Using wkhtmltopdf to generate a PDF from eZ Platform

I wanted to produce a polished PDF of restricted content managed by https://ezplatform.com/ with the least amount of effort possible and I wanted an approach that would allow me to run a single command on the command line.

My first approach was to try to use a browser's print to PDF feature, but I wasn't happy with the results. So I tried https://wkhtmltopdf.org/,

After tinkering with various roles/permissions and firewall configurations with eZ Platform and Symfony I chose to create a PDF user which was allowed to log in and view only the target content. I used curl to log in, extracted the eZ session id cookie and passed it to wkhtmltopdf for rendering.


#!/bin/bash
if [ "$#" -ne 1 ]; then
    echo "Usage: $0.sh url"
    exit;
fi;
USER=pdfuser
PASS=somepassword

URL=$1;
PDF=`echo $1 | sed "s/.*\/\([^\/]\+\)$/\1/"`
FOOTER_LEFT=${PDF^^}
LOGIN_URL=`echo $1 | sed "s/^\(https\?:\/\/[^\/]\+\/\).*$/\1login/"`
CSRF_TOKEN=`curl -s -X GET --cookie-jar cookie "$LOGIN_URL" | grep -o "name=\"_csrf_token\" value=\"\([^\"]\+\)\"" | sed "s/.*value=\"\([^\"]\+\)\"/\1/"`
LOGIN_DATA="_username=$USER&_password=$PASS&_csrf_token=$CSRF_TOKEN"
curl -L -s -b cookie --cookie-jar cookie -d "$LOGIN_DATA" "$LOGIN_URL"_check > /dev/null
COOKIE=`grep -o "\(eZSESS.*\)$" cookie | sed "s/\s\+/ /g"`
wkhtmltopdf --cookie $COOKIE --print-media-type --margin-left .5in --margin-right .5in --margin-top .5in --margin-bottom .5in "$URL" --footer-left "$FOOTER_LEFT" --footer-center 'Page [page] of [topage]' --footer-font-name 'Open Sans' --footer-font-size 8 --footer-right 'Updated [date]' "$PDF.pdf"

Thanks to: https://serverfault.com/a/306360/311430 for help with the cookies in the Apache log

Quiz analysis with Canvas API

I wanted to perform Item Analysis on quiz results under Canvas LMS.

Without working very hard.

I used a Web Inspector to look at the traffic, found the URL for quiz statistics and started trying to use curl to authenticate in and get the data.

When it didn't work right away, I Googled and found the Canvas REST API documentation. I spent a little time roaming through and found ...

curl -H "Authorization: Bearer REDACTED" https://canvas.instructure.com/api/v1/courses/:course_id/quizzes/:quiz_id/statistics

A single curl request delivers all the stats for a quiz. Absolutely excellent.

To create a .csv of the data, I used PHP, then translated it to Node.js


<?php
// The JavaScript code handles more types of questions - refer to that if you are planning to use this code
$shortopts = "f:q:";

$longopts  = [
    "file:",
    "questions:"
];
$options = getopt($shortopts, $longopts);

$filename = $options['f'];
if (is_file($filename)) {
        $questionList = $options['q'];
        $questions = explode(',', $questionList);
        if (count($questions) < 1) {
                die('No questions');
        }       
} else {
        die('File not found');
}

$keys = [ 'responses', 'answered', 'correct', 'partially_correct', 'incorrect' ];
$assessmentData = json_decode(file_get_contents('stats.json'));
if (json_last_error() === JSON_ERROR_NONE) {
        $quizStatistics = $assessmentData->quiz_statistics[0];
        echo 'question,'.implode(',', $keys).PHP_EOL;
        foreach ($questions as $q) {
                $data = getData($keys, $q, $quizStatistics->question_statistics[$q]);
                echo implode(',', $data).PHP_EOL;
        }
}

function getData($keys, $question,$data) {
        $lineData = [ $question ];
        foreach ($keys as $k) {
                $lineData[] = isset($data->$k) ? $data->$k : 0;
        }
        return $lineData;
}


"use strict";

// Thanks to: https://stackabuse.com/command-line-arguments-in-node-js/
// Thanks to: https://flaviocopes.com/how-to-check-if-file-exists-node/
// Thanks to: https://code-maven.com/reading-a-file-with-nodejs

const fs = require('fs');

const path = process.argv[2];
try {
	if (fs.existsSync(path)) {
		let questionList = process.argv[3];
		let questions = questionList.split(',');
		if (questions.length < 1) {
			throw new Error('No questions');
		}
		fs.readFile(path, 'utf8', function(err,contents) {
				let csvData = createCSVData(contents,questions);
				process.stdout.write(csvData + '\n');
				});	
	} else {
		throw new Error('File not found');
	}
} catch (err) {
	console.error(err);
}

function createCSVData(fileData,questions){
	const keys = [ 'responses', 'answered', 'correct', 'partially_correct', 'incorrect' ];
	let csvData = '';
	let assessmentData = JSON.parse(fileData);
	if (assessmentData !== null && typeof assessmentData !== "undefined") {
		let quizStatistics = assessmentData.quiz_statistics[0];
		csvData = 'question,'+ keys.join(',') + '\n';
		questions.forEach((q) => { 
                                q--; // questions are zero based
				let data = getData(keys, q, quizStatistics.question_statistics[q]);
				csvData += data.join(',') + '\n';
				});
	}
	return csvData;
}

function getData(keys, question, data) {
	let lineData = [ question ];
	keys.forEach(k => {
		let val = 0;
		if (typeof data[k] !== "undefined") {
			val = data[k];
		} else {
			if (typeof data[k + '_student_count'] !== "undefined") {
				val = data[k + '_student_count'];
			}
		}
		lineData.push(val);
	});
	switch (data.question_type) {
		case "essay_question":
			data.point_distribution.forEach(o => {;
				switch (o.score) {
					case data.full_credit:
						lineData[keys.indexOf('correct')+1] = o.count;
						break;
					case 0:
						lineData[keys.indexOf('incorrect')+1] = o.count;
						break;
					default:
						lineData[keys.indexOf('partially_correct')+1] += o.count;
						break;
				}					
			});
			break;
		case "true_false_question":
		case "short_answer_question":
			data.answers.forEach(o => {;
				if (o.correct === true) {
					lineData[keys.indexOf('correct')+1] = o.responses;
				} else {
					lineData[keys.indexOf('incorrect')+1] = o.responses;
				}					
			});
			break;			
		default:
		    	//console.log(question + ". "+data.question_type);
			//console.log(data);
	}
	return lineData;
}

Sample output:

question,responses,answered,correct,partially_correct,incorrect
2,11,11,3,4,6

Ref: https://canvas.instructure.com/doc/api/
Ref: https://canvas.instructure.com/doc/api/file.oauth.html#manual-token-generation - get a user auth token

AMI - upgrade PHP from 7.1 to 7.3

AMI - upgrade PHP from 7.1 to 7.3
PHP logo

Don't do this on a production system

I ran this on an Amazon Linux AMI - it's probably fine on CentOS, etc.

Get all the PHP 7.1 packages and make a file called php. You might have to change the .x86_64 to .i386/.i686

sudo yum list installed php71* | grep php | cut -f1 -d' ' | tr -d '.x86_64' | tr "\n" ' ' | sed "s/71/73/g" > php

Remove PHP 7.1 (remember I said not to do this on a production machine)

sudo yum remove php71*

Now edit your php file and add

sudo yum install at the beginning of the list of packages

It should look something like this

sudo yum install php73 php73-cli php73-common php73-gd php73-imap php73-intl php73-json php73-mbstring php73-mysqlnd php73-opcache php73-pdo php73-pecl-apcu php73-pecl-igbinary php73-pecl-memcached php73-pgsql php73-process php73-soap php73-ml

Run the php file with

source php

And, if you are using memcached, run this too

sudo yum install php7-pear php73-devel
sudo pecl7 install memcached
sudo pecl7 update-channels

Add this into php.ini somewhere ...

extension=memcached.so

Restart Apache

sudo apachectl restart

Bask in the glory

Raspberry Pi - web page as a sign (or kiosk)

TL;DR

This is the stuff that lets you get the Pi up and running - meaning displaying a web page. It won't solve all your problems.

Raspberry Pi - web page as a sign (or kiosk)

Raspberry Pis are about the size of a deck of cards and extremely easy to work with. If all you need is to display a web page, they're great.

Setup steps for Pi

Setup WiFi

You're probably going to need an internet connection. This describes how to set up WiFi. If you're using a wired connection, you can skip this.

I apologize for providing a link, but it is the best way to explain what needs to be done. You'll figure it out.

Ref: https://www.raspberrypi.org/documentation/configuration/wireless/wireless-cli.md

sudo raspi-config

Set the locale and hostname of the Pi

This is important for several reasons:

  • You can use the hostname to identify the Pi to the web page you're displaying
  • The locale affects the keyboard layout. If you don't set it properly, when you press a key, you might not get what you want.
  • You need to set the timezone for the Pi, or it won't know what time it is.

You will need to reboot after this

Install vim

Okay, this isn't strictly necessary and if you prefer to use nano or some other editor, it's fine.

sudo apt-get install vim

Screensaver

You probably don't want the display to shut off. The easiest way to do this is to use xscreensaver.

Install xscreensaver, so it is easy to shut off the screensaver


sudo apt-get install xscreensaver

Now shut off the screensaver using the GUI

Make the Pi open a browser in kiosk mode

Set it up to load the browser and display the sign on boot

Add this command as the last line in ~/.profile.


chromium-browser --kiosk --window-position=0,0 --window-size=1920,1080 https://www.example.com?kiosk=`hostname`

Set the window size to match your monitor. Set the URL to work with your site.

Test it out a few times on the command line to make sure it does what you want.

Ctrl-Alt-F2 escapes from kiosk mode, default Pi username/password is pi/raspberry. To start the desktop (if you cancel out of kiosk mode, you may need this), use startx

Add a cron job to shut the Pi down

If you're using the Pi to run a sign in the office, there's no point in having the sign display during off hours. Turn it off.

Create a file called: /etc/cron.d/stop


0 19 * * * root /sbin/shutdown -h now > /dev/null 2>&1

Final thoughts

  • Be sure to consider authentication/security. Since these will probably be limited access, you can use the IP address to identify it. Only trusted sources. Or you may argue public access is fine. It's a sign, right? Anyway - be sure to consider whether you are willing to allow access to the web page from sources other than the sign.
  • The Pi doesn't turn on automatically. You need to choose a way to turn the sign on, or never shut it off. It can be a simple switch, a timer, a remote controlled outlet, a smartphone control, or ... http://www.uugear.com/witty-pi-realtime-clock-power-management-for-raspberry-pi/ (which looks like the most fun)
  • You may want to install Teamviewer so you can support it remotely.

This post courtesy of Game Creek Video.

Returning custom headers with FOSRestBundle

The Content-Range header supports a (dgrid) OnDemandGrid

Returning custom headers with FOSRestBundle


use FOS\RestBundle\View\View as FOSRestView;
...
$view = FOSRestView::create();
$view->setData($data);
$view->setHeader( 'Content-Range', 'items '.$offset.'-'.($offset+$limit).'/'.$count);
$handler = $this->get('fos_rest.view_handler');
return $handler->handle($view);

Ref: https://symfony.com/doc/1.5/bundles/FOSRestBundle/2-the-view-layer.html

Symfony 4