Webpage Screenshot Server

How to build a Webpage Screenshot Server with the Raspberry Pi 2. First we need "xvfb" (X virtual framebuffer), ImageMagick then we need "firefox-esr" and last we need "apache" with php to listen on external requests and deliver the Webpage Screenshots. Many tanks to Leonard Teo (Taking Server Side Screenshots of Websites) I followed his instructions for Ubuntu and adapted them for the Raspberry Pi.

1.) Update your Raspberry Pi Packages

run the commands:

pi@raspberrypi ~ $ sudo apt-get update
pi@raspberrypi ~ $ sudo apt-get upgrade

2.) Install xvfb, firefox-esr, imagemagick, apache2, php5

run the commands:

pi@raspberrypi ~ $ sudo apt-get install xvfb
pi@raspberrypi ~ $ sudo apt-get install firefox-esr
pi@raspberrypi ~ $ sudo apt-get install imagemagick
pi@raspberrypi ~ $ sudo apt-get install apache2
pi@raspberrypi ~ $ sudo apt-get install php5 php5-gd libapache2-mod-php5

to remove unused packages run the command:

pi@raspberrypi ~ $ sudo apt-get --purge remove sonic-pi
pi@raspberrypi ~ $ sudo apt-get autoremove

3.) Check the Package "ttf-mscorefonts-installer"

Many webpages use common fonts that are included in the "ttf-mscorefonts-installer" package, check if it is already installed or you might want to install it.

pi@raspberrypi ~ $ dpkg --get-selections
pi@raspberrypi ~ $ sudo apt-get install ttf-mscorefonts-installer

4.) Get the "xvfb" process running

create a file in /etc/init.d called xvfb with te following shell script:

#! /bin/sh
### BEGIN INIT INFO
# Provides:          xvfb
# Required-Start:    $remote_fs $syslog
# Required-Stop:     $remote_fs $syslog
# Default-Start:     2 3 4 5
# Default-Stop:      0 1 6
# Short-Description: Start/stop xvfb for the Screenserver
### END INIT INFO

# 11.03.2017 david jufer

XVFB_OUTPUT=/tmp/Xvfb.out
XVFB=/usr/bin/X11/Xvfb
XVFB_OPTIONS=":5 -screen 0 1920x1920x24 -fbdir /var/run"

do_start()
{
    echo -n "Starting : X Virtual Frame Buffer "
    $XVFB $XVFB_OPTIONS >>$XVFB_OUTPUT 2>&1&
    RETVAL=$?
    echo
    return $RETVAL
}

do_stop()
{
    echo -n "Shutting down : X Virtual Frame Buffer"
    echo
    killall Xvfb
    echo
    return 0
}

case "$1" in
    start)
        do_start
    ;;
    stop)
        do_stop
    ;;
    status)
        status xvfb
    ;;
    restart)
        do_stop
        do_start
    ;;
    *)
        echo "Usage: xvfb {start|stop|status|restart}"
        exit 1
    ;;
esac

:


now you can start and stop the xvfb server with:

pi@raspberrypi ~ $ sudo service xvfb start
pi@raspberrypi ~ $ sudo service xvfb stop

with the following command we achiev that the process "xvfb" starts automatically on startup (on CentOS it's "chkconfig"):

pi@raspberrypi ~ $ sudo service --status-all
pi@raspberrypi ~ $ sudo update-rc.d xvfb defaults

5.) Check the basic functions

Let us check the basic functions. Start the virtual framebuffer, run the Firefox-Browser an take a screenshot of it. DISPLAY=:5 tells xvfb to render to display 5 (virtual), nohup silence the output firefox loads, firefox jufer.info starts Iceweasel and loads the given URL.

pi@raspberrypi ~ $ sudo service xvfb start
pi@raspberrypi ~ $ sudo DISPLAY=:5 nohup firefox jufer.info &
pi@raspberrypi ~ $ sudo DISPLAY=:5 import -window root /var/www/html/screenshot.png

Check the PNG-Image. Kill firefox by typing fg then CTRL+C.

6.) Some Config for Apache

Setup the rewrite module and configuration overriding (.htaccess)

pi@raspberrypi ~ $ sudo a2enmod rewrite
pi@raspberrypi ~ $ sudo nano /etc/apache2/sites-enabled/000-default.conf

change the settings for "Directory /var/www/" to look like this

<VirtualHost *:80>
  ServerAdmin webmaster@localhost
  DocumentRoot /var/www/html
  # david jufer 11.03.2017
  <Directory /var/www/html>
    AllowOverride All
  </Directory>
  ErrorLog ${APACHE_LOG_DIR}/error.log
  CustomLog ${APACHE_LOG_DIR}/access.log combined
</VirtualHost>

Restart Apache

pi@raspberrypi ~ $ sudo service apache2 restart

7.) Some Config for Firefox/Iceweasel

After closing firefox with "kill %1" the profile ist writen to /root/.mozilla/firefox/<random>.default/

now check /root/.mozilla/firefox/profiles.ini to look something like this

pi@raspberrypi ~ $ sudo nano /root/.mozilla/firefox/profiles.ini

[General]
StartWithLastProfile=0

[Profile0]
Name=default
IsRelative=1
Path=sa8a1kyt.default
Default=1

open the file "prefs.js"

pi@raspberrypi ~ $ nano /root/.mozilla/firefox/sa8a1kyt.default/prefs.js

add the folowing content at the end of the file:

# david jufer 11.03.2017
user_pref("browser.fullscreen.animateUp", 0);
user_pref("browser.fullscreen.autohide", false);
user_pref("browser.link.open_newwindow", 1);
user_pref("browser.sessionstore.resume_from_crash", false);
user_pref("browser.startup.page", 0);
user_pref("browser.tabs.warnOnOpen", false);

open /root/.mozilla/firefox/<random>.default/xulstore.json an set the following values for with, height, screenX, screenY

pi@raspberrypi ~ $ sudo nano /root/.mozilla/firefox/sa8a1kyt.default/xulstore.json

...
,"main-window":{"width":"1920", "height":"1920", "screenX":"0", "screenY":"0",
...

8.) Create the scripts needed on the Apache

We want the URL-Call to be something like this to create a Image of the Webpage www.jufer.info:
   http://domain/screenserver/www.jufer.info
or
   http://domain/screenserver/1280/1800/www.jufer.info
to get a screenshot of the size 1280x1800.
Fist crate a folder /var/www/html/screenserver

pi@raspberrypi ~ $ sudo mkdir /var/www/html/screenserver
pi@raspberrypi ~ $ sudo chown www-data:root /var/www/html/screenserver

Create the 2 empty files jobKey.txt and jobUrl.txt make shure the owner is "www-data". With these 2 files we are achieving that multiple requests are serialized.

Create the /screenserver/.htaccess file with the following rules:

RewriteEngine on

RewriteCond %{REQUEST_FILENAME} !-d
RewriteCond %{REQUEST_FILENAME} !-f
RewriteRule ^([0-9]+)/([0-9]+)/(.+)[/]?$ index.php?url=$3&width=$1&height=$2 [PT]

RewriteCond %{REQUEST_FILENAME} !-d
RewriteCond %{REQUEST_FILENAME} !-f
RewriteRule ^([0-9]+)/(.+)[/]?$ index.php?url=$2&width=$1 [PT]

RewriteCond %{REQUEST_FILENAME} !-d
RewriteCond %{REQUEST_FILENAME} !-f
RewriteRule ^(.+)[/]?$ index.php?url=$1 [PT]


The index.php file contains the function of the "frontend" an look like this: