Basics

What is varnish

Varnish has been specifically developed as HTTP-Accelerator. It acts as a reverse proxy server. All requests to port 80 directed at the servers are captured. Varnish places all content in RAM memory. If the content is re-requested it will be read from memory. In summary Varnish prevents every request to be processed by PHP en SQL. It relieves the server and the request will be dealt with much faster.

Varnish is configured in a stand alone server or a cluster of servers environment. In a multiple server environment a single varnish server acting as a webproxy is used. In this article we describe the varnish configuration in a single webserver with Magento environment. Varnish ships with its own language VCL: (Varnish Configuration Language).

Configuration

Varnish is configured per domain. On a single server multiple instances of varnish may be loaded.

Beware: The port numbers use in this manual may be already in use.

Create a Varnish instance

To run Varnish it has to be configured:

First create a varnish configuration: ~/domains/<domein>/var/etc/default.vcl

vcl 4.0;
backend default {
    .host = "127.0.0.1";
    .port = "8080";
}

258/5000 In this example we created an empty varnish configuration. The configuration of this is application specific. You need a secret file to protect your varnish instance from the outside. You do this with the following command:

cat /dev/urandom | tr -dc 'a-zA-Z0-9' | fold -w 32 | head -n 1 > ~/domains/<domain>/var/etc/varnish.secret

In the example configuration below the varnish portnumber 6181 and the admin port is 6182 are defined. These ports may vary if multiple varnish instances are running on the same server.

Chcek the ports in use, by running this command:

$ netstat -atun | grep LISTEN

tcp        0      0 0.0.0.0:3306            0.0.0.0:*               LISTEN
tcp        0      0 0.0.0.0:6182            0.0.0.0:*               LISTEN
...
...
...
tcp6       0      0 :::2049                 :::*                    LISTEN
tcp6       0      0 :::54500                :::*                    LISTEN
tcp6       0      0 :::6181                 :::*                    LISTEN

In this example the ports 6181 en 6182 are not in use and ready to be used with varnish.

Starten / restart

a configuration for Supervisor needs to be created, to ensure the proper functioning of varnish.

file: ~/supervisor/supervisor.d/varnish-<domainname>.conf

[program:varnish]
command=/usr/sbin/varnishd -p feature=+esi_ignore_other_elements -p vcc_allow_inline_c=on -p cli_buffer=16384 -a :6181 -T localhost:6182 -f /home/<user>/domains/<domein>/var/etc/default.vcl -S /home/<user>/domains/<domein>/var/etc/varnish.secret -s malloc,256m -F -n /home/<user>/domains/<domain>/var/run
autorestart=true
stdout_logfile=/home/<user>/domains/<domain>/var/log/varnish.log
redirect_stderr=true
environment=HOME='/home/<user>/',PATH='/bin:/sbin:/usr/bin:/usr/sbin:/usr/local/bin:/usr/local/sbin'
directory=/home/<user>/domains/<domain>/var/run

Also the quantity of memory allocated to varnish can be configured using the '-s malloc,256m' parameter In this case 256MB is allocated to varnish which can be leveled up if the shop uses many pages.

Beware Too much memory allocated to Varnish causes serious problems for applications like MySQl, reddish or other services if their memory becomes depleted.

Now the service has been added and Supervisor can start it.

$ supervisorctl reread
varnish: available

$ supervisorctl update
varnish: added process group

A final control if varnish is running:

$ supervisorctl status varnish
varnish RUNNING pid 4581, uptime 0:00:01

Use the following command to restart Varnish:

$ supervisorctl restart varnish

Nginx configuration adjustment

Varnish is an http accelerator reverse proxy and captures requests before they are forwarded to PHP. At the first request varnish forwards it and the server processes the request. The result is stored in varnish and the next time the same request is answered directly from the Varnish cache preventing a high php / sql / hhvm load on the server.

Because all modern websites answer all traffic from port 80 (http) immediately with a redirect response to https, only a configuration for port 443 is added.

This is configured in ~/domains/<domain>/var/etc/port-443/varnish.nginx.conf.

location ~* ^\/.* {
    proxy_pass http://127.0.0.1:6181;
    proxy_set_header Host $http_host;
    proxy_set_header X-Forwarded-Host $http_host;
    proxy_set_header X-Real-IP $remote_addr;
    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    proxy_set_header Ssl-Offloaded $https;
    proxy_set_header X-Forwarded-Proto $scheme;
}

Beware Check if the ports below are equal to the earlier mentioned ports.

You need to reload the nginx configuration using this command:

nginx-reload

a request is processed as follows:

+-----+    +-----------+    +--------------+    +------------+    +---------+
| WEB +--->+ NGINX 443 +--->+ VARNISH 6181 +--->+ NGINX 8080 +--->+ PHP|FPM |
+-----+    +-----------+    +--------------+    +------------+    +---------+

Application & SSL offloading

Because the requests are directly answered from cache, PHP considers these requests no longer to be http request by default. If your application also redirects to https at the same time it may cause a inifite redirect loop. Specifically, the PHP server variable $ _SERVER ['HTTPS] is no longer set to "on".

In the case of such a redirect, modern applications no longer only consider the server variable $ _SERVER but also the SSL offload header. If this is not configured, go to your entry point (usually index.php) add this piece of code.

if (isset($_SERVER['HTTP_SSL_OFFLOADED'])) {
    $_SERVER['HTTPS'] = 'on';
}

Varnish commands

Beecause Vanrish does not run from a default location the varnish commands need an extra argument, these look like this:

varnishadm -n ~/domains/<domain>/var/run/
varnishhist -n ~/domains/<domain>/var/run/
varnishlog -n ~/domains/<domain>/var/run/
varnishncsa -n ~/domains/<domain>/var/run/
varnishreload -n ~/domains/<domain>/var/run/
varnishstat -n ~/domains/<domain>/var/run/
varnishtop -n ~/domains/<domain>/var/run/

To speed up the typing of the commands, you may add aliases to your bash configuration. Add the following lines to your ~/.bash_aliases file. You need to logout and login to activate the aliases.

alias varnishadm="varnishadm -n ~/domains/<domein>/var/run/"
alias varnishhist="varnishhist -n ~/domains/<domein>/var/run/"
alias varnishlog="varnishlog -n ~/domains/<domein>/var/run/"
alias varnishncsa="varnishncsa -n ~/domains/<domein>/var/run/"
alias varnishreload="varnishreload -n ~/domains/<domein>/var/run/"
alias varnishstat="varnishstat -n ~/domains/<domein>/var/run/"
alias varnishtop="varnishtop -n ~/domains/<domein>/var/run/"