I made a statement in the Joomla forums offering to describe how I use Varnish and Joomla together. Well...today is the day! Due to an unforeseen summer cold - I'm not doing the outside work that I thought I'd be doing, so it's time to make good on that promise.
What you see in the image above is a generic (simplified) version of the topology I use for my high traffic sites. There might be more than one load balancer, maybe more than one Varnish server, more than one internal NGINX or PHP server, but this is the general layout. I like to put all of the applications behind a firewall, in a private network, and leave only the load balancers accessible to the Internet. This is just my preference, a configuration that has served me well over the years. By no means is this the only way to do it, it's just what works for me and it's an easy configuration available on most server hosting platforms.
Simplified Varnish and Joomla
I am NOT going to impose a 7-server configuration on you. Varnish and Joomla setup can be even further simplified to just 2 servers. In this way, I can more easily convey necessary configurations and required software. I'm going to show the Ubuntu 24.04 version of this software stack. Your environment may be slightly different (in process or package name) if you're using something other than Ubuntu. You'll have to rely on your own knowledge to set up the server OS, I'm only dealing with the aspects directly related to Varnish and Joomla!
Everyone runs SSL these days, so I feel it needs to be mentioned that the certificate is installed on the load balancer, and all communication between the load balancer and Joomla or Varnish and Joomla is on port 80. The load balancer handles the encrypted traffic with the public. Joomla is configured to use the "behind a load balancer" setting in global configuration.
To make the config files more clear, I'm giving each server an entry in /etc/hosts. This way I can refer to meaningful names like varnish and liveserver instead of IP addresses. You could easily use IP addresses if you wanted.
/etc/hosts
127.0.0.1 varnish
192.168.1.2 liveserver
Load Balancer + Varnish
This could be 2 separate servers, but for the purposes of this howto - we can squeeze them into one without any issues. This is the required package list (using Ubuntu package names):
- varnish
- nginx
- libnginx-mod-http-headers-more-filter
Yup, that's it. Configs will be detailed later.
Linux NGINX MySQL PHP
As with the load balancer server, this could be many different servers, and if you wanted to use Apache instead of NGINX, you could do that - but you'd lose some of the load balancer magic..no big deal. Plenty of guides exist to help you configure the basic stack.
What is / why NGINX? I prefer NGINX (pronounced Engine-X) to Apache. I made the switch years ago, and found that it's faster and easier to configure and maintain.
Here's the required package list (again, using Ubuntu package names):
- nginx
- mariadb-server
- php-fpm
- php-mysqlnd
- php-redis
- php-json
- php-simplexml
- php-dom
- php-zip
- php-gd
- php-intl
This is a pretty standard installation for Joomla on LNMP stack.
Some Basic Rules
Before we get started, some basic rules about setting up this Varnish and Joomla site tech stack. You'll need to be very careful about forms. If a form resides on a cached page, it's not going to work because the cached CSRF token won't be valid anymore. That means no login modules, at least on pages that can be cached.
Any page that submits data or displays any kind of dynamic result needs to be non-cached. Search results pages, user profiles, contact forms... It's a pain in the ass sometimes, but your site will be fast.
Next, we'll start with configuration on the load balancer.
NGINX
The load balancer is the only server with services listening on the Internet. This requires specific configuration on the OTHER servers to prevent them from listening on public networks that may be attached.
Many writeups exist explaining now to turn NGINX into a load balancer. I utilize snippets to define potential destinations based on accessed locations in NGINX configuration. This way I can re-use the snippets in multiple locations.
/etc/nginx/snippets/ssl.conf
Out of scope for this how-to, but it keeps SSL configurations neatly out of site configurations. Maybe I'll write another article explaining how to get an A+ with SSL Labs.
/etc/nginx/snippets/varnish.conf
proxy_set_header X-Real-Ip $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header Host $host;
proxy_set_header REMOTE_ADDR $remote_addr;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_hide_header X-Powered-By;
more_clear_headers Server;
more_clear_headers X-Logged-In;
more_clear_headers Via;
more_clear_headers X-Varnish;
more_clear_headers X-Cache;
more_clear_headers X-Cache-Hits;
proxy_read_timeout 300;
proxy_pass http://upstream_varnish;
proxy_redirect http://upstream_varnish https://$host;
/etc/nginx/snippets/liveserver.conf
proxy_set_header X-Real-Ip $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header Host $host;
proxy_set_header REMOTE_ADDR $remote_addr;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_pass http://upstream_liveserver;
proxy_read_timeout 3600;
proxy_connect_timeout 300;
proxy_redirect http://upstream_liveserver https://$hostname;
proxy_http_version 1.1;
proxy_set_header Connection "";
/etc/nginx/sites-available/default.conf
# port 80 forwards all traffic to 443
server {
listen 80 default_server;
listen [::]:80 default_server;
server_name www.yourdomain.com yourdomain.com;
location / {
return 301 https://$host$request_uri;
}
}
server {
listen 443 ssl http2;
listen [::]:443 ssl http2;
server_name www.yourdomain.com yourdomain.com;
ssl_certificate /path/to/your/fullchain.pem;
ssl_certificate_key /path/to/your/privkey.pem;
include /etc/nginx/snippets/ssl.conf;
upstream upstream_liveserver {
least_conn; #only if your backend servers are NGINX
server liveserver weight=5;
# server liveserver1;
# server liveserver2 backup;
}
upstream upstream_varnish {
server varnish weight=5;
# server varnish1;
# server varnish2 backup;
}
location / {
include /etc/nginx/snippets/varnish.conf;
}
location ~ ^/administrator {
include /etc/nginx/snippets/liveserver.conf;
}
}
Though simplified for this write-up, this is the basic configuration of the load balancer. It's not anything special, or complicated. Populate your upstream servers, modify the path to your certificates, update the server name, maybe add some access and error logs and you're good to go.
It might look like ALL traffic outside of /administrator is being cached with Varnish, but we'll handle cached and non-cached URLs in the next step.
Creating an NGINX configuration file on the actual backend server is well-documented elsewhere.
Varnish Server
Probably the biggest reason Varnish isn't seen more often in Joomla setups is its configuration. Varnish VCL configuration files are complex and large but they're really not that complicated.
I started with a configuration created by fevangelou titled: The perfect Varnish configuration for WordPress, Joomla, Drupal & other (common) CMS based websites. I found that configuration to be less than ideal for Joomla. The limitations are pretty extensive, but I'll sum them up in just a few bullets:
- Config contains Wordpress and Drupal specific configs
- List of pages to NOT cache is static
- Not well documented
- It has comments, but the Wordpress, Drupal, and Joomla configs are all mixed together. I only care about the Joomla configs.
None of that is a criticism for the original author - their config was my starting point.
Using my (free) System - Expires Headers plugin, we can reduce the non-caching page list to just a few entries that are universal to Joomla installations. Using the plugin to control cached pages is discussed below the Varnish config.
/etc/varnish/default.vcl
Um... I'm not putting the whole config file here. Even pared down, it's still over 300 lines long. I'll cover the highlights here, and just let you download the config file. Remember to edit it for your environment!!!
There are some basic configs that I'm just going to breeze over. For the most part, there are 4 sections you'll want to understand. Let's breeze over those bits first.
These are commented, so you can pretty much read what each bit does in the comments above it.
- vcl - defines which version of Varnish your config is intended for.
- import - allows you to bring in additional functionality
- backend - allows you to define the source where Varnish retrieves its content
- director - useful if you have multiple backends and wish to load-balance
Action Sections
These are where the Varnish magic happens. This is also where you'll find the main differences between my configuration and that provided by fevangelou. I chose to focus on Joomla configuration, and to simplify the configuration using a plugin to define non-cached pages. This removed dozens of lines from the config file.
I'm not going into detail, this is a broad overview of what's happening in each section. The sections are heavily commented, and should be relatively self-explanatory.
When you see the following commands, this is what they mean:
- return(pass) - tells Varnish to pass the request to the backend rather than try and serve it from cache.
- return(pipe) - means Varnish stops inspecting each request, and just sends bytes straight to the backend.
- return(hash) - tell Varnish to deliver content from cache even if the request otherwise indicates that the request should be passed.
- return(deliver) - send the page to the browser
sub vcl_recv
This section examines the incoming request and determines what to do with it. We do fun things in this section in preparation for the response. One of the first things is attaching the client IP address to a special header in order to hand it over to Joomla. We also strip out cookies that are unnecessary.
We also inspect several header and cookie values to determine if the request should NOT be served from cache. I opt to ALWAYS serve fresh pages to users who are logged in, so we look for the joomla_user_state cookie and act upon it if it's found
Likewise, there are some URLs that will never get cached - 2, to be precise. /administrator and com_ajax. If the page is requested with XMLHttpRequest, it's also some kind of AJAX call, and those aren't cached.
Basically, you'll want to take a peek at this section and determine if there's something it's doing that you don't want it to do (or something it's NOT doing, perhaps another URL or cookie to pay attention to.)
The good news is, I stripped out all of the WP and Drupal stuff, so if it's in there - it's only stuff that's Varnish and Joomla related.
sub vcl_backend_response
Super similar to the previous section, this section deals with the response from the backend servers. There are various details you'll want to read through, but here are the highlights:
- We don't cache errors (5xx status codes)
- We don't cache if the page has ANY negative caching headers (no-store, no-cache, must-revalidate, Pragma=no-cache).
- The same pages that aren't allowed in the previous section aren't allowed to be cached here.
- POST requests are not cached
- Cookies are unset for guests to prevent them from being stored with cached pages
- We set some cache duration values
Again, I stripped out anything to do with WP or Drupal - the only things left in this section is Varnish and Joomla related.
sub vcl_deliver
Final checks, setting Expires headers if they aren't already set, some cache headers (that the load balancer removes), and finally sending the response to the browser.
Cached Page Control
Varnish and Joomla don't naturally play nice. Varnish wants to cache everything, and requires a list of pages to NOT cache, and there isn't really a mechanism within Joomla to indicate which pages should be cached and which not. It occurred to me that I wrote a plugin to do that exact thing - but with browser cache. So why not adapt that for use in Varnish? I mean, Varnish can inspect headers...
Varnish and Joomla controlled by System - Expires Headers
My plugin is pretty simple to use. You can define any number of cache configurations and assign those to menu items. More specifically, you can define which pages NOT to cache with a few simple settings and menu item selections. To be extremely specific, these are the settings to use in order to trigger my version of Varnish config to NOT cache a page:
- Expires: Off
- Cache-Control: On
- Cacheable: no-cache
- Check all 3: no-store, must-revalidate, and Pragma: no-cache
Then just apply it to as many of your menu items as need to be NOT cached. One thing to remember, if the page is ONLY visible to logged in users, it will never be cached - so no need to choose those menu items.
Varnish and Joomla has never been easier.
Joomla Cache
This setup eliminates the need to use Joomla's Page Cache plugin. The ability to set global expires headers using the Expires Headers plugin, and Varnish level caching removes the caching burden from Joomla. One less plugin means less activity and faster dynamic page loads. This doesn't do anything to other cached content types (module cache, plugin cache, etc).
Final Thoughts
Is this the absolute best configuration for Varnish and Joomla? Absolutely not. This is as much a starting point for me as it is for you. Maybe I have a little bit of a head start, but certainly there is someone smarter out there willing to devote some time to making this better. I sincerely hope they're willing to share their work as I have done here.
I am open to both suggestions and criticisms. By all means, tell me what I did wrong (or right, if you feel generous). I'm certain that there are improvements to be made and I'm not too proud to ask for them.
Good luck, and may your sites be super fast.
