DIY Mercury installation for super fast WordPress performance
- Running huge imports with WP All Import reliably (yes, even if you use Cloudflare!) - September 19, 2023
- New SQL enhancement for Scalability Pro to fix WooCommerce long-running query in the product-hero block - August 31, 2023
- Create a static favicon.ico to avoid surplus PHP requests for rush traffic - July 25, 2023
I’ve just released a new service to set up, migrate and configure your WordPress site for you on Digital Ocean using the latest and greatest Mercury Install. BUT, maybe you want to set this up yourself? If so, read on for the full guide.
This includes:
- Nginx – web server (far faster than Apache)
- Memcached and APC – caching engine which works with W3 Total Cache
- Varnish – HTTP accelerator
- HHVM with fallback to PHP-FPM – compiled PHP for much faster pages
- Percona DB – faster than MySQL (100% compatibility)
Installing just Nginx will make your site a helluva lot faster and able to support many many more simultaneous users, but add in all these other features and you’ll be the envy of all your WordPress friends.
If you want to see some charts and stuff, check out the High Performance WordPress Installation Service page.
Table of Contents
DIY Mercury Installation Steps
You need to know SSH and some Linux commands in order to build this yourself. Here’s the steps:
- Get yourself a Digital Ocean droplet (or any Ubuntu 14.04+ VPS but Digital Ocean use Raid SSD so they’re a great choice) (this affiliate link gets you $10 off)
- If you’re migrating, point new.yourdomain.com at your Digital Ocean IP address. If this is for a new site, point yourdomain.com at your Digital Ocean IP address
- Follow the instructions using this Ansible Playlist to the letter
- Install cURL, mcrypt and set up your firewall –
sudo apt-get install php5-curl
sudo apt-get install php5-mcrypt
sudo ufw allow ssh
sudo ufw allow 80/tcp
sudo ufw allow 443/tcp
sudo ufw allow 25/tcp
sudo ufw enable - Set up PHP7 for your faster fallback plan:
sudo add-apt-repository ppa:ondrej/php
sudo apt-get update
sudo apt-get install -y tmux curl wget nginx php7.0-fpm php7.0-cli php7.0-curl php7.0-gd php7.0-intl
sudo apt-get install php7.0-mysql php-memcached php7.0-mbstring php7.0-zip php7.0-xml
vi /etc/nginx/conf.d/upstream.conf(Change php socket to new php7 socket)
upstream php {
server unix:/var/run/php/php7.0-fpm.sock;
} - Set up fail2ban to protect your server from attack (edit the jail.local file once you’ve created it, find the nginx section and change enabled to true):
sudo apt-get install fail2ban
Then vi /etc/fail2ban/jail.conf Change nginx enabled=true (instead of false) Change bantime at top to 86400 (1 day instead of 10 minutes) Add the following after nginx section:
cp /etc/fail2ban/jail.conf /etc/fail2ban/jail.local
vi /etc/fail2ban/filter.d/wordpress.conf (add following lines)
[Definition]
failregex = ^<HOST>.*POST.*(wp-login\.php|xmlrpc\.php).* (403|499)
[wordpress]
enabled = true
port = http,https
filter = wordpress
logpath = /var/log/nginx/*access.log
maxretry = 3 - Set up sendmail (not needed if you use an external service like Mandrill or Mailgun – note: more work needed to set up SPF records and make email get past Junk folders – use external service if in doubt)
sudo apt-get install sendmail
sudo sendmailconfig (choose Y and Y for answers to the questions) - Set up unattended-upgrades (add “kernel”; to the blacklist section because if you are on Digital Ocean, you should update the kernel through the console to ensure snapshots and other things continue to work):
sudo apt-get install unattended-upgrades
vi /etc/apt/apt.conf.d/50unattended-upgrades - Run
update-rc.d xxx disable (where xxx is the service you wish to switch off)
for any services you do not wish to run. Check services with service –status-all. Note: If you have 1GB RAM or less on your server, switch off Varnish and HHVM and use PHP7 – you will need to change your /etc/nginx/sites-available/domain.com file – change three things: 1) Change port to 80 instead of 8080, 2) Change primary method from hhvm to php, 3) Remove the fallback line in the primary method
- Set up a proper cron job – locate your wp-config.php file (/var/www/html/yourdomain/wp-config.php) and add:
define('DISABLE_WP_CRON', 'true');
Then run crontab -e and add:
*/1 * * * * wget -q -O – http://YOURWEBSITE.COM/wp-cron.php?doing_wp_cron - Change your timezone:
sudo dpkg-reconfigure tzdata
- Optionally configure longer timeouts and debug – useful if you’re going to be running large imports and such stuff
Edit wp-config.php:define ('WP_DEBUG', true);
define('WP_DEBUG_DISPLAY', false);
define('WP_DEBUG_LOG', true);
Edit /etc/nginx/sites-available/domain.com:
add fastcgi_read_timeout 120; to your location block
Edit /etc/php/7.0/fpm/php.ini:
Change max_execution_time to 120 Change upload_max_filesize to something better than 2M
- Increase the number of child processes that get spawned:
Edit /etc/php/7.0/fpm/pool.d/www.conf and change:pm.max_children 25
pm.start_servers 5
pm.max_spare_servers 5
(you can play with above settings depending on the RAM on your droplet)
- Increase the number of connections available to nginx – edit /etc/nginx/nginx.conf – something like this at the beginning:
events {
worker_connections 20000;
multi_accept on;
use epoll;
}
worker_rlimit_nofile 42000;
- Configure MySQL – vi /etc/mysql/my.cnf [mysqld section] (depends on memory available, engine used, and what you’re doing with wordpress)
key_buffer = 512M
max_allowed_packet = 512M
thread_stack = 192K
thread_cache_size = 8
query_cache_limit = 512M
query_cache_size = 4192M (or 0 can improve performance, it depends)
innodb_buffer_pool_size = 20000M (according to your memory - as a guide, use 60% of your memory here)
innodb_buffer_pool_instances = 16 (according to how many cores you have)
innodb_io_capacity = 5000 (because Digital Ocean use fast SSDs so we don't want InnoDB to be limited)
Restart your key services (nginx, varnish, hhvm, php7.0-fpm) or reboot your server with shutdown -r now. Your new server and wordpress installation is now ready, fast and secure.
Migrating your existing site
- Log onto your original server
- cd to your wp-content folder
- Create a backup of the mysql database using: mysqldump -u root -p dbname > sqldump.sql
- If you don’t have SSH access, create a template file and add the following code: <?php shell_exec(‘mysqldump -u root -p dbname > sqldump.sql’); ?> then create a page with that template and view the page – you should now have a backup file – this is useful to get round stupid hosts like Hostgator or Godaddy if they’ve disabled SSH and backups
- Log onto your new server
- cd to your wp-content folder
- Connect with FTP to your old server: sftp root@ipaddress
- Get everything using: get -r *
- Exit FTP
- Restore your database using:
- mysql -u root -p
- create database dbname;
- use dbname;
- source sqldump.sql; — replace sqldump.sql with the full path to your sqldump.sql file (e.g. /var/www/html/websitename/wp-content/sqldump.sql)
- Exit mysql and delete the sqldump.sql file
- You might need to copy across the contents of your wp-config.php file – have a look at the differences for both using FTP.
- Update /etc/nginx/sites-available/domain.com to point to the folder you downloaded from your old server
- Once you’re happy, switch your DNS settings to point your domain name at your new server
Set up W3 Total Cache
- Log onto your new website
- Install W3 Total Cache
- Set up page caching, minification, database caching all using Opcode: APC caching. There is a complete guide to setting up W3 Caching here BUT use Disk:Enhanced only for page caching – use OpCode:APC Caching for everything else. If you’re using Datafeedr (or for maximum compatibility with other plugins), don’t add object caching.
- Optionally set up Cloudflare or similar CDN – if you do, also install the Cloudflare plugin as it fixes some peculiarities with IP address origination
- Add the following to your wp-config.php file:
Configure email
Edit /etc/hosts – find the 127.0.0.1 line and add your fully qualified domain name (hostname.yourdomain.com):
127.0.0.1 hostname.yourdomain.com hostname
Then run the following:
apt-get install exim4-daemon-light mailutils
dpkg-reconfigure exim4-config
From the screens that follow, choose:
- Internet site
- Enter your fully qualified domain name
- Add your hostname and localhost to the list of servers that can send (as well as the fully qualified domain name) separated by semi-colons
- Leave ‘Domains to relay for’ and ‘Machines to relay for both blank
- Say ‘No’ to ‘Keep number of DNS-queries minimal (Dial-on-Demand)’?
- Leave other options at their defaults
Now your server can send mail which doesn’t bounce from WordPress.
That’s it – you now have a superfast WordPress installation – the fastest it POSSIBLY can CURRENTLY be for your server configuration.
The above runs on $5pcm Digital Ocean upwards, although $20pcm+ is recommended otherwise you’ll need to disable some services – e.g. only use one engine (HHVM or PHP7), disable varnish and disable memcached. You still get great performance and better than anything else you’ll find as this is tuned for WordPress.
If you don’t want to follow the steps above, don’t have time, or are confused, I’ve listed a service where you can pay us to set this up for you. Get that fast wordpress installation service here.
Adding a new website
The best way to add a new website is to copy the file in /etc/nginx/sites-available and configure the new file as per your new site.
February 22, 2015 @ 10:50 pm
Thanks Dave for this exceptional article! It works like a charm and i’m really excited to get my datafeedr API site up and running now on Digital Ocean.
March 26, 2015 @ 3:25 pm
This setup is fantastic, our sites are running much much faster! Thanks!
P.S. have you had any issues with this setup and the Datafeedr Bulk import tool getting stuck on the same image/product?
March 26, 2015 @ 3:32 pm
One thing to watch out for with Datafeedr is to ensure you don’t enable ‘Object caching’ in W3 Total Cache as it’s not compatible.
I’ve not experienced Datafeedr Bulk Import tool getting stuck on same image/product – make sure you have a proper cron job set up so you’re not relying on visitor traffic in order for your batch jobs to run – I made a guide for that here: https://www.wpintense.com/2014/11/22/set-real-cron-job-datafeedr-woocommerce-product-sets/
May 4, 2015 @ 6:55 pm
Hi Simon
This issue should be fixed in the newest version of the Datafeedr Product Sets plugin. It was an issue related to caching plugins and the Object cache.
Eric
May 4, 2015 @ 9:27 pm
That’s great news. I’ve got a client I’m setting up the mercury service for, so I’ll include the object caching and we can see what impact it makes.
March 27, 2015 @ 2:15 pm
We have this running on an 8gb 8 core server, what in your professional opinion would be the optimal hhvm settings for memory etc?
May 4, 2015 @ 9:30 pm
Wow – sorry I missed this comment. I’d need to measure the impact – it can depend on the size of your database, or how long your pages are taking to process. I’ve got plenty sites with 256MB script setting in HHVM that are running very well. You can track your uptime using http://www.uptimerobot.com for free as well, which is important if you’re scaling.
February 10, 2016 @ 8:16 am
i tried to install this diy mercury stack to digitalocean droplet with your kind explanation. and i think it was done succesfully .
but i couldn’t upload my wordpress theme . i did almost everything in the google so that solve this issues but it’s still not work.
what can i do for it?? can you give me some advice?
February 10, 2016 @ 9:56 am
Hi – is there any error message specifically? If you are saying you can’t upload your wordpress theme, I presume that means you got WordPress installed. Probably it’s your /etc/php/7.0/fpm/php.ini file – if you edit this you will find a line like this:
upload_max_filesize = 2M
Change the 2M to 24M and you should be able to upload your theme. That’s just a guess at what’s wrong as you didn’t give me very much to go on!
Or it could be your /etc/hhvm/php.ini file – same line (you will need to ADD it to the hhvm/php.ini file as it doesn’t exist in the first place). Depends what you have set up in your nginx /etc/nginx/sites-available/yourdomain.com file to process your PHP (either PHP 7 or HHVM)
February 10, 2016 @ 2:42 pm
thanks for your reply. the exist message from wordpress uploader was ‘are you really do this?’
i did edit php.ini php7.0 folder. and it didn’t works. i will try same things in hhvm. thanks a lot!
February 10, 2016 @ 4:54 pm
Now, another error code has been generated during uploading theme.
Error 503 Service Unavailable
Service Unavailable
Guru Meditation:
XID: 1771580132
Varnish cache server
should i off varnish??
February 10, 2016 @ 7:07 pm
If you have varnish enabled, make sure you access your back end with :8080 port which bypasses varnish.
e.g. http://www.yourdomain.com:8080/wp-admin/
Alternatively, disable varnish using the disable command above and:
service varnish stop
And then edit the port number in /etc/nginx/sites-available/yourdomain.com to 80 instead of 8080.
February 13, 2016 @ 9:06 am
Awesomesauce! Followed your instructions, step-by-step.
I got a bit confused as to how to change the configuration to disable Varnish and HHVM… but I think it’s working. *optimistic*
February 14, 2016 @ 4:44 pm
Good – glad to hear it – let me know how it goes. I’ll keep working on improving this article to make it easier to follow for future readers.
February 14, 2016 @ 3:09 pm
hhmm i was destory my droplet on digital ocean. then when i tried to reinstall mercury stack to droplet , during the process for playbook.yml
the message like this had appeared and failed.
February 14, 2016 @ 4:37 pm
Judo – what message are you seeing when running the playbook? If you’ve got error messages, the likeliest scenario is you’re trying to install to a different operating system other than Ubuntu 14.04 64 bit – e.g. maybe you went for 32 bit by mistake?
February 14, 2016 @ 4:52 pm
Hi, Dave.
Thanks for the guide. It’s truly awesome!
This mercury install uses and outdated version of nginx and varnish (v3 instead of v4).
How can I use this mercury installation with the latest version of nginx and varnish (v4 sintax is not compatible with the v3 syntax).
Thanks for your time!
February 14, 2016 @ 7:17 pm
Hi – one way to get the latest version of nginx and varnish is to run:
apt-get update
apt-get upgrade
Soon I’ll be making my own ansible script – maybe next week, maybe the week after – the main thing I’m aiming for is obviously the latest versions of nginx and varnish but most importantly I want nginx to be built with the ngx_http_realip_module built into it. You have to build from source to get this. It’s required if you’re using Cloudflare and Fail2ban together so that Fail2ban doesn’t end up banning the Cloudflare IP addresses.
February 15, 2016 @ 3:29 pm
That’s great news!
I was able, in a clean Ubuntu installation, to first install nginx (latest version) and only then run the ansible script for the mercury environment. This seems to run just fine.
Unfortunately this doesn’t work for varnish because when I run the mercury installation it fails because the syntax of the config file of varnish is incompatible with version 4.
Regarding SSL (https) I was able to configure nginx as a proxy for 443 port so I can have both nginx and varnish running on https since varnish doesn’t support SSL. It works fine! But I have an issue… When trying to log in in wp-admin the port defaults to 8080 (I’m using this is to bypass varnish in the backend) but because of that SSL stops working (it works fine in the frontend). I get this error messages in Chrome: SSL connection error and ERR_SSL_PROTOCOL_ERROR. Still trying to figure this out :/
February 15, 2016 @ 3:33 pm
Also, when you make your ansible script, please add the pagespeed mod and the newest version of openssl 1.0.2 so we can have ALPN when using http/2. That would be a great addition 🙂
Thanks once again for all the contributions you’ve made. The mercury environment is awesome!
March 12, 2016 @ 5:38 pm
hi, whats the meaning of this?
worker_rlimit_nofile 42000;
and how to adjust pm.max_request correctly?
thanks
April 5, 2016 @ 11:19 am
Hi – the worker_rlimit_nofile controls how many files can be opened at once by worker processes for nginx. This is important if you’re using a disk/file-based caching system like that offered by W3 Total Cache.
April 14, 2016 @ 12:00 am
Hello,
Great tutorial, But I need to upgrade the current Varnish 3.0 installed through guide here to Varnish 4, can you please help with that ?
May 27, 2016 @ 2:12 pm
This guide should help: https://uk.godaddy.com/help/how-to-install-varnish-on-ubuntu-1404-12387
June 2, 2016 @ 2:26 am
I think you missed what should be add in the wp-config in
Set up W3 Total Cache
5: Add the following to your wp-config.php file:
July 5, 2016 @ 1:33 pm
Yes – you’re right – to complete your W3 Total Cache config if you visit your site /wp-admin/admin.php?page=w3tc_install it will tell you the extra lines to enter to your nginx config file
June 30, 2016 @ 8:46 pm
Hi Dave,
I’ve been around Windows Servers since NT and 2000 versions, hosted on my own hardware. Now I’m learning Linux and love being able to spin up a $5 or $10 VPS at Digital Ocean in a few minutes.
I’m mostly hosting WordPress sites and find them on the slow side. I will also be installing WooCommerce on WP. Your Mercury stack seems the way to go. I have a few questions that might also benefit your other readers.
Q1: You discussed a newer Ansible Playbook that includes the newer NGINX and Varnish. If now available, can you please direct me? Otherwise, do you have any suggestions about how to update your tutorial to get the latest please.
Q2: You state only Ununtu 14.04 should be used. It’s now a year later so I’m wondering if Ubuntu 16.04 can work?
Q3: I would like to use the AJENTI.org control panel. Where in your tutorial should I install Ajenti?
Q4: With respect to WordPress and all the plugins, is PHP 7, for the fallback, better than PHP 5.6? I saw your post about fixes for major plugins for PHP7, but that leaves other plugins I might use with problems (potentially).
Thank you.
July 4, 2016 @ 4:11 pm
1. I don’t have it to hand right now and google is failing me – you can run this tutorial and then upgrade varnish and/or rebuild nginx if you wish
2. I haven’t yet tested it with Ubuntu 16.04 – I’d love it if someone would try and feedback?
3. I don’t have Ajenti installation in the tutorial yet. It’s something I’m offering for people who purchase the installation service but I can tell you it’s not that easy or straightforward to install. I’ll try and pull a guide together, but I may yet back off from Ajenti if I can’t get it to do everything I want.
4. It’s your choice – you can have PHP 7 + PHP 5.6 fallback if you wish, or HHVM + PHP 5.6 fallback. This ansible script installs PHP 5.6, so you will have that available as an option if you wish.
September 8, 2016 @ 7:32 am
for the life of me i could not get php7 working.. i searched for the php7.0-fpm.sock file and it wasn’t even in any of my directories.. anything come to mind as to why that could be?
November 17, 2016 @ 12:51 pm
The php7 file is in a different folder – a different subdirectory. /var/run/php/php7.0-fpm.sock
https://www.wpintense.com/2015/02/04/diy-mercury-installation-super-fast-wordpress-performance/
May 31, 2017 @ 12:54 pm
I’ve got the same issue, I have followed every step, but WordPress is telling me I’m using 5.6.99 as a fallback and I’ve also got a Max upload size of 0 – any ideas?
May 31, 2017 @ 1:03 pm
Yes if you followed the above script then the php file that contains the .sock reference needs to be edited. Note: don’t just change the PHP7 reference, there’s also a folder difference – that’ll likely be the issue.
May 31, 2017 @ 1:24 pm
Thanks Dave,
Sorry if I sound a bit thick, but could you elaborate one that a bit more, cannot find a php7.0-fpm.sock anywhere
December 5, 2016 @ 7:26 pm
Dear, I’m starting to test this optimized installation that seems to be very good as well as your article, however when you access the link: https://github.com/zach-adams/hgv-deploy-full I came across this warning: WARNING: DEPRECATED
Unfortunately I do not have the time to upkeep this project or provide updates for issues. I would recommend using Carl Alexander’s DebOps for WordPress project which does the same thing as this project. You can find it here:
https://github.com/carlalexander/debops-wordpress
Is it still worth following the old project or the new project?
March 21, 2017 @ 12:49 pm
Yes – the old script still works fine. But yes – the Debops replacement is superior and we’ve used it as the basis of our new Rocket Stack – https://www.wpintense.com/shop/rocket-stack/
March 10, 2017 @ 6:20 pm
This is great, but unless I’m missing something (which is very possible!) I don’t see Percona being installed in the instructions above.
March 21, 2017 @ 12:47 pm
Percona is installed through the ansible script.
February 5, 2018 @ 4:12 pm
Is this tutorial still relevant for 2018?
February 14, 2018 @ 9:45 am
Yes it still works and is still relevant, BUT there’s this improved article which is better: https://www.wpintense.com/2017/06/12/installing-and-configuring-fastest-possible-wordpress-stack-digital-ocean/
Or if you’re looking to make massive sites, build yourself a cluster. It’s really easy. https://www.wpintense.com/2018/01/09/setting-up-a-wordpress-cluster-for-huge-sites/