DIY Mercury installation for super fast WordPress performance

February 4, 2015 Dave Hilditch Stack Guides Server Stack, WordPress Speed 39

Author
Recent Posts

Talk to me

Dave Hilditch

Founder at WP Intense

Author of the WPI performance guides and creator and coder of the plugins sold by WP Intense.

Talk to me

DIY Mercury Installation Steps

You need to know SSH and some Linux commands in order to build this yourself. Here’s the steps:

Get yourself a Digital Ocean droplet (or any Ubuntu 14.04+ VPS but Digital Ocean use Raid SSD so they’re a great choice) (this affiliate link gets you $10 off)
If you’re migrating, point new.yourdomain.com at your Digital Ocean IP address. If this is for a new site, point yourdomain.com at your Digital Ocean IP address
Follow the instructions using this Ansible Playlist to the letter

Install cURL, mcrypt and set up your firewall –

sudo apt-get install php5-curl
sudo apt-get install php5-mcrypt
sudo ufw allow ssh
sudo ufw allow 80/tcp
sudo ufw allow 443/tcp
sudo ufw allow 25/tcp
sudo ufw enable

Set up PHP7 for your faster fallback plan:

sudo add-apt-repository ppa:ondrej/php
sudo apt-get update
sudo apt-get install -y tmux curl wget nginx php7.0-fpm php7.0-cli php7.0-curl php7.0-gd php7.0-intl 
sudo apt-get install php7.0-mysql php-memcached php7.0-mbstring php7.0-zip php7.0-xml

vi /etc/nginx/conf.d/upstream.conf

(Change php socket to new php7 socket)

upstream php {
 server unix:/var/run/php/php7.0-fpm.sock;
}

Set up fail2ban to protect your server from attack (edit the jail.local file once you’ve created it, find the nginx section and change enabled to true):

sudo apt-get install fail2ban
cp /etc/fail2ban/jail.conf /etc/fail2ban/jail.local

vi /etc/fail2ban/filter.d/wordpress.conf (add following lines)

[Definition]
failregex = ^<HOST>.*POST.*(wp-login\.php|xmlrpc\.php).* (403|499)

Then vi /etc/fail2ban/jail.conf Change nginx enabled=true (instead of false) Change bantime at top to 86400 (1 day instead of 10 minutes) Add the following after nginx section:

[wordpress]

enabled = true

port = http,https

filter = wordpress

logpath = /var/log/nginx/*access.log

maxretry = 3

Set up sendmail (not needed if you use an external service like Mandrill or Mailgun – note: more work needed to set up SPF records and make email get past Junk folders – use external service if in doubt)
```
sudo apt-get install sendmail
sudo sendmailconfig (choose Y and Y for answers to the questions)
```
Set up unattended-upgrades (add “kernel”; to the blacklist section because if you are on Digital Ocean, you should update the kernel through the console to ensure snapshots and other things continue to work):
```
sudo apt-get install unattended-upgrades
vi /etc/apt/apt.conf.d/50unattended-upgrades
```
Run
```
update-rc.d xxx disable (where xxx is the service you wish to switch off)
```
for any services you do not wish to run. Check services with service –status-all. Note: If you have 1GB RAM or less on your server, switch off Varnish and HHVM and use PHP7 – you will need to change your /etc/nginx/sites-available/domain.com file – change three things: 1) Change port to 80 instead of 8080, 2) Change primary method from hhvm to php, 3) Remove the fallback line in the primary method

Set up a proper cron job – locate your wp-config.php file (/var/www/html/yourdomain/wp-config.php) and add:

define('DISABLE_WP_CRON', 'true');
Then run crontab -e and add:
*/1 * * * * wget -q -O – http://YOURWEBSITE.COM/wp-cron.php?doing_wp_cron

Change your timezone:
```
sudo dpkg-reconfigure tzdata
```

Optionally configure longer timeouts and debug – useful if you’re going to be running large imports and such stuff
Edit wp-config.php:

define ('WP_DEBUG', true);

define('WP_DEBUG_DISPLAY', false);

define('WP_DEBUG_LOG', true);

Edit /etc/nginx/sites-available/domain.com:

add fastcgi_read_timeout 120; to your location block

Edit /etc/php/7.0/fpm/php.ini:

Change max_execution_time to 120
Change upload_max_filesize to something better than 2M

Increase the number of child processes that get spawned:
Edit /etc/php/7.0/fpm/pool.d/www.conf and change:
```
pm.max_children 25
```
```
pm.start_servers 5
```
```
pm.max_spare_servers 5
```
(you can play with above settings depending on the RAM on your droplet)
Increase the number of connections available to nginx – edit /etc/nginx/nginx.conf – something like this at the beginning:
```
events {
```
```
 worker_connections 20000;
```
```
 multi_accept on;
```
```
 use epoll;
```
```
}
```
```
worker_rlimit_nofile 42000;
```

Configure MySQL – vi /etc/mysql/my.cnf [mysqld section] (depends on memory available, engine used, and what you’re doing with wordpress)

key_buffer = 512M
max_allowed_packet = 512M
thread_stack = 192K
thread_cache_size = 8
query_cache_limit = 512M
query_cache_size = 4192M (or 0 can improve performance, it depends)
innodb_buffer_pool_size = 20000M (according to your memory - as a guide, use 60% of your memory here)
innodb_buffer_pool_instances = 16 (according to how many cores you have)
innodb_io_capacity = 5000 (because Digital Ocean use fast SSDs so we don't want InnoDB to be limited)

Restart your key services (nginx, varnish, hhvm, php7.0-fpm) or reboot your server with shutdown -r now. Your new server and wordpress installation is now ready, fast and secure.

Migrating your existing site

Log onto your original server
cd to your wp-content folder
Create a backup of the mysql database using: mysqldump -u root -p dbname > sqldump.sql
1. If you don’t have SSH access, create a template file and add the following code: <?php shell_exec(‘mysqldump -u root -p dbname > sqldump.sql’); ?> then create a page with that template and view the page – you should now have a backup file – this is useful to get round stupid hosts like Hostgator or Godaddy if they’ve disabled SSH and backups
Log onto your new server
cd to your wp-content folder
Connect with FTP to your old server: sftp root@ipaddress
Get everything using: get -r *
Exit FTP
Restore your database using:
1. mysql -u root -p
2. create database dbname;
3. use dbname;
4. source sqldump.sql; — replace sqldump.sql with the full path to your sqldump.sql file (e.g. /var/www/html/websitename/wp-content/sqldump.sql)
Exit mysql and delete the sqldump.sql file
You might need to copy across the contents of your wp-config.php file – have a look at the differences for both using FTP.
Update /etc/nginx/sites-available/domain.com to point to the folder you downloaded from your old server
Once you’re happy, switch your DNS settings to point your domain name at your new server

Set up W3 Total Cache

Log onto your new website
Install W3 Total Cache
Set up page caching, minification, database caching all using Opcode: APC caching. There is a complete guide to setting up W3 Caching here BUT use Disk:Enhanced only for page caching – use OpCode:APC Caching for everything else. If you’re using Datafeedr (or for maximum compatibility with other plugins), don’t add object caching.
Optionally set up Cloudflare or similar CDN – if you do, also install the Cloudflare plugin as it fixes some peculiarities with IP address origination
Add the following to your wp-config.php file:

Configure email

Edit /etc/hosts – find the 127.0.0.1 line and add your fully qualified domain name (hostname.yourdomain.com):

127.0.0.1 hostname.yourdomain.com hostname

Then run the following:

apt-get install exim4-daemon-light mailutils

dpkg-reconfigure exim4-config

From the screens that follow, choose:

Internet site
Enter your fully qualified domain name
Add your hostname and localhost to the list of servers that can send (as well as the fully qualified domain name) separated by semi-colons
Leave ‘Domains to relay for’ and ‘Machines to relay for both blank
Say ‘No’ to ‘Keep number of DNS-queries minimal (Dial-on-Demand)’?
Leave other options at their defaults

Now your server can send mail which doesn’t bounce from WordPress.

That’s it – you now have a superfast WordPress installation – the fastest it POSSIBLY can CURRENTLY be for your server configuration.

The above runs on $5pcm Digital Ocean upwards, although $20pcm+ is recommended otherwise you’ll need to disable some services – e.g. only use one engine (HHVM or PHP7), disable varnish and disable memcached. You still get great performance and better than anything else you’ll find as this is tuned for WordPress.

If you don’t want to follow the steps above, don’t have time, or are confused, I’ve listed a service where you can pay us to set this up for you. Get that fast wordpress installation service here.

Adding a new website

The best way to add a new website is to copy the file in /etc/nginx/sites-available and configure the new file as per your new site.

Share1 Tweet LinkedIn Email

39 Comments

Affiliater
February 22, 2015 @ 10:50 pm

Thanks Dave for this exceptional article! It works like a charm and i’m really excited to get my datafeedr API site up and running now on Digital Ocean.

Reply
Simon Haynes
March 26, 2015 @ 3:25 pm

This setup is fantastic, our sites are running much much faster! Thanks!

P.S. have you had any issues with this setup and the Datafeedr Bulk import tool getting stuck on the same image/product?

Reply
- Anon
  March 26, 2015 @ 3:32 pm
  
  One thing to watch out for with Datafeedr is to ensure you don’t enable ‘Object caching’ in W3 Total Cache as it’s not compatible.
  
  I’ve not experienced Datafeedr Bulk Import tool getting stuck on same image/product – make sure you have a proper cron job set up so you’re not relying on visitor traffic in order for your batch jobs to run – I made a guide for that here: https://www.wpintense.com/2014/11/22/set-real-cron-job-datafeedr-woocommerce-product-sets/
  
  Reply
- Eric
  May 4, 2015 @ 6:55 pm
  
  Hi Simon
  
  This issue should be fixed in the newest version of the Datafeedr Product Sets plugin. It was an issue related to caching plugins and the Object cache.
  
  Eric
  
  Reply
  - Anon
    May 4, 2015 @ 9:27 pm
    
    That’s great news. I’ve got a client I’m setting up the mercury service for, so I’ll include the object caching and we can see what impact it makes.
    
    Reply
Simon Haynes
March 27, 2015 @ 2:15 pm

We have this running on an 8gb 8 core server, what in your professional opinion would be the optimal hhvm settings for memory etc?

Reply
- Anon
  May 4, 2015 @ 9:30 pm
  
  Wow – sorry I missed this comment. I’d need to measure the impact – it can depend on the size of your database, or how long your pages are taking to process. I’ve got plenty sites with 256MB script setting in HHVM that are running very well. You can track your uptime using http://www.uptimerobot.com for free as well, which is important if you’re scaling.
  
  Reply
judo
February 10, 2016 @ 8:16 am

i tried to install this diy mercury stack to digitalocean droplet with your kind explanation. and i think it was done succesfully .

but i couldn’t upload my wordpress theme . i did almost everything in the google so that solve this issues but it’s still not work.

what can i do for it?? can you give me some advice?

Reply
- Anon
  February 10, 2016 @ 9:56 am
  
  Hi – is there any error message specifically? If you are saying you can’t upload your wordpress theme, I presume that means you got WordPress installed. Probably it’s your /etc/php/7.0/fpm/php.ini file – if you edit this you will find a line like this:
  
  upload_max_filesize = 2M
  
  Change the 2M to 24M and you should be able to upload your theme. That’s just a guess at what’s wrong as you didn’t give me very much to go on!
  
  Or it could be your /etc/hhvm/php.ini file – same line (you will need to ADD it to the hhvm/php.ini file as it doesn’t exist in the first place). Depends what you have set up in your nginx /etc/nginx/sites-available/yourdomain.com file to process your PHP (either PHP 7 or HHVM)
  
  Reply
  - judo
    February 10, 2016 @ 2:42 pm
    
    thanks for your reply. the exist message from wordpress uploader was ‘are you really do this?’
    
    i did edit php.ini php7.0 folder. and it didn’t works. i will try same things in hhvm. thanks a lot!
    
    Reply
  - judo
    February 10, 2016 @ 4:54 pm
    
    Now, another error code has been generated during uploading theme.
    
    Error 503 Service Unavailable
    
    Service Unavailable
    
    Guru Meditation:
    
    XID: 1771580132
    
    Varnish cache server
    
    should i off varnish??
    
    Reply
    - Anon
      February 10, 2016 @ 7:07 pm
      
      If you have varnish enabled, make sure you access your back end with :8080 port which bypasses varnish.
      
      e.g. http://www.yourdomain.com:8080/wp-admin/
      
      Alternatively, disable varnish using the disable command above and:
      
      service varnish stop
      
      And then edit the port number in /etc/nginx/sites-available/yourdomain.com to 80 instead of 8080.
      
      Reply
Marc
February 13, 2016 @ 9:06 am

Awesomesauce! Followed your instructions, step-by-step.

I got a bit confused as to how to change the configuration to disable Varnish and HHVM… but I think it’s working. *optimistic*

Reply
- Anon
  February 14, 2016 @ 4:44 pm
  
  Good – glad to hear it – let me know how it goes. I’ll keep working on improving this article to make it easier to follow for future readers.
  
  Reply
judo
February 14, 2016 @ 3:09 pm

hhmm i was destory my droplet on digital ocean. then when i tried to reinstall mercury stack to droplet , during the process for playbook.yml

the message like this had appeared and failed.

Reply
- Anon
  February 14, 2016 @ 4:37 pm
  
  Judo – what message are you seeing when running the playbook? If you’ve got error messages, the likeliest scenario is you’re trying to install to a different operating system other than Ubuntu 14.04 64 bit – e.g. maybe you went for 32 bit by mistake?
  
  Reply
Cala
February 14, 2016 @ 4:52 pm

Hi, Dave.

Thanks for the guide. It’s truly awesome!

This mercury install uses and outdated version of nginx and varnish (v3 instead of v4).

How can I use this mercury installation with the latest version of nginx and varnish (v4 sintax is not compatible with the v3 syntax).

Thanks for your time!

Reply
- Anon
  February 14, 2016 @ 7:17 pm
  
  Hi – one way to get the latest version of nginx and varnish is to run:
  apt-get update apt-get upgrade
  
  Soon I’ll be making my own ansible script – maybe next week, maybe the week after – the main thing I’m aiming for is obviously the latest versions of nginx and varnish but most importantly I want nginx to be built with the ngx_http_realip_module built into it. You have to build from source to get this. It’s required if you’re using Cloudflare and Fail2ban together so that Fail2ban doesn’t end up banning the Cloudflare IP addresses.
  
  Reply
  - Cala
    February 15, 2016 @ 3:29 pm
    
    That’s great news!
    
    I was able, in a clean Ubuntu installation, to first install nginx (latest version) and only then run the ansible script for the mercury environment. This seems to run just fine.
    
    Unfortunately this doesn’t work for varnish because when I run the mercury installation it fails because the syntax of the config file of varnish is incompatible with version 4.
    
    Regarding SSL (https) I was able to configure nginx as a proxy for 443 port so I can have both nginx and varnish running on https since varnish doesn’t support SSL. It works fine! But I have an issue… When trying to log in in wp-admin the port defaults to 8080 (I’m using this is to bypass varnish in the backend) but because of that SSL stops working (it works fine in the frontend). I get this error messages in Chrome: SSL connection error and ERR_SSL_PROTOCOL_ERROR. Still trying to figure this out :/
    
    Reply
Cala
February 15, 2016 @ 3:33 pm

Also, when you make your ansible script, please add the pagespeed mod and the newest version of openssl 1.0.2 so we can have ALPN when using http/2. That would be a great addition 🙂

Thanks once again for all the contributions you’ve made. The mercury environment is awesome!

Reply
Ferri
March 12, 2016 @ 5:38 pm

hi, whats the meaning of this?

worker_rlimit_nofile 42000;

and how to adjust pm.max_request correctly?

thanks

Reply
- Anon
  April 5, 2016 @ 11:19 am
  
  Hi – the worker_rlimit_nofile controls how many files can be opened at once by worker processes for nginx. This is important if you’re using a disk/file-based caching system like that offered by W3 Total Cache.
  
  Reply
Mohammad Farouk
April 14, 2016 @ 12:00 am

Hello,

Great tutorial, But I need to upgrade the current Varnish 3.0 installed through guide here to Varnish 4, can you please help with that ?

Reply
- Anon
  May 27, 2016 @ 2:12 pm
  
  This guide should help: https://uk.godaddy.com/help/how-to-install-varnish-on-ubuntu-1404-12387
  
  Reply
matthew
June 2, 2016 @ 2:26 am

I think you missed what should be add in the wp-config in

Set up W3 Total Cache

5: Add the following to your wp-config.php file:

Reply
- Anon
  July 5, 2016 @ 1:33 pm
  
  Yes – you’re right – to complete your W3 Total Cache config if you visit your site /wp-admin/admin.php?page=w3tc_install it will tell you the extra lines to enter to your nginx config file
  
  Reply
Peter Ode
June 30, 2016 @ 8:46 pm

Hi Dave,
I’ve been around Windows Servers since NT and 2000 versions, hosted on my own hardware. Now I’m learning Linux and love being able to spin up a $5 or $10 VPS at Digital Ocean in a few minutes.

I’m mostly hosting WordPress sites and find them on the slow side. I will also be installing WooCommerce on WP. Your Mercury stack seems the way to go. I have a few questions that might also benefit your other readers.

Q1: You discussed a newer Ansible Playbook that includes the newer NGINX and Varnish. If now available, can you please direct me? Otherwise, do you have any suggestions about how to update your tutorial to get the latest please.

Q2: You state only Ununtu 14.04 should be used. It’s now a year later so I’m wondering if Ubuntu 16.04 can work?

Q3: I would like to use the AJENTI.org control panel. Where in your tutorial should I install Ajenti?

Q4: With respect to WordPress and all the plugins, is PHP 7, for the fallback, better than PHP 5.6? I saw your post about fixes for major plugins for PHP7, but that leaves other plugins I might use with problems (potentially).

Thank you.

Reply
- Anon
  July 4, 2016 @ 4:11 pm
  
  1. I don’t have it to hand right now and google is failing me – you can run this tutorial and then upgrade varnish and/or rebuild nginx if you wish
  
  2. I haven’t yet tested it with Ubuntu 16.04 – I’d love it if someone would try and feedback?
  
  3. I don’t have Ajenti installation in the tutorial yet. It’s something I’m offering for people who purchase the installation service but I can tell you it’s not that easy or straightforward to install. I’ll try and pull a guide together, but I may yet back off from Ajenti if I can’t get it to do everything I want.
  
  4. It’s your choice – you can have PHP 7 + PHP 5.6 fallback if you wish, or HHVM + PHP 5.6 fallback. This ansible script installs PHP 5.6, so you will have that available as an option if you wish.
  
  Reply
lance
September 8, 2016 @ 7:32 am

for the life of me i could not get php7 working.. i searched for the php7.0-fpm.sock file and it wasn’t even in any of my directories.. anything come to mind as to why that could be?

Reply
- Dave H.
  November 17, 2016 @ 12:51 pm
  
  The php7 file is in a different folder – a different subdirectory. /var/run/php/php7.0-fpm.sock
  
  https://www.wpintense.com/2015/02/04/diy-mercury-installation-super-fast-wordpress-performance/
  
  Reply
  - Shane Barrett
    May 31, 2017 @ 12:54 pm
    
    I’ve got the same issue, I have followed every step, but WordPress is telling me I’m using 5.6.99 as a fallback and I’ve also got a Max upload size of 0 – any ideas?
    
    Reply
    - Anon
      May 31, 2017 @ 1:03 pm
      
      Yes if you followed the above script then the php file that contains the .sock reference needs to be edited. Note: don’t just change the PHP7 reference, there’s also a folder difference – that’ll likely be the issue.
      
      Reply
      - Shane Barrett
        May 31, 2017 @ 1:24 pm
        
        Thanks Dave,
        
        Sorry if I sound a bit thick, but could you elaborate one that a bit more, cannot find a php7.0-fpm.sock anywhere
Anderson Oliveira
December 5, 2016 @ 7:26 pm

Dear, I’m starting to test this optimized installation that seems to be very good as well as your article, however when you access the link: https://github.com/zach-adams/hgv-deploy-full I came across this warning: WARNING: DEPRECATED

Unfortunately I do not have the time to upkeep this project or provide updates for issues. I would recommend using Carl Alexander’s DebOps for WordPress project which does the same thing as this project. You can find it here:

https://github.com/carlalexander/debops-wordpress

Is it still worth following the old project or the new project?

Reply
- Anon
  March 21, 2017 @ 12:49 pm
  
  Yes – the old script still works fine. But yes – the Debops replacement is superior and we’ve used it as the basis of our new Rocket Stack – https://www.wpintense.com/shop/rocket-stack/
  
  Reply
Aaron
March 10, 2017 @ 6:20 pm

This is great, but unless I’m missing something (which is very possible!) I don’t see Percona being installed in the instructions above.

Reply
- Anon
  March 21, 2017 @ 12:47 pm
  
  Percona is installed through the ansible script.
  
  Reply
Mariyan Nenchev
February 5, 2018 @ 4:12 pm

Is this tutorial still relevant for 2018?

Reply
- Anon
  February 14, 2018 @ 9:45 am
  
  Yes it still works and is still relevant, BUT there’s this improved article which is better: https://www.wpintense.com/2017/06/12/installing-and-configuring-fastest-possible-wordpress-stack-digital-ocean/
  
  Or if you’re looking to make massive sites, build yourself a cluster. It’s really easy. https://www.wpintense.com/2018/01/09/setting-up-a-wordpress-cluster-for-huge-sites/
  
  Reply

DIY Mercury installation for super fast WordPress performance

DIY Mercury Installation Steps

Migrating your existing site

Set up W3 Total Cache

Configure email

Adding a new website

Related Posts

Installing Xdebug remotely to debug WordPress and gather profile traces

Migrating your WordPress website to Cloudways safely and quickly

How to configure nginx for Yoast SEO sitemaps

39 Comments

Leave a Reply Cancel reply