DIY Mercury installation for super fast WordPress performance

I’ve just released a new service to set up, migrate and configure your WordPress site for you on Digital Ocean using the latest and greatest Mercury Install. BUT, maybe you want to set this up yourself? If so, read on for the full guide.

This includes:

  • Nginx – web server (far faster than Apache)
  • Memcached and APC – caching engine which works with W3 Total Cache
  • Varnish – HTTP accelerator
  • HHVM with fallback to PHP-FPM – compiled PHP for much faster pages
  • Percona DB – faster than MySQL (100% compatibility)

Installing just Nginx will make your site a helluva lot faster and able to support many many more simultaneous users, but add in all these other features and you’ll be the envy of all your WordPress friends.

If you want to see some charts and stuff, check out the High Performance WordPress Installation Service page.

DIY Mercury Installation Steps

You need to know SSH and some Linux commands in order to build this yourself. Here’s the steps:

  1. Get yourself a Digital Ocean droplet (or any Ubuntu 14.04+ VPS but Digital Ocean use Raid SSD so they’re a great choice) (this affiliate link gets you $10 off)
  2. If you’re migrating, point new.yourdomain.com at your Digital Ocean IP address. If this is for a new site, point yourdomain.com at your Digital Ocean IP address
  3. Follow the instructions using this Ansible Playlist to the letter
  4. Install cURL, mcrypt and set up your firewall –
    sudo apt-get install php5-curl
    sudo apt-get install php5-mcrypt
    sudo ufw allow ssh
    sudo ufw allow 80/tcp
    sudo ufw allow 443/tcp
    sudo ufw allow 25/tcp
    sudo ufw enable
  5. Set up PHP7 for your faster fallback plan:
    sudo add-apt-repository ppa:ondrej/php
    sudo apt-get update
    sudo apt-get install -y tmux curl wget nginx php7.0-fpm php7.0-cli php7.0-curl php7.0-gd php7.0-intl
    sudo apt-get install php7.0-mysql php-memcached php7.0-mbstring php7.0-zip php7.0-xml

    vi /etc/nginx/conf.d/upstream.conf
    (Change php socket to new php7 socket)
    
    upstream php {
    server unix:/var/run/php/php7.0-fpm.sock;
    }
  6. Set up fail2ban to protect your server from attack (edit the jail.local file once you’ve created it, find the nginx section and change enabled to true):
    sudo apt-get install fail2ban
    cp /etc/fail2ban/jail.conf /etc/fail2ban/jail.local

    vi /etc/fail2ban/filter.d/wordpress.conf (add following lines)

    [Definition]
    failregex = ^<HOST>.*POST.*(wp-login.php|xmlrpc.php).* (403|499)
    Then vi /etc/fail2ban/jail.conf Change nginx enabled=true (instead of false) Change bantime at top to 86400 (1 day instead of 10 minutes) Add the following after nginx section:
    [wordpress]

    enabled = true

    port = http,https

    filter = wordpress

    logpath = /var/log/nginx/*access.log

    maxretry = 3
  7. Set up sendmail (not needed if you use an external service like Mandrill or Mailgun – note: more work needed to set up SPF records and make email get past Junk folders – use external service if in doubt)
    sudo apt-get install sendmail
    sudo sendmailconfig (choose Y and Y for answers to the questions)
  8. Set up unattended-upgrades (add “kernel”; to the blacklist section because if you are on Digital Ocean, you should update the kernel through the console to ensure snapshots and other things continue to work):
    sudo apt-get install unattended-upgrades
    vi /etc/apt/apt.conf.d/50unattended-upgrades
  9. Run
    update-rc.d xxx disable (where xxx is the service you wish to switch off)

    for any services you do not wish to run. Check services with service –status-all. Note: If you have 1GB RAM or less on your server, switch off Varnish and HHVM and use PHP7 – you will need to change your /etc/nginx/sites-available/domain.com file – change three things: 1) Change port to 80 instead of 8080, 2) Change primary method from hhvm to php, 3) Remove the fallback line in the primary method

  10. Set up a proper cron job – locate your wp-config.php file (/var/www/html/yourdomain/wp-config.php) and add:
    define('DISABLE_WP_CRON', 'true');
    Then run crontab -e and add:
    */1 * * * * wget -q -O – http://YOURWEBSITE.COM/wp-cron.php?doing_wp_cron
  11. Change your timezone:
    sudo dpkg-reconfigure tzdata
  12. Optionally configure longer timeouts and debug – useful if you’re going to be running large imports and such stuff
    Edit wp-config.php:
    define ('WP_DEBUG', true);
    define('WP_DEBUG_DISPLAY', false);
    define('WP_DEBUG_LOG', true);
    Edit /etc/nginx/sites-available/domain.com:
    add fastcgi_read_timeout 120; to your location block
    Edit /etc/php/7.0/fpm/php.ini:
    Change max_execution_time to 120
    Change upload_max_filesize to something better than 2M
  13. Increase the number of child processes that get spawned:
    Edit /etc/php/7.0/fpm/pool.d/www.conf and change:
    pm.max_children 25
    pm.start_servers 5
    pm.max_spare_servers 5

    (you can play with above settings depending on the RAM on your droplet)

  14. Increase the number of connections available to nginx – edit /etc/nginx/nginx.conf – something like this at the beginning:
    events {
     worker_connections 20000;
     multi_accept on;
     use epoll;
    }
    worker_rlimit_nofile 42000;
  15. Configure MySQL – vi /etc/mysql/my.cnf [mysqld section] (depends on memory available, engine used, and what you’re doing with wordpress)
    key_buffer = 512M
    max_allowed_packet = 512M
    thread_stack = 192K
    thread_cache_size = 8
    query_cache_limit = 512M
    query_cache_size = 4192M (or 0 can improve performance, it depends)
    innodb_buffer_pool_size = 20000M (according to your memory - as a guide, use 60% of your memory here)
    innodb_buffer_pool_instances = 16 (according to how many cores you have)
    innodb_io_capacity = 5000 (because Digital Ocean use fast SSDs so we don't want InnoDB to be limited)

Restart your key services (nginx, varnish, hhvm, php7.0-fpm) or reboot your server with shutdown -r now. Your new server and wordpress installation is now ready, fast and secure.

Migrating your existing site

  1. Log onto your original server
  2. cd to your wp-content folder
  3. Create a backup of the mysql database using: mysqldump -u root -p dbname > sqldump.sql
    1. If you don’t have SSH access, create a template file and add the following code: <?php shell_exec(‘mysqldump -u root -p dbname > sqldump.sql’); ?> then create a page with that template and view the page – you should now have a backup file – this is useful to get round stupid hosts like Hostgator or Godaddy if they’ve disabled SSH and backups
  4. Log onto your new server
  5. cd to your wp-content folder
  6. Connect with FTP to your old server: sftp root@ipaddress
  7. Get everything using: get -r *
  8. Exit FTP
  9. Restore your database using:
    1. mysql -u root -p
    2. create database dbname;
    3. use dbname;
    4. source sqldump.sql; — replace sqldump.sql with the full path to your sqldump.sql file (e.g. /var/www/html/websitename/wp-content/sqldump.sql)
  10. Exit mysql and delete the sqldump.sql file
  11. You might need to copy across the contents of your wp-config.php file – have a look at the differences for both using FTP.
  12. Update /etc/nginx/sites-available/domain.com to point to the folder you downloaded from your old server
  13. Once you’re happy, switch your DNS settings to point your domain name at your new server

Set up W3 Total Cache

  1. Log onto your new website
  2. Install W3 Total Cache
  3. Set up page caching, minification, database caching all using Opcode: APC caching. There is a complete guide to setting up W3 Caching here BUT use Disk:Enhanced only for page caching – use OpCode:APC Caching for everything else. If you’re using Datafeedr (or for maximum compatibility with other plugins), don’t add object caching.
  4. Optionally set up Cloudflare or similar CDN – if you do, also install the Cloudflare plugin as it fixes some peculiarities with IP address origination
  5. Add the following to your wp-config.php file:

Configure email

Edit /etc/hosts – find the 127.0.0.1 line and add your fully qualified domain name (hostname.yourdomain.com):

127.0.0.1 hostname.yourdomain.com hostname

Then run the following:

apt-get install exim4-daemon-light mailutils

dpkg-reconfigure exim4-config

From the screens that follow, choose:

  • Internet site
  • Enter your fully qualified domain name
  • Add your hostname and localhost to the list of servers that can send (as well as the fully qualified domain name) separated by semi-colons
  • Leave ‘Domains to relay for’ and ‘Machines to relay for both blank
  • Say ‘No’ to ‘Keep number of DNS-queries minimal (Dial-on-Demand)’?
  • Leave other options at their defaults

Now your server can send mail which doesn’t bounce from WordPress.

That’s it – you now have a superfast WordPress installation – the fastest it POSSIBLY can CURRENTLY be for your server configuration.

The above runs on $5pcm Digital Ocean upwards, although $20pcm+ is recommended otherwise you’ll need to disable some services – e.g. only use one engine (HHVM or PHP7), disable varnish and disable memcached. You still get great performance and better than anything else you’ll find as this is tuned for WordPress.

If you don’t want to follow the steps above, don’t have time, or are confused, I’ve listed a service where you can pay us to set this up for you. Get that fast wordpress installation service here.

Adding a new website

The best way to add a new website is to copy the file in /etc/nginx/sites-available and configure the new file as per your new site.

39 Comments
Show all Most Helpful Highest Rating Lowest Rating Add your review
  1. Is this tutorial still relevant for 2018?

  2. This is great, but unless I’m missing something (which is very possible!) I don’t see Percona being installed in the instructions above.

  3. Dear, I’m starting to test this optimized installation that seems to be very good as well as your article, however when you access the link: https://github.com/zach-adams/hgv-deploy-full I came across this warning: WARNING: DEPRECATED

    Unfortunately I do not have the time to upkeep this project or provide updates for issues. I would recommend using Carl Alexander’s DebOps for WordPress project which does the same thing as this project. You can find it here:

    https://github.com/carlalexander/debops-wordpress

    Is it still worth following the old project or the new project?

  4. for the life of me i could not get php7 working.. i searched for the php7.0-fpm.sock file and it wasn’t even in any of my directories.. anything come to mind as to why that could be?

  5. Hi Dave,
    I’ve been around Windows Servers since NT and 2000 versions, hosted on my own hardware. Now I’m learning Linux and love being able to spin up a $5 or $10 VPS at Digital Ocean in a few minutes.

    I’m mostly hosting WordPress sites and find them on the slow side. I will also be installing WooCommerce on WP. Your Mercury stack seems the way to go. I have a few questions that might also benefit your other readers.

    Q1: You discussed a newer Ansible Playbook that includes the newer NGINX and Varnish. If now available, can you please direct me? Otherwise, do you have any suggestions about how to update your tutorial to get the latest please.

    Q2: You state only Ununtu 14.04 should be used. It’s now a year later so I’m wondering if Ubuntu 16.04 can work?

    Q3: I would like to use the AJENTI.org control panel. Where in your tutorial should I install Ajenti?

    Q4: With respect to WordPress and all the plugins, is PHP 7, for the fallback, better than PHP 5.6? I saw your post about fixes for major plugins for PHP7, but that leaves other plugins I might use with problems (potentially).

    Thank you.

    • 1. I don’t have it to hand right now and google is failing me – you can run this tutorial and then upgrade varnish and/or rebuild nginx if you wish

      2. I haven’t yet tested it with Ubuntu 16.04 – I’d love it if someone would try and feedback?

      3. I don’t have Ajenti installation in the tutorial yet. It’s something I’m offering for people who purchase the installation service but I can tell you it’s not that easy or straightforward to install. I’ll try and pull a guide together, but I may yet back off from Ajenti if I can’t get it to do everything I want.

      4. It’s your choice – you can have PHP 7 + PHP 5.6 fallback if you wish, or HHVM + PHP 5.6 fallback. This ansible script installs PHP 5.6, so you will have that available as an option if you wish.

  6. I think you missed what should be add in the wp-config in

    Set up W3 Total Cache

    5: Add the following to your wp-config.php file:

    • Yes – you’re right – to complete your W3 Total Cache config if you visit your site /wp-admin/admin.php?page=w3tc_install it will tell you the extra lines to enter to your nginx config file

  7. Hello,

    Great tutorial, But I need to upgrade the current Varnish 3.0 installed through guide here to Varnish 4, can you please help with that ?

  8. hi, whats the meaning of this?

    worker_rlimit_nofile 42000;

    and how to adjust pm.max_request correctly?

    thanks

    • Hi – the worker_rlimit_nofile controls how many files can be opened at once by worker processes for nginx. This is important if you’re using a disk/file-based caching system like that offered by W3 Total Cache.

  9. Also, when you make your ansible script, please add the pagespeed mod and the newest version of openssl 1.0.2 so we can have ALPN when using http/2. That would be a great addition 🙂

    Thanks once again for all the contributions you’ve made. The mercury environment is awesome!

  10. Hi, Dave.

    Thanks for the guide. It’s truly awesome!

    This mercury install uses and outdated version of nginx and varnish (v3 instead of v4).

    How can I use this mercury installation with the latest version of nginx and varnish (v4 sintax is not compatible with the v3 syntax).

    Thanks for your time!

    • Hi – one way to get the latest version of nginx and varnish is to run:

      apt-get update
      apt-get upgrade

      Soon I’ll be making my own ansible script – maybe next week, maybe the week after – the main thing I’m aiming for is obviously the latest versions of nginx and varnish but most importantly I want nginx to be built with the ngx_http_realip_module built into it. You have to build from source to get this. It’s required if you’re using Cloudflare and Fail2ban together so that Fail2ban doesn’t end up banning the Cloudflare IP addresses.

      • That’s great news!

        I was able, in a clean Ubuntu installation, to first install nginx (latest version) and only then run the ansible script for the mercury environment. This seems to run just fine.

        Unfortunately this doesn’t work for varnish because when I run the mercury installation it fails because the syntax of the config file of varnish is incompatible with version 4.

        Regarding SSL (https) I was able to configure nginx as a proxy for 443 port so I can have both nginx and varnish running on https since varnish doesn’t support SSL. It works fine! But I have an issue… When trying to log in in wp-admin the port defaults to 8080 (I’m using this is to bypass varnish in the backend) but because of that SSL stops working (it works fine in the frontend). I get this error messages in Chrome: SSL connection error and ERR_SSL_PROTOCOL_ERROR. Still trying to figure this out :/

  11. hhmm i was destory my droplet on digital ocean. then when i tried to reinstall mercury stack to droplet , during the process for playbook.yml

    the message like this had appeared and failed.

    • Judo – what message are you seeing when running the playbook? If you’ve got error messages, the likeliest scenario is you’re trying to install to a different operating system other than Ubuntu 14.04 64 bit – e.g. maybe you went for 32 bit by mistake?

  12. Awesomesauce! Followed your instructions, step-by-step.

    I got a bit confused as to how to change the configuration to disable Varnish and HHVM… but I think it’s working. *optimistic*

  13. i tried to install this diy mercury stack to digitalocean droplet with your kind explanation. and i think it was done succesfully .

    but i couldn’t upload my wordpress theme . i did almost everything in the google so that solve this issues but it’s still not work.

    what can i do for it?? can you give me some advice?

    • Hi – is there any error message specifically? If you are saying you can’t upload your wordpress theme, I presume that means you got WordPress installed. Probably it’s your /etc/php/7.0/fpm/php.ini file – if you edit this you will find a line like this:

      upload_max_filesize = 2M

      Change the 2M to 24M and you should be able to upload your theme. That’s just a guess at what’s wrong as you didn’t give me very much to go on!

      Or it could be your /etc/hhvm/php.ini file – same line (you will need to ADD it to the hhvm/php.ini file as it doesn’t exist in the first place). Depends what you have set up in your nginx /etc/nginx/sites-available/yourdomain.com file to process your PHP (either PHP 7 or HHVM)

      • Now, another error code has been generated during uploading theme.

        Error 503 Service Unavailable

        Service Unavailable

        Guru Meditation:

        XID: 1771580132

        Varnish cache server

        should i off varnish??

      • thanks for your reply. the exist message from wordpress uploader was ‘are you really do this?’

        i did edit php.ini php7.0 folder. and it didn’t works. i will try same things in hhvm. thanks a lot!

  14. We have this running on an 8gb 8 core server, what in your professional opinion would be the optimal hhvm settings for memory etc?

    • Wow – sorry I missed this comment. I’d need to measure the impact – it can depend on the size of your database, or how long your pages are taking to process. I’ve got plenty sites with 256MB script setting in HHVM that are running very well. You can track your uptime using http://www.uptimerobot.com for free as well, which is important if you’re scaling.

  15. This setup is fantastic, our sites are running much much faster! Thanks!

    P.S. have you had any issues with this setup and the Datafeedr Bulk import tool getting stuck on the same image/product?

  16. Thanks Dave for this exceptional article! It works like a charm and i’m really excited to get my datafeedr API site up and running now on Digital Ocean.

Leave a reply

Super Speedy Plugins
Logo