Transfer files quickly from Amazon S3
- Breaking Through WooCommerce Restrictions with Super Speedy Filters: The Power of Pretty Permalinks - May 10, 2023
- Configuring Cloudflare for WordPress and WooCommerce: Boost Your Website Speed and Security - May 10, 2023
- Boost Your Marketplace Performance: Combining WCFM with the Super Speedy Pack - April 24, 2023
It may be that you’re running a migration for someone, or you’ve decided to switch to using Digital Ocean volumes. Whatever the reason, downloading files from S3 using rsync + fuse is very slow. Here’s how to transfer files quickly.
apt install python-pip
pip install awscli --upgrade --user
pip install --upgrade pip
apt install awscli
Now, that’s AWS installed, you need to configure it with your access key and secret key.
aws configure
It’ll ask you for your access key and secret key, so paste them into the window. Once done, you can download your bucket.
aws s3 sync s3://mybucket /root/s3dest/
That’s it – it’ll download the entire bucket now. Performance-wise, I have not benchmarked this, but it’s about 4 or 5 times faster than using fuse + rsync.