It may be that you’re running a migration for someone, or you’ve decided to switch to using Digital Ocean volumes. Whatever the reason, downloading files from S3 using rsync + fuse is very slow. Here’s how to transfer files quickly.
apt install python-pip pip install awscli --upgrade --user pip install --upgrade pip apt install awscli
Now, that’s AWS installed, you need to configure it with your access key and secret key.
It’ll ask you for your access key and secret key, so paste them into the window. Once done, you can download your bucket.
aws s3 sync s3://mybucket /root/s3dest/
That’s it – it’ll download the entire bucket now. Performance-wise, I have not benchmarked this, but it’s about 4 or 5 times faster than using fuse + rsync.
Chat to me directly through our on-site chat bubble.
Latest posts by Dave Hilditch (see all)
- Minor update for Scalability Pro released - December 2, 2018
- External Images 1.97 released - December 2, 2018
- External Images upgrade released – gallery support added - November 30, 2018