tcS3 -- Upload directly to Amazon S3 from your Wordpress install

Amazon’s inexpensive, unlimited cloud storage system is an excellent asset backend for all websites and this all-inclusive plugin uses the AWS SDK for PHP to facilitate uploads directly from your WordPress instance to your S3 bucket.  This plugin requires nothing special from you — it has been tested for performance on shared hosting, VPS and dedicated servers and worked on each, out of the box, both on Apache and nginX. tcS3 has been tested on wordpress 3.7 – 4.0 and worked well on all versions.

How does it work?

There are several hooks which are fired during the WordPress upload process. This plugin listens for two of them, depending on the file type. For images it waits for the derivatives (all of the various sizes and crops) to be generated as independent files and then uploads all of them at once using the multipart upload capabilities of the AWS SDK for PHP. This minimizes how much time is added on to the upload (you’ll see some extra time spent on the “crunching stage” as this is when the upload happens… depending on the server’s load this time can vary). Delete operations within your library are also repeated on S3 — many other plugins in there will simply remove the entry but leave your indexable file in your bucket to suck up bandwidth and space-usage charges.

Can I use it in a multisite setting?

Yep! As the web administrator for the CUNY Graduate School of Journalism I have network activated this bad boy and all uploads go right to a central bucket for the school. Our environment hosts 800+ sites across 3 application servers and 2 database servers, so having the centralized bucket is absolutely necessary in order to keep assets available to clients hitting our load balancer without having to replicate files. We also benefit by having S3 make the lift for serving the images.

If you don’t network activate it, your users have the option of selecting it from the plugins menu and configuring it for their own private S3 bucket. Since WordPress is one of those CMSes that gets attacked a lot you may want to encourage your users to configure separate IAM keys only for their WordPress authentication and grant those keys access to their S3 bucket only… that way if they are exploited, their keys aren’t used to spin up a bunch of big EC2 boxes. They can just expire the keys and generate new ones, with minimal consequence if there is a breach. As a sysadmin you should employ the same practice when network activating.

Any recommendations for cutting cost?

We’re all on a budget, right? Well, my plugin is free, so you have that. Aside from that, S3 is pretty inexpensive on its face but there is one thing I do for clients in my managed hosting environment and for CUNY… Cloudflare. The DNS & Caching service fronts my S3 bucket so initial requests get cached and subsequent hits have their bandwidth requests handled by Cloudflare, which is free. Cloudflare is also a genuine CDN so it makes delivery time much faster for the folks who are accessing the web from some place other than the US’ east coast (where my buckets are). 

My plugin also has some cache parameters with default settings… even if you opt not to front your bucket with Cloudflare’s free CDN service, these settings will encourage your user’s browsers to cache images on initial download so subsequent hits will be pulled from their local storage as opposed to results in a request to your S3 bucket.

Where can I get it?

It’s right in the WordPress Plugin repository. It’s also on Github.

 

I installed the plugin and my site got slower?

This can happen on shared hosting, smaller servers or high traffic sites. By default the plugin assumes your server is NOT flawless and may fail to push some files to S3. It first captures the headers of the assumed file path on S3 to see if they come back with a response code between 200 and 399 as any of those 199 code values means the image resolved and loaded. If it doesn’t get a HTTP code in that range is checks your local server for the same file and, if the code comes back between 200 and 399, serves up that file instead via a 301 redirect. All of that checking can slow down PHP and if a lot of it is happening it can clog things up. I have written Apache and nginx configs which bypass wordpress entirely when the url contains tcS3_media and, at the web server level (as opposed to within the application layer of PHP) performs a hard 301 to S3 without first checking for the image. In my experience faster sites with some broken images are a much better user experience than a very slow site with all of its images.

nginx example config

location ~*/tcS3_media/ {
		rewrite ^.*/tcS3_media/(.+)$ http://bucketname.s3.amazonaws.com/uploadsubdirectory/$1 permanent; ##bucketname.s3.amazonaws.com should correlate to your bucket name. Upload subdirectory should be used if you have specified a directory within your bucket for uploads to be sent to. If you are uploading to the root just skip that level and go right to the $1
	}

Apache example config

    RewriteEngine On ##May already be set since WordPress uses this
    RewriteRule ^tcS3_media/(.*)$ http://bucketname.s3.amazonaws.com/uploadsubdirectory/$1 [R=301,L] ##bucketname.s3.amazonaws.com should correlate to your bucket name. Upload subdirectory should be used if you have specified a directory within your bucket for uploads to be sent to. If you are uploading to the root just skip that level and go right to the $1

 

I have a comment/complaint/bug to report. Where should I do it?

Creating a ticket on Github will be the most logical place. I’ll see it there. A review in WordPress will likely go unnoticed. Emailing me will probably be ignored/forgotten, especially if you fall into the category of “I read the message on my phone, on the LIRR and then forgot about it.”  Commenting this post may also help you get eyes on it.