Adding NGINX caching on-top of this is pretty trivial.
Also, heads up, in the directive proxy_cache_path, they should consider enabling "use_temp_path". This directive instructs NGINX to write them to the same directories where they will be cached. We recommend that you set this parameter to off to avoid unnecessary copying of data between file systems. use_temp_path was introduced in NGINX version 1.7.10 and NGINX Plus R6.
use_temp_path=off
Also, they should enable "proxy_cache_revalidate". This saves on bandwidth, because the server sends the full item only if it has been modified since the time recorded in the Last-Modified header.
This is vulnerable to path expansion attacks. If someone passes a URL such as your site/s3/..EVIL_BUCKET/EVIL.js all of a sudden your site is serving someone else's content. Bad idea. Use virtual host style buckets instead, i.e S3_bucket.S3host/content.
Immutable blobs are really the right choice with s3, as it's eventually-consistent (when using it as a blob store anyway. If you're hosting a static site or similar it's a bit tricky to immutableize and not necessarily worth the effort).
Yep, file shas are a great choice. UUIDs are typically fine too.
One sort of weird case is if I have an image key (sha-based) and want to store thumbnail sizes:
'bae6ff187e4c491e5de9cfa3b039ce7da8255798' makes sense as a base key, but really I want bae6ff187e4c491e5de9cfa3b039ce7da8255798/400x400 for thumbnails rather than storing individual thumbnail shas, hah.
Good config, but if you're not defining the proxy in an upstream {} block, you can't make use of the keepalive parameter, which keeps a number of connections to the backend alive at any time, reducing the RTT for an actual request.
This is bad for stuff like this because nginx doesn't re-resolve DNS records after process startup. So if an IP address behind the hostname changes, things will just hard stop working. Using it explicitly as a variable coerces nginx into actually resolving DNS regularly to pick up changes like a normal client.
Also, heads up, in the directive proxy_cache_path, they should consider enabling "use_temp_path". This directive instructs NGINX to write them to the same directories where they will be cached. We recommend that you set this parameter to off to avoid unnecessary copying of data between file systems. use_temp_path was introduced in NGINX version 1.7.10 and NGINX Plus R6.
Also, they should enable "proxy_cache_revalidate". This saves on bandwidth, because the server sends the full item only if it has been modified since the time recorded in the Last-Modified header.