image-serving

How it is built

Architecture overview

An image-serving system stores one original of each picture and produces every size and format a page needs at the moment someone asks, instead of pre-rendering a fixed set. A URL names the original and the transformation (width=300, crop, format=avif); the server fetches the original, decodes, resizes, re-encodes and returns the bytes with cache headers, and a CDN in front keeps the result so that the second viewer never reaches the server. The design is therefore a cache hierarchy wrapped around an expensive pure function: transform(original, options) -> bytes. Three properties make it work. The function is deterministic, so any result can be cached or recomputed. The URL is authenticated, so strangers cannot ask for endless new variants. And the function runs in bounded memory, so a hostile or huge image cannot take the server down. Open examples are imgproxy, imaginary and thumbor; the commercial ones (Cloudinary and others) work the same way.

Parts:

• URL contract — what a request means and who may ask — Signed URLs, Message authentication codes (HMAC, CMAC, Poly1305, KMAC)
• Admission — keep load and variant count bounded — Token bucket
• Edge cache — most requests end here — CDN and edge, HTTP caching semantics
• Origin cache and routing — keep transformed images near the CPU that made them — Consistent hashing, S3-FIFO, W-TinyLFU, request-coalescing (no page yet)
• Source access — fetch and bound the original — Object storage
• Transform engine — decode, resize, convert colour, encode — tiled-image-pipeline (no page yet), Image resampling filters, Color space conversion, chroma-subsampling (no page yet), discrete-cosine-transform (no page yet), SIMD programming, Arena allocator
• Operations — keep it alive, observable and safe to change — Process supervision, Graceful draining, hdr-histogram (no page yet), Mutual TLS, Primary-backup replication, Point-in-time recovery
• Self-test — the decoders see hostile input — Coverage-guided fuzzing, Deterministic simulation testing

 browser ──GET /sig/w:300,fmt:avif/<source id>──► CDN edge ──hit──► bytes (Cache-Control, Vary: Accept)
                                                   │ miss
                                                   ▼
                          origin router: verify signature ─► rate limit ─► hash(variant key) → instance
                                                   ▼
                          instance: LRU/FIFO cache ──hit──► bytes
                                      │ miss: coalesce identical requests (one leader)
                                      ▼
                           fetch original (object store, size and time limits)
                                      ▼
                    header probe → reject if too many pixels → decode (tiled) → resize → colour → encode
                                      ▼
                        bytes ─► cache ─► response ─► CDN stores it

How the parts fit together

URL contract

The URL is the API. imgproxy's form is /%signature/%processing_options/%encoded_url.%extension (source: imgproxy docs, Signing a URL): signature first, then options, then the source and the requested format. Signed URLs are there for a security reason that the imgproxy docs state plainly: signing stops "an attacker" from "requesting multiple different image resizes" as a denial-of-service attack. Each unseen (source, options) pair costs CPU, so an open resize service is a CPU amplifier. The signature is a Message authentication code: in imgproxy it is an HMAC-SHA256 over the salt followed by the path, URL-safe Base64 encoded, and checking is disabled until a key and salt are configured, which the docs recommend for production. The verifier needs no lookup, which keeps the check cheap and stateless on every instance; the cost is that a leaked signed URL works until the key is rotated, so rotation and, if needed, an expiry in the signed part are part of the contract.

A second way to bound the variant space is not to let clients choose options at all. imgproxy has presets (named option sets) and a presets-only mode (IMGPROXY_ONLY_PRESETS) in which the URL carries only preset names such as thumbnail:watermarked (source: imgproxy docs, Presets). Use signing when many callers need free-form sizes; use presets when the design has a handful of image roles (avatar, card, hero) and you want a small, cacheable set of keys.

The URL contract also fixes the cache key: source id plus the normalised option set plus the negotiated output format. Normalise (sort options, apply defaults, clamp sizes) before hashing, or w=300,h=200 and h=200,w=300 become two entries.

Admission

Before any expensive work the origin checks load limits. A Token bucket per client key or source address allows bursts (a page of 30 thumbnails) but caps the sustained rate. In front of the CPU there is also a hard limit on concurrent transforms: imgproxy's IMGPROXY_WORKERS is the number of images processed at once without queueing, and it defaults to twice the number of CPU cores; IMGPROXY_REQUESTS_QUEUE_SIZE bounds the wait queue and requests beyond it are rejected with HTTP 429 (default 0, meaning unlimited; source: imgproxy docs, Configuration). The two controls do different jobs: the token bucket protects fairness between callers, the worker and queue limits protect the machine. Refusing early with 429 beats a queue that grows until requests time out, because retries feed the backlog.

Edge cache

CDN and edge holds the bytes close to viewers and absorbs almost all traffic; the origin sees only misses. What the CDN may do is set by HTTP caching semantics (RFC 9111): Cache-Control sets freshness; a validator such as ETag lets a stale entry be revalidated with a cheap conditional request; and Vary says which request headers select the variant. For images, Vary: Accept is the contract when the format is negotiated, because the same URL returns AVIF to one browser and JPEG to another; a cache that ignores it serves a format a client cannot decode. imgproxy can detect AVIF, WebP and JPEG XL support from the request and, when the URL omits an extension, choose the result format, with priority JPEG XL, then AVIF, then WebP (source: imgproxy docs); and its default Cache-Control: max-age is one year (IMGPROXY_TTL = 31,536,000 seconds), which is safe only because URLs are immutable: a changed original gets a new source id or version in the URL, never the same URL (source: same docs). IMGPROXY_ETAG_BUSTER exists to invalidate validators when processing configuration changes.

The alternative is to put the format in the URL (the page picks it), which gives the CDN the simplest key at the cost of moving the decision into every page.

Origin cache and routing

When a request misses the CDN it reaches one of several origin instances. If instances were chosen at random, each would cache a random 1/n of the variants and the miss ratio would be high. Instead Consistent hashing maps the cache key to an instance, so a variant is computed and stored in one place, and adding an instance moves only about 1/n of the keys. The cost is a hot-key problem, which the CDN normally absorbs.

Each instance keeps recent results in memory or on local disk, and the eviction policy matters because image traffic is a long tail of one-hit wonders next to a few hot images. S3-FIFO evicts items that were never reused quickly through a small queue and does no list moves on a hit; W-TinyLFU admits a newcomer only if its estimated frequency beats the victim's. Both are designed to beat plain LRU and CLOCK on such traces, so one-hit variants do not push out the hot ones.

request-coalescing (no page yet), also called singleflight, is the part that saves the most CPU at the moment of a cache miss. When a new image goes viral or the CDN is purged, hundreds of identical requests arrive within a millisecond; without coalescing, each one fetches the source and runs the transform. With it, the first becomes the leader, the others wait on its result, and the work runs once. The contract: the key is the full cache key, waiters have their own deadline, and a failed leader shares its error rather than letting all waiters retry at once.

Source access

The original lives in Object storage, reached by a source id, not by an arbitrary URL taken from the request. imgproxy defaults are instructive: the source allow-list IMGPROXY_ALLOWED_SOURCES is blank by default, which allows all source URLs, and the download timeout is 5 seconds (source: imgproxy docs); a production deployment restricts sources, because a server that fetches whatever URL the client names is an SSRF (server-side request forgery) hole into your network. Fetches stream into the decoder with a size limit; the original is read once per transform and never per request, which is what the caches are for. Small images make request price matter more than bytes.

Transform engine

This is the part that costs CPU and memory. The key idea of tiled-image-pipeline (no page yet) is demand-driven evaluation: nothing is computed until the output asks for it, and only the pixels in flight are kept in memory. libvips describes it as "only keeping the pixels currently being processed in RAM" together with threaded I/O; an image is read through regions (rectangular sub-areas) and a "partial image" stores a function that can produce any rectangle on demand (source: libvips docs, How it works). A resize of a 50-megapixel photo to 300 pixels wide then needs a few strips of the source at a time instead of a full decoded copy, and by default libvips uses as many threads as there are cores for one image (source: libvips docs, Using threads). imgproxy wraps libvips; imaginary uses it through a Go binding. The alternative, decoding the whole image into a flat buffer, is simpler and is what most scripts do, but its memory is proportional to the pixel count of the source, so a single hostile upload can exhaust a machine.

The stages and their blocks:

• Decode. Image and video codecs rest on block transforms such as the DCT (discrete-cosine-transform, no page yet); for JPEG the decoder can reduce resolution while decoding (at 1/2, 1/4 or 1/8 scale), which skips most of the work when the target is small (the resampling page lists it as the lever for large shrink factors).
• Chroma. chroma-subsampling (no page yet): JPEG and AVIF store colour at lower resolution than brightness (4:2:0 is common), so the decoder upsamples and the encoder subsamples again; the order matters for quality.
• Resize. Image resampling filters: separable passes with kernels that widen with the shrink factor, so a 10x reduction averages all the source pixels it replaces and does not alias.
• Colour. Color space conversion: resizing in linear light is more accurate than in the encoded (gamma) values, and wide-gamut sources must be converted or tagged for browsers that expect sRGB.
• Encode. AVIF, WebP and JPEG encoders trade CPU time for size; for AVIF the encode is typically the slowest stage (a general observation, not measured here).

SIMD programming matters in resize and colour loops, where the same arithmetic is applied to millions of pixels. An Arena allocator serves the many small, same-lifetime buffers of one transform (per-row weights, scratch strips) and frees them all at once when the request ends. A real problem the imgproxy docs describe: libvips uses GLib's allocator, which can fragment in a heavily multi-threaded process, so imgproxy documents MALLOC_ARENA_MAX, jemalloc and TCMalloc as remedies (source: imgproxy docs, Memory usage tweaks).

Safety limits belong to the engine's contract, checked on the header before decoding: imgproxy rejects sources above IMGPROXY_MAX_SRC_RESOLUTION (default 50 megapixels), limits animation frames (default 1) and can cap the result dimension (source: imgproxy docs). A "decompression bomb" is a small file that decodes into a huge raster, so the check is on declared pixels, not file size.

Operations and self-test

Process supervision restarts a crashed worker (native decoders do crash on bad input), and Graceful draining lets a deploy finish in-flight transforms: imgproxy waits for ongoing requests for IMGPROXY_GRACEFUL_STOP_TIMEOUT, by default twice the processing timeout, which defaults to 10 seconds. Latency is recorded in an hdr-histogram (no page yet), split by cache hit, miss and format, because a mean over hits and misses describes neither. Mutual TLS secures CDN-to-origin and origin-to-store links if they cross networks you do not control. The store that holds originals needs Primary-backup replication or the replication its provider offers and Point-in-time recovery against accidental deletes; derived images need neither, since they can be recomputed.

Decoders parse attacker-controlled bytes, which makes them a classic target for Coverage-guided fuzzing; libvips shows an OSS-Fuzz status badge in its README. Fuzz every codec you enable; disable formats you do not serve. Deterministic simulation testing fits the coordination logic (coalescing, the admission queue, cache stampede on a purge) with simulated time and a seeded scheduler.

Data and control flow

A cache miss, end to end:

1. The browser requests a signed URL; the CDN does not have it and forwards the request to the origin.
2. The origin verifies the HMAC signature over salt and path (reject before any other work), applies the token bucket, and parses the options against the presets or the allowed ranges.
3. It builds the cache key, hashes it, and routes to the owning instance.
4. The instance looks in its cache; on a miss it joins or starts a coalesced fetch for that key.
5. The leader downloads the original from object storage, bounded by the download timeout and size limit.
6. It reads the header, rejects too many pixels, then decodes in tiles, shrinks on load where possible, resizes, converts colour, encodes.
7. The result goes into the instance cache, the waiters are released, the response is sent with Cache-Control, ETag and Vary, and the CDN stores it. Time goes first to the encode and decode (CPU), then the source fetch (network), and only a little to signature checking; a hit costs a hash and a memory read.

A source update: the new original is uploaded under a new id or version, the page references the new URL, and old variants age out without a purge. Where the URL cannot change, a purge by key on the CDN is the expensive path.

Scaling

Limits come in this order:

1. CPU on the origin. A miss costs a decode, a resize and an encode. Levers: shrink-on-load, a cheaper encoder setting for variants that are rarely viewed, workers capped at the core count, SIMD kernels.
2. Miss ratio at the CDN. Fewer distinct URLs means more hits: presets or a small allowed size set, rounding requested widths up to the next bucket, format in the URL or a controlled Vary.
3. Memory per transform. Tiled evaluation, resolution limits, allocator choice.
4. Origin cache efficiency. Hash routing to keep one copy per variant, a scan-resistant eviction policy, coalescing.
5. Source fetch. Origin shielding (a second cache layer at the CDN) and keeping the store in the same region.
6. Hot keys. A viral image concentrates on one instance after hashing; the CDN is the first defence, replicas of the hottest keys the second.

Failure modes

Real implementations

The alternative: pre-generate a fixed set of sizes at upload. It is the simplest design, has no CPU amplifier and no cache stampede, and fits a small catalogue with few image roles. On-the-fly resizing pays off when designs change often, when the number of sizes and formats is large, or when new formats (AVIF) should reach old images without a batch job.

Sources

• imgproxy documentation: Signing a URL, Configuration options, Presets, Memory usage tweaks. docs.imgproxy.net
• libvips documentation: How it works (technical background), Using threads. libvips.org
• libvips README (demand-driven, horizontally threaded; fuzzing badge). github.com/libvips/libvips
• RFC 9111, HTTP Caching. rfc-editor.org
• RFC 9110, HTTP Semantics (the Vary header). rfc-editor.org
• imaginary and thumbor READMEs. imaginary, thumbor