forward-proxy

How it is built

Architecture overview

A forward proxy (egress gateway) sits between a group of clients and the internet and makes requests on their behalf. Clients are configured to use it, or the network redirects their traffic to it. For plain HTTP the proxy reads the request, applies policy, answers from its cache if it may, and otherwise fetches from the origin. For HTTPS it cannot see inside the connection, so by default it opens a tunnel (HTTP CONNECT) and relays bytes; if the organisation wants to inspect or cache encrypted traffic it intercepts TLS: it terminates the client's TLS with a certificate it mints for the requested host and starts a second TLS session to the origin. Organisations use it to control and log outgoing traffic, to save bandwidth with a shared cache, and to give servers in locked-down networks a single audited way out. The one idea: the proxy is a policy decision point on a relay, and the whole design is about deciding early, from as little information as possible (client identity, host name, port), and then relaying bytes cheaply.

Parts:

• Listener and protocol front — accepts connections, parses requests, multiplexes thousands of sockets — epoll event loop, HTTP/1.1 message parsing, HTTP/2
• Policy — who may go where: authentication, address and host rules, rate limits — Authorization models (RBAC, ABAC, ReBAC), OAuth 2 and OpenID Connect, radix-trie (no page yet), Bloom filter, Token bucket
• Tunnel and relay — CONNECT handling and the byte pump — http-connect-tunneling (no page yet), Zero-copy I/O
• TLS interception — decrypts, inspects, re-encrypts when policy says so — tls-interception (no page yet), TLS 1.3, x509-certificates (no page yet), Mutual TLS, Secrets management
• Name resolution — finds the origin address — DNS message format and client lookups, recursive-dns-resolution (no page yet), dnssec (no page yet), stale-while-revalidate (no page yet)
• Cache — stores responses and decides what may be reused — HTTP caching semantics, LRU and CLOCK, S3-FIFO, Buffer pool
• Upstream connections — reaches origins and parent proxies — connection-pooling (no page yet), Circuit breaker, Exponential backoff with jitter, Health checking
• Operations — keeps the proxy up and measurable — Process supervision, Graceful draining, hdr-histogram (no page yet), Coverage-guided fuzzing, Deterministic simulation testing, Primary-backup replication, Point-in-time recovery

 clients ──► (explicit proxy setting | transparent redirect)
               │
        ┌──────▼───────────────────────────────────────────────────┐
        │ listener ─► parse ─► auth ─► ACL / policy ─► route       │
        │                                   │                      │
        │   plain HTTP ───► cache lookup ──hit──► respond          │
        │                       │miss                              │
        │   CONNECT ──► splice (relay)   or   bump (TLS intercept) │
        │                       ▼                                  │
        │             DNS ─► upstream pool ─► origin | parent proxy│
        └──────────────────────────────────────────────────────────┘
          logs, metrics ─► access log, histograms

How the parts fit together

Listener and protocol front

The proxy is an I/O-bound relay with many mostly idle sockets, so the standard shape is an event loop over non-blocking sockets (epoll event loop): one thread per worker handles all of its connections, and several workers share the listening port. Squid, the long-lived open reference, runs this way and scales by starting several worker processes ("SMP mode"); its documentation describes workers that accept and process requests, and disker processes for disk I/O (Squid wiki, SmpScale).

A forward proxy receives requests in two forms. A request to a proxy carries the full target (GET http://example.com/path HTTP/1.1, the absolute form of RFC 9112) so the proxy knows the origin; a tunnel is requested with CONNECT example.com:443. The parser (HTTP/1.1 message parsing) must be strict because it is the first thing the internet sees: request smuggling comes from disagreements over message boundaries, so reject ambiguous framing (both Content-Length and Transfer-Encoding, duplicate lengths) rather than guessing. Inbound HTTP/2 to the proxy is optional and mostly a client-side concern; to the origins the proxy usually speaks HTTP/1.1 or HTTP/2 depending on what the origin offers.

In intercepting (transparent) mode the network steers port 80 and 443 traffic to the proxy without the client knowing. The proxy then lacks the explicit target: it has the original destination address (from the redirect), the Host header for HTTP, and the SNI hostname in the TLS ClientHello (RFC 6066) for HTTPS. Policy must work on those. Explicit mode is easier to reason about and to authenticate; transparent mode is easier to deploy.

Policy

Policy runs in a fixed order: identify the client (source address, then proxy authentication), check the target (host, port, method, URL pattern), apply per-user limits, and either allow, deny or redirect. Authentication uses the 407 Proxy Authentication Required challenge and Proxy-Authorization header of RFC 9110, backed by a directory or an identity provider (OAuth 2 and OpenID Connect tokens in modern deployments); the result feeds Authorization models, which usually reduce to rules over user groups and destination categories. Rule engines have two hot data structures:

• Address and host matching: CIDR ranges in a radix-trie (no page yet), host-suffix lookups in a reversed-label trie or hash set. A Bloom filter in front of a big blocklist makes the common "not blocked" answer cheap, with the real lookup only for hits.
• Rate limits and bandwidth shaping: a Token bucket per user or per network. Squid's delay pools are exactly buckets replenished at a set rate, grouped per host, per network or aggregate (Squid wiki, DelayPools). Safe defaults matter: the stock Squid configuration allows CONNECT only to port 443 (acl SSL_ports port 443 and http_access deny CONNECT !SSL_ports in the shipped config), because a proxy that tunnels to any port can be turned into a relay for mail or scans. Never expose a forward proxy to the internet without authentication: an open proxy is abused within hours.

Tunnel and relay

After the proxy accepts a CONNECT, it connects to the target and replies 200; from then on it copies bytes in both directions without parsing them (http-connect-tunneling, no page yet; the method is defined in RFC 9110). A tunnel needs two buffers and two directions of backpressure: stop reading from the side that is faster when the other side's send buffer is full, or memory grows without bound. Where the platform allows, use kernel splicing or Zero-copy I/O so the bytes do not pass through user space. Policy for tunnels is limited to what is known before encryption: client, destination host and port, SNI. Tunnels need idle timeouts, because long-lived idle connections otherwise accumulate.

TLS interception

When policy requires seeing inside HTTPS, the proxy does what is called bumping, or splicing when it chooses not to: tls-interception (no page yet) gives the proxy a private certificate authority whose root is installed on all managed clients. For each host the proxy mints a leaf certificate (x509-certificates, no page yet) signed by that CA, presents it to the client, and opens an independent TLS 1.3 (or older) session to the origin. The two sessions are separate: the proxy sees the plaintext and must itself validate the origin's certificate, or it turns an attacker's bad certificate into a trusted one.

Squid's peek and splice shows the decision logic: the proxy peeks at the ClientHello to learn the server name and then decides, per site, to splice (relay untouched) or bump, and avoids bumping non-TLS traffic (Squid wiki, SslPeekAndSplice). Splice sites such as banking and health, and apps with pinned certificates; bump only what policy requires. Constraints you will meet:

• In TLS 1.3 the server's certificate and later handshake messages are encrypted (RFC 8446), so the proxy cannot learn the real certificate by looking at the wire; it has to complete its own handshake first.
• Applications that pin their certificates, or clients that present a client certificate, break under bumping; keep them on the splice list.
• The CA private key is the most valuable secret in the system: whoever has it can impersonate every site to every managed client. Keep it in an HSM or a locked-down service (Secrets management), minting certificates through a narrow interface, and cache minted certificates per host to avoid a signature per connection.
• Privacy and legal rules often restrict what may be decrypted (health, finance); the splice list is the control.

Name resolution

The proxy resolves the origin name itself (for tunnels and for the absolute-form target), so it is a DNS client at volume (DNS message format and client lookups). Run a local caching recursive resolver or use the site's (recursive-dns-resolution, no page yet), keep results for the TTL, cache negative answers briefly, and serve a stale answer while refreshing in the background (stale-while-revalidate, no page yet) so a slow resolver does not stall requests. With DNSSEC (dnssec, no page yet) validation, a poisoned answer is rejected rather than cached. A transparent proxy has a second trap: it must connect to the address the client intended, not to whatever the proxy's own lookup of the Host header returns, or an attacker who controls the header can steer the proxy to a different server than the one policy checked.

Cache

The shared cache is what turns a forward proxy into a bandwidth saver. Rules for what may be stored and reused are those of HTTP caching semantics (RFC 9111): freshness lifetime, validators for conditional requests, Cache-Control, Vary, and private versus shared. A forward proxy is a shared cache, so responses meant for one user (private, or any response to an authenticated request unless marked shareable) must not be stored. Eviction is LRU and CLOCK in the classic design; S3-FIFO is a newer, more scan-resistant option. The in-memory part holds hot objects in a Buffer pool; the disk part is a store file with its index in memory. Squid's Rock store uses a database-style layout so several workers can share the disk cache, and its documentation says the disk object cache is shared among workers only with Rock (Squid wiki, SmpScale, RockStore). Collapsed forwarding, which makes concurrent requests for one object share one origin fetch, is a configuration option in Squid (collapsed_forwarding) and prevents a thundering herd on a popular new object.

Upstream connections

To reach origins, the proxy keeps idle connections for reuse (connection-pooling, no page yet): a new TCP and TLS handshake costs round trips, and reuse is the biggest latency win for plain HTTP traffic. Pools are keyed by origin and by the TLS identity used; never reuse a connection across security contexts. When an origin fails, the Circuit breaker stops sending it traffic and Exponential backoff with jitter governs retries. Chained setups send requests to a parent proxy (a regional or security gateway); Health checking detects a dead parent and falls back to direct or to a second parent.

Operations

Process supervision restarts crashed workers; Graceful draining finishes in-flight tunnels before a restart (tunnels can be hours long, so set a drain limit). Latency is tracked as histograms (hdr-histogram, no page yet) per destination class, not as averages. The parser and TLS code are attack surface; Coverage-guided fuzzing is the standard tool for the HTTP parser and the ICAP or helper interfaces.

The proxy is stateless apart from its cache, but its configuration (policy rules, blocklists, credentials, certificate authority material) is state with an owner. Primary-backup replication keeps that store on a primary that orders all changes and ships them to standbys that proxy nodes read from; the acknowledgement rule decides whether a confirmed rule change can be lost in a crash, and an external arbiter decides who is primary. Point-in-time recovery covers a wrong change that every standby has already copied, such as a blocklist entry that denies a whole domain: the store is rebuilt to the instant before it from a base backup plus the archived change log, and the nodes reload.

Deterministic simulation testing runs the listener, policy, tunnel and upstream pool in one thread against a simulated network and clock under a seeded scheduler, with half-closed tunnels, origins that stall, resets in the middle of a CONNECT and slow clients, so the byte pump's close-order and timeout bugs replay from a seed.

Data and control flow

One plain HTTP request:

1. The client sends GET http://origin/path to the proxy; the event loop reads it and the parser builds the request.
2. Authentication: no credentials yields 407; with credentials, the helper (directory or token check) returns the user. Results are cached for a short time because a directory call per request would dominate.
3. Policy: rules over user, destination, method; a deny returns an error page, an allow continues.
4. Cache lookup by method and URL (plus Vary): a fresh hit is served; a stale hit with a validator triggers a conditional request; a miss goes upstream.
5. Resolve the origin, take a pooled connection or open a new one, forward the request (strip hop-by-hop headers, add Via and optionally forwarding headers).
6. Stream the response to the client while storing it if it is cacheable. Time goes to the origin round trip on misses; on hits it goes to memory and socket copies.
7. Write an access-log line; update counters and histograms.

One HTTPS request through a tunnel (splice):

1. The client sends CONNECT host:443; authentication and policy run on host and port (and user).
2. Resolve and connect; reply 200; start the relay loop with backpressure on both directions.
3. The proxy never parses the data; the session ends when either side closes or the idle timer fires; the log records bytes and duration.

One HTTPS request with interception (bump): as above, but after the 200 the proxy finishes a TLS handshake with the client using a minted certificate for the SNI name, opens its own validated TLS session to the origin, and then treats the plaintext as ordinary HTTP requests: policy per URL, cache lookup, and optionally a content-inspection step.

Scaling

Limits in the order they bite:

1. Concurrent connections. An event loop handles tens of thousands per worker; raise file-descriptor limits, tune socket buffers, enable timeouts for idle sockets.
2. TLS handshakes when bumping: asymmetric crypto per connection on both sides. Add workers (one per core), session resumption, cached leaf certificates, hardware or faster curves.
3. Authentication and policy lookups. Cache helper results, keep rules in memory-resident structures, move heavy categorisation into a periodically refreshed data set.
4. Disk I/O for the cache. Use the shared-store design (Rock-like), separate disk processes, SSDs, and keep only objects worth storing.
5. Bandwidth. Hit rate decides it; forward-proxy hit rates are limited by how much traffic is cacheable, and encrypted traffic is invisible to the cache unless bumped, so bump only where it pays.
6. Single machine. Run several proxies behind a load balancer (consistent hashing on the destination host raises per-node hit rate), or a hierarchy with parent proxies.

Failure modes

Real implementations

The alternative: a layer-4 egress firewall with a DNS allow list, or a cloud NAT with no HTTP awareness. It is cheaper, never decrypts anything, and gives control only by IP and name; pick it when destination control is enough and per-request visibility is not required. A L7 proxy in the reverse direction solves the mirror problem for traffic coming in.

Sources

• IETF RFC 9110, HTTP Semantics (CONNECT, proxy authentication, Via). rfc
• RFC 9111, HTTP Caching. rfc
• RFC 9112, HTTP/1.1 (message framing, absolute-form). rfc
• RFC 8446, The Transport Layer Security (TLS) Protocol Version 1.3. rfc
• RFC 6066, TLS Extensions: Extension Definitions (server name indication). rfc
• RFC 3507, Internet Content Adaptation Protocol (ICAP), for external content inspection. rfc
• Squid wiki: SslBump peek and splice, SMP scale, Rock store, Delay pools; configuration reference
• mitmproxy, How mitmproxy works. docs