rum

How it is built

Architecture overview

Real-user monitoring (RUM) measures what actual visitors experience, from inside their browsers. A small script in the page reads the browser's own performance instrumentation (navigation and resource timings, paint and layout metrics, JavaScript errors), optionally records the session so it can be replayed, and sends the result to a collector with a beacon. The collector enriches each event (country and network from the IP address, browser and page from the user agent and URL, readable stack traces from minified code), samples, and writes it to two places: a stream that maintains windowed aggregates for dashboards and alerts, and a columnar store for ad-hoc questions.

The one idea that makes it work is that the data volume is set by traffic, not by what you want to know, so every stage reduces it as early as it can: sample at the client, aggregate to mergeable sketches in the stream, and keep raw events only in a compressed, sorted, columnar form for a limited time. Percentiles are the headline number, and a percentile can only be combined across pages, countries and minutes if it is stored as a sketch.

Parts:

• Browser agent — collects timings, vitals, errors and a session event stream inside the page, and sends them without blocking it — browser-performance-apis (no page yet)
• Session recorder — records DOM snapshots and changes so a session can be replayed, with sensitive fields masked — session-replay (no page yet), built on Snapshot delta compression, deflate (no page yet) and Data masking
• Sampler — decides which sessions and events are kept, and caps the rate — telemetry-sampling (no page yet), built on Token bucket
• Enricher — adds geography and network from the IP, normalises user agents and URLs — longest-prefix-match (no page yet), Regex engine
• Error symbolicator — turns minified stack frames into source locations — symbolication (no page yet), built on Interval tree
• Stream aggregator — maintains per-window counts and distributions as events arrive — Stream windowed aggregation
• Sketch layer — percentiles and distinct counts in small mergeable states — quantile-sketches (no page yet), HyperLogLog
• Rollup and store — pre-aggregated rollups at several resolutions plus raw events in a column store — Rollup aggregation, Columnar storage

 browser ──► agent (PerformanceObserver, error hooks, recorder)
              │ beacon: batched, compressed (deflate), sampled-in sessions only
              ▼
          collector ──► rate limit / sampler ──► enrich: IP prefix → geo/ASN, UA, URL template
              │                                         │ errors: symbolicate with source maps
              ▼                                         ▼
        event log (partitioned) ──► stream aggregator (event-time windows, watermark)
              │                         │ sketches: quantiles, HLL, counts
              │                         ▼
              │                    rollups: 1 min → 1 h → 1 day  ──► dashboards, alerts
              ▼
        column store (raw events, sorted by time, dictionary + delta + bit-packed)
              └──► ad-hoc queries, session list ──► replay viewer (events from the recorder)

Two readers use the output. Dashboards and alerts read rollups: small, precomputed, cheap. Investigation reads the raw columns and, for a single user, the session recording. Keeping them separate lets each be tuned for its own access pattern.

How the parts fit together

Browser agent

The browser already measures most of what matters. The agent subscribes to the standard APIs (browser-performance-apis, no page yet): Navigation Timing for the page load phases, Resource Timing for each fetched resource (cross-origin resources expose detailed timings only when the server sends a Timing-Allow-Origin header), Paint Timing and Largest Contentful Paint for rendering, and layout-shift and interaction events for stability and responsiveness. The three Core Web Vitals are Largest Contentful Paint, Interaction to Next Paint and Cumulative Layout Shift; the published guidance is a good LCP within 2.5 s and a CLS of 0.1 or less, judged at the 75th percentile of page loads, segmented by mobile and desktop (web.dev). That last point decides the backend: the target statistic is a percentile over a population, not a mean.

Contracts for the agent:

• Never block the page. Observe, buffer, and send from idle time or on page hide using a beacon, so the measurement does not worsen what it measures.
• Size budget. The script is on the critical path of every page view; keep it small and load it async.
• Dimension capture. Attach only low-cardinality dimensions on the client (page template, release, device class); derive the rest on the server.
• Reliability is partial. Pages are closed mid-flight and beacons lost. RUM numbers describe the visitors whose beacon arrived, a bias to state when comparing with server logs.

Session recorder

Session replay (session-replay, no page yet) reconstructs what a visitor saw. The approach used by rrweb is an initial serialised snapshot of the DOM and its state, followed by a stream of incremental events for each mutation and user interaction; a player rebuilds the snapshot and re-applies the events in order. This is a snapshot-plus-delta design, the same idea as Snapshot delta compression: send the full state once, then only what changed. The recording is compressed before upload with deflate (no page yet) since DOM text compresses well.

Replay is where privacy risk concentrates. Data masking is applied in the browser, before data leaves the device: form inputs are replaced by placeholders, elements marked as private are not serialised, and nothing sensitive reaches the collector to be scrubbed later. Recordings are also the most expensive data in the system, so they get the most aggressive sampling.

Sampler

Sampling (telemetry-sampling, no page yet) bounds cost and must keep results unbiased:

• Decide per session, not per event. Hash the session id (hash-based sampling, no page yet) and keep sessions below a threshold, so every event of a kept session is present and funnels and replays are complete. Sampling events independently would break any per-session metric.
• Record the rate. Each kept event carries its sampling probability so counts can be scaled back up. The OpenTelemetry consistent-probability sampling specification describes how a threshold carried with the data lets later stages stay consistent.
• Always keep the interesting ones. Errors and very slow loads can be kept at a higher rate than ordinary sessions, provided their rates are recorded and used when computing totals.
• Protect the collector. A Token bucket per client or per site caps the rate on ingest; a buggy page that fires beacons in a loop is dropped at the edge, not billed. reservoir-sampling (no page yet) is the tool when a bounded number of examples is wanted from an unbounded stream, such as "ten example sessions per error".

Enricher

Enrichment adds dimensions that the browser cannot or should not supply.

• Where the visitor is. The IP address is matched against a table of network prefixes to get country, region and network operator: longest-prefix-match (no page yet). The table is rebuilt from a geolocation database on a schedule. Store the derived fields, then drop or truncate the address according to the privacy policy.
• What they use. The Regex engine classifies user-agent strings and rewrites URLs into templates (/user/123/orders becomes /user/:id/orders), which keeps the page dimension at a few hundred values instead of millions. Use a linear-time regex engine here: these patterns run on untrusted input at ingest rate, so a backtracking engine is a denial-of-service risk.

Error symbolicator

Production JavaScript is minified, so a stack trace names positions in one huge line. Symbolication (no page yet) maps them back using the source maps (no page yet) uploaded at build time (ECMA-426 specifies the format), keyed by the release identifier the agent attaches. Looking up the original position for a generated one is a range query over sorted mappings, a job for an Interval tree or a binary search over the sorted segments. Group errors by a fingerprint computed from the symbolicated top frames and the message with variable parts removed, so a thousand reports of the same bug are one issue.

Stream aggregator

Raw events flow into Stream windowed aggregation: tumbling or sliding windows over event time (when it happened in the browser), not arrival time. Mobile and flaky networks deliver late, so the aggregator keeps a window open until a watermark (watermarks, no page yet) says no more events are expected, then emits the result; stragglers after that go to a correction path. Window timers are held on a Hierarchical timing wheel. Per window and dimension set it keeps a counter, a sketch of the metric values, and a sketch of distinct sessions.

Sketch layer

Two questions need sketches, because exact answers need unbounded state or cannot be merged.

• Percentiles. LCP at the 75th percentile for a page, per country, per hour. A quantile sketch (quantile-sketches, no page yet) keeps a bounded summary with a stated error bound, and two summaries merge into one, so minute windows roll up to hours and countries roll up to a global figure. Averaging percentiles is wrong; merging sketches is right.
• Distinct counts. "How many users hit this error?" HyperLogLog counts distinct ids in a small fixed state (12 KB at 16,384 registers) with a relative error of about 1.04/√m for m registers (Flajolet et al. 2007), and also merges, by taking the maximum per register.

Rollup and store

Rollup aggregation keeps the aggregate states, not the events, at coarser resolutions (1 minute, 1 hour, 1 day) in time partitions: recent data at fine resolution, old data coarse, so a 30-day chart reads a few hundred rows. Raw events go to Columnar storage, in which each field is stored as its own array. Low-cardinality fields (country, browser, page template) compress with dictionary encoding (no page yet), timestamps with Delta encoding, and small integers with Bit packing; zone maps (no page yet) let a query skip blocks whose min/max exclude the filter. A query for "p75 LCP of /checkout on mobile in Germany last week" then scans three or four columns and skips most blocks. Raw events are kept for days to weeks (a retention choice), rollups far longer.

Data and control flow

One page view

1. The agent is loaded. A hash of the session id decides whether this session is sampled in (rate recorded).
2. The browser fires its timing and paint entries; the agent records them and attaches release, template and device class.
3. On page hide the agent sends one batched, compressed beacon (plus recorder chunks for replay sessions).
4. The collector applies the rate limit, validates the schema, and appends to the event log, partitioned by site.
5. The enricher resolves the IP prefix to geography, parses the user agent, normalises the URL and, for errors, symbolicates and fingerprints.
6. The stream aggregator assigns the event to its event-time windows and updates counters and sketches; late events within the allowed lateness update an already-open window.
7. When a window closes, its states are written as a rollup row; the same event is appended to the column store.

One dashboard query: pick the coarsest rollup that satisfies the requested resolution, read the sketch states for the filter, merge them across the dimensions being collapsed, and read the percentile from the merged sketch. This is milliseconds of work on small states; the raw store is used only when a dimension was not rolled up.

One investigation: a spike in p75 LCP on one release leads to a filter on that release in the raw columns, a list of slow sessions, and a replay of one of them if it was recorded.

Scaling

1. Ingest rate. Levers: client sampling per session, token buckets at the edge, a stateless collector fleet behind a load balancer, a partitioned log between collector and processors.
2. Cardinality. URL templates, a fixed set of dimensions, caps per dimension with an "other" bucket. Unbounded dimensions (raw URLs, free-form error text) are the common cause of exploding cost.
3. Aggregation state. Windows × dimension sets × sketch size. Partition the stream by dimension key; keep sketches small; shorten allowed lateness.
4. Storage. Compress with column encodings, partition by time, drop raw data first, keep rollups.
5. Replay. Recordings dominate bytes: sample hardest, cap length, store separately from events.
6. Query. Pre-aggregation first, then column pruning and block skipping, then more nodes.

Failure modes

Real implementations

The alternative: synthetic monitoring (see Synthetic monitoring) measures one scripted client from fixed places on a fixed schedule: stable and available before launch, but blind to real devices and networks. RUM is noisy and only exists once users arrive. Most teams run both.

Sources

• web.dev, Web Vitals: thresholds and the 75th-percentile assessment. page
• W3C, Navigation Timing Level 2. spec; Resource Timing. spec; Paint Timing. spec; Largest Contentful Paint. spec
• P. Flajolet, É. Fusy, O. Gandouet, F. Meunier, HyperLogLog: the analysis of a near-optimal cardinality estimation algorithm, 2007. paper
• C. Masson, J. E. Rim, H. K. Lee, DDSketch, 2019. paper
• OpenTelemetry, Probability sampling with consistent thresholds. spec
• ECMA-426, Source map format specification. spec
• rrweb README (snapshot, record, replay). repo