marketing-measurement

How it is built

Architecture overview

A marketing measurement system answers one question: what did each euro of spend cause, as opposed to what merely happened while it was spent? No single method answers it well, so the system runs three estimators of different strength and makes them check each other. A marketing mix model (MMM) fits weekly or daily aggregate sales against spend per channel, with carry-over and diminishing returns, and covers every channel including offline. Incrementality experiments (geo-lift tests, holdouts) create a counterfactual on purpose and give a causal number for one channel at one time. Attribution works from individual touchpoint paths and credits conversions to touches; it is fine-grained and fast, but only correlational. A budget optimiser reads the fitted response curves and proposes a spend split. The one idea that makes it work: experiments are rare and expensive, the model is cheap and always on, so experiment results are fed into the model as priors, and the model is trusted only where it agrees with them. Google Meridian, PyMC-Marketing and Robyn are the open MMM frameworks; CausalImpact and GeoLift cover the experiment side.

Parts:

• Event data and identity — touchpoint paths and a person key to join them — Sessionization (no page yet), Identity resolution (no page yet), Private set intersection (no page yet)
• Mix model — channel response with carry-over and saturation — adstock-saturation-transforms (no page yet), Generalized linear models, Time-series forecasting
• Bayesian inference engine — posterior over model parameters — Markov chain Monte Carlo (no page yet), Hamiltonian Monte Carlo and NUTS (no page yet), Variational inference (no page yet)
• Incrementality tests — causal lift per channel from planned experiments — Hypothesis testing, CUPED, Sequential hypothesis testing (no page yet), difference-in-differences (no page yet), synthetic-control (no page yet), Bootstrap confidence intervals (no page yet)
• Attribution — credit for conversions along a path — touchpoint-attribution (no page yet), Shapley values and TreeSHAP (no page yet)
• Budget optimiser — best spend split under constraints — Quadratic programming (no page yet)

 ad platforms, CRM, web/app events ──► sessionize ──► identity join ──► touchpoint paths ──► attribution
        │ spend, impressions, price,                  (clean room: PSI)                         │ (fast, correlational)
        │ promotions, weather, macro                                                            ▼
        └──► weekly/daily panel ──► MMM (adstock · saturation · trend · season) ──► posterior ──► response curves
                                        ▲ priors from lift ▲                                       │
 geo/holdout experiments ──► DiD / synthetic control ──► lift ± interval ───────────────────────────┤
                                                                                                    ▼
                                                                 budget optimiser ──► spend plan ──► next flight

How the parts fit together

Event data and identity

Attribution and user-level experiments need paths: for each person, the ordered ad touches and the conversion. Raw events (impressions, clicks, visits) become paths through Sessionization (no page yet), which cuts an event stream into visits by an inactivity gap, and through Identity resolution (no page yet), which links the cookies, device ids and logins of one person. The two sides of a path usually sit in different companies: the publisher has the ad exposure, the advertiser has the purchase. Private set intersection (no page yet) lets them compute the overlap and aggregates over it without handing each other their customer lists. Contract with everything downstream: a path table keyed by a person id, plus daily aggregates by channel and region. The mix model never needs the paths; it needs only the aggregates, which is why it survives cookie loss and privacy rules (Meridian's documentation says as much: MMM uses aggregated data and no user-level information).

Mix model

The model explains a KPI (sales, sign-ups) per period, and per geography where available, as a baseline plus channel contributions. Two transforms turn raw spend into effect. Adstock (adstock-saturation-transforms, no page yet) spreads an impression's effect over following periods, usually with a geometric decay, because an ad seen today still works next week. Saturation bends the response so that the tenth million buys less than the first, commonly a Hill-type or logistic curve. Robyn describes exploring adstock rates and saturation curves; PyMC-Marketing lets the user choose among adstock and saturation functions.

The baseline carries what is not marketing: trend, seasonality, holidays, price, promotions. This is the territory of Time-series forecasting, and it matters because a model that misses a seasonal peak credits it to whichever channel was turned up at the time. The regression machinery underneath is a Generalized linear model with a link suited to the KPI (log for revenue-like, positive outcomes), with the transforms as non-linear terms. Why a regression and not a flexible learner: the object of interest is the channel coefficient, with an interval, not the best forecast, and a handful of channels over a few hundred periods cannot support a flexible model. The same collinearity that plagues any regression is severe here, because teams raise all channels together at peak season; the data alone often cannot separate them, which is the case for priors and for experiments.

Bayesian inference engine

With many parameters (a few per channel, plus baseline terms, plus geography effects) and weak data, point estimates are fragile; the Bayesian answer is a posterior, the full range of parameter values the data and the priors support. Sampling it is the job of Markov chain Monte Carlo (no page yet), in practice the No-U-Turn variant of Hamiltonian Monte Carlo (no page yet). Meridian's README states that its model uses NUTS and recommends GPUs because sampling is compute intensive; PyMC-Marketing offers several NUTS implementations (BlackJax, NumPyro, Nutpie). Variational inference (no page yet) is the cheaper alternative: fit a simple distribution to the posterior by optimisation, in seconds instead of minutes, at the price of underestimated uncertainty. Use it for exploration and model search, and NUTS for the numbers that go to the CFO.

Contract with the optimiser: a set of posterior draws, so every downstream number (ROI, marginal ROI, optimum) comes with an interval by simply computing it on each draw.

Incrementality tests

A planned experiment is the only direct causal evidence. Two shapes dominate. A user-level holdout randomises people into exposed and withheld; the analysis is a hypothesis test on a conversion difference, sharpened with CUPED (use pre-period behaviour to remove noise) and, if the team peeks at results while the test runs, with Sequential hypothesis testing (no page yet), because repeated looks inflate false positives. A geo experiment withholds or adds spend in some regions when user-level randomisation is impossible (TV, outdoor, or platforms that cannot hold users out). Google's geo-experiment paper (Vaver and Koehler) describes the design; analysis compares treated regions with control regions over the same weeks with difference-in-differences (no page yet), or builds a weighted blend of control regions that tracks the treated region before the test, synthetic-control (no page yet), and reads the post-period gap as lift. CausalImpact does this with a Bayesian structural time-series model (Brodersen et al.); GeoLift builds on synthetic control methods and includes power calculators for choosing which markets to test. Bootstrap confidence intervals (no page yet) give an interval without distributional assumptions when the lift is a ratio or a custom metric.

Contract with the model: for channel c over a window, a lift estimate and an interval, expressed as incremental outcome per unit of spend. The model takes it as a prior on that channel's ROI; PyMC-Marketing documents lift-test calibration and Meridian lists calibration with experiments as a feature.

Attribution

touchpoint-attribution (no page yet) assigns each conversion's credit across the touches on its path. Rule-based variants (last touch, first touch, linear, time decay) are cheap and biased in known ways. Data-driven variants model the path: Shao and Li's multi-touch attribution models fit conversion probability as a function of the touches, and removal-effect models ask how conversion falls when a channel is deleted from the paths. Credit as a fair split is a cooperative-game question, and Shapley values (no page yet) are the standard answer: a channel's credit is its average marginal contribution over all orders of the other channels, computed exactly for a handful of channels and by sampling or tree-specific algorithms for more. The weakness is built in: paths show who was reached, not who would have bought anyway, so attribution over-credits channels that target likely buyers (search brand terms, retargeting). Use it for fast, within-channel optimisation, such as which creative or keyword, and use the MMM and the experiments to set the level of each channel.

Budget optimiser

Given response curves with posteriors, the optimiser finds the spend split that maximises expected outcome under a total budget and per-channel minimum and maximum. With saturating curves the objective is concave, so a local optimum is the global one. Quadratic programming (no page yet) is the solver family when the response is approximated quadratically around the current spend, or when a variance penalty on the posterior makes the plan risk-averse; general constrained solvers handle the full curves. Robyn describes gradient-based optimisation for budget allocation and PyMC-Marketing ships a budget optimisation module. Contract: the plan is only valid inside the range of spend the model has seen; the optimiser must bound each channel near history, since a saturation curve extrapolated to ten times past spend is a guess.

Data and control flow

One measurement cycle (quarterly, with monthly refreshes):

1. Pull spend, impressions, prices, promotions and outcomes by channel, region and week; join external drivers such as weather or holidays. Check the panel for gaps and for channels that always move together.
2. Fit the MMM with weakly informative priors; inspect fit on held-out weeks and the posterior of each channel's ROI. Use variational inference first, NUTS for the final fit.
3. Compare channels where an experiment exists. A model ROI far outside the experiment's interval means the model is wrong or the world changed; fix the model, do not discard the experiment.
4. Choose the next experiments where the posterior is widest and the spend is largest. Run them as geo or holdout tests with a power calculation first.
5. Feed new lift results back as priors, refit, then run the optimiser for the next flight's plan with each channel bounded near its history.

One conversion path (daily, attribution):

1. Join exposure logs with conversions through the person key (clean room if the data sit with two parties).
2. Cut the paths with a session gap; drop paths outside the lookback window.
3. Assign credit by the chosen rule or model, with Shapley-style averaging where a model is used.
4. Report credit by channel and campaign; compare its channel totals with the MMM to see where they disagree.

Scaling

In the order the limits bite:

1. Identifiability limits an MMM before compute does: few periods, many correlated channels. Levers: add geographies (a geo-level hierarchical model multiplies the data), add experiments as priors, vary spend on purpose, drop channels that are below noise.
2. Experiment cost and power: a geo test with few markets or small effects cannot detect a plausible lift. Levers: power calculations before launch, CUPED-style pre-period adjustment, longer tests, pooled tests over several channels.
3. Sampling time: NUTS on a large geo-level model is hours. Levers: GPU, variational inference for iteration, fewer parameters, refresh the model monthly instead of weekly.
4. Path data volume: billions of events a day. Levers: sessionize on a stream, keep only paths with a conversion and sampled non-converting paths, compute credit on aggregates.
5. Privacy: user-level joins shrink. Lever: lean on the aggregate methods and run joins in a clean room.

Failure modes

Real implementations

The alternative: attribution only. It is cheap, needs no model fitting and reports within hours, but it measures exposure, not causation, and degrades as user-level data disappear. Pick it for tactical decisions inside one channel; pick MMM plus experiments when the question is how much to spend on each channel.

Sources

• Y. Jin, Y. Wang, Y. Sun, D. Chan, J. Koehler, Bayesian Methods for Media Mix Modeling with Carryover and Shape Effects, Google Research, 2017. paper
• J. Vaver, J. Koehler, Measuring Ad Effectiveness Using Geo Experiments, Google Research, 2011. paper
• K. Brodersen et al., Inferring causal impact using Bayesian structural time-series models, Annals of Applied Statistics, 2015. DOI 10.1214/14-AOAS788
• A. Abadie, A. Diamond, J. Hainmueller, Synthetic Control Methods for Comparative Case Studies, Journal of the American Statistical Association, 2010.
• X. Shao, L. Li, Data-driven multi-touch attribution models, KDD 2011.
• M. Hoffman, A. Gelman, The No-U-Turn Sampler, JMLR 2014. arXiv:1111.4246
• S. Lundberg, S.-I. Lee, A Unified Approach to Interpreting Model Predictions, NeurIPS 2017. arXiv:1705.07874
• Meridian documentation and README; PyMC-Marketing site; Robyn and GeoLift READMEs (repos above).