A single marketing mix model can produce a clean chart, a tidy R², and a confident reallocation recommendation. It can also be wrong by seven figures before anyone notices. The problem isn't the math; it's that every MMM encodes assumptions about how marketing works, and those assumptions shape the output as much as the data does.
Adstock windows determine how long a channel's effect lingers. Saturation curves dictate how quickly returns diminish. Priors and regularization encode beliefs about plausible effect sizes. Change any of these, and the same spend history produces a different story. As Ben Vigneron recently argued in Search Engine Land, a single model is a first opinion, not a verdict.
The CFO-safe move is to run multiple models against the same inputs before making a budget decision. Not because more models are better, but because disagreement between models tells you where your uncertainty actually lives.
Where the Assumptions Hide
Most MMM conversations focus on outputs: channel contributions, response curves, optimal allocations. Fewer focus on the inputs that determine those outputs before a single regression runs.
Three assumptions do the heaviest lifting. First, decay windows. A model that assumes paid social's effect fades in three days will attribute less lift to that channel than one assuming a two-week window. Neither is objectively correct; both are defensible depending on your sales cycle and consideration period.
Second, saturation curves. These determine how quickly a channel hits diminishing returns. A model with aggressive saturation will recommend pulling budget earlier; a conservative one will keep spending longer.
Third, priors. Bayesian models let you encode beliefs about plausible effect sizes. Strong priors stabilize estimates when data is sparse, but they also constrain what the model can learn.
The danger isn't that any single assumption is wrong. It's that you don't know which assumptions are driving your reallocation until you see what happens when you change them.
The Multi-Model Protocol
The practical solution is to run the same data through at least two, ideally three, modeling frameworks with different structural assumptions. Vigneron's approach uses three open-source tools: Meta's Robyn (ridge regression with evolutionary hyperparameter search), Google's Meridian (Bayesian with hierarchical priors), and PyMC-Marketing (flexible Bayesian with customizable priors). Each makes different default assumptions about decay, saturation, and regularization.
The goal isn't to average the outputs or pick the model with the best fit statistics. It's to identify where the models agree and where they diverge.
Agreement is a green light. If all three models say paid search is saturated and brand spend is underweight, you have convergent evidence. The recommendation is robust to modeling choices.
Divergence is a yellow light. If Robyn says display is your most efficient channel while Meridian says it's your least, you've found a decision that depends heavily on assumptions you can't validate from historical data alone. That's not a failure; that's information. It tells you where to run an incrementality test before committing budget.
What Divergence Actually Tells You
When models disagree, the instinct is to ask which one is right. That's the wrong question. The right question is: what would have to be true for each model to be correct?
Say Model A attributes 18% of conversions to YouTube while Model B attributes 6%. The difference might trace to decay assumptions (Model A uses a longer window), saturation curves (Model B assumes faster diminishing returns), or how each handles the correlation between YouTube spend and seasonal demand spikes.
Map the disagreement back to the assumption. Then ask whether you have any external evidence to adjudicate. Did you run a geo-holdout on YouTube last year? Do you have brand lift studies that suggest a longer consideration window? If yes, weight the model whose assumptions align with that evidence. If no, you've identified a testable hypothesis.
This is the real value of multi-model comparison: it converts vague uncertainty into specific questions you can answer with experiments.
Connecting Models to Experiments
MMM and incrementality testing aren't competing methodologies; they're complementary. As Vigneron notes, models generate and rank hypotheses, experiments confirm which ones matter, and confirmed results feed back into the models as priors.
The practical workflow looks like this. Run your multi-model comparison quarterly. Identify the two or three channels where models diverge most. Prioritize incrementality tests (geo-lift, holdout, or on/off) on those channels. Use test results to update priors in your Bayesian models or to constrain hyperparameter ranges in your frequentist ones.
You can't test every channel every quarter. Budget and operational constraints make that impossible. But you can test the channels where your models disagree most, which are precisely the channels where a wrong decision costs the most.
The Operational Checklist
For teams implementing this approach, the sequence matters.
Start with data alignment. All three models must ingest identical spend, impression, and outcome data at the same granularity. The Trade Desk's MMM best practices emphasize breaking out programmatic by channel, format, and buying model rather than treating it as a single line item. Inconsistent granularity across models makes comparison meaningless.
Standardize your output format. Each model will produce channel contributions, response curves, and optimal allocations in slightly different structures. Build a comparison template that normalizes these into a single view: channel, contribution share, marginal efficiency, and recommended reallocation.
Document the assumptions. For each model, record the decay window, saturation function, and prior distributions used. When models disagree, this documentation lets you trace the divergence to its source.
Set a decision rule. Before running the comparison, define what level of agreement you require to act without further testing. A reasonable threshold: if all models agree on direction (increase, decrease, or hold) and magnitude is within 20%, proceed. If not, flag for incrementality testing.
The Board-Ready Framing
When you present a reallocation recommendation, the CFO will ask how confident you are. A single model gives you a point estimate with no honest way to quantify uncertainty. A multi-model comparison gives you a range and a map of where that range comes from.
"All three models recommend shifting $400K from display to connected TV. The estimated lift ranges from 8% to 14% depending on saturation assumptions. We're running a geo-holdout in Q4 to narrow that range before committing the full reallocation."
That's a recommendation Finance can underwrite. It shows the math, acknowledges the uncertainty, and specifies how you'll reduce it. Model or it didn't happen, but multiple models show you where the model might be wrong.
