The marketing platform says 5x. Your backend says 2x. You split the difference, call it 3x, and move on with your pipeline review. That compromise feels reasonable. It's also mathematically incoherent.
Benjamin Wenner's analysis in Search Engine Land puts the problem plainly: the platform claims the sale, your backend gives credit elsewhere, and neither can tell you if the ad caused anything. Both systems are answering different questions than the one you actually need answered. The platform asks "what touched the customer before conversion?" and counts generously. The backend asks "what was the last thing the customer touched?" and counts stingily. Neither asks "what would have happened if we hadn't run the ad at all?"
That last question is the only one your CFO cares about. It's the only one that determines whether your spend is creating value or subsidizing conversions that would have happened anyway.
Two Thermometers, Zero Fever
Wenner uses a thermometer analogy that's worth stealing for your next board prep: one thermometer runs five degrees hot, one runs five degrees cold. Averaging them doesn't give you the patient's actual temperature. It gives you a number that describes nobody.
The platform's 5x is inflated by design. View-through conversions, modeled conversions where consent was never granted, and conversion windows that credit clicks from three weeks ago all flow into that number. Marketing platforms grade their own homework and set your spend on the result.
The backend's 2x is deflated by design. Most backend revenue reporting is last-click or something close to it. The order gets attributed to whatever the customer touched last, often a brand search or a direct visit. The paid click that started the journey three weeks earlier gets nothing. Your CRM isn't lying to you. It's answering a narrower question than you think you asked.
Same ambiguity, opposite defaults. The gap between 5x and 2x is mostly the size of that disagreement, not evidence of fraud or incompetence.
The Incrementality Gap Nobody Wants to Model
Here's where the math gets uncomfortable for most marketing orgs. If your backend systematically strips credit from the touchpoint that opened the journey, then whatever did the opening gets underpaid. The error runs in one consistent direction: away from the top of the funnel.
But the error doesn't fall evenly. It falls hardest on the channels that can't be clicked at all. Display impressions, video views, podcast mentions, OOH: the backend can't see ads nobody clicks. These channels might be doing real work, or they might be doing nothing. Your current measurement stack can't tell you which.
Evan Carroll's LinkedIn post captures the downstream problem:
You can hit 5x ROAS and still lose money. You can hit 2x ROAS and be wildly profitable.
Evan Carroll
ROAS only shows revenue divided by ad spend. It doesn't show whether that revenue covers your OpEx, which sales actually came from your ads, or your actual cash position.
The metric is a ratio. Ratios are easy to game and easier to misread. A 4x ROAS means nothing if you're losing money on every order because your contribution margin can't support the acquisition cost.
What Incrementality Actually Measures
Incrementality testing asks a different question: what revenue would we have lost if we hadn't run this campaign? The answer requires a holdout, a control group that doesn't see the ad, and a comparison of conversion rates between exposed and unexposed populations.
The math is straightforward. If your exposed group converts at 4% and your holdout converts at 3.2%, your incremental lift is 0.8 percentage points. That 0.8 points, multiplied by your audience size and average order value, gives you the revenue your ad actually caused. Everything else was going to happen anyway.
This is harder to run than it sounds. You need statistical power, which means sample size. You need clean holdouts, which means geographic or audience-based exclusions that don't contaminate. You need patience, which means waiting for enough conversions to reach significance. Most orgs don't have the discipline for any of these.
But the alternative is worse. Without incrementality, you're optimizing toward a number that's either inflated (platform) or deflated (backend), and you have no way to know which direction you're wrong.
The CFO Conversation You're Avoiding
Your CFO doesn't care about ROAS. Your CFO cares about CAC payback, gross margin, and whether marketing spend is generating returns above the cost of capital. ROAS is a proxy for those things, but it's a proxy that breaks down the moment you try to compare channels or allocate budget.
Here's the conversation you should be having instead:
First, what's our blended CAC payback period across all channels? Not platform-reported, not backend-reported, but based on cohort analysis of actual customer behavior over time.
Second, what's our incremental CAC for the marginal dollar of spend in each channel? This requires incrementality testing, not attribution modeling.
Third, what's the sensitivity of our forecast to a 20% reduction in paid spend? If you can't answer this with a confidence interval, you're guessing.
Carroll's post suggests tracking:
- MER (all revenue divided by all marketing spend)
- CAC vs LTV
- Contribution margin in dollars
- Actual cash flow
These are the right metrics. They're also the metrics that require you to stop trusting platform-reported ROAS as a source of truth.
A Two-Week Pilot to Find Your Real Number
If you're running significant paid spend and you've never done an incrementality test, start small. Pick one channel, one geography, and one two-week window. Hold out 10-15% of your target audience from seeing ads. Compare conversion rates. Calculate lift.
The assumptions you need to document:
- Baseline conversion rate
- Expected lift
- Minimum detectable effect
- Sample size required for 80% power
If your expected lift is small and your conversion rate is low, you'll need a larger holdout or a longer window.
The risks: you'll lose some revenue during the test period. You'll also learn whether that revenue was incremental or whether you were paying for conversions that would have happened anyway. The second piece of information is worth more than the first.
The Number That Matters
Your paid media ROAS isn't 5x. It isn't 2x. It's whatever your incrementality test says it is, minus the conversions that would have happened without the ad, adjusted for the contribution margin of those conversions, and compared against your cost of capital.
That's a harder number to calculate. It's also the only number that tells you whether your spend is creating value or destroying it. Model or it didn't happen.
