Attribution answers the question of which ad a player touched before installing. Incrementality answers a different and more important one: would that player have installed anyway? The gap between the two answers is where a great deal of user acquisition budget quietly disappears. Brand search ads collect installs from people who were already looking for the game, retargeting reaches players who were about to come back on their own, and an ad network can be credited with installs that a viral moment on social media actually caused.
Incrementality testing measures the installs, purchases and revenue that advertising actually causes, by comparing a group that saw the ads with a similar group that did not. This guide explains the main ways to do it, walks through a geo lift test step by step, and shows where it matters most for game advertisers.
Attribution versus incrementality
Attribution is bookkeeping. A mobile measurement partner receives clicks and impressions from every network and assigns each install to one of them by a set of rules, usually the last click within a time window. It is necessary for paying networks and optimising campaigns, but it cannot see the counterfactual: what would have happened without the ad. An install attributed to an ad may well have happened without it.
Incrementality is an experiment. You deliberately withhold advertising from a comparable group and measure the difference. The result is an incremental number of installs, and from it an incremental cost per install: spend divided by the installs the spend actually caused. That figure is almost always higher than the attributed cost per install, and the size of the gap is the point.
The main test designs
User-level holdouts
The platform randomly splits its audience, shows your ads to one group and holds out the other, then compares conversions. Meta and Google offer lift studies of this kind to eligible advertisers. Some networks show the control group an unrelated ad, often called a ghost ad or a public service ad, so both groups are measured in the same conditions. Holdouts are statistically clean but run inside one platform, so they measure that platform alone, and on iOS they are limited by what can be observed without tracking consent.
Geo lift tests
Instead of splitting people, you split places. Pick comparable regions or countries, keep advertising in some and stop or change it in others, then compare installs and revenue before and after. Geo tests need no user-level data, which makes them robust to privacy changes, and they work across every channel at once. Their weakness is noise: regions differ, and a holiday or a viral video in one of them can swamp the effect.
Pause tests
The simplest version: switch a channel or campaign off for a period and see what happens to total installs. It is cheap and quick, and it is often the most persuasive evidence a finance team will accept. It is also the crudest, because anything else that changes during the pause gets mistaken for the effect.
Marketing mix modelling
Marketing mix modelling (MMM) is a statistical model of total installs or revenue over time as a function of spend in each channel, plus seasonality, pricing, launches and other factors. It needs no user-level tracking at all, which is why it has returned to fashion since privacy changes on iOS. Open-source tools are available, such as Meta's Robyn and Google's Meridian. MMM works best with a long history and varied spend; it is strongest when calibrated with the results of real experiments like the ones above.
Running a geo lift test, step by step
- Write down the question. For example: what does our brand search spend add, or how many installs does our in-app network budget cause in mid-sized markets?
- Pick the metric that matters: total installs from all sources, or better, revenue or retained players at day 7, so cannibalised organic installs show up as what they are.
- Choose test and control regions that have tracked each other closely in the past. The closer their history, the smaller the effect you can detect.
- Decide the duration in advance. It must be long enough to separate the effect from day-to-day noise, typically several weeks, and should avoid big holidays or launches.
- Change one thing in the test regions: stop the channel, double it, or switch the creative. Leave the control regions untouched.
- Measure the difference against what the control regions predict the test regions would have done, then compute incremental installs and incremental cost per install.
- Repeat. One test is an estimate with a wide margin of error; a programme of tests becomes a reliable picture.
An illustration, not a benchmark: a channel spends 20,000 in the test regions and is credited with 10,000 installs, an attributed CPI of 2. The test shows total installs fell by only 5,000 when it was paused. Its incremental CPI is 4, twice what the dashboard said.
Where incrementality matters most in games
- Brand search. People searching your game's name are already interested. Brand campaigns protect against competitors, but much of what they are credited with is organic. See Apple Ads for mobile games.
- Retargeting. Lapsed players reached by an ad sometimes return because of it and sometimes would have returned anyway after an update or an event. Playable ads for retargeting covers designing for them.
- Cross-promotion. Moving a player from one of your games to another can create a new player or simply relocate an existing one.
- Channels that overlap. When several networks reach the same people, each claims the same installs, and the sum of their attributed installs exceeds what the budget really caused.
Incrementality and creative
Most creative testing happens inside a platform, comparing installs per thousand impressions between ads shown to similar audiences. That is a fair test of which creative performs better within the channel, and playable ad A/B testing covers it. Incrementality adds a second question: does a new creative bring new players, or just reach the same players more cheaply? Creatives that show the game honestly and attract a different audience tend to grow total installs; creatives that merely win the auction for people who would have installed anyway do not.
This is one reason teams test genuinely different concepts rather than small variations. Playable Ads Maker exports one project in the format each of 13 ad networks asks for, so a new concept can run on every channel in a geo test at once rather than in one network's corner.
Common mistakes
- Measuring only attributed installs during a test, so organic installs that replaced paid ones are missed.
- Ending a test early because the first days look good.
- Comparing regions that never tracked each other in the first place.
- Running several tests in the same regions at once.
- Treating one result as permanent. Incrementality changes as the game, its audience and its competition change.
Create playable end cards in minutes—no code required.
Open Playable Ads Maker