Incrementality Testing (ASO)
Incrementality testing isolates what a specific ASO change or ad campaign actually caused, as distinct from installs that would have happened anyway, by comparing against a holdout or a paused period.
Raw install numbers after a metadata change or an ad campaign don't prove the change caused them — some of those installs might have happened regardless. Incrementality testing addresses this by creating a comparison: pausing a campaign for a period and watching whether installs drop, or holding one storefront's metadata constant as a control while updating others.
This matters most for expensive or uncertain decisions: before committing to ongoing paid spend on a keyword, a short pause-and-compare test can show how much of the current install rate is actually attributable to the ads versus organic demand that existed anyway.
For organic metadata changes, a cleaner but slower version of the same logic applies: compare a keyword's install rate for a period before and after the change, watching for a shift that outpaces normal week-to-week variance rather than assuming any change was the update's doing.
Frequently asked questions
- Do I need special tooling to run an incrementality test?
- Not necessarily. A basic version just requires comparing performance before and after a change, or pausing one variable (like ad spend on a keyword) while holding others constant, using the analytics you already have access to.
Related terms
- Organic vs. Paid InstallsOrganic installs come from users who found an app through unpaid search or browse; paid installs come from Apple Search Ads or other ad campaigns — the distinction matters for measuring ASO's real impact separately from ad spend.
- Halo Effect (Paid-to-Organic Spillover)The halo effect describes organic App Store installs that increase as a side effect of running paid Apple Search Ads, driven by paid-install velocity lifting an app's ranking and visibility for the same keyword.
- App Store Connect AnalyticsApp Store Connect Analytics is Apple's own native reporting on impressions, product page views, conversion, and retention for your app, distinct from and complementary to third-party ASO tools.
Browse the glossary
Put this into practice
Score App Store keywords by popularity and difficulty across 109 storefronts — the numbers behind incrementality testing (aso).
Start keyword research