How to Read an Inconclusive App Store Screenshot Test
Interpret App Store product page test confidence and decide what to do when traffic cannot support a clear result.
A screenshot treatment can show a higher observed conversion rate while Apple's test still has too little evidence to call it better. Use the test status and confidence alongside the displayed change before replacing your product page.
Check that the test has enough observations to appear
Open Product Page Optimization in App Store Connect and confirm the test and treatment dates. Apple documents a minimum of five attributed first-time downloads for a test to appear in Analytics. Meeting that minimum only makes results visible; it does not make the estimate conclusive.
Read performance and confidence together
Compare the treatment with the baseline using the displayed improvement and confidence. If the result is still collecting data or likely to be inconclusive, avoid declaring a winner based on the point estimate alone. Do not use another app's traffic or result as a sample-size target for yours.
Choose a next step before rerunning
Use three separate decisions. Keep collecting data when the test still has time, traffic is steady, and no release or campaign change is about to change the comparison. Keep the current page and record the result as inconclusive if the decision deadline arrives without enough evidence. Retest if a release or shift in acquisition disrupted the comparison, or if a simpler test could answer a clearer question.
For example, 4 downloads from 40 visits and 5 from 40 differ by one download. The raw rates are 10% and 12.5%, a difference of 2.5 percentage points. This is a simple observed difference, not Apple’s reported relative improvement or confidence estimate.
