Google Play Experiments: Test for the Best

August 8th, 2019

by David Quinn

VP of Strategy & Partnerships at Gummicube, Inc.

A/B testing is an essential part of an App Store Optimization strategy. It’s crucial to test all aspects of your app’s metadata with different variants of its creatives and description to understand what users react to. Google Play offers Store Listing Experiments, which enable developers to test variants of their app’s page and determine what performs best.

Play Store Listing Experiments

Google Play Experiments allow developers to run multiple versions of an app’s listing at once, with each variant appearing before a certain percentage of live users. Published apps can run variants to test against their current version before deciding which variant to move forward with or if more tests are needed.

Google Play offers two experiment types:

Default Graphics Experiments - Tests creatives such as icon, screenshots and promo video for the default language
Localized Experiments - Tests creatives and descriptions in up to five languages. Each variant is shown to users in the selected languages only.

The experiments provide developers with key information and insights into how the variants performed. This includes the number of variants being served to a percentage of users, how long the experiment ran for, the type of experiment and the overall results.

The display provides clear information, so developers can see how well each variant performed against the other. For instance, if there are two variants, it will show how many installs each variant gained and how well they performed compared to the existing version with a “90% confidence interval.”

Play Store Experiment Best Practices

When running tests, it’s important to test one attribute at a time. If more than one thing changes in a single variant, it will be difficult to determine which change users are reacting well to. For instance, you can test a different set of screenshots against each other, but if one set also includes a different description, any changes to conversions could be due to either one of those.

Google also recommends testing all listing attributes, starting with the app icon then moving on to the screenshots, short description, then long description. The experiments should run for at least 7 days, in order to capture weekday and weekend behavior with a meaningful amount of traffic.

The experiments should use the largest possible test audience; preferably a 50% split between two different versions, or a 25% split if four variants are being tested. A large audience portion means you can reach a greater number of users for results that more accurately reflect the overall userbase, while an even split means it will be easier to see how each version is being received by those who see it.

Just as ASO requires constant reiterations based on learnings and successes, so too should experiments be repeated and tested again with variations built on the successes of the last. Take what you’ve learned from each experiment and apply it to determine the best performing variant possible.

Get Started Google Experiments Testing

Google Play Store Experiments are helpful for A/B testing variations of an app’s user-facing elements. Developers can try different variants of their icons, screenshots, short descriptions and long descriptions to see what users respond best to. This can be used to determine the most optimal variants to use for App Store Optimization, helping to ensure an app is fully optimized and performing well.

Need help with your mobile A/B tests? Get in touch with Gummicube today!