Why version matters

When you analyze app reviews by version, you can tie what users say to what you shipped. An overall rating mixes reviews from people on last year's build with people on yesterday's. Splitting by version shows whether a release fixed a known problem, introduced a new one, or changed nothing users noticed.

Getting reviews by version

  • In Play Console, go to Ratings and reviews > Reviews and filter by app version. Google lists app version among the available filters.
  • For analysis, download the review reports from Google Cloud Storage; keep the app version column alongside date, rating and text.
  • Note that not every review is tied to a version in the same way; keep reviews with no version in a separate group.
Source

Google's help page on filtering and exporting reviews.

View and analyse your app's ratings and reviews (Play Console Help)

The method

  1. Classify each review into topics with sentiment: login, payments, checkout, search, performance, crashes, delivery, content.
  2. Group reviews by app version.
  3. For each version, calculate each topic's share of reviews that complain about it.
  4. Compare each version with the one before it.
  5. Flag topics whose complaint share rose sharply, with enough reviews behind them.
  6. Read the reviews behind every flag.
Rates, not counts

The current version always has the most reviews because most users run it. Compare the share of reviews mentioning each problem, not the number.

Account for adoption

In the first days after a release, reviews on the new version come mainly from users with automatic updates and those who update quickly. They may differ from your typical users. Wait until a version has a reasonable number of reviews before comparing, and compare the same number of days after each release where you can.

A worked example

  • Version 8.3: 1,900 reviews; payment complaints 4% of reviews; login complaints 6%.
  • Version 8.4: 700 reviews in its first three weeks; payment complaints 11%; login complaints 2%.

Login complaints fell, which matches the login fix in 8.4. Payment complaints nearly tripled. Reading the reviews shows a specific wallet failing at the final step, which was changed in 8.4. The regression goes to the payments team with the reviews attached.

Regressions

How to spot release regressions in app reviews.

Spotting release regressions in app reviews

Free review snapshot

Want this done for your own reviews?

No commitment. We use your details only to send your snapshot and follow up. Privacy policy

Pitfalls

  • Comparing raw counts across versions.
  • Judging a version on its first day of reviews.
  • Treating rating changes as version effects without checking topics.
  • Ignoring reviews in languages other than English.
  • Forgetting server-side changes, which affect all versions at once and look like a version effect on the newest one.

Common questions

What if I release weekly?

Group consecutive small releases, or compare by week instead, and check topics after each meaningful change.

Can I do this for iOS too?

The same method works for any store that records the app version with each review.

Why did complaints rise on an old version?

Users on old versions may hit server-side changes or deprecations; check what changed on the backend.

How many reviews do I need per version?

Enough that one or two reviews do not swing the rate; a few hundred is comfortable, a few dozen is a reason to read rather than calculate.

Checklist

  • Reviews exported with app version.
  • Topics classified with sentiment.
  • Complaint rates compared version to version.
  • Adoption period allowed for.
  • Reviews read behind every flagged change.

Reporting version results

Share a short table with each release review: version, reviews so far, and complaint share for the five largest product areas, with the change from the previous version and a note for anything flagged. Add three quotes for each flag. Keeping the same format every release lets the team see at a glance whether a release made things better, worse or no different.

Doing this with Falcon

Falcon classifies every Google Play review of your app into themes, sub-themes and specific issues with the sentiment of each mention, kept separate from your store reviews but on the same taxonomy. Negative share by topic and month shows which problems are new or growing, and every issue opens the reviews behind it, with the supporting words highlighted.