Why rankings are popular

A store rating ranking is easy to make and easy to read: sort by Google rating, show the top ten and bottom ten. It feels objective. It is also one of the most misleading charts in retail reporting.

Four problems

  • Sample size: a store with 30 reviews can sit at 4.9 or 4.1 by chance; a store with 3,000 cannot.
  • History: all-time averages include years-old reviews; a changed store keeps its old rating.
  • Format: a highway outlet and a mall flagship serve different customers.
  • No reason: the ranking says a store is low, not what is wrong.

How chance creates top and bottom stores

Among many small stores, some will have unusually good or bad months by chance alone. Rankings put exactly those stores at the extremes. Next quarter, many of them move back towards the middle, and the ranking appears to show improvement or decline that never happened.

What to use instead

  • Peer groups of similar stores.
  • A recent window, such as 90 days.
  • Issue rates by topic, compared with the peer group's typical range.
  • Review counts beside every figure; small samples flagged.
  • Trend against the previous period.
Fair comparisons

How to compare store performance fairly.

How to compare store performance fairly

Free review snapshot

Want this done for your own reviews?

No commitment. We use your details only to send your snapshot and follow up. Privacy policy

An example

A chain's quarterly "top ten" included four stores with fewer than 40 reviews each; three were in the bottom ten the previous quarter. Replacing the ranking with peer-group issue rates showed a different picture: two large stores with rising billing complaints that the ranking had placed comfortably mid-table.

If leadership wants a list

Give a list of stores needing attention, each with the issue, the evidence and the trend, rather than a ranking. It answers the question leadership is really asking: where should we act?

Triage

Which stores need attention: a triage method.

Which stores need attention a triage method

Common mistakes

  • Ranking all-time ratings.
  • Rewarding stores for rankings driven by chance.
  • Publishing rankings to store teams without context.
  • Ignoring the reasons behind positions.

Common questions

Are rankings ever useful?

Within a peer group, with enough reviews and a recent window, as a starting point.

Should we stop showing ratings?

No; show them with counts and recent windows, alongside topic measures.

Why did our best store fall to the bottom?

With few reviews, chance alone can move a store a long way.

What do store managers prefer?

Comparisons with similar stores and the reviews behind the numbers.

Checklist

  • Peer groups used.
  • Recent window.
  • Counts beside figures.
  • Issue rates, not just ratings.
  • Trends shown.

In short

Rating rankings mix sample sizes, histories and formats and give no reasons. Compare stores within peer groups on recent issue rates with counts and trends, and list stores needing attention instead.

Explaining the change to leadership

Show last quarter's top ten and bottom ten alongside each store's review count and the previous quarter's position. The movement of small stores between top and bottom usually makes the point on its own.

Doing this with Falcon

Falcon shows negative share by topic, location and month on one heatmap, flags locations with too few reviews to compare, and lets you compare any period with the one before or the same weeks last year, so a location's problem can be told apart from a network-wide one. Every number opens the reviews behind it, in customers' own words, so a comparison can always be checked.