How to normalize review data across busy and quiet stores

Why raw counts mislead

To normalize review data means making numbers comparable across stores that receive very different volumes. A store with 800 reviews a month will have more complaints than one with 40, even if it is better run. Counts measure size, not quality.

Rates that work

• Prevalence: reviews complaining about a topic divided by all reviews at the store. "What share of customers who reviewed complained about billing?"
• Negative share: negative mentions of a topic divided by all mentions of it. "When billing comes up, how often is it a complaint?"

Both are useful. Report which one each number is.

Windows

Quiet stores need longer windows to accumulate enough reviews. Use the same window for stores you compare directly; if quiet stores need a quarter while busy ones use a month, compare them only with each other.

Sample flags

Even after normalising, small samples are noisy. Flag any store below a minimum review count in the window, and show its numbers separately rather than ranking it.

What about footfall?

Dividing complaints by footfall or transactions can be informative, but only if those figures are reliable and comparable across stores. Review rates also depend on how hard each store asks for reviews, which varies. Use review-based rates as the primary measure and footfall as context.

A worked example

• Store A: 900 reviews, 90 billing complaints. Prevalence 10%.
• Store B: 60 reviews, 12 billing complaints. Prevalence 20%, but on a small sample.
• Store C: 400 reviews, 72 billing complaints. Prevalence 18%, well supported.

By counts, Store A looks worst. Normalised, Store C has a well-supported problem, Store B needs watching, and Store A is fine.

Common mistakes

• Comparing counts.
• Mixing prevalence and negative share without labels.
• Comparing stores on different windows.
• Normalising by unreliable footfall.

Common questions

Checklist

• Rates, not counts.
• Measure labelled.
• Same window for direct comparisons.
• Small samples flagged.

In short

Compare stores on rates, prevalence or negative share, over the same window, with small samples flagged; treat footfall as context, not the base.

Reporting normalised data

Show every rate with its count, label whether it is prevalence or negative share, and group stores by format so readers compare like with like. A footnote explaining the measure saves many questions in review meetings.

Network averages

For network-wide figures, weight by review volume: add up all complaints and all reviews across stores before dividing, rather than averaging the store rates. Averaging rates gives a small store the same weight as a large one, which distorts the network picture.

Doing this with Falcon

Falcon shows negative share by topic, location and month on one heatmap, flags locations with too few reviews to compare, and lets you compare any period with the one before or the same weeks last year, so a location's problem can be told apart from a network-wide one. Every number opens the reviews behind it, in customers' own words, so a comparison can always be checked.