Small sample size reviews: when to trust a location's numbers

Why small samples mislead

Small sample size reviews produce numbers that look precise and are not. If 3 of a store's 12 reviews this month complain about staff, that is 25%. One more complaint makes it 31% (4 of 13); one more good review makes it 23% (3 of 13). The store did not change; the sample did.

How much numbers can move

A rough guide for proportions: the uncertainty shrinks with the square root of the sample. A complaint rate around 20% on 25 reviews could plausibly be anywhere from roughly 9% to 39%; on 400 reviews, roughly 16% to 24%. Statistical intervals such as the Wilson interval give these ranges properly.

Set a minimum

• Choose a minimum number of reviews per location and period before ranking.
• Below it, show the location separately as "not enough reviews".
• Use the same minimum for every location so the rule is fair.

Show uncertainty

• Display ranges, not just points, for small samples.
• Flag rather than hide; nobody should assume a quiet store is fine.
• Sort by evidence strength as well as by rate.

Learning from quiet locations

• Use longer windows: a quarter or six months instead of a month.
• Group similar small stores together for some analyses.
• Read every review; with few, reading beats calculating.
• Treat serious issues (safety, hygiene) individually regardless of sample.

An example

A regional manager is shown that a small-town store has the worst staff complaint rate in the region: 30%. It is based on ten reviews, three of them from one weekend. Over six months, the rate is 9%, in line with peers. The flag that should have been shown was "not enough reviews", not "worst in region".

Common mistakes

• Ranking stores with very different volumes together.
• Dropping small stores silently.
• Reacting to a single bad month at a quiet store.
• Using all-time averages to compensate, which hides recent change.

Common questions

Checklist

• Minimum sample set.
• Small samples flagged, not hidden.
• Ranges shown where useful.
• Longer windows for quiet stores.
• Serious issues handled individually.

In short

Numbers built on few reviews swing widely. Set a minimum, flag small samples instead of ranking them, show ranges, use longer windows for quiet stores, and read their reviews.

Explaining it to managers

Store managers often push back when a small sample makes their store look bad, and they are right to. Showing the review count beside every number, and using a clear "not enough reviews" flag, makes the rule fair and visible. It also stops managers dismissing well-supported findings at larger stores as noise.

At network level

Small samples matter less when you add stores together. A rate across 300 stores is stable even if each store's is not. Report network and regional trends with confidence, and treat individual small stores with care; the same data supports both, at different levels.

Doing this with Falcon

Falcon builds the theme list from your own reviews, then labels every review against it with a multilingual model, scoring sentiment for each mention and keeping the exact quote behind each label. Texts too short to judge are marked unclear rather than guessed, and low-confidence labels are flagged for a person to check. Negative share by topic, location and month then shows what is rising where, with small samples flagged.