Why an overall sentiment score misleads

The appeal and the problem

A sentiment score is attractive: one number per store, per month, easy to chart. But a single score averages together everything customers felt about everything they mentioned. The result moves for reasons nobody can see.

What a score hides

• Polarisation: half delighted and half furious averages to neutral.
• Mixed reviews: praise for staff and anger about billing cancel out.
• Topic changes: one topic getting worse while another improves leaves the score flat.
• Volume: the same score from 20 reviews and from 2,000 looks identical.

Why it is hard to read

Is 0.21 good? Is a fall to 0.18 important? Scores depend on the model and scale used, so they cannot be compared across tools, and their changes are hard to explain to a store manager. Compare that with "billing complaints rose from 9% to 15% of mentions", which anyone can understand and check.

What to report instead

• Negative share by topic: the fraction of each topic's mentions that are negative.
• Positive share for strengths.
• The number of mentions behind each share.
• Change since the last period.

An example

A store's overall sentiment score holds steady for a quarter. By topic, staff sentiment improved after training while billing complaints doubled after a system change. The steady score told leadership nothing had changed; the topic view showed one success and one problem.

When a headline number helps

If leadership wants one number, use a familiar outcome measure such as the recent-window rating or CSAT, and put the topic breakdown directly beneath it. Do not invent a composite sentiment index that nobody can explain.

Common mistakes

• Tracking one score per store.
• Comparing scores from different tools.
• Reporting scores without counts.
• Building composite indices nobody can explain.

Common questions

Checklist

• No single sentiment score as a KPI.
• Topic negative share with counts.
• Change over time.
• A familiar headline measure if needed.

In short

A single sentiment score hides polarisation, mixed reviews and topic changes, and is hard to interpret. Report negative share by topic with counts and trend instead.

Moving away from a score

If your organisation already reports a sentiment score, run both for a quarter: the score and a small table of topic negative shares with counts. Use the table in discussions. Most teams stop looking at the score once they have seen which topics move it.

A second example

Two stores both score 0.3. At one, 65% of mentions are positive and 35% negative. At the other, 50% are positive, 20% negative and 30% neutral. The first has a real complaint problem hidden by strong praise; the second has fewer complaints but less enthusiasm. The same score describes two different stores.

Testing your own score

Take last quarter's score for each store and compare it with the store's topic negative shares. Look for stores with similar scores but very different topic patterns. If you find them, and most networks do, the score is hiding differences that matter.

Doing this with Falcon

Falcon measures sentiment for every theme a review mentions rather than one score per review, keeps the exact words that support each label, and reports negative share by topic, store and month with small samples flagged and unclear texts kept visible. Every number opens the reviews behind it.