Side by side
| Aspect | By hand | With AI |
|---|---|---|
| Accuracy | The reference, when done carefully | Can come close on well-defined categories |
| Consistency | People drift and disagree | Same rules each time, if model and prompt stay unchanged |
| Speed | A few hundred comments a day | Thousands in minutes |
| Languages | Those the tagger reads | Many, with multilingual models |
| Cost | Grows with volume | Grows slowly with volume |
| Understanding | Teaches you what customers say | Teaches nobody unless people read the output |
| Best for | Building categories, low volume, accuracy checks | Thousands of comments across locations and languages |
- Accuracy: careful manual tagging is the reference; constrained AI can come close on well-defined categories.
- Consistency: people drift and disagree; AI applies the same rules every time, if the model and prompt do not change.
- Speed: a person tags a few hundred comments a day; AI tags thousands in minutes.
- Languages: people tag the languages they read; multilingual models cover many.
- Cost: manual cost grows with volume; AI cost grows slowly.
- Understanding: hand-tagging teaches you what customers say; AI does not teach anyone unless people read its output.
When hand-tagging is right
- At the start, to build and test categories.
- At low volume, a few hundred comments a month.
- For the monthly accuracy sample used to check AI.
- For sensitive cases that need judgement.
When AI is right
- Thousands of comments a month.
- Many locations and languages.
- When you need history reclassified after a category change.
- When consistency over time matters more than nuance in each comment.
Combining them
- People write the categories, definitions and examples.
- AI classifies every comment against them and quotes the supporting words.
- People check a random sample monthly and fix definitions where AI and people disagree.
- People read the feedback behind every important finding.
How to use AI to analyse customer reviews you can trust.
How to use ai to analyse customer reviewsFree review snapshot
Want this done for your own reviews?
No commitment. We use your details only to send your snapshot and follow up. Privacy policy
An example
A team tagged 300 reviews a month by hand for a year and knew its categories well. When the brand expanded to 80 stores and four languages, it moved to AI tagging with the same categories, kept one person checking 100 AI-tagged reviews a month, and reclassified the previous year so the trends continued unbroken.
Common mistakes
- Switching to AI without defined categories.
- Never checking AI output against people.
- Letting the AI invent categories.
- Assuming hand-tagging is consistent because people did it.
Common questions
How accurate is AI tagging?
It depends on the categories and setup; measure it on your own sample.
Will AI replace analysts?
It replaces tagging; analysts still interpret, check and explain.
Is hand-tagging biased?
It can be; two people tagging the same sample reveals how much.
Can we mix both in one dataset?
Yes, if both use the same categories and you record which method tagged each item.
Checklist
- Categories defined by people.
- AI constrained to them.
- Monthly human check.
- Model and prompt versions recorded.
In short
Tag by hand to learn your categories and to check accuracy; let AI apply them at scale. People define, AI applies, people check and interpret.
Thinking about cost and time
Hand-tagging costs people's time, roughly in proportion to volume: double the reviews, double the hours. AI costs mostly setup and checking time, plus model usage that grows slowly with volume. The crossover point depends on your volume and languages, but most multi-store brands pass it as soon as they want monthly, consistent results across all locations.
Count the hidden cost of hand-tagging too: results arrive weeks late, and history is rarely reclassified when categories change, so trends break.
Keeping AI honest
Record which model, prompt and category version produced each label, keep the supporting quote for every label, and compare a random sample against a person each month. If accuracy drops after a model update, you will see it in the sample before it distorts a quarter's reporting.
Doing this with Falcon
Falcon builds the theme list from your own reviews, then labels every review against it with a multilingual model, scoring sentiment for each mention and keeping the exact quote behind each label. Texts too short to judge are marked unclear rather than guessed, and low-confidence labels are flagged for a person to check. Negative share by topic, location and month then shows what is rising where, with small samples flagged.