Customer feedback classification, explained

What classification is

Customer feedback classification is the step that turns free text into something you can count. Each comment, or each part of a comment, is assigned to one or more categories (such as Billing > Waiting time) and usually a sentiment. Everything downstream, from dashboards to root-cause analysis, depends on how well this step is done.

Four methods

• Manual tagging: a person reads and tags. Accurate when done carefully, slow, and hard to keep consistent between people.
• Keyword rules: "queue" means Waiting. Fast and transparent, but brittle with synonyms, negation and other languages.
• Trained classifiers: machine learning models trained on labelled examples. Consistent at scale, but need training data and retraining when categories change.
• Language models: read the text in context and choose from your categories. Flexible across languages and wording, but must be constrained and checked.

Review-level vs mention-level

Classifying a whole review into one category forces you to choose between its topics. Mention-level classification labels each topic a review mentions, with its own sentiment, so mixed reviews keep both their praise and their complaint.

What good classification looks like

• A fixed set of categories with definitions and examples.
• Labels at the mention level, with sentiment.
• Evidence: the words that justify each label.
• An "unclear" option for text with no usable topic.
• Stable results over time, with model and category versions recorded.

How to evaluate a method

• Have two people tag the same 200 comments independently.
• Where they agree, use that as the reference.
• Run the method on the same comments and compare.
• Look at disagreements by category: some categories are always harder.
• Repeat monthly on a fresh sample.

An example

A retailer's keyword rules counted every review mentioning "wait" as a waiting complaint, including "can't wait to come back!". Switching to a language model that chose from defined categories and quoted the supporting words fixed the false positives, and showed that true waiting complaints were half what the rules had reported.

Common mistakes

• Letting the method invent new categories every run.
• Forcing every comment into a category.
• Never checking accuracy after launch.
• Changing categories without reclassifying history.

Common questions

Checklist

• Categories defined with examples.
• Mention-level labels with sentiment.
• Evidence kept for each label.
• Unclear texts flagged.
• Monthly accuracy check.

In short

Classify feedback against fixed, defined categories at the mention level, keep the words behind each label, allow unclear, and check accuracy on your own data every month.

Classification across languages

Feedback in markets like India arrives in many languages and in mixtures of English and local languages. Keyword rules struggle badly here, because every word list must be rebuilt for every language and script. Multilingual models handle mixed text far better, but accuracy still varies by language, so check samples in each language you receive, not only in English.

Keep the original text alongside any translation, and label in the original where possible; translation can shift meaning, especially for sarcasm and idioms.

Doing this with Falcon

Falcon builds the theme list from your own reviews, then labels every review against it with a multilingual model, scoring sentiment for each mention and keeping the exact quote behind each label. Texts too short to judge are marked unclear rather than guessed, and low-confidence labels are flagged for a person to check. Negative share by topic, location and month then shows what is rising where, with small samples flagged.