Sampling Strategies: How Much Customer Feedback Data Do You Actually Need?

Why Sample at All

Surveying everyone doesn't guarantee accuracy. If only a small slice responds, you still have a sample, just one you didn't choose. Qualtrics' 2025 Consumer Trends report found that fewer than a third of consumers give feedback directly to companies. Choosing who you ask gives you more control over who answers. Sampling also makes analysis easier. Your team can read and act on a few hundred well-chosen responses much faster than tens of thousands of scattered ones.

Over-surveying has a cost

Shep Hyken's 2024 research found that 23% of customers stopped doing business with a company because it sent too many surveys. Sampling protects those relationships.

How Big Should Your Sample Be?

Sample size depends on how precise you need to be. Andrew Mercer, a principal methodologist at Pew Research Center, explains that “the margin of error for a survey is driven primarily by the sample size.” He notes that a simple random sample of 1,067 cases gives a margin of error of about plus or minus 3 percentage points. Many CX questions don't need that much precision. Around 400 random responses gives roughly plus or minus 5 points, which is often enough to spot a real trend.

Plan for segments

Most CX decisions happen at the segment level. Mercer also points out that subgroups have fewer cases, so “their margins of error are larger.” If you need reliable scores for each region, plan a big enough sample for each region. For very small segments, you may need to ask everyone in the group. That's fine as long as contact limits stay in place.

Pick the Right Sampling Method

• Random sampling gives every customer an equal chance to be asked.
• Stratified sampling sets quotas for each segment so small groups aren't drowned out.
• Event-based sampling asks a share of customers after a specific interaction.
• Targeted sampling focuses on accounts you need to hear from, such as upcoming renewals.

Avoid convenience samples

Surveying whoever is easiest to reach, like your most active users, skews results. Pull a fresh random list for each wave so the same customers aren't asked every time. Our post on response bias explains why.

Let Unstructured Data Fill the Gaps

Support tickets and reviews come from customers you didn't sample. They come with their own skew. They still add depth when you read them alongside survey results. Use them to spot emerging issues, then confirm the size of the problem with a structured sample.

Watch data quality

Research from Precisely and Drexel University in 2024 found that 67% of data professionals don't completely trust the data they use for decisions. A clean, well-planned sample earns more trust than a huge, messy one. Our guide to feedback deduplication covers cleanup, and our post on survey fatigue explains contact limits. You can track results by segment in NPS analysis, or book a demo.