How Many Calls Should You Review? A Practical QA Sampling Guide
How to decide how many calls to review for quality: random and targeted samples, per-agent minimums, small-team shortcuts and how to avoid bias in your sample.
"How many calls should we review?" is the first question every new quality programme asks, and the honest answer is: it depends on what you want to learn and how much time you have. Reviewing a tiny share of calls can miss real problems, but reviewing too many creates a backlog nobody has time to coach from. This guide explains how to think about sampling in practical terms, without a statistics degree.

It is part of a wider quality routine described in Quality Management for Calls; the sample you choose should feed the sessions in A Simple Framework for Coaching Agents on Calls.
Start with the purpose of the sample
Different goals call for different samples. Decide which of these you are trying to do:
What is the sample for?
- Fair agent feedbackEach agent gets a regular, comparable set of reviewed calls.
- Spotting systemic issuesA broad view of what customers ask and where processes fail.
- Risk and complianceTargeted review of calls where mistakes would matter most.
- Training and recognitionFinding excellent calls to share and learn from.
One sample rarely does all four. Teams usually combine a small random sample for fairness with targeted calls for risk.
Random versus targeted sampling
Two ways to pick calls
| Random | Targeted | |
|---|---|---|
| How calls are chosen | Picked by chance from all calls in the period | Picked by a signal such as length, sentiment, outcome or keywords |
| Strength | Unbiased view of typical calls | Finds problems and risks faster |
| Weakness | May miss rare serious issues | Gives a skewed view if used alone |
| Best for | Agent scores and trends | Compliance, complaints, coaching ideas |
Use both. A common pattern is two or three random calls per agent each period, plus a few targeted calls reviewed for the whole team. Keep the two kinds separate in your reports; do not use targeted calls to calculate an agent's overall score, or you will punish agents for taking the hardest calls.
A practical way to choose the number
Work backwards from the time you have. A careful review of a call usually takes longer than the call itself: listening, scoring and writing a comment. Estimate honestly.
Choosing a sample size
- Estimate review timeFor example, a ten-minute call may need fifteen to twenty minutes to review and comment.
- Decide your weekly review budgetHow many hours can reviewers spend each week, after coaching time?
- Divide by review timeThis gives the number of calls you can review properly.
- Share across agentsGive each agent at least a minimum, and split the rest by risk and need.
- Check after a monthAdjust if reviews pile up or if you finish early.
Minimums per agent
Many teams set a floor, such as a few calls per agent per month, so that feedback is regular. Newer agents or those on a development plan may get more; experienced agents with stable results may get fewer. A larger sample for one agent does not mean a worse agent; it can simply mean extra support.
Small teams
With five to ten agents, you may be able to review a high share of calls. Do not worry about scientific precision. Pick a manageable number, such as a few calls per agent per month, review them well and coach from them. What matters is that it happens every month.
Avoiding bias in the sample
A sample can mislead if it is not chosen carefully. Watch for these traps:
- Only reviewing short calls because they are quicker. Long calls often hide the real difficulty.
- Reviewing the same agents more because they are noticed more. This looks like favouritism or a witch hunt.
- Reviewing only complaints. You will get a gloomy picture and miss the good practice.
- Always reviewing the same shift. Different times of day bring different callers and pressures.
- Letting agents choose all their calls. They will choose the best ones. A mix works better; see the coaching framework.
Make selection rules written and visible. When agents know how calls are chosen, they are more likely to accept the results. Also check that the sample covers all channels and call types you handle, such as inbound, outbound, and different queues.
Fairness to agents
Agents accept review when they believe the process is fair. Explain how the sample is chosen, show them their own results in the same format as everyone else, and let them see the comments reviewers write. Give extra attention to the way you treat part-time agents, night staff and people on different call types, because a sample that is not comparable creates unfair scores.
- Use the same scorecard for every agent on the same kind of call.
- Do not compare scores between agents who handle different call types without noting the difference.
- Allow agents to question a score, with a simple route to a second opinion.
- Keep score data private to the agent, their lead and reviewers.
When you find a recurring weakness across the sample, treat it as a team issue first: unclear process, missing information, or poor tools. Individual coaching comes after the system has been checked.
Useful signals for targeted review
Signals help you find calls that deserve a closer look without listening to everything:
| Signal | Why it is useful | Example use |
|---|---|---|
| Very long calls | May show confusion, a hard issue or inefficiency | Review for process gaps |
| Customer negative at the end | Possible failure or unmet need | Review for service recovery |
| Transfers and repeat contacts | Possible misrouting or incomplete answer | Review for routing and knowledge |
| Flagged words or phrases | Compliance or care requirements | Check against the rule |
| Very short calls | Hang-ups, wrong numbers, abrupt endings | Check opening and handling |
| Excellent customer feedback | Good practice to learn from | Add to the library of good calls |
Using the sample well
A sample is worthless if nothing follows it. Share results with each agent quickly, so they remember the call. Aggregate across agents to find recurring weak items, and fix those with training, scripts or knowledge updates rather than blaming individuals. Run calibration so that reviewers score consistently; see Running QA Calibration Sessions.
A simple sampling plan
- Purpose of the review stated
- Random calls per agent per period
- Targeted calls per period and the signals used
- Minimums for new agents or those on a plan
- Selection rules written and shared
- Review time budgeted per week
- Calibration scheduled
- Results shared within a few days
Sampling in Call
In Call's Review & coaching area, the review queue lists calls waiting for a review, longest and most negative first, and shows up to twelve at a time, so reviewers start with the calls most likely to need attention. Per-agent results show evaluations, average score, failed calls and the sampling plan in use. A section for "Possible mis-selling" or "Customers needing extra care" lists calls flagged by phrase rules, with the words that matched, but only when something was flagged.
Where an AI agent answers calls, the Automatic review tab in Conversation AI can score recent calls that no human has reviewed and rank them by risk, with the reasons shown, and it groups calls into topics by the words customers used. It also lets you sample AI summaries and mark each one Accurate, Partly or Wrong. Use these tools to choose calls, but keep a human in the loop for the final judgement.
Frequently asked questions
Is there a standard percentage of calls to review?
No. Choose a number your team can review and coach from consistently. A small sample reviewed well is more useful than a large one that sits unread.
How many calls per agent per month is reasonable?
Many teams start with a few per agent and adjust. Newer agents often get more. The key is that every agent is reviewed regularly.
Should agents know which calls are reviewed?
They should know the selection rules and that calls may be reviewed. They need not know in advance which specific calls, though transparency about the process helps trust.
Can software choose the sample for us?
Software can rank calls by signals such as sentiment, length or flags, which saves time. Use that for targeted review and keep some random calls so the picture stays fair.
How do we avoid reviewers being biased?
Write selection rules, rotate reviewers, run calibration sessions and make sure samples cover all shifts and call types.
What if we cannot review enough calls?
Reduce the scorecard to the most important items, review fewer calls more carefully and use targeted signals. Add automation only where the rule is clear and reliable.