Quality and team

How Many Calls Should You Review? A Practical QA Sampling Guide

How to decide how many calls to review for quality: random and targeted samples, per-agent minimums, small-team shortcuts and how to avoid bias in your sample.

Call editorial team · Published · 5 min read

"How many calls should we review?" is the first question every new quality programme asks, and the honest answer is: it depends on what you want to learn and how much time you have. Reviewing a tiny share of calls can miss real problems, but reviewing too many creates a backlog nobody has time to coach from. This guide explains how to think about sampling in practical terms, without a statistics degree.

A quality analyst at a desk selecting recordings from a call list on a monitor
Illustrative image

It is part of a wider quality routine described in Quality Management for Calls; the sample you choose should feed the sessions in A Simple Framework for Coaching Agents on Calls.

Start with the purpose of the sample

Different goals call for different samples. Decide which of these you are trying to do:

What is the sample for?

  • Fair agent feedbackEach agent gets a regular, comparable set of reviewed calls.
  • Spotting systemic issuesA broad view of what customers ask and where processes fail.
  • Risk and complianceTargeted review of calls where mistakes would matter most.
  • Training and recognitionFinding excellent calls to share and learn from.

One sample rarely does all four. Teams usually combine a small random sample for fairness with targeted calls for risk.

Random versus targeted sampling

Two ways to pick calls

RandomTargeted
How calls are chosenPicked by chance from all calls in the periodPicked by a signal such as length, sentiment, outcome or keywords
StrengthUnbiased view of typical callsFinds problems and risks faster
WeaknessMay miss rare serious issuesGives a skewed view if used alone
Best forAgent scores and trendsCompliance, complaints, coaching ideas

Use both. A common pattern is two or three random calls per agent each period, plus a few targeted calls reviewed for the whole team. Keep the two kinds separate in your reports; do not use targeted calls to calculate an agent's overall score, or you will punish agents for taking the hardest calls.

A practical way to choose the number

Work backwards from the time you have. A careful review of a call usually takes longer than the call itself: listening, scoring and writing a comment. Estimate honestly.

Choosing a sample size

  1. Estimate review timeFor example, a ten-minute call may need fifteen to twenty minutes to review and comment.
  2. Decide your weekly review budgetHow many hours can reviewers spend each week, after coaching time?
  3. Divide by review timeThis gives the number of calls you can review properly.
  4. Share across agentsGive each agent at least a minimum, and split the rest by risk and need.
  5. Check after a monthAdjust if reviews pile up or if you finish early.

Minimums per agent

Many teams set a floor, such as a few calls per agent per month, so that feedback is regular. Newer agents or those on a development plan may get more; experienced agents with stable results may get fewer. A larger sample for one agent does not mean a worse agent; it can simply mean extra support.

Small teams

With five to ten agents, you may be able to review a high share of calls. Do not worry about scientific precision. Pick a manageable number, such as a few calls per agent per month, review them well and coach from them. What matters is that it happens every month.

Avoiding bias in the sample

A sample can mislead if it is not chosen carefully. Watch for these traps:

Make selection rules written and visible. When agents know how calls are chosen, they are more likely to accept the results. Also check that the sample covers all channels and call types you handle, such as inbound, outbound, and different queues.

Fairness to agents

Agents accept review when they believe the process is fair. Explain how the sample is chosen, show them their own results in the same format as everyone else, and let them see the comments reviewers write. Give extra attention to the way you treat part-time agents, night staff and people on different call types, because a sample that is not comparable creates unfair scores.

When you find a recurring weakness across the sample, treat it as a team issue first: unclear process, missing information, or poor tools. Individual coaching comes after the system has been checked.

Useful signals for targeted review

Signals help you find calls that deserve a closer look without listening to everything:

SignalWhy it is usefulExample use
Very long callsMay show confusion, a hard issue or inefficiencyReview for process gaps
Customer negative at the endPossible failure or unmet needReview for service recovery
Transfers and repeat contactsPossible misrouting or incomplete answerReview for routing and knowledge
Flagged words or phrasesCompliance or care requirementsCheck against the rule
Very short callsHang-ups, wrong numbers, abrupt endingsCheck opening and handling
Excellent customer feedbackGood practice to learn fromAdd to the library of good calls

Using the sample well

A sample is worthless if nothing follows it. Share results with each agent quickly, so they remember the call. Aggregate across agents to find recurring weak items, and fix those with training, scripts or knowledge updates rather than blaming individuals. Run calibration so that reviewers score consistently; see Running QA Calibration Sessions.

A simple sampling plan

  • Purpose of the review stated
  • Random calls per agent per period
  • Targeted calls per period and the signals used
  • Minimums for new agents or those on a plan
  • Selection rules written and shared
  • Review time budgeted per week
  • Calibration scheduled
  • Results shared within a few days

Sampling in Call

In Call's Review & coaching area, the review queue lists calls waiting for a review, longest and most negative first, and shows up to twelve at a time, so reviewers start with the calls most likely to need attention. Per-agent results show evaluations, average score, failed calls and the sampling plan in use. A section for "Possible mis-selling" or "Customers needing extra care" lists calls flagged by phrase rules, with the words that matched, but only when something was flagged.

Where an AI agent answers calls, the Automatic review tab in Conversation AI can score recent calls that no human has reviewed and rank them by risk, with the reasons shown, and it groups calls into topics by the words customers used. It also lets you sample AI summaries and mark each one Accurate, Partly or Wrong. Use these tools to choose calls, but keep a human in the loop for the final judgement.

Frequently asked questions

Is there a standard percentage of calls to review?

No. Choose a number your team can review and coach from consistently. A small sample reviewed well is more useful than a large one that sits unread.

How many calls per agent per month is reasonable?

Many teams start with a few per agent and adjust. Newer agents often get more. The key is that every agent is reviewed regularly.

Should agents know which calls are reviewed?

They should know the selection rules and that calls may be reviewed. They need not know in advance which specific calls, though transparency about the process helps trust.

Can software choose the sample for us?

Software can rank calls by signals such as sentiment, length or flags, which saves time. Use that for targeted review and keep some random calls so the picture stays fair.

How do we avoid reviewers being biased?

Write selection rules, rotate reviewers, run calibration sessions and make sure samples cover all shifts and call types.

What if we cannot review enough calls?

Reduce the scorecard to the most important items, review fewer calls more carefully and use targeted signals. Add automation only where the rule is clear and reliable.

Find calls worth reviewing first with Call's review queue

In the demo console you can try incoming calls in a simulation.

Related guides