Using Call

Practice Calls: Testing Your AI Agent Before It Goes Live

How simulated practice calls let you test an AI phone agent on hard cases first, how to read the grades and lessons, and how to turn findings into fixes.

Call editorial team · Published · 6 min read

Any team that has trained new staff knows the first week on the phones is the riskiest. The new person knows the policies but has not yet met the caller who talks over them, the one who changes subject mid-sentence or the one who just wants somebody to blame. Good teams rehearse these moments with role-play before the real thing.

A team lead reviewing call scores and transcripts on a monitor before a launch
Illustrative image

An AI phone agent needs the same rehearsal, and it can have far more of it, far faster. This guide explains how to design practice, how to read the results and how to feed lessons back. It builds on setting up an AI agent and writing prompts that work.

Why practise before going live

The first real calls shape what customers think of the whole service. An agent that mishears a name, talks over the caller or invents an answer costs more than a missed call, because the customer now has a story to tell. Practice finds those problems while the only audience is your own team.

Practice is not proof. A good score on simulated callers does not mean real callers will behave the same way, and live voice behaviour can differ from a simulation. Treat a good result as permission to try with real calls carefully, not as a certificate.

What to test

The value of a batch depends on the situations it covers. Mix easy and hard cases deliberately.

Scenarios that expose weaknesses

  • InterruptionsA caller who talks over the agent. Does it stop and listen, or plough on?
  • Language switchingA caller who starts in one language and moves to another. Does the agent follow?
  • Asking for a personThe caller says "let me talk to someone". Is the hand-over clear and prompt?
  • Price questionsA caller asks what it costs. Does the agent answer only what it knows and avoid guessing?
  • ComplaintsAn upset caller. Does the agent stay calm and offer a next step?
  • Everyday questionsQuestions taken from the agent's own knowledge base. Are the answers right and short?

The balance matters. A batch of only hard cases will always look bad and teach little about everyday performance; a batch of only easy questions will flatter an agent that fails under pressure. A mix of both is usually the most informative start.

Running a practice batch in Call

In Call, practice lives under AI agents, in What it learned, in the Practice calls tab. AI callers phone your AI agents in a simulation, in Japanese and English, and every call is graded. You choose the agents (up to ten at once; an agent that cannot answer calls yet is greyed out), how many Japanese and English calls each should get (from none to twenty-five of each), and the scenario mix.

Scenario mixWhat it coversUse it when
BalancedEach hard case once, the rest questions from the agent's knowledge baseYou are testing a new agent for the first time
Knowledge baseOnly questions drawn from what the agent knowsYou changed its knowledge and want to check accuracy
Edge casesOnly the hard casesYou changed its behaviour rules or prompt

Running a batch

  1. Open Practice callsGo to AI agents, then What it learned, then Practice calls.
  2. Choose agents and countsTick the agents and set the number of calls in each language.
  3. Pick a mixBalanced, Knowledge base or Edge cases.
  4. Start the batchThe number beside the button is the total: agents multiplied by calls in both languages.
  5. Follow progressThe list refreshes itself; open a batch to see scores and call results.

The start button stays greyed out at zero calls or above a total of three hundred, so large tests are run in portions. You can also cancel a batch that is waiting or running; calls not started are cancelled and a call already in progress finishes. You can turn on voice for a batch to add a play button to each turn, and the audio is made only when you press it.

Reading the results

Open a finished batch and start with the summary, not a single transcript. Totals show calls done, the average score, corrected replies, questions that need an answer and style tips. Below them come the score by criterion, the top issues, with each agent's average, and a table of calls with their scenario, language, score and flags.

  1. Look at the score by criterion to find the weakest area.
  2. Read the top issues; if one appears across agents, it is probably a shared prompt or rule.
  3. Sort the calls by score and read the worst three transcripts in full.
  4. Read the grader's note on each reply, not only the final grade.
  5. Compare languages; a gap may mean one language needs its own wording.

Applying the lessons

Practice produces lessons, and they wait for your review rather than changing the agent automatically. You can apply or dismiss each one. Corrected replies and questions that need an answer also wait in Suggestions. This is deliberate: a person decides what the agent should learn.

Reviewing lessons

  • Does the lesson match your policy and tone
  • Is the corrected reply actually correct, not merely plausible
  • Does it apply to every caller, or only to this scenario
  • Are two lessons contradicting one another
  • Will the agent need a new knowledge entry, not a prompt change

After applying changes, run a small batch again and compare. Change one thing at a time where you can, otherwise you will not know which change helped. See knowledge sources for where factual answers belong.

A sensible practice cadence

Practice pays off most when it is a habit, not a one-off event. Tie batches to moments when something changes.

MomentWhat to runWhat to look for
New agent, before launchA balanced batch in each language you serveWeak criteria, hand-over behaviour
After a prompt or rule changeAn edge-case batchWhether the change fixed the problem without breaking another
After adding knowledgeA knowledge-base batchCorrect, short answers and no invented facts
Before a busy periodA balanced batch on the lines that will be busiestStability under a typical mix of questions
After a real-call problemA batch that mimics the failureWhether the fix holds

Keep a short log: the date, what you changed, the batch you ran and the headline result. After a few months this history shows which changes helped, and it protects you from repeating an experiment that already failed.

A hypothetical clinic could run a balanced batch for its reception agent before opening the new line, an edge-case batch after adding a rule about urgent symptoms, and a knowledge-base batch whenever the opening hours or services change. Three small batches, run at the right moments, say more than one huge test run once.

Beyond simulation

A simulation is a rehearsal. Before launch, also make a real test call to your own phone and listen as a customer would. Go live carefully: begin with a low-risk line or limited hours, keep an easy route to a person (see handing a call to a person) and listen to the first real calls daily. For human agents the same idea applies; see a coaching framework for rehearsal and feedback with people.

Frequently asked questions

Are practice calls real phone calls?

They are simulated calls in which AI callers phone your AI agents. Use a separate test call to your own phone to hear the agent on a real line.

Can I practise in more than one language?

The practice tool runs simulated calls in Japanese and English. Set how many of each you want per agent.

How many calls should I run?

Start small, with a few of each scenario per agent, and read the results closely. Increase only when you have a reason, such as testing a change across languages.

Do lessons change my agent automatically?

No. Lessons wait for your review. You apply the ones you agree with and dismiss the others.

What does the grade tell me?

It shows how the simulated call went against the criteria, with the grader's notes on each reply. It is evidence, and real callers may still behave differently.

What if an agent is greyed out?

An agent that cannot answer calls yet is marked as unable to answer and cannot be selected. Finish its setup first.

Start a practice batch for your AI agent in Call

In the demo console you can try incoming calls in a simulation.

Related guides