Practice Calls: Testing Your AI Agent Before It Goes Live
How simulated practice calls let you test an AI phone agent on hard cases first, how to read the grades and lessons, and how to turn findings into fixes.
Any team that has trained new staff knows the first week on the phones is the riskiest. The new person knows the policies but has not yet met the caller who talks over them, the one who changes subject mid-sentence or the one who just wants somebody to blame. Good teams rehearse these moments with role-play before the real thing.

An AI phone agent needs the same rehearsal, and it can have far more of it, far faster. This guide explains how to design practice, how to read the results and how to feed lessons back. It builds on setting up an AI agent and writing prompts that work.
Why practise before going live
The first real calls shape what customers think of the whole service. An agent that mishears a name, talks over the caller or invents an answer costs more than a missed call, because the customer now has a story to tell. Practice finds those problems while the only audience is your own team.
- Coverage. You can test dozens of calls in a batch, including unusual ones that might not occur in the first weeks.
- Repeatability. After you change a prompt, run the same kind of batch and compare.
- Safety. Mistakes cost nothing with real customers.
- Evidence. Scores and flags give you something to discuss other than opinions.
Practice is not proof. A good score on simulated callers does not mean real callers will behave the same way, and live voice behaviour can differ from a simulation. Treat a good result as permission to try with real calls carefully, not as a certificate.
What to test
The value of a batch depends on the situations it covers. Mix easy and hard cases deliberately.
Scenarios that expose weaknesses
- InterruptionsA caller who talks over the agent. Does it stop and listen, or plough on?
- Language switchingA caller who starts in one language and moves to another. Does the agent follow?
- Asking for a personThe caller says "let me talk to someone". Is the hand-over clear and prompt?
- Price questionsA caller asks what it costs. Does the agent answer only what it knows and avoid guessing?
- ComplaintsAn upset caller. Does the agent stay calm and offer a next step?
- Everyday questionsQuestions taken from the agent's own knowledge base. Are the answers right and short?
The balance matters. A batch of only hard cases will always look bad and teach little about everyday performance; a batch of only easy questions will flatter an agent that fails under pressure. A mix of both is usually the most informative start.
Running a practice batch in Call
In Call, practice lives under AI agents, in What it learned, in the Practice calls tab. AI callers phone your AI agents in a simulation, in Japanese and English, and every call is graded. You choose the agents (up to ten at once; an agent that cannot answer calls yet is greyed out), how many Japanese and English calls each should get (from none to twenty-five of each), and the scenario mix.
| Scenario mix | What it covers | Use it when |
|---|---|---|
| Balanced | Each hard case once, the rest questions from the agent's knowledge base | You are testing a new agent for the first time |
| Knowledge base | Only questions drawn from what the agent knows | You changed its knowledge and want to check accuracy |
| Edge cases | Only the hard cases | You changed its behaviour rules or prompt |
Running a batch
- Open Practice callsGo to AI agents, then What it learned, then Practice calls.
- Choose agents and countsTick the agents and set the number of calls in each language.
- Pick a mixBalanced, Knowledge base or Edge cases.
- Start the batchThe number beside the button is the total: agents multiplied by calls in both languages.
- Follow progressThe list refreshes itself; open a batch to see scores and call results.
The start button stays greyed out at zero calls or above a total of three hundred, so large tests are run in portions. You can also cancel a batch that is waiting or running; calls not started are cancelled and a call already in progress finishes. You can turn on voice for a batch to add a play button to each turn, and the audio is made only when you press it.
Reading the results
Open a finished batch and start with the summary, not a single transcript. Totals show calls done, the average score, corrected replies, questions that need an answer and style tips. Below them come the score by criterion, the top issues, with each agent's average, and a table of calls with their scenario, language, score and flags.
- Look at the score by criterion to find the weakest area.
- Read the top issues; if one appears across agents, it is probably a shared prompt or rule.
- Sort the calls by score and read the worst three transcripts in full.
- Read the grader's note on each reply, not only the final grade.
- Compare languages; a gap may mean one language needs its own wording.
Applying the lessons
Practice produces lessons, and they wait for your review rather than changing the agent automatically. You can apply or dismiss each one. Corrected replies and questions that need an answer also wait in Suggestions. This is deliberate: a person decides what the agent should learn.
Reviewing lessons
- Does the lesson match your policy and tone
- Is the corrected reply actually correct, not merely plausible
- Does it apply to every caller, or only to this scenario
- Are two lessons contradicting one another
- Will the agent need a new knowledge entry, not a prompt change
After applying changes, run a small batch again and compare. Change one thing at a time where you can, otherwise you will not know which change helped. See knowledge sources for where factual answers belong.
A sensible practice cadence
Practice pays off most when it is a habit, not a one-off event. Tie batches to moments when something changes.
| Moment | What to run | What to look for |
|---|---|---|
| New agent, before launch | A balanced batch in each language you serve | Weak criteria, hand-over behaviour |
| After a prompt or rule change | An edge-case batch | Whether the change fixed the problem without breaking another |
| After adding knowledge | A knowledge-base batch | Correct, short answers and no invented facts |
| Before a busy period | A balanced batch on the lines that will be busiest | Stability under a typical mix of questions |
| After a real-call problem | A batch that mimics the failure | Whether the fix holds |
Keep a short log: the date, what you changed, the batch you ran and the headline result. After a few months this history shows which changes helped, and it protects you from repeating an experiment that already failed.
A hypothetical clinic could run a balanced batch for its reception agent before opening the new line, an edge-case batch after adding a rule about urgent symptoms, and a knowledge-base batch whenever the opening hours or services change. Three small batches, run at the right moments, say more than one huge test run once.
Beyond simulation
A simulation is a rehearsal. Before launch, also make a real test call to your own phone and listen as a customer would. Go live carefully: begin with a low-risk line or limited hours, keep an easy route to a person (see handing a call to a person) and listen to the first real calls daily. For human agents the same idea applies; see a coaching framework for rehearsal and feedback with people.
Frequently asked questions
Are practice calls real phone calls?
They are simulated calls in which AI callers phone your AI agents. Use a separate test call to your own phone to hear the agent on a real line.
Can I practise in more than one language?
The practice tool runs simulated calls in Japanese and English. Set how many of each you want per agent.
How many calls should I run?
Start small, with a few of each scenario per agent, and read the results closely. Increase only when you have a reason, such as testing a change across languages.
Do lessons change my agent automatically?
No. Lessons wait for your review. You apply the ones you agree with and dismiss the others.
What does the grade tell me?
It shows how the simulated call went against the criteria, with the grader's notes on each reply. It is evidence, and real callers may still behave differently.
What if an agent is greyed out?
An agent that cannot answer calls yet is marked as unable to answer and cannot be selected. Finish its setup first.