Voice AI technology

Preventing Made-Up Answers in Voice AI on Phone Calls

Why voice agents sometimes state things that are not true, and the practical steps that limit it: grounded sources, safe fallbacks, trap questions and review.

Call editorial team · Published · 5 min read

Ask a voice agent for your opening hours and it will answer smoothly. The problem arises when it answers just as smoothly about something it was never told. This is what people mean by a hallucination: output that sounds right but has no basis in your information.

A person listening carefully on a phone call at their desk
Illustrative image

On a phone line the risk is sharper than in a chat window. The caller hears one confident sentence and no link, no source and no easy way to go back. This guide explains why hallucinations occur, how to reduce them and how to check that your fixes work.

What a hallucination is, and why it happens

A language model produces the most plausible continuation of a conversation. It was trained on huge amounts of text and has no built-in sense of which statements are true for your business. When the answer is missing from the prompt or the sources, the model can fill the gap with something that looks reasonable.

Typical causes on a phone agent

  • Missing informationThe caller asks something nobody wrote down, so the model improvises.
  • Outdated informationAn old price or opening time remains in a source.
  • Ambiguous sourcesTwo documents disagree and the agent picks one.
  • Pressure to be helpfulA prompt that says always answer encourages guessing.
  • Speech recognition errorsA misheard word changes the question, so the agent answers a different one.

The last cause is easy to overlook. If the transcript says Tuesday when the caller said Thursday, the answer may be perfectly accurate for the wrong question. See Speech Recognition on Phone Calls for how recognition errors arise.

What it looks like in practice

A hypothetical dental clinic might tell its agent about opening hours and the services it offers. A caller asks whether the clinic accepts a particular insurance plan. With no information on that, a careless agent might say yes, because many clinics do. The caller turns up expecting cover and is unhappy. The fix is not a smarter model but a clear instruction and a source that says what to do when the answer is unknown.

Common high-risk topics are prices, availability, legal or medical advice, refund promises and anything that sounds like a commitment. Mark these as areas where the agent must not improvise.

Why voice makes it worse

A written chatbot answer can carry a link, a disclaimer or a visible list of sources. Speech carries none of these. Synthetic voices are also smooth and steady, so a guess sounds exactly like a fact; there is no hesitation to warn the caller. Callers often act on what they hear immediately, by driving somewhere, cancelling something or paying something.

That is why the design bias for phone agents should lean toward restraint. A short honest answer that points to a person costs a few seconds. A confident wrong answer can cost a customer. When you decide how strict to be, think about the worst plausible consequence of a wrong answer on each topic and set the rule accordingly: relaxed for opening hours, strict for anything involving money, health or legal rights.

How to reduce made-up answers

A layered approach

  1. Ground the agentGive it approved facts and tell it to answer only from them.
  2. Define the unknownWrite what it should say when it lacks an answer.
  3. Restrict commitmentsForbid promises it cannot keep, such as discounts or confirmed bookings.
  4. Offer a routeAlways provide a person, a message or a callback as the next step.
  5. Test and reviewTry questions outside the sources and read real transcripts.

Ground it in approved sources

Retrieval means looking up relevant passages from your documents at the moment of the question and giving only those to the model. It keeps answers tied to text you wrote. Retrieval for Voice Agents explains the idea, and Knowledge Sources in Call shows how it appears in the product.

Write short, unambiguous facts

One fact per entry, with the condition included, works better than a long brochure. Instead of We are open on weekends, write Saturday 9 to 13, closed on Sunday. If a topic has exceptions, state them in the same entry.

Use exact answers for key questions

For questions where wording matters, such as cancellation rules, give the agent the exact answer to read out. In Call you can add a question and the exact answer the agent should give, so it need not paraphrase.

Teach it to say I do not know

An honest I cannot confirm that on this call, but I can take your details and have someone get back to you is far better than a guess. In Call, the agent is designed to say plainly that it cannot confirm something when nothing in its instructions or sources answers the question, and to offer a person where the line can put callers through. It also never invents prices, hours, dates or other facts that your instructions and sources do not state, and your instructions cannot switch these rules off.

Grounded versus ungrounded answers

Same question, two set-ups

Ungrounded agentGrounded agent
Source of the answerGeneral knowledge and plausible guessesYour approved facts only
When information is missingInvents something fluentStates it cannot confirm, offers a next step
Risk to the callerMay act on wrong detailsKnows what is uncertain
MaintenanceEasy to start, hard to trustNeeds sources kept current

Test for hallucinations on purpose

Do not wait for a customer to find the gaps. Build a small set of trap questions and run them whenever you change the prompt or sources.

A hallucination test set

  • Ask about a service you do not offer
  • Ask for a price that is not in the sources
  • Ask about a policy exception nobody wrote down
  • Ask for a date far in the future
  • Ask a question in a different phrasing from your written answers
  • Ask something that sounds similar to a covered topic
  • Check that every unknown ends with a safe next step

Repeat the set after changes, and add every real failure you find to it. The wider testing process is described in Testing a Voice AI Agent.

Keep watching after launch

Hallucination control is not a one-off task. Information changes, new questions appear and prompts drift as people edit them. Set a routine: read a sample of transcripts each week, search for phrases that signal doubt, and update the sources. In Call, you can open the full conversation of any call, search all stored transcripts for a phrase and see the history of corrections made to the agent's words. Remember that transcripts are kept for a limited time, so review them promptly.

Applying this in Call

Start by writing your most-asked questions as exact question and answer pairs, add short factual sources for everything else, and attach the agent to a line only after testing. If you want help drafting prompts that discourage guessing, read Prompting an LLM Phone Agent. The guardrails that sit alongside this work are covered in Guardrails for Voice AI Agents.

Frequently asked questions

What is a voice AI hallucination?

It is an answer that sounds confident but is not supported by your information, for example a made-up price or policy. It happens when the model fills a gap instead of admitting it does not know.

Can hallucinations be eliminated completely?

No tool can promise that. You can reduce them a lot by limiting what the agent may say, keeping sources current and giving it a safe fallback, but you should keep testing and reviewing.

Why does the agent answer a question I never trained it on?

Language models are built to produce plausible text. Without a clear instruction to stay within your sources and say when it cannot confirm something, it may improvise.

Is a misheard word the same as a hallucination?

Not exactly. A misheard word is a recognition error, but it can lead to a wrong answer that looks like a hallucination. Check the transcript to see which one happened.

Which topics are riskiest?

Prices, availability, medical or legal advice, refunds and any promise or commitment. Give the agent exact wording or a handover for these.

How often should I review transcripts?

Daily or every few days at launch, then weekly. Review more often after you change the prompt or sources.

Ground your AI agent in approved answers in Call

In the demo console you can try incoming calls in a simulation.

Related guides