Prompt Design for Phone AI Agents: Instructions That Hold Up
How to write instructions for a voice AI agent that answers phone calls: role, tone, allowed answers, handoff rules, spoken style and how to test them.
The instructions you give a voice AI agent decide most of how it behaves on a call. Vague instructions produce vague, sometimes wrong, conversations. Over-long instructions bury what matters. Good instructions are short, specific and tested.

This guide is for people writing those instructions, often called a system prompt, for an agent that answers a business phone line. It assumes you know the basics from What Is AI Phone Reception? and How Voice AI Works. The aim is not clever wording but a reliable, honest agent.
Why phone prompts differ from chat prompts
A chat reply can be long, formatted and full of links. A spoken reply cannot. Callers hear each sentence once, in sequence, while waiting for the next. Everything about a phone prompt follows from that.
Chat versus phone
| Text chat | Phone call | |
|---|---|---|
| Length | Paragraphs are fine | One or two short sentences at a time |
| Format | Lists, links, bold | None of these; speak plainly |
| Errors | Reader can scroll back | Caller cannot; misheard details are costly |
| Interruptions | Rare | Common; callers talk over the agent |
| Names and numbers | Typed | Spoken and heard, so easily wrong |
A structure that works
Keep instructions in clear parts. Order them so the most important things come first.
Parts of a good phone-agent prompt
- RoleWho the agent is and for which business.
- GoalWhat a good call looks like.
- ToneHow it should sound, such as calm and friendly.
- Facts and sourcesWhat it may rely on and what to do when it does not know.
- RulesWhat it must always or never do.
- HandoffWhen and how to pass the call to a person or take a message.
Role and goal
One or two sentences are enough: "You answer the phone for a dental clinic. Your goal is to help callers with opening hours and directions, and to take a request for an appointment or a message for the team." Avoid grand personalities. A clear job beats a clever character.
Tone
Name the tone you want and keep it consistent: calm, polite, concise. If your business has a style, such as formal Japanese honorifics, say so explicitly. Do not ask for humour or chattiness unless it suits your callers.
Writing for spoken replies
- Short. One or two sentences, then stop and let the caller respond.
- No lists, links or emoji. They cannot be spoken well.
- One question at a time. "May I have your name?" then "And your number?"
- Read back important details. Names, numbers and dates are easily misheard. Have the agent repeat them to confirm.
- Spell out how to say things. Tell it how to read your business name, product names and any unusual words.
Facts, limits and "I don't know"
The biggest risk with a language model is a confident wrong answer. Reduce it by giving the agent trusted material and a clear rule for gaps.
Four rules that prevent most problems
- Answer only from supplied materialHours, services and policies that you provide and keep up to date.
- Say when it does not know"I cannot confirm that on this call; I can take a message or put you through."
- Never invent specificsNo prices, dates or promises that are not in your material.
- Stay in scopePolitely decline topics outside the business.
For larger sets of information, see Retrieval for Voice Agents and Voice AI Guardrails.
Be careful with bookings and commitments
An AI agent should not confirm a booking, give a discount or promise a callback time on its own unless your system truly supports it and you have decided it should. A safer pattern is that the agent takes the request, reads it back, and says that the team will confirm. State this in the prompt in plain terms.
Handoff and escalation
Say exactly when the agent should hand over. Typical triggers are: the caller asks for a person, the caller is upset, the matter is urgent or sensitive, or the agent has failed to help after a couple of attempts. Also say what to do when nobody is available: take a message, offer a callback, or give opening hours. Connect it with Letting Callers Reach a Person Quickly.
Handoff rules to include
- The caller asks for a person
- The caller sounds upset or reports an emergency
- The question is outside the agent's material
- The agent has misunderstood twice in a row
- The topic involves payment, legal or medical decisions
- What to do if no one is available
A short example prompt
Here is a deliberately plain example for a hypothetical clinic. Adapt the wording; do not copy it unchanged.
- "You answer the phone for Riverside Clinic. Speak calmly and politely, in one or two short sentences."
- "Help callers with opening hours, the address and how to request an appointment. Use only the information provided to you."
- "If you are not sure, say you cannot confirm that on this call and offer to take a message or put them through."
- "Never give medical advice. If a caller describes an emergency, tell them to contact emergency services and offer to put them through to a person."
- "To take an appointment request, ask for their name, number and preferred time, read these back, and say that the team will confirm."
Notice what is absent: no long backstory, no promises about speed, and no instruction to be impressive. Each line either defines the job, limits it or says how to hand over. As you learn from calls, add rules for the real problems you see, and remove lines that no longer earn their place.
Testing and revising
A prompt is a draft until it has met real conversations. Test deliberately and change little at a time.
- Place practice calls covering typical questions, odd questions, silence and interruptions.
- Include callers with accents, noise and unusual names.
- Read the transcripts and note exactly where the agent went wrong.
- Change one thing in the prompt, then run the same calls again.
- Save a copy of each version that works, so you can roll back.
Finally, treat the prompt as living documentation. Date each version, note why a line was added, and review the whole prompt now and then to remove rules that no longer matter.
Doing it in Call
In Call, an agent's instructions sit in its system prompt, read together with its persona and the knowledge it can use. Persona settings include a name, role, tone, language, goals and rules, and a choice of model that trades speed against capability. The agent is built to answer only from your instructions, its attached script and matching passages of its knowledge, to say plainly when it cannot confirm something and to offer a person where the line can put callers through. It replies in short spoken sentences. Instructions cannot switch off these built-in rules. Test calls and the playground let you hear the result, and restore points save a copy of an agent before a big change so you can roll back. See Setting Up an AI Agent in Call.
Frequently asked questions
How long should a phone agent prompt be?
As short as it can be while covering role, goal, tone, sources, rules and handoff. Long prompts bury the important instructions.
Why should replies be so short?
Callers hear each sentence once and cannot scroll back. Short replies are easier to follow and let the caller interrupt naturally.
How do I stop the agent making things up?
Supply approved material, tell it to answer only from it, and give it a standard phrase for when it cannot confirm something.
Can the agent confirm appointments?
Only if your system and rules truly support it. A safer default is that it takes the request and the team confirms.
How should I test a prompt?
Place practice calls with ordinary and awkward questions, read the transcripts, change one thing and repeat.
What should I do if a change makes things worse?
Roll back to the last working version. Keeping a saved copy of each good version makes this quick.