Never Send £42.50 to a Voice Model
A voice agent does not usually fail by sounding robotic. It fails on one string: a price read as digits, a date read month-first, a postcode read as a word, a street called Saint. The caller forgives a slightly synthetic voice and does not forgive being told the wrong number.
The good news is that this class of failure has a name, a forty-year literature, and a fix that lives in your code rather than in the voice model. The bad news is that the industry's latency race is actively making it worse, and the markup most teams reach for fails silently by design.
We wrote about the input side, the agent mishearing the caller, in the names and postcodes guide. This is the output side: what the agent says back.
Key takeaways
- Reading "£42.50" aloud is a pipeline stage called text normalisation, not a voice-quality problem. It is fixable in text.
- The SSML
<say-as>values everyone relies on were never standardised, and the spec requires unsupported values to fail silently.- ElevenLabs documents that normalisation is disabled by default on Flash v2.5 "to maintain the low latency". The fast model is fast partly because it skips the step that reads your prices right.
- Pronunciation control is per-model, not per-vendor. ElevenLabs phoneme tags are documented on exactly one model.
- The reliable fix is pre-normalisation: send the voice "forty-two pounds fifty", never "£42.50".
Why does the voice read prices and dates wrong?
Because "£42.50" is not a word, and turning it into words is a separate pipeline stage that can fail independently of the voice. The W3C SSML specification names the stage: "Text normalization is an automated process of the synthesis processor" that converts written forms like currency amounts and dates into spoken forms, before any audio is generated.
The research community has had a name for the inputs since 2001. Sproat and colleagues called them non-standard words: numbers, abbreviations, dates, currency amounts and acronyms that "cannot be found in a dictionary" and cannot be pronounced by ordinary letter-to-sound rules. Their paper also documents why "St" keeps going wrong: engines guess Saint or Street from capitalisation of the neighbouring words, and address data from a CRM breaks that heuristic constantly.
Why the model alone cannot be trusted
Modern models normalise implicitly inside the network, and a 2016 study by Google researchers, published as a preprint, showed why that is not enough: neural normalisers make rare but catastrophic errors, the wrong number or the wrong unit, precisely the errors that "convey completely the wrong message" in a speech application. Their fix was a deterministic rule-based filter on top. Keep that shape in mind, because it is the same fix we land on below.
Isn't SSML supposed to fix this?
Less than almost everyone believes, for two documented reasons.
First, the <say-as> values were never standardised. The SSML 1.1 Recommendation says plainly that it "does not enumerate the possible values" for the attributes; the familiar list, date, time, telephone, cardinal, characters, comes from a 2005 W3C Working Group Note that describes itself as a draft. Every vendor implements its own subset with its own behaviour.
Second, failure is silent by specification: "When the value for the interpret-as attribute is unknown or unsupported by a processor, it MUST render the contained text as if no interpret-as value were specified." Markup that works on one engine degrades invisibly on another. No error, no warning, just a phone number read as one enormous number.
The UK trap is codified in the standard itself. The 2005 Note says "1/2/2004" is ambiguous between the first of February and the second of January, and resolves it with a format attribute, dmy for British order. Azure's own documented example defaults to month-first. If your integration never sets the format, your 15/09 booking is being read as a date in the wrong month by any engine following the US default.
Where the engines actually stand
| Engine | Documented position |
|---|---|
| Azure | Fullest <say-as> set surveyed, including currency, telephone, date with dmy. Support is language-gated, and the date default is month-first |
Supports say-as; currency verbalisation follows the voice's locale unless you set a language attribute. On Chirp 3: HD voices, SSML is in Preview and "SSML tags are not currently supported for streaming requests" |
|
| ElevenLabs | No <say-as>. Normalisation is a model setting, and "By default, normalization is disabled for Flash v2.5 to maintain the low latency" |
| OpenAI | No pronunciation or normalisation markup of any kind. Controls are voice choice, a speed parameter and free-text instructions |
Read the Google row twice if you run a phone agent. Streaming is what a live call is, so the newest voices' SSML does not apply on exactly the requests a phone agent makes.
The fix is to write it the way you want it said
Render everything speakable in your application, before the text reaches the voice. This is not a workaround, it is what ElevenLabs' own documentation recommends for exactly the models a phone agent uses, down to target renderings: currency as words, phone numbers as grouped digits.
| Never send | Send instead |
|---|---|
| £42.50 | forty-two pounds fifty |
| 07700 900123 | oh seven seven oh oh, nine hundred, one two three |
| AL1 3QR | A L one, three Q R |
| 15/09 | the fifteenth of September |
| 18:30 | half past six in the evening |
| Dr Patel, Station Rd | Doctor Patel, Station Road |
Two implementation notes. The text your voice speaks is written by an LLM, so the first half of the rule lives in the system prompt: numbers, prices, times and postcodes are always written as spoken words. The second half is a deterministic post-pass that catches what the model misses, a small function that rewrites currency, phone and postcode patterns into words. That is the same architecture the 2016 normalisation study reached: model plus rule-based filter, because for high-stakes strings the deterministic layer is the one you can trust.
This also happens to be our house style for agent scripts for a different reason: it is what the caller hears either way.
How do you fix one mispronounced word?
With whatever your vendor offers, and the table is messy. Control is per-model, not per-vendor, and it changes between model generations.
| Vendor | Documented mechanism |
|---|---|
| ElevenLabs | SSML <phoneme> tags (IPA or CMU Arpabet) documented on eleven_flash_v2 only, one word per tag. PLS pronunciation dictionaries, at most three per request; phoneme rules apply only on supported models, alias rules elsewhere. v3 uses inline IPA, self-rated at 80 to 90% |
| Cartesia | Inline per-word IPA in the transcript, or a sounds-like form. A locale field on Sonic 3.6 where en-GB flips date reading to day-first |
| OpenAI | No markup. A documented prompting convention for Realtime voices: a reference-pronunciations section in the system prompt |
| Deepgram | Docs state Aura has no pronunciation-control feature and direct you to respellings in the text |
custom_pronunciations, documented as available across en-AU, en-GB, en-IN and en-US on Chirp 3: HD |
|
| Azure | <phoneme> with four alphabets. Uniquely, an invalid phone fails loudly with an HTTP error rather than silently |
The universal fallback is respelling, and ElevenLabs documents it with a UK name as its own example: write "Cloff-ton" for Claughton. It works on every engine because it is not a feature. Its cost is that the respelling leaks into everything downstream that reads the same text, transcripts, captions and your CRM notes, which is an argument for applying it at the last moment before the voice and nowhere else.
UK place names are the test set for all of this. A 2019 survey by linguists behind a language-learning app, reported by BBC News, put Frome top of the most-mispronounced list, and any agent serving Leicester, Marylebone or Southwark should be tested on saying them before a caller hears it guess.
How much control do you get over delivery?
Less each year, which is the finding in this section that surprised us. The industry is removing knobs, not adding them.
Azure's newest HD voices are documented as not supporting <prosody> or <emphasis>, the controls its standard voices have carried for years; the HD models infer emotion from text instead. Amazon Polly's neural, long-form and generative engines dropped controls the 2016-era standard engine still documents. Cartesia's changelog for sonic-2-2025-04-16 states it removed its experimental speed and emotion controls outright. ElevenLabs never had SSML prosody; its whole delivery surface is four sliders, with its own conversational guidance suggesting stability around 0.30 to 0.50, and v3 replaces <break> tags with punctuation and audio tags.
What replaces the knobs
Prompting. OpenAI's TTS takes free-text instructions about tone and pacing, and Hume's Octave formalises the same idea as acting instructions. The trade is real: prompt-driven delivery is more natural on average and less deterministic on any given sentence, and there is no markup to pin the one word that must be stressed.
The practical consequence for a phone agent: do not build your design around fine prosody control, because the model you migrate to next year probably will not have it. Control the text, which you will always own.
Does faster mean worse?
On normalisation, documented yes: ElevenLabs' fast model disables it by default to hit its latency target. The latency race and the correct reading of your prices are directly in tension, and knowing that is most of the defence.
On the headline numbers themselves, read the units before comparing. ElevenLabs' roughly 75ms figure for Flash v2.5 is model inference time, and its own engineering guidance for real deployments is 100 to 150ms time-to-first-byte over websockets in Europe and North America. Cartesia markets "sub-90ms" with no stated conditions. Rime publishes the best-conditioned figures we found, 37ms P50 and 56ms P90 time-to-first-audio, measured at its own edge. OpenAI publishes no latency figures at all, only model guidance. Independent benchmarks that measure what a caller experiences, first audio at the client, consistently land two to three times above vendor headlines, not because anyone is lying but because the metrics measure different spans.
The end-to-end budget, and why time-to-first-audio is only one term in it, is in the latency guide.
Should it sound as human as possible?
The evidence says clarity beats maximal humanness, and the popular version of the "uncanny valley of voice" is not what the research shows. Studies that go looking for a valley find discomfort driven by voices deviating from typical organic speech, distorted or category-ambiguous ones, rather than by near-human quality as such.
Meanwhile detection has collapsed as a defence. A 2025 study in Scientific Reports found listeners matched a cloned voice to its real speaker about 80% of the time yet identified the clone as AI only about 60% of the time, and a large-scale 2026 study, still a preprint, found listeners' accuracy on real voices falling, people now misjudge humans as machines.
Whether passing pays
So the question is not whether you can pass, it is whether passing pays, and the field data says be careful. In the Luo et al. Marketing Science field experiment, disclosing the bot up front cut purchases by more than 79.7%; on inbound service lines, the evidence in the disclosure guide runs the other way, disclosure reduces hang-ups. A Management Science field experiment by Xu, Dai and Yan adds the useful nuance: identity disclosure cut customer response by around 11%, and humanising the chatbot's conversational style improved response by over 5.6% to 24.9% depending on the feature, which the authors say may potentially counteract the disclosure penalty.
Our reading for a disclosed UK business line: make the voice pleasant, paced and warm, disclose it as AI, and spend the remaining effort on saying the price, the time and the postcode correctly. Nobody hangs up because the voice was slightly synthetic. They hang up because it read their phone number back wrong twice.
Frequently asked questions
How do I know if say-as works on my platform?
Test the audio, never the markup. The specification requires an engine that does not support your value to read the text as if the markup were absent, so acceptance proves nothing. Send a price, a UK-format date, a phone number and a postcode through your actual model and listen to what comes back, on the streaming path if calls are streamed, since at least one vendor documents different SSML support for streaming and synchronous requests.
What is the cheapest way to make it say a name right? Respelling in the text you send to the voice, which works on every engine and is documented by ElevenLabs as the fallback for models without phoneme support. Apply it at the final step before synthesis so the respelt form never reaches transcripts or your CRM. If your volume justifies it, move staff names and street names into a pronunciation dictionary on a model that honours one, and keep the respelling map as the portable fallback for the next migration.
Which TTS vendor is fastest? The honest answer is that the published numbers are not comparable. Vendor figures mostly measure model inference; what your caller experiences is time to first audio at their ear, across your region, your telephony leg and your chunking strategy, and independent measurements of that run well above the headline figures everywhere. Benchmark the two or three candidates from your own infrastructure on telephone-band audio, and remember the documented trade: the fastest models may ship with normalisation off.
Do fillers and breathing sounds make an agent better? The measured evidence is from text chatbots rather than voice: Xu, Dai and Yan found humanising conversational features improved customer response by over 5.6% to 24.9%. Voice-specific measurement is thin, and fillers cost real milliseconds and occasionally transcribe badly. We treat them as seasoning: a small amount in greetings and acknowledgements, never inside a price, a time or a readback, where clarity is the entire job.