Every voice system mishears. Ours does, yours will, and the vendor telling you theirs does not has either not measured it or is not telling you.
What separates a usable system from a liability is not the error rate. It is what happens to an error once it occurs. Here is why phone audio is genuinely hard, and the four failures worth understanding because they are the ones that reach customers.
Why a phone line is the hard case
The audio is narrowband. Traditional telephony carries roughly 300 to 3,400 hertz — a fraction of what you hear in a room. The frequencies that distinguish "s" from "f", or "fifteen" from "fifty", sit partly outside that range. Then it is compressed for transmission. A recogniser that performs beautifully on a podcast is working from far less information on a phone call.
The room is against you. A dryer, a dining room, a street, a car. The caller is frequently on speakerphone while doing something else.
People do not speak in sentences. They trail off, restart, talk over you, say "um", and change their mind mid-request. Written language is tidy; speech is not.
Names and service words are the worst case. A recogniser leans on what is statistically likely in general English. Your technician's name and your service names are not statistically likely, which is exactly why they are misheard most.
The four failures worth knowing
These are all real, all from our own test calls, and all found before a customer met them.
Words become digits. "The pedicure one" came back as "The Pedicure 1". The system read the 1 as an appointment number, moved a different booking, and confirmed it as the pedicure. The mishearing was small; the consequence was not.
AM and PM swap. "Two PM" returned as "2 AM", with high confidence. Most systems can reason that a salon is shut at two in the morning. The question is what they do next — ask, or quietly book the nearest real slot without mentioning that they changed the time.
Speech becomes a name. "Actually, wait" was transcribed as "Ashley Way", which sounds exactly like a person, and the caller's profile was renamed. The interruption itself worked perfectly. What the system did with the interruption did not.
Confidence does not track correctness. This is the one that matters most. A recogniser returning the wrong words is usually not flagging uncertainty — it returns a wrong answer as confidently as a right one. So "only act on high-confidence transcriptions" is not the safeguard it sounds like.
The thing that actually protects the caller
Not accuracy. Consequence-awareness.
The right design question is not "how often is it right" but "what is the worst thing this action can do if the words were wrong". Those two questions produce very different systems.
Reading back a price is low-consequence — if it is wrong, the caller corrects you. Moving an existing appointment is high-consequence — if it is wrong, nobody finds out until someone arrives.
So the actions that can quietly damage something must confirm first. "I've got a pedicure on Thursday at two — is that the one?" costs two seconds and eliminates the entire class of failure above. That sentence is worth more than several percentage points of recognition accuracy.
We go through the machinery underneath all this in how voice AI understands a phone call.
What you can do about it
Give it your vocabulary. Service names, staff names, the words your business uses. Most systems accept a list of terms to weight, and it makes a real difference to exactly the words that fail most.
Read the transcripts for the first fortnight. This is the single most useful hour you will spend. You will find out what your callers actually say, which is reliably not what you assumed, and you will see which words fail.
Test the high-consequence paths specifically. Not "does it book" — does it name the booking before moving it. Does it query 2 AM. Does it treat a name arriving mid-interruption as something to confirm rather than save.
The question to ask a vendor
Not "what is your accuracy rate?" — you will get a number measured on clean audio that tells you nothing about your dryer.
Ask instead: "What does it do when it is not sure?"
A good answer describes asking. A bad answer describes accuracy. A system whose plan for uncertainty is to be accurate has no plan for uncertainty, and the phone will find its limits within a week.
The five specific tests, with what to listen for in each, are in five calls to make before you trust an AI receptionist.