What is speech to text (STT)?
The system that turns a caller's spoken words into text the language model can actually read and act on.
Speech to text (STT) — Speech to text (STT) converts spoken audio into written words, and in a voice agent it turns what the caller says into text the language model can act on. Its accuracy on accents, noise and names decides whether the agent understood.
Also known as: STT, transcription
Why it matters
Whatever the caller says has to become text before a language model can use it, and STT is entirely responsible for that step. If the transcription is wrong, everything downstream is wrong too: the model reasons over words the caller never said, and no amount of prompt design fixes an agent that mis-heard the question. STT accuracy is the ceiling on how well a voice agent can perform, not the model's reasoning.
The hard cases are the ones demos rarely show: strong accents, background noise, a bad connection, and proper nouns, names, addresses, product names, that no general model was trained to expect. A voice agent that works well on a quiet studio recording of clear, standard speech and has never been tested against a real caller on a speakerphone in a noisy room is untested where it matters.
What STT does in a call
- 1
The caller speaks
Audio arrives over the phone line, often already degraded by the call itself.
- 2
Audio is transcribed
The STT engine converts speech to text, usually streaming partial results as the caller talks.
- 3
The model reads the transcript
The language model reasons over the text, not the original audio.
- 4
Errors compound downstream
A mis-transcribed word or name changes what the model believes it heard, with nothing to correct it.
The mistake to watch for
Questions
- What is the difference between speech to text and text to speech?
- They convert in opposite directions. Speech to text turns the caller's spoken audio into text the language model can read. Text to speech turns the model's written reply into the audio the caller hears. A voice agent needs both working well.
- Why do voice agents struggle with names and addresses?
- Speech-to-text engines are generally trained on common vocabulary and struggle more with proper nouns that don't appear often in that training data, like unusual last names, street names or product names specific to one business. This is a common source of errors that a quiet, clear-speech demo will not reveal.
- Does background noise really affect accuracy that much?
- Yes. Real calls come in from cars, job sites, retail floors and speakerphones, and accuracy on that kind of audio is meaningfully lower than on a clean recording. Testing an agent only with clear audio hides this gap until it shows up on real calls.