What is Speech recognition?

Speech recognition turns audio of someone talking into text. You see it in meeting transcripts, voice typing, captions, voicemail summaries and every voice assistant. It is also called speech-to-text, automatic speech recognition (ASR) or simply transcription.

How it works

Modern systems are neural networks trained on large collections of audio paired with correct transcripts. The model takes in sound, usually as a spectrogram (a picture of which frequencies are present over time), and predicts the words. Because it learns from many speakers, it can handle accents, speeds and background noise far better than older rule-based systems.

There are two common modes:

• Batch transcription. You upload a finished recording and get the full text back. Accuracy is best here because the model can use the whole recording as context.
• Streaming (realtime) transcription. Text appears while the person is still talking. It has to guess with less context, so it trades a bit of accuracy for low delay. Realtime matters for captions and voice assistants.

Why it matters to you

Speech recognition is the front door of any voice product. If it mishears a name, a number or a product term, everything after it, including the answer from a large language model, is built on a mistake. That is why people judge it by word error rate, the share of words transcribed wrongly, and by how it performs on their own audio, not on a vendor's demo.

An example

A sales team records customer calls. A speech recognition model produces the transcript, then a language model summarizes it and pulls out action items. The summary can only be as good as the transcript underneath it.

Things that trip it up

• Heavy accents, crosstalk and poor microphones.
• Jargon, product names and unusual proper names. Many services let you supply a custom vocabulary.
• Numbers, dates and spelled-out codes.
• Audio privacy: a recording is personal data, so check where it is processed and stored.

Related terms

Speech recognition is one half of voice AI. The other half, producing spoken audio from text, is text-to-speech. Put the two together with a model in the middle and you have a voice agent. Speech recognition also depends on inference speed, since realtime use needs results in a fraction of a second.