Definition
Speaker diarization is Speaker diarization is the process of working out who spoke when in an audio recording, labeling each stretch of speech as Speaker 1, Speaker 2 and so on, usually without knowing anyone's name.
Speaker diarization answers the question "who spoke when?" Given a recording with several voices, such as a meeting, an interview or a customer call, it splits the audio into segments and tags each one with a speaker label like Speaker 1 or Speaker 2. Paired with speech recognition, it turns a wall of text into a transcript you can follow as a conversation.
It does not usually say who the speakers are. Matching a voice to a named person is a different task called speaker identification, which needs a reference sample of that person's voice.
A typical pipeline has a few stages:
Newer systems often fold several of these steps into one neural model, and some do diarization and transcription together.
Accuracy drops with overlapping speech, quick back-and-forth, similar-sounding voices, distant microphones, background noise and recordings with many speakers. Accuracy is commonly reported as diarization error rate, which counts missed speech, false alarms and speakers assigned to the wrong label, and it varies a lot by test audio, much like word error rate for transcription.
Diarization separates speakers and labels them generically, like Speaker 1. Identification matches a voice to a known, named person using a reference sample.
Some systems do it live and some only after a recording ends. Live diarization is generally harder and less accurate.
Mistral released Voxtral 3 on Oct. 20: new transcription and speech models plus a voice agent API at $0.04 a minute when it reasons with Mistral Small 4. Skip the benchmark charts. The per-minute price and the choice of where audio is processed are the story.