Skip to content
LiveNext 10:46:07

Definition

What is Small language model?

Small language model is a small language model is an AI language model with far fewer parameters than a frontier model, built to run faster and cheaper on narrow, repetitive tasks and sometimes on a laptop or phone.

What it is

A small language model (SLM) is a large language model that has been kept deliberately compact. It has fewer parameters, so it needs less memory and less computing power for each answer. There is no official cutoff. People usually use the label for models from a few hundred million up to roughly ten billion parameters, and for the cheap, fast tier that labs sell beside their flagship models.

How it works

A small model is built the same way as a big one: it is a transformer trained to predict the next token. The difference is size, and the tricks used to make that size work. Common ones are:

  • Careful training data. Cleaner, more focused data lets a small model learn more per parameter.
  • Distillation. A big "teacher" model produces answers, and the small "student" is trained to imitate them.
  • Quantization. Storing the numbers inside the model at lower precision shrinks it again.

Why it matters to you

Smaller means cheaper and quicker. A small model answers with lower latency and costs a fraction as much per million tokens, and it can run on your own hardware or on a device, so private data never leaves your machine.

The trade is depth. A small model is usually weaker at long chains of reasoning, rare knowledge and messy, open-ended requests. It does well on work with a clear shape: sorting email into categories, pulling fields out of a document, rewriting text in a fixed style, or deciding which tool an agent should call next.

An example

Think of a kitchen. The head chef (a frontier model) designs the menu and handles the hard dishes. The line cook (a small model) makes the same fifty orders of fries all night, quickly and cheaply. You do not want the head chef frying potatoes, and you do not want the line cook inventing the tasting menu.

That is why teams often send easy, high-volume steps to a small model and keep the hard ones for a large one, a pattern called model routing. You test the small model on your own examples first, because a benchmark chart will not tell you whether it handles your data.

Small models are often open-weight models, which you can download and fine-tune cheaply for one job. At the other end of the scale sits the frontier model, the largest and most capable systems a lab offers.

Questions people ask

What is the difference between a small and a large language model?

A small language model has far fewer parameters, so it runs faster and costs less, but it is usually weaker on hard reasoning and rare knowledge. A large model is the reverse.

Can a small language model run on a laptop or phone?

Often yes. Models in the low billions of parameters, especially quantized ones, can run on consumer hardware, which keeps your data on the device.

Small language model in the news

#03

Mistral prices a full voice agent at $0.04 a minute

Mistral released Voxtral 3 on Oct. 20: new transcription and speech models plus a voice agent API at $0.04 a minute when it reasons with Mistral Small 4. Skip the benchmark charts. The per-minute price and the choice of where audio is processed are the story.

#01

Treasury proposes banning US investment in China AI

Treasury's Oct. 22 draft rule would bar US investment in the development of any AI system by companies tied to China and other countries of concern, agent wrappers included. The compute threshold is gone, and so is the safe harbor for small AI deals.