What it is
A small language model (SLM) is a large language model that has been kept deliberately compact. It has fewer parameters, so it needs less memory and less computing power for each answer. There is no official cutoff. People usually use the label for models from a few hundred million up to roughly ten billion parameters, and for the cheap, fast tier that labs sell beside their flagship models.
How it works
A small model is built the same way as a big one: it is a transformer trained to predict the next token. The difference is size, and the tricks used to make that size work. Common ones are:
- Careful training data. Cleaner, more focused data lets a small model learn more per parameter.
- Distillation. A big "teacher" model produces answers, and the small "student" is trained to imitate them.
- Quantization. Storing the numbers inside the model at lower precision shrinks it again.
Why it matters to you
Smaller means cheaper and quicker. A small model answers with lower latency and costs a fraction as much per million tokens, and it can run on your own hardware or on a device, so private data never leaves your machine.
The trade is depth. A small model is usually weaker at long chains of reasoning, rare knowledge and messy, open-ended requests. It does well on work with a clear shape: sorting email into categories, pulling fields out of a document, rewriting text in a fixed style, or deciding which tool an agent should call next.
An example
Think of a kitchen. The head chef (a frontier model) designs the menu and handles the hard dishes. The line cook (a small model) makes the same fifty orders of fries all night, quickly and cheaply. You do not want the head chef frying potatoes, and you do not want the line cook inventing the tasting menu.
That is why teams often send easy, high-volume steps to a small model and keep the hard ones for a large one, a pattern called model routing. You test the small model on your own examples first, because a benchmark chart will not tell you whether it handles your data.
Small models are often open-weight models, which you can download and fine-tune cheaply for one job. At the other end of the scale sits the frontier model, the largest and most capable systems a lab offers.