Definition
Distillation is Distillation is a training technique where a smaller "student" AI model learns to imitate the outputs of a larger "teacher" model, keeping much of its ability at a fraction of the size and cost.
Distillation is how you teach a small model by letting it study a big one. Instead of learning only from raw data, the small model (the student) is trained to reproduce the answers of a larger, more capable model (the teacher).
Normally a model learns from text with a single right answer: the next word. In distillation, the student also sees what the teacher produced for the same input. The teacher's output carries more information than a bare right answer. It shows how confident the teacher was in each option, or, for a reasoning model, the whole chain of thought it wrote on the way to a conclusion.
The idea was laid out in a 2015 paper by Geoffrey Hinton, Oriol Vinyals and Jeff Dean, "Distilling the Knowledge in a Neural Network." Since then it has become a standard way to build compact models.
Think of an apprentice who copies a master's finished work and notes, rather than learning the trade from scratch. The apprentice gets good faster, though rarely better than the master.
A big model is expensive to run. Every answer costs inference compute. A distilled student can be a fraction of the size, answer faster and cost less per token, which is why many small, cheap models in a lab's lineup are distilled from its flagship. It is often combined with fine-tuning and other post-training steps.
Distillation does not have to happen inside one company. Anyone with access to a model's outputs, for example through an API, can collect them and train another model on them. Many AI providers forbid this in their terms of service when the goal is to build a competing model. Labs now treat large-scale extraction of their models' outputs as a security and business problem, and they try to detect and block it. Whether and when it crosses a legal line is a matter for contracts and courts, not for the technique itself.
Distillation is different from fine-tuning, which adapts one model to new examples, though a distillation run is often a form of it. It is also different from shrinking a model by other means. The size of the student is usually described in parameters.
It is training a smaller student model to imitate the outputs of a larger teacher model, so the student keeps much of the teacher's ability while being cheaper and faster to run.
The technique is legal and widely used internally. Training on a rival's outputs often breaks the provider's terms of service, and the legal questions depend on contracts and jurisdiction.
Anthropic says new defenses cut the Claude reasoning data distillers harvest by about 90 to 99%, and it has banned tens of thousands of accounts. The fine print reaches honest developers too: new API accounts can't edit Claude's prior context, and blocked requests are billed.