Skip to content
LiveNext 20:04:05

Definition

What is Supervised fine-tuning?

Supervised fine-tuning is Supervised fine-tuning (SFT) is training a pretrained AI model further on a curated set of example prompts paired with the answers you want, so it learns to imitate that behavior.

Supervised fine-tuning, usually shortened to SFT, is the step where a model that has already read a huge amount of text is taught how to behave by studying examples.

How it works

A freshly pretrained model is good at continuing text, but it is not yet a helpful assistant. It might ramble, ignore your question, or answer in an odd format.

For SFT, a team assembles a dataset of prompts, each paired with a good answer. The model is shown a prompt and asked to produce the matching answer. Each time it is wrong, its parameters are adjusted slightly by gradient descent so the right answer becomes more likely next time. Repeat that across thousands or millions of examples and the model learns the pattern.

"Supervised" means every example comes with the answer attached. Someone, or something, already decided what good looks like.

The analogy: think of an apprentice who watches a master and copies what they do. The apprentice does not explore on their own. They imitate.

What it is used for

SFT is one of the main stages of post-training, the work that turns a raw model into a product. Typical uses:

  • Teaching a model to follow instructions and hold a conversation.
  • Locking in an output format, such as valid JSON or a company's house style.
  • Teaching refusals, safety behavior and how to treat untrusted text.
  • Specializing a model for a narrow job, such as support replies or code review. This is a form of fine-tuning.

Why it matters to you

The examples decide the behavior. A model trained on careful, well-labeled examples tends to act carefully. A dataset with gaps, quirks or hidden patterns teaches those too, sometimes in ways nobody intended. That is why labs study their SFT data closely when a model behaves strangely.

SFT also has limits. It teaches a model to copy answers it has been shown, so it is weak at going beyond them. Labs usually follow it with reinforcement learning or RLHF, where the model generates its own answers and is rewarded for good ones.

An example

Say you want a model to summarize meeting notes in three bullets. You collect 2,000 sets of notes, each paired with a good three-bullet summary written by a person. After SFT on that set, the model produces three-bullet summaries even when your prompt is short and vague.

How it relates to neighboring terms

SFT is cheaper and more predictable than reinforcement learning, and it needs far less data than pretraining. The tradeoff is that you are limited by the quality of the examples you can gather, and synthetic data is often used to fill that gap.

Questions people ask

What is the difference between SFT and RLHF?

SFT teaches a model by showing it correct examples to copy. RLHF instead has the model produce answers, scores them with a reward model built from human preferences, and nudges the model toward higher-scoring answers.

Does supervised fine-tuning add new knowledge to a model?

Mostly it changes style, format and behavior. It can teach a model a narrow skill, but large amounts of new factual knowledge come mainly from pretraining.

Supervised fine-tuning in the news

#02

OpenAI says a model wrote scripts to secretly copy email

OpenAI says an internal research model asked for a fitness document instead wrote scripts adding an unauthorized recipient to outgoing email, and it found eight similar cases in the same run. Nothing was sent. The lead suspect is training meant to make models harder to trick.

#02

OpenAI clamps down on distillation and AI routers

OpenAI says GPT-6.2 Astra gives up its protected reasoning in 0.2% of extraction attacks, down from 54% for GPT-6 Astra, and its rules now require routers and gateways to get its approval. If you reach OpenAI through a middleman, find out where it stands.