What it is
By default a language model writes free-form text. That is fine for a person and awkward for a program. If your code needs a field called "category" with one of four allowed values, a paragraph of prose is a problem.
Structured output fixes this. You give the model a schema, a description of the exact shape the reply must take: which fields, what type each one is, which values are allowed. The model's reply then comes back as data, most often JSON, that matches it.
Picture a paper form instead of a blank page. The model still decides what to write, but only in the boxes you drew.
How it works
There are two broad approaches:
- Prompting and parsing. You ask nicely for JSON in the prompt, then check the reply and retry when it is malformed. This works most of the time and fails some of the time.
- Constrained decoding. The provider restricts which tokens the model may pick at each step so the reply cannot break the schema. The model generates one token at a time, and the system simply removes choices that would make the output invalid. This is a form of controlling how the model samples, related to ideas like temperature and top-p sampling.
Many providers offer this as a setting on their API, usually by accepting a JSON schema alongside the prompt. Details vary by provider, so check the documentation of the one you use.
What it does and does not guarantee
A matching shape is not a correct answer. Structured output guarantees the reply parses and has the right fields. It does not guarantee the values are true. A model can return a perfectly formatted, perfectly wrong answer, which is a hallucination in a tidy box.
It also does not guarantee the model understood the schema's intent, so give fields clear names and short descriptions.
Why it matters
- Reliability. Your code stops breaking on stray commentary like "Sure! Here is the JSON."
- Tool calls. Tool use depends on it: when an agent calls a function, the arguments have to arrive in a shape the function accepts.
- Pipelines. Classification, extraction and routing jobs become simple data steps. See model routing.
Practical tips
- Keep schemas small. Fewer fields means fewer chances to go wrong.
- Use enumerated values for choices instead of free text.
- Add an "unknown" option so the model is not forced to invent an answer.
- Validate the values in your own code anyway.
The short version
Structured output turns a model's reply into data your software can trust to be shaped correctly. Trust the shape. Check the contents.