Skip to content
LiveNext 10:46:08

Definition

What is Structured output?

Structured output is Structured output is a model feature that forces an AI's reply to follow a fixed format, usually JSON that matches a schema you supply, so software can read it without guessing.

What it is

By default a language model writes free-form text. That is fine for a person and awkward for a program. If your code needs a field called "category" with one of four allowed values, a paragraph of prose is a problem.

Structured output fixes this. You give the model a schema, a description of the exact shape the reply must take: which fields, what type each one is, which values are allowed. The model's reply then comes back as data, most often JSON, that matches it.

Picture a paper form instead of a blank page. The model still decides what to write, but only in the boxes you drew.

How it works

There are two broad approaches:

  • Prompting and parsing. You ask nicely for JSON in the prompt, then check the reply and retry when it is malformed. This works most of the time and fails some of the time.
  • Constrained decoding. The provider restricts which tokens the model may pick at each step so the reply cannot break the schema. The model generates one token at a time, and the system simply removes choices that would make the output invalid. This is a form of controlling how the model samples, related to ideas like temperature and top-p sampling.

Many providers offer this as a setting on their API, usually by accepting a JSON schema alongside the prompt. Details vary by provider, so check the documentation of the one you use.

What it does and does not guarantee

A matching shape is not a correct answer. Structured output guarantees the reply parses and has the right fields. It does not guarantee the values are true. A model can return a perfectly formatted, perfectly wrong answer, which is a hallucination in a tidy box.

It also does not guarantee the model understood the schema's intent, so give fields clear names and short descriptions.

Why it matters

  • Reliability. Your code stops breaking on stray commentary like "Sure! Here is the JSON."
  • Tool calls. Tool use depends on it: when an agent calls a function, the arguments have to arrive in a shape the function accepts.
  • Pipelines. Classification, extraction and routing jobs become simple data steps. See model routing.

Practical tips

  • Keep schemas small. Fewer fields means fewer chances to go wrong.
  • Use enumerated values for choices instead of free text.
  • Add an "unknown" option so the model is not forced to invent an answer.
  • Validate the values in your own code anyway.

The short version

Structured output turns a model's reply into data your software can trust to be shaped correctly. Trust the shape. Check the contents.

Questions people ask

What is structured output in AI?

It is a feature that makes a model's reply follow a schema you provide, typically JSON, so your code can parse it reliably.

Does structured output stop hallucinations?

No. It guarantees the format, not the facts. A reply can match the schema and still contain wrong values.

Structured output in the news

#01

Microsoft's Decision-1 makes the yes/no call cheap

Microsoft released Decision-1, a Qwen-based model that scores fixed choices, at $0.042 per million input tokens with free output. The benchmarks are Microsoft's own. The price is the reason to move your agent's small calls off big models.

#04

OpenAI's Decisions API goes GA with 400 questions a call

OpenAI's Decisions API left beta on Oct. 22, now taking up to 400 typed questions per request, up from 200, at $0.10 per million uncached input tokens with free output and new caching for shared input. The badge is minor. The doubled question count is what cuts your bill.