OpenAI: Structured Outputs That Do Not Drift

Structured output is the difference between a feature and a demo. A json_schema response format with strict: true makes the model adhere to your schema. JSON mode (json_object) only promises valid JSON, a much weaker promise than most people read it as.

Strict mode removes a whole class of parsing bugs on day one. It introduces two quieter ones, both from the same place: a shape guarantee is not a content guarantee.

Why this exists

The first quiet bug is deleted error handling. Once the JSON always parses, the code that coped with malformed output gets removed and validation stops at “did it parse.” But strict mode guarantees shape and says nothing about whether the values are true. A schema-valid invoice dated 1970-01-01 with a total of zero passes every check you have left.

The second is absence. In strict mode every property has to appear in required. There is no optional. The documented way to express one is a union type that includes null — which means your schema has no “missing.” It has “present and null.”

That sounds like a technicality. It is the source of the worst reporting bug in these systems, because a bare null collapses three different situations into one value: the field was not in the document, the field does not apply to this document type, and the model could not read it. Downstream somebody counts nulls and calls it a data quality number. It is three numbers added together, and you cannot tell which one is growing.

How to use it

Put a reason next to every nullable value. Never ship a nullable field alone. Ship the value plus a short closed enum saying why it is null: found, absent_in_source, not_applicable, unreadable. Three distinguishable failures instead of one ambiguous one, and the enum is countable, so it can drive a triage queue.

Set additionalProperties: false on nested objects, not just the root. Nested objects are where surprise keys appear.

Branch on refusal. The response carries a refusal field when the model declines. Code that only reads parsed content treats a refusal as an empty result — the same silent zero as a failed extraction, from a different cause.

Version the schema inside the payload. One field, schema_version, written by your code and not by the model. When the schema changes you can tell which rows came from which shape without matching deploy timestamps against row timestamps.

Make every change additive. Add fields; never repurpose a name. Redefining what status means is how a report is wrong for a month before anyone notices, because nothing errors.

Keep enums short and closed, with an other member and a free-text sibling. A long enum is a taxonomy you are asking the model to guess. A short one with an escape hatch tells you what the taxonomy is missing, because you can read what landed in the sibling.

Validate meaning after parsing. Ranges, cross-field consistency, and one rule most schemas omit: if the model returns a quoted span, assert that the span actually appears in the source. That check catches more real extraction errors than the rest of your validation together.

Where it goes wrong

Strict schemas make a bad task look tidy. If the job is genuinely ambiguous — two plausible totals on a page, a category a careful person would argue about — the schema does not resolve it. It forces a guess and formats it neatly. Ambiguity needs a field that can say “ambiguous,” not a cleaner container.

Deep nesting is a false economy. One call producing a five-level tree is harder to debug, harder to evaluate, and fails whole. Two flatter calls usually cost less than the retries they save. There are published caps on nesting depth, property counts and enum sizes; read them when you design and again when you add a field, not when an error finds them for you.

And strict mode is not an evaluation set. Shape conformance is the easiest property to test and the least informative. You still need documents with known correct answers and a number that moves when quality does.