Short version for the impatient: strict: true guarantees the shape of your JSON. It guarantees nothing about whether the values inside it are right. If you already knew that, skip ahead to the bit about optional fields, because that’s where I actually lost a day.
Here’s the setup. I had an extraction endpoint pulling line items out of scanned supplier invoices. Schema locked down, strict: true, Pydantic model on the other end. It had been quiet for weeks. Then finance flagged an invoice where the tax amount came back as 0.0 on a document that clearly showed 5% VAT.
My first instinct was that the API had regressed. It hadn’t. The response was perfectly schema-valid. It was just wrong. Constrained decoding had done exactly what it promised and nothing more, and I’d quietly started treating “parses cleanly” as “is correct” somewhere around week three.
Strict mode is a grammar, not a fact checker
The mechanism is worth understanding, because it explains precisely which class of bugs this feature kills and which it leaves alone.
OpenAI compiles your JSON Schema into a context-free grammar, then masks the sampler at every step so only tokens valid under that grammar can be emitted. Their announcement post on Structured Outputs walks through this: once the model has produced {"val, an opening brace is no longer a legal next token, so its probability gets zeroed. That’s also why the first request against a brand new schema is slow. The grammar has to be preprocessed and cached, and OpenAI says complex schemas can take up to a minute the first time.
So the guarantee is real, and it’s a token-level guarantee. Braces balance. Enums stay inside their allowed values. Required keys show up. What no grammar can check is whether 5.0 should have been 12.50. OpenAI is direct about this in their own limitations section: structured outputs “doesn’t prevent all kinds of model mistakes,” and they specifically call out wrong values inside a well-formed object.
I find it useful to think of it as the difference between a type system and a test suite. Nobody ships a TypeScript codebase and declares it correct because it compiles. Same energy here. I made a similar mistake in the other direction once, stuffing so much context in that the model started confidently misreading it, which I wrote up in the anti-hallucination prompt that backfired on me.
The before and after is genuinely dramatic
Credit where it’s due. The old pattern was miserable. This is roughly what my invoice extractor looked like in 2023, on JSON mode, which guaranteed valid JSON and absolutely nothing about the schema:
import json
SYSTEM = """Return ONLY valid JSON matching this shape:
{"vendor": str, "total": float, "tax": float, "currency": "AED"|"USD"}
Do not include markdown fences. Do not explain."""
def extract(text, attempts=4):
for i in range(attempts):
r = client.chat.completions.create(
model="gpt-4-0613",
response_format={"type": "json_object"},
messages=[
{"role": "system", "content": SYSTEM},
{"role": "user", "content": text},
],
)
raw = r.choices[0].message.content
try:
data = json.loads(raw)
except json.JSONDecodeError:
continue
# JSON mode gave us valid JSON. It did not give us OUR JSON.
if not {"vendor", "total", "tax", "currency"} <= data.keys():
continue
if data["currency"] not in ("AED", "USD"):
continue
return data
raise RuntimeError(f"gave up after {attempts} attempts")
Four attempts. Manual key checking. An enum I had to police by hand. And a retry loop that burned tokens every time the model decided "amount_total" was a nicer key name than "total".
The current version:
from typing import Literal
from pydantic import BaseModel
class Invoice(BaseModel):
vendor: str
total: float
tax: float
currency: Literal["AED", "USD"]
completion = client.chat.completions.parse(
model="gpt-4.1",
messages=[
{"role": "system", "content": "Extract the invoice fields."},
{"role": "user", "content": text},
],
response_format=Invoice,
)
invoice = completion.choices[0].message.parsed
The retry loop is gone. The key checking is gone. The enum policing is gone. That’s a real improvement and I’m not going to pretend otherwise. The SDK converts the Pydantic model to a schema, sets strict: true, and deserializes the response back into a typed object.
What survived, unchanged, is the possibility that tax is 0.0 when it should be 12.50.
Every field is required, and that’s where I lost the day
This is the rule that produced my actual bug, and it’s the one I’d tattoo on a junior dev if they let me.
In strict mode, every property you declare must appear in required. Not “should.” Must. You also need additionalProperties: false on every object in the schema, no exceptions. Microsoft’s Azure OpenAI structured outputs docs spell out both rules, and they mirror OpenAI’s own subset exactly.
So how do you express a field that genuinely might not exist on the document? You don’t make it optional. You union it with null:
{
"type": "object",
"properties": {
"vendor": { "type": "string" },
"total": { "type": "number" },
"tax": { "type": ["number", "null"] },
"po_number":{ "type": ["string", "null"] }
},
"required": ["vendor", "total", "tax", "po_number"],
"additionalProperties": false
}
Every key is required. Two of them are allowed to be null. Fine so far.
Now read that schema the way the model reads it. tax is a number-or-null and it is mandatory. The model must emit something in that slot on every single document. When the tax line is smudged, or written in a layout the model hasn’t seen, or expressed as “VAT included,” the grammar offers exactly two escape hatches: a number, or null. There is no “I couldn’t tell.” So it picks one. And a plausible number often beats null on the model’s own priors, because most invoices do have a tax amount.
That’s my 0.0. Not a hallucination in the dramatic sense. A forced choice under a constraint I built.
What fixed it was making “unknown” a first-class value instead of an absence:
from typing import Literal, Union
from pydantic import BaseModel
class Known(BaseModel):
status: Literal["found"]
value: float
source_text: str # verbatim span it read the number from
class Unknown(BaseModel):
status: Literal["not_found"]
reason: str
class Invoice(BaseModel):
vendor: str
total: float
tax: Union[Known, Unknown]
Two changes doing the work. not_found is now a legal, cheap answer, so the model stops guessing to satisfy the grammar. And source_text forces it to quote the span it read the number from, which means I can grep the OCR text for that string and reject the row when it isn’t there. Schema compliance I get for free. Groundedness I have to design in.
One caveat: the root object can’t be an anyOf, so this union pattern works on nested fields, not at the top level.
The keywords your schema is quietly ignoring
Here’s the part that stings if you came from ordinary JSON Schema validation, where pattern and minimum actually mean something. In strict mode a large chunk of the vocabulary is accepted and then ignored.
Per the Azure docs, the ignored list includes minLength, maxLength, pattern and format on strings; minimum, maximum and multipleOf on numbers; minItems, maxItems and uniqueItems on arrays. Objects lose patternProperties, propertyNames, minProperties and maxProperties. There’s also a ceiling of 100 object properties total and five levels of nesting. Recursion via $ref and shared definitions in $defs are both supported, which surprised me the first time I tried it.
Which means this schema is lying to you:
{
"email": { "type": "string", "format": "email" },
"iban": { "type": "string", "pattern": "^AE[0-9]{21}$" },
"rating": { "type": "integer", "minimum": 1, "maximum": 5 }
}
You’ll get a string in email. It may not contain an @. You’ll get an integer in rating. It may be 47.
Two things help. Push what you can into enum, which is enforced, so rating becomes "type": "integer", "enum": [1,2,3,4,5]. Then validate the rest after the fact, in your own code, where the constraint actually runs:
from pydantic import BaseModel, EmailStr, Field, field_validator
class Contact(BaseModel):
email: EmailStr
iban: str = Field(pattern=r"^AE[0-9]{21}$")
@field_validator("iban")
@classmethod
def check_mod97(cls, v: str) -> str:
# the model can produce a well-formed IBAN that isn't a real one
if not iban_mod97_valid(v):
raise ValueError("checksum failed")
return v
Note that Pydantic will happily send pattern up in the generated schema, where it does nothing, and then enforce it locally on the way back, where it does. That asymmetry caught me out for a while.
Three ways you still get something that isn’t your schema
The grammar holds right up until it doesn’t, and there are exactly three documented exits.
The model can refuse. Safety refusals bypass the schema entirely and come back on a separate refusal field on the message. If you’re reading .parsed without checking .refusal first, you’ll get a None and a confusing traceback three functions later.
The generation can hit max_tokens mid-object. You get truncated, invalid JSON. Always check finish_reason == "stop" before you trust the payload, especially on schemas with unbounded arrays, since maxItems isn’t enforced and nothing stops the model from emitting four hundred line items.
And parallel tool calls break it. Structured outputs aren’t compatible with parallel function calling. Set parallel_tool_calls: false when you’re using strict: true on tools.
msg = completion.choices[0].message
if msg.refusal:
raise ModelRefused(msg.refusal)
if completion.choices[0].finish_reason != "stop":
raise Truncated(completion.choices[0].finish_reason)
invoice = msg.parsed
Nine lines. I’ve now added them to every extraction path I own, including the ones in the document pipelines I build for clients.
What I’d do this week
Pick your highest-volume structured call and open its schema. Find every field typed as ["something", "null"] and ask what the model does when it can’t tell. If null is the only alternative to a value, you’re pushing it toward a guess. Add an explicit not_found branch and a source_text field, then sample fifty recent responses and check how often that quoted span actually appears in your input. My hit rate was 91%. I’d assumed 100%, which was the whole problem.
The parsing problem is solved. I’d started acting like the extraction problem was solved too, and those aren’t the same thing. Same lesson I keep relearning every time a framework makes something easy, most recently when AI SDK 7 let me delete my own agent loop and I briefly forgot I still had to think about what the loop was doing.
If you want the full list of what the subset does and doesn’t accept, OpenAI’s structured outputs guide is the reference, and it’s worth twenty minutes before you design a schema rather than after.
