Structured outputs that don't break
Getting JSON out of a model is easy; getting JSON your code can trust on every call is a system. Schemas, provider structured-output modes versus prompting, validation with targeted retries, and rendering partial JSON while it streams.
The moment a model's output feeds code rather than a person, its format becomes a contract. An insurance intake system that reads a claim amount, a support bot that routes on a category field, an agent that calls a function with arguments: each one breaks, or worse, silently does the wrong thing, when the output is almost-but-not-quite right. This post builds the layers that make structured output dependable, with a complete standard-library example.
Layer 1: write the contract as a JSON Schema
Describe the output once, in JSON Schema, and use that single description everywhere: in the prompt, in the provider's structured-output mode, and in your validator. Our running example extracts a claim from an insurance intake note.
requiredfor every field your code reads. Missing fields are the most common failure.additionalProperties: false, so a misspelled key such asclaimTypefails loudly instead of being ignored.enumfor categories. A model will happily invent"car"when you meant"auto".- Normalised formats: ISO dates (
2026-09-03), numbers as numbers, currency as a separate ISO code. Ambiguity like03/09/2026is a bug waiting for a different country.
Layer 2: prompting versus provider modes
There are three ways to get JSON, in increasing strength.
| Approach | Guarantees | Still need |
|---|---|---|
| Prompt + example | Nothing; usually works | Parse, validate, retry |
| Tool / function call | Shape, often strict | Business-rule checks |
| Schema-constrained mode | Valid JSON, schema shape | Rule checks, truncation |
Prompting means describing the format and showing an example, which is few-shot prompting. It works most of the time with modern models, and “most of the time” is the problem: one malformed reply in a few hundred is still a daily incident at volume. Typical failures are a markdown fence around the JSON, a trailing explanation, a string where a number belongs, or a missing field.
Provider structured-output modes fix the syntax at the source. With constrained decoding, at each step the model is only allowed to pick tokens that keep the output valid against the schema: the scores of every other token are masked out. Willard and Louf (2023) describe how to do this efficiently by compiling the schema into a state machine. OpenAI exposes it as structured outputs with a strict JSON schema; Anthropic exposes it as output_config.format with a JSON schema, and as strict: true on tool definitions for tool calls.
Default: use the provider's schema mode wherever it is available, send it the structural part of the schema, and validate the full schema yourself. Fall back to prompting plus validation only for models without a schema mode.
Layer 3: validate, then retry with the exact errors
Validation turns “the model got it wrong” from a crash into a decision. The useful trick is what you do next: send the model its own answer and the specific errors, not the original prompt again. A model told $.claim_type: 'car' not one of ['auto', 'property', 'travel'] fixes that field; a model asked the same question again may make a different mistake.
Here is the whole loop in standard-library Python: a validator for the subset of JSON Schema used above, a parser that tolerates a markdown fence and nothing else, the retry loop, and a scripted fake model so it runs offline. Save it as structured.py and run python3 -I structured.py.
"""Validate model JSON against a schema, retry with the errors, and render partial streams.
Stdlib only: python3 -I structured.py"""
import json
import re
CLAIM_SCHEMA = {
"type": "object",
"required": ["policy_number", "incident_date", "claim_type", "amount", "injuries"],
"additionalProperties": False,
"properties": {
"policy_number": {"type": "string", "pattern": r"^POL-\d{6}$"},
"incident_date": {"type": "string", "pattern": r"^\d{4}-\d{2}-\d{2}$"},
"claim_type": {"enum": ["auto", "property", "travel"]},
"amount": {"type": "number", "minimum": 0},
"currency": {"type": "string", "pattern": r"^[A-Z]{3}$"},
"injuries": {"type": "boolean"},
"parties": {"type": "array", "items": {"type": "string"}},
},
}
TYPES = {"object": dict, "array": list, "string": str, "boolean": bool,
"number": (int, float), "integer": int}
def validate(value, schema, path="$"):
"""Return a list of human-readable errors (empty list = valid). A subset of JSON Schema."""
errors = []
expected = schema.get("type")
if expected:
ok = isinstance(value, TYPES[expected])
if expected in ("number", "integer") and isinstance(value, bool):
ok = False # in Python, True is an int; in JSON it is not a number
if not ok:
return [f"{path}: expected {expected}, got {type(value).__name__}"]
if "enum" in schema and value not in schema["enum"]:
errors.append(f"{path}: {value!r} not one of {schema['enum']}")
if "pattern" in schema and isinstance(value, str) and not re.search(schema["pattern"], value):
errors.append(f"{path}: {value!r} does not match {schema['pattern']}")
if "minimum" in schema and isinstance(value, (int, float)) and value < schema["minimum"]:
errors.append(f"{path}: {value} is below minimum {schema['minimum']}")
if isinstance(value, dict):
props = schema.get("properties", {})
for key in schema.get("required", []):
if key not in value:
errors.append(f"{path}: missing required field '{key}'")
for key, item in value.items():
if key in props:
errors.extend(validate(item, props[key], f"{path}.{key}"))
elif schema.get("additionalProperties") is False:
errors.append(f"{path}: unexpected field '{key}'")
if isinstance(value, list) and "items" in schema:
for i, item in enumerate(value):
errors.extend(validate(item, schema["items"], f"{path}[{i}]"))
return errors
def parse_json(text):
"""Accept bare JSON or JSON wrapped in a markdown fence; nothing cleverer."""
fenced = re.search(r"```(?:json)?\s*(.*?)```", text, re.S)
return json.loads(fenced.group(1) if fenced else text)
def extract(call_model, prompt, schema, max_attempts=3):
"""Ask, parse, validate; on failure send the exact errors back. Raises after max_attempts."""
messages = [prompt]
for attempt in range(1, max_attempts + 1):
raw = call_model("\n\n".join(messages))
try:
data = parse_json(raw)
errors = validate(data, schema)
except json.JSONDecodeError as exc:
errors = [f"not valid JSON: {exc.msg} at char {exc.pos}"]
if not errors:
return data, attempt
messages += [f"Your previous answer:\n{raw}",
"It failed validation:\n- " + "\n- ".join(errors) + "\nReturn corrected JSON only."]
raise ValueError(f"no valid output after {max_attempts} attempts: {errors}")
def _close(text):
"""Close an open string and any open brackets so the prefix can be parsed."""
stack, in_str, escaped = [], False, False
for ch in text:
if in_str:
if escaped:
escaped = False
elif ch == "\\":
escaped = True
elif ch == '"':
in_str = False
elif ch == '"':
in_str = True
elif ch in "{[":
stack.append("}" if ch == "{" else "]")
elif ch in "}]":
stack.pop()
return text + ('"' if in_str else "") + "".join(reversed(stack))
def parse_partial(text):
"""Best-effort parse of a truncated JSON stream: back off to the last comma until it parses."""
while text:
try:
return json.loads(_close(text))
except json.JSONDecodeError:
cut = text.rfind(",")
if cut == -1:
return None
text = text[:cut]
return None
if __name__ == "__main__":
scripted = iter([
'```json\n{"policy_number": "POL-48213", "incident_date": "03/09/2026", '
'"claim_type": "car", "amount": "2,400", "injuries": false}\n```',
'{"policy_number": "POL-482130", "incident_date": "2026-09-03", '
'"claim_type": "auto", "amount": 2400, "currency": "EUR", "injuries": false}',
])
seen = []
def fake_model(prompt):
seen.append(prompt)
return next(scripted)
data, attempts = extract(fake_model, "Extract the claim from this intake note: ...", CLAIM_SCHEMA)
print("first-attempt errors:")
print(seen[1].split("It failed validation:\n")[1].split("\nReturn")[0])
print("valid after", attempts, "attempts:", data)
stream = '{"policy_number": "POL-482130", "claim_type": "auto", "parties": ["Driver A", "Dri'
for cut in (30, 62, 75, len(stream)):
print(cut, parse_partial(stream[:cut]))Running it prints:
first-attempt errors:
- $.policy_number: 'POL-48213' does not match ^POL-\d{6}$
- $.incident_date: '03/09/2026' does not match ^\d{4}-\d{2}-\d{2}$
- $.claim_type: 'car' not one of ['auto', 'property', 'travel']
- $.amount: expected number, got str
valid after 2 attempts: {'policy_number': 'POL-482130', 'incident_date': '2026-09-03', 'claim_type': 'auto', 'amount': 2400, 'currency': 'EUR', 'injuries': False}
30 {'policy_number': 'POL-482130'}
62 {'policy_number': 'POL-482130', 'claim_type': 'auto'}
75 {'policy_number': 'POL-482130', 'claim_type': 'auto', 'parties': ['Driver A']}
82 {'policy_number': 'POL-482130', 'claim_type': 'auto', 'parties': ['Driver A', 'Dri']}Details that matter in that code:
boolis rejected as a number. In PythonTrueis anint; in JSON it is not. Without that line,"amount": truewould pass.- The parser is deliberately dumb. It strips one markdown fence. It does not try to repair broken JSON on a final answer, because repaired JSON can mean something the model did not say. Broken output goes back through the retry loop.
- All errors at once. The validator collects every problem in one pass, so one retry can fix four fields instead of needing four retries.
- A hard attempt budget. Three attempts, then raise. If a prompt needs more than one retry on more than a few percent of your golden set, fix the prompt or the schema; retries are a safety net, not a strategy.
Retries for format errors are different from retries for transport errors. Timeouts and rate limits deserve exponential backoff on the same request; validation failures deserve an immediate new request that includes the errors. Keep the two loops separate so each is easy to reason about.
Layer 4: the provider mode, with a backstop
With a schema mode, the retry loop rarely fires, but you still check two things: that the reply was not cut off, and that it passes the rules the provider did not enforce. This snippet needs ANTHROPIC_API_KEY and imports the schema and validator from structured.py.
# Needs ANTHROPIC_API_KEY. Provider-enforced schema: the reply is constrained to the schema's structure.
import json
import anthropic
from structured import CLAIM_SCHEMA, validate
def strip_keywords(schema, drop=("pattern", "minimum")):
"""Provider modes enforce a subset of JSON Schema; send the structure, check the rest locally."""
if isinstance(schema, dict):
return {k: strip_keywords(v, drop) for k, v in schema.items() if k not in drop}
return schema
client = anthropic.Anthropic(timeout=60.0, max_retries=2)
note = "Caller: policy POL-482130, rear-ended on 3 Sept 2026, repair quote EUR 2,400, nobody hurt."
response = client.messages.create(
model="claude-opus-5-5",
max_tokens=2048,
messages=[{"role": "user", "content": f"Extract the insurance claim from this intake note:\n{note}"}],
output_config={"format": {"type": "json_schema", "schema": strip_keywords(CLAIM_SCHEMA)}},
)
if response.stop_reason != "end_turn":
raise RuntimeError(f"incomplete output: {response.stop_reason}") # max_tokens, refusal
text = next(b.text for b in response.content if b.type == "text")
claim = json.loads(text)
assert validate(claim, CLAIM_SCHEMA) == [] # patterns and minimums are checked here
print(claim)The stop_reason check is not optional. A reply that hit max_tokens is truncated, and truncated JSON is the most common way schema-constrained output still fails to parse. Set max_tokens with headroom over your largest expected object.
Layer 5: rendering partial JSON while it streams
Streaming makes a long extraction feel fast, but halfway through a stream the JSON is not valid yet. To show fields as they arrive, close what is open and parse; if that fails because the cut landed mid-key or mid-value, back off to the last comma and try again. The parse_partial function in the example does exactly that, and the last four output lines show it filling in an object field by field.
Key takeaways
- Write the contract once as a JSON Schema with
required,enumandadditionalProperties: false, and use it in the prompt, the provider mode and your validator. - Provider schema modes mask invalid tokens so output always parses and has the right shape; they do not make it true and may not enforce every keyword.
- Validate every reply yourself; on failure, retry with the exact error list and a hard budget of about three attempts.
- Check for truncation (
max_tokens) before parsing; it is the most common way constrained output still breaks. - Parse partial JSON for display while streaming, and act only on the final validated object.
Sources
- Efficient Guided Generation for Large Language Models — Willard & Louf, 2023paper
- JSON Schema specification (draft 2020-12) — JSON Schema, 2020spec
- Structured Outputs guide — OpenAI, 2024docs