Structured Outputs That Survive a Provider Swap

What is solved, and what is not

As of early 2026, OpenAI, Anthropic and Google Gemini all support native structured output. The ecosystem has converged on the capability, and the argument that dominated 2023 and 2024 — whether a language model can be made to emit parseable JSON reliably — is finished. On a single provider, with constrained decoding switched on, schema compliance at the syntactic level is effectively guaranteed. Not likely. Guaranteed, by construction, for the reasons set out in the next section.

That is a genuine and underappreciated win. It is also, in most production systems, about a third of the problem. Two things remain unsolved, and both of them are the sort of problem that shows up long after the demo.

The first is portability. A schema is not a neutral artefact that every provider reads the same way. As of 2026, Anthropic does not support some numeric and length bound keywords; OpenAI restricts union types and imposes hard ceilings on schema size and nesting; Gemini has limits that are not fully documented. The consequence is blunt: the same Zod or Pydantic model can be entirely legal on the provider you built against and rejected outright by the provider you migrate to. Nothing in your code changed. The schema is the same file. The request fails.

The second is semantics. Syntactic validity is not semantic usability. A response can satisfy your schema perfectly and still be unusable: an empty string in a field you required, a plausible-looking enum member that the source document never supported, a date in exactly the right format and exactly the wrong value. There is 2026 research on precisely this gap — When Correct Isn't Usable: Improving Structured Output Reliability in Small Language Models (arXiv:2605.02363) — and the finding matches what anyone running an extraction pipeline at volume will already have noticed: your schema pass rate went to essentially one hundred per cent and your incident rate did not go to zero.

What follows is the design discipline for both. In outline:

• Constrained decoding solves syntax on one provider. It does not solve portability or meaning.
• The production architecture is three layers: constrained decoding, boundary validation, and a typed retry.
• Design your schemas to the intersection subset of every provider you might ever ship against.
• Express the constraints the intersection cannot carry in the validator, not in the schema.
• Pin provider behaviour with contract tests that run on a schedule, because the limits drift.
• Add semantic assertions, because a schema cannot tell you a valid value is a wrong one.

One note on dates before we start. Every provider-specific statement in this guide is qualified with a year for a reason. Provider limits are moving targets: keywords get added, ceilings get raised, restrictions get relaxed and occasionally tightened. If you read this in 2028, the discipline will still hold and some of the specifics will not. That is exactly why the contract tests in section six matter more than the table in section four — the table tells you where things stood, the tests tell you where they stand.

How constrained decoding actually works

It is worth understanding the mechanism, because two important consequences fall straight out of it and both get missed by teams who treat structured output as a flag they switch on.

At each generation step, the model produces a distribution over the whole vocabulary. Constrained — or grammar-constrained — decoding inserts itself between that distribution and the sampler. The inference engine compiles your schema into a finite state machine, tracks which state the partially generated output is in, and masks out every token that would move the output into an invalid state. If the grammar says the next character must be a quotation mark, every token that does not begin with a quotation mark has its probability driven to zero before sampling. The model does not decide to comply. It physically cannot emit a token that violates the schema.

That mechanism has two consequences you should design around.

Consequence one: syntactic compliance is mechanical, not statistical. You do not need a retry loop for malformed JSON on a provider path with constrained decoding enabled, and you should not write one. Retries there are dead code that will confuse whoever reads the module in a year. The parse-failure retry belongs on the paths where constrained decoding is not available — an older endpoint, an open-weight model you serve yourself without a grammar engine, or a provider that rejected your schema and fell back to free text.

Consequence two: your schema is now part of your latency and cost surface. The grammar has to be compiled before generation can start, and that compilation is proportional to how complicated the schema is. A large schema with deep nesting and many alternatives is not free. It also occupies space in the request itself, which matters if you are already fighting for context — the same discipline that applies to bounding agent tool output in a cache-aware way applies to the schema you send with the call. Most engines cache compiled grammars, so a stable schema reused across thousands of calls amortises well and a schema generated per request does not. If you are building the schema dynamically, hash it and reuse it rather than constructing a fresh object every time.

There is a third consequence that is more subtle and that section nine returns to. Masking constrains form, not content. The decoder's job is to guide the model into producing something that fits, and if the source material does not contain a value for a required field, something is exactly what you will get.

For local and open-weight deployments the same machinery is available outside the big three: Outlines is the best-known library for grammar-constrained decoding against models you host yourself, and it makes the mechanism explicit in a way the hosted APIs deliberately hide. If you want to understand what your provider is doing, reading how a local constrained decoder works is a productive afternoon.

The three layers you actually need

The recommended production architecture has three layers: parameter validation, failure retry, and constrained decoding. They are usually described in that order and they run in the reverse of it, which is part of why teams implement one or two and call it done.

Constrained decoding sits at the model call and buys you syntax. Parameter validation sits at the boundary where the response enters your system and buys you your actual contract — the empty-string rule, the range, the pattern, the cross-field consistency, everything the provider either would not accept in the schema or has no way to express. The typed retry wraps both and decides, per failure class, whether to try again, to escalate, or to stop.

Skipping the middle layer is the common error, and the reasoning behind it is superficially sound: the provider guaranteed the schema, so why validate? Because the provider guaranteed a schema — the reduced one you were able to send it — and because your guarantee should not evaporate the day you route traffic to a second provider whose guarantees are shaped differently. Keeping the validator authoritative means a migration changes what you send and never changes what you trust.

# Three layers, in the order they execute.
#   1  constrained decoding at the model call   -> syntax
#   2  Pydantic validation at the boundary      -> your contract
#   3  a typed retry wrapped around both        -> recovery

from enum import Enum
from typing import Optional
from pydantic import BaseModel, ValidationError


class Currency(str, Enum):
    GBP = "GBP"
    INR = "INR"
    EUR = "EUR"
    USD = "USD"


class WireInvoice(BaseModel):
    """What we send to the provider. Deliberately plain: no regex,
    no bounds, no unions. Every provider we ship against accepts it."""
    supplier_name: str
    tax_id: str
    invoice_number: str
    currency: Currency
    total_minor_units: int      # 12345 means 123.45. Never a float.
    issued_on: str              # loose string here, ISO date enforced below
    due_on: Optional[str] = None
    line_count: int


PROMPT = (
    "Extract the invoice fields from the document below.\n"
    "If a value is not present in the document, return null for that "
    "field rather than guessing.\n\n"
    "{feedback}"
    "DOCUMENT:\n{document}\n"
)


def extract_invoice(document: str, max_attempts: int = 3) -> "StrictInvoice":
    schema = portable_schema(WireInvoice)     # see the next section
    feedback = ""
    last_error: Optional[Exception] = None

    for attempt in range(1, max_attempts + 1):
        # Layer 1 — the provider adapter sets its own structured-output
        # parameter and returns the raw JSON text.
        raw = call_model(
            prompt=PROMPT.format(document=document, feedback=feedback),
            json_schema=schema,
        )
        try:
            # Layer 2 — the contract, enforced by us, not by them.
            return StrictInvoice.model_validate_json(raw)
        except ValidationError as exc:
            # Layer 3 — typed retry. Section eight decides the branch.
            last_error = exc
            feedback = describe_violations(exc) + "\n"

    raise ExtractionFailed(f"{max_attempts} attempts exhausted") from last_error

In TypeScript the shape is identical: Zod replaces Pydantic, schema.parse() or schema.safeParse() replaces model_validate_json, and a JSON Schema emitter replaces model_json_schema(). On the Python side, Instructor is the widely used wrapper that ties the Pydantic model to the provider call and handles the retry plumbing for you; if you use it, understand what it is doing rather than treating it as magic, because the portability decisions in the next section are still yours to make.

The portability problem

This is the centre of the article. The industry solved constrained decoding provider by provider and did not standardise what a schema may contain, so JSON Schema support is a ragged union rather than a common floor. Each provider implements a subset, documents part of it, and enforces the rest at request time.

As of 2026 the divergences that bite most often are these. Anthropic does not support some numeric and length bound keywords, so a schema that expresses minimum, maximum or a string length constraint may not survive the trip. OpenAI restricts union types — an anyOf that models "either a person or a company" is a natural way to think and a fragile way to send — and imposes hard ceilings on schema size and nesting depth; check the current limit in the provider's documentation rather than trusting a number you read anywhere, including here. Gemini has limits that are not fully documented, which is its own category of problem: you discover them empirically, in production, unless you go looking first.

The table below is a working map, valid as of 2026 and written to be superseded. Where the guidance says "verify in the provider's docs", that is not evasion — it means the behaviour is either undocumented, inconsistent between providers, or the sort of thing that changes between two model releases. Treat the third column as the durable part.

Schema feature	Status across providers, as of 2026	What to do
Object with named string, number and boolean properties	Safe. The universal core of every implementation.	Build everything out of this.
Required fields	Safe. Declaring a field required is supported everywhere.	But read section nine first — required is not free.
Optional fields	Verify in the provider's docs. Providers differ in how optionality is expressed and what a missing key means.	Prefer an explicitly nullable field over an absent key, and normalise both in the validator.
String enums	Safe as a mechanism. The member chosen is not guaranteed to be the right one.	Use freely; add a semantic check that the member is evidenced by the source.
Union types / anyOf / oneOf	Risky. OpenAI restricts union types as of 2026.	Flatten to one object with a discriminator enum plus nullable branch fields.
Nested objects, one or two levels	Safe in practice, within the provider's overall size ceiling.	Keep the nesting shallow and deliberate.
Deep nesting	Risky. OpenAI imposes hard ceilings on nesting; Gemini's limits are not fully documented.	Flatten. Check the current limit in each provider's docs and encode it as a test.
Very wide objects (many properties)	Risky. Hard ceilings exist on total schema size.	Split into two calls, or extract a list of typed records instead of one giant object.
Arrays of a single item type	Safe.	Use freely.
Array item-count bounds (minItems, maxItems)	Verify in the provider's docs. Not reliably honoured everywhere.	Bound in the validator; mention the expected count in the field description.
Numeric bounds (minimum, maximum, exclusiveMinimum)	Risky. Anthropic does not support some numeric bound keywords as of 2026.	Strip from the wire schema. Enforce in Pydantic or Zod.
String length bounds (minLength, maxLength)	Risky. Anthropic does not support some length bound keywords as of 2026.	Strip from the wire schema. Enforce in the validator.
String pattern (regex)	Verify in the provider's docs. Support and regex dialect both vary.	Never rely on it. Send a plain string; enforce the pattern at the boundary.
String format (date-time, email, uri)	Verify in the provider's docs. Frequently accepted and silently ignored.	Treat as a hint to the model, never as a guarantee. Parse and validate yourself.
additionalProperties: false	Verify in the provider's docs. Some require it, some ignore it.	Set it if the provider wants it; enforce strictness in the validator regardless.
Recursive or self-referencing schemas	Risky. Support is inconsistent and interacts badly with nesting ceilings.	Model as a flat list of nodes with parent identifiers, and rebuild the tree yourself.
$ref and $defs indirection	Verify in the provider's docs. Emitted by default by most schema generators.	Inline the definitions before sending if any target provider is unhappy with them.
Property description strings	Safe, and the most underused portable feature there is.	Write them properly. They are prompt surface, not documentation.

That last row deserves emphasis because it is where the portable subset gives something back. Everything the schema cannot enforce, the description can suggest — units, expected format, what to do when the value is absent, what the field is definitely not for. This is the same insight that makes tool descriptions decide which tool gets called: the words in your schema are read by the model on every single request, and they are portable in a way that keywords are not. A field annotated "ISO 8601 date, YYYY-MM-DD. Null if the document does not state one." gets you most of what a format keyword would have, on every provider, for free.

Design to the intersection subset

The discipline is straightforward to state and requires a decision most teams never consciously make: design your schemas to the intersection of what all your target providers accept, and express everything richer in the validator.

Deciding the subset takes about an hour and starts with an honest list. Which providers do you actually support? Not which one you use — which ones could you be asked to run on. Include the provider a large client mandates in a contract. Include the regional endpoint that a data-residency requirement might force you onto. Include the open-weight model you would fall back to if a price rise made the current arrangement untenable. Then take the intersection of the schema features that all of them accept, write it down in one module, and make that module the only place schemas are constructed.

The mechanical part is a schema emitter that strips the non-portable keywords on the way out. The constraints are not discarded; they move.

import re
from datetime import date
from typing import Any, Optional
from pydantic import BaseModel, field_validator

UK_VAT   = re.compile(r"^GB\d{9}$")
IN_GSTIN = re.compile(r"^\d{2}[A-Z]{5}\d{4}[A-Z][A-Z\d]Z[A-Z\d]$")

# Keywords that are not accepted, or not honoured, by every provider
# we ship against as of 2026. Review this set when the contract tests
# in the next section change colour.
NON_PORTABLE_KEYWORDS = {
    "minimum", "maximum", "exclusiveMinimum", "exclusiveMaximum",
    "minLength", "maxLength", "pattern", "format",
    "minItems", "maxItems", "uniqueItems", "multipleOf",
}


def portable_schema(model: type[BaseModel]) -> dict[str, Any]:
    """Emit JSON Schema, then remove every keyword the intersection
    subset excludes. Nothing is lost: StrictInvoice enforces all of it."""
    def strip(node: Any) -> Any:
        if isinstance(node, dict):
            return {k: strip(v) for k, v in node.items()
                    if k not in NON_PORTABLE_KEYWORDS}
        if isinstance(node, list):
            return [strip(v) for v in node]
        return node

    return strip(model.model_json_schema())


class StrictInvoice(WireInvoice):
    """Never leaves the process. This is where the contract lives, and
    it is identical no matter which provider produced the payload."""

    @field_validator("supplier_name", "invoice_number")
    @classmethod
    def non_blank(cls, v: str) -> str:
        if not v.strip():
            raise ValueError("field was present but empty")
        return v.strip()

    @field_validator("tax_id")
    @classmethod
    def known_tax_id(cls, v: str) -> str:
        # The regex we could not portably put in the wire schema.
        if not (UK_VAT.match(v) or IN_GSTIN.match(v)):
            raise ValueError(f"tax_id {v!r} is neither a UK VAT number nor a GSTIN")
        return v

    @field_validator("issued_on", "due_on")
    @classmethod
    def iso_date(cls, v: Optional[str]) -> Optional[str]:
        if v is None:
            return None
        date.fromisoformat(v)      # raises on anything that is not an ISO date
        return v

    @field_validator("total_minor_units", "line_count")
    @classmethod
    def non_negative(cls, v: int) -> int:
        # The numeric bound we could not portably put in the wire schema.
        if v < 0:
            raise ValueError("value must not be negative")
        return v

Two worked substitutions

A union becomes a discriminator. Suppose the natural model is "the counterparty is either an individual or a company", which JSON Schema expresses as an anyOf over two object shapes. Because union types are restricted on at least one major provider, send a single object instead: a counterparty_type enum with the values individual and company, plus the fields for both branches, all nullable. Then enforce the real rule in a model validator — if the type is company then the company registration number must be present and the date of birth must be absent, and vice versa. The wire schema is dull and portable; the semantics are intact and enforced in one place.

A recursive tree becomes a flat list. Extracting a document outline, an org chart or a nested requirements structure invites a self-referencing schema, which is the least portable construct in common use. Send an array of flat records instead, each with an id, a parent_id and a depth, and rebuild the tree in your code. You gain portability, a much smaller compiled grammar, and the ability to reject a malformed hierarchy with a clear error rather than a truncated object.

Contract tests catch provider drift

A contract test, in this setting, answers one question that no unit test answers: will this exact schema still be accepted by this exact provider? Your own tests validate payloads against your own validator, so they pass forever regardless of what any provider does. Only a real request finds out whether the provider still agrees with you.

The suite has two halves. The first submits every schema you ship, to every provider you support, with a minimal prompt and a tiny output budget, and asserts acceptance. The second feeds a corpus of known-bad payloads to your validator and asserts rejection with the expected message — which is what stops a well-meaning refactor from quietly loosening a rule.

# tests/contract/test_schema_portability.py
import pytest
from pydantic import ValidationError

from myapp.schemas import PORTABLE_SCHEMAS   # {"invoice": {...}, "claim": {...}}
from myapp.providers import PROVIDERS        # {"openai": adapter, "anthropic": ...}
from myapp.models import StrictInvoice

TINY_PROMPT = "Return the fields for a single fictitious example record."


@pytest.mark.contract          # excluded from the default `pytest` run
@pytest.mark.parametrize("provider_name", sorted(PROVIDERS))
@pytest.mark.parametrize("schema_name", sorted(PORTABLE_SCHEMAS))
def test_provider_accepts_schema(provider_name: str, schema_name: str) -> None:
    provider = PROVIDERS[provider_name]
    schema = PORTABLE_SCHEMAS[schema_name]

    result = provider.structured_call(
        prompt=TINY_PROMPT,
        json_schema=schema,
        max_output_tokens=128,      # keep the suite cheap enough to run nightly
    )

    assert result.accepted, (
        f"{provider_name} rejected schema {schema_name!r}: {result.error}\n"
        f"Remove the offending keyword from the intersection subset, "
        f"move the constraint into the validator, and re-run."
    )


KNOWN_BAD = [
    ('{"supplier_name": "", "tax_id": "GB123456789", "invoice_number": "A1",'
     ' "currency": "GBP", "total_minor_units": 1000, "issued_on": "2026-01-04",'
     ' "due_on": null, "line_count": 1}',
     "field was present but empty"),
    ('{"supplier_name": "Acme", "tax_id": "NOTATAXID", "invoice_number": "A1",'
     ' "currency": "GBP", "total_minor_units": 1000, "issued_on": "2026-01-04",'
     ' "due_on": null, "line_count": 1}',
     "neither a UK VAT number nor a GSTIN"),
    ('{"supplier_name": "Acme", "tax_id": "GB123456789", "invoice_number": "A1",'
     ' "currency": "GBP", "total_minor_units": -50, "issued_on": "2026-01-04",'
     ' "due_on": null, "line_count": 1}',
     "must not be negative"),
]


@pytest.mark.parametrize("payload,expected", KNOWN_BAD)
def test_validator_rejects_known_bad(payload: str, expected: str) -> None:
    with pytest.raises(ValidationError) as caught:
        StrictInvoice.model_validate_json(payload)
    assert expected in str(caught.value)

The scheduling matters as much as the tests. Your schemas may not change for six months while the providers change underneath you continuously, so a suite that runs only when someone touches the code will never catch drift. Run the contract half nightly or weekly against every provider, on a schedule rather than on commit, and let a change in provider behaviour arrive as a red build on a Tuesday morning instead of as an incident at three in the morning during a migration.

Three practical details make the difference between a suite people keep and one they delete. Tag it so a provider outage does not block ordinary pull requests — a failed contract test should notify, not gate the merge queue. Keep the prompts trivial and the output budget small so the whole run costs a rounding error; the same instinct that drives cost-aware evaluation, measured as quality per pound applies here, and a suite that is expensive to run gets switched off. And make the failure message name the provider, the schema and the exact keyword that was rejected, with the remedy stated, because the person reading it at eight in the morning will not have this article open.

Contract tests are also what makes the table in section four safe to publish. The specific limits will drift; some of them will have drifted by the time you read this. The tests are what protect you from that drift, because they re-derive the answer from the providers themselves every night rather than from a document that was accurate once.

Valid is not the same as usable

Constrained decoding guarantees the shape of the answer and has no opinion whatsoever about its content. That is the gap the 2026 paper When Correct Isn't Usable (arXiv:2605.02363) names directly, and it is the gap that produces the most confusing incidents, because every metric you are watching says the system is healthy.

Four shapes account for most of it. Present but empty: a required string arrives as an empty string, or as a placeholder like "N/A" or "unknown", because the grammar demanded a string and the document did not supply one. A plausible invented enum member: the model selects a value that is genuinely in your allowed list and genuinely not supported by the source, which is the hardest of the four to detect because it is indistinguishable from a correct answer without checking the source. Correctly formatted and wrong: a date that parses cleanly as ISO 8601 and refers to a day that appears nowhere in the document, or is in the future, or predates the company. In range and nonsensical: a total with no line items, a due date before the issue date, a quantity of zero on a line that has a price.

The remedy is a separate pass of semantic assertions, run after validation and before the value is used. Keep them out of the Pydantic validators, because these checks need the source document as well as the parsed object, and because you want to be able to return them as warnings for review rather than as hard failures.

from datetime import date

PLACEHOLDERS = {"", "n/a", "na", "none", "null", "unknown", "string", "tbd"}

CURRENCY_TOKENS = {
    Currency.GBP: ("GBP", "£"),
    Currency.INR: ("INR", "₹", "Rs", "Rs."),
    Currency.EUR: ("EUR", "€"),
    Currency.USD: ("USD", "US$", "$"),
}


def semantic_problems(inv: StrictInvoice, document: str,
                      today: date) -> list[str]:
    """Everything the schema cannot express. Returns human-readable
    reasons; an empty list means the record is safe to use."""
    problems: list[str] = []
    haystack = document.lower()

    # 1. Present, valid, and carrying no information.
    for field in ("supplier_name", "invoice_number"):
        value = getattr(inv, field)
        if value.strip().lower() in PLACEHOLDERS:
            problems.append(f"{field} is a placeholder, not a value")

    # 2. Groundedness — a literal we copied should appear in the source.
    if inv.invoice_number.lower() not in haystack:
        problems.append("invoice_number does not appear in the document")

    # 3. A legal enum member with nothing in the source to support it.
    if not any(t.lower() in haystack for t in CURRENCY_TOKENS[inv.currency]):
        problems.append(f"currency {inv.currency.value} is not evidenced")

    # 4. Right format, wrong value. (StrictInvoice already proved it parses.)
    issued = date.fromisoformat(inv.issued_on)
    if issued > today:
        problems.append("issued_on is in the future")
    if issued.year < today.year - 7:
        problems.append("issued_on is implausibly old")
    if inv.due_on and date.fromisoformat(inv.due_on) < issued:
        problems.append("due_on precedes issued_on")

    # 5. Each number is in range; together they do not make sense.
    if inv.line_count == 0 and inv.total_minor_units > 0:
        problems.append("a non-zero total with no line items")
    if inv.line_count > 0 and inv.total_minor_units == 0:
        problems.append("line items with a zero total")

    return problems

Two design notes on that function. It returns reasons rather than raising, because a semantic problem is often a signal about the source document rather than about the model — a genuinely illegible scan will produce the same warnings every time, and you want that routed to a human queue rather than retried into the ground. And the groundedness check in point two is the cheapest high-value assertion available for any extraction task: if you claim to have copied a literal out of a document, the literal ought to be in the document. It catches invented identifiers, hallucinated reference numbers and transposed digits in a single line of code.

For the judgement calls that a rule cannot settle — is this the correct supplier when the document names three, is this summary faithful — a second model pass is the usual escalation, and our guide to second-model verification and what the critique actually costs covers when the extra call is worth its price and when it is theatre. Use it selectively, on the subset of records that the cheap assertions have already flagged.

Retry design that does not burn the budget

Retries are where a well-designed extraction pipeline turns into an expensive one. The rule that keeps it honest is to branch on the error class rather than retrying on anything that raised. Format errors — a missing field, a wrong type, a violated pattern — are worth retrying, because the model can self-correct when you quote the specific violation back to it. Business validation failures generally are not, because a blind retry against a rule the model cannot see just produces a different wrong answer at the same price.

Failure class	What it looks like	Retry?	Action
Transport or rate limit	429 or 503 from the provider	Yes, identical request	Exponential backoff with jitter, inside the hard cap.
Schema rejected by the provider	400 naming an unsupported keyword	Never	Fail fast and page the owner. A contract test should have caught this in CI.
Truncated or unparseable output	Output hit the token ceiling mid-object	Once, with a larger budget	Raise the output limit or narrow the schema. Do not simply repeat the call.
Missing field or wrong type	Validation error on a path without constrained decoding	Yes, with the violation quoted	Append the specific error to the prompt. The model self-corrects well here.
Validator constraint violated	Regex, length or range enforced at the boundary	Yes, up to twice	Quote the rule in plain language, not the raw stack trace.
Semantic problem	Empty value, ungrounded literal, impossible date	Once at most	Then route to review — the source often genuinely lacks the value.
Business rule violated	Amount over an approval limit, unknown supplier id	No	Escalate to a human or the calling service. Never retry blindly.
Identical failure repeating	Same error on every attempt	No	Circuit-break, record the payload, take the fallback path.

from enum import Enum


class Outcome(str, Enum):
    RETRY_SAME      = "retry_same"        # transient; resend unchanged
    RETRY_WITH_HINT = "retry_with_hint"   # the model can self-correct
    ESCALATE        = "escalate"          # a person or the caller decides
    FAIL_FAST       = "fail_fast"         # no prompt will fix this


MAX_ATTEMPTS = 3      # a hard cap, not a per-call-site setting


def classify(error: Exception) -> Outcome:
    if isinstance(error, (RateLimited, UpstreamUnavailable, TruncatedOutput)):
        return Outcome.RETRY_SAME
    if isinstance(error, SchemaRejected):
        # The provider refused the schema itself. Retrying is pure spend.
        return Outcome.FAIL_FAST
    if isinstance(error, ValidationError):
        return Outcome.RETRY_WITH_HINT
    if isinstance(error, SemanticProblem):
        return Outcome.RETRY_WITH_HINT
    if isinstance(error, BusinessRuleViolation):
        return Outcome.ESCALATE
    return Outcome.FAIL_FAST


def run(document: str) -> InvoiceResult:
    hint = ""
    for attempt in range(1, MAX_ATTEMPTS + 1):
        try:
            return extract_once(document, hint)
        except Exception as err:
            outcome = classify(err)
            if outcome is Outcome.FAIL_FAST:
                raise
            if outcome is Outcome.ESCALATE:
                return queue_for_review(document, err)
            if outcome is Outcome.RETRY_WITH_HINT:
                # A semantic problem that survives one correction is
                # usually the document, not the model. Stop paying for it.
                if isinstance(err, SemanticProblem) and attempt > 1:
                    return queue_for_review(document, err)
                hint = describe(err)
            sleep(backoff(attempt))

    return queue_for_review(document, AttemptsExhausted(MAX_ATTEMPTS))

The failure mode this structure exists to prevent is the loop that burns real money on a schema the model can never satisfy. It happens when a required field asks for something the document simply does not contain, or when a validator rule is stricter than anything the source could support. Each attempt looks individually reasonable, the retry count is "only" three, and at ten thousand documents a day you have quietly tripled the bill on your worst-performing tenth of the corpus. The hard cap, the escalation branch and the fallback path together turn that from an unbounded cost into a bounded one with a queue attached.

When a strict schema is the wrong tool

Structured output is not free of cost, and there are cases where imposing it makes the system worse. Three are worth naming.

Exploratory extraction. If you do not yet know which fields matter, a schema freezes a guess into the pipeline and hides everything you did not think to ask for. Run the first few hundred documents with a loose instruction and read what comes back, then design the schema from evidence. Locking the structure too early is how teams end up with an extraction pipeline that is precisely wrong.

Reasoning you intend to parse afterwards. Forcing a model to write its analysis directly into a rigid structure can cost you the working-out that made the analysis good, because the grammar keeps steering it back into field boundaries. The pattern that works is two calls: let the model reason in prose, then extract from that text with a cheap, tightly constrained second call. The second call has an easy job — the answer is in front of it — and the first call has room to think. This is straightforwardly an instance of the wider shift that our reporting on context engineering replacing prompt engineering describes: designing what the model sees and when, rather than arguing with it inside a single request.

A required field is a fabrication incentive

This is the point most worth internalising, and it follows mechanically from how constrained decoding works. If a field is required, the finite state machine will not let generation finish until that field has a value. The model cannot decline. It cannot leave the field out. Its only available move is to emit something that fits the type — and what a language model produces when it must produce something is a plausible value. You have not asked for a fact; you have asked for a token sequence, and you will get one.

Every required field is therefore a small standing invitation to fabricate, and the invitation is strongest exactly where the data is worst: poor scans, partial documents, edge-case formats. Treat optionality as a deliberate per-field design decision rather than a default that falls out of your type annotations. For each field, ask whether a real source document might legitimately not contain it. If the answer is yes, make it nullable, say in the description what null means, and instruct the model explicitly that null is the correct answer when the value is absent. Then measure: a sudden drop in nulls after a model upgrade is a signal worth investigating, not a quality improvement.

The same reasoning applies to agent tool arguments, where a required parameter the model cannot fill honestly produces a confidently wrong call, and where progressive tool disclosure is the structural answer to keeping the surface small enough that each argument is genuinely answerable.

Portability is a delivery requirement, not a preference

It is tempting to file all of this under good engineering hygiene, the sort of thing you would do if there were time. In both of AI Tech Connect's markets it is closer to a contractual obligation, and for different reasons.

For Indian services firms and the global capability centres in Bengaluru, Hyderabad, Pune and Chennai, the pattern is familiar to anyone who has run delivery: the same extraction pipeline gets built four times for four clients, and each client mandates a different provider. One has an enterprise agreement with one vendor, one has standardised on a hyperscaler's model garden, one insists on a specific regional endpoint, one has an internal open-weight deployment. If your schema module only works on the provider your first client happened to choose, every subsequent engagement pays a rewrite tax that was never in the estimate. Design to the intersection subset and the fourth engagement is a configuration entry and a contract-test run. That is a margin decision as much as a technical one, and it is the sort of thing that decides whether a fixed-price extraction project is profitable.

For UK teams the pressure comes from a different direction: data residency, sector-specific procurement rules, and public-sector frameworks that constrain which supplier and which region you may use, sometimes decided long after the build started. A financial services client's second-line review, an NHS trust's information governance process or a local authority's procurement cycle can land on your project mid-delivery with an instruction to move. If the schema is portable and the validator is authoritative, that instruction costs you a config change, a contract-test run and a regression pass. If it is not, it costs you a rewrite you cannot bill for, during a period when the client is already anxious. The commercial dynamics behind that fragmentation — three major vendors each pulling their SDKs and formats in different directions — are the subject of our reporting on the agent SDK wars between OpenAI, Google and Anthropic, and there is no sign of a common standard arriving to rescue anyone.

If you take one thing away, make it this sequence. Write down which providers you support, including the ones you might be forced onto. Build the schema module to their intersection and strip the rest on the way out. Put every richer constraint in a Pydantic or Zod model at the boundary, and let that model be the single source of truth about what your system will accept. Add the semantic assertions the schema cannot express, starting with groundedness and empty-value checks. Then write the contract tests and run them on a schedule, so that the day a provider changes its mind about a keyword, you find out from a build rather than from a customer. The specific limits in this article will be out of date before the discipline is.