LLM Structured Outputs: Why Valid JSON Can Still Be Wrong

Documentation and benchmark claims checked against OpenAI, Anthropic, Google, and arXiv sources on October 6, 2026. Provider behavior and model support can change.

Your LLM pipeline can return valid JSON on every request and still put the wrong information into your database. A response may contain every required key, use the expected types, and pass Pydantic or JSON Schema validation while quietly changing a date, choosing the wrong category, or adding a detail that was never present in the source.

This guide explains the gap between schema validity and value accuracy, shows the failure modes that create schema-valid errors, and gives you a reproducible evaluation protocol. The experiment in this article is a proposed protocol, not a claimed measurement from PromptSphereHub. The published figures are attributed to their original sources.

What structured outputs actually guarantee

Provider-native structured outputs use a schema or response format to constrain the form of the model’s response. OpenAI distinguishes JSON mode from Structured Outputs: JSON mode aims to produce valid JSON, while Structured Outputs also targets adherence to a supplied schema when the schema is supported. Anthropic describes constrained decoding for parseable, type-safe JSON, and Google documents response schemas for predictable structured data.

These are useful guarantees. They reduce malformed responses, missing required keys, inconsistent types, and many retry-worthy formatting failures. They do not mean that the model has verified every value against the source. A schema can say that severity must be one of low, medium, or high. It cannot, by itself, determine whether the source actually supports the selected label.

There are also provider-specific boundaries. OpenAI documents supported-schema restrictions and edge cases such as refusals or truncated outputs. Anthropic’s SDK helpers transform unsupported constraints such as minimum, maximum, minLength, and maxLength into field descriptions before sending the schema. Google’s documentation says that while the output is syntactically correct JSON, you should still validate values in your application. Read the documentation for the exact provider and version you deploy; do not treat “structured outputs” as one identical implementation.

Schema validity is not value accuracy

Use two metrics instead of one:

The second metric must be tied to ground truth. For an invoice, that may mean the exact invoice date and total. For a support ticket, it may mean the category and product version that a human annotator can justify from the ticket. A schema validator knows the allowed shape; it does not know which value the source document supports.

The 2026 Structured Output Benchmark evaluated 21 models across native text, OCR-processed images, and audio-derived contexts. Its best reported Value Accuracy was 83.0% for text, 67.2% for images, and 23.7% for audio. The benchmark’s point is not that every production pipeline will achieve those numbers. It is that a response can be structurally correct while still losing exact value accuracy, especially when the source is long, noisy, or difficult to interpret.

Four ways valid JSON can still be wrong

1. Constrained hallucination

An enum makes the output structurally neat. It does not create a correct answer when none of the options is supported. If the source says “the customer did not specify a priority” and the schema allows only low, medium, or high, the model must choose a valid-looking value unless you provide an escape value such as unknown or null.

2. Constraint laundering

A constraint can look strict in your local schema while being weaker at the provider boundary. Anthropic’s documentation explains that its SDK helpers remove some unsupported constraints and add them to descriptions. That can still guide the model, but it is not the same as grammar-level enforcement. If a business rule matters, enforce it again after the model responds.

3. Plausible-value substitution

Models often produce values that sound reasonable but are not copied from the source. A date may be normalized incorrectly. A version may drift toward the latest version mentioned elsewhere in the context. A long description may preserve the general idea while changing the exact fact. All of these can remain valid strings.

4. Mode-specific quality changes

Prompt-only JSON, provider-native structured outputs, and open-source constrained-decoding engines do not have identical behavior. The 2025 constrained-decoding study found differences in schema coverage and noted that tokenization and distributional shifts can affect task quality. If you migrate providers or change a schema, rerun the value-accuracy set instead of assuming that a 100% format rate means the behavior is unchanged.

A two-layer scoring protocol

Run structural validation first. Then, only for responses that pass that layer, compare every leaf field with the ground truth. Keep the stages separate because a parse error and a wrong value require different engineering fixes.

import json
from jsonschema import validate
from jsonschema.exceptions import ValidationError as SchemaError

def score_response(raw: str, schema: dict, ground_truth: dict) -> dict:
    # Layer 1: syntax and schema
    try:
        obj = json.loads(raw)
    except json.JSONDecodeError:
        return {"schema_valid": False, "fields": {}, "reason": "parse_error"}

    try:
        validate(instance=obj, schema=schema)
    except SchemaError as exc:
        return {
            "schema_valid": False,
            "fields": {},
            "reason": f"schema_error: {exc.message}",
        }

    # Layer 2: exact field comparison against annotated truth
    fields = {}
    for path, expected in ground_truth.items():
        actual = obj
        try:
            for key in path.split("."):
                actual = actual[int(key)] if isinstance(actual, list) else actual[key]
            fields[path] = "correct" if actual == expected else "wrong"
        except (KeyError, IndexError, TypeError, ValueError):
            fields[path] = "missing"

    return {"schema_valid": True, "fields": fields}

For nested arrays, use numeric path segments such as items.0.name; the scorer converts those segments to list indexes instead of treating them as dictionary keys.

At dataset level, report at least these values:

• Schema validity rate: structurally valid responses divided by all responses.
• Conditional value accuracy: correct leaf fields divided by scored leaf fields among structurally valid responses.
• End-to-end exact success: records that pass both layers divided by all records.
• Unknown-forcing errors: valid enum values selected even though the source does not support any known option.

The end-to-end rate is useful as a headline, but keep both component rates visible. A drop in schema validity points toward parsing, schema, or provider handling. A drop in value accuracy points toward grounding, ambiguity, source quality, schema design, or model behavior.

The reproducible experiment to run on your data

Status: proposed protocol. The following design is complete enough to run, but PromptSphereHub has not collected results from it. Do not treat the setup as a benchmark result.

Question

Does provider-native structured output improve format reliability without hiding a value-accuracy problem? Does adding forced enums change the error pattern?

Dataset

Create 30 to 50 records from one real extraction task, such as support tickets or invoices. For each record, store the source text, a schema, and field-level ground truth written before testing. Define annotation rules for dates, aliases, partial matches, and “not stated” cases. Keep the set versioned so you can rerun it after a model or prompt change.

Compare two modes

1. Prompt-only baseline: ask for JSON in plain instructions, parse it, and validate it locally.
2. Native structured output: use the provider’s current JSON Schema or response-schema feature with the same model, source records, output limit, and grounding instructions.

Run the same records in the same order where possible. Log model ID, provider, schema version, source ID, raw response, refusal or truncation status, token usage, structural result, and every field-level semantic result. If you want to study enum pressure, run a second schema variant that includes unknown or nullable fields rather than forcing a known category.

for mode in ["prompt_only", "native_structured"]:
    for record in dataset:
        raw = call_model(
            source=record.text,
            mode=mode,
            schema=record.schema,
        )
        result = score_response(
            raw=raw,
            schema=record.schema,
            ground_truth=record.ground_truth,
        )
        log_result(mode, record.id, result)

summarize_by_field_and_mode()

Do not infer a winner from one or two records. A small set is useful for finding failure modes, not for making universal claims. If a difference is small, repeat the test with more records and review the ground truth. If the native mode improves schema validity but lowers value accuracy, redesign the schema and grounding instructions before disabling the feature.

How to harden the pipeline

1. Separate shape from meaning. Put types, required keys, and safe structure in the schema. Put source-grounding instructions in field descriptions and prompts.
2. Audit provider support. Check which keywords are truly enforced by the API and which are transformed, ignored, or rejected. Re-run local validation after every response.
3. Allow “not stated.” Use a nullable field or an explicit unknown enum value whenever the source can legitimately omit information.
4. Keep a gold set. Re-run the same annotated records after changing the model, prompt, schema, provider, or source pre-processing.
5. Log both layers. Production monitoring should show format failures, semantic spot-check results, refusal/truncation states, and field-level error patterns.
6. Block silent regressions. If value accuracy falls below an agreed threshold on a high-stakes field, stop the release or route the record to human review.

If your extraction is part of a retrieved workflow, also review the RAG evaluation metrics and hallucination checks. A perfect extractor cannot recover a fact that the retrieval stage failed to return.

Common mistakes

• Using strict: true as a quality score. It describes a format contract, not proof that the source was understood correctly.
• Monitoring only parse failures. Native structured outputs can make parse errors rare while wrong values remain invisible.
• Over-constraining every field. A narrow enum can turn uncertainty into a valid but unsupported answer.
• Copying one provider’s schema to another. Supported subsets and SDK transformations differ. Treat migration as a new test condition.
• Repairing JSON and discarding the evidence. A repair library can help a prompt-only path recover syntax, but a high repair rate is a signal to investigate the model or prompt.

When exact value scoring is the wrong tool

Exact leaf matching is useful for extraction, classification, and tool-input preparation. It is not the right metric for every output. Summaries and open-ended writing need criteria such as factuality review, citation checks, rubric-based evaluation, or human assessment. For noisy OCR and audio, separate source-transcription errors from extraction errors; otherwise the metric will blame the model for information that was already lost upstream.

For larger evaluation programs, the evaluation dataset before production guide offers a broader approach to fixture design. For production monitoring, connect the field-level results to an AI observability playbook rather than leaving them in a one-off notebook.

Practical checklist

• Schema validity and value accuracy are defined separately.
• A versioned gold set contains at least 30 annotated records, with rules for ambiguous fields.
• Every response is parsed and validated locally, even when the provider claims schema adherence.
• Nullable or unknown values exist where the source can be silent.
• Field-level accuracy is reported by mode, model, and source type.
• Refusals, truncation, repair events, and provider schema errors are logged separately.
• Schema and prompt changes trigger a regression run.
• High-stakes accuracy failures route to review instead of silently entering a database or tool call.

Concrete example: a support-ticket extractor

Suppose your schema contains customer_name, issue_category, severity, and first_mentioned_version. A ticket says: “Since the 4.2 update the export button does nothing. This is blocking our month-end close, and we are a paying customer.”

A defensible annotation might be issue_category: bug, severity: high, and first_mentioned_version: 4.2. A model could return medium because severity language is ambiguous or return a newer version mentioned elsewhere in a long context. Both values can satisfy a simple enum and string schema. Only comparison with the annotated source exposes the mistake.

The fix is not to make the schema more decorative. It is to define the evidence rule, permit uncertainty where uncertainty is real, and measure whether the rule works on records the system did not see during prompt writing.

Sources and further reading

1. The Structured Output Benchmark (SOB) — supports the 21-model design, exact leaf-value metric, and reported 83.0% text, 67.2% image, and 23.7% audio best Value Accuracy figures.
2. OpenAI Structured Outputs documentation — supports the distinction between JSON mode and schema adherence, supported-schema caveats, and refusal/truncation edge cases.
3. Anthropic Structured Outputs documentation — supports constrained-decoding guarantees, SDK schema transformation behavior, and invalid-output edge cases.
4. Google Gemini structured-output documentation — supports response-schema behavior, supported JSON Schema subsets, and the recommendation to validate semantic values in the application.
5. JSONSchemaBench: A Rigorous Benchmark of Structured Outputs for Language Models (Geng et al., 2025) — supports the explanation of constrained decoding, schema coverage differences, and possible quality effects from tokenization and distributional shifts.
6. JSON Schema: Getting Started — background reference for what a schema describes and validates.