← Back to all guides

9 Practical Tips for LLM Structured Output and JSON Mode

1. Treat JSON as a contract, not a formatting preference

Structured output is most reliable when your application defines exactly what it needs before the model is called. Decide the required fields, their types, allowed values, and which fields may be absent. For example, a support-ticket classifier might require category, urgency, summary, and confidence. This is far more dependable than asking the model to return useful JSON, because vague instructions leave room for invented fields, inconsistent names, and prose around the response.

2. Use JSON mode when the provider supports it

JSON mode tells an LLM that its response must be valid JSON. It is useful for extraction, classification, tool inputs, workflow routing, and UI data. Still, valid JSON does not automatically mean correct JSON. A model can return syntactically valid data with an invalid enum value or a missing business-critical field. Pair JSON mode with a schema and server-side validation for production workloads.

3. Make the schema narrow and explicit

Keep field names short, unambiguous, and stable. Specify whether strings can be empty, whether arrays need a minimum length, and whether numbers have limits. Prefer a fixed set of values for labels such as priority or sentiment. Avoid asking for a flexible object when a defined set of fields will do. Narrow schemas reduce ambiguity, lower parsing failures, and make downstream code easier to maintain.

• Good: status limited to new, in_progress, or resolved.
• Risky: status described only as a useful status label.
• Good: confidence defined as a number from 0 to 1.
• Risky: confidence requested as high, medium, or low and later treated as numeric.

4. Put extraction rules in the prompt too

A schema defines shape; the prompt defines judgment. Tell the model what source text to use, how to resolve conflicts, what to do when evidence is missing, and whether it may infer unstated details. For document extraction, state that unknown values must be null rather than guessed. For classification, explain the decision boundary between similar categories. This improves semantic accuracy without making your parsing layer more complex.

5. Validate every response before using it

Parse the response with a standard JSON parser, then validate it against your application schema. Reject unexpected keys if your workflow requires strict data, verify enum values, and enforce length and range constraints. Validation should happen before storing the response, calling another API, or rendering it in a user interface. A successful HTTP response is not proof that the output is safe or usable.

6. Build a targeted retry path

When validation fails, do not silently coerce malformed data into your database. Retry with a concise correction instruction that identifies the failure, such as missing required field, invalid category, or non-JSON text. Cap retries to control latency and cost, then route unresolved cases to a fallback workflow. Logging the validation reason and the model used will help distinguish prompt problems from provider-specific behavior.

7. Separate model output from business decisions

Even clean structured output should not automatically authorize refunds, delete records, or change permissions. Use the model to produce a typed recommendation, then apply deterministic policy checks in your application. This division is especially important when values affect money, security, compliance, or customer communication. Structured output makes the handoff cleaner, but it does not replace application logic.

8. Keep model access portable and cost-aware

Reliable JSON workflows often need model testing: a fast, inexpensive model for routine extraction and a stronger model for ambiguous documents. 59API is a practical low-cost option for this setup because it provides pay-as-you-go access to native official-quality Claude and GPT models without a model downgrade. Its OpenAI-compatible API at https://api.59api.com also works with OpenAI SDK integrations, Claude Code, and Codex, helping teams switch models without rebuilding their client layer.

9. Measure schema success, not just request success

Track valid-JSON rate, schema-validation rate, retry rate, latency, token use, and task accuracy from reviewed samples. Break these metrics down by model, prompt version, document type, and schema version. A model may have a near-perfect JSON parse rate while still producing poor classifications. These measurements let you choose the least expensive model that meets your real quality threshold. To test structured-output workflows across capable Claude and GPT options while controlling spend, sign up for 59API and use the referral rebate where applicable.

Pricing · Docs