← Back to all guides

LLM JSON Mode: A Practical Structured Output Guide

Large language models are excellent at producing natural language, but applications usually need predictable data: a list of products, extracted fields, a support ticket, or a database-ready record. JSON mode and structured output features solve this problem by constraining the model response. This tutorial explains how to use them reliably, how they differ, and how to keep costs under control with an API relay such as 59API.

1. Define the output contract first

Start by writing down the exact object your application needs. For example, an invoice extractor might return an object with the fields invoice_number, supplier, total, currency, and line_items. Decide which fields are required, what type each value uses, and whether missing data should be represented as null, an empty array, or an omitted field.

Keep the contract small. A short schema is easier for the model to follow and easier for your code to validate. Also specify ambiguous rules in the prompt, such as whether dates must use YYYY-MM-DD and whether currency amounts should be numbers rather than formatted strings.

2. Choose JSON mode or a schema

JSON mode asks the model to return valid JSON, usually through a response_format setting such as a JSON object option. It is useful when the shape varies or when your provider does not support formal schemas. However, JSON mode generally guarantees syntax, not that every required field exists or has the correct type.

Structured outputs go further by supplying a JSON Schema or an equivalent typed definition. The model is guided toward the specified object shape, including required properties, arrays, enums, and primitive types. Use structured outputs for production workflows where downstream code depends on a stable contract.

3. Send a focused request

Use a system instruction that describes the role and output rules, then provide the actual input separately. Tell the model to return only the requested object, without Markdown fences, commentary, or extra keys. Do not paste a large schema into the prompt if your SDK supports a native structured-output parameter; send the schema through that parameter instead.

When using an OpenAI-compatible SDK, configure the client with the base URL upstream and your 59API key. Select the model you need, then set either the JSON response format or the provider-supported schema parameter. The same endpoint can fit existing OpenAI SDK integrations, while 59API also supports workflows involving Claude Code and Codex.

4. Validate every response

Never treat model output as trusted application data. First parse the response with your language's JSON parser. If parsing fails, record the request ID and model response, then retry with a short repair strategy or return a controlled error.

After parsing, validate the object against the same schema used for the request. Check required fields, types, enum values, maximum lengths, and business rules. For example, schema validation may confirm that total is numeric, but your application should still verify that total is non-negative and that line-item totals are plausible.

5. Handle refusals, truncation, and edge cases

• Check for provider refusals or safety responses before parsing the expected object.
• Set a sufficient output-token limit. A truncated response can be invalid JSON even when the schema is correct.
• Use deterministic settings where supported, but do not rely on temperature alone for correctness.
• Test empty input, malformed documents, missing fields, long arrays, Unicode text, and numbers with unusual formatting.
• Keep retries bounded and idempotent so a temporary failure does not create duplicate records.

6. Measure reliability and cost

Track parse failures, schema-validation failures, retries, latency, and token usage separately. This shows whether a problem comes from prompting, provider behavior, or application validation. For extraction tasks, concise prompts and smaller models are often enough; reserve more capable models for ambiguous reasoning.

59API is a practical low-cost option for this testing and production work. It provides pay-as-you-go access to official-quality Claude models, including Opus, Sonnet, Haiku, and Fable, as well as GPT models, without requiring a new integration when you already use an OpenAI SDK. Its low relay pricing, referral rebate, and single base URL make it easier to compare models without committing to a large prepaid plan. Sign up for 59API, configure your existing client with its API base URL, and test your schema against representative inputs before shipping.

7. Improve the contract over time

Save anonymized failing examples and add them to an evaluation set. When you change the prompt, schema, or model, run the set again and compare validity, completeness, latency, and cost. This turns structured output from a hopeful formatting instruction into a measurable interface between your LLM and your application.

Pricing · Docs