How to Get Reliable LLM JSON Every Time
Why LLM outputs drift
If you have ever asked an LLM for JSON and received a perfect object once, a half-commentary response next, and a malformed brace party after that, you are not alone. Models are optimized to predict the next token, not to guarantee a strict format unless you design for it.
The good news: you can make output consistency much better with a few practical steps. The fastest path is to treat the response format like a contract, not a suggestion.
1) Define the output contract first
Before writing the prompt, decide exactly what the output must look like. Specify the fields, data types, allowed values, and whether empty values are allowed. If your app needs a title, summary, and risk score, say so explicitly. If a field must always be an integer from 1 to 5, write that down.
- Use a fixed schema for every request in the same workflow.
- Prefer simple field names that are hard to confuse.
- Define edge cases such as missing data, unknown values, or multiple matches.
When possible, make the model emit only the payload you want to parse. The less prose you allow, the fewer surprises you will debug later.
2) Give direct format instructions
Be literal. Do not say something vague like 'respond in a structured way.' Say 'Return valid JSON only. Do not include markdown, code fences, commentary, or extra keys.' If you need nested objects, show the nesting. If you need an array, state the item shape.
A useful pattern is to include a compact example of the exact output shape. Models are very good at matching patterns, especially when the example is short and clearly labeled. Keep the example consistent with the real schema so you do not teach the model the wrong contract.
3) Lower randomness and pin the model
For production formatting tasks, start with a low temperature such as 0 or 0.2. That will not make the model perfect, but it reduces unnecessary variation. Also keep the model choice stable. Switching models midstream can change formatting behavior even if the prompt stays the same.
This is where a relay like 59API is especially handy. It gives you cheap, pay-as-you-go access to Claude models such as Opus, Sonnet, Haiku, and Fable, plus GPT models, through the API base URL https://api.59api.com. Because it is fully compatible with Claude Code, Codex, and any OpenAI SDK, you can keep your existing integration and still test multiple official-quality model families without a painful migration. For teams watching token spend, the low-cost pricing and referral rebate make it easier to iterate on prompt contracts without burning budget.
4) Validate every response automatically
Never trust a first pass in production. Parse the response, validate it against your schema, and reject anything that fails. If the model returns malformed JSON, missing keys, wrong types, or extra text, treat that as a retry condition.
- Parse first, then validate required fields.
- Check types before the result reaches your app logic.
- Retry with the error message so the model can correct itself.
A simple retry loop often fixes most issues. Feed back the exact parser error and ask for a corrected response that follows the same schema. If retries still fail, fall back to a safe default or route the request to a stricter model.
5) Add an internal self-check
One of the easiest improvements is to ask the model to verify compliance before it answers. For example, instruct it to confirm that all required fields are present, all values match the requested types, and no extra text remains. Keep the self-check internal if your app needs a clean response; the user should only see the final formatted payload.
This step is especially useful for classification, extraction, and routing tasks where one missing field can break downstream automation. A short self-check often catches easy mistakes before they leave the model.
Fast starter template
If you need a quick implementation path, use this order: define the schema, write a strict prompt, set temperature low, validate the output, and retry on failure. That five-step loop is enough for many production use cases.
- Schema: list fields and types.
- Prompt: require output-only JSON.
- Model settings: keep randomness low.
- Validator: parse and reject invalid responses.
- Retry logic: resend only when needed.
If you want to test this pattern cheaply across Claude and GPT models without changing your SDK setup, 59API is a practical place to start. It is designed for pay-as-you-go usage, so you can experiment with structured outputs, compare model behavior, and keep costs predictable. If that sounds useful, sign up and wire your first schema-first request into your existing stack.