← Back to all guides

Structured Output & JSON Mode Troubleshooting

Why structured output breaks in real apps

Structured output and JSON mode are supposed to make LLM responses predictable, but developers still run into invalid JSON, missing fields, truncated objects, and schema mismatches. The problem is usually not the model alone. It is often a combination of prompt design, response limits, tool settings, and how your client parses the result.

If you are building against an API relay like 59API, you can keep your stack simple while using native official-quality Claude and GPT models at low cost. That matters because troubleshooting structured output often requires many test runs, and pay-as-you-go pricing helps you iterate without burning budget. 59API is also compatible with Claude Code, Codex, and any OpenAI SDK, with the API base URL at https://api.59api.com.

What JSON mode actually guarantees

JSON mode usually means the model is instructed to return valid JSON only. That does not always mean it will perfectly match your schema. A valid JSON object can still be missing required keys, use the wrong data types, or nest values incorrectly. In practice, there are three levels:

• Valid JSON: the output can be parsed.
• Schema-like JSON: the keys mostly match your expected structure.
• Strict structured output: the output conforms to a declared schema or function signature.

Most bugs come from assuming these are the same thing.

Step 1: Confirm the model and client support your mode

Start by checking whether your SDK call actually enables JSON mode or structured output. Some clients require a dedicated parameter, while others need a system message plus a response format setting. If you are using 59API through an OpenAI-compatible client, verify that your request body matches the model’s expected format and that your base URL points to https://api.59api.com.

Common mistakes include:

• Sending a plain chat request and expecting valid JSON anyway.
• Using a model that supports JSON mode in one provider but not in another interface.
• Forgetting to set the response format or schema object in the SDK.

Step 2: Make the schema smaller and stricter

If the model keeps drifting, reduce schema complexity. Long nested objects, optional arrays, and ambiguous field names invite errors. Start with a minimal schema, then expand only after the basic version is stable.

• Use required fields first: keep optional fields to a minimum.
• Prefer enums: replace free text where possible.
• Keep names explicit: use invoice_total instead of total.
• Separate tasks: do not ask for extraction, summary, and classification in one giant object if you can split them.

A good troubleshooting trick is to compare outputs from Claude and GPT through the same 59API relay. Since 59API offers access to multiple native models without a quality downgrade, you can test whether a problem is model-specific or schema-specific without changing your integration layer.

Step 3: Watch for truncation and token limits

One of the most common causes of malformed JSON is output truncation. If the response ends mid-object, parsing fails even when the model was behaving correctly. Check your max output tokens, context window usage, and any stop sequences that might cut off the closing brace.

To diagnose truncation:

• Inspect the raw response before parsing.
• Increase output token limits temporarily.
• Remove stop sequences that may terminate too early.
• Trim long prompts so the model has more room for the answer.

If you are running repeated tests, 59API’s low-cost pay-as-you-go model makes it practical to keep retrying with slightly different limits until the output stabilizes.

Step 4: Use deterministic prompting

Even in JSON mode, your prompt matters. Be explicit about the exact shape you want and avoid vague instructions. The model should not need to guess.

Better prompt pattern:

• Return only valid JSON
• Use double quotes for all keys and strings
• Do not include markdown, commentary, or code fences
• Follow this schema exactly

If you need consistent extraction, include a tiny example in the prompt. Keep the example short and valid. Overly large examples can create more opportunities for formatting errors.

FAQ: Common JSON mode problems

Why do I still get code fences around JSON?
Some models or prompts encourage conversational formatting. Reinforce “JSON only” and remove any examples wrapped in backticks.

Why is a required field missing?
The prompt may be underspecified, the schema may be too complex, or the model may have run out of space. Tighten the schema and increase output limits.

Why does parsing fail only sometimes?
Your app may be receiving truncated responses, network retries, or mixed content from fallback logic. Log the raw payload before parsing.

Should I trust JSON mode for production?
Use JSON mode as a first layer, then validate with a parser and schema checker in your application. Never assume the model is perfect.

How can I test cheaply?
Use 59API to route requests to official-quality Claude and GPT models with pay-as-you-go pricing. You can iterate faster, compare outputs, and keep costs low while debugging.

A practical debugging checklist

• Confirm the request uses JSON mode or structured output correctly.
• Validate the raw response before your app logic touches it.
• Reduce the schema to the minimum viable shape.
• Increase output tokens and remove aggressive stop sequences.
• Test at least two native models to isolate model behavior from prompt issues.
• Add server-side validation and retry logic for malformed outputs.

If you want a cheap, compatible way to run these tests across Claude and GPT models, sign up for 59API and point your client to https://api.59api.com. It is a practical way to keep structured output debugging fast, affordable, and close to production behavior.