← सभी गाइड पर लौटें

Reliable LLM JSON Mode: A Production Workflow

Why JSON Mode Alone Is Not a Production Guarantee

Structured output turns an LLM from a chat interface into a dependable component in an application workflow. Instead of asking for a free-form answer, you request data that your software can use directly: extracted invoice fields, product classifications, support-ticket routing labels, or content metadata.

JSON mode is helpful because it strongly encourages valid JSON, but valid JSON is not always usable JSON. A model can return syntactically correct data with an unexpected field name, a missing required value, or an enum value your application does not recognize. A reliable implementation therefore combines a narrow schema, clear instructions, server-side validation, and a controlled retry path.

This walkthrough uses a realistic support-ticket triage workflow. The same pattern applies to extraction, moderation, lead qualification, and agent tool calls.

Step 1: Define the Smallest Useful Schema

Begin with the data your downstream code truly needs. For a support ticket, define four fields: category, urgency, summary, and needs_human_review. Keep category constrained to values such as billing, technical, account, or other. Keep urgency constrained to low, normal, high, or critical. Make summary a short string, and make needs_human_review a boolean.

Do not ask the model to generate fields your system can calculate itself, such as a ticket ID, ingestion timestamp, or the authenticated user ID. Smaller schemas reduce ambiguity, lower token use, and make validation much easier. They also create cleaner analytics because every output follows the same vocabulary.

Step 2: Write Instructions That Match the Schema

Your system prompt should state the task, the decision rules, and the output constraint. For example: classify the ticket using only the permitted categories; mark critical only for service outages, security concerns, or data loss; set needs_human_review to true when the ticket lacks enough evidence or contains a sensitive request.

Then explicitly tell the model to return only the requested JSON object. If your selected endpoint supports schema-based structured outputs, provide the JSON Schema and make required fields mandatory. If it supports JSON mode only, use the JSON response setting and include the field definitions in the prompt. Schema-based output is generally stronger, while JSON mode is a valuable compatibility fallback.

Capability details can differ by provider and model, so test the exact model you deploy. This is especially important when switching between fast low-cost models for routine classification and more capable models for difficult cases.

Step 3: Use an OpenAI-Compatible Client Configuration

With an OpenAI SDK-compatible integration, point your client at https://api.59api.com, supply your 59API API key, select the model, and send your messages with the appropriate structured-output or JSON-mode option. 59API is useful for this workflow because it provides pay-as-you-go access to native official-quality Claude and GPT models while remaining compatible with existing OpenAI SDK patterns, Claude Code, and Codex setups.

That compatibility makes model evaluation less disruptive. You can keep one application workflow and compare a fast model for high-volume ticket routing against a stronger model for ambiguous or high-risk requests. Since structured-output tasks can involve many small calls, a low-cost relay is particularly helpful for testing realistic volumes rather than judging quality from a handful of prompts.

Step 4: Validate Every Response in Your Application

After receiving the model response, parse the JSON and validate it with your application schema library. Validation should check that all required keys exist, additional keys are rejected when appropriate, enum values are allowed, strings meet length limits, and booleans are actual booleans rather than text such as “yes.”

• Parse failure: the response is not valid JSON.
• Schema failure: the JSON is valid but does not match your contract.
• Business-rule failure: the JSON matches the schema but violates your operational logic, such as marking a password-reset issue as critical.

Log these failure types separately. This distinction tells you whether to improve the prompt, tighten the schema, adjust model selection, or fix an application-side assumption.

Step 5: Add One Focused Retry, Then Escalate Safely

Never silently accept malformed output. On a parse or schema failure, retry once with the original ticket plus a concise correction message: “Your previous response failed validation because category was not an allowed value. Return only an object that matches the required schema.” Include the validation reason, but do not expose internal secrets or unrelated customer data.

If the second attempt fails, return a safe fallback: category other, urgency normal, and needs_human_review true. Queue the ticket for a person or a more capable model. Limit retries to avoid surprise cost, latency spikes, and loops caused by adversarial input.

Step 6: Measure Quality Before Expanding the Workflow

Create a test set of real anonymized tickets, including vague requests, multilingual messages, prompt-injection attempts, missing details, and multiple issues in one ticket. Track JSON validity rate, schema-valid rate, correct category rate, unsafe escalation misses, latency, and cost per ticket. Evaluate each candidate model with the same test set before changing production routing.

59API makes this kind of controlled comparison practical by offering economical access to Claude models including Opus, Sonnet, Haiku, and Fable, as well as GPT models, without downgrading the underlying model quality. Once your schema, validation, and fallback logic are stable, you can optimize for cost with confidence. If you are building structured AI features, consider signing up for 59API to test the workflow against the models that best fit your accuracy and budget targets.