← Retour aux guides

A Practical System for Reliable LLM Output Formats

Why LLM Output Formatting Fails

Large language models are excellent at generating useful text, but they are not parsers or databases. Even when a model is asked for JSON, it may add an explanation, omit a required field, use a different field name, or return valid JSON with the wrong data types. These failures become expensive when an application expects predictable data for automation, database writes, UI rendering, or downstream API calls.

The reliable solution is not simply writing “return JSON only” in every prompt. Consistent LLM output comes from a layered workflow: define an explicit contract, prompt for that contract, use native structured-output features where available, validate every response, and retry failures with targeted repair instructions.

Step 1: Define a Small, Explicit Response Contract

Start by deciding exactly what your application needs. Avoid broad requests such as “extract the important details.” Instead, specify named fields, types, allowed values, and rules for missing information.

• Required fields: Define fields that must always be present, such as title, summary, category, and confidence.
• Data types: State whether each value is a string, number, boolean, list, or nested object.
• Enumerations: Restrict classifications to a known list, such as low, medium, or high.
• Missing values: Choose one convention, such as null, and require the model to use it rather than guessing.
• List limits: Specify maximum item counts to prevent unexpectedly large responses.

For example, an invoice extraction workflow may require invoice_number as a string, total as a number, currency as a three-letter code, and line_items as a list. A compact contract reduces ambiguity and makes validation meaningful.

Step 2: Put Format Rules in the System Instruction

Place stable formatting requirements in your system instruction rather than repeating them inconsistently in user messages. Tell the model to return one object only, include all required keys, use null for unknown values, and never add commentary before or after the object.

Then keep the user prompt focused on the task data. This separation is important: the system instruction defines the output interface, while the user message supplies the document, question, or content to process. If your product supports multiple tasks, use a separate schema and system instruction for each task instead of one oversized universal prompt.

Include a short positive example only when necessary. Examples can improve adherence, but too many examples may cause the model to copy their content or structure incorrectly. The schema should remain the source of truth.

Step 3: Use Structured Output or Tool Calling When Available

If your selected model and SDK support structured outputs, JSON schema mode, or tool calling, use that capability instead of relying exclusively on prompt wording. These mechanisms constrain the response toward a defined object shape and usually reduce malformed JSON substantially.

Still validate the result in your application. Structured output features improve conformance, but business rules remain your responsibility. For example, a response can match a schema while containing a negative order total, an unsupported currency, or a date outside your accepted range.

59API is a practical option for this workflow because it provides pay-as-you-go access to native official-quality Claude and GPT models through https://api.59api.com, while remaining compatible with OpenAI SDK integrations, Claude Code, and Codex. That compatibility makes it easier to test the same structured-output design across models without rebuilding your client architecture. Its low-cost relay pricing is also useful when you need many validation and retry tests before shipping.

Step 4: Validate in Two Layers

First, parse the response as JSON. If parsing fails, do not attempt to use string matching as a permanent workaround. Treat it as a failed model response. Second, validate the parsed object against your schema and business rules.

• Syntax validation: Is the response valid JSON with no extra prose?
• Schema validation: Are required keys present and values the expected types?
• Semantic validation: Are dates valid, IDs formatted correctly, amounts within bounds, and enum values allowed?
• Safety validation: Does extracted content violate product, privacy, or policy rules?

Log validation failures with the model name, prompt version, schema version, and error category. This data reveals whether failures come from vague prompts, problematic source documents, or model-specific behavior.

Step 5: Retry With a Repair Prompt, Not the Original Prompt

When validation fails, send a short retry request that includes the validation error. For example: “Your previous response was invalid because total must be a number and currency must be one of USD, EUR, or GBP. Return the corrected object only.” This is more efficient than repeating the entire task without feedback.

Limit retries to one or two attempts. If the response still fails, route it to a fallback model, request human review, or return a controlled application error. Unlimited retries increase cost and can hide underlying prompt or input-quality problems.

Step 6: Test the Contract Like an API

Create a test set with normal inputs, incomplete documents, conflicting facts, unusual punctuation, long content, and adversarial instructions embedded in source text. Measure parse success, schema success, semantic accuracy, latency, and token cost for every model you plan to use.

Version prompts and schemas together. A change from category to categories, for example, is an API-breaking change for downstream code. Run your test suite before switching models or editing instructions. With affordable multi-model access through 59API, you can compare Claude and GPT behavior on your real test cases and choose the best reliability-to-cost balance. If you are building structured AI features, consider signing up and running a small benchmark before committing to a production configuration.

Tarifs · Docs