← Retour aux guides

How to Get Stable LLM Output Every Time

Why LLM output formats drift

If you have ever asked an LLM to return JSON, a table, or a fixed set of fields, you have probably seen the same failure modes: extra commentary, swapped field names, missing commas, or a helpful but unusable paragraph before the answer. The model is not being random; it is balancing instruction-following with language generation. That means the real job is not just “ask for JSON,” but design a workflow that makes the format easy to produce and easy to verify.

The best teams treat output format as an interface contract. The model generates a draft, your app validates it, and if the result is off-spec, you repair or retry. Once you adopt that mindset, consistency improves fast.

Step 1: Define the contract before you prompt

Start with a precise schema for the output you need. Be explicit about required keys, value types, allowed enums, and whether empty values should be null, an empty string, or omitted. If the model has to guess, it will. A clear contract removes that guesswork.

• Good: title is a string, sentiment is one of positive, neutral, negative, and score is an integer from 1 to 5.
• Bad: “Return a useful analysis object.”

For production workflows, keep the contract small. The more fields you require, the more chances there are for format drift. Ask only for what the downstream step really needs.

Step 2: Put formatting rules in the system message

Don’t bury formatting instructions at the end of a long user prompt. Put them in a system or developer message so they are always present. Include the output shape, one short example, and the one rule that matters most: no extra text outside the schema.

A useful pattern is to say: “Return only valid JSON matching this schema. Do not wrap it in markdown. Do not add commentary. If a value is unknown, use null.” That one sentence removes a lot of ambiguity.

If you want even better consistency, show a single, fully valid example. Models imitate patterns very well, especially when the example is close to the task.

Step 3: Validate immediately, not later

Never trust raw model output in a production pipeline. Parse it right away and fail fast if it does not match the contract. In practical terms, that means:

• Attempt to parse the response as soon as it arrives.
• Check required keys and value types.
• Reject any extra top-level text if your pipeline expects pure JSON.
• Log the failure reason so you can tune the prompt later.

This is where many teams stop. But validation alone does not fix the problem; it just tells you when the problem happened. The next step is recovery.

Step 4: Build a repair loop

When validation fails, send the model a short correction prompt that includes the broken output and the exact reason it failed. Keep the retry narrow. For example: “The output was not valid JSON because the key sentiment was missing. Return the same object again, with only valid JSON.”

In my experience, one repair attempt is usually enough. More than one retry often means the prompt or schema is too vague. If you need repeated repairs, simplify the schema or split the task into two smaller calls.

Step 5: Test with ugly inputs

Consistency is easy on clean examples. It gets harder with edge cases: empty text, extremely long text, mixed languages, or inputs that contain quotes and newlines. Build a small test set that intentionally breaks your assumptions. Run it every time you change the prompt.

A good test set includes:

• Very short inputs
• Very long inputs
• Inputs with special characters
• Ambiguous cases where the model should return null

If your format still holds under those cases, you are close to production-ready.

Why cost matters when you validate and retry

Structured output workflows often use more than one call: generate, validate, repair, and sometimes compare multiple candidates. That is why low cost matters. If every retry is expensive, teams become reluctant to enforce quality. If the model is affordable, you can validate aggressively without worrying about the bill.

That is where 59API fits nicely. It is a cheap pay-as-you-go AI API relay with native official-quality Claude models and GPT models, so you do not have to trade consistency for lower cost. It is fully compatible with Claude Code, Codex, and any OpenAI SDK, and you can point your client to the base URL https://api.59api.com. For teams shipping structured workflows, that compatibility means you can keep your existing code and simply swap the endpoint.

Because 59API is among the cheapest relays and includes a referral rebate, it is especially practical when you are running lots of validation tests or retry logic in development. You get the same workflow, but with less friction on the budget.

A simple production checklist

• Define the output schema first.
• Use a strict system prompt with one valid example.
• Parse and validate immediately.
• Retry once with a focused repair prompt.
• Test against messy real-world inputs.
• Keep the model endpoint cheap enough that quality checks are affordable.

If you want to make your LLM outputs dependable without overpaying for experimentation, sign up for 59API and wire it into your existing OpenAI-compatible workflow. It is a straightforward way to keep format consistency high while staying cost-conscious.

Tarifs · Docs