← सभी गाइड पर लौटें

Cut LLM Costs with Structured Output and JSON Mode

Why Structured Output Lowers LLM Spend

Structured output is not only a developer-experience feature. It is a direct cost-control mechanism. When an LLM returns predictable JSON, your application can parse results automatically instead of sending follow-up prompts such as “format this as JSON,” “extract only the fields,” or “fix the invalid response.” Each repair request consumes input and output tokens, adds latency, and increases the chance of another failure.

Consider a customer-support classifier that processes 100,000 tickets per month. If 6% of responses require a 500-token repair call, that is 3,000 extra requests and 1.5 million avoidable tokens before counting the original context. Reducing malformed outputs from 6% to 1% saves 1.25 million repair tokens monthly. The exact dollar amount depends on the model, but the operational savings are consistent: fewer API calls, fewer queues, and fewer edge-case failures.

Use JSON Mode for Syntax, Schemas for Meaning

JSON mode generally instructs the model to produce valid JSON. It is valuable, but valid JSON alone does not guarantee that the response has the fields your system needs. A response like {"answer":"unknown"} may parse correctly while still failing your workflow.

Use a schema or an explicit output contract to define the required shape. For example, an order-routing task can require intent, priority, reason, and confidence. Restrict intent to a small set such as refund, shipment, product_question, or other. Restrict priority to integers from 1 through 5. Tight constraints reduce ambiguous answers and make downstream code simpler.

• Keep field names short and stable: use sku, not productStockKeepingUnitIdentifier.
• Use enums for categories rather than asking the model to invent labels.
• Set maximum string lengths for summaries, notes, and extracted text.
• Require null for unavailable values instead of allowing prose such as “not found.”
• Return identifiers and machine-readable values separately from display text.

Design Prompts That Avoid Expensive Retries

Put the output contract near the end of the instruction, after task context and business rules. Tell the model what to do when evidence is missing. For example: “Set confidence to 0 and reason to null when the source does not support a classification.” This prevents the model from filling gaps with explanations that break parsing or trigger human review.

Do not ask for chain-of-thought or lengthy reasoning when the application only needs a decision. Request a compact rationale instead: one sentence capped at 160 characters, or an array containing no more than three evidence IDs. A 300-token explanation across 100,000 requests adds 30 million output tokens per month. Limiting it to 60 tokens saves 24 million tokens while preserving an audit trail.

For extraction jobs, send only relevant source text. If a PDF contains 40 pages but the invoice total is on page 1, retrieve and submit that page rather than the full document. At 1,200 unnecessary input tokens per request and 50,000 documents per month, trimming context saves 60 million input tokens.

Validate Locally Before Calling the Model Again

Always validate the parsed response in application code. Check JSON parsing, required fields, enum membership, numeric ranges, and maximum lengths. Most invalid responses should not immediately trigger a full retry. First attempt deterministic recovery: strip accidental code fences, parse the JSON, and validate it. Then send one concise repair prompt containing only the invalid JSON and validation errors, not the entire original conversation.

Cap repair attempts at one for routine workflows. If the repair also fails, route the item to a fallback model, a rules-based path, or a review queue. Unlimited retries are a common source of surprise API bills. Track structured-output success rate, retry rate, average input tokens, average output tokens, and cost per completed task. A useful target is a first-pass valid-schema rate above 98% for stable production prompts.

Choose an Economical Model Path

Use the smallest model that meets the accuracy target. A practical pattern is to run classification, tagging, and straightforward extraction on a lower-cost model, then escalate only uncertain cases. For example, send records with confidence below 0.75 or failed validation to a stronger model. If 85% of requests complete on the lower-cost route, only 15% pay for premium reasoning.

59API is a practical option for this routing strategy because it provides pay-as-you-go access to native official-quality Claude models, including Opus, Sonnet, Haiku, and Fable, plus GPT models, without a model downgrade. Its OpenAI-compatible interface and API base URL, https://api.59api.com, let teams keep existing OpenAI SDK integrations while evaluating lower-cost model choices. It also works with Claude Code and Codex, which can reduce migration effort for developer workflows.

Start by measuring your current malformed-output rate and token use for one high-volume task. Then introduce JSON mode, a strict schema, local validation, and confidence-based escalation. Teams looking to reduce relay costs can sign up for 59API and use its referral rebate as an additional way to lower ongoing API spend.

मूल्य · दस्तावेज़