← Retour aux guides

Balance Input and Output Token Costs in AI Apps

Input vs Output Tokens: The Cost Model You Actually Need

Every LLM request has two billable parts: input tokens are the system prompt, chat history, retrieved documents, and current user message; output tokens are the model's generated response. Output tokens often cost more per token, but input costs can become the larger problem when an application repeatedly sends long conversations or large RAG context.

The practical goal is not simply to minimize tokens. It is to spend tokens where they improve the result: enough input context for correct answers, and enough output budget for a complete answer without paying for unnecessary prose.

Start With a Per-Request Token Budget

Define a maximum input and output budget for each workflow before writing prompts. A support-answer endpoint might allow 3,000 input tokens and 350 output tokens. A code-review job may need 12,000 input tokens and 1,500 output tokens. These limits give you a concrete basis for model selection, observability, and cost alerts.

For OpenAI-compatible clients, explicitly cap generated tokens. 59API works with standard OpenAI SDK integrations through https://api.59api.com, so this approach can be used with compatible GPT and Claude model endpoints without maintaining separate client code.

Example request settings:

base_url = "https://api.59api.com/v1"
model = "claude-sonnet"
max_tokens = 500

A 500-token ceiling prevents one verbose response from consuming the budget intended for several routine requests. Set the limit based on the UI: a short answer panel needs far fewer output tokens than a document-generation workflow.

Reduce Input Tokens Without Removing Useful Context

Input-token savings are usually won by controlling history and retrieval. Do not resend every past message forever. Keep the latest turns verbatim, summarize older decisions, and retain structured facts such as account tier, repository name, or selected language in a compact state object.

• Keep the system prompt specific and short. Replace repeated policy text with concise rules.
• Trim chat history to the turns relevant to the current task.
• Retrieve only the top relevant document chunks, rather than attaching an entire knowledge base.
• Store a rolling conversation summary after several turns.
• Send source code diffs or focused files instead of a full repository snapshot.

For example, a coding assistant should send the failing function, stack trace, dependency versions, and relevant interface definitions. Sending 20 unrelated files increases cost and can lower answer quality by burying the actual issue.

Control Output Cost Through Prompt Design

Use an explicit output shape. Ask for a maximum number of steps, bullets, fields, or lines. This is more reliable than merely saying "be concise," and it reduces the chance of an answer ending mid-task because the token cap was too low.

For a classifier, request a single label and confidence. For extraction, ask for named fields only. For code repair, request a patch plus a short explanation. Reserve long output budgets for work that truly needs them, such as migration plans, test generation, or customer-facing drafts.

A useful instruction is: Return the diagnosis, the smallest safe code change, and up to three verification steps. It constrains output while preserving the information a developer needs to act.

Choose the Model for the Expensive Side of the Request

Model choice should follow the dominant cost and quality requirement. If a request contains a large input context but needs a short answer, prioritize a model with strong long-context comprehension and economical input pricing. If a request has little context but produces long content, compare output prices and writing quality carefully.

Use a lower-cost model for routing, classification, tagging, summaries, and straightforward extraction. Escalate only difficult reasoning, complex code changes, or high-stakes final responses to stronger models. With 59API, developers can access official-quality Claude models including Opus, Sonnet, Haiku, and Fable alongside GPT models on a pay-as-you-go relay, making this routing strategy practical without a large commitment.

Measure Before You Optimize

Log model name, input tokens, output tokens, latency, task type, and whether the response was accepted or required a retry. Then calculate cost per successful outcome, not just cost per request. A cheap response that causes two retries is not actually cheap.

Review the data weekly. Look for endpoints with growing input size, unusually long completions, and repeated requests. Tighten retrieval, tune max output tokens, or route the workflow to a more appropriate model. 59API is a low-cost choice for testing these combinations because it offers cheap pay-as-you-go access, native model quality, and a referral rebate. Sign up when you are ready to benchmark your real workloads and set budgets from actual usage data.

Tarifs · Docs