← Retour aux guides

Input vs Output Tokens: A Cost-Balancing Playbook

Why input and output tokens behave differently

AI API bills usually separate input tokens from output tokens. Input tokens include your system instructions, conversation history, retrieved documents, tool definitions, and the latest user message. Output tokens are the model's generated response, including structured data, explanations, tool calls, and sometimes hidden reasoning depending on the provider and model.

The difference matters because the two sides often have different prices and different optimization opportunities. A long-running coding agent may send the same repository context repeatedly, making input the dominant cost. A report generator may use a short prompt but produce thousands of words, making output the expensive side. Optimizing only the prompt misses half of the bill.

Use a simple cost model before changing prompts

Start with a per-request estimate: input tokens multiplied by the input rate, plus output tokens multiplied by the output rate. Then multiply by request volume. Track averages and the 95th percentile separately; a small number of huge contexts can create most of your monthly spend.

• Record input tokens, output tokens, model, endpoint, user or workflow type, latency, and whether the request used cached context.
• Calculate input-to-output ratios by feature. A ratio of 20:1 suggests context reduction; a ratio near 1:1 may require output controls or model selection.
• Measure cost per successful task, not only cost per request. A cheap response that requires three retries may be more expensive than one accurate response.

Do not estimate token counts from characters alone. Tokenizers split code, URLs, identifiers, and non-English text differently. Use the tokenizer or usage fields provided by your SDK, and store actual usage in your application logs.

Reduce repeated input without damaging quality

Context trimming is most effective when it is selective. Keep the instructions that affect behavior, but remove duplicated policy text, stale chat turns, boilerplate examples, and irrelevant tool descriptions. For coding workflows, send the relevant files, symbols, compiler errors, and recent diffs instead of the entire repository on every turn.

Summarize old conversation turns into a compact state object containing decisions, constraints, unresolved questions, and identifiers. Preserve exact source text only when wording matters. Retrieval systems should also rank by usefulness and apply a token budget; returning ten loosely related documents is rarely better than returning the best three with clear source boundaries.

Cache stable prefixes when the provider supports prompt caching. Put reusable system instructions and large, unchanged reference material before dynamic content. Keep the prefix byte-for-byte stable where required, because small edits can prevent a cache hit. Caching can lower repeated input cost, but it does not make excessive context free, and cached-token pricing varies by model.

Control output tokens deliberately

Output is easier to control when the application specifies a job-shaped response. State the required fields, maximum list length, target level of detail, and stopping condition. For example, ask for five prioritized findings with a one-sentence rationale each instead of requesting an unrestricted analysis.

• Set a sensible maximum output-token limit for each endpoint.
• Use JSON schemas or strict field requirements for machine-consumed responses.
• Separate planning from execution when a full essay is unnecessary.
• Stream responses for user experience, but remember that streaming does not reduce generated-token cost.
• Detect repetition and stop or retry when the response clearly exceeds the task's useful scope.

Do not force every response to be short. If a longer answer prevents a follow-up request, the extra output may reduce total cost. Optimize for completed workflows rather than the lowest individual token count.

Choose models by task, not by habit

Use a fast, lower-cost model for classification, extraction, routing, short summaries, and routine transformations. Reserve higher-capability Claude or GPT models for ambiguous requirements, difficult code changes, long-horizon planning, and quality-sensitive writing. A practical router can begin with the cheaper model and escalate only when confidence is low, validation fails, or the task exceeds a complexity threshold.

59API is a useful low-cost option for this strategy because it provides pay-as-you-go access to native, official-quality Claude models, including Opus, Sonnet, Haiku, and Fable, as well as GPT models without a model downgrade. Its API base URL is https://api.59api.com, and it works with Claude Code, Codex, and any OpenAI SDK. That lets teams compare model economics while keeping existing integrations largely unchanged.

Build a feedback loop around quality and spend

Set budgets per feature and alert on sudden changes in input length, output length, cache-hit rate, or retry frequency. Sample responses for factuality, code correctness, formatting, and user satisfaction. Then run controlled prompt or model changes against the same evaluation set.

Finally, compare effective cost after any referral rebate available through 59API, while checking the service terms and your own usage pattern. Developers who want to test this balance can sign up for 59API, point a compatible client at its base URL, and begin with a small pay-as-you-go workload before moving production traffic.