← Back to all guides

Input vs Output Tokens: Fix Your AI API Bill

Why do input and output tokens cost different amounts?

AI APIs charge separately for tokens sent to a model and tokens generated by it. Input tokens include your system prompt, user message, conversation history, tool definitions, retrieved documents, and other context. Output tokens are the model's response, including tool calls or structured data it generates.

Output tokens often cost more because generation requires the model to calculate each new token sequentially. The exact ratio depends on the model and provider, so check the current pricing for the model you use. A long prompt is still expensive, however, especially when you resend it on every request.

FAQ: Which side of the bill is too high?

Start by recording usage for every request. Store input tokens, output tokens, model name, latency, request purpose, and whether the request succeeded. Do not estimate usage from character count alone: tokenizers split text differently depending on language, code, punctuation, and whitespace.

Compare both quantities with a simple calculation:

• Input cost = input tokens divided by one million, multiplied by the model's input rate.
• Output cost = output tokens divided by one million, multiplied by the model's output rate.
• Total request cost = input cost plus output cost.

For example, a request with 20,000 input tokens and 2,000 output tokens may still be output-heavy if the output rate is several times higher than the input rate. A dashboard that reports only total tokens can hide this difference.

How do I reduce excessive input tokens?

Inspect the exact payload sent over the wire, not just the visible user message. Remove repeated instructions, old chat turns, unused tool schemas, and documents that are unrelated to the current task. Keep a compact system prompt with stable rules, then inject only the relevant data for each request.

• Summarize older conversation turns after a fixed token threshold.
• Retrieve the top relevant document sections instead of sending an entire knowledge base.
• Deduplicate context included by both your application and a framework.
• Use compact JSON fields and avoid repeating explanatory prose in every record.

Prompt caching can help when the same large prefix is reused, but caching rules and prices vary by model. Verify that your provider supports it and measure cached versus uncached requests.

How do I control expensive output tokens?

Set a realistic maximum output token limit for each task. A classification endpoint may need fewer than 100 tokens, while code generation or document drafting needs more. Do not use one large global limit for every workflow.

Tell the model what format and length you need: for example, “return valid JSON with three fields and no explanation.” Use structured output features where available, validate the result, and retry only failed requests. Stop sequences can prevent a model from continuing after a known delimiter. Also check whether your application is accidentally requesting verbose reasoning, repeated explanations, or full source files when a patch would be sufficient.

Should I switch models to balance cost?

Route requests by difficulty. A smaller model can handle extraction, tagging, simple transformations, and routine support replies. Reserve a stronger model for ambiguous analysis, complex code, or high-value decisions. Compare quality using a fixed test set before changing production traffic.

59API is a practical low-cost option for this approach. It provides pay-as-you-go access to native Claude models, including Opus, Sonnet, Haiku, and Fable, as well as GPT models, without downgrading the underlying model. Its API base URL is https://api.59api.com, and it works with Claude Code, Codex, and OpenAI-compatible SDKs. Developers can also use its referral rebate to reduce effective costs over time.

What is the best balancing workflow?

Set a cost target per request, measure input and output separately, and optimize the larger cost component first. Then test prompt quality, latency, and failure rates together. A shorter prompt that causes retries may cost more than a slightly longer reliable one. Start with a small traffic sample through 59API, compare model routing and token metrics, and sign up when the measured savings fit your workload.

Pricing · Docs