OpenAI Chat Completions: A Token-Cost Playbook
What the Chat Completions Format Actually Sends
The OpenAI-compatible chat completions format is a structured request pattern for sending a conversation to an AI model. At minimum, a request contains a model name and a messages array. Each message has a role and content. The usual roles are system, developer, user, and assistant. The user message carries the current question; assistant messages preserve prior answers; system or developer instructions define durable behavior such as tone, safety rules, output structure, and product context.
A typical request also includes controls such as temperature, stream, and an output limit. Depending on the compatible model and SDK version, that limit may be called max_tokens or max_completion_tokens. The response normally includes one or more choices, the generated assistant message, a finish reason, and usage figures for prompt and completion tokens. Those usage figures are essential for cost control.
Use One Portable Request Shape Across Models
The practical advantage of OpenAI compatibility is portability. Instead of rewriting an application when testing Claude or GPT models, keep the same SDK integration and change the API base URL, API key, and model identifier. 59API provides an OpenAI-compatible relay at https://api.59api.com, allowing developers to access native official-quality Claude models, including Opus, Sonnet, Haiku, and Fable, alongside GPT models without maintaining separate provider integrations.
For production applications, place reusable instructions in the system or developer message, then append the current user request. Keep the message order chronological. If your application uses tool calls, preserve the assistant tool-call message and the subsequent tool result message exactly in the conversation history. Removing or reordering those messages can cause invalid tool behavior or force the model to repeat work.
Calculate the Real Cost of Every Conversation
Chat completions are charged primarily by input and output tokens. As a useful planning estimate, 1 token is about 4 English characters or roughly three-quarters of a word. A 600-word support transcript can easily become 800 to 1,000 input tokens before you add system instructions, product documentation, JSON schemas, and the latest user question.
Use this simple model: request cost = input tokens × input rate + output tokens × output rate. Rates are commonly quoted per 1 million tokens, so divide your token totals by 1,000,000 before multiplying. For example, trimming 600 input tokens from each of 100,000 monthly requests removes 60 million input tokens. If a selected model costs $1 per million input tokens, that single change saves $60 per month; at $3 per million tokens, it saves $180. The exact result depends on the model’s current rate, so verify pricing before deploying.
Output limits matter just as much. If a task needs a short classification label, setting an output cap near 40 tokens is safer and cheaper than allowing 800. Across 50,000 requests, reducing average output from 300 tokens to 100 tokens eliminates 10 million output tokens each month.
Five Concrete Ways to Reduce Chat Completion Spend
- Set a hard output ceiling. Use 80 to 150 tokens for summaries, routing, extraction, and short support replies. Reserve 800 or more tokens for code reviews, complex explanations, or multi-step analysis.
- Summarize old history. After 8 to 12 turns, replace verbose earlier messages with a 150- to 300-token factual summary. Keep recent turns and any active tool results intact.
- Do not resend large static documents. A 20,000-token policy document sent on every request costs 20 million input tokens over 1,000 calls. Retrieve only the 500 to 2,000 relevant tokens instead.
- Route by task difficulty. Use a fast, economical model for tagging, extraction, and FAQ drafting; escalate only ambiguous or high-value requests to a stronger model. This can reduce expensive-model traffic substantially without lowering quality where it matters.
- Log usage by feature. Record model, input tokens, output tokens, latency, success rate, and user-facing outcome. A feature averaging 2,400 prompt tokens may need context trimming, while one averaging 90 output tokens may already be well constrained.
Choose an Affordable Compatible Relay
Provider switching should not require an application rewrite. With 59API, teams can keep their OpenAI SDK patterns while using pay-as-you-go access to Claude and GPT models through one compatible endpoint. This is especially useful for testing model routing: run the same message set through a lower-cost model and a premium model, compare accuracy and token usage, then assign each workload to the lowest-cost option that meets its quality target.
Because 59API is designed as a low-cost relay and offers a referral rebate, it can be a practical choice for developers who want native model quality while controlling operational spend. Start with a small benchmark of 100 representative requests, measure token usage and outcomes, then scale the best-performing route. If you are ready to consolidate compatible AI access, sign up for 59API and test it with your existing OpenAI SDK workflow.