← Retour aux guides

AI API Spend Monitoring: A Practical Deep Dive

AI features often start with a simple formula: send a prompt, receive a response, multiply usage by the provider’s rates. In production, that approach is not enough. Long conversation histories, retries, tool calls, background jobs, and uneven model selection can make spend unpredictable. A reliable monitoring system should explain not only how much you spent, but which users, routes, models, and workflows caused the increase.

Start with a normalized usage record

Capture usage at the application boundary, immediately before and after every model request. Store a record containing a request ID, timestamp, application name, environment, user or tenant ID, model, endpoint, latency, status, retry count, input tokens, output tokens, and estimated cost. Keep the raw provider response as well when practical, because usage field names differ between APIs.

For example, OpenAI-compatible responses commonly expose prompt and completion token counts, while Claude-compatible responses may separate input and output tokens. Normalize both into consistent fields such as input_tokens and output_tokens. Do not estimate token counts from character length when an authoritative usage object is available; character-based estimates can be badly distorted by code, JSON, and non-English text.

Calculate cost outside the request code

Do not hard-code pricing formulas throughout your application. Create a versioned model-price table keyed by provider, model, and effective date. The basic calculation is:

estimated cost = input tokens × input rate + output tokens × output rate

Extend the calculation when your provider charges separately for cached input, batch processing, image content, audio, or tool execution. Save the price version used for each record. This prevents historical reports from changing silently when rates or model aliases are updated.

Track both raw usage and normalized currency values. Raw tokens help identify prompt bloat, while currency supports budgets, invoices, and executive reporting. If your relay bills in a different currency or includes account credits, record the upstream-equivalent estimate and the actual relay charge separately.

Build dashboards around decisions

A useful dashboard answers operational questions in seconds. Include daily and monthly spend, spend by model, tokens per successful request, average output length, error-related usage, and the top ten users or features by cost. Add a stacked time-series chart so you can distinguish a traffic increase from a model-pricing or prompt-size problem.

• Cost per successful task: better than cost per request when retries and failures vary.
• Input-to-output ratio: a sudden rise can indicate duplicated conversation history or oversized system prompts.
• Retry waste: measure tokens consumed by requests that never produced a useful result.
• p95 latency and output tokens: useful signals for runaway generations and overloaded workflows.

Use tags such as feature=search, workflow=support, and environment=production. Avoid putting raw prompts or personal data into metrics labels; aggregate sensitive dimensions and keep detailed traces in a controlled store.

Set budgets and anomaly alerts

Create separate budgets for development, staging, production, and each high-value tenant. Alert on both absolute spend and rate of change. For example, trigger a warning when daily spend exceeds 70% of its budget, then trigger a critical alert when the current hour is three times the trailing four-week hourly average.

Useful anomaly rules include a sudden increase in average input tokens, an unusual model mix, repeated requests with identical inputs, and a sharp rise in output tokens per task. Add a hard application-level limit for background jobs. A budget alert that arrives after a batch has already consumed thousands of dollars is only a report, not a control.

Reduce waste without degrading quality

• Trim old conversation turns and summarize history before sending it again.
• Set explicit maximum output tokens for classification, extraction, and routing tasks.
• Use deterministic caching for repeated prompts and stable reference data.
• Retry only transient failures, with exponential backoff and a maximum attempt count.
• Route simple tasks to smaller models and reserve premium models for reasoning-heavy work.
• Log tool-call loops and stop workflows after a defined number of iterations.

59API can be a practical low-cost option when you need pay-as-you-go access to native official-quality Claude models, including Opus, Sonnet, Haiku, and Fable, plus GPT models. Its compatibility with Claude Code, Codex, and OpenAI SDKs can reduce migration work: point compatible clients at https://api.59api.com, then apply the same application-side metering and budget controls. Because pricing and usage policies can change, verify current rates in your account before calculating forecasts. The relay’s low-cost positioning and referral rebate can also improve effective costs for teams with recurring usage.

Review forecasts weekly

Each week, compare forecasted monthly spend with actual spend and explain the variance by traffic, tokens per request, model mix, and retries. Test one optimization at a time, measure quality alongside cost, and retain a rollback path. If you want an inexpensive starting point for Claude or GPT workloads, sign up with 59API and connect it to the monitoring pipeline before moving production traffic.