← Voltar aos guias

LLM API Rate Limits: A Retry Playbook for 2026

Handling rate limits and retries with LLM APIs requires more than catching HTTP 429. For production systems in 2026, build one coordinated policy for error classification, traffic pacing, retry deadlines, and recovery. Use this implementation checklist.

1. Classify failures before retrying

Inspect the HTTP status and structured error body. A 429 can mean temporary throttling or exhausted credits; only the former is normally solved by waiting. Authentication failures, invalid parameters, and billing problems need corrective action, not repeated requests. Connection failures and temporary server errors may qualify for bounded retries, subject to whether replaying the operation is safe. ([developers.openai.com](https://developers.openai.com/api/docs/guides/error-codes?utm_source=openai))

Record the error code, requested model, attempt number, and request ID. Keep credentials and sensitive prompts out of routine logs.

2. Respect Retry-After, then add jitter

When present and valid, Retry-After is your minimum waiting period. HTTP permits either delay-seconds or an HTTP date. Parse both formats rather than assuming every value is an integer. Add a small random delay after that minimum to spread recovering clients. ([rfc-editor.org](https://www.rfc-editor.org/rfc/rfc9110.html))

Without a server hint, use capped exponential backoff with full jitter. One illustrative policy chooses a random delay between zero and min(30, 2^n) seconds, where n starts at zero for the first retry. This produces expanding windows rather than synchronized retry waves. ([aws.amazon.com](https://aws.amazon.com/builders-library/timeouts-retries-and-backoff-with-jitter/))

Set explicit budgets: for example, three total attempts and a 60-second end-to-end deadline, including requests and sleeps. These are starting values, not provider limits. If Retry-After exceeds the remaining deadline, queue the job or return a recoverable error; never shorten the server's requested wait.

3. Avoid multiplying SDK retries

The official OpenAI Python SDK retries eligible failures twice by default. Wrapping it in three application-level attempts can therefore produce up to nine underlying attempts. Choose one retry owner: configure max_retries=0 when your application owns the policy, or keep SDK retries and account for them in the overall budget. Configure connection and read timeouts explicitly rather than relying on defaults. ([github.com](https://github.com/openai/openai-python/blob/main/README.md?plain=1&utm_source=openai))

4. Prevent throttling with shared admission control

Track requests per minute and token limits separately; some providers distinguish input and output token budgets. Limits can also apply across shared account or model groups. Bursty traffic may fail even when its minute-level average looks acceptable. ([developers.openai.com](https://developers.openai.com/api/docs/guides/rate-limits?utm_source=openai))

• Coordinate workers: use shared request and token counters across application instances.
• Reserve capacity: estimate each request's counted tokens before dispatch, then reconcile using returned usage and documented accounting rules.
• Bound concurrency: limit in-flight calls and place excess work in a bounded queue.
• Leave headroom: if an illustrative budget allows 60,000 tokens per minute and each request consumes 1,500 counted tokens, the ceiling is 40 requests per minute, even with a 120-request limit. Start below that ceiling.

5. Treat streaming and tool execution separately

A streaming response can fail after HTTP 200 has already arrived, so monitor stream events and terminal completion, not just the initial status. Do not silently append a fresh generation to partial text. Mark the answer incomplete and restart explicitly unless the API documents resumability. ([platform.claude.com](https://platform.claude.com/docs/en/api/errors))

A timeout does not prove an operation failed. Before retrying agents that send emails, charge customers, or modify records, persist tool execution IDs and deduplicate side effects. Use endpoint-supported idempotency where documented; an arbitrary header does not create that guarantee. ([aws.amazon.com](https://aws.amazon.com/builders-library/timeouts-retries-and-backoff-with-jitter/))

6. Measure successful work, not cheap attempts

Track attempts per completed task, queue age, 429 frequency, tail latency, and total token cost. Test missing headers, exhausted credits, repeated server errors, and interrupted streams before launch. Evaluate fallback models against your own quality checks rather than assuming interchangeable behavior.

7. Keep multi-model access cost-conscious

59API is a practical low-cost option to evaluate: its published offering combines pay-as-you-go token billing, OpenAI SDK compatibility, and access to models such as Claude Haiku 4.5 and GPT-6.1 Sol. It also offers a referral rebate. Compare current model-specific prices and measure cost per successful task, including retries. ([59api.com](https://59api.com/))

Before production rollout, verify which rate-limit headers the relay exposes and which quotas apply; client compatibility does not establish identical upstream limits. If that fits your workflow, sign up for 59API and benchmark a small workload with these safeguards enabled.

Preços · Docs