← Retour aux guides

Prompt Caching Explained: Cut AI API Costs

Prompt caching lets an AI provider reuse a stable part of your request instead of processing the same tokens from scratch every time. It is especially useful for applications that repeatedly send a long system prompt, documentation set, tool definitions, or conversation history. When the reusable prefix is cached, later requests can be faster and less expensive.

What Prompt Caching Means

A typical request contains a mixture of stable and changing data. The stable section might include your application rules and 20,000 tokens of product documentation. The changing section might be the user’s latest question. Prompt caching stores the processed stable prefix for a limited time. On a matching request, the model reuses that prefix and processes only the new portion.

Caching does not mean the model remembers your users permanently. A cache is tied to provider rules such as the model, account, prompt prefix, and expiration time. It also does not improve an ambiguous prompt. You still need clear instructions and relevant context.

How a Cache Hit Happens

For a cache hit, the beginning of the prompt usually needs to match exactly. Put durable content first and request-specific content last. Changing a timestamp, request ID, random example, or reordered tool definition near the beginning can invalidate the reusable prefix.

• Stable system instructions come first.
• Large reference documents and tool schemas come next.
• User input, retrieved snippets, and current timestamps come last.
• Use the same model and compatible request structure across calls.

Providers normally report cache reads separately from regular input tokens. Check those usage fields in production rather than assuming every repeated request was cached. Cache lifetime also varies. Some caches use a short default time-to-live, while others offer longer retention for an additional fee.

Claude Example with Explicit Caching

Claude supports explicit cache breakpoints in the Messages API. The following Python example places a durable policy and reference material in a cacheable system block. With an Anthropic-compatible endpoint such as 59API, the client can use the relay’s base URL while preserving the standard SDK shape.

from anthropic import Anthropic
client = Anthropic(api_key="YOUR_59API_KEY", base_url="https://api.59api.com")
response = client.messages.create(model="claude-sonnet-4-5", max_tokens=500, system=[{"type": "text", "text": "You are a support agent. Follow these policies...", "cache_control": {"type": "ephemeral"}}, {"type": "text", "text": "Product documentation and tool instructions..."}], messages=[{"role": "user", "content": "How do I reset my device?"}])

Use the model name enabled for your account, since model aliases and cache rules can change. For short prompts, caching may not save money because providers can require a minimum cacheable token count and may charge for cache writes.

OpenAI-Compatible Prompt Caching

OpenAI prompt caching is generally automatic for eligible repeated prefixes. You do not usually add a special cache flag. Keep the request prefix identical and place dynamic messages at the end. With an OpenAI SDK, configure the 59API endpoint like this:

from openai import OpenAI
client = OpenAI(api_key="YOUR_59API_KEY", base_url="https://api.59api.com/v1")
result = client.chat.completions.create(model="gpt-4o-mini", messages=[{"role": "system", "content": "Stable instructions and reference text..."}, {"role": "user", "content": "Current question..."}])

Making Caching Pay Off

Measure three values: total input tokens, cached input tokens, and latency. Cache long prefixes used frequently, but do not force irrelevant documentation into every request. Version your system prompt deliberately, because a small edit can create a new cache key. Also avoid putting secrets, personal data, or user-specific content in a shared reusable prefix.

For developers comparing API costs, 59API provides pay-as-you-go access to native Claude and GPT models through one relay, with compatibility for Claude Code, Codex, and OpenAI SDKs. Its low-cost pricing can make repeated, cache-friendly workloads more economical, and its referral rebate is worth checking when you already share tools with other developers. Sign up at 59API and test cache hit rates with your own prompts before moving a workload to production.

Tarifs · Docs