← Voltar aos guias

Prompt Caching Explained: Fix Misses and Cut API Cost

What is prompt caching?

Prompt caching stores a reusable part of an AI request so the model provider does not need to fully reprocess identical input every time. This is especially useful when requests repeatedly include a large system instruction, tool definition, documentation set, policy file, or codebase context.

On a cache hit, the provider can reuse the stored prompt prefix and process only the new portion of the request. Depending on the model and API, cached input may be charged at a lower rate, processed faster, or both. Prompt caching is different from response caching: it usually reuses input context, not the model's final answer.

How does prompt caching work?

Most implementations create a cache key from an ordered portion of the request. The key can include the model, system message, tools, and earlier messages. If a later request begins with the same cacheable content, the provider can match it to the stored prefix.

• Stable prefix: Put instructions, reference material, and tool schemas that rarely change at the beginning.
• Dynamic suffix: Put the user's current question, timestamps, random IDs, and conversation-specific data after the stable content.
• Expiration: Cached entries normally have a provider-defined time-to-live. A cache hit is not permanent storage.
• Provider rules: Minimum token sizes, supported models, cache controls, and billing behavior differ between Claude, GPT, and other APIs.

Some APIs automatically detect reusable prefixes. Others require an explicit cache marker or cache-control setting. Always check the documentation for the exact model and endpoint you are calling.

Why is my prompt cache not hitting?

The most common problem is that requests are not identical where the cache key is calculated. Even a small change can prevent a match.

• Move changing values out of the beginning of the prompt.
• Keep system instructions and tool definitions in the same order.
• Do not insert the current date, request ID, user name, or random text into a supposedly static prefix.
• Use the same model identifier and compatible API settings for repeated calls.
• Confirm that the prompt is larger than the provider's minimum cache threshold.
• Allow for expiration and cold starts after deployment, scaling, or a long idle period.

Whitespace and serialization can also matter. Generate structured tool definitions consistently, avoid rebuilding JSON with unpredictable key ordering, and compare the actual serialized requests rather than only the source objects.

How can I verify a cache hit?

Inspect the API response and usage fields. Providers may report cached input tokens, cache creation tokens, cache read tokens, or a similar metric. A response containing only total input tokens is not enough to prove that caching occurred.

Test with three requests: send the same large prefix twice, change one character in that prefix, and then send the original request again after the expected TTL. Record latency, token usage, and cost. This separates a real cache hit from normal variation in model response time.

Does prompt caching work with Claude and GPT?

It can, but support is model- and API-specific. Claude APIs commonly expose explicit prompt-cache controls, while GPT-compatible endpoints may offer different caching behavior or automatic prefix caching. Do not assume that a feature available in one provider's native API is available through every OpenAI-compatible relay.

When using 59API at https://api.59api.com, check the current endpoint and model documentation for cache-control parameters and returned usage fields. 59API provides pay-as-you-go access to official-quality Claude Opus, Sonnet, Haiku, Fable, and GPT models, and works with Claude Code, Codex, and OpenAI SDK integrations. That makes it practical to test caching without committing to a large fixed plan, while keeping model behavior intact rather than using a downgraded substitute.

Can prompt caching reduce my API bill?

Yes, when your workload repeatedly sends the same large context and the provider gives cached input a lower price. Calculate the result using your own traffic: identify the repeated prefix size, cache-hit rate, cache-read price, cache-write price, and TTL. Caching helps less when prompts are short, highly unique, or rarely repeated.

To control costs, cache stable instructions and reference content, trim obsolete conversation history, monitor cache-read and cache-write tokens, and set a budget alert. If you want a low-cost way to compare Claude and GPT usage, sign up for 59API and run a small controlled test before moving production traffic.

Preços · Docs