← Retour aux guides

Prompt Caching Explained: Cache Misses, Costs & FAQs

What is prompt caching?

Prompt caching lets an AI provider reuse previously computed processing for repeated prompt content. If your requests repeatedly include the same instructions, tool definitions, or reference document, caching can reduce the work needed to process that input. Depending on the provider, this can lower input costs and improve time to first token.

Prompt caching does not mean returning a saved answer. The model still generates a response for each request. Output tokens remain billable under the applicable pricing, and cached content still occupies space in the model’s context window.

How does prompt caching work?

Many implementations reuse processing for an identical prompt prefix: the beginning of the input, up to a supported cache boundary. Providers may enable this automatically or require explicit cache controls. Minimum eligible lengths, retention periods, pricing, and supported content vary by model and endpoint.

Consider a support assistant with a long product manual followed by a customer’s question. Keeping the manual unchanged at the beginning gives later requests a reusable prefix. Placing a changing timestamp before the manual can prevent reuse of everything after that timestamp.

For workloads using Claude Sonnet 5.5 or GPT-6.1 Sol, confirm the selected route’s caching support before relying on savings. A model name or compatible API format alone does not establish the caching behavior.

How can I structure requests for cache hits?

• Put stable content first. Start with persistent instructions, fixed tool definitions, and reusable reference material. Place the current question and request-specific data afterward.
• Keep serialization consistent. Preserve message order, whitespace, tool ordering, and document formatting. Avoid regenerating equivalent content with different formatting.
• Remove unnecessary variation. Move request IDs, timestamps, and changing metadata out of the reusable prefix.
• Check eligibility. Verify the provider’s minimum token threshold, supported message types, cache boundaries, and retention rules.
• Repeat a controlled request. Send the same substantial prefix twice within the retention window, then inspect usage metadata. Keep the model, endpoint, and relevant settings consistent.

Why am I getting cache misses?

The prefix changed. Compare the actual serialized requests, including system instructions and tool schemas. A small early change can leave only a shorter prefix eligible for reuse.

The input is too short or the cache expired. Short prompts may fall below the caching threshold. A repeat request outside the retention window may need to create the cache again.

The route handles caching differently. A relay may expose different providers or deployment paths. Check whether your route passes through cache controls and reports cache usage. SDK compatibility does not guarantee identical caching semantics.

The conversation was rearranged. Editing earlier messages or rebuilding history in a different order can invalidate the reusable prefix. Appending new messages generally preserves more of the beginning.

How do I verify savings?

Read the response’s usage fields before comparing invoices. OpenAI-style responses may report prompt_tokens_details.cached_tokens. Anthropic-style responses may expose cache_read_input_tokens and cache_creation_input_tokens. Availability depends on the endpoint; missing fields leave caching unconfirmed.

Calculate uncached input, cache reads, cache writes where applicable, and output separately. For example, if 80% of input receives a hypothetical 90% cache-read discount, input cost falls by 72% before write charges. Total request savings will be smaller when output accounts for a large share of the bill.

Does caching always make responses faster?

No. Reusing a long prefix can shorten input processing, but network latency, queueing, and response length still matter. Measure time to first token separately from total completion time across several comparable requests.

Can I use prompt caching through 59API?

59API provides low-cost, pay-as-you-go access to native, official-quality models without downgrades. Its API base is https://api.59api.com, with compatibility for Claude Code, Codex, Gemini CLI, and OpenAI SDKs. That makes it a practical option for testing repeated-context workloads while controlling spend.

Confirm cache support, usage reporting, and billing for your chosen route, then compare the total cost of representative requests. Sign up for 59API to try your workload; a referral rebate is also available.

Tarifs · Docs