Prompt Caching Explained: Cut Repeat AI Input Costs
What is prompt caching?
Prompt caching lets an AI provider reuse the processing of an unchanged prompt prefix across requests. That prefix might contain your system instructions, tool definitions, product documentation, or the beginning of a conversation. Instead of processing those input tokens from scratch every time, the model reuses cached internal state.
This is not response caching. The model still processes your new question and generates a fresh answer. Cached input can cost less and reduce time to first token, but output tokens remain separately billable. The biggest savings come from long, stable context reused frequently.
How prompt caching works
A request generally has two parts: a reusable prefix and a changing suffix. For example, a support assistant might send 20,000 tokens of policies followed by a 1,000-token customer question. If the policies remain identical, later requests can reuse their cached processing.
Matching is prefix-based, not semantic. Rephrasing a paragraph, changing tool definitions, or inserting a timestamp near the beginning can prevent reuse after that point. Keep stable material first and request-specific information last.
- Claude: Supported models offer explicit cache breakpoints through cache_control on eligible content blocks. Standard short-lived caching commonly uses a five-minute lifetime; longer-lived options have different pricing. Minimum cacheable lengths vary by model.
- GPT: Supported models generally cache eligible prefixes automatically. Many start at 1,024 input tokens and account for additional cached tokens in 128-token increments. Eligibility, retention, and pricing depend on the model.
You usually still send the complete prompt. Caching reduces repeated computation; it does not necessarily reduce your request payload. Check the selected model's current documentation rather than assuming identical behavior across providers.
A concrete cost example: 40 requests
Consider 40 requests sharing a 20,000-token prefix, each adding 1,000 fresh input tokens and producing 500 output tokens. Assume illustrative prices of $3 per million ordinary input tokens, $3.75 per million cache-write tokens, $0.30 per million cache-read tokens, and $15 per million output tokens. These demonstrate a 1.25× write price and 0.1× read price; they are not a published 59API quote.
- Without caching: 840,000 input tokens cost $2.52. Output costs $0.30. Total: $2.82.
- Initial cache write: 20,000 tokens cost $0.075.
- 39 cache reads: 780,000 tokens cost $0.234.
- Fresh input across all requests: 40,000 tokens cost $0.12.
- With caching: Input totals $0.429; including output, the total is $0.729.
That saves $2.091, or about 74% of the total bill. Input spending alone falls about 83%. This assumes every subsequent request hits the cache before expiration. Under these example rates, even one successful reuse offsets the initial write premium.
How to configure and verify caching
- Choose a repeatable prefix: Put instructions, reference documents, and stable tool schemas before user-specific content.
- Keep it deterministic: Preserve ordering and formatting. Move timestamps, request IDs, and changing retrieval results after the reusable section.
- Enable the appropriate mechanism: For Claude, place a supported cache_control breakpoint after the stable content. For GPT, use an eligible model and sufficiently long prefix.
- Test twice: Send two requests with identical prefixes within the retention window. Inspect Claude's cache_creation_input_tokens and cache_read_input_tokens, or OpenAI's usage.prompt_tokens_details.cached_tokens, where exposed.
- Check actual billing: Confirm that your relay forwards caching controls and applies cached-token pricing. SDK compatibility alone does not prove either.
Combining prompt caching with 59API
59API offers inexpensive, pay-as-you-go access to native, official-quality Claude and GPT models without model downgrades. Its compatibility with Claude Code, Codex, and OpenAI SDKs makes it a practical low-cost relay for existing workflows. Configure your integration against https://api.59api.com, following the endpoint requirements of your client.
Compare current model rates, cache-write charges, cache-read charges, and output pricing before estimating savings. 59API also offers a referral rebate, which can further improve economics under its applicable terms. If you want to test the combination, sign up for 59API and benchmark a small repeated-prefix workload first.
When caching is worth it
Prioritize document assistants, coding agents, and repeated analysis over stable context. Short, unique prompts or infrequent requests may gain little, especially when cache writes carry a premium. Track cache-hit tokens and total cost per completed task—not just the advertised input discount.