Prompt Caching: A 10-Minute Developer Quick Start
What is prompt caching?
Prompt caching lets an AI provider reuse computation for input it has already processed. If your requests repeatedly include the same instructions, tool definitions, or reference document, the provider may reuse that work instead of processing the entire input again.
For developers, the potential benefits are lower input-token charges and faster time to first token. Caching is especially useful for coding assistants, document chat, and agents with large, stable system prompts. It does not eliminate output-token charges or guarantee faster token generation.
How prompt caching works
Before generating an answer, a language model processes your input during a stage called prefill. Prompt caching commonly preserves reusable internal state from this stage, often involving attention key/value data. A later request with a matching prefix can reuse that state and process only the new material.
This is not response caching. The model still generates a fresh answer. It is also different from semantic caching, which retrieves an earlier answer to a similar question.
Providers generally expose caching in one of two ways:
- Automatic prefix caching: eligible repeated prefixes are detected without a separate cache-creation request.
- Explicit caching: you mark cacheable boundaries or create a cached-content resource and reference it in later requests.
Minimum token counts, expiration periods, pricing, and supported inputs vary by provider and model. A repeated prompt is not automatically a billable cache hit.
Step 1: Put reusable content first
Arrange requests from most stable to most variable. A practical order is system instructions, fixed tool definitions, reference material, conversation history, and finally the newest user message.
For example, a support assistant might send the same 10,000-token product handbook followed by a different 500-token question. The handbook is the reusable prefix; the question is new input.
- Keep shared instructions identical between requests.
- Keep tool definitions and their ordering stable.
- Move timestamps, request IDs, and personalized details after shared content.
- Avoid adding irrelevant text merely to reach a caching threshold.
With prefix caching, a change near the beginning can prevent reuse of everything after that point. Similar meaning is not enough: treat an exact, provider-compatible prefix as the target.
Step 2: Confirm support on your API route
59API offers low-cost, pay-as-you-go access to native official-quality models without downgrading them. Its base URL is https://api.59api.com, and it works with Claude Code, Codex, Gemini CLI, and OpenAI SDKs.
You could evaluate routes for Claude Sonnet 5.5 or GPT-6.1 Sol, depending on your workload. Before implementation, confirm whether the selected route supports automatic caching, explicit cache controls, or neither. SDK compatibility alone does not guarantee that every provider-specific caching feature is forwarded.
Check the route’s endpoint configuration, cache eligibility rules, expiration behavior, usage reporting, and actual cached-input prices. Do not assume discounts advertised by an upstream provider apply unchanged through a relay.
Step 3: Run a controlled two-request test
- Send a request containing a substantial, stable prefix and a short question.
- Send another request soon afterward with the identical prefix and a different final question.
- Inspect both responses’ usage metadata and the billing records.
- Repeat several times to distinguish cache behavior from ordinary latency variation.
Depending on the API, useful fields may include prompt_tokens_details.cached_tokens or cache_read_input_tokens. Field names and availability vary. Record cache-write tokens or charges separately when reported; the first request may cost more than subsequent reads.
Step 4: Measure savings, not just hits
Suppose a request contains 10,000 cached input tokens and 500 uncached input tokens. If, purely illustratively, cached input costs 20% of the regular rate, its input cost equals 2,500 regular-rate tokens: 10,000 × 0.20 + 500. That excludes output, cache writes, storage, and other fees.
Track total cost per task, cached-token share, and time to first token. Keep an uncached baseline. Cache expiration, eviction, model changes, and early-prefix edits can all reduce reuse, so requests must still work on a miss.
Check cache isolation and retention rules before sending sensitive data. For an inexpensive starting point, sign up for 59API and benchmark one repetitive workload against its actual usage and billing data.