← Voltar aos guias

Prompt Caching: A 10-Minute Developer Quick Start

What is prompt caching?

Prompt caching lets an AI provider reuse computation for a previously processed prompt prefix. Instead of processing the same instructions, documents, or tool definitions from scratch on every request, the model can reuse cached internal state. This can reduce input-token costs and time to first token.

It is not response caching. The model still processes your new input and generates a fresh answer. For example, a coding assistant can reuse a long repository overview while answering different questions about that repository.

How prompt caching works

During inference, a model builds internal attention state, commonly called a key-value or KV cache. Provider-side prompt caching preserves reusable state across requests. When a later request begins with an eligible matching prefix, the provider can retrieve that state rather than recompute it.

Prefix matching is the important constraint. Reusing the same paragraph somewhere inside a different prompt does not necessarily produce a hit. Changes near the beginning can invalidate reuse for everything after them. Matching rules, minimum lengths, retention windows, and billing vary by provider and model.

1. Put stable content first

Arrange requests from least-changing to most-changing content:

• First: system instructions, security rules, output requirements, and stable tool definitions.
• Next: reusable reference documents, repository context, or product documentation.
• Last: the current user question, request-specific data, and other changing values.

Keep ordering and formatting consistent. Avoid inserting timestamps, random IDs, or changing user metadata before reusable content. In chat applications, append messages rather than rewriting earlier conversation history whenever practical.

2. Enable the right provider behavior

OpenAI: supported models generally apply prompt caching automatically once the prompt meets their minimum length. Many supported models start eligibility at 1,024 tokens, but check the selected model's current documentation. You usually do not need a cache-control block.

Claude: a common explicit approach is to mark a supported content block at the end of your reusable prefix with "cache_control": {"type": "ephemeral"}. A system text block can look like this: {"type":"text","text":"Your stable instructions and reference material...","cache_control":{"type":"ephemeral"}}.

Claude's minimum cacheable length depends on the model. Its standard ephemeral cache duration is commonly five minutes, with longer-duration options available in supported configurations. Cache writes can cost more than ordinary input, while cache reads cost less, so repeated reuse matters.

3. Connect through 59API

59API offers low-cost, pay-as-you-go access to Claude and GPT models through the relay host https://api.59api.com. It supports Claude Code, Codex, and OpenAI SDK workflows, making it convenient for developers who want native, official-quality models without downgrades.

Create an API key, store it in an environment variable, and configure your client's base URL and credentials. Follow 59API's endpoint instructions for your chosen client, including whether it expects a versioned path. Select a supported model identifier from its current catalog.

Client compatibility alone does not guarantee caching support. Confirm that your selected route forwards provider-specific cache controls and returns cache-usage fields. Also check 59API's cached-input pricing: upstream discounts do not automatically establish a relay's billing rates.

4. Verify hits before measuring savings

Send a request with a sufficiently long stable prefix, then repeat it within the retention window while changing only the final question.

• For OpenAI-style Chat Completions responses, inspect usage.prompt_tokens_details.cached_tokens when available.
• For Claude responses, inspect usage.cache_creation_input_tokens and usage.cache_read_input_tokens.
• Compare billed cost and time to first token across several requests, not just one pair.

If reads remain zero, check prefix differences, minimum length, cache-control placement, expiration, and route support. Do not add meaningless padding just to reach a threshold.

When is prompt caching worth it?

Start with repeated, substantial context: coding agents, document assistants, support bots, and shared tool schemas. Short, unrelated requests usually offer less benefit. Caching also does not eliminate output-token costs or make generated answers deterministic.

For an inexpensive trial, consider signing up for 59API and benchmarking one realistic repeated-context workflow. Its pay-as-you-go pricing keeps experimentation flexible, and its referral rebate offers another potential saving under the program's terms.

Preços · Docs