What is context caching? Repeated prompt prefixes can cost far less

If every request starts with the same long block — a system prompt, a product manual, tool definitions — models that support context caching store that prefix. When the same prefix comes back, the matched part is billed at the cache-read price. Take gpt-5.4-mini: normal input is ¥1.5 per 1M tokens, cache reads only ¥0.15. The longer the fixed prefix and the more often you call, the more you save.

How caching works

To read a prompt, a model runs a computation over every token. If two requests begin with the same long block, the second can reuse the first one's results, and the saved compute shows up in the price.

That is why caching matches on the prefix: only text that is identical from the very first token counts as a hit. Change one character in the middle and nothing after it can be matched.

How each vendor caches

These are the vendors' published general rules. Minimum lengths, lifetimes and other details change, so check each model's official documentation. Whether a model has a cache price shows in its cache-read column in Models and pricing.

Reading cache prices

Here is Claude Sonnet 5.5 in every group:

In its cheapest group, a cache write costs ¥4 per 1M tokens, more than normal input, but every later hit is billed at the cache-read price. The more a prefix is reused, the better the deal.

Input and cache-read prices for a few other common models (per 1M tokens, cheapest group):

• deepseek-v4.1-flash: input ¥2.8, cache read ¥0.056
• gemini-3.8-flash: input ¥1.5, cache read ¥0.15
• gpt-5.5: input ¥10, cache read ¥1

How much it saves

A support bot sends a fixed 6,000-token system prompt plus product material with every request, about 1,000 tokens of question and recent turns, and gets a 400-token answer, 100,000 times a month.

Without cache hits:

With the fixed 6,000 tokens served from cache:

Real hit rates stay below 100%: expired entries and edited prefixes send some requests back to full price. Once you are live, compute your actual hit rate from the cached tokens in usage and budget with that.

Writing prompts that hit the cache

1. Put what never changes first and what changes last. A good order: system prompt, tool definitions, fixed material, chat history, the current question.
2. Keep values that change on every request — current time, request IDs, user names — out of the system prompt. If you need them, put them in the last user message.
3. Keep tool definitions and examples in a fixed order, and serialize JSON with a stable key order.
4. Send similar requests close together. Cache entries expire if they are not hit, and writing them again costs money.
5. Claude needs explicit marking: put cache_control on the last block of the fixed content.

import os
import anthropic

client = anthropic.Anthropic(base_url="https://open.tutujin.com", api_key=os.environ["TUTUJIN_API_KEY"])

with open("product_manual.md", encoding="utf-8") as f:
    manual = f.read()  # several thousand tokens of fixed material

resp = client.messages.create(
    model="claude-sonnet-5-5",
    max_tokens=1024,
    system=[
        {"type": "text", "text": "You are product support. Answer only from the manual below."},
        {"type": "text", "text": manual, "cache_control": {"type": "ephemeral"}},
    ],
    messages=[{"role": "user", "content": "How long is the warranty?"}],
)
print(resp.usage.cache_creation_input_tokens, resp.usage.cache_read_input_tokens)

The first request writes the cache and reports cache_creation_input_tokens; later requests within the cache lifetime hit it and report cache_read_input_tokens.

Checking for hits

When you call other vendors' models through the OpenAI-compatible API, whether the cache field is returned depends on the model. The Usage page in the console is the record of what each request actually cost. For savings beyond caching, see How to control API costs.