Prompt Caching Explained: How Claude, OpenAI, and Gemini Actually Cut Your LLM API Costs

If your team shipped an LLM feature and the API bill is bigger than expected, the cause is almost never the model being too expensive per token. It's that you're paying full price to reprocess the same system prompt, the same tool definitions, and the same document context on every single request, when most of that text didn't change since the last call. Prompt caching is the fix, and all three major providers now support some version of it, but they implement it differently enough that copying an OpenAI pattern into a Claude integration will quietly cost you money instead of saving it.

What's actually being cached

When a transformer model processes a prompt, it computes attention key-value (KV) tensors for every token in that prompt before it can generate anything. That computation is the expensive part. Prompt caching stores those computed KV tensors server-side, keyed to a specific prefix of your prompt, so that the next request sharing that same prefix can skip recomputing it and load the cached result instead. It's an infrastructure optimization on the provider's servers, not a change to the model's behavior or output quality.

The catch is the word "prefix." Caching only works on an exact, unbroken match from the start of the cached block. If you change a single character anywhere before your cache boundary, timestamp, user ID, retrieved document order, anything, the whole cache miss cascades and you pay full price again. This is the single most common way teams accidentally cache nothing while believing they've implemented it correctly.

Claude: you decide what's cached and for how long

Anthropic's approach is opt-in. You mark a block of your prompt with a cache_control field, and the system caches everything up to and including that marker. You get two TTL options: a 5-minute cache at a 1.25x write premium over the base input rate, or a 1-hour cache at a 2x write premium, both refreshed automatically on every cache hit within the window. Reads cost 10% of the base input rate regardless of which TTL you chose. According to Anthropic's own pricing documentation, on Claude Sonnet 5 that's $2 per million tokens for a fresh request, $2.50 per million to write a 5-minute cache entry, and $0.20 per million to read it back, meaning the write premium is recovered after exactly one cache hit.

Minimum cacheable length varies by model, from 512 tokens on Claude Opus 5 up to 4,096 on Haiku 4.5 and Haiku 3.5, and Anthropic's prompt caching documentation is explicit that shorter blocks silently fail to cache with no error thrown, you have to check cache_creation_input_tokens and cache_read_input_tokens in the response to confirm it actually happened. Anthropic also documents the exact failure mode worth memorizing: placing the cache marker on content that changes every request, like an inline timestamp, guarantees a miss on every call because the hash of everything up to that marker never repeats.

OpenAI: automatic, which cuts both ways

OpenAI's implementation requires no code changes at all. Per OpenAI's prompt caching guide, the API hashes the beginning of every incoming prompt, routes matching requests to a server that already holds that prefix in memory, and applies the cached discount automatically whenever it finds a match, no cache_control field, no opt-in flag. The minimum prefix length to qualify is 1,024 tokens. On GPT-5.6 and newer, cache writes now carry a 1.25x premium over standard input pricing (older models wrote to cache for free), while reads get a steep discount versus a fresh request, mirroring the Claude tradeoff without you having to configure it. The practical downside of "automatic" is that you also lose the ability to force what gets cached, or extend a cache's lifetime past whatever OpenAI's infrastructure happens to retain, which matters if your traffic pattern is bursty rather than steady.

Gemini: implicit by default, explicit when you need control

Google splits this into two mechanisms. Implicit caching is on by default for Gemini 2.5 and newer models with no setup required, per Google's context caching documentation, and applies automatically to matching prefixes across both stateful and stateless requests. Explicit caching lets you create a named cache object yourself and reference it by ID across calls, useful when you want a guaranteed-available cache rather than a best-effort one, at the cost of paying an hourly storage fee on top of the discounted read rate. Per Google's published pricing, Gemini 2.5 Flash charges $0.03 per million cached tokens against a $0.30 standard input rate, a 90% discount, plus $1.00 per million tokens per hour for explicit cache storage. Gemini 2.5 Pro follows the same 90% discount ratio at higher absolute prices. Minimum prefix length is 2,048 tokens on both 2.5 models. This is the same underlying "don't recompute what you already computed" idea our AI agent memory comparison touches on from a different angle, memory systems persist facts across sessions, caching persists computation within a session.

Where caching quietly backfires

There are three ways teams implement this and end up paying more, not less.

• Placing the cache breakpoint after content that changes on every call, which produces a permanent cache miss and, on Claude and GPT-5.6+, an unnecessary write premium on top of it
• Caching a low-traffic endpoint where requests arrive slower than the TTL expires, so you're paying the write premium repeatedly and never collecting enough reads to recover it
• Assuming a stable system prompt is "cached" without verifying it in the response payload, since a prompt below the model's minimum token threshold caches nothing and fails silently on every provider

The fix for the second case matters most for founders running lean: a support bot answering a handful of tickets an hour on Claude's 5-minute TTL will mostly pay the 1.25x write premium and rarely hit a read, while the same bot on the 1-hour TTL pays 2x once and then reads cheap for the rest of the hour. Match the TTL to your actual request cadence, not the default.

A rollout checklist that actually works

Put the largest, most stable block of your prompt (system instructions, tool schemas, retrieved documents) first, and anything that changes per request (user message, current timestamp) last, since only unbroken prefixes cache. Confirm every deployment by reading the cache-hit fields in the response instead of trusting that a correctly-placed marker worked. If you're on Claude, choose the 1-hour TTL for anything called less than once every five minutes, and the 5-minute default otherwise. If you're on OpenAI or Gemini's implicit mode, there's nothing to configure, but you still need to check the actual cached_tokens count in production traffic, because automatic caching only helps if your prefix genuinely repeats byte-for-byte across real user requests, not just in your test script.

None of this replaces the retrieval and architecture decisions that determine whether an LLM feature works at all, our RAG vs fine-tuning breakdown and vector database guide cover that ground. Caching is a cost lever you pull after the architecture is right, and it's one of the few LLM cost optimizations that requires no quality tradeoff at all, you get the identical output, just without paying to recompute context you already sent five seconds ago. The AI Product Development Bootcamp walks through wiring this into a real production pipeline alongside the retrieval and evaluation decisions that determine whether the feature is worth running in the first place.