Prompt caching: the AI cost lever most teams ignore

Every team building on LLM APIs watches their output token spend — it’s the obvious line item. What most skip is the equally large (sometimes larger) hit from re-sending the same system prompt, knowledge base, or tool definitions on every single request. Prompt caching is the fix. It’s table-stakes infrastructure in 2025, it’s available on all three major providers, and operators consistently report that it was the first lever they should have pulled.

What’s actually happening under the hood

When you call a frontier model, the API processes every token you send through the model’s attention layers before generating a single output token. When multiple requests share identical prefixes — system prompts, few-shot examples, or document context — the KV cache computation repeats unnecessarily. Prompt caching stores the key-value representations from that attention pass so subsequent requests with the same prefix can skip recomputation entirely. When the same prompt prefix is used, the model retrieves the cached K-V pairs, eliminating the need to recompute them — only the new, dynamic parts of the prompt need processing.

The result: beyond cost savings, Anthropic reports latency reductions of up to 85% for long prompts — a 100K-token book example showed response time dropping from 11.5s to 2.4s with caching enabled.

Provider breakdown as of June 2025

Anthropic (Claude)

Prompt caching (both automatic and explicit) is supported on all active Claude models. The pricing structure has two modes:

5-minute cache write tokens are 1.25× the base input token price; 1-hour cache write tokens are 2× the base input token price; cache read tokens are 0.1× the base input token price.

Translation: a cache hit costs 10% of the standard input price, which means caching pays off after just one cache read for the 5-minute duration (1.25× write), or after two cache reads for the 1-hour duration (2× write).

There’s a minimum prompt size to qualify. The minimum cacheable prompt length is 4,096 tokens for Claude Mythos Preview, Claude Opus 4.7, and Opus 4.6, and 1,024 tokens for Claude Sonnet 4.5, Opus 4.1, Sonnet 4, and related models. Shorter prompts are silently processed without caching — no error, just no savings.

For teams using Claude Code or building agents on the Anthropic API, this is the fastest ROI available. For any application with a large, reused system prompt or document context, it is the single most impactful change you can make to your Anthropic bill.

OpenAI

Prompt caching works automatically on all API requests and has no additional fees. It is enabled for all recent models, GPT-4o and newer.

Caching is enabled automatically for prompts that are 1,024 tokens or longer.

The tradeoff vs. Anthropic: you lose explicit control but gain zero-config simplicity. Prompt caching can reduce time-to-first-token latency by up to 80% and input token costs by up to 90%. Cache discount magnitude varies by model — cache discounts can be significant, and as OpenAI’s inference stack has become more efficient, their newest models have been able to offer steeper cache discounts.

One important constraint: cache hits are only possible for exact prefix matches within a prompt. Place static content like instructions and examples at the beginning of your prompt, and put variable content such as user-specific information at the end.

Google (Gemini)

The Gemini API offers two different caching mechanisms: implicit caching (automatically enabled on Gemini 2.5 and newer models, no cost saving guarantee) and explicit caching (can be manually enabled on most models, with a cost saving guarantee).

The pricing model is structurally different from the others — and it matters for your math. Google (Vertex/Gemini) charges a per-hour cache storage fee in addition to cache hit pricing.

Cache reads cost 10% of base input price, but storage costs $1–$4.50 per million tokens per hour depending on the model. That hourly storage line means Google caching is most economical for high-traffic, long-session applications — low-volume workloads can end up paying more than they save.

When caching actually helps

The math only works in your favor when context is large, stable, and re-used frequently. The highest-ROI scenarios:

• Multi-turn agents with big system prompts — every conversation turn re-sends the same instructions. Relevance AI and Lindy workflows built on the Anthropic API benefit directly here.
• RAG pipelines with shared document context — if every query against a knowledge base sends the same retrieval template and source documents, cache those.
• Workflow automation at volume — n8n workflows that call a model in a loop with the same context block are ideal candidates.
• Tool definition-heavy agents — long tool schemas get sent on every step; cache them.

Assume cached tokens cost 10% of normal, your prefix is 3,000 tokens, your suffix is 1,000 tokens, and you make 100 requests in the cache window: without caching that’s 400,000 full-price input tokens; with caching it drops to roughly 132,700 full-price equivalent tokens — about a 67% reduction on the input side.

The gotchas that eat your savings

Prefix invalidation is silent and expensive. Cache hits require an exact match on the prefix. A single character change — even a whitespace difference, a timestamp injected into the system prompt, or a dynamic user name interpolated before static content — busts the cache. Operators consistently discover this when they see mysteriously low cache hit rates after a seemingly minor prompt change.

Short TTLs punish low-frequency callers. On Anthropic, cached entries have a minimum lifetime of 5 minutes (standard) or 1 hour (extended), after which they are promptly, though not immediately, deleted.

If your app prompts less than once every five minutes, you’ll be losing money on 5-minute TTL cache writes that never get read. Use 1-hour caching only when session frequency justifies the higher write cost.

Google’s storage fee changes the calculus for cold workloads. Unlike Anthropic and OpenAI, Google charges per-hour storage on cached content. Context caching offers a price optimization where the long prefix prompt can be reused between requests, but the hourly extra charge means it isn’t a default optimization for all cases — only high-traffic applications running long prompt systems will see significant savings.

Parallelism can kill cache hits. Cache entries become available after the first response begins — they’re not available for concurrent parallel requests. Burst-parallel workloads need warm-up logic or staggered calls to avoid cache misses on the first wave.

Tools that don’t implement cache_control correctly silently drain your budget. It could be about how the requests are being structured before they even reach Claude — specifically, whether prompt caching is working the way Anthropic designed it to. Claude subscription limits, prompt caching, and third-party integrations are more tightly connected than most users realize. If you’re hitting spend limits faster than expected with an abstraction layer between you and the API, check whether that layer is actually setting cache breakpoints.

Stacking caching with other discounts

Caching compounds. The caching multipliers stack with other pricing modifiers such as the Batch API discount and data residency. For non-real-time workloads, combining prompt caching with batch processing can push input token costs well below 10% of list price. Teams using Vellum for prompt management can track cache hit rates as a first-class metric and monitor ROI per workflow over time.

Bottom line

Prompt caching is not an advanced optimization — it’s basic API hygiene for any production workload with repeated context. The Anthropic implementation requires explicit cache_control markers and rewards careful prompt structure; OpenAI’s is automatic but less controllable; Google’s adds a storage dimension that warrants its own cost model. The single most common mistake is injecting dynamic content into the prefix — audit your prompt construction before assuming caching is working.