Prompt Caching Explained: How I Cut My Claude, GPT, and Gemini API Bills 75% in 2026

Prompt caching is the single most underused cost-reduction technique in AI engineering in 2026. Almost every team I work with is paying full price for inputs the model has already seen — sometimes hundreds of times in a single hour. Turning on caching takes about an afternoon, requires no model retraining, and routinely cuts API bills by 50 to 90 percent.

I am going to walk you through what prompt caching is, how it works on each of the three major APIs, a real case study where my team cut a bill by 75 percent, and the design patterns that make caching land instead of fail. This is the article I wish someone had shoved at me in 2024.

The problem: every API call repays for the same input

A normal LLM API call charges you for two things — input tokens and output tokens. Most production AI applications send the same long input over and over. A customer-service bot reattaches the same 50-page handbook on every question. A code agent reattaches the same project context on every step. A document-Q&A app reattaches the same PDF on every query.

Without caching, you pay full price for those repeated tokens every single call. Multiply by hundreds of users, dozens of turns per conversation, and weeks of operation, and the bill becomes the largest line item in your AI startup.

What prompt caching actually is

When a model processes a prompt, it does an expensive operation called the "prefill" — it reads every input token and computes the internal state of the network. The output generation runs on top of that internal state.

Prompt caching saves the internal state from a previous call so the next call can skip the prefill. The economic effect: input tokens that hit the cache cost a fraction of the standard input price (typically 10–25%), and the latency drops because the model is not redoing work it already did.

The mechanics across providers are slightly different, but the rule is the same: long, stable prefixes that are repeated frequently are the perfect cache target. System prompts, tool definitions, large reference documents, retrieved context that does not change — all good. The dynamic part of your prompt — the user's latest message, today's data, the live query — should be at the end, after the cache breakpoint.

Claude prompt caching

Claude's prompt caching is opt-in and very explicit. You mark the cache breakpoint by adding cache_control to a content block in your messages. Anthropic's pricing in 2026:

• Cache write: 1.25× the standard input token price (a small premium for the first call).
• Cache read: 0.10× the standard input token price (90% discount on subsequent calls).
• Cache TTL: 5 minutes by default, with a 1-hour extended-cache option for an additional fee.

The pattern in practice: put your large system prompt and any large reference documents in the cached portion, then add the user's question after the cache breakpoint. The first call costs slightly more than baseline; every call after that within the TTL costs roughly one-tenth of the cached portion plus full price for the new user content.

Claude caching also composes with Claude Code, the agentic CLI. When you point Claude Code at a large project, it uses caching automatically so that the project context is paid for once and reused across many tool calls. This is one of the reasons Claude Code feels so cheap to operate compared to its raw token count.

OpenAI prompt caching

OpenAI's prompt caching is automatic for prompts above 1024 tokens on most production models. You do not need to add a flag — if your prompt prefix matches a recently seen prefix, you get the discount. Key facts:

• Cached input price: roughly 50% off standard input pricing on most current production models. Some newer models offer larger discounts.
• TTL: 5 to 10 minutes typically, longer during off-peak.
• Granularity: the cache hit is computed on the longest matching prefix from the start of the prompt. So you must keep the stable parts at the front.

The implication: in OpenAI, the order of your message blocks matters enormously. System prompt first, then large reference content, then conversation history, and the user's new message last. If you reverse that order or shuffle it call-to-call, you destroy your cache hit rate without even noticing.

Gemini context caching

Google's Gemini context caching takes a different approach. You explicitly create a named cache containing your large reference content, give it a TTL, and reference the cache by ID on subsequent calls. Pricing in 2026:

• Cache storage cost: per million tokens per hour. Small but real if your cache is large and lives long.
• Cache read cost: typically 25% of standard input pricing for the cached tokens.
• TTL: you choose, from minutes to hours.

Gemini's design is the most explicit of the three. You manage cache objects like database records. The advantage is predictability — you know exactly which calls will hit cache. The disadvantage is operational overhead — you need to handle cache creation, expiration, and cleanup yourself.

Real case study: 75% cost reduction

Last month one of my federal-AI projects was running a document-Q&A workload over a 320-page acquisition regulation document. Every user question reattached the full document as context, then asked the LLM to find the answer. The pre-caching numbers were ugly:

• Average prompt size: 165,000 tokens (320 pages plus system prompt plus user question).
• Average response: 800 tokens.
• Calls per day: about 2,400 (peak, internal users).
• Daily input cost on Claude Sonnet 4.6 at $3 per million input tokens: ~$1,188.
• Daily output cost: ~$58.
• Daily total: ~$1,246.

I added a single cache_control breakpoint after the 160,000-token regulation block. The first call of each 5-minute window paid the 1.25× write premium; every subsequent call paid the 0.10× cache read price for the 160,000 stable tokens, plus normal pricing for the small user question and the response. The new daily total: ~$310. That is a 75 percent reduction with one config change and zero accuracy loss.

The math works similarly on OpenAI and Gemini. The exact percentage depends on your prefix-to-question ratio, but anything above 60 percent reduction is normal for document-heavy workloads, and customer-service bots with long handbooks routinely exceed 80 percent.

How to design your prompts for cache hits

1. Put stable content first. System prompt, tool definitions, large reference documents — front-load them.
2. Put dynamic content last. The user's new message, today's data, the timestamp, and any randomized strings come after the cache breakpoint.
3. Avoid shuffling. If you build prompts by concatenating arrays, sort them deterministically. A different order kills the cache.
4. Watch your TTL. If your traffic is bursty with long quiet periods, the 5-minute TTL may cost you. Use Anthropic's 1-hour extended cache or Gemini's longer TTL.
5. Measure cache hit rate. Both Anthropic and OpenAI return cache statistics in the response. Log them. If your hit rate is below 70%, your prefix is not stable enough.
6. Cache tools, not just text. Tool/function definitions are part of the prompt. If you have many tools, they are an excellent stable prefix to cache.

Watch-outs

One, cached content is not state. The model still does not "remember" between calls in a stateful sense. Caching only saves the prefill computation. If you need true conversational memory, you still need to manage history yourself.

Two, cache TTLs are short by default. If your traffic dips below the TTL, your cache evicts and the next call pays the write premium again. Plan your traffic patterns or pay for extended cache.

Three, security boundaries matter. A cache key tied to your account is fine. A cache that crosses tenants in a multi-tenant SaaS is a privacy incident waiting to happen. Use per-tenant cache scopes if your provider supports them.

Four, the cache hit is your responsibility to verify. Always log the cache statistics returned by the API. If you assume caching is working without measuring, you will find out three months and ten thousand dollars later.

Where to go from here

Prompt caching is the highest-ROI optimization in production AI engineering in 2026. One afternoon of work, a single config change, and your bill drops by half or more on most workloads. There is no model retraining, no accuracy hit, and no architectural rework. It is the rare thing that is genuinely free.

If you run any production AI workload that sends more than 1,000 tokens of stable context, turn caching on this week. Measure your hit rate. Iterate on prompt structure until your hit rate is above 70 percent. Watch the bill drop. Then go build the next feature.