Prompt Caching: Cut Your LLM Bill Up to 90% Across Claude, GPT and Gemini

What you need to know

Most production LLM bills contain a large amount of the same tokens, sent over and over. A support classifier re-sends its 6,000-token policy and schema on every ticket. A retrieval-augmented answer bot re-sends the same instructions and the same retrieved passages for a burst of follow-up questions. A coding agent re-sends the same tool definitions and the same repository context on every turn. In each case the provider reprocesses that identical prefix from scratch and charges you full input price for it — again and again.

Prompt caching removes that waste. The provider stores the processed form of your stable prefix and, on the next request that starts with the same bytes, reuses it and bills the reused portion at a steep discount. As of July 2026, that discount is up to roughly 90% off cached input tokens on the current models from Anthropic, OpenAI and Google. Nothing the user sees changes: same model, same output, same format. You simply stop paying to re-read what the provider already read.

The entire technique rests on one durable habit that survives every price change and model release: structure the prompt so the stable content comes first and the variable content comes last. Caching is a prefix match, so a reusable prefix only exists if you build one. Get that ordering right and the provider-specific mechanics — automatic on OpenAI and Gemini, an explicit breakpoint on Claude — are a small amount of plumbing on top. This guide covers the pattern, the Python, how the three majors differ, what it actually saves at three traffic tiers, and the silent mistakes that quietly leave the discount on the table. It is distinct from the batch API, which cuts async jobs 50%; caching is the per-request lever, and the two stack.

How prompt caching actually works

Caching operates on the front of your prompt. When a request arrives, the provider hashes the leading tokens and checks whether it has already processed a prompt with that exact prefix. If it has, it loads the cached internal state instead of recomputing it, and charges you the cache-read rate — roughly a tenth of the normal input price — for that reused span. Everything after the point where your request first diverges from the cached one is processed normally, at full price.

The prefix-match rule that governs everything

The one invariant you must internalise: caching is a byte-exact prefix match, and any change anywhere in the prefix invalidates everything after it. There is no fuzzy matching and no reordering. If the first 2,000 tokens of two requests are identical and the 2,001st differs, the cache reuses the first 2,000 and reprocesses the rest. If the very first token differs — a fresh request ID, a current timestamp, a per-user greeting at the top — then nothing matches and you pay full price for the lot, no matter how much identical content sits further down.

This is why ordering is not a nice-to-have but the whole game. The provider renders your request in a fixed order — broadly tools, then system prompt, then the message history — and the cache key is built from those bytes in sequence. Stable content has to physically precede volatile content, or there is no reusable prefix to cache. An Indian fintech running a KYC-extraction pipeline and a UK legal-tech team summarising contracts face the identical constraint: the shared schema goes at the front, the document under review goes at the back.

Prompt caching is not the batch API

These two cost levers are frequently confused because both save roughly the same order of magnitude, so it is worth drawing the line clearly. The batch API is about time: you hand the provider a pile of non-urgent requests, accept that results may take up to a day, and pay about half price for the flexibility. Prompt caching is about repetition: it cuts the cost of the prefix you re-send on every synchronous request, and the reply still comes back in real time. Because they act on different parts of the bill, they multiply rather than compete — a nightly extraction run can be batched and share a cached schema, compounding the two discounts. Our companion guide on the cache, route and compress playbook shows how the levers layer; this piece drills into the caching one.

The one pattern that makes it work: stable prefix first, variable last

Here is the pattern in Python against the Anthropic API, because Claude requires you to place the cache breakpoint explicitly, which makes the structure visible. The same ordering discipline is what earns you the automatic cache hits on OpenAI and Gemini too — the only difference there is that you do not mark the breakpoint yourself.

First, the anti-pattern most teams ship by accident. It reads naturally, and it never caches:

from uuid import uuid4
from datetime import datetime
from anthropic import Anthropic

client = Anthropic()
SYSTEM_PROMPT = load_policy_and_schema()   # ~6,000 tokens, identical every call

# BEFORE: volatile content sits at the top of the prefix, so the cache never hits.
def classify_before(ticket_text, user_id):
    return client.messages.create(
        model="claude-opus-4-8",
        max_tokens=512,
        system=(
            f"Request ID: {uuid4()}\n"           # changes every call — kills the cache
            f"Timestamp: {datetime.now()}\n"      # so does this
            f"{SYSTEM_PROMPT}\n"
            f"User under review: {user_id}"       # varies per request
        ),
        messages=[{"role": "user", "content": ticket_text}],
    )

Every request begins with a fresh UUID and a live timestamp, so the byte-exact prefix is unique every time and the 6,000-token schema behind them is reprocessed at full price on every single call. Now the fixed version: pull the stable schema to the very front, mark the breakpoint at its end, and push everything that varies into the message body after it.

# AFTER: stable prefix first with an explicit cache breakpoint; variable content last.
def classify_after(ticket_text, user_id):
    return client.messages.create(
        model="claude-opus-4-8",
        max_tokens=512,
        # 1. Stable prefix first — identical bytes every call — cached explicitly.
        system=[
            {
                "type": "text",
                "text": SYSTEM_PROMPT,
                "cache_control": {"type": "ephemeral"},   # breakpoint at end of the stable span
            }
        ],
        # 2. Everything that varies goes AFTER the breakpoint.
        messages=[
            {
                "role": "user",
                "content": f"User under review: {user_id}\n\n{ticket_text}",
            }
        ],
    )

# Verify the cache is working — this must be non-zero on the second call onward.
resp = classify_after("card declined at checkout", "user_8842")
print(resp.usage.cache_read_input_tokens)   # 0 on the first write, ~6000 on hits

Three things carry the whole change. The UUID and timestamp are gone from the prefix — if you genuinely need them, they belong after the breakpoint where they invalidate nothing before them. The schema is now the first thing rendered, so its bytes are stable across every call. And the cache_read_input_tokens field is your proof: zero on the first request that writes the cache, then roughly the size of the prefix on every subsequent hit. On OpenAI and Google you would not write the cache_control line at all — caching is automatic — but you would make the exact same move of pulling the schema to the front and the variable text to the back, because their caches are prefix matches too. If your prompts are already versioned through a prompt management and versioning workflow, this is a one-line reorder in the template, applied everywhere at once.

How the three majors compare

The pattern is universal; the mechanics differ. The table below summarises the state of play as of July 2026 — write premium, read discount, minimum cacheable size, cache lifetime, and whether you control the breakpoint yourself. Treat the numbers as a shape, not a contract: providers revise pricing and thresholds regularly, so confirm the current figures on each provider's own pricing and documentation pages before you design around them.

Figures as of July 2026 and approximate; discounts, thresholds and TTLs vary by model and change over time. Verify against each provider's own pricing and caching documentation before committing.

Anthropic Claude: explicit breakpoints, 90% off reads

Claude gives you the most control and asks the most in return. You mark the end of a stable span with a cache_control breakpoint (up to four per request), and reads then cost about a tenth of the base input price — roughly 90% off. The catch is the write: creating or refreshing a cache entry costs about 1.25× the base input rate on the default five-minute lifetime, or 2× if you opt into the one-hour lifetime. That write premium means caching only pays once the prefix is reused enough to amortise it — with the five-minute window, two hits already put you ahead; with the one-hour window you need roughly three. There is also a top-level automatic mode that places the breakpoint on the last cacheable block for you, which suits growing conversations where the reusable prefix extends every turn.

OpenAI GPT: automatic, zero code

OpenAI is the least effort: caching is on by default for every request over roughly 1,024 tokens, with no code change, no explicit breakpoint and no write premium. If your prompt has a reusable prefix, you get the discount automatically; if it does not, you get nothing extra to configure. As of July 2026 the cached-input discount reaches up to about 90% on current models, though the exact figure varies by model — several GPT models have historically discounted cached input by 50%, so check the rate for the specific model you run rather than assuming the headline number. Cached entries live for roughly five to ten minutes of inactivity, up to about an hour, with longer retention on some newer models. Because you cannot force a breakpoint, your only lever is prompt structure — which is the lever that matters anyway.

Google Gemini: implicit plus explicit with storage billing

Gemini offers both modes. Implicit caching is automatic on Gemini 2.5 and newer, discounting cached tokens by up to roughly 90% (about 75% on the older 2.0 generation) with no storage cost and no setup — the same put-stable-content-first advice applies. Explicit caching is a managed object you create and reference, which guarantees the hit but bills differently: you pay a per-hour storage cost for the cached tokens (on the order of a dollar per million tokens per hour on Flash-class models, more on Pro-class) for as long as the cache lives, and explicit caches carry a much higher minimum size, in the region of 32,768 tokens. That storage-billing model is the key difference from the other two: with Gemini explicit caching you are renting space over time, so it pays off for a large context reused heavily within its lifetime, and can lose money for a large cache queried rarely. For most builders, implicit caching plus good prompt ordering is the place to start.

What it saves: a worked monthly model

Numbers make the case better than adjectives, so here is a worked example you can re-run with your own figures. Picture a classification or extraction service — an Indian fintech tagging KYC documents, say, or a UK insurer triaging claims emails. Each request carries a 6,000-token stable prefix (instructions plus schema), about 400 tokens of variable input (the document under review), and produces about 200 tokens of output. The illustrative rates are Claude Opus 4.8 as of July 2026: input $5 per million tokens, output $25 per million, and a cache read at roughly $0.50 per million. Under steady traffic the five-minute cache stays warm, so the write premium is a rounding error we can set aside.

Illustrative, as of July 2026, on Claude Opus 4.8 rates and the token mix stated above. Per-request cost falls from ~$0.037 to ~$0.010. Your real figure depends on the prefix-to-variable ratio, the model tier and region.

Read across a row and the shape is clear. Without caching, the 6,000-token prefix costs about $0.030 per request; with caching it costs about $0.003 — the 90% cached-read discount doing its work. The variable input and the output are unchanged, so the total request cost falls from roughly $0.037 to $0.010, about 73% off. The savings scale linearly with volume: the same reorder that saves a prototype a few thousand dollars a month saves a business at scale a few hundred thousand a year, for a one-line change to how the prompt is assembled.

The lever that moves this number is the prefix-to-variable ratio. Because caching only discounts the shared prefix, the more of each request that is fixed, the closer your total saving gets to the headline 90%. A bulk-extraction job with a huge schema and a tiny input approaches that ceiling; a free-form chat where most of each request is fresh user text sees far less, because there is little to reuse. Audit your traffic and cache first where the prefix dominates — that is where the money is.

Gotchas that quietly kill your cache hit rate

Caching fails silently. There is no error when a cache misses — you simply pay full price and never notice, because the response looks identical. Almost every failure traces back to the byte-exact prefix rule, so the audit is always the same: something changed in the prefix that you did not intend to change. The usual suspects are a timestamp or a fresh UUID interpolated near the top of the system prompt; JSON serialised without sorted keys, so the byte order drifts request to request; a tool list assembled in a different order per user; or a conditional block that quietly adds or removes a section depending on a flag, creating a different prefix for every flag combination. Each of these puts a moving byte ahead of your stable content and invalidates everything after it.

The second failure mode is time. The default cache lifetime is short — around five minutes on most providers — so a workload where requests arrive further apart than the window pays a fresh write every time and never banks a read. For bursty traffic with gaps, either move the caching logic behind a shared layer that keeps the entry warm, or opt into a longer lifetime where the provider offers one and the reuse volume justifies the higher write cost. Centralising this in a resilient LLM gateway alongside your retries and rate limits is often the cleanest home for it, because the gateway sees every request and can enforce a consistent, cache-friendly prompt shape across all of them.

Where caching fits in your cost stack

Prompt caching is the first lever to reach for, precisely because it changes nothing the user sees — same model, same output, same format — so it carries no quality risk. That makes it safer than switching to a cheaper model tier or trimming context, both of which can degrade results. Pull caching first, measure, then layer the other levers on the smaller bill it leaves behind.

From there the stack is orthogonal. Move offline, latency-tolerant work to the batch API for a further 50% off, and where a batch shares a schema, stack caching on top for a combined discount. Route each task to the cheapest sufficient model tier so the caching saving compounds on a lower base rate. And if your volume is high enough and your compliance needs point that way, weigh self-hosted serving with quantisation and batching, where the equivalent win comes from KV-cache reuse rather than a provider discount. Caching, batching, routing and self-hosting each act on a different part of the cost, so they multiply rather than overlap.

The bottom line for builders

Prompt caching is the highest-leverage, lowest-risk cost move available to anyone shipping LLM features, and the durable skill behind it is not provider-specific plumbing but a habit: build every prompt with the stable content first and the variable content last, so a reusable prefix exists to be cached. Do that, add the one breakpoint Claude needs, lean on the automatic caching that OpenAI and Gemini already give you, and instrument the cached-token field so you can prove it is working. As of July 2026 that pattern reliably takes up to 90% off the repeated portion of your bill across all three majors, with no change to what your users experience. Structure the prompt once, and the saving keeps paying out on every request you make for as long as the pattern holds — which is a long time, because it is a property of how caching works, not of this month's price sheet.

Sources

• Anthropic — Prompt caching documentation
• OpenAI — Prompt caching guide
• Google — Gemini API context caching