Definition: Prompt caching is a provider feature that saves the work a model already did on the unchanged opening stretch of a request, usually the system prompt and the reference material sitting at the top of the context window, so the next request that begins with the same tokens skips that reading and goes straight to inference on whatever is new.
TL;DR: Prompt caching reuses the model's reading of the unchanged front of your prompt instead of paying for it twice. Providers bill those cached tokens at a fraction of the normal input rate, in some cases about a tenth, and the answer starts sooner. It only works when that front section is identical every time. Build one free →
You already do a version of this. When a colleague asks you a fourth question about the same contract, you do not reread the contract from page one. You keep it open, you keep what you already worked out, and you read only the new question. A model has no such habit. It is stateless, so left alone it rereads the entire contract on every single question. Prompt caching is the shortcut that lets it stop doing that.
Why Prompt Caching Matters in 2026
Prompt caching is the largest cost lever most AI products have, because the biggest part of a modern request is the part that never changes. A system prompt, a catalogue of tools, a style guide, and a few reference documents can run to tens of thousands of tokens, and every turn of a conversation resends all of it. In a ten-turn chat you pay to read the same opening ten times. Caching collapses that to one full read plus nine cheap lookups. The saving compounds hardest exactly where AI is heading: long agent sessions, multi-agent teams that all carry the same standing brief, and retrieval systems that paste a large block of source material in front of every question. It also shortens the wait, because reading the prompt is a real share of the delay before the first word appears.
How Prompt Caching Works
A model does not swallow your prompt whole. It works through the text in order, and by the time it reaches the end it has built an internal reading of everything before that point. Prompt caching stores that reading at a marked spot, filed under the exact text that produced it. The next request that opens with identical text loads the stored reading instead of rebuilding it.
- Split the request. Everything that stays the same run after run sits at the front: the brief, the rules, the tool descriptions, the documents. Everything that varies goes last. Deciding what belongs where is a core move in context engineering.
- Check for a match. The provider compares the front of your request against what it stored recently. The comparison is exact and runs over tokens, so a single changed character breaks it. Most providers also set a floor, commonly around a thousand tokens, below which nothing is cached at all.
- On a hit, load the saved reading. The stored state drops in and the model carries on as if it had read the front itself. Because the attention work over that stretch was already done, it is not repeated, which is where both the money and the time come back.
- On a miss, read it once and keep it. The front is processed normally, billed at the normal rate, and stored. Some providers charge slightly more than a plain read for that first storing pass, which is why caching pays off on repetition rather than on a one-off question.
- Process the new part. Only your latest message, or the latest incoming record, is genuinely new work. That final stretch is the only part running at full price. See inference.
- Let it expire. Caches live for minutes of inactivity, not days. A busy agent session keeps its cache warm by using it; an idle one quietly falls back to a full read on the next request.
Two habits decide whether any of this fires. Put stable content first and variable content last, and keep the stable part genuinely stable. A timestamp, a greeting with the user's name, or a freshly shuffled list at the top of the prompt looks harmless and costs you every hit, because the front is no longer identical. This is the practical half of resource-aware optimization: the same words, in a better order.
Prompt Caching vs Memory
People hear "cache" and assume the AI is starting to remember them. It is not. Prompt caching changes what a request costs and how fast it returns; it never changes what the model knows or what it says. Agent memory is the opposite: it changes what the model knows, and it costs more, not less, because remembered facts are extra text to send.
| Question | Prompt caching | Agent memory |
|---|---|---|
| What it stores | The model's reading of text you already sent | Facts, preferences, and results worth keeping |
| What it changes | Price and speed | What the agent actually knows |
| Does the answer change | No, you get the same answer | Yes, that is the point |
| How long it lasts | Minutes of inactivity | Across sessions, until you change it |
| Who controls it | The provider and the platform | You |
| What breaks it | One changed character at the front | Little; you edit or delete it deliberately |
| How you notice it | A cheaper bill and a shorter wait | The agent knows your situation |
A three-turn conversation makes the billing shape concrete. The standing brief is identical each time, so only the first turn pays full price for it:
TURN 1 [ 20,000 tokens of standing brief ] + [ your question ]
cache miss -> the brief is read in full, then stored
TURN 2 [ the same 20,000 tokens ] + [ your next question ]
cache hit -> the brief is loaded, billed at a fraction
TURN 3 [ the same 20,000 tokens ] + [ your next question ]
cache hit -> loaded again
Ten turns like this pay for the brief once at full price,
not ten times.
Connection to Taskade
You never configure a cache in Taskade, and the shape of a Taskade request is already the shape caching rewards. A Taskade AI Agent carries a standing brief that barely moves: the role you wrote for it, the built-in tools it can call, and the projects and records you pointed it at. That is the stable front. Your message, or the record that just changed, is the small new tail. The same holds for automations that fire hundreds of times a day against one unchanging instruction set, and for multi-agent teams where several agents share the same opening context. Model choice is handled for you: the Auto setting routes each job to one of 15+ frontier models from OpenAI, Anthropic, Google, and open-weight providers, so you are never pinning versions or reading provider caching docs. What you see is credits and results, not tokens and cache keys.
What You Would Build in Taskade
Picture the policy questions your team asks all week. Refunds, shipping exceptions, what the contract says about renewals. The answers all live in one long handbook that nobody rereads, so the same three people get interrupted instead. In Taskade that becomes a policy desk: you point an agent at the handbook and the relevant projects, and anyone can ask it a question in plain language. The handbook is the unchanging front of every request. The question is the tail. Twenty questions in a morning read the handbook properly once and stay quick and cheap after that, which is the whole practical payoff of prompt caching without you thinking about it. Add an automation and the desk also posts a weekly list of the questions people keep asking, which is usually the fastest signal about what the handbook is missing.
Describe yours and build it free →
Related Concepts
- Token: the unit caching is priced in
- Context Window: the space the cached front sits in
- Inference: the work that still runs at full price
- System Prompt: the most commonly cached block of all
- Context Engineering: deciding what goes first and what goes last
- Agent Session: why activity keeps a cache warm
- Resource-Aware Optimization: spending less without doing less
- Stateless vs Stateful AI: why the rereading happens in the first place
- AI Agents in Taskade: agents whose standing brief barely moves
Frequently Asked Questions About Prompt Caching
What is prompt caching in simple terms?
Prompt caching means a provider saves the work it already did on the unchanged opening part of your request and reuses it next time. You still pay for the new part of the prompt and for the answer. You pay far less for the system prompt and reference material that repeat on every turn.
How much does prompt caching actually save?
It depends on the provider and on how much of your prompt repeats. Cached input is commonly billed at a fraction of the normal input rate, in some cases about a tenth, so a request that is mostly standing brief can drop sharply in price. Answers also start sooner, because the reading step is skipped.
What is a prefix cache?
Same thing, different name. "Prefix" is just the front of the prompt, the stretch before anything varies. A prefix cache matches that opening run of tokens exactly, from the very first character forward, which is why caching only helps when the stable material is placed first and the changing material last.
What are cache tokens on my bill?
They are the tokens that were served from a cache rather than read fresh. Most providers report them as separate lines: tokens written into the cache on the first pass, and tokens read from it afterwards. Reads are the cheap ones. A high read count against a low write count means your context is well ordered.
Why did my prompt cache stop working?
Almost always because something at the front changed. A date, a user's name, a reordered list, or an edited line in the system prompt is enough to make the opening text different, and the match is exact. The other common cause is time: caches expire after minutes of inactivity, so a quiet gap ends them.
How do I reduce the cost of running an AI model?
Order the prompt so everything stable sits first and everything variable sits last, so caching can fire. Send only the material the task needs instead of everything you have. Keep sessions focused rather than sprawling. Match the job to the right model instead of routing everything to the largest one. See context engineering and resource-aware optimization.
Is my cached prompt visible to other companies?
No. Providers scope caches to your own account or organization, so a cache entry is only ever matched against your own later requests. Caching is a billing and speed mechanism, not a shared pool of text, and it does not add your content to anyone's training data.
Do I need to turn on prompt caching in Taskade?
No. There is no setting to find. Every Taskade AI Agent and every automation is assembled with a stable standing brief and a small changing tail, which is the structure caching rewards, and the Auto model routing handles provider differences for you. You describe the work; Taskade EVE handles the assembly inside Taskade Genesis.