Prompt caching is a feature of some AI APIs that stores the work done on the start of an input so it can be reused. If the next request begins exactly the same way, that part is billed at a lower rate and processed faster. It saves money when many requests share long instructions or documents.
A question from the app. Pick an answer.
Your app sends the same long instructions with every request. What does prompt caching do?
That is history on your device. Prompt caching lives in the service and keeps the work already done on the repeated start of requests.
If a request starts the same as a recent one, the service reuses what it already computed for that start and bills those tokens for less. Storing it may cost a bit.
That would be storing answers, and it would not help with different questions. Prompt caching keeps the start of the request: the model still answers every time.
Before answering, the model processes the whole input, all its tokens together; what comes out one token at a time is the answer. If a thousand requests start with the same instructions, processing them a thousand times is repeated work. With caching, the provider keeps that intermediate result for a while and reuses it when a request arrives with the same beginning.
It does not store answers. The model writes a new answer each time, so output costs the same and quality does not change. What goes down is the cost of the repeated part of the input and the time until the first word.
Made up rates: normal input costs 1 USD per million tokens and input read from cache, 0.10 USD. A support app sends a 5,000 token manual with every request, plus the customer's question of about 100. With a thousand questions a day, without caching you send 5.1 million tokens: 5.10 USD. With caching, the manual is billed at the low rate (0.50 USD) and the questions at the normal one (0.10 USD): 0.60 USD.
This is an ideal case: it assumes every question hits the cache and leaves output out. In practice, some providers charge a bit more the first time the cache is written, and each has its own rates. The real savings depend on how many requests reuse the same beginning. For how the full bill adds up, see the AI cost page.
Caching only works for the beginning of the prompt, and that beginning has to be identical, character by character. That is why order matters: fixed parts first (instructions, documents, examples) and what changes at the end (the user's question).
Common mistake: putting "Today is Monday the 3rd, 2:05 PM" or "Hi, Sarah" at the top. Change one letter and everything after it stops matching. Another case: if your app translates its instructions into English, Spanish and Portuguese, each language is a different beginning with its own cache. With little traffic in one language, that cache can expire before it is reused.
If the repeated beginning is short, many providers do not even store it: there is usually a minimum number of tokens. If requests come far apart, the cache expires (it usually lasts minutes without use) and you pay for the full input again. And some APIs apply it on their own while others ask you to mark what to store: check the provider's docs before counting on the savings.