更高的缓存命中率，以及帮助持久化智能体运行更快、成本更低的新工具。
GPT‑6 让持久化智能体能够连续数小时处理复杂任务，从重构代码库到产出经过充分调研的文档和演示文稿。支撑这些智能体的应用会发起一系列相互依赖的 API 请求，往往会沿用早前轮次中的相同指令、工具定义和上下文。OpenAI 会缓存这些共享上下文，以便在多个请求之间复用计算，从而缩短响应时间，并为开发者提供最高 90% 的缓存输入 token 折扣。
借助 GPT‑6 系列，我们推出了改进的提示词缓存系统，默认即可实现更高的缓存命中率。我们现在对在 30 分钟窗口内复用的符合条件的共享前缀提供缓存折扣。我们还推出了新工具，帮助开发者监控缓存性能、诊断未命中情况，并选择要缓存提示词的多少部分。
“OpenAI 的提示词缓存在帮助 GitHub Copilot 大规模提供快速、高效体验方面发挥着关键作用。在过去几个月里，相对于此前的基线，我们在向 OpenAI 模型发起的数十亿次请求中，将需要全新处理的提示词 token 占比降低了 50% 以上。其结果是更高效的推理栈，以及开发者获得首次响应的时间更快。”
监控缓存并诊断缓存未命中
全新的 提示词缓存仪表盘 可展示你的应用输入中有多少来自缓存。你可以追踪一段时间内的命中率，并使用输入构成图表来对比已缓存和未缓存的 token。这些视图有助于你发现缓存命中的下降，并评估应用改动对缓存性能的影响。
当你遇到意外的缓存未命中时，可使用 提示词缓存诊断工具 来了解发生了什么。将某个请求与近期的响应进行对比，以识别导致无法复用缓存的模型、工具、设置或输入变化。受影响的 token 估算数量可帮助你评估影响规模，并决定如何优化你的集成以最大化缓存命中率。
{
"prompt_cache_diagnostics": {
"type": "cache_miss",
"reason": "tools_changed",
"comparison_reusable_tokens": 5629,
"cache_missed_tokens": 5629
}
}
为你的应用优化缓存
选择要缓存的内容。 显式缓存断点让你可以选择要复用哪些提示词前缀。更新后的 提示词缓存指南 解释了如何使用它们、缓存前缀保持有效的时间，以及工具和输入的变化如何影响复用。
调整推理强度而不破坏缓存。 在 GPT‑6 模型上，你现在可以在多次响应之间 更改推理强度 而不破坏缓存。对于更难的任务可以提高强度，对于常规的后续跟进可以降低强度，只需追加一个 configuration_update，同时保持请求级别的推理强度不变。这让你可以调整任务所需的推理量，同时保留可复用的上下文。
在工具和指令发生变化时保留缓存。当你的智能体的工具使用需求发生变化时，保持工具定义、schema 和顺序稳定，以便较早的上下文保持可复用。使用 allowed_tools 让只有相关的工具可被调用，或者在不需要工具时将 tool_choice 设为 none，而不是移除定义。使用新的 developer 消息在上下文末尾追加新指令，以覆盖较早的指令。参见我们的关于管理工具变更的指南。
预热缓存以降低延迟。预热会提前准备好已知上下文，这样当请求到达时，模型可以更快开始响应。例如，应用可以在启动期间预热共享指令、工具定义或参考资料，在用户提出第一个问题之前完成。这样就把处理过程移出了用户的等待时间。
这些可选控制项建立在引擎的默认性能之上，帮助你根据自身工作负载定制缓存。
“OpenAI 的提示词缓存诊断和仪表盘帮助我们将缓存命中率提高了几个百分点，成本降低了 20%。现在当缓存意外失效时我们会收到告警，并使用 Codex 智能体来诊断根本原因。显式断点还让我们能够缓存稳定的上下文，同时将频繁变化的内容保留在提示词末尾。这使得为后台任务分叉对话、同时复用几乎全部共享上下文在经济上变得可行。”
Higher cache hit rates and new tools to help persistent agents run faster and cost less.
GPT‑6 enables persistent agents to work for hours on complex tasks, from refactoring codebases to producing well-researched documents and presentations. The applications behind these agents make a series of API requests that build on one another, often carrying forward the same instructions, tool definitions, and context from earlier turns. OpenAI caches that shared context to reuse computation across requests, reducing response times and giving developers discounts of up to 90% on cached input tokens.
With the GPT‑6 family, we launched an improved prompt caching system that delivers higher cache hit rates by default. We now give cache discounts for eligible shared prefixes reused within a 30-minute window. We’re also introducing new tools to help developers monitor cache performance, diagnose misses, and choose how much of a prompt to cache.
“OpenAI’s prompt caching plays a critical role in helping GitHub Copilot deliver fast, efficient experiences at scale. Over the past several months, we’ve reduced by more than 50% the share of prompt tokens requiring fresh processing across billions of requests to OpenAI models, relative to our previous baseline. The result is a more efficient inference stack and faster time to first response for developers.”
Monitor caching and diagnose cache misses
The new Prompt Caching Dashboard shows how much of your application’s input is served from cache. Track hit rates over time and use the input composition chart to compare cached and uncached tokens. These views help you spot drops in cache hits and evaluate how changes to your application impact caching performance.
When you see an unexpected cache miss, use the prompt caching diagnostics tool to understand what happened. Compare a request with a recent response to identify changes to the model, tools, settings, or input that prevented reuse. The estimated number of affected tokens helps you assess the size of the impact and decide how you can optimize your integration to maximize cache hit rates.
{
"prompt_cache_diagnostics": {
"type": "cache_miss",
"reason": "tools_changed",
"comparison_reusable_tokens": 5629,
"cache_missed_tokens": 5629
}
}
Optimize caching for your application
Choose what to cache. Explicit cache breakpoints let you choose which prompt prefixes to reuse. The refreshed prompt caching guide explains how to use them, how long cached prefixes remain eligible, and how changes to tools and inputs affect reuse.
Adjust reasoning effort without breaking cache. On GPT‑6 models, you can now change reasoning effort between responses without breaking cache. Raise effort for a harder task or lower it for a routine follow-up by appending a configuration_update while leaving request-level reasoning effort unchanged. This lets you adjust how much reasoning a task needs while preserving reusable context.
Preserve cache as tools and instructions change. As your agent’s tool use needs change, keep tool definitions, schemas, and ordering stable so earlier context stays reusable. Use allowed_tools to make only the relevant tools callable, or set tool_choice to none when no tools are needed, instead of removing definitions. Use new developer messages to append new instructions towards the end of the context to override older ones. See our guidance on managing tool changes.
Prewarm the cache to reduce latency.Prewarming prepares known context ahead of time so the model can start responding sooner when a request arrives. For example, an application can prewarm shared instructions, tool definitions, or reference material during startup, before the user asks their first question. This moves processing out of the user’s wait time.
These optional controls build on the engine’s default performance, helping you tailor caching to your workload.
“OpenAI’s prompt caching diagnostics and dashboard helped us improve cache hit rates by a few percentage points, reducing costs by 20%. We now get alerts when caching breaks unexpectedly and use Codex agents to diagnose the root cause. Explicit breakpoints also let us cache stable context while keeping frequently changing content at the end of the prompt. That’s made it economically viable to fork conversations for background tasks while reusing nearly all of the shared context.”
Monitor cache hit rates in the Prompt Caching Dashboard.
Investigate unexpected misses with the diagnostics tool.
Follow the prompt caching guide to improve your setup, or use Codex to review your code, apply improvements, and measure results.