随着智能体逐渐成熟并学会处理更具挑战性的任务，token 开销的分布也发生了变化。如今智能体运行时间更长，并在每一步之间携带更多上下文，这使得我们组装和管理这些上下文的方式变得愈发重要。
智能体推理开销的去向
生产流量 · 宽度 = 占总开销的比例 · 深浅 = 计费类型
- 输出
- 未缓存输入
- 缓存输入
智能体推理开销的去向
生产流量 · 宽度 = 占总开销的比例 · 深浅 = 计费类型
- 输出
- 未缓存输入
- 缓存输入
说明：系统与工具定义包含压缩摘要。用户文本包含手动附加的技能。技能与插件包含技能描述、MCP 工具描述，以及放入静态上下文的规则。
过去几个月里，我们通过提升 Cursor 智能体框架的效率来应对这一变化。该框架让我们能够直接控制每个请求如何组装、上下文如何复用，以及工作何时在多个智能体之间分配。这些层面的改动将用户的 token 成本降低了 7%，同时没有降低智能体的质量。
精简系统提示词
智能体的每一轮交互都会包含 Cursor 在模型开始工作之前提供的上下文。这包括系统提示词以及智能体可使用的工具定义。由于这些上下文会贯穿整个对话，它已成为我们完全可以控制的最大开销来源之一。
当模型能力较弱时，我们不得不详细写明工具使用、任务管理和代码变更工作流的指令。我们还必须防范一些奇怪的行为，比如极长的哈希转储、二进制输出和 emoji。
随着模型不断改进，其中大部分指导都变得不再必要。我们不再需要罗列一长串“不要这样做”“你必须”“重要”之类的指令，只需定义工具的行为方式，模型通常就会遵从。这一点在各个模型系列中都成立，使我们得以将系统提示词削减约 66%。
随着时间推移，我们会持续增删指令，因为新模型需要新的指导，而这些指导又会流入未来模型的训练之中。利用大规模用户群进行 A/B 测试，对于针对真实流量有效优化 harness 至关重要。虽然评测可以作为一种快速且有用的替代指标，但它们往往代表的是“困难”问题，并不能准确反映用户请求的真实分布。
仅在需要时加载工具
系统提示词只是 Cursor 在每一轮对话中提供的上下文的一部分。另一部分是工具定义，随着我们为 Cursor 智能体添加更多强大的能力——包括后台 shell 监控、云端子智能体，以及更可靠的网页内容访问——这些定义在过去一年中急剧膨胀。这些工具大多很重要，但每一个都只在不到 20% 的对话中会被用到。
这就创造了一个提升效率的机会：让工具保持可用，但不必在每次请求中都包含它们的完整定义。今年早些时候，我们在将 MCP 工具迁移到动态上下文时解决过类似的问题——仅在需要时才加载它们。这使得调用 MCP 工具的会话中总 token 数减少了 46.9%。
现在，我们将同样的技术应用到了我们自己的内置工具上。
为了决定哪些工具保留在静态上下文中，我们基于每个工具的使用频率以及模型是否需要从一开始就看到它，对多种配置进行了 A/B 测试。我们追踪了 token 用量、成本、延迟、工具调用错误以及智能体的整体使用情况，以确保节省不会导致质量下降。
最常调用的工具
至少调用一次各工具的智能体会话占比
最常调用的工具
至少调用一次各工具的智能体会话占比
最终，我们将读取、搜索、编辑以及在静态上下文中使用 shell 的高频工具保留了下来。我们还保留了 ask_question，因为一些模型往往会幻觉出对它的调用，以及对于特定产品流程至关重要的工具，例如 Plan Mode 中的 create_plan。其余工具现在会在智能体需要时再加载。
卸载内置工具使静态上下文描述 token 减少了 60%
- 保留在静态上下文中
- 卸载到动态上下文
卸载内置工具使静态上下文描述 token 减少了 60%
- 保留在静态上下文中
- 卸载到动态上下文
提升缓存复用
在减少每次请求中的静态上下文数量后，我们提升了重复上下文在多个轮次之间被缓存的有效性。
智能体的每一轮都会重新发送一个长请求，其中包含工具、系统指令、设置以及到目前为止的对话。开头的大部分内容在相邻轮次之间保持不变，而末尾的对话则持续增长。
提示词缓存让模型提供商能够复用那段未更改的前缀。不过，缓存的可配置性因提供商而异。在 GPT-5.6 之前，缓存边界是根据最新请求自动确定的。尽管工具和系统指令很少变化，但它们本身并未被清晰地标记为可复用。
自 GPT-5.6 起，OpenAI API 允许客户端在其默认的隐式缓存之外，显式标记缓存断点。我们现在将断点放在请求的稳定层之后、不断增长的对话之前，让后续轮次能够复用更多未更改的前缀。
断点只有在缀本身保持稳定时才有用，因此我们还收紧了每个请求最前面的内容。为此，我们将工具和系统指令保留给很少变化的内容，并把更多易变的设置移过缓存边界，放入我们的“幻影用户消息”中。这里存放用户和请求特定的上下文，例如技能、子智能体和环境信息。
这些改动将冷缓存未命中率降低了 20%。
压缩文件读取
token 消耗的另一大来源是智能体在工作过程中添加的上下文，其中很大一部分来自读取文件。
Cursor 的智能体通过 Read 工具读取文件，该工具传统上会为每一行编号，因为模型本身不擅长数行，并且需要为用户引用特定片段。
单个行号只占用大约三到五个 token，但当智能体在一次会话中读取数万行代码时，为每一行都编号会额外增加相当可观的上下文量。
我们通过仅每隔十行标注行号来降低这一开销。这样的频率仍然足以让模型正确引用代码，而且这一改动将缓存读取 token 减少了 1.6%，质量没有任何下降。
策略性地使用子智能体
更长的智能体运行会创造更多将工作委派给子智能体的机会。这可以减少 token 消耗，因为每个子智能体通常从全新的上下文窗口开始，而不必携带父智能体的完整对话。一旦子智能体报告了结果，父智能体就可以继续运行，而无需携带子智能体的完整工作上下文。
不过，智能体与子智能体之间的这种上下文隔离确实会带来协调成本，因为不共享上下文的智能体可能会重复工作，或去执行已不再必要的任务。
我们做了两处改动，以在不增加不必要协调的前提下获得效率收益。首先，我们移除了强烈鼓励智能体使用子智能体进行代码库探索的指令。随着子智能体在训练数据中越来越普遍，研究人员也将其纳入后训练，模型已经原生学会了这一模式。移除额外的提示后，子智能体的使用变得更加均衡。
我们还收紧了子智能体选择模型的方式。Cursor 可以使用我们提供的任意模型来派生子智能体，从而能够弥补不同模型之间的盲点，或者将一个昂贵的规划模型与一个更便宜的模型配对用于实现。我们更新了工具参数，使智能体仅在用户或 harness 指示时才选择不同的模型。
持续提升 harness 效率
我们将继续测量上下文在更长运行中如何累积，并测试 harness 在哪些环节可以减少重复处理而不影响智能体质量。随着时间推移，我们预计这将使 token 用量的增长速度远低于智能体所能完成的工作量的增长速度。我们也将这些经验带到了 Grok Bot，正在努力优化其独特的 harness，以便用户能以最低成本完成最多的工作。
As agents have matured and learned to tackle more ambitious tasks, token spend has shifted. Agents now work for longer and carry more context from one step to the next, making the way we assemble and manage that context increasingly important.
Where agent inference spend goes
Production traffic · width = share of total spend · shade = billing type
- Output
- Uncached input
- Cached input
Where agent inference spend goes
Production traffic · width = share of total spend · shade = billing type
- Output
- Uncached input
- Cached input
Notes: System & tool defs includes compaction summaries. User text includes manually attached skills. Skills & plugins includes skill descriptions, MCP tool descriptions, and rules that go in static context.
Over the past few months we've responded to this shift by improving the efficiency of Cursor's agent harness. The harness gives us direct control over how each request is assembled, how context is reused, and when work is divided across agents. Changes across each of these layers reduced token costs for users by 7% without reducing agent quality.
Trimming the system prompt
Every agent turn includes context supplied by Cursor before the model begins working. This includes the system prompt and definitions for the tools the agent can use. Because this context is included throughout a conversation, it had become one of the largest sources of spend that we fully control.
When models were less capable, we had to spell out instructions for tool usage, task management, and code-change workflows. We also had to guard against strange behaviors like extremely long hash dumps, binary output, and emojis.
As models improved, much of that direction became unnecessary. Instead of long lists of "DO NOT do this," "You must," or "Important" instructions, we could simply define how a tool behaves and models would generally comply. This was true across model families, allowing us to trim roughly 66% of our system prompt.
Over time, we continue to add and remove instructions as new models require new guidance, which then flows into the training of future models. Leveraging A/B tests on a large user base is crucial to effectively optimizing the harness for real traffic. While evals can be a fast and useful proxy, they often represent "hard" problems and don't properly reflect the true distribution of user requests.
Loading tools only when needed
The system prompt is only one part of the context Cursor supplies on every turn. Another is tool definitions, which had grown dramatically over the course of the year as we added more powerful capabilities to the Cursor agent, including background shell monitoring, cloud subagents, and more reliable access to web content. Most of these tools are important, but each is needed in fewer than 20% of conversations.
That created an opportunity to improve efficiency by keeping tools available without including their full definitions in every request. We'd solved a similar problem earlier this year when we moved MCP tools into dynamic context, loading them only when needed. This reduced total tokens by 46.9% across sessions that called an MCP tool.
We have now applied the same technique to our own built-in tools.
To decide which tools to keep in static context, we A/B tested several configurations based on how often each tool was used and whether models needed to see it from the start. We tracked token usage, cost, latency, tool-call errors, and overall agent usage to make sure the savings did not degrade quality.
Most commonly invoked tools
Share of agent conversations invoking each tool at least once
Most commonly invoked tools
Share of agent conversations invoking each tool at least once
Ultimately, we kept the high-frequency tools for reading, searching, editing, and using the shell in static context. We also retained ask_question, which some models tended to hallucinate calls for, and tools that are crucial to specific product flows, such as create_plan in Plan Mode. The remaining tools now load when the agent needs them.
Offloading built-in tools cut static-context description tokens by 60%
- Kept in static context
- Offloaded to dynamic context
Offloading built-in tools cut static-context description tokens by 60%
- Kept in static context
- Offloaded to dynamic context
Improving cache reuse
After reducing the amount of static context in each request, we improved how effectively repeated context could be cached across turns.
Every agent turn resends a long request containing tools, system instructions, setup, and the conversation so far. Much of the beginning stays the same from one turn to the next, while the conversation at the end continues to grow.
Prompt caching allows the model provider to reuse that unchanged prefix. However, caching configurability can vary by provider. Before GPT-5.6, the cache boundary was determined automatically based on the latest request. Even though tools and system instructions rarely changed, they were not cleanly marked as reusable on their own.
Since GPT-5.6, the OpenAI API allows clients to mark explicit cache breakpoints alongside its default implicit caching. We now place breakpoints after stable layers of the request and before the growing conversation, allowing later turns to reuse more of the unchanged prefix.
Breakpoints only help if the prefix itself stays stable, so we also tightened what sits at the front of each request. We did this by reserving tools and system instructions for content that rarely changes, and by moving more variable setup past the cache boundaries into our "phantom user message." This holds user- and request-specific context like skills, subagents, and environment info.
These changes reduced the rate of cold cache misses by 20%.
Compressing file reads
Another large source of token spend is the context an agent adds as it works, much of which comes from reading files.
Cursor's agent reads files through a Read tool, which traditionally numbered every line because models are not good at counting lines on their own and need to cite specific sections for the user.
A single line number uses only around three to five tokens, but when an agent reads tens of thousands of lines during a session, numbering every one adds a meaningful amount of context.
We reduced that overhead by including line numbers only on every tenth line. This is still frequent enough for models to cite code properly, and the change reduced cache-read tokens by 1.6% with no reduction in quality.
Using subagents strategically
Longer agent runs create more opportunities to delegate work to subagents. This can reduce token spend because each subagent typically starts with a fresh context window rather than carrying the parent agent's full conversation. Once it reports its results, the parent can continue without carrying the subagent's full working context.
This kind of context isolation between agents and subagents does carry a coordination tax, though, because agents that do not share context can duplicate work or pursue tasks that are no longer necessary.
We made two changes to capture the efficiency benefits without adding unnecessary coordination. First, we removed instructions that strongly encouraged agents to use subagents for codebase exploration. As subagents became more prevalent in training data and researchers incorporated them into post-training, models learned this pattern natively. Removing the extra prompting produced more balanced subagent usage.
We also tightened how subagents select models. Cursor can spawn subagents using any of our available models, which makes it possible to shore up blind spots across models or pair an expensive planning model with a cheaper one for implementation. We updated the tool arguments so agents choose a different model only when directed by the user or the harness.
Continuing to improve harness efficiency
We'll continue measuring how context accumulates across longer runs and testing where the harness can reduce repeated processing without affecting agent quality. Over time, we expect this will allow token use to grow far more slowly than the amount of work agents can complete. We've also taken these learnings to Grok Bot, where we're working to optimize its unique harness so that users can accomplish the most work at the lowest cost.