GitHub Copilot CLI、GitHub Copilot app 和 GitHub Copilot SDK 全都由 Copilot agent runtime 提供支持,这是一个可以嵌入到应用程序和服务中的智能体运行框架。它最初是用 TypeScript 编写、运行在 Node.js 和 V8 JavaScript 引擎之上,服务于如今被称为 GitHub Copilot cloud agent(CCA)的产品,而随着该运行时及其能力快速扩展,它一直沿用着这套技术栈。
如今,这一切已经改变。借助 GitHub Copilot app 和 Copilot CLI,我们将该运行时完全重写为超过 800,000 行生产级 Rust 代码。AI 智能体编写了其中大部分代码,横跨 128 个 pull request,这些 PR 陆续合入 main 并逐步发布,而不是等到最后一次性切换。
少数不可避免的回归问题也在过程中被迅速发现并修复,同时运行时的性能提升了好几个数量级。一个在智能体出现之前需要一整个开发团队花上一两年才能完成的项目,如今主要由一名开发者仅用几个月就完成了,而团队其他成员在此期间还在持续大幅扩展该运行时的能力和覆盖范围。
为什么我们需要移植
Copilot 智能体运行时不仅仅是 Copilot CLI 背后的引擎。它还支撑着越来越多的 Microsoft、GitHub 以及生态系统解决方案,对于其中每一个而言,从架构上看,AI 支持都是围绕同一个运行时构建的外壳,再加上该解决方案所需的定制化内容。这不仅包括 GitHub Copilot CLI 和 GitHub Copilot 应用,还包括最新发布的 VS Code、Visual Studio、CCA、Copilot Code Review(CCR)、Copilot Cowork、Copilot Studio,以及 Excel、Outlook、PowerPoint、Word……还有更多。
这些是非常不同的产品,它们中没有哪一个想要、也不应该需要去实现生产级智能体 harness 所包含的一切。它们想要的是全部的智能、安全、可靠性和性能,并且希望这些能够被共享,这样一处修复就能修复所有产品。上一段中列出的大多数产品最初都实现了自己的智能体循环,但此后已将其替换为 GitHub Copilot SDK,后者是进入 Copilot 智能体运行时的入口。
这样做使它们能够专注于自身的核心业务价值,而把细节交给运行时处理。考虑到行业的节奏,以及所采用的智能体循环需要在激烈竞争中始终保持最佳水准,这一点就更加重要。
所以,共享运行时,很好。问题在于被共享之物的本质。
如果我们看一下这个 CLI,它在逻辑上就是一个构建在智能体循环之上的终端 UI(TUI)。实际情况是,整个技术栈都是用 TypeScript 实现的,使用 Node.js 作为框架、V8 作为执行引擎,并用 Ink 和 React 来做 UI。
对于一个 TUI 应用来说,这是相当合理的选择;TypeScript 和 Node.js 门槛低、易于上手,能实现非常快速的应用开发。而且对于控制台应用的需求而言,它在启动速度、响应性、吞吐量和内存消耗方面的性能影响也是合理的。
但遗憾的是,当你考虑到这套实现要被用在其他环境中、面对其他约束、并且需要满足诸如快速启动和因低内存开销而带来的出色服务器密度等要求时,这些性能表现就远没有那么合理了。
CLI 及其运行时的架构也加剧了这里的挑战。整个行业跑得极快,在这种背景下,真正聪明的人会为了交付速度和市场覆盖而做决策。Copilot CLI 最初是快速编写并发布的,在这样做的时候,TUI 和运行时相当紧密地交织在一起,而不是被分离成离散的层。后来当需要一个 SDK 来以编程方式访问该运行时时,由于各层之间没有清晰的分离,人们做出了一个务实的决定:把 SDK 叠加在 CLI 之上,尽管从逻辑上讲你会期望相反的架构。CLI 不再只能通过用户在命令行提供的命令来访问,而是被更新为一种可以无头运行的模式,从 stdin 读取类似的命令并将响应写入 stdout。随后可以使用 JSON-RPC 协议,将来自外部进程的函数调用与 CLI 之间进行编组。这样 SDK 就可以嵌入到任意的消费程序中,这些程序会生成一个 CLI 进程,在进程外托管智能体循环,而 SDK 则通过这种 JSON-RPC 机制调用远程进程中的函数。很巧妙。能快速出门。很灵活。但对于那些消费类应用的性能(启动、内存、吞吐量)和可靠性来说并不理想。从 SDK 创建一个新的 CopilotClient 意味着要再生成一个进程:
const client = new CopilotClient();
await client.start(); // spawns the CLI as a subprocess
const session = await client.createSession({
/* ... */
}); 这个过程需要启动并托管 Node 和 V8。这意味着要解析 CLI 中由 TypeScript 代码生成的大量 JavaScript,为其生成字节码,并可能在后续的 JIT 层级中优化热点代码。这意味着要承担与 V8 相关的全部内存开销。
这意味着要继承 Node 的线程模型,而该模型默认会迫使我们把所有 CPU 密集型工作串行化。这意味着仅仅为了进行函数调用,就不得不进行强制性的跨进程通信。这意味着每一种语言的每一个 SDK 使用者,都要附带 Node.js 或一个包含 V8 的打包二进制文件。
这意味着 C#、Python、Go、Java 和 Rust 的 SDK 都要为每个客户端额外付出整整一个语言运行时的代价,最低工作集大约在 100 MB 量级,而这个运行时对其应用而言别无他用。这意味着每一个事件、每一条消息,以及每一次抽象会话文件系统的读写,都要跨越进程边界。
这意味着 Node 的一次崩溃会连带整个会话一起崩溃。这也意味着任何部署它的人,至少都要监督、监控和调试两个进程。
相反,我们想要一个运行时:
- 它不包含 TUI,TUI 是它自己的库,TUI 以及其他应用和服务可以干净地分层构建在其之上。
- 以依赖极少、开销极低的语言实现。
- 以能够干净地嵌入进程内、而非被迫置于进程外的方式实现。
- 以在性能、可扩展性和可靠性方面具备顶尖特性的语言实现。
- 用一种非常适合互操作的语言实现,从而能够通过各技术栈的外部函数接口(FFI)机制,被全部六种 Copilot SDK 语言版本(C#、TypeScript、Python、Rust、Go、Java)干净地使用。
- 用一套提供更现代安全态势的工具链实现,供应链风险更低,并且对构造即正确(correct-by-construction)代码有更好的支持。
基于以上所有理由,以及一些较软性的理由(例如团队经验和行业方向),我们选择了 Rust。这绝不意味着每一个大型 TypeScript 程序都应该改成 Rust。我们的需求强调通过 C ABI 进行嵌入、低启动开销和稳态开销,以及可预测的资源使用。Rust 让这些目标成为可能,代价是带来了其他复杂性,例如我们必须显式地表示生命周期和共享状态(后文讨论的生命周期回归问题正凸显了这一影响)。合适的目标语言确实因应用而异。
随后开展了两项关键且相关的任务:
- 将 TUI 专属代码与运行时分离,使前者严格分层构建于后者之上,更具体地说,严格分层构建于 SDK 的公开接口之上。如今,CLI 仍在若干处直接调用运行时内部实现;将其完全迁移到 SDK 的接口之上仍是进行中的工作。
- 将该运行时层 100% 移植到 Rust,最终形成一个纯原生二进制文件,对外暴露 C ABI 供所有语言前端在进程内调用,同时提供一个基于 stdin/stdout 或基于 socket 的服务器,以备仍需进程外调用的情况。
本文主要讨论第二点:将运行时移植到 Rust。
之前的样子
2026 年 5 月初的初步移植计划估计该运行时大约有 130,000 行 TypeScript 代码。就界定范围而言,这一初步测量相当准确,但事实证明,它在两个关键方面也极具误导性。在移植的同时:
- 仍封装在 TUI 层中的部分正被下推到运行时层。最初在估算中被忽略的整个组件以及相当大比例的代码,随后又被认为与移植相关。
- 提交的 Pull Request 带来了大量新增 TypeScript 代码,不断推高仓库中 TypeScript 的总量。数十名借助智能体辅助的开发者每周合并数百个 Pull Request。
把所有因素都算进去,我估计大约有 430,000 行生产环境 TypeScript 代码最终经过了这次移植。这些同样的因素也让人难以看清过程中的进展:直到接近尾声时,生产环境 TypeScript 的代码量看起来仍相对稳定,甚至略有增长,因为移植的速度一直勉强跟上新进来的工作。
这一点更加令人困惑,因为在这段时间里,除了移植工作之外,还有新进来的 Rust 代码;在移植工作早期,新进来的代码更可能以 TypeScript 为主,而到了后期,则更可能以 Rust 为主。
在移植过程中,运行时接收了约 300,000 行生产环境的 TypeScript 代码,同时移除了约 430,000 行;而约 1,200,000 行生产环境的 Rust 代码进入,约 365,000 行离开。换句话说,上图中 TypeScript 行数表面上的稳定,实际上掩盖了大量的 TypeScript 代码变动。
原地移植策略
该图表还凸显了这次移植方式的一个重要方面:原地进行。
对于这种规模的改写,主要有两种方法:
- 大爆炸式。新的 Rust 运行时作为完整的替代方案进行开发,然后在准备就绪时一次性整体替换。这种大爆炸式切换有两种变体。a. 全员停工。在改写进行期间,所有人停止在
main分支上的其他工作,改写工作在main中完成。b. 并行开发。改写在一个功能分支中进行,同时主分支上的工作继续进行,改写工作不断尝试跟上并合并主分支的变更。 - 原地进行。这是以逐组件移植的方式完成的,运行时被一块一块地增量重写。这种原地方式也有两种变体。a. 原子替换。每一块都从 TypeScript 原子性地切换为 Rust,剩余的 TypeScript 与新的 Rust 之间的互操作提供了连续性。随着时间推移,生产运行时中 TypeScript 越来越少,Rust 越来越多,直到有一天,不再有 TypeScript,只有 Rust。b. A/B。与其在组件被移植后就删除它们,不如将 TypeScript 和 Rust 组件都作为可热插拔的选项加以维护,一旦信心趋于平稳,就删除 TypeScript。
出于多种原因,我们选择了方案 2a:
- 没有人会经历工作中断。主分支继续保持活跃。每一位未直接参与移植的开发者都可以照常继续工作,只有当他们在长时间内一直在推进的某个 pull request 恰好触及了同时被移植的代码时才会受到影响,在这种情况下,他们需要 rebase,并让他们的智能体帮助只移植他们正在推进中的改动。
- 运行时的主分支始终可发布。每个 pull request 都用一层调用 Rust 的薄垫片替换现有的 TypeScript 实现,并在一次原子性变更中删除旧代码。新代码会立即在原地被实际运行。
- 重写是增量式且可审查的。每个 pull request 只移植单个组件或切片,因此变更范围更小,diff 也更容易审查,无论是人工、智能体还是两者兼有。
- 大多数移植都相当小且自成一体,从而最大限度地减少了与并发 pull request 之间的偏移。在某些情况下,当 TypeScript 组件过于庞大时,可以先将它们重构为更易于移植的组件。
- 所有现有的端到端测试,涵盖 CLI 和 SDK,在每一步都针对新的 Rust 代码运行,这给了我们信心和大量的验证。如果某个 pull request 导致必需的测试失败,它就不会被合入。
我们也避开了 2b 方案的一个变体,即同时维护同一组件的多个版本。过去几个月里,每周都有数百个 pull request 被合并进仓库,代码库在不断演进,而且速度很快。让同一份代码以两种不同语言、两套不同依赖库的形式存在,会带来极大的复杂性。而且其中一些组件并非完全隔离;有些在逻辑上是独立的,其余系统可以通过简单的 API 访问它们,但另一些则有大量牵连,要让这张图按组件热插拔简直是一场噩梦。最有可能从谨慎的并行切换中获益的子系统,恰恰是并行最难的那些。例如,会话编排并不是一个纯函数,不能靠某个实验开关用 if/else 去调用两个不同版本。它持有可变状态,双向驱动回调,并贯穿几乎所有其他子系统,因此“两个都跑并对比”意味着要维护两份分叉的副本——一份承载对话状态与服务的组件副本,还要祈祷它们能在数百次并发编辑中保持同步。让一个组件难以移植的那种耦合,也正是让它几乎不可能在不引入比避免的更多回归的情况下进行影子运行的那种耦合。能够以这种方式切换所带来的好处,主要在于获得信心,而我们也可以通过其他方式获得信心。
验证也通过增量式发布来完成。如果采用一次性切换的方式,我们会把所有内容放在一个长期分支中,移植整个运行时,然后一次性切换。这意味着使用者会一次性体验到所有移植的代码行,包括在仓库内测试中漏掉的所有回归问题。以增量方式分批发布变更——这里两个组件、那里一个组件——使我们能够在已部署的构建中获得最后一公里的验证,并伴随真实的使用者使用(通常是 Microsoft 和 GitHub 内部的第一方使用者),同时将回归风险降到最低。在大约十四周半的移植窗口期内,main 发布了 135 个版本,其中包括 100 个预发布版本和 35 个稳定版本,平均每天约 1.3 个版本。每天也大约有 1.3 个移植 pull request 被创建,因此每个版本都携带一组规模小且可确知的已移植组件(我们通常尝试先在预发布版本中发布移植,但并非总能成功)。在最近七天的 npm 采样中,预发布版本仅占下载量的 10.5%,这表明在监控反馈渠道以发现故障信号并在下一个预发布版本中迅速修复期间,初始暴露面相对有限。所报告的问题更容易与已知的近期变更相关联,也更容易定位根因并快速修复。通过这种方式,在较长时间内以增量方式进行移植实际上是一种优势而非阻碍(也就是说,更快并不总是更好)。到 8 月 21 日,运行时已 100% 为生产 Rust:832,378 行生产 Rust 代码和 468,689 行 Rust 单元测试,此外还有 174,675 行 E2E TypeScript 测试。独立的 GitHub Copilot SDK 仓库又增加了约 130,000 行 E2E 测试代码,涵盖 Node.js、Python、Go、C#、Rust 和 Java。
开始上手
在全力投入之前,我们先建立了信心并验证了可行性。我们首先提交了两个 pull request,搭建了 Rust 工作区、工具链、lint 规则、CI、构建流水线和编码规范,随后引入了运行时 crate 以及代码生成和互操作模式,同时移植了一批纯逻辑原语——这些原语是特意挑选的,因为它们没有 I/O 或共享状态,并且已有完善的测试。
只有在这些合并之后,第一个主要移植 pull request 才将三个无副作用的辅助函数走完了完整流程。这些起到了试点发布的作用,把关于仓库布局、FFI、打包、测试和审查的假设转化为约定,供后续规模大得多的移植工作复用。
基本上,我们对整套机制进行了端到端测试。计划继续按照从叶到根的顺序推进工作,以纯辅助函数、内容排除、shell 工具和会话文件系统操作来建立翻译和测试模式。随后是有状态子系统,工具、hooks、模型客户端和 MCP 则构建在这些基础之上。
会话编排(迄今为止运行时中耦合度最高、最不适合自然并行的部分)被安排在接近尾声时进行。
| 周期 | Pull request 数 | 变更行数中位数 |
|---|---|---|
| 5 月 1 日–15 日 | 8 | 3,250 |
| 5 月 16 日–31 日 | 2 | 9,421 |
| 6月1日–15日 | 40 | 5,073 |
| 6月16日–30日 | 31 | 8,253 |
| 7月1日–15日 | 10 | 9,514 |
| 7月16日–31日 | 14 | 28,159 |
| 8月1日–15日 | 19 | 13,861 |
| 8月16日–30日 | 4 | 99,445 |
早期的移植工作,即那些小型叶子组件,推进得很快。但更大的子系统并不是以单个原子步骤完成迁移的;例如,MCP 支持经历了七个专门的 pull request 才逐步推进,而工具则通过一个由六部分组成的系列逐步推进,之后还需要额外的工作来迁移编排逻辑并淘汰剩余的 TypeScript。Hooks、auth、telemetry、plugins、settings 和 persistence 也走了类似的路径。
实际上,有用的移植单元并不总是“一个组件”。它往往是一次穿越相关行为区域的浪潮:先迁移纯逻辑,再迁移状态归属,然后迁移编排,接着移除回退逻辑,最后在临时互操作层消失后简化 Rust 代码。
互操作
在这次移植中,涉及互操作的主要有两个层面:
- 临时内部互操作。每当一个函数被移植到 Rust,该函数都需要能够被调用初始 TypeScript 函数的任何 TypeScript 代码所调用。同样,我们也需要让 Rust 函数能够调用 TypeScript 回调。这种互操作需求属于实现细节,且极其多变。随着 Rust 内部接口面积的增长,所需的 TypeScript 垫片数量也随之增长,因为它们与需要从 TypeScript 调用的 Rust 方法一一对应。当那些调用方被移植到 Rust 后,现有的垫片层就会被删除,并换上新的层。最终,当我们到达运行时库的公共入口点时,垫片就消失了。
- SDK 层面。所有 SDK 库都需要能够架设在运行时之上并暴露其功能。在移植前的世界里,这是通过一个双向 JSON-RPC 层来暴露运行时实现的:SDK 将函数调用请求作为 JSON-RPC 方法调用负载发送,运行时解析请求并调用相关 API,然后通过同一传输通道将结果发回,供 SDK 解析并返回。反方向同样存在;运行时需要能够回调 SDK 客户端,例如用于钩子通知和权限请求,这些在 SDK 客户端中以回调形式呈现,使用各语言中被视为惯用的语言特性(例如 C# 中的委托)。
我们通过 napi-rs 项目的 napi Rust crate 实现了第(1)点,该 crate 的存在就是为了用 Rust 构建 Node 原生插件。你只需给一个函数标注 #[napi],napi-rs 宏就会生成 N-API 注册胶水代码,使该函数可从 JavaScript 调用,并在生成的 index.d.ts 中为其生成 TypeScript 声明。一个同步的 Rust 函数会变成普通的 JavaScript 函数,一个 async fn 会变成返回 promise 的 JavaScript 函数,而标注了 #[napi(object)] 的结构体会在另一侧变成普通对象。
流量还必须双向流动。许多已移植的组件暂时依赖于尚未移植的东西,因此 Rust 需要回调到 TypeScript,例如 Rust 中的工具实现向仍是 TypeScript 的模型层请求推理,或触发一个 hook,或为它想要运行的命令请求权限决策。
napi-rs 通过“线程安全函数”来处理这一点,它让运行在 Tokio 工作线程上的 Rust 代码能够回调到 Node 主线程上的 JavaScript 回调。Node 安装一次回调,Rust 持有它,并在需要反向调用时随时调用它。
每一个这样的回调从构造上就是临时的:回调之所以存在,只是因为另一端的东西仍是 TypeScript,而当那个东西被移植后,它就会被删除。
这个临时接缝在 8 月 3 日达到峰值,有 2,019 个内部 N-API 导出和 3,356 个 TypeScript 调用点。完成时,运行时完全由 Rust 构成,因此没有内部互操作:0 个临时内部 N-API 导出和 0 个 TypeScript 调用点留存。(我前面提到过,CLI 仍然对运行时有一些内部访问,我们正在努力移除;那些导出不在此处计数。)
第二层互操作层,即 SDK 表面,是两者中永久性的那一层。Copilot SDK 面向六种语言发布:TypeScript、Python、Go、C#、Java 和 Rust。它们全都使用相同的双向 JSON-RPC 契约,最初它们都以相同的方式接入它:以无头模式将 Copilot CLI 作为子进程启动,并通过管道或套接字与之通信。
在移植期间,这仍然是默认方式。这也意味着任何语言的 SDK 使用者都要随附或定位一个完整的 Node 实现,在每一个事件和每一条消息上都要付出一次进程跳转的代价,并且要监督两个进程而不是一个。
将运行时移植到 Rust 正是让另一个选项变得可行的原因。所发布的 runtime.node 是一个普通的平台共享库(.node 扩展是 Node.js 原生插件的约定;其底层是一个 .dll、.so 或 .dylib),而它现在为同一个引擎提供了两扇前门。一扇是 napi 门,Node 进程将其作为原生插件加载;这是 CLI 的路径(目前……未来,意图是它将完全走 SDK 路径)。另一扇是 C ABI 门,任何语言都可以将其加载到自己的进程中并通过 FFI 调用。同一个进程内运行时通过每种语言的原生互操作机制被选用:
| SDK | 原生桥接 | 进程内客户端选择 |
|---|---|---|
| C# | P/Invoke | new CopilotClient(new CopilotClientOptions { Connection = RuntimeConnection.ForInProcess() }) |
| Go | purego | copilot.NewClient(&copilot.ClientOptions{Connection: copilot.InProcessConnection{}}) |
| Java | JNA | new CopilotClient(new CopilotClientOptions().setConnection(RuntimeConnection.forInProcess())) |
| Python | cffi | CopilotClient(connection=RuntimeConnection.for_inprocess()) |
| Rust | libloading | Client::start(ClientOptions::new().with_transport(Transport::InProcess)).await? |
| TypeScript | koffi | new CopilotClient({ connection: RuntimeConnection.forInProcess() }) |
Rust 重写与进程内、进程外托管方式的选择是两个相互独立的维度。已完成的 Rust 运行时同时支持两者:它既可以运行在 SDK 使用方的进程内,也可以运行在现有 JSON-RPC 服务器边界之后。目前这些进程内入口点属于选择性启用,因为我们还在逐步建立与使用方应用共享同一进程、进而共享同一故障边界的信心。
传输层之上的一切仍保持相同的 SDK API:会话、事件、工具、权限和回调并不关心它们的 JSON-RPC 字节是通过管道还是函数调用传递的。
第二扇门的有趣之处在于它的规模。它只导出了 19 个函数:4 个用于服务器生命周期,4 个用于会话注册与配置,8 个用于连接,还有 3 个用于嵌入式宿主。在这些函数背后,共享契约目前包含 364 条分发路由:其中 340 条可由 SDK 使用方调用,另有 24 条以相反方向运行,作为运行时到 SDK 的回调。
napi 这扇门则大得多,需要为每一条分发路由都提供函数。C ABI 这扇门是基于分发的:API 方法根本不会获得导出,而是以 JSON-RPC 字节的形式写入连接中传输,结果、事件以及服务器到客户端的请求则通过宿主提供的回调返回。
新增、修改或移除一个 API 方法只会触及引擎的分发表,而绝不会触及 ABI。SDK 只需一次性绑定这 19 个入口点,就能通过它们动态触达整个仍在不断增长的 API 表面。
这就引出了一个显而易见的问题:在一次不再跨越进程边界的调用中,为什么仍然存在 JSON-RPC?
答案是,这让进程内托管成为即插即用,而非重写。每个 SDK 都已经有了一个可用的 JSON-RPC 客户端,具备分帧、请求与响应关联,以及处理服务器到客户端方向的处理程序。将 FFI 作为该客户端底层的又一种传输方式挂载上去,就把字节路径从管道或套接字转移到了函数调用,而上层的一切保持不变。
六个 SDK 以增量式、可选启用的传输方式获得了进程内托管,现有的传输方式则保持不变。如果我们改为为每个 API 方法定义一个带类型的 C 函数,那么每个 SDK 都需要第二层绑定,每个新增的 API 方法都需要再多六个绑定,而 ABI 也会变成一个我们必须对其进行版本管理的二进制兼容性表面。
对于真正处于远程的运行时——无论是跨越子进程边界还是通过 TCP——我们仍然需要 JSON-RPC。在进程内保持相同的协议,意味着只需维护一套双向 API 和分发系统,而不必为远程连接维护 JSON-RPC、再为本地连接维护一套按方法划分的 FFI 接口。
这是一个实实在在的权衡,而不是一个显而易见的决定。我们避免了进程跳转,但每次调用仍然要付出 JSON-RPC 的开销。对于以推理为主的工作负载,与模型往返相比,这种序列化开销通常微不足道。在高吞吐的本地工作负载中它仍然可测量,但如今还不足以证明值得在六个 SDK 绑定中重复实现数百个方法。
而且这是一个我们日后可以轻松修改的决定,只要性能需求真的出现。载荷编码是两端之间的私有细节,把 JSON 换成 MessagePack 之类更紧凑的格式,不会改变任何一个已声明的导出。按方法划分的类型化导出也可以日后为热路径添加,调用同一个引擎和同一批处理器,而无需替换字节通道——该通道仍将是流式传输、服务端到客户端请求,以及那些极少调用的长尾方法的底层基础,为这些方法定制导出毫无收益。
会话数据说明了什么
这篇博文中的几乎每一个数字都来自以下两个来源之一。第一个来源是私有 github/copilot-agent-runtime 仓库的 GitHub 历史记录:pull request 及其 diff、审查评论、CI 运行等。第二个来源是智能体会话日志。运行时(以及 CLI、应用等)会为其运行的每个会话写入结构化事件日志:每行一个 JSON 对象,随会话发生而追加。这些日志可能包含提示词、命令、命令输出、文件路径,以及工具可能暴露的密钥,因此必须作为敏感数据处理。日志存储在会话运行所在机器的本地;远程会话功能在启用时也可以上传日志,但须遵循产品设置和组织政策。
以下是所有构成移植 pull request 的数据汇总:
| 指标 | 数量 |
|---|---|
| 事件 | 12,760,995 |
| 用户消息 | 31,247 |
| 助手消息 | 1,385,214 |
| Hook 开始和结束事件 | 6,438,562 |
| 工具启动 | 1,857,409 |
| 编译命令 | 23,096 |
| 测试命令 | 19,485 |
| 变基命令 | 2,496 |
| 提交命令 | 7,410 |
| 推送命令 | 5,554 |
| 已完成的压缩 | 5,116 |
这 31,247 条 user 角色消息并不是我本人敲下的 31,247 条提示词;它们包括技能指令、自动化的合并标记、跨会话消息以及子智能体流量,远超我输入或口述的大约 2,600 条,约占十二分之一。同样,1,385,214 条 assistant 消息包括子智能体和面向工具的消息,而不只是对话式 UI 中展示给我的文本。
该语料库包含 68 种不同的事件类型和 67 个不同的工具名称;1,130,921 次工具调用中,有 61% 来自子智能体,而非主会话线程。
这个数字并不能说明我为什么要把自己插入约 2,600 次。为此,我让 Copilot 为会话日志语料库中每条由人类撰写的消息分配一个主要意图。
前三个类别占了我交互的 63%。其中只有约 40 次是我可识别的会话启动,因为这主要发生在我先创建一个聊天来探索下一个前沿,然后要求该聊天会话为每个期望的切片创建实际的移植会话。我的角色与其说是“分配任务然后等待”,不如说是“操作控制回路”:检查结果、质疑技术决策、执行质量门禁,并在智能体把中间停止点当作终点时推动其继续。
尽管智能体在做“工作”,人类判断仍然深度参与其中。我的参与只是向上移动了……不再是负责编写语法,而是负责框定问题、定义边界、选择策略、裁决例外情况,以及总体上确保一切朝着好的方向推进。
一切都关乎缓存
大语言模型服务商通常对输入 token(你发送给它们的内容)和输出 token(它们返回给你的内容)分别按不同费率计费。计费之所以常以 token 为单位,是因为 token 能较好地近似完成推理所需的计算量:对于每个输入 token,模型必须读取它、将其纳入内部表示,并在决定下一个 token 的计算中加以使用。
不过,服务商往往支持缓存这些计算的结果,这样一来,如果某个提示词的相同前缀已经被处理过,服务商就可以复用缓存中的中间计算结果,而不必从头重新计算。这降低了处理这些 token 的成本,而节省下来的费用可以传递给用户。因此,输入 token 通常会公布多种费率,其中包括从缓存中读取的输入 token 的费率。
折扣力度非常大!服务商通常对缓存命中按 90% 的折扣计费,例如某服务商可能对 100 万输入 token 收费 $2.00,而对 100 万缓存输入读取 token 仅收费 $0.20。换句话说,你非常非常需要维护好提示词缓存,这样你的账单就能少一个数量级。
这次移植工作的数据表明,我们在这方面做得很好。提示词缓存命中率为 96.22%:缓存读取量除以所有输入侧 token 总量(缓存读取加缓存写入加全新输入)。缓存写入占 3.07%,全新输入占 0.71%。这并非偶然。GitHub Copilot 专门塑造了智能体循环,以维护一个长而稳定的前缀(先是系统提示词,然后是工具定义,再是累积的对话),因此每一轮都向模型已经处理过的上下文追加内容。
上下文中昂贵的部分只需支付一次,之后每次调用都以低一个数量级的成本重新读取。这也正是长时间自主会话在经济上能够成立的原因。一次三百小时的移植,如果在数万次调用中的每一次都从头重新读取整个不断增长的上下文,其成本将与我们看到的相差一个数量级。
智能体框架开发者花费大量精力试图避免破坏提示词缓存,而模型厂商也经常发布新功能来帮助他们做到这一点。
上下文压缩(Compaction)则讲述了另一个相辅相成的故事。在所有移植会话中,GitHub Copilot 自动压缩上下文 5,116 次(即会话填满上下文窗口后,为了继续运行而自我总结的时刻)。那个单独的 sessions-infrastructure 移植 pull request 在其持续多天的生命周期中压缩了 647 次,而有一个小型移植则从未压缩过一次。
持续数百小时的自主工作之所以可能,正是因为智能体能够一遍又一遍地回收其工作记忆而不丢失线索。这数千次总结中的每一次,都是一个有损交接本可能悄然让移植脱轨的节点,而大多数情况下并没有。Copilot 的子智能体在最小化压缩方面也起到了很大作用。
每个子智能体都拥有自己的上下文,因此父会话可以有效地提出问题,让子智能体去计算答案并在此过程中消耗相当多的上下文,然后只把答案报告回父会话。父会话的上下文无需被所有这些中间信息所影响。
上面所说的“大多数情况下并没有”,在会话日志中清晰可见。我让 Copilot 将每一次成功的压缩与其前后至少各有 20 次工具调用的工作配对。这产生了约 4,000 个可比较的窗口。智能体在压缩前 20 次工具调用中所做的事情,与压缩后所做的事情在规模上看起来相似(探索:压缩前 46.5%,压缩后 48.1%;修改:压缩前 8.4%,压缩后 6.0%;验证:压缩前 4.7%,压缩后 4.0%;失败:压缩前 1.0%,压缩后 1.5%)。
如果压缩经常丢失思维线索,我们会预期压缩后一侧明显偏向重新定位,出现阅读量激增而编辑量骤降,因为智能体在重新发现自己在哪、应该做什么。然而,实际上只出现了朝那个方向的轻微偏移。
是的,静态分析有帮助
有一个流行的说法认为,Rust 是 AI 生成代码格外理想的目标,因为 Rust 严格的编译器能捕捉到模型出错的地方。这些会话日志让我们得以检验这一理论,至少对于类似这次移植工作的任务而言。
直接验证命令的结果捕获了 8,678 次 rustc 错误代码的出现。四个最大的诊断类别覆盖了 84%:
- 37%:名称与导入解析,其中以
E0425为主(“cannot find value in this scope”) - 22%:缺失的方法或字段
- 14%:类型不匹配
- 11%:未满足的 trait 约束
这些全都是普通的接线问题:某个名称输出略有偏差、某个签名对不上、某个字段被重命名了、某个抽象尚未实现。这类错误正是批量翻译容易且不经意间产生的,也正是编译器能极快捕捉到的。
但请注意这份清单里缺少的东西:任何真正属于 Rust 特有的内容。这四个类别中的每一个都是最基础的静态类型检查,C#、Java 或 Go 编译器同样能全部捕获,其中好几项还能给出更友好的诊断信息,而且速度都要快得多。如果这就是把智能体引向 Rust 的理由,那它实际上是把智能体引向任何静态类型语言的理由。一个强类型编译器和/或具备出色静态分析与 lint 能力的语言,确实非常适合这类工作,智能体可以把它当作快速反馈回路。在 4,478 次直接 cargo check 运行中(更严格的结果匹配器捕获到了结果),87.1% 返回干净结果,这正是以小增量编辑并不断重新编译所得到的效果。
相比之下,所有权、借用和生命周期错误加起来仅占已编码诊断的 1.7%。借用检查器——这个在每一场关于 Rust 难学的讨论中占据主导地位的东西——只是一个安静的后台存在。编译器几乎把所有报错精力都花在了无聊的机械性错误上。
智能体喜欢阅读
我们还可以检查会话事件语料库中的工具调用数据,并从中提取一些关于智能体如何分配时间的有趣观察。
| 工具 | 调用次数 | 中位数 | 实测小时数 |
|---|---|---|---|
powershell | 630,423 | 3 s | 2,833.9 |
view | 590,988 | 0 s | 621.7 |
rg | 281,783 | 1 s | 408.4 |
grep | 126,483 | 1 s | 115.3 |
apply_patch | 53,715 | 0 s | 17.0 |
edit | 40,591 | 1 s | 24.1 |
read_powershell | 36,728 | 90 s | 1,203.9 |
task | 13,080 | 274 s | 2,329.0 |
我从中得到的第一个结论是,智能体花在收集证据上的时间远远多于修改代码的时间。从所展示的文件读取和搜索工具与编辑工具的对比来看,它们所做的探索是修改的 10 倍。读取文件、搜索代码仓库以及运行诊断命令占据了主导地位;编辑相比之下只是一项很小的投入。AI 疯狂输出代码的流行印象几乎恰恰相反;在这个规模上,工作看起来更像是反复的调查——检查当前状态、形成假设、做出有针对性的修改、如此循环往复。
委派机制放大了这一模式。子智能体主要用于在相互独立的问题上并行展开探索,而主智能体则更可能负责实际的编辑并整合答案。对于这类项目而言,这是一种有用的分工方式:多个上下文可以并行调查,但将变更操作保持在协调智能体附近,可以减少冲突性修改,并保持连贯的实现策略。
shell 流量还表明,自主软件工作中有多大比例是在做状态管理。只读的 Git 检查是最常见的命令模式,因为智能体在不断询问的实际上是“我现在在哪里?”它们在查找哪些内容发生了变化、一次 rebase 做了什么、另一个会话提交了什么,以及一个分支与快速演进的 main 之间偏离了多远。这种定位工作使得许多长时间运行的任务能够在同一个不断变化的代码库上运作,而不会盲目地相互覆盖。
深入查看 shell 工具流量后,最常见的命令类别让定位与验证之间的这种平衡更加清晰:
| 命令类别 | 调用次数 | 中位数 | 实测小时数 |
|---|---|---|---|
git inspect | 300,530 | 2 s | 608.1 |
git 其他 | 89,865 | 3 s | 243.1 |
| 搜索 | 85,482 | 2 s | 147.6 |
pnpm test | 13,852 | 22 s | 219.1 |
pnpm lint | 9,757 | 29 s | 177.0 |
cargo test | 8,437 | 120 s | 364.2 |
git commit | 7,410 | 11 s | 39.7 |
cargo fmt | 5,223 | 18 s | 77.2 |
cargo check | 4,492 | 120 s | 176.9 |
pnpm build | 3,630 | 180 s | 215.6 |
cargo clippy | 2,115 | 135 s | 107.4 |
git rebase | 2,496 | 7 s | 9.9 |
cargo build | 566 | 104 s | 20.3 |
模型选择
GitHub Copilot 允许单个会话在对话中途切换模型,也允许不同会话运行不同的模型,因此模型选择变成了按切片(per-slice)做出的决策。日志中出现了两种不同类型的模型决策。在主线程上,也就是驱动每个端口的那条线程,由我们来选择模型和推理强度。而在会话内部,当智能体启动子智能体或子会话去探索或攻克一项有边界的任务时,则由负责编排的模型来选择这些模型。
对于子智能体而言,模型组合看起来略有不同,因为是由智能体而非人类来优化吞吐量和成本,而不是做出最困难的判断。它最常生成的子智能体运行在 Claude Opus 4.8、GPT-5.6 Sol、Claude Haiku 4.5 和 GPT-5.5 上,其次是 Gemini 3.1 Pro 和 Claude Opus 5。然而,至少在移植进行的那段时间,有三个频繁使用的智能体定义固定了它们的模型选择(explore 和 task 使用 Claude Haiku,research 使用 Claude Sonnet),因此这一用量中有相当一部分是由子智能体的选择决定的,而不是单独选择模型的结果。
与智能体集群协作
GitHub Copilot 应用支持可视化活跃的拉取请求会话、查看每个会话的状态,并在它们之间轻松切换,这使其非常适合管理移植工作所固有的大量并发任务。但真正让它出彩的一点,是它能够让会话与其他会话进行交互。
一个会话可以创建其他会话,并且可以在其他会话运行时向它们发送消息。每个会话,无论是父会话还是子会话,都拥有自己的工作树、自己的分支和自己的智能体循环;它独立于创建它的会话,而不是在其中运行的东西。这与子智能体不同,子智能体在父级自己的工作区内运行,并将其答案交回父级的上下文中。两者都是对不同用途有用的构造。
举一个会话可能创建其他会话的例子,最困难的移植之一针对的是 session.ts 文件。这个文件有机地增长到了约 30,000 行 TypeScript。它代表了会话的骨干,实际上横向贯穿整个运行时,几乎与每一个组件相互触碰、相互影响,处于状态、事件、工具、模型、钩子和入口点访问的中心。因此,我把它留到了移植过程的接近尾声,从技术栈底部向上、跨越所有垂直方向推进,直到它们全部在 session.ts 处走到尽头。接手它的移植会话并没有一上来就直接动手写 Rust。它最初的五十六分钟都在阅读,在创建任何东西之前进行了 122 次工具调用,构建出这个文件实际拥有什么、接缝在哪里的图景。直到那时它才开始委派,把这个文件按逻辑拆分,并将切片委派给子会话。在整个 25 小时的运行中,它自己进行了 222 次 shell 调用、205 次文件查看和 197 次 ripgrep 搜索,这还不算它的子会话所做的一切。

那就是 15 个子会话,每一个都是独立的分支,拥有自己的工作树和独立的智能体,全部由顶层的父会话隐式创建。父会话在大约三个小时内分七波创建了它们:第一波创建了五个,第二波在大约二十分钟后创建了另外两个,然后又在二十分钟后创建了另外两个,接着在接下来的两个小时里零散地创建单个和成对的会话。
模型选择是按切片逐一决定的:15 个中有 10 个运行在 GPT-5.6 Sol 上,5 个运行在 Claude Opus 4.8 上。全部 15 个都在 GitHub Copilot 的 autopilot 模式下启动,该模式允许会话朝着某个目标持续推进,而不必在每一步都停下来等待批准。
启动提示词的中位数长度约为 1,100 个字符,足以承载归属边界和各项约束,但又短到让子会话必须自行摸索出实现路径。我向父智能体发出提示,随后是父智能体——而非人类——为每个子会话撰写那些启动提示词。
除了这 15 个子会话之外,同一个父会话还动用了五个子智能体:三个 explore 智能体与第一波并行触发,一个 code-review,以及一个 rubber-duck。这些子智能体负责探究问题,并把父智能体在决定下一步该做什么之前、其上下文中所需的答案反馈回去。子智能体让父智能体无需耗费自己的上下文窗口去推导,就能获得经过深入思考的答案。
相比之下,子会话则着手进行实际的移植工作——这些工作会产生 diff,并且需要与其他并行移植者隔离。子会话的工作涉及仓库中 140 个不同的文件,其中 120 个文件恰好只被一个会话触及。那 20 个存在争用的文件都是枢纽文件,比如 session.ts 本身。但每个会话都在自己的工作树中工作,因此能够不受兄弟会话干扰地推进。当然,父会话为此在协调上付出了代价。它花费了大量精力与子会话沟通,充当信息中介,轮询它们的状态 60 次,并发送了 89 条协调消息。当子会话各自宣布完成时,父会话将它们的提交 cherry-pick 到自己的分支中并解决冲突。这些合并也并非特别干净,父智能体花了相当多的时间来调和这些编辑。
我们可以在时间线上看到这一点,其中展示了父会话及其大多数子会话。
注意那些大的空档。我在进行这次移植工作的同时还在旅行,不得不在多个时间点合上笔记本电脑。(我随后改变了工作流程,加入了可以远程接入的云端虚拟机。)
这些并行的子会话对那台笔记本电脑造成了显著影响。有一段时间,并发移植进行得非常顺利。随后,同一台机器上的全部 15 个并发智能体各自尝试构建和测试,我那可怜的笔记本电脑直接卡死了。我向父会话发出提示,让它转告其子会话,必须全部停止构建和测试。父会话把这一约束向外传达,它们所幸终止了构建,随后以最低的 CPU 活动继续工作。我随后更新了自己的常驻指令,要求子智能体和子会话在移植期间避免大型构建和测试运行,改为只由父智能体来执行这些操作。
后来我又更进一步,把一个原本普通的聊天会话变成了八个独立移植会话的构建调度器。提示词简单得令人尴尬:向每个打开的会话发送一条策略,要求尽可能避免 CPU 密集型的构建和测试,规定在必须构建时需向本会话请求许可,并充当闸门,一次只向一个会话发放构建权限。
基本上,我把这个聊天会话变成了一个智能体互斥锁。这个闸门维护着明确的持有者和队列,并通过各会话原本就用于协调代码的同一套跨会话消息机制,一次只授予一个租约。那些请求租约但被拒绝的会话,往往会在此期间做其他工作来等待,比如从自己的待办清单里挑事情做。

这个 session.ts 端口还牵涉到我在整个运行时移植过程中所目睹的最酷、最令人唏嘘、也无疑最出人意料的一次交互。正如我提到的,我们主要采用自底向上的方式来做移植,这也是为什么 session.ts——它实际上位于所有其他组件之上——是最后移植的组件之一。始终位于 session.ts 之上的,只有运行时的所有入口点,也就是从 SDK 暴露出来、并出现在前面讨论过的分发表中的那些公共函数。这样的函数有数百个。虽然我知道其中许多会立即调用到 session.ts,但我想抢先推进移植工作,于是在启动 session.ts 会话之后,我又启动了一个会话来移植所有入口点。我告诉它在 session.ts 边界处停下。我估摸着可能会有一些白费的工作,在 rebase 时也需要投入一些精力或消耗一些 token,但这样会加速整体移植。然后我就去睡觉了。再然后……它们找到了彼此。
我给入口点会话写的启动提示词确实告诉了它,会话移植和六个组件移植正在并发运行,因为我想让它知道自己的边界,以及应当避免移植哪些内容,从而尽可能减少冲突。显然,我的提示词起了反效果。刚过四分钟,在盘点完入口路径、并大概对它们的重叠程度形成了判断之后,它调用了一个应用内置的 orchestrate 技能,该技能的作用是协调跨会话的工作。从那里开始:
- 入口点会话枚举了每一个活跃会话,并向那些它认为存在重叠的会话发送了消息。
session.ts会话用一个 2,001 字符的清单作了回答,标题为“关于stephentoub-port-session-to-rust的具体重叠”- entrypoints 会话读取了
session.ts会话的工作树,以确认它刚刚被告知的内容(我猜是“信任但要验证”)。 - entrypoints 会话询问
session.ts会话是否准备好协调其 760 个文件的 diff。 session.ts会话基本上让它别来烦自己:“还没准备好提交/集成。”- entrypoints 会话接着又问了同样的问题三次,每次它都从
session.ts会话那里得到同样的回答。 - 就在这时,entrypoints 会话决定不再在意
session.ts会话怎么想,直接伸手进入它的工作树,抓取了另一个会话的所有更改,并合并到了自己的里面。 - 然后两个会话就各自愉快地继续下去了。
我从这次交互中有几点收获:
- 明确表达意图很重要。启动提示词列出了其他正在运行的会话,这样这个会话就会知道哪些不要碰。但我没有把“不要碰”这部分说清楚,所以我没有阻止智能体去做某件事,反而鼓励了它去做。我需要在意图和指导上表达得更加明确。
- 你提供的任何东西,都可能被智能体判定为适用。这个
orchestrate技能随 GitHub Copilot 应用一起提供,它自我描述为用于并行运行独立的工作流。我的提示词中完全没有提到它。模型自己发现了自身所处的情况,将其与该描述进行匹配,然后加载了它。你所暴露的能力集合,就是你可能得到的行为集合,包括在你从未设想过的情境中。 - 对等方需要一个裁决者。两个会话都无法强制对方。当
session.ts会话四次表示尚未准备好集成时,这一拒绝没有任何分量,因此愿意单方面行动的会话默认获胜。针对相邻代码的并行会话需要一个指定的协调者,或者需要一个人,而这两个会话两者皆无。 - “自主运行”需要对超出你自己分支范围的决策设置例外。我真正的意思是“不要因为设计细节来叫醒我”。它(并非不合理地)理解为吞并一个对等方也在范围内。再说一次,我本应在我的指导中说得更明确。
- 这里的根本原因是我自己。我同时以自顶向下和自底向上的方式划分这项工作,而这两个方向在中间相遇于代码库中连接最密集的那个文件。我太贪心,想要向前推进。上述一切都由此而来。
谢天谢地,这整个交互是一个有趣的异类。在整个运行时移植工作中,大多数叶子组件的移植都是简单的单会话任务。较大的子系统往往涉及多个子会话和子智能体。不过,这些部分在移植过程中如何参与,差异很大。
模型编排层的移植——即真正与提供商通信的那一层——很好地展示了其中一种模式。它的主会话运行了 42 个挂钟小时,启动了 126 个子智能体。最繁忙时,有 22 个同时在工作。然而,大多数时间里,只有主智能体在运行,然后时不时地会在某个时间窗口内生成大量子智能体。
那张图中有三点很突出。第一,几乎所有代码生成都在前 12 小时内完成;之后一天的工作全是验证。第二,底行的颜色从左到右变化,从以蓝色和绿色为主(阅读、构建)转变为以蓝色和橙色为主(阅读、审查);这在逻辑上说得通,但能在实践中看到它还是很不错的。第三,那个例子——以及更普遍地这种模式——各阶段之间有非常清晰的分离。扩展运行时的移植则是一个反例。
它花了 88 小时而不是 42 小时,并且结构非常不同:
- 写作和审查显著重叠。在前一个例子中,工作是高度瀑布式的(先代码生成,再审查),而在这里,审查在写作停止之前很久就开始了,两者在大部分工作时间内并行推进并重叠。
- 最下面一行的颜色分布则杂乱无章。 在模型编排中,颜色从绿色变为橙色,对应着从编写到检查的过渡;而这一行从头到尾都是阅读、构建和审查的混合。在 88 小时的会话中,阅读调用中间的一半分布在 49 小时的时间跨度内,编写分布在 33 小时内,审查分布在 27 小时内。每个类别都散布在整个运行过程的大部分时间里。
- 空闲被推迟到了最后。 整个集群在前 56 小时内几乎不间断地工作。
- 比例仍然吻合。 这里编写 Rust 占工具调用的 2%,那边占 1%;阅读占 44%,那边占 57%;审查占 23%,那边占 27%。两个会话对待完成的工作内容看法一致,只是发生的时间不同。
末尾那些空白的切片也直观地反映了一个在这个智能体编程时代越来越普遍的问题:等待审批。团队中的某个人和/或某个智能体审查代码并留下反馈,然后有一段短暂的活动期,智能体处理反馈并让 CI 重新变绿,接着又是更多的等待,如此反复,直到最终我们得到那个令人多巴胺飙升的批准印章。
这两个例子各自代表了一种主导模式。大约四分之一的会话看起来更像模型编排那个会话,而四分之三看起来像扩展运行时那个。干净利落的阶段推进反而是例外;常见的情况是智能体自始至终都在规划、编写和审查。
大规模代码审查
前面图表中体现出的对审查的高度关注,很大程度上源于我明确的提示。我创建了一个简单的“提示词即技能”,我称之为 rust-rebase-review(此外还有我们已经合并到仓库中的通用 Rust 编码技能)。由于变更涌入速度很快,其中许多还相互冲突,我需要频繁地 rebase。通过自定义指令,我鼓励 harness 在流程中的适当时机调用这个技能,同时我也会不时手动调用它。这个提示词随着时间推移有所演变,但大致是以下这个版本的变体:
Squash into a single commit, then rebase on the latest in origin/main, resolving all conflicts, and force push. As part of rebasing, pay extra special attention to anything that has changed, been added, been removed, and ensure that logic is all ported over to the corresponding Rust code correctly. Always do the rebasing yourself / in the main agent; do not spawn a subagent for it.
Then enter a review/fix loop where you launch a subagent per opus 5, gpt-5.6-sol, and grok 4.6.
- That subagent should do a line-by-line comparison of the old TypeScript and the new Rust, confirming behavioral equality.
- Look for anything introducing any kind of incompatibility; our goal is to move this code into Rust with as close as is possible to 100% the same semantics. If you hit anything questionable, ask me about it.
- We want to ensure we're writing as efficient and idiomatic Rust code as we can; look for opportunities to simplify, to use routines like from the memchr crate to optimize searches instead of open-coded loops, avoid unnecessary allocation, use traits for reuse and loose coupling, etc.
- Ensure that all defunct TypeScript code (e.g. code that has been fully ported, tests that are now no longer necessary because they're duplicative, unnecessary napi shims, etc.) has been deleted.
- Ensure that we've ported as much code as possible, e.g. if there's any TypeScript remaining in touched files and that TypeScript is more than just a shim, that's a red flag. If new TypeScript that's not just a super thin shim is being added, that's a red flag. Look for any callers of TypeScript shims to see whether those callers can instead be ported to Rust, pushing the boundary as far as reasonably possible. Our goal is to soon get to 100% Rust in the runtime layer.
- Validate that no E2E tests have been deleted or changed. Such changes are an indication of a porting bug.
If a review surfaces issues, validate them, and then if there are any to fix, fix them, and iterate to do another full review. Continue iterating with reviewing/fixing until all reviews come back clean. After every set of changes in response to review feedback, commit and push so that CI validation runs concurrently with subsequent reviews.
Don't bother running full test suites; that'll be handled in CI. Try to minimize CPU-consuming efforts to the bare minimum, as we'll likely have many operations happening concurrently. 在频繁 rebase 的情况下,涌入的变更很容易被意外丢失。但我们发现原地原子替换带来了一个意想不到的好处:在添加对应 Rust 代码的同时删除 TypeScript 代码,我们实际上隐式地制造了与 rebase 引入的、针对该 TypeScript 的变更之间的冲突:一个分支修改它,另一个分支删除它。这保证了我们会注意到对已移植代码的变更,而不需要为每一行涌入的代码去判断它是否可能触及了此前已移植的代码。
我自己的审查当然只是所执行的智能体审查中的一部分。除了每次提交都会运行的 CCR 之外,团队还有多个专门的代码审查机器人,每个都有自己的方法和提示词,在每次提交时运行并提供详细反馈。所有这些最终都会变成 pull request 上的评论,然后需要逐一处理。幸运的是,处理所有这些智能体反馈也可以(大部分)由智能体来完成。
在本次评审中,我聚焦于架构、设计、约定和方法。智能体负责做详尽的新旧对比;测试和静态分析检查可机械验证的属性;人类评审者则专注于架构、API 契约、风险,以及其他各层所暴露出的任何可疑之处。
我选定了目标架构,决定了哪些行为至关重要,划分了工作,解决了模糊的权衡取舍,评判了证据,人工审查了高风险区域,审查了智能体对反馈的回应,并做出了最终的合并决策。智能体改变了一名工程师所能监督的代码量。它们并没有消除对这样一名工程师的需求:他理解整个系统,并能为方向、护栏和发布背书。
自动化内循环
GitHub Copilot 应用是这项工作的核心。它管理着大量并发的活跃会话,使得在它们之间切换变得很容易,并且每个会话都携带着所有相关的配对上下文(关联的终端窗口、浏览器窗口、画布等等)。这里最重要的功能是智能体合并:

Agent merge 是内置于该应用中的一个循环(CLI 也通过 /pr auto 提供了这一功能)。它会按定时器触发,或响应外部刺激(比如来自 GitHub 的 CI 完成通知或审查评论),届时应用会查看发生了什么变化。如果留下了审查评论,它会调用智能体来决定是拒绝该评论,还是接受并处理它(同时回复,并注明是自动化在响应)。如果测试失败,它会下载日志、调查失败原因并修复 bug。如果出现冲突,它会调用智能体进行合并或 rebase。实际上,它把我们作为人类开发者都会做的这个循环自动化了:推动我们的 pull request 走向“全绿”、获得批准,并最终合并。
Agent merge 处理了每一个移植 pull request。不过在大多数情况下,我们并没有走到真正的“合并”那一步。智能体会修复所有 CI 失败、处理并回复所有评论,并确保所有冲突都得到解决。在合并之前,我会抽查智能体实际做了什么,尤其是它如何处理反馈。我是否不同意它对审查者的某些回复?所应用修复的总体方向是否可取且合理?对于这些移植,我通常会保留最后那一项不勾选。
最后一个复选框不止一次发挥了作用。在一次合并循环中,移植过程删除了一个暴露给 SDK 的函数。我们仓库的 schema 兼容性 CI 环节恰好履行了职责并失败了。智能体的回应是给仓库打上 schema-break-ok 自动化标签,这是让检查通过的逃生通道。在合并前审查这个 pull request 时,我问了那个显而易见的问题:“schema 破坏是什么?你给这个 pull request 加了 schema-break-ok 标签;为什么这样没问题?”它并不没问题。这个方法存在于 main 上;移植过程只是把它弄丢了。我称这是不可接受的回归,并告诉智能体用 Rust 把它完整恢复回来。二十一秒后,豁免被移除,该方法以原生 Rust 实现被恢复。
我们既在局部也在全局层面回应失败,修复个别案例,但随后也修复系统,使它们更不容易再次发生。我们不断演进提供给编码和审查智能体的指令,以进一步降低这些问题在未来移植 pull request 中再次发生的可能性。我们把会话日志转化为评估。在某些情况下,我们实际上利用学到的经验来改进运行时本身,通过调整提示词、工具描述或 autopilot 的运作方式。
一次迁移,两件事
语言重写几乎从来不只是语言重写。运行时依赖的每一个库也都必须被替换,而且与我们在 Rust 代码中编写并拥有的部分不同,这些替换并非由我们来做忠实还原。有些是同一理念换了个名字。有些需要好几个 crate 才能覆盖一个 npm 包曾经完成的工作。还有少数根本没有可接受的现成答案,只能手工编写(由智能体完成)。
CLI 和运行时目前位于同一个仓库中,共享一个 package.json。在移植过程中,我们移除了约 60 个 npm 依赖,因为它们只被已移植到 Rust 的运行时代码所使用。这只是一个移除数量的下限,因为有些包虽然被替换用于运行时,但 CLI 仍然需要它们。例如,zod 是一个 TypeScript 模式声明与验证库,CLI 和运行时都在使用它。随着 Rust 移植的推进,运行时现在使用 serde、schemars 和 jsonschema 的组合来满足相同的用途,但 zod 仍然因为 CLI 的缘故留在清单中。
有很多例子是一个 npm 包变成了一个做同样工作的 crate。js-tiktoken 变成了 tiktoken-rs,具有相同的 o200k_base 编码。ignore 变成了同名 crate,具有相同的 gitignore 语义。minimatch 变成了 globset,fast-myers-diff 变成了 similar,dompurify 变成了 ammonia,而 github/keytar 变成了 keyring。
在其他情况下,我们无法将包与 crate 一一对应地替换。相反,一个包变成了多个 crate,或者多个包合并为更少的 crate。依赖工作的大部分都花在了这里。八个 opentelemetry/* 包变成了四个 crate,外加一个手写的跟踪器状态机和文件导出器。三个 Web 内容包——mozilla/readability、linkedom 和 turndown——变成了两个 crate:readability 和 htmd。sharp、image-size、file-type 变成了 image 和 imagesize。以此类推。还有五个案例,我们完全用自定义实现替换了 npm 包。
有多少 unsafe?
人们关于智能体编写的 Rust 代码提出的另一个问题是,其中有多少悄悄放弃了安全保证。Rust 的安全性是一种可以用关键字关闭的属性,因此当智能体遇到无法满足的借用检查时,就有一个显而易见的逃生通道。在整个 runtime crate 中,我们现在有 158 个 unsafe 块,仅分布在 36 个文件中(与之相伴的还有 26 个 unsafe fn 声明、26 个 unsafe extern 块,以及 9 个 unsafe impl trait 实现)。重要的是,其中每一个都与外部组件的互操作有关。
| unsafe 块为何存在 | 块 | 占比 |
|---|---|---|
| C ABI 边界 | 51 | 32.3% |
| Windows API | 49 | 31.0% |
| POSIX / libc | 46 | 29.1% |
| SQLite C API | 7 | 4.4% |
| 动态库加载 | 4 | 2.5% |
| 进程环境 | 1 | 0.6% |
C ABI 代码块是 SDK 宿主进入的前门,因此它们会从调用方接收原始指针和长度,而 Rust 编译器对这些调用方没有任何控制权。Windows 和 POSIX 代码块是系统调用:注册表读取、凭据握手、进程树、sysconf。SQLite 是一个 C 库。动态库加载是 dlopen,从构造上就不可能安全,哪怕仅仅因为解析出的符号可能并非你期望的函数。进程环境 unsafe 代码块之所以存在,是因为 Rust 2024 将多线程进程中对进程全局环境状态的修改视为不安全操作。这些 unsafe 代码块中的每一个都标记了 Rust 所提供保证真正终止的地方:另一侧是 C 函数、系统调用、来自外部运行时的指针,或进程全局的宿主状态。
有用的特性在于,unsafe 让我们所拥有的 Rust 代码中所有这些位置都可审计。TypeScript 运行时中的等价代码恰好跨越了相同的边界,穿过 Node 的 C++ 内部实现和原生 npm 包,而我们的源码中没有任何东西标记出受检查的世界在哪里终止。当然,这并不是交付系统中每一个安全边界的完整清单:依赖项、构建工具、C 库、安全封装以及规格错误的 FFI 契约仍然可能包含或暴露不安全代码。
unsafe 在哪些地方被使用的细节很有意思,但我更感兴趣的是它没有在哪些地方被使用。它没有被用在模型客户端中、MCP 层中、智能体层中或提示词层中。而且移植工作中已知的回归问题,没有一个涉及 unsafe 块。
回归问题
移植代码很容易。让它正确却很难。而对于像 Copilot 智能体运行时这样庞大且复杂的代码库,回归问题是意料之中的。
到 2026 年 9 月 14 日,我们已经追踪了数十个已知的移植回归问题,全部已修复。大多数是正确性 bug,还有一小部分是性能回归。这些是在从零编写的约 832,000 行生产 Rust 代码的基础上出现的。
当然,并非所有这些回归问题都发布了。有些在仓库中开发时就被发现了。另一些出现在预发布版本中,但在进入稳定版本之前就被修复了。还有一些进入了稳定版本,通常是因为它们足够不起眼,以至于在一轮或多轮预发布使用中都未被发现。
当然,这并非为零,而且我百分之百确信,实际数量比我们所知道的还要多。这些是我们注意到的或被报告出来的,但如此规模的迁移绝对还带出了另外一些,它们足够安静,以至于至今还没有人踩到。就像一般的 bug 一样,我预计随着技术栈更偏远的角落被在真实环境中猛烈地反复使用,我们会继续发现零星的边角案例回归问题。
绝对数量其实也没那么重要。更重要的是所有这些问题的“原因”,这样我们才能从中吸取教训,避免将来重蹈覆辙。几乎所有的正确性回归都落在三大类中:新代码实现了不同的行为契约;状态、所有权或生命周期行为发生了变化;或者迁移的某一部分被遗漏、只部分应用,或在 rebase 中丢失了。还有一小部分来自宿主或互操作边界处的要求,甚至来自那些自信地验证了错误行为的测试。这些类别在若干反复出现的模式中变得更加具体:
语义模糊。若干回归问题源于源语言未明确表达的行为。TypeScript 只有一种 number 类型;Rust 则要求在多种类型中做出选择,包括某个值是否可以为小数。而智能体猜错了。概念上属于整数的字段变成了 f64,于是 Rust 将诸如 42.0 这样的值序列化,而不是 42:像 Go 和 C# 这类强类型 SDK 无法将仓库 ID 反序列化为 int64,并拒绝了 hook 时间戳和任务时长。反方向上,某个智能体将 timeToFirstTokenMs 声明为 i64,但流式路径却输出了诸如 5446.712845 这样的值,导致已写入的会话无法读取、无法恢复。还有一个更隐蔽的案例与类型无关:event.error || "Unknown error" 变成了 .unwrap_or("Unknown error")。JavaScript 的 || 会替换空字符串;Rust 的 unwrap_or 则保留它,因此一个空的子智能体错误仍然保持为空。哎。
环境行为。另一个反复出现的来源是 JavaScript 或 Node 在不可见的情况下提供的行为。配额代码使用了 toLocaleDateString,它会继承宿主时区;而 Rust 需要显式传入该时区。但尽管 Intl.DateTimeFormat().resolvedOptions().timeZone 的类型标注为 string,它却可能返回 undefined,而 napi 无法将其转换为 Rust 的 String,从而导致模型列表加载失败。在另一处,将环境变量的读取从某个 await 之前移到之后,意味着等待期间宿主的变化可能会改变结果。一个原生插件加载器调用 process.report.getReport() 仅仅是为了识别平台,但在 Windows 上这会遵循 _NT_SYMBOL_PATH,并可能在渲染 CLI 之前花费数分钟下载 PDB。其他环境输入还包括工作目录、仓库身份、PATH 以及会话认证。将所有权移入 Rust 需要决定何时捕获每一项、如何携带它,以及何时刷新它。
只移植了一对中的一半。若干回归问题源于成对操作失去同步。一次轮次上限检查更新了原生注册表的终止状态,但没有取消进程内的模型循环,导致又有一个请求逃逸出去。在另一处,任务完成被持久化并发出,但没有投射到活动会话状态中,因此 Autopilot 在完成任务后仍继续运行。
阻塞主线程。 CLI 仍然从 Node 的单线程事件循环驱动 Rust 运行时,因此跨 napi 边界的同步工作会冻结 UI。/chronicle reindex 以这种方式解析了数百个会话文件,导致渲染和输入被阻塞近一分钟。将导出改为 async 并把工作移到阻塞线程池线程后解决了问题。一次审计又发现了几个可能阻塞的入口点,由此形成了一条长期规则:真正干活的 napi 导出必须是异步的,并在必要时使用 spawn_blocking。
窗口闪烁。 在 Windows 上,生成子进程时如果不带 CREATE_NO_WINDOW,会短暂弹出一个控制台窗口。Node.js 运行时此前通过猴子补丁修改了进程生成逻辑以添加该标志,从而对移植智能体隐藏了这一要求;Rust 替代实现遗漏了它。一次审计又修复了两处额外的进程生成点,不过这些是原本就存在的遗漏,而非移植引入的回归,我们随后将该规则加入了 copilot 指令。
生命周期管理。 最大的一类问题涉及生命周期、释放、所有权、顺序或竞态。将状态移入 Rust 后,TypeScript 往往持有一个指向存储于原生表中实例的不透明句柄。与对象引用不同,该句柄的生命周期可能长于实例本身。一个 hook 在请求中途被释放,导致一个 tool_use 块成为孤儿,卡住了对话,因为模型 API 要求有匹配的结果。一个 shell 在“已宣告”和“已启动”之间被取消,泄漏了一个孤儿进程,使会话保持活跃。一个沙箱开关更新了一个代数计数器,却没有更新其原生对应物,导致 shell 卡在“重新配置中”。
被忽视的功能。另一组问题涉及移植过程中直接遗漏的功能。有一处移植省略了 SDK 回调,并删除了对应的端到端测试,这促使团队制定了一条新规则:智能体未经明确同意不得修改 E2E 测试。一次会话中止保留了原生那一半,却丢失了用于中断等待工具返回的轮次所需的进程内取消能力。SDK 替换内置工具搜索的能力取决于启用开关、面向模型的描述和 schema,以及将执行路由到 SDK 回调。这次移植把这三样全丢了(一致性万岁?),悄无声息地把某个使用方的自然语言搜索替换成了正则表达式搜索。
不同库,不同主张。还有几处回归源于用更严格的 Rust 等价实现替换 JavaScript 库或 API。当时,Rust MCP SDK(rmcp)会对格式错误的 JSON-RPC 输入做出响应,而 TypeScript SDK 则不会。面对一个用更多格式错误输出回应错误的服务器,这种礼貌变成了无限循环,导致启动卡死。不同生态系统中相关的库很少表现完全一致。
擦肩而过。少数回归源于分支漂移或 rebase。在一个每周有数百个 pull request 的仓库里,一个开了好几天的会话会不断累积变更和冲突。这次移植需要数千次 rebase;即便成功率非常高,也仍会有失败。
还有那些慢吞吞的家伙。最后一组在功能上是正确的,但速度更慢。有些回归丢失了已有的效率优化,比如记忆化、完全异步等待,或有界日志流式传输。另一些则在 Rust-TypeScript 边界引入了迁移特有的开销:冗余序列化、加锁、轮询、无界原生并发,以及原生到宿主侧的跨界调用。这些并不是后续可以优化的机会点;每一项都是移植过程中引入的性能退化。有一次只读扫描把 260 MB 的事件日志做了深拷贝,而不是借用它。在持续的事件流量下,另一个实现只完成了所请求通道刷新的一小部分,同时为每个事件保留一个异步句柄,直到 V8 耗尽堆内存。
几十个回归听起来很多。但在一项生成了超过 800,000 行代码的移植中,坦白说,我惊讶且欣慰的是我们没有遇到多一个数量级的回归。我们还可以通过观察公开的 github/copilot-cli 和 github/copilot-sdk 仓库中 issue 的趋势,来了解这些问题是否被广泛“感知”到。在这里,我们将 1 月到 8 月期间开启的 issue 分类为质量相关,条件是它们带有 bug 标签,或者标题中使用了常见的故障术语,如“bug”“regression”“crash”“hang”“timeout”“broken”或“incorrect”。与移植工作进行期间及之后相比,这些水平与移植工作之前基本没有变化:
| 仓库 | 1 月–4 月,重写之前 | 5 月–8 月,重写期间 / 之后 |
|---|---|---|
github/copilot-cli | 22.9%(454 / 1,982) | 23.7%(354 / 1,496) |
github/copilot-sdk | 36.2%(190 / 525) | 32.3%(135 / 418) |
这并不是一个可用性指标,也不是对逃逸缺陷的精确计数……正如回归问题本身一样,一个问题所代表的含义、其范围等等都可能存在大量差异。但它确实提供了一个有用的检验:尽管产品变更的规模异常庞大,面向产品的问题反馈渠道在迁移期间并未显示出有意义的质量问题激增。
只要能编译,就能编译
前面我提到了一个流行的梗:Rust 因其严格的编译器而非常适合 AI 生成的代码。还有一个相关的流行 Rust 梗:如果代码能编译,它就是正确的。并非如此。我们已知的回归问题清单对此给出了很好的回应:语料库中的每一个回归问题都被合并到了 main,这意味着它们都成功编译了。编译器接受了这些有 bug 的版本,因为对编译器而言,它们每一个都是合法的 Rust。
编译器可以证明某个 f64 被一致地使用。但它无法知道仓库 ID 必须序列化为整数,也无法知道一个以尾部 .0 序列化的时间戳会被线路另一端的每一个强类型 SDK 拒绝。编译器可以防止它所能看到的代码中出现未同步的数据竞争,但它无法阻止一个完全同步的状态机编码出错误的状态。一个队列可以被锁保护,却仍然让两个发送方各自认定对方会将其排空。事件可以在线程之间安全传递,却仍然以错误的顺序到达。一个同步的 napi 函数可以是内存安全的,却仍然阻塞 Node 的主线程长达一分钟。编译从定义上讲也几乎无法检测到那些不存在的东西。当一次 rebase 悄悄移除了一个防护及其测试,或者当一个进程启动器忘记了抑制控制台窗口弹出的 Windows 标志时,编译器不会提出异议。它也对每次读取都克隆一份 250 MB 的事件日志没有任何意见。编译器检查的是你写的程序在内部是否自洽。它无法检查你是否写完了整个程序、是否保留了旧的契约、是否按正确的顺序调用、是否满足了宿主未明说的要求,或者是否以可接受的成本完成了工作。
这绝不是反对 Rust 编译器的论据。与任何静态类型语言一样,编译器消除了一大类机械性错误,并为智能体提供了一个极其有用的内循环。但“只要能编译,就是正确的”只有当作笑话时才有用。
性能,性能,还是性能
那么,这一切换来了什么?这次移植刻意保持了行为不变。它并非要重新设计算法或修复 bug;事实上,我反复把智能体从机会主义的优化方向上推开,因为同时改变语言和行为会让你很难判断究竟是哪一个把你搞坏了。不过,这次重写的一个关键目标确实是性能和可扩展性(此外还有可靠性等因素)。
当人们问我为什么要用 Rust 重写运行时,我的回答往往是这样的:“我并不是要转向 Rust,我是要摆脱 Node.js 和 V8。”这一转变确实显著提升了我们的性能表现。
在移植前后,我通过 C# SDK 对运行时的多个场景进行了基准测试(TypeScript、Python、Go、C#、Java 和 Rust SDK 都通过相同的传输架构接入同一个引擎)。基线是移植前的 SDK 和 CLI 构建版本,其中 TypeScript 运行时由 Node 托管,并通过 stdio 访问。
8 月 21 日的结果使用的是 Rust 运行时,既作为进程外服务器,也通过 FFI 在进程内加载。这是对交付系统的端到端比较,而非试图隔离语言变更所带来的影响;同一时期还落地了其他改动,因此这些数字需要打些折扣来看。
每个计时的轮次都发往运行在 localhost 上的一个确定性聊天补全服务器,产生固定的小规模响应。换句话说,这些数字刻意剔除了模型推理和网络延迟。它们衡量的是我们所改变的那部分:客户端启动、进程启动、会话创建、事件处理、持久化、拆除等。
| 场景 | 5 月 12 日 | 8 月 21 日进程外 | 8 月 21 日进程内 |
|---|---|---|---|
| 客户端、会话、单轮对话 | 5.25 s | 1.33 s(4.0x) | 292 ms(18.0x) |
| 恢复 32 轮会话 | 5.64 s | 1.52 s(3.7x) | 264 ms(21.4x) |
| 十个并发客户端生命周期 | 12.34 s | 4.18 s(3.0x) | 742 ms(16.6x) |
| 1,000 个单轮会话生命周期 | 132.52 s | 22.53 s(5.9x) | 20.93 秒(6.3 倍) |
“客户端、会话、单轮”这个数字是最容易直观感受到的。它指的是创建一个客户端、创建一个会话、执行单轮交互,然后拆除一切。进程外开销的很大一部分来自启动 Node、初始化 V8,以及在第一轮交互开始之前加载、解析并为由 TypeScript 代码生成的 JavaScript 应用生成字节码。Rust 运行时消除了这些 Node/V8 和 JavaScript 加载成本。
“1,000 次单轮会话生命周期”压力测试代表了我们可以构建的东西发生了阶跃式变化。它使用单个共享客户端,测量运行 100 个并发流水线的表现,每个流水线创建一个会话、完成一次完整的模型轮次、销毁会话,并连续执行十次。移植前的 TypeScript CLI 每秒完成 7.55 次这样的生命周期。Rust 进程外完成 57.45 次。Rust 进程内完成 120.0 次。
这是一个特定于工作负载的结果;Rust 运行时并非普遍“快 15.9 倍”。但这恰恰是服务器托管方关心的工作负载:许多独立会话共享一个运行时。而且挂钟时间并没有把工作隐藏到另一个核心上。在对同一 100×10 工作负载进行的单独资源采样中,移植前的进程树消耗了 312 秒的累计 CPU。Rust 配置消耗了约 110 秒。这是托管方可以用于更多会话的 CPU 容量。
内存数据也讲述了同样的故事,同时附上那个老生常谈的警告:内存指标极易被误用。观察十客户端批处理期间新增的常驻私有内存,移植前的进程树峰值比基线高出 1,383 MB。Rust 进程外方案峰值仅为 247 MB。而 Rust 进程内方案为 126 MB,少了一个数量级。虽然这些数字当然会因用途和机器而异,但它们触及了核心目标:一台机器上的服务可以承载多得多的客户端和会话,之后内存、进程数或 CPU 才会成为限制性资源。
最棒的是,这还只是基线移植。大部分实现仍然由忠实转写为 Rust 的 TypeScript 风格算法构成。我们尚未开展新的所有权模型、并发模型和进程内架构所支持的广泛重新设计工作。在优化工作开始之前,就能让一个进程内客户端完成创建、一个完整的单轮会话和销毁仅需约 55 毫秒,将共享客户端推至每秒 120 次单轮会话生命周期,并将实测的十客户端内存增量削减 91%,这是一个非常好的起点。
移植的代价
那么……这一切在金钱上花了多少?请鼓掌……
我在所有移植工作上的 token 消耗总计约 1363 亿 token,其中包括约 1306 亿缓存输入读取 token、约 42 亿缓存输入写入 token、约 9 亿全新输入 token,以及约 6 亿输出 token。所有这些 token 的账单金额约为 $120,000。
当然,这些 token 并不会自己花掉。使用开发者大量时间来引导这些智能体的成本也应计入其中。不过,我并没有把所有时间都投入到这个项目上。智能体式开发很大程度上是“赶紧提交然后等待”;提交一个提示词,让编码智能体自行其是,时不时查看一下以便可能引导它,但在它完成之前,去做其他事情。
这意味着开发者不再一次只处理一个编码任务;他们把等待时间重叠起来,从而能够同时处理许多事情。在移植窗口期内,这些 Rust 移植 PR 约占我向所有贡献仓库提交的 PR 的 20%。如果我们粗略估计 PR 的占比近似于我所花时间的占比,那大约相当于三周专门用于移植的时间。
换句话说,这次移植工作的大致账单约为 120,000 美元的归属 token 支出,外加三周的开发者时间。
话虽如此,端到端的 Rust 迁移并非完全由一位开发者完成;这是一项团队协作的成果。@stevesandersonms提供了napi-oop的设计与实现,即临时的进程外互操作性层,以及六个 SDK FFI 实现中的五个,@edburns提供了第六个。@roji实现了 SDK 的打包,以便正确发布和使用 Rust 二进制文件。@caarlos0一直在帮助将 Rust 代码拆分为许多小的子 crate,以缓解开始变得棘手的构建时间问题,而@criemen则帮助改进了资源缓存,以加速 CI 和本地构建。@devm33、@examon、@MRayermannMSFT、@dereklegenzoff以及其他人为无数 pull request 的审查和批准提供了帮助。而所有为 copilot-agent-runtime 仓库做出贡献的人,在世界于他们脚下发生巨变之时,始终给予了支持与配合。
经验教训
这次经历强化了几条超越本次重写本身的经验教训。以下是我们下次会带入的一些做法:
- 目标必须被清晰、完整地陈述出来。 我们早期的指令过于模糊。“把 XYZ 组件移植到 Rust”被理解成只处理热路径,或者只处理逻辑部分,而智能体一再把 I/O 和编排视为超出范围。一旦我们明确说明最终状态是一个由 100% Rust 代码库构建的原生二进制文件,即便我们想保留也没有任何留给 TypeScript 的执行环境,它们就变得擅长得多,能够自主地朝着这个目标推进。
- 端到端测试绝对、毫无疑问地至关重要。 除了一个例外,所有涉及功能缺失的回归,以及许多其他回归,都是由于缺乏足够的端到端测试造成的。对于任何此类性质的移植,你都必须拥有可用于验证移植正确性的测试,而且这些测试本身不能在移植过程中被重写,否则你就会失去你的判定基准。我们这次移植工作的初始计划就指出了这一点,并指出我们需要在开始移植之前大幅改善我们的 E2E 测试状况。我们做到了,但做得还不够。我们确信,如果在开始移植之前增加更多的 E2E 测试,真正专注于确保大多数有意义的行为都被它们覆盖,我们一路上的回归会比实际出现的更少。
- 保护判定基准免受智能体影响。 不能允许改动实现的智能体同时通过削弱测试、更新快照、提高兼容性基线或套用逃生舱标签来悄悄重新定义正确性,至少不能在缺乏监督的情况下这样做。尽可能让行为契约保持独立,把敏感的护栏置于单独的所有权或审批之下,并用具有不同失效模式的检查层层叠加,这样单一个错误就不足以让一次重大回归上线。
- 先翻译,再重新设计。 保持行为不变并沿用现有算法,可以让同时变动的变量数量保持在可控范围内。一旦旧实现和过渡期的脚手架被清除,就可以在一个稳定的基线上重新设计归属关系、并发和性能。我曾几次偏离这一原则——出于不安分,或难以对同事说不,或坚信这次情况不同——事后看来,每一次我都后悔。每一次偏离都比坚持原路线付出了更多的回归问题、更多的时间或更多的 token。
- 把反复出现的失败转化为未来的成功。 AI 智能体会偏离正轨:当它们偏离时,要从中学习。当某种失败模式出现两次时,它就应该被写入常驻指令、可复用技能、评测、受保护的基线,或者 harness 本身。
- 当智能体进入循环时,开发者的内循环更重要,而不是更不重要。 对于我们开发者来说,我们生活和呼吸都在内循环中——我们能多快做出改动、构建、测试、迭代。当其中的工具链环节耗时过长时,我们会感到沮丧。人们很容易认为,当智能体在做更底层的工作时,这一点就不再适用。但事实恰恰相反,它更加适用。AI 智能体完成这些任务中思考和编写代码的部分极其迅速,但它们仍然需要构建,仍然需要测试。而且,随着它们在思考和编写上花的时间更少、在快速验证的内循环中花的时间更多,它们用于构建和测试的时间占比实际上在增加。花一些时间在前期优化内循环,并针对多件事情同时发生的情况来优化它(例如,就像你同时在多个 worktree 中处理多个任务一样)。你日后会感谢自己做了这项投资。
接下来是什么?
它成功了。五月时还完全是 TypeScript 的执行运行时,到了八月已经完全是 Rust,而且整个过程持续交付给真实用户,而不是在最后以一次令人胆战心惊的整体切换落地。
我并没有简单地让一个 AI 智能体“把整个代码库从 TypeScript 移植到 Rust”。即便这是整个行业的发展方向,我们也绝对还没到那一步。相反,智能体让一整类项目变得可行。在一个运行中的系统里就地重写,产出数十万行生产级 Rust 代码,main,由一名工程师在团队支持下完成——这在智能体出现之前是不会被批准的提案。那需要一整个团队和一两年的时间,它还要与那个团队本可以交付的每一项功能竞争,而且它会输(说实话,也应该输)。智能体把成本降到了让这个项目变得可行的程度。
这次移植本身已经完成:运行时的生产实现 100% 是 Rust,临时的内部 TypeScript/N-API 接缝已经移除。不过,我们仍有许多想做的事:进一步改进构建系统和开发者内循环、清理移植后的结构、围绕 Rust 的所有权与并发模型重新设计,以及追求进一步的性能提升。
这次移植是一次翻译,而且是有意为之(也是以此为目标促成的),我们尽可能让行为保持接近 100% 一致,而不是趁机修复额外的 bug、重构组件,或进一步提升性能和可扩展性(超出重写本身自然带来的部分)。在微观层面,大部分代码已是地道的 Rust,但在宏观层面,仍有相当多最初用 TypeScript 编写的算法披着 Rust 的语法外衣。
如今这些算法底层的约束已经改变,重新审视这些决策,才是真正有意思的收益所在。
最让我兴奋的是这次移植所开启的可能性。SDK 现在可以直接加载进六种语言中任意一种的宿主进程,依赖链中不再有 Node.js 或 V8,也不需要再监督第二个进程——而这正是我们从采用该 SDK 的合作伙伴那里听到的最常见的单一摩擦点。
运行时实例的成本只有过去的零头,意味着宿主在耗尽机器资源之前可以运行多得多的并发会话。而且运行时现在可以去到 Node.js 永远无法跟随的地方,覆盖从云端到桌面、到设备、再到嵌入式系统的整个谱系。这些都不是终点。它是我们如今得以在其之上构建 GitHub Copilot 未来的基础,而在三个月里看着智能体重写运行它们自己的那个引擎之后,我很期待看到它能走多远。
The GitHub Copilot CLI, GitHub Copilot app, and GitHub Copilot SDK are all backed by the Copilot agent runtime, an agentic harness that can be embedded into applications and services. It was originally written in TypeScript on Node.js and the V8 JavaScript engine for what is now the GitHub Copilot cloud agent (CCA), and the runtime stayed on that stack as the runtime and its capabilities grew rapidly.
That has now changed. Using the GitHub Copilot app and the Copilot CLI, we completely rewrote the runtime into more than 800,000 lines of production Rust. AI agents wrote most of the code, spanning 128 pull requests that landed in main and shipped incrementally rather than waiting for a single cutover at the end. The few inevitable regressions were discovered and fixed quickly along the way, while the performance of the runtime improved by orders of magnitude. A project that would have taken a whole team of developers a year or two before agents was now completed primarily by a single developer, in only a few months, all while the rest of the team continued to greatly expand the runtime’s capabilities and reach.
Why we needed to port
The Copilot agent runtime isn’t just the engine behind the Copilot CLI. It backs a growing set of Microsoft, GitHub, and ecosystem solutions, for each of which AI support is, architecturally, a shell around the same runtime plus whatever customizations that solution needs. This includes not only the GitHub Copilot CLI and the GitHub Copilot app, but also the latest releases of VS Code, Visual Studio, CCA, Copilot Code Review (CCR), Copilot Cowork, Copilot Studio, and Excel and Outlook and PowerPoint and Word and… it goes on.
These are very different products, and none of them wants to or should need to implement everything that goes into a production agent harness. They want all of the intelligence, security, reliability, and performance, and they want it shared so that a fix in one place fixes it in all of them. Most of the products listed in the previous paragraph initially implemented their own agent loop, but have since replaced it with the GitHub Copilot SDK, which is the entry point to the Copilot agent runtime. Doing so enables them to focus on their core business value and leave the details to the runtime. That’s all the more important given the pace of the industry and the employed agent loop needing to stay always best-of-breed in the face of intense competition.
So, shared runtime, good. The problem was the nature of the thing being shared.
If we look at the CLI, it’s logically a terminal UI (TUI) on top of an agent loop. As it happened, the whole stack was implemented in TypeScript, using Node.js as the framework and V8 for the execution engine, with Ink and React for UI. That’s a respectable choice for a TUI application; TypeScript and Node.js are broadly accessible and enable very rapid application development. And for the needs of a console application, the performance implications in terms of startup, responsiveness, throughput, and memory consumption are also reasonable. They are, unfortunately, much less reasonable when you think about that implementation being used in other environments, with other constraints, with demands for things like fast startup and excellent server density due to low memory overhead.
The architecture of the CLI and its runtime also contributed to challenges here. The whole industry is running extremely fast, and in that context, really bright people make decisions for delivery speed and market reach. The Copilot CLI was initially written and shipped quickly, and in doing so, the TUI and the runtime were fairly intertwined rather than separated into discrete layers. Then when an SDK was needed for programmatic access to that runtime, without clear separation of the layers, a pragmatic decision was made to layer the SDK on top of the CLI, even though logically you’d expect the inverse architecture. Rather than only being accessible via commands provided by the user at the command line, the CLI was updated with a mode where it could be run headless, reading similar commands from stdin and writing responses to stdout. A JSON-RPC protocol could then be used to marshal function calls from an external process to and from the CLI. The SDK could then be embedded in arbitrary consuming programs, which would spawn a CLI process to host the agent loop out-of-process, with the SDK calling functions in the remote process via this JSON-RPC mechanism. Neat. Fast to get out the door. Flexible. But not great for the performance (startup, memory, throughput) and reliability of those consuming applications. Creating a new CopilotClient from the SDK meant spawning another process:
const client = new CopilotClient();
await client.start(); // spawns the CLI as a subprocess
const session = await client.createSession({
/* ... */
}); The process would need to launch and host Node and V8. It meant parsing the significant amount of JavaScript produced from the TypeScript code in the CLI, generating bytecode for it, and potentially optimizing hot code in later JIT tiers. It meant all the memory overhead associated with V8. It meant inheriting Node’s threading model, which by default pushes us towards a model of all CPU-bound work being serialized. And it meant forced out-of-process communication just to make function calls. It meant every SDK consumer, in every language, ships Node.js or a bundled binary containing V8. It meant the C#, Python, Go, Java, and Rust SDKs all paid for a whole second language runtime per client, on the order of 100 MB of working set minimum, for a runtime their application otherwise had no use for. It meant every event, every message, and every abstracted session file system read and write was pushed across a process boundary. It meant a crash in Node took the session with it. And it meant anyone deploying this had, at a minimum, two processes to supervise, monitor, and debug.
Instead, we wanted a runtime:
- that does not include the TUI, that’s its own library the TUI and other applications and services can be properly layered on top of cleanly.
- implemented in a language with minimal dependencies and minimal overhead.
- implemented in a way that it can be cleanly embedded in-process rather than being forced out-of-process.
- implemented in a language with top characteristics around performance and scalability and reliability.
- implemented in a language that’s great for interop, such that it can be used cleanly by all six Copilot SDK language versions (C#, TypeScript, Python, Rust, Go, Java) with that stack’s foreign function interface (FFI) mechanism.
- implemented with a tool chain that provides a more modern security posture, with less supply chain risk and greater support for correct-by-construction code.
For all those reasons, as well as softer reasons (such as team experience and industry direction), we chose Rust. This is in no way a claim that every large TypeScript program should become Rust. Our requirements emphasized embedding through a C ABI, low startup and steady-state overhead, and predictable resource use. Rust made those goals possible, at the expense of other complications, e.g. we had to represent lifetimes and shared state explicitly (the lifecycle regressions discussed later highlight the implications of that). The right target language legitimately varies from application to application.
There were then two key related tasks undertaken:
- Separating the TUI-specific code from the runtime, so that the former is layered strictly on top of the latter, and more specifically layered strictly on top of the SDK’s public surface area. Today, the CLI still calls directly into runtime internals in several places; moving it fully onto the SDK’s surface area is ongoing work.
- Porting that runtime layer to 100% Rust, resulting in a pure native binary exposing a C ABI for in-process consumption by all the language front-ends and a stdin/stdout-based or socket-based server for when out-of-process is still desired.
This post primarily covers the second: porting the runtime to Rust.
What it looked like before
The initial porting plan in early May 2026 estimated the runtime at roughly 130,000 lines of TypeScript. For scoping purposes, this initial measurement was reasonably accurate, but, as it turned out, also wildly misleading, in two key ways. Concurrent with porting:
- Pieces still wrapped up in the TUI layer were being pushed down to the runtime layer. Entire components and significant percentages of code initially ignored in the estimates were then later considered relevant to porting.
- Pull requests contributing significant amounts of new TypeScript were constantly raising the amount of TypeScript in the repo. Tens of agentically assisted developers merging hundreds of pull requests per week.
Everything factored in, I estimate approximately 430,000 lines of production TypeScript ended up passing through the port. Those same factors also made it hard to see progress along the way: until close to the end, production TypeScript volume appeared to be holding relatively steady, if not increasing slightly, as porting kept pace with incoming work.
This is confused further because there was also incoming Rust code, separate from the port, over the timeframe; early in the porting effort, incoming code was more likely to be dominated by TypeScript, whereas later in the effort, it was more likely to be dominated by Rust.
During the port, the runtime took in ~300,000 production lines of TypeScript and shed ~430,000, while ~1,200,000 production Rust lines entered and ~365,000 left. In other words, the apparent stability of the TypeScript line in the above graph was actually hiding significant amounts of TypeScript churn.
In-place porting strategy
That chart also highlights an important aspect of how the port was done: in place.
There are two main approaches to a rewrite of this scale:
- Big bang. The new Rust runtime is developed as a complete alternative and then swapped in all at once when it’s ready. Such a big-bang cutover has two variations. a. Stop the world. Everyone ceases other work on the
mainbranch while the rewrite happens, with the rewrite being done inmain. b. Parallel development. The rewrite happens in a feature branch while work continues in the main branch, with the rewrite constantly trying to keep up with and merging in changes from the main branch. - In place. This is done as a component-by-component port, where the runtime is incrementally rewritten one piece at a time. Such an in-place approach also has two variations. a. Atomic replacement. Each piece is flipped atomically from TypeScript to Rust, with interop between the remaining TypeScript and the new Rust providing continuity. Over time, less and less of the production runtime is TypeScript, and more and more is Rust, until one day, there’s no more TypeScript, only Rust. b. A/B. Rather than deleting components as they’re ported, both the TypeScript and the Rust components are maintained as hot-swappable options, with the TypeScript being deleted once confidence has plateaued.
We went with option 2a, for a variety of reasons:
- No one experiences work stoppage. The main branch continues to be active. Every developer not directly involved in the port gets to keep on keepin’ on, impacted only when a pull request they may have in flight for a prolonged period of time happens to touch code that gets ported concurrently, in which case they need to rebase and have their agents help port just their in-flight changes.
- The runtime’s main branch is always shippable. Each pull request replaces the existing TypeScript implementation with a thin shim that calls into Rust, and deletes the old code in one atomic change. The new code is immediately exercised, in-situ.
- The rewrite is incremental and reviewable. Each pull request ports a single component or slice, so the scope of change is smaller and the diff is easier to review, whether by a human or by agents or both.
- Most ports are reasonably small and self-contained, minimizing drift from concurrent pull requests. In some cases, where TypeScript components were too large, they could first be refactored into more easily ported components.
- All existing end-to-end tests, across the CLI and SDK, run against the new Rust code at every step, giving us confidence and lots of validation. If a pull request caused a required test to fail, it didn’t land.
We also shied away from the option 2b variant that involved maintaining multiple versions of the same component concurrently. With hundreds of pull requests being merged into the repo per week for the last several months, the codebase is constantly evolving, and quickly. Having two different versions of the same code in two different languages and using two different sets of dependency libraries adds a ton of complexity. Some of these components aren’t perfectly isolated, either; while some are logically standalone with simple APIs for accessing them by the rest of the system, others have significant tendrils, and making that graph hot swappable per component is a nightmare. The subsystems that would possibly benefit the most from a cautious parallel cutover are exactly the ones where parallel is hardest. For example, session orchestration isn’t a pure function you can call two different versions of with an if/else on some experimentation flag. It owns mutable state, drives callbacks in both directions, and threads through nearly every other subsystem, so “run both and compare” would mean maintaining two divergent copies of the component that holds the conversation’s state and services, and praying they stay in sync across hundreds of concurrent edits. The coupling that makes a component hard to port is the same coupling that makes it near impossible to shadow without risking introducing more regressions than it avoids. The benefits of being able to swap in this manner are primarily about gaining confidence, which we could do in other ways.
Validation also happened through incremental rollout. With a big-bang cutover approach, we would hold everything in a long-lived branch, port the whole runtime, and cut over once. That means consumers experience every ported line at once, including all regressions that slipped through in-repo testing. Rolling out portions of the change incrementally, two components here, one component there, enabled us to get a last-mile of validation in deployed builds with real consumer usage (most often first-party within Microsoft and GitHub) but while keeping the risk of regression to a minimum. Over the roughly fourteen-and-a-half-week porting window, main shipped 135 releases, inclusive of 100 pre-release versions and 35 stable versions, averaging around 1.3 releases per day. Roughly 1.3 port pull requests opened per day, as well, such that each release carried a small and knowable set of ported components (we generally tried but didn’t always succeed in shipping ports in a pre-release first). In a trailing seven-day npm sample, pre-release versions accounted for only 10.5% of downloads, indicating that initial exposure was relatively limited while we monitored feedback channels for signals of things breaking and quickly turned around fixes in the next pre-release. Reported issues were more easily correlated with known recent changes, and more easily root caused and quickly fixed. In this manner, doing the porting incrementally over a longer period of time was actually a feature rather than a hindrance (i.e. faster is not always better). By August 21, the runtime was 100% production Rust: 832,378 lines of production Rust and 468,689 lines of Rust unit tests, in addition to 174,675 lines of E2E TypeScript tests. The separate GitHub Copilot SDK repository added another ~130,000 lines of E2E test code across Node.js, Python, Go, C#, Rust, and Java.
Getting going
Before going all in, we gained confidence and proved things out. We started with two pull requests that established the Rust workspace, toolchain, lint rules, CI, build pipeline, and coding instructions, and then introduced the runtime crate plus code generation and interop patterns while porting a collection of pure-logic primitives chosen specifically because they had no I/O or shared state and already had strong tests. Only after those landed did the first primary port pull request take three side-effect-free helpers through the full process.
These functioned as shipping pilots, turning assumptions about repository layout, FFI, packaging, testing, and review into conventions the subsequent much larger ports would then reuse. Basically, we tested the machinery end to end. The plan continued by ordering the work from the leaves inward, with pure helpers, content exclusion, shell utilities, and session filesystem operations establishing the translation and testing pattern. Stateful subsystems followed, and tools, hooks, model clients, and MCP built on those pieces. Session orchestration (by far the most coupled and least naturally parallel part of the runtime) would come near the end.
| Period | Pull requests | Median changed lines |
|---|---|---|
| May 1–15 | 8 | 3,250 |
| May 16–31 | 2 | 9,421 |
| Jun 1–15 | 40 | 5,073 |
| Jun 16–30 | 31 | 8,253 |
| Jul 1–15 | 10 | 9,514 |
| Jul 16–31 | 14 | 28,159 |
| Aug 1–15 | 19 | 13,861 |
| Aug 16–30 | 4 | 99,445 |
The early ports, small leaf components, moved quickly. But larger subsystems didn’t move in one atomic step; MCP support, for example, progressed through seven dedicated pull requests, while tools progressed via a six-part series and then needed additional work to move orchestration and retire the remaining TypeScript. Hooks, auth, telemetry, plugins, settings, and persistence followed similar paths.
In effect, the useful unit of porting wasn’t always “a component.” It was often a wave through regions of related behavior: first move the pure logic, then move state ownership, then move orchestration, then remove fallbacks, and finally simplify the Rust after the temporary interop was gone.
Interop
There are two main layers in this port involving interop:
- Temporary internal interop. Any time a function was ported to Rust, that function needed to be invocable from whatever TypeScript was calling the initial TypeScript function. Similarly we needed to enable Rust functions to invoke TypeScript callbacks. This interop need is an implementation detail and extremely fluid. As the Rust internal surface area grows, so too do the number of TypeScript shims needed, as they’re 1:1 with whatever Rust methods need to be called from TypeScript. As those callers get ported to Rust, that existing layer of shim is deleted, and a new layer is put in place. Eventually we reach the public entrypoints into the runtime library, and the shims evaporate.
- The SDK surface. All of the SDK libraries need to be able to sit on top of the runtime and expose its functionality. In the pre-port world, this was done by having the runtime exposed via a bidirectional JSON-RPC layer, with the SDK sending function call requests as JSON-RPC method call payloads, the runtime parsing the request and invoking the relevant API, then sending back the result via the same transport for the SDK to parse and return. The inverse direction also exists; the runtime needs to be able to call back to the SDK client, for example for hook notifications and permission demands, which surface in the SDK clients as callbacks using whatever language feature is considered idiomatic (e.g. delegates in C#).
We achieved (1) via the napi Rust crate from the napi-rs project, which exists to build Node native addons in Rust. You annotate a function with #[napi], and a napi-rs macro generates the N-API registration glue that makes the function callable from JavaScript, plus a TypeScript declaration for it in a generated index.d.ts. A synchronous Rust function becomes an ordinary JavaScript function, an async fn becomes a JavaScript function returning a promise, and structs annotated #[napi(object)] become plain objects on the other side.
Traffic has to move in both directions, too. Many ported components temporarily depended on something that hadn’t been ported yet, so Rust needed to call back into TypeScript, for example a tool implementation in Rust asking the still-TypeScript model layer for inference, or raising a hook, or requesting a permission decision for a command it wanted to run. napi-rs handles this with “threadsafe functions,” which let Rust code running on a Tokio worker thread invoke a JavaScript callback back on Node’s main thread. Node installs the callback once, Rust holds it and calls it whenever it needs to go the other way. Every one of these is temporary by construction: the callback exists only because the thing on the other end is still TypeScript, and it gets deleted when that thing is ported.
The temporary seam peaked on August 3, with 2,019 internal N-API exports and 3,356 TypeScript call sites. At completion, the runtime was entirely Rust, therefore no internal interop: 0 temporary internal N-API exports and 0 TypeScript call sites remained. (I mentioned earlier that the CLI still has some internal access to the runtime we’re working to remove; those exports aren’t counted here.)
The second interop layer, the SDK surface, is the permanent one of the two. The Copilot SDK ships for six languages: TypeScript, Python, Go, C#, Java, and Rust. All of them speak the same bidirectional JSON-RPC contract, and originally all of them reached it the same way: spawn the Copilot CLI in headless mode as a subprocess, and talk to it over a pipe or a socket. That remained the default during the port. It also meant an SDK consumer in any language ships or locates a full Node implementation, pays a process hop on every event and every message, and supervises two processes instead of one.
Porting the runtime to Rust is what makes the other option viable. The shipped runtime.node is an ordinary platform shared library (the .node extension is the Node.js native-addon convention; underneath it is a .dll, .so, or .dylib), and it now presents two front doors onto the same engine. There’s the napi door, which a Node process loads as a native addon; that’s the CLI’s path (today… in the future, the intent is it’ll go through the SDK path fully). And there’s a C ABI door, which any language can load into its own process and call through FFI. The same in-process runtime is selected through each language’s native interop mechanism:
| SDK | Native bridge | In-process client selection |
|---|---|---|
| C# | P/Invoke | new CopilotClient(new CopilotClientOptions { Connection = RuntimeConnection.ForInProcess() }) |
| Go | purego | copilot.NewClient(&copilot.ClientOptions{Connection: copilot.InProcessConnection{}}) |
| Java | JNA | new CopilotClient(new CopilotClientOptions().setConnection(RuntimeConnection.forInProcess())) |
| Python | cffi | CopilotClient(connection=RuntimeConnection.for_inprocess()) |
| Rust | libloading | Client::start(ClientOptions::new().with_transport(Transport::InProcess)).await? |
| TypeScript | koffi | new CopilotClient({ connection: RuntimeConnection.forInProcess() }) |
The Rust rewrite and the choice between in-process and out-of-process hosting are separate dimensions. The completed Rust runtime supports both: it can run inside the SDK consumer’s process or behind the existing JSON-RPC server boundary. Those in-process entry points are currently opt-in while we gain confidence in sharing a process, and therefore a failure boundary, with the consuming application. Everything above the transport remains the same SDK API: sessions, events, tools, permissions, and callbacks do not care whether their JSON-RPC bytes crossed a pipe or a function call.
The interesting thing about the second door is its size. It has just 19 exported functions: four for server lifecycle, four for session registration and configuration, eight for connections, and three for the embedded host. Behind those functions, the shared contract currently contains 364 dispatch routes: 340 are callable by SDK consumers, while 24 run in the other direction as runtime-to-SDK callbacks. The napi door is much larger, needing functions for every one of those dispatch routes. The C ABI door is dispatch-based: API methods don’t get exports at all, but rather travel as JSON-RPC bytes written into a connection, and results, events, and server-to-client requests come back on host-supplied callbacks. Adding, changing, or removing an API method touches the engine’s dispatch table but never touches the ABI. An SDK binds those 19 entry points once and reaches the entire, still-growing API surface dynamically through them.
Which raises the obvious question: why is there still JSON-RPC in a call that no longer crosses a process boundary?
The answer is that it made in-process hosting a drop-in rather than a rewrite. Every SDK already had a working JSON-RPC client, with framing, request and response correlation, and handlers for the server-to-client direction. Mounting FFI as one more transport underneath that client moves the byte path from a pipe or socket to a function call and leaves everything above it untouched. Six SDKs got in-process hosting as an additive, opt-in transport, with the existing ones unchanged. Had we instead defined a typed C function per API method, every SDK would have needed a second binding layer, every new API method would have needed six more bindings, and the ABI would have become a binary compatibility surface we had to version.
We also still need JSON-RPC for runtimes that genuinely are remote, whether across a subprocess boundary or over TCP. Keeping the same protocol in-process means maintaining one bidirectional API and dispatch system rather than JSON-RPC for remote connections plus a second per-method FFI surface for local ones. That’s a real tradeoff rather than a no-brainer decision. We avoid the process hop, but we still pay JSON-RPC overhead on every call. For inference-dominated workloads, that serialization is generally small potatoes compared to the model round trip.
It is still measurable in high-throughput local workloads, but not enough today to justify duplicating hundreds of methods across six SDK bindings. And it’s a decision we can easily revise later should the performance need present itself. The payload encoding is a private detail of the two ends, and swapping JSON for something denser like MessagePack would not change a single declared export. Typed per-method exports can also be added later for hot paths, calling the same engine and the same handlers, without replacing the byte channel, which would remain the substrate for streaming, server-to-client requests, and the long tail of rarely called methods where a bespoke export buys nothing.
What the session data shows
Nearly every figure in this post comes from one of two sources. The first source is the GitHub history of the private github/copilot-agent-runtime repository: pull requests and their diffs, review comments, CI runs, etc. The second source is the agent session logs. The runtime (and thus the CLI, app, etc.) writes a structured event log for every session it runs: one JSON object per line, appended as the session happens. Those logs can include prompts, commands, command output, file paths, and potentially secrets surfaced by tools, so they must be handled as sensitive data. The log is local to the machine where the session ran; remote-session features can also upload it when enabled, subject to product settings and organizational policy.
Here is a summary of the data across all the constituent porting pull requests:
| Metric | Count |
|---|---|
| Events | 12,760,995 |
| User messages | 31,247 |
| Assistant messages | 1,385,214 |
| Hook start and end events | 6,438,562 |
| Tool starts | 1,857,409 |
| Compilation commands | 23,096 |
| Test commands | 19,485 |
| Rebase commands | 2,496 |
| Commit commands | 7,410 |
| Push commands | 5,554 |
| Completed compactions | 5,116 |
Those 31,247 user-role messages aren’t 31,247 prompts I personally typed; they include skill instructions, automated merge ticks, cross-session messages, and child-agent traffic, beyond the roughly 2,600 I typed or spoke, about one in 12. Similarly, the 1,385,214 assistant messages include subagents and tool-oriented messages, not just text shown to me in a conversational UI. The corpus contains 68 distinct event types and 67 distinct tool names; 1,130,921 tool calls, 61%, came from subagents rather than the main session thread.
The count doesn’t say why I was inserting myself ~2,600 times. For that, I had Copilot assign one primary intent to each of the human-authored messages in the session log corpus.
The first three buckets account for 63% of my interactions. Only ~40 were recognizable session kickoffs from me, as that largely happened by me first creating a chat to explore the next horizon and then asking that chat session to create actual porting sessions for each desired slice. My role was less “assign a task and wait” and more “operate the control loop”: inspect the result, challenge technical decisions, enforce quality gates, and push when an agent treated an intermediate stopping point as the finish line. Human judgment was still very heavily involved even though agents were doing “the work.” My involvement just moved upward… instead of being responsible for writing syntax, I was responsible for framing the problems, defining boundaries, choosing strategies, adjudicating exceptions, and overall ensuring everything was moving in a good direction.
It’s all about caching
LLM providers typically charge one rate for input tokens (what you send to them) and another rate for output tokens (what they send to you). Billing is often done by tokens because they are a useful approximation of the computational work required to do the inference: for each input token, the model must read it, incorporate it into its internal representation, and use it as part of the computation that determines the next token. However, providers often support caching the results of those computations, such that if an identical prefix of a prompt has already been processed, the provider can reuse intermediate computations from the cache rather than recomputing them from scratch. That reduces the cost of processing those tokens, and that savings can be passed along to the consumer. As such, input tokens are often advertised with multiple rates, including a rate for input tokens that are read from the cache.
The discounts are steep! Often providers bill cache hits at a 90% discount, so for example a provider might charge $2.00 for 1 million input tokens but only $0.20 for 1 million cached input read tokens. In other words, you really, really want to maintain good prompt caching so that your bill is an order of magnitude smaller.
The data from the porting effort shows we did well here. The prompt-cache hit rate was 96.22%: cache reads divided by all input-side token volume (cache reads plus cache writes plus fresh input). Cache writes were 3.07%, and fresh input was 0.71%. This is not an accident. GitHub Copilot shapes the agent loop specifically to preserve a long and stable prefix (the system prompt, then the tool definitions, then the accumulated conversation), so each turn appends to context the model has already processed. The expensive part of the context is paid for once and then re-read at an order of magnitude less monetary cost on every subsequent call. It’s also the reason the economics of long autonomous sessions hold together at all. A three-hundred-hour port that re-read its entire growing context from scratch on each of tens of thousands of calls would cost a different order of magnitude than what we saw. Agent harness developers spend a great deal of energy trying to avoid breaking the prompt cache, and model vendors routinely ship new features to help them do so.
Compaction tells a complementary story. Across the port sessions, GitHub Copilot compacted context automatically 5,116 times (the moments when a session had filled its context window and summarized itself in order to keep going). The single sessions-infrastructure port pull request compacted 647 times over its many-day lifespan, while one small port never compacted once. Sustained multi-hundred-hour autonomous work is only possible because the agent can recycle its working memory over and over without losing the thread. Every one of those thousands of summarizations was a point where a lossy handoff could have quietly derailed the port, and mostly didn’t. Copilot’s subagents also greatly factor into minimizing compaction. Each subagent gets its own context, so a parent session can effectively ask a question, have a subagent go off and exert a fair amount of context in computing the answer, and then report just the answer back to the parent. The parent’s context needn’t be impacted by all of that intermediate information.
The “mostly didn’t” above is visible in the session logs. I had Copilot pair each successful compaction with the work surrounding it when at least 20 tool calls existed on both sides. That produced ~4,000 comparable windows. The mix of what the agent did in the 20 tool calls before compacting looks similar in scale to what it did after compacting (exploration 46.5% before and 48.1% after, mutation 8.4% before and 6.0% after, validation 4.7% before and 4.0% after, failures 1.0% before and 1.5% after). If compaction were regularly dropping the thread of thought, we’d expect the after side to be visibly re-orientation-heavy, with a spike in reading and a collapse in editing while the agent rediscovered where it was and what it should be doing. Instead, there’s only a mild shift in that direction.
Yes, static analysis helps
There’s a popular meme that Rust is an unusually good target for AI-generated code because Rust’s strict compiler catches what the model gets wrong. The session logs let us test that theory, at least for tasks that look like this porting effort.
Direct validation-command results captured 8,678 occurrences of rustc‘s error codes. The four largest diagnostic families cover 84%:
- 37%: Name and import resolution, dominated by
E0425(“cannot find value in this scope”) - 22%: Missing methods or fields
- 14%: Type mismatches
- 11%: Unsatisfied trait bounds
Every one of those is ordinary wiring: a name output slightly wrong, a signature that didn’t line up, a field that had been renamed, an abstraction left unimplemented. These are the kinds of mistakes bulk translation easily and accidentally produces and exactly the ones a compiler catches very quickly.
But note what’s absent from that list: anything truly specific to Rust. Every one of those four categories is bread-and-butter static typing, and a C# or Java or Go compiler would catch all of them just as well, several of them with friendlier diagnostics, and all of them a great deal faster. If this is the argument for pointing agents at Rust, it’s really an argument for pointing them at any statically typed language. A strongly typed compiler and/or a language with excellent static analysis and linting is genuinely a good fit for this work, with the agents using it as a fast feedback loop. Across the 4,478 direct cargo check runs for which the stricter result matcher captured an outcome, 87.1% came back clean, which is what you get from editing in small increments and recompiling constantly.
In contrast, ownership, borrowing, and lifetime errors combined were only 1.7% of coded diagnostics. The borrow checker, the thing that dominates every conversation about Rust being hard, was a quiet background presence. The compiler spent almost all of its erroring energy on boring mechanical mistakes.
Agents like reading
We can also examine the corpus of session events for tool call data, and from that extract some interesting observations about how agents spend their time.
| Tool | Calls | Median | Measured hours |
|---|---|---|---|
powershell | 630,423 | 3 s | 2,833.9 |
view | 590,988 | 0 s | 621.7 |
rg | 281,783 | 1 s | 408.4 |
grep | 126,483 | 1 s | 115.3 |
apply_patch | 53,715 | 0 s | 17.0 |
edit | 40,591 | 1 s | 24.1 |
read_powershell | 36,728 | 90 s | 1,203.9 |
task | 13,080 | 274 s | 2,329.0 |
My first takeaway here is that the agents spent far more time gathering evidence than changing code. Across the displayed file-reading and search tools versus the editing tools, they did 10x as much exploration as mutation. Reading files, searching the repository, and running diagnostic commands dominated; edits were a comparatively small effort. The popular image of AI spewing code is almost backwards; at this scale, the work looked much more like iterative investigation, inspecting the current state, forming a hypothesis, making a targeted change, rinsing and repeating.
Delegation amplified that pattern. Subagents were used primarily to fan out exploration across independent questions, while the main agent was more likely to own the edits and integrate the answers. That’s a useful division of labor for this kind of project: many contexts can investigate in parallel, but keeping mutation closer to the coordinating agent reduces conflicting changes and preserves a coherent implementation strategy.
The shell traffic also shows how much of autonomous software work is state management. Read-only Git inspection was the most common command pattern because the agents were constantly asking, effectively, “where am I?” They were looking for what had changed, what a rebase had done, what another session had landed, and how far a branch had drifted from a rapidly evolving main. That orientation work enabled many long-running efforts to operate against the same moving codebase without blindly overwriting one another.
Looking inside the shell-tool traffic, the most common command families make that balance between orientation and validation even clearer:
| Command family | Calls | Median | Measured hours |
|---|---|---|---|
git inspect | 300,530 | 2 s | 608.1 |
git other | 89,865 | 3 s | 243.1 |
| search | 85,482 | 2 s | 147.6 |
pnpm test | 13,852 | 22 s | 219.1 |
pnpm lint | 9,757 | 29 s | 177.0 |
cargo test | 8,437 | 120 s | 364.2 |
git commit | 7,410 | 11 s | 39.7 |
cargo fmt | 5,223 | 18 s | 77.2 |
cargo check | 4,492 | 120 s | 176.9 |
pnpm build | 3,630 | 180 s | 215.6 |
cargo clippy | 2,115 | 135 s | 107.4 |
git rebase | 2,496 | 7 s | 9.9 |
cargo build | 566 | 104 s | 20.3 |
Model selection
GitHub Copilot lets a single session change models mid-conversation and lets different sessions run different models, so model choice became a per-slice decision. Two different kinds of model decision show up in the logs. On the main thread, the one driving each port, we chose the model and the reasoning effort. Inside a session, when the agent spun up subagents or subsessions to explore or to grind through a bounded task, the orchestrating model chose those models.
For subagents, the model mix looks a bit different, with an agent rather than a human optimizing for throughput and cost rather than for the hardest judgment calls. The subagents it spawned most often ran on Claude Opus 4.8, GPT-5.6 Sol, Claude Haiku 4.5, and GPT-5.5, followed by Gemini 3.1 Pro and Claude Opus 5. However, at least at the time the ports were happening, three frequently used agent definitions pinned their model choice (explore and task to Claude Haiku, research to Claude Sonnet), so a significant part of that volume was determined by the choice of subagent rather than by choosing a model separately.
Working with agent fleets
The GitHub Copilot app’s support for visualizing active pull request sessions, the status of each, and easily switching between them made it ideal for managing lots of concurrent work inherent to the port. But one of the things that really made it shine was its ability for sessions to interact with other sessions.
A session can create other sessions, and it can message other sessions while they run. Each session, parent or child, gets its own worktree, its own branch, and its own agent loop; it’s separate from the session that spawned it rather than something running inside it. That’s different from a subagent, which runs inside the parent’s own workspace and hands its answer back into the parent’s context. Both are useful constructs for different things.
As an example of how a session might create other sessions, one of the hardest ports was for the session.ts file. This file had grown organically to be ~30,000 lines of TypeScript. It represented the backbone of a session and effectively spanned horizontally across the whole runtime, touching and being touched by practically every component, sitting at the center of state, events, tools, models, hooks, persistence, and entrypoint access. As a result, I left it for close to the end of the porting process, working up from the bottom of the stack across all the verticals until they all dead ended at session.ts. The porting session that took it on did not start by just diving in and writing Rust. It spent its first fifty-six minutes reading, with 122 tool calls before it created anything, building a picture of what the file actually owned and where the seams were. Only then did it start delegating, logically splitting up the file and delegating slices to subsessions. Across the whole 25-hour run, it made 222 shell calls, 205 file views, and 197 ripgrep searches of its own, on top of everything its child sessions did.

That’s 15 child sessions, each one a separate branch with its own worktree and a separate agent, all created implicitly by the parent session at the top. The parent session created them in seven waves over about three hours: the first wave created five, the second another two about twenty minutes later, then another pair twenty minutes after that, then singles and pairs spread out over the next two hours.
Model choice was made per slice: 10 of the 15 ran on GPT-5.6 Sol and five on Claude Opus 4.8. All 15 were started in GitHub Copilot’s autopilot mode, which lets a session pursue an objective without stopping for approval at each step. The median kickoff prompt was about 1,100 characters, long enough to carry the ownership boundary and the constraints, but short enough that the child session had to work out the approach itself. I prompted the parent agent, and then the parent agent, not a human, wrote those kickoff prompts to each child session.
Alongside those 15 subsessions, the same parent session also employed five subagents: three explore agents fired in parallel with the first wave, one code-review, and one rubber-duck. These subagents explored questions, feeding back to the parent answers it needed in its context before it could decide what to do next. The subagents allowed the parent to get deeply thought out answers without needing to spend its own context window on deriving them.
In contrast, child sessions went to work on the actual porting, the work that produced a diff and needed isolation from the other parallel porters. The child sessions’ work touched 140 distinct files in the repo, 120 of which were touched by exactly one session. The 20 contended files were all hubs, such as session.ts itself. But each session was working in its own worktree, and so was able to proceed undisturbed by its siblings. The parent, of course, paid for that in coordination. It spent a good deal of effort communicating with its child sessions, acting as an information broker, polling their state 60 times and sending 89 coordination messages. When the child sessions each announced their completion, the parent cherry-picked their commits into its own branch and resolved the conflicts. These were not particularly clean merges, either, and the parent agent spent a decent amount of time reconciling the edits.
We can see that on a timeline, which shows the parent session and most of its child sessions.
Note those large gaps. I was traveling while working on this port, and I had to close my laptop at various points. (I subsequently changed my workflow to incorporate cloud-based virtual machines I could remote into.)
These parallel child sessions had a significant impact on that laptop. For a while, the concurrent porting was going swimmingly. Then all 15 concurrent agents on one machine each tried to build and test, and my poor laptop ground to a halt. I prompted to the parent, asking it to relay to its child sessions that they must all stop building and testing. The parent relayed that constraint outward, and they thankfully killed their builds and proceeded to work with minimal CPU activity. I subsequently updated my standing instructions that subagents and subsessions should avoid large builds and test runs while porting, instead deferring that to be done only by the parent agent.
Later I took that one step further and made an otherwise ordinary chat session into a build scheduler for eight independent porting sessions. The prompt was embarrassingly simple: send every open session a policy to avoid CPU-intensive building and testing where possible, require it to ask this session for permission when a build was necessary, and act as a gate, handing out the ability for one session at a time to build. Basically I turned the chat session into an agentic mutex. The gate kept an explicit owner and queue and granted one lease at a time through the same cross-session messaging mechanism the sessions already used to coordinate code. Sessions that asked for and were denied the lease often waited by doing other work in the meantime, like picking off things from its todo list.

This session.ts port was also involved in one of the coolest, saddest, and certainly most unexpected interactions I witnessed during the whole runtime port. As I mentioned, we did the port primarily bottom-up, which is why session.ts that effectively sits on top of every other component was one of the last components ported. The only thing consistently above session.ts are all of the entrypoints into the runtime, namely the public functions that are exposed from the SDK and that show up in the previously discussed dispatch table. There are hundreds of these. And while I know that many of them immediately call into session.ts, I wanted to get a jump on the porting, and so after launching the session.ts session, I launched a session to port all of the entrypoints. I told it to stop at the session.ts boundary. I figured there may be a bit of throwaway work and some amount of effort or number of tokens needed in a rebase, but that it would accelerate the overall porting. Then I went to bed. And then… they found each other.
My kickoff prompt for the entrypoints session did tell it that the session port and six component ports were running concurrently, as I wanted it to know its boundaries and what it should avoid porting to avoid as many conflicts as possible. Apparently my prompting had the opposite effect. Just over four minutes in, having inventoried the ingress paths and presumably formed a view of how much they overlapped, it invoked an app built-in orchestrate skill, whose purpose is coordinating work across sessions. From there:
- The entrypoints session enumerated every active session and sent messages to the ones with perceived overlap.
- The
session.tssession answered with a 2,001 character inventory titled “Concrete overlap onstephentoub-port-session-to-rust“ - The entrypoints session read the
session.tssession’s worktree to confirm what it had just been told (trust but verify, I guess). - The entrypoints session asked the
session.tssession whether it was ready to reconcile its 760 file diff. - The
session.tssession basically told it to get lost: “Not ready to commit/integrate.” - The entrypoints session proceeded to ask the same question three more times, and each time it got back the same answer from the
session.tssession. - At which point the entrypoints session decided it didn’t care what the
session.tssession thought and simply reached into its worktree and grabbed all of the other session’s changes and merged them into its own. - Then both sessions went on their merry way.
A few things I took from this interaction:
- It’s important to be explicit about intent. The kickoff prompt named the other running sessions so this one would know what to leave alone. But I didn’t make that “leave alone” part explicit, so instead of blocking the agent from doing something, I ended up encouraging it to do it. I needed to be much more explicit in my intent and guidance.
- Whatever you make available is something an agent may decide applies. The
orchestrateskill ships in the GitHub Copilot app and describes itself as being for running independent workstreams in parallel. Nothing in my prompt mentioned it. The model discovered its own situation, matched it against that description, and loaded it. The set of capabilities you expose is the set of behaviors you might get, including in situations you never pictured. - Peers need a tiebreaker. Neither session could compel the other. When the
session.tssession said it was not ready to integrate four separate times, that refusal carried no weight, so the session willing to act unilaterally won by default. Parallel sessions over adjacent code need a designated coordinator, or they need a human, and these had neither. - “Run autonomously” needs an exception for decisions that reach outside your own branch. I really meant “don’t wake me up over design details.” It heard (not unreasonably) that annexing a peer was in scope. Again, I should have been more explicit in my guidance.
- The root cause here was me. I partitioned this work top-down and bottom-up at the same time, and the two directions met in the middle at the single most connected file in the codebase. I was too greedy to make forward progress. Everything above follows from that.
This whole interaction was, thankfully, an interesting outlier. Across the whole runtime porting effort, most of the leaf component ports were straightforward single-session tasks. The larger subsystems often involved multiple subsessions and subagents. How those pieces participated over the course of a port varied significantly, though.
The port of model orchestration, the layer that actually talks to the providers, provides a good example of one pattern. Its main session ran 42 wall-clock hours and started 126 subagents. At its busiest, 22 were working at the same time. However, the majority of the time, it was only the main agent, and then now and again it would spawn a significant number of subagents for a window of time.
Three things in that picture stand out. First, effectively all of the code generation was done in the first 12 hours; the next day of work after that is all validation. Second, the color of the bottom row shifts left to right, from predominantly blue and green (reading, building) to predominantly blue and orange (reading, reviewing); that makes logical sense, but it’s neat to see it in practice. Third, that example, and more generally this pattern, has very clean separation between phases. The extension-runtime port is a counter-example.
It took 88 hours instead of 42 and had very different structure:
- Writing and reviewing overlap significantly. Whereas in the previous example, the work was very waterfall (first code generation, then review), here reviewing starts long before writing stops, and both proceed and overlap for most of the duration of the effort.
- The bottom row is all over the place color-wise. Where model orchestration shifts from green to orange as it moves from writing to checking, this one is the same mixture of reading, building, and reviewing from beginning to end. The middle half of the reading calls is spread across a 49-hour span, the writing across 33 hours, and the reviewing across 27, inside an 88-hour session. Every category is spread across most of the run.
- Idling is deferred to the end. The fleet works nearly continuously for the first 56 hours.
- The proportions still match. Writing Rust is 2% of tool calls here against 1% there, reading 44% against 57%, reviewing 23% against 27%. The two sessions agree on what the work to be done is, just not on when it happens.
Those blank slices at the end are also a visual representation of a problem becoming more and more common in this agentic coding era: waiting for approval. Someone on the team and/or an agent reviews the code and leaves feedback, there’s a brief period of activity where the agent addresses the feedback and drives CI to green again, and then more waiting, rinse and repeat, until eventually we get the dopamine-inducing stamp of approval.
These two examples each represent a dominant pattern. About a fourth of the sessions look more like the model orchestration session, while three quarters look like the extensions runtime one. The clean progression through phases was the exception; the common case was the agent planning, writing, and reviewing all the way through.
Code review at scale
Much of that focus on review visible in the previous graphs was due to my explicit prompting. I created a simple prompt-as-a-skill I called rust-rebase-review (in addition to a general Rust coding skill we have merged in the repo). Due to the fast rate of incoming changes, many of which conflicted, I was frequently rebasing. Via custom instructions, I encouraged the harness to invoke this skill at appropriate points in the process, and also manually invoked it from time to time. The prompt evolved a bit over time, but it was a variation of this:
Squash into a single commit, then rebase on the latest in origin/main, resolving all conflicts, and force push. As part of rebasing, pay extra special attention to anything that has changed, been added, been removed, and ensure that logic is all ported over to the corresponding Rust code correctly. Always do the rebasing yourself / in the main agent; do not spawn a subagent for it.
Then enter a review/fix loop where you launch a subagent per opus 5, gpt-5.6-sol, and grok 4.6.
- That subagent should do a line-by-line comparison of the old TypeScript and the new Rust, confirming behavioral equality.
- Look for anything introducing any kind of incompatibility; our goal is to move this code into Rust with as close as is possible to 100% the same semantics. If you hit anything questionable, ask me about it.
- We want to ensure we're writing as efficient and idiomatic Rust code as we can; look for opportunities to simplify, to use routines like from the memchr crate to optimize searches instead of open-coded loops, avoid unnecessary allocation, use traits for reuse and loose coupling, etc.
- Ensure that all defunct TypeScript code (e.g. code that has been fully ported, tests that are now no longer necessary because they're duplicative, unnecessary napi shims, etc.) has been deleted.
- Ensure that we've ported as much code as possible, e.g. if there's any TypeScript remaining in touched files and that TypeScript is more than just a shim, that's a red flag. If new TypeScript that's not just a super thin shim is being added, that's a red flag. Look for any callers of TypeScript shims to see whether those callers can instead be ported to Rust, pushing the boundary as far as reasonably possible. Our goal is to soon get to 100% Rust in the runtime layer.
- Validate that no E2E tests have been deleted or changed. Such changes are an indication of a porting bug.
If a review surfaces issues, validate them, and then if there are any to fix, fix them, and iterate to do another full review. Continue iterating with reviewing/fixing until all reviews come back clean. After every set of changes in response to review feedback, commit and push so that CI validation runs concurrently with subsequent reviews.
Don't bother running full test suites; that'll be handled in CI. Try to minimize CPU-consuming efforts to the bare minimum, as we'll likely have many operations happening concurrently. With the frequent rebasing, it’s easy for incoming changes to get lost accidentally. But we found an unexpected benefit to the in-place atomic swap: by deleting the TypeScript at the same time as we were adding the corresponding Rust, we were implicitly creating conflicts with rebase-induced incoming changes to that TypeScript: one branch changing it and the other deleting it. This guaranteed we’d notice changes to already ported code, rather than needing to rationalize for every incoming line whether it was something that might have been touching previously ported code.
My own reviews were of course only a portion of the agentic review being performed. In addition to CCR running on every commit, the team has multiple dedicated code review bots, each with their own approaches and prompts, running on every commit and providing detailed feedback. All of this would end up in comments on the pull requests that would then need to be addressed. Thankfully, handling all of that agentic feedback can also be handled agentically (mostly).
For my review, I focused on architecture, design, conventions, and approach. The agents did the exhaustive old-versus-new comparisons; tests and static analysis checked mechanically enforceable properties; human reviewers concentrated on architecture, API contracts, risk, and any suspicious places surfaced by those other layers.
I chose the destination architecture, decided what behavior mattered, partitioned the work, resolved ambiguous trade-offs, judged the evidence, manually reviewed high-risk areas, reviewed agentic responses to feedback, and made the final merge decisions. The agents changed the amount of code one engineer could supervise. They did not remove the need for an engineer who understood the system and could vouch for the direction, the guardrails, and the release.
Automating the inner loop
The GitHub Copilot app was central to this work. It manages lots of concurrent active sessions, making it easy to switch between them and carry along with each all the relevant paired context (associated terminal windows, browser windows, canvases, and so on). The feature that mattered most here is agent merge:

Agent merge is a loop built into the app (the CLI has it as well with /pr auto). On a timer or in response to external stimuli (like notifications from GitHub of CI completion or a review comment), the app will look to see what has changed. If a review comment has been left, it’ll invoke the agent to decide whether to reject the comment or to accept and address it (and respond, noting it’s automation that’s responding). If a test fails, it’ll download the logs, investigate the failure, and fix the bug. If a conflict occurs, it’ll invoke the agent to merge or rebase. Effectively, it automates the loop we as human developers all do, driving our pull requests to “green”, getting sign-off, and eventually merging.
Agent merge handled every single porting pull request. In most cases we stopped short of the actual “merge” part, however. The agent would fix all CI failures, address and respond to all comments, and make sure all conflicts were resolved. Before merging, I’d spot check what the agent actually did, and in particular how it addressed feedback. Did I disagree with any of its responses to reviewers? Was the high-level direction of applied fixes desirable and sound? For these ports, I typically left that last entry unchecked.
That last checkbox mattered more than once. In one merge loop pass, the port had deleted one of the functions exposed to the SDK. Our repo’s schema compatibility CI leg did exactly its job and failed. The agent’s response was to apply the repo’s schema-break-ok automation label, which is the escape hatch to make the check pass. In reviewing the pull request before merge, I asked the obvious question: “What is the schema break? You added the schema-break-ok label to the pull request; why is it ok?” It wasn’t. The method existed on main; the port had simply lost it. I called it an unacceptable regression and told the agent to bring it back fully in Rust. Twenty-one seconds later, the waiver was removed, and the method was restored with a native Rust implementation.
We responded to failures both locally and globally, fixing the individual cases but then also fixing the system so that they’d be less likely to recur. We continually evolved the instructions fed to the coding and review agents to further reduce the chances of these same issues happening again in future porting pull requests. We turned session logs into evals. And in some cases, we actually used the lessons learned to improve the runtime itself, via tweaks to prompts or tool descriptions or how autopilot operated.
Two migrations in one
A language rewrite is almost never only a language rewrite. Every library the runtime leaned on had to be replaced, too, and unlike in the Rust code we wrote and owned, those replacements weren’t ours to make faithful. Some were the same idea under a different name. Some took several crates to cover what one npm package had done. A few had no acceptable off-the-shelf answer at all and had to be written by hand (by agent).
The CLI and runtime are currently in the same repo, sharing a package.json. Over the course of the port, we removed ~60 npm dependencies because they were only being used by runtime code that was ported to Rust. This is a lower bound for removals, because there were packages replaced for the runtime but that were still needed by the CLI. For example, zod is a TypeScript schema declaration and validation library that both the CLI and the runtime were using. With the Rust port, the runtime now uses a combination of serde, schemars, and jsonschema to satisfy the same purposes, yet zod is still in the manifest for the CLI’s sake.
There were a bunch of examples where one npm package became one crate doing the same job. js-tiktoken became tiktoken-rs with the same o200k_base encoding. ignore became the crate of the same name with the same gitignore semantics. minimatch became globset, fast-myers-diff became similar, dompurify became ammonia, and github/keytar became keyring.
In other cases, we weren’t able to replace packages with crates one-to-one. Instead one package became several crates, or several collapsed into fewer. This is where the majority of the dependency work went. Eight opentelemetry/* packages became four crates plus a hand-written tracker state machine and file exporter. Three web-content packages, mozilla/readability, linkedom, and turndown, became two crates, readability and htmd. sharp, image-size, file-type became image and imagesize. And so on. There were also five cases where we replaced an npm package entirely with a completely custom implementation.
How much unsafe?
Another question people ask about agent-written Rust is how much of it quietly opted out of safety guarantees. Rust’s safety is a property you can turn off with a keyword, so an agent that hits a borrow it can’t satisfy has an obvious escape hatch. Within the entire runtime crate, we now have 158 unsafe blocks, across only 36 files (alongside them are 26 unsafe fn declarations, 26 unsafe extern blocks, and nine unsafe impl trait implementations). Importantly, every single one of these is about interop with external components.
| Why the unsafe block exists | Blocks | Share |
|---|---|---|
| C ABI boundary | 51 | 32.3% |
| Windows API | 49 | 31.0% |
| POSIX / libc | 46 | 29.1% |
| SQLite C API | 7 | 4.4% |
| Dynamic library loading | 4 | 2.5% |
| Process environment | 1 | 0.6% |
The C ABI blocks are the front door SDK hosts come through, so they receive raw pointers and lengths from a caller the Rust compiler has no control over. The Windows and POSIX blocks are system calls: registry reads, credential handshakes, process trees, sysconf. SQLite is a C library. Dynamic library loading is dlopen, which cannot be safe by construction if for no other reason than because the symbol you resolve might not be the function you expected. The process environment unsafe block exists because Rust 2024 treats mutation of process-global environment state as unsafe in a multi-threaded process. Every one of those unsafe blocks marks a place where the guarantees Rust makes genuinely end: the thing on the other side is a C function, a syscall, a pointer from a foreign runtime, or process-global host state.
The useful property is that unsafe makes all of these places in the Rust code we own auditable. The equivalent code in the TypeScript runtime crossed exactly the same boundaries, through Node’s C++ internals and native npm packages, and nothing in our source marked where the checked world stopped. Of course, that’s not a complete inventory of every safety boundary in the delivered system: dependencies, build tools, C libraries, safe wrappers, and incorrectly specified FFI contracts can still contain or expose unsafety.
The details of where unsafe is used are interesting, but I’m more interested in where it isn’t used. It’s not used in the model clients, in the MCP layer, in the agent layer, or in the prompt layer. And not one of the known regressions from the porting effort involved an unsafe block.
The regressions
Porting code is easy. Making it correct is hard. And with a codebase as large and as complicated as the Copilot agent runtime, regressions are to be expected.
By September 14, 2026, we’d traced dozens of known port regressions, all fixed. Most were correctness bugs, with a smaller set of performance regressions. Against the ~832,000 lines of production Rust written from scratch.
Of course, not all of those regressions shipped. Some were caught just by developing in the repo. Others appeared in a pre-release but were fixed before making it to a stable release. And some reached a stable release, often after being unnoticeable enough to make it through one or more rounds of pre-release usage undetected.
This isn’t zero, of course, and I’m 100% sure there are more than the ones we know about. These are the ones we noticed or that were reported, but a migration this size absolutely shipped a few more that have been quiet enough that nobody has hit them yet. As with bugs in general, I expect we’ll continue to discover a trickle of corner-case regressions as the further reaches of the stack are exercised aggressively in the wild.
The absolute number also isn’t all that important. What’s more important is the “why” all of these happened, so that we can learn from the issues and avoid repeating them in the future. Nearly all of the correctness regressions land in three large families: the new code implemented a different behavioral contract; state, ownership, or lifetime behavior changed; or some part of the migration was omitted, only partially applied, or lost in rebasing. A smaller set came from requirements at the host or interop boundary, and even from tests that confidently validated the wrong behavior. Those families become more concrete in a handful of recurring patterns:
Ambiguous semantics. Several regressions came from behavior the source language left implicit. TypeScript has one number type; Rust requires choosing among several, including whether a value can be fractional. And the agents guess wrong. Fields that were conceptually integers became f64s, so Rust serialized values such as 42.0 instead of 42: strongly typed SDKs like Go and C# couldn’t unmarshal a repo ID into an int64 and rejected a hook timestamp and task duration. In the other direction, an agent declared timeToFirstTokenMs as i64, but the streaming path emitted values such as 5446.712845, making written sessions unreadable and unresumable. A subtler case wasn’t about types: event.error || "Unknown error" became .unwrap_or("Unknown error"). JavaScript’s || replaces an empty string; Rust’s unwrap_or preserves it, so an empty subagent error remained empty. Oops.
Ambient behaviors. Another recurring source was behavior JavaScript or Node supplied invisibly. Quota code used toLocaleDateString, which inherits the host time zone; Rust needed that zone passed explicitly. But although Intl.DateTimeFormat().resolvedOptions().timeZone is typed as string, it can return undefined, which napi couldn’t convert to a Rust String, breaking model-list loading. Elsewhere, moving an environment-variable read from before an await to after it meant a host change during the wait could alter the result. A native-addon loader called process.report.getReport() only to identify the platform, but on Windows that honored _NT_SYMBOL_PATH and could spend minutes downloading PDBs before rendering the CLI. Other ambient inputs included the working directory, repository identity, PATH, and session authentication. Moving ownership into Rust required deciding when to capture each one, how to carry it, and when to refresh it.
Only porting half of a pair. Several of the regressions came from paired operations falling out of sync. A turn-cap check updated the native registry’s abort state but didn’t cancel the in-process model loop, allowing one more request to escape. Elsewhere, task completion was persisted and emitted but not projected into active session state, so Autopilot continued after completing the task.
Blocking the main thread. The CLI still drives the Rust runtime from Node’s single-threaded event loop, so synchronous work across the napi boundary freezes the UI. /chronicle reindex parsed hundreds of session files this way, blocking rendering and input for nearly a minute. Making the export async and moving the work to a blocking thread-pool thread fixed it. An audit found several more potentially blocking entry points, leading to a standing rule: napi exports that do real work must be asynchronous and, when necessary, use spawn_blocking.
Flashing windows. On Windows, spawning a child process without CREATE_NO_WINDOW briefly opens a console window. The Node.js runtime had monkey patched process spawning to add the flag, hiding the requirement from the porting agent; the Rust replacement omitted it. An audit fixed two additional spawn sites, though those were preexisting omissions rather than port regressions, and we added the rule to the copilot instructions.
Lifecycle management. The largest cluster involved lifecycle, disposal, ownership, ordering, or races. Moving state into Rust often left TypeScript holding an opaque handle to an instance stored in a native table. Unlike an object reference, that handle can outlive the instance. One hook disposed mid-request orphaned a tool_use block, wedging the conversation because the model API required a matching result. A shell canceled between “announced” and “started” leaked an orphan that kept the session active. A sandbox toggle updated one generation counter but not its native twin, leaving the shell stuck “reconfiguring.”
Overlooked features. Another group involved features the port simply missed. One port omitted SDK callbacks and deleted their end-to-end test, prompting a new rule that agents must not change E2E tests without explicit consent. A session abort kept its native half but lost the in-process cancellation needed to interrupt a turn waiting in a tool. The SDK’s ability to replace built-in tool search depended on enablement, the model-facing description and schema, and routing execution to the SDK callback. The port dropped all three (yay for consistency?), silently replacing one consumer’s natural-language search with regular-expression search.
Different libraries with different opinions. Several more regressions came from replacing JavaScript libraries or APIs with stricter Rust equivalents. At the time, the Rust MCP SDK (rmcp) responded to malformed JSON-RPC input while the TypeScript SDK didn’t. Against a server that answered errors with more malformed output, that politeness became an infinite loop that hung startup. Related libraries in different ecosystems rarely behave identically.
Ships passing in the night. A handful of regressions came from branch drift or rebasing. In a repo with hundreds of pull requests per week, a session open for days accumulates constant changes and conflicts. The port required thousands of rebases; even a very high success rate leaves failures.
And the slowpokes. The last group was functionally correct but slower. Some regressions lost existing efficiencies such as memoization, fully asynchronous waiting, or bounded log streaming. Others added migration-specific overhead at the Rust-TypeScript boundary: redundant serialization, locking, polling, unbounded native concurrency, and native-to-host crossings. These weren’t later optimization opportunities; each was a degradation introduced by the port. One read-only scan deep-copied a 260 MB event log instead of borrowing it. Under sustained event traffic, another implementation completed only a tiny fraction of the requested channel flushes while retaining an asynchronous handle per event until V8 exhausted its heap.
Dozens of regressions sounds like a lot. But in a port that generated more than 800,000 lines of code, frankly I’m surprised and pleased we didn’t encounter an order of magnitude more. We can also get a sense for whether these were broadly “felt” by looking at trends in issues in the public github/copilot-cli and github/copilot-sdk repos. Here we’ve classified issues opened from January through August as quality related when they carried a bug label or when their title used common failure terms such as “bug,” “regression,” “crash,” “hang,” “timeout,” “broken,” or “incorrect.” The levels were essentially unchanged from before the porting work as compared to during and after it:
| Repository | Jan–Apr, before the rewrite | May–Aug, during / after the rewrite |
|---|---|---|
github/copilot-cli | 22.9% (454 / 1,982) | 23.7% (354 / 1,496) |
github/copilot-sdk | 36.2% (190 / 525) | 32.3% (135 / 418) |
That’s not an availability metric or an exact count of escaped defects… as with the regressions themselves, there can be a ton of variability in what an issue represents, its scope, and so on. But it does provide a useful check: despite the extraordinary volume of product change, the product-facing issue channels didn’t show a meaningful quality concern spike during the migration.
If it compiles, it compiles
Earlier I referred to a popular meme that Rust is great for AI-generated code because of its strict compiler. There’s another related and popular Rust meme, that if the code compiles, it’s correct. Nope. Our known regression list provides a good response to that: every regression in the corpus was merged to main, which means it successfully compiled. The compiler accepted the buggy versions because, as far as the compiler was concerned, every one of them was valid Rust.
The compiler can prove that an f64 is used consistently. It cannot know that a repository ID must be serialized as an integer or that a timestamp serialized with a trailing .0 will be rejected by every strongly-typed SDK at the other end of the wire. The compiler can prevent unsynchronized data races in the code it can see, but it can’t prevent a perfectly synchronized state machine from encoding the wrong states. A queue can be protected by locks and still let two senders each conclude the other will drain it. Events can move safely between threads and still arrive in the wrong order. A synchronous napi function can be memory-safe and still block Node’s main thread for a minute. Compilation is also almost by definition unable to detect something that isn’t there. The compiler can’t object when a rebase quietly removes a guard and its test, or when a process spawner forgot the Windows flag that suppresses console window pop-ups. It also had no opinion about cloning a 250 MB event log on every read. A compiler checks whether the program you wrote is internally coherent. It cannot check whether you wrote the whole program, preserved the old contract, called things in the right order, met the host’s unwritten requirements, or did the work at an acceptable cost.
That’s in no way an argument against Rust’s compiler. As with any statically-typed language, the compiler eliminated a huge class of mechanical mistakes and gave the agents an exceptionally useful inner loop. But “if it compiles, it’s correct” is useful only as a joke.
Perf, perf, and more perf
So what did we buy with all of this? The port was deliberately behavior-preserving. It did not set out to redesign algorithms or fix bugs; in fact, I repeatedly pushed the agents away from opportunistic optimization because changing language and behavior at the same time makes it much harder to know which one broke you. A key goal of the rewrite, however, was indeed performance and scalability (in addition to reliability and other factors). When folks asked me why did I set out to rewrite the runtime in Rust, my answer was often along the lines of “I didn’t set out to move to Rust, I set out to move away from Node.js and V8”. That move has indeed significantly moved our needle on performance.
I benchmarked several scenarios for the runtime via the C# SDK before and after the port (the TypeScript, Python, Go, C#, Java, and Rust SDKs all reach the same engine through the same transport architecture). The baseline against a pre-port build of the SDK and CLI, with the TypeScript runtime hosted by Node and reached over stdio. The August 21 result uses the Rust runtime, both as an out-of-process server and loaded in-process through FFI. This is an end-to-end comparison of the delivered systems rather than an attempt to isolate the effect of the language change; other changes landed during the same period, so the numbers need to be taken with some grains of salt.
Each timed turn went to a deterministic chat completion server running on localhost, producing a fixed, small response. In other words, these numbers deliberately remove model inference and network latency. They measure the part we changed: client startup, process launch, session creation, event handling, persistence, teardown, etc.
| Scenario | May 12 | Aug 21 out-of-process | Aug 21 in-process |
|---|---|---|---|
| Client, session, one turn | 5.25 s | 1.33 s (4.0x) | 292 ms (18.0x) |
| Resume 32-turn session | 5.64 s | 1.52 s (3.7x) | 264 ms (21.4x) |
| Ten concurrent client lifecycles | 12.34 s | 4.18 s (3.0x) | 742 ms (16.6x) |
| 1,000 one-turn session lifecycles | 132.52 s | 22.53 s (5.9x) | 20.93 s (6.3x) |
The “client, session, one-turn” number is the easiest one to feel. It’s creating a client, creating a session, doing a single turn, and tearing everything down. A large part of the out-of-process overhead came from starting Node, initializing V8, and loading, parsing, and generating bytecode for the JavaScript application produced from the TypeScript code before the first turn could begin. The Rust runtime removes those Node/V8 and JavaScript-loading costs.
The “1,000 one-turn session lifecycles” pressure test represents a step function change in what we can build. It uses a single shared client and measures running 100 concurrent pipelines, each creating a session, taking a complete model turn, disposing the session, and doing that ten times in sequence. The pre-port TypeScript CLI completed 7.55 of those lifecycles per second. Rust out-of-process completed 57.45. Rust in-process completed 120.0.
That is a workload-specific result; the Rust runtime is not universally “15.9x faster.” But it is exactly the workload server hosts care about: many independent sessions sharing one runtime. And wall-clock time isn’t hiding the work on another core. In a separate resource-sampling pass over the same 100-by-10 workload, the pre-port process tree consumed 312 seconds of aggregate CPU. The Rust configurations consumed about 110 seconds. That is CPU capacity the host can spend on more sessions.
Memory tells the same story, with the usual warning that memory metrics are annoyingly easy to misuse. Looking at resident private memory added during the ten-client batch, the pre-port process tree peaked 1,383 MB above baseline. Rust out-of-process peaked at only 247 MB. And Rust in-process at 126 MB, an order of magnitude less. While these numbers of course will differ from use to use and machine to machine, they get to the core goal: a service can host far more clients and sessions on the same machine before memory, process count, or CPU become the limiting resources.
The best part is this is the baseline port. Much of the implementation still consists of TypeScript-shaped algorithms faithfully rendered in Rust. We haven’t yet done the broad redesign work that the new ownership model, concurrency model, and in-process architecture enable. Getting an in-process client through creation, a complete one-turn session, and teardown in about 55 milliseconds, pushing a shared client to 120 one-turn session lifecycles per second, and cutting the measured ten-client memory delta by 91% before that optimization work begins is a very good place to start.
What the port cost
So… how much did all this cost, monetarily? Drum roll, please….
My token spend for all of the porting work was ~136.3 billion total tokens, including ~130.6 billion cached input read tokens, ~4.2 billion cached input write tokens, ~900 million fresh input tokens, and ~600 million output tokens. The monetary bill for all those tokens came to ~$120,000.
Of course, these tokens weren’t spending themselves. The cost for using a significant amount of a developer’s time guiding these agents should also be factored in. However, I wasn’t devoting all of my time just to this project. Agentic development is a lot of “hurry up and wait;” submit a prompt, let the coding agent do its thing, check in on it from time to time to possibly steer it, but in the meantime until it completes, do other things. That means developers are no longer working on just one coding task at a time; they’re overlapping that waiting to enable working on many things concurrently. During the porting window, these Rust porting PRs represented ~20% of my PRs across all repos to which I was contributing. If we wave our hands and estimate that the share of pull requests approximates the share of my time, that works out to roughly three weeks dedicated to the port.
In other words, the rough bill for the porting effort was about $120,000 in attributed token spend plus three weeks of a developer’s time.
That said, the end-to-end Rust migration was not 100% a single developer; it’s been a team effort. @stevesandersonms provided the design and implementation for napi-oop, the temporary out-of-process interoperability layer, as well as five of the six SDK FFI implementations, with @edburns providing the sixth. @roji implemented the packaging for the SDK to ship and utilize the Rust binaries correctly. @caarlos0 has been helping to split the Rust code into many small subcrates in order to help with build times that started to become problematic, while @criemen helped with improved asset caching to speed up CI and local builds. @devm33, @examon, @MRayermannMSFT, @dereklegenzoff, and others have helped with countless pull request reviews and approvals. And everyone contributing to the copilot-agent-runtime repo has been supportive and accommodating as the world has shifted beneath their feet.
Lessons learned
The experience reinforced a few lessons that apply beyond this rewrite. Here are some things we’d carry into a next time:
- The goal needs to be clearly and fully stated. Our early instructions were too vague. “Port XYZ component to Rust” got read as only the hot paths, or only the logic, and the agents repeatedly treated I/O and orchestration as being out of scope. Once we were explicit that the end state was a native binary built from a 100% Rust codebase, with no execution environment left for TypeScript even if we wanted one, they got much better at driving autonomously to that goal.
- End-to-end tests are absolutely, unequivocally critical. With one exception, all of the regressions that involved missing features, and many of the others, were due to lack of sufficient end-to-end tests. For any port of this nature, you must have tests that can be used to validate the correctness of the port, and those tests can’t themselves be re-written during the porting, or else you lose your oracle. Our initial plan for this porting effort called this out, and called out that we needed to significantly improve our E2E testing posture before starting on the port. We did that, but we didn’t do it enough. We’re sure that if we’d added even more E2E tests before starting the port, really focusing on ensuring most meaningful behaviors were covered by them, we would have had fewer regressions than we did along the way.
- Protect the oracle from the agent. The agent changing an implementation cannot also be allowed to silently redefine correctness by weakening a test, updating a snapshot, raising a compatibility baseline, or applying an escape-hatch label, at least not without oversight. Keep the behavioral contract independent where possible, put sensitive guardrails behind separate ownership or approval, and layer checks with different failure modes so that one mistake isn’t enough to ship a big regression.
- Translate first, redesign second. Preserving behavior and existing algorithms kept the number of variables moving at once manageable. Once the old implementation and the transitional scaffolding are gone, ownership, concurrency, and performance can be redesigned against a stable baseline. I veered away from this a few times, out of restlessness or difficulty saying no to a peer or a conviction that this case was different, and in hindsight I regret every one of them. Each cost more regressions, more time, or more tokens than staying the course would have.
- Turn repeated failures into future successes. AI agents will wander off track: when they do, learn from it. When a failure mode shows up twice, it belongs in standing instructions, a reusable skill, an eval, a protected baseline, or the harness itself.
- The developer inner loop matters more with agents in the loop, not less. For us as developers, we live and breathe in our inner loop, how quickly we can make a change, build, test, iterate. We get frustrated when the tooling part of that takes too long. And it’d be forgivable to think that doesn’t apply when agents are doing the lower-level work. It does. More so. AI agents do the thinking and writing code parts of these tasks incredibly quickly, but they still need to build and they still need to test. And the proportion of the time they build and test actually increases as they spend less time thinking and writing and more time in fast validation inner loops. Spend some time up front optimizing the inner loop, and optimizing it for multiple things happening at the same time (e.g. as if you’re working on multiple tasks in multiple worktrees at the same time). You will thank yourself later for the investment.
What’s next?
It worked. The execution runtime that was entirely TypeScript in May is now entirely Rust in August, and it shipped to real users continuously the whole way rather than landing as one terrifying cutover at the end.
I didn’t simply ask an AI agent to “port this whole codebase from TypeScript to Rust.” Even if that’s where we’re heading as an industry, we’re definitely not there yet. Instead, agents made an entire category of project feasible. A rewrite producing hundreds of thousands of lines of production Rust in a live system, done in place, in main, by one engineer supported by a team, is not a proposal that would have been accepted before agents. It would have needed a whole team and a year or two, it would have competed against every feature that team could have shipped instead, and it would have lost (and, honestly, should have lost). Agents moved the price to where the project became tenable.
The port itself is complete: the runtime’s production implementation is 100% Rust, and the temporary internal TypeScript/N-API seam is gone. There is still plenty of work we desire to do, though: further improving the build system and developer inner loop, cleaning up translated structures, redesigning around Rust’s ownership and concurrency models, and pursuing further performance wins. The port was a translation, deliberately so (prompted as such), and we kept behavior as close to 100% identical as possible, rather than opportunistically fixing additional bugs, rearchitecting components, or further improving performance and scalability (beyond what came implicitly from the rewrite). Much of the code at the micro level is idiomatic Rust, but at the macro level there’s a good deal of initially TypeScript algorithms wearing Rust’s syntax. Revisiting those decisions now that the constraints underneath them have changed is where the interesting wins will be.
I’m most excited about what the port makes possible. The SDK can be loaded directly into a host process in any of six languages, with no Node.js or V8 in the dependency chain, and no second process to supervise, which is the single most common piece of friction we’ve heard from partners adopting the SDK. A runtime instance that costs a fraction of what it used to means a host can run far more concurrent sessions before it runs out of machine. And the runtime can now go places Node.js was never going to follow it, across the spectrum of cloud to desktop to device to embedded systems. None of that is the finish line. It’s the base we now get to build the future of GitHub Copilot on top of, and after three months of watching agents rewrite the very engine that runs them, I’m looking forward to seeing just how far it goes.