DEF CON CTF 是黑客界的奥运会。它的挑战赛题属于整个网络安全领域最难的题目之列，通常涉及对二进制文件进行逆向工程并构造可用的漏洞利用。在 DEF CON 预选赛的历史上，这些任务通常需要一支超过20 名世界级黑客组成的团队——有时甚至多达数百人——昼夜不停地工作，才能在为期三天的窗口期内解出这些题目。
今年，有了 AI，情况有所不同。在 UC Santa Barbara 和 UC Berkeley，我们构建了 SageCTF：一个 CTF 专用智能体，基于我们下一代智能体脚手架OpenSage搭建。它以单人选手身份参加预选赛，取得了令人瞩目的强劲成绩：
- 在 7 道高难度挑战中夺回 8 个 flag，而其中每一道题都需要一支专业黑客团队花费数小时才能解出
- 总共获得 1,743 分——位列所有队伍的前 5%
- 表现优于所有声称不使用 AI 或仅少量使用 AI 的队伍
- SageCTF 总共尝试了15道非交互式挑战。在未能拿到 flag 的题目中，有4道非常接近——距离解出仅差一到两步
SageCTF 标志着一次重大突破，展示了 AI 智能体在全球一些最困难的安全挑战上日益增长的能力。
从技术上讲，OpenSage 让 AI 自行构建其智能体系统，而不是遵循预先指定的拓扑结构或工作流。它还为智能体提供了分层记忆，从而实现细粒度的知识管理。这两者都非常适合 CTF 这类任务，因为这类任务具有超长周期，并且需要复杂的长期推理。
SageCTF 显著优于 Claude Code
除了现场 DEF CON CTF 之外，我们还在 50 个 CTF 挑战上进一步评估了 SageCTF 和 Claude Code。这些挑战是来自近期 TAMU CTF、UMass CTF 和 RITSEC CTF 的难题，以及一个公开的 NYU CTF Benchmark 数据集。为了实现完全对等的比较，两个智能体使用了相同的评估设置：
- 主模型：Claude-Opus-4.6，高推理强度
- 沙箱：预装工具的定制 Docker 容器
- 预装工具：mcp（ida-pro、ghidra）、python 库（pwntools、angr、playwright 等）、实用工具（z3、sagemath、qemu 等）
- 时间限制与预算：每个挑战 10 小时和 $200
SageCTF 与 Claude Code 在 CTF 基准上的对比。
SageCTF 解决了 50 个挑战中的 39 个，而 Claude Code 解决了 13 个。更重要的是，Claude Code 解决的每一个挑战也都由 SageCTF 解决，没有任何仅由 Claude Code 解决的挑战。
SageCTF 在 DEF CON CTF 2026 上的卓越表现
免责声明
CTF 是由人类选手构建、为人类选手而生的艺术形式：一个展示真实安全技能、分享技术、与其他黑客一同享受这门手艺的地方。我们希望在不干扰这种文化、也不干扰现场比赛的前提下评测 SageCTF。由于 DEF CON CTF 2026 不允许完全自动化的机器人提交 flag，我们使用 SageCTF 来解题，但没有提交任何 flag。
SageCTF 端到端地解题（不提交 flag）。
DEF CON CTF 2026 是一场为期 48 小时的线上 Jeopardy 式 CTF，涵盖 web、crypto、pwn、misc、逆向工程、koth（King-of-the-Hill）和 LiveCTF 共 23 道挑战题。
超过一千支队伍报名，686 支队伍出现在计分排行榜上。在计分队伍中，根据主办方的 AI 使用政策，36 支自报为“No AI”，139 支自报为“Low AI”，511 支自报为“Human-Led AI”。
SageCTF 尝试了 15 道挑战题，解出了其中 7 道，共获取 8 个 flag，总计 1,743 分。这一成绩使 SageCTF 跻身 686 支计分队伍的前 5%，领先于许多世界知名的黑客团队。
这一成绩在该竞赛的 AI 使用类别下同样引人注目。SageCTF 的排名领先于 每一支自报为使用 No AI 或 Low AI 的计分队伍，这两类合计共 175 支队伍。
SageCTF 解出了极其困难的挑战
DEF CON CTF 的挑战全都很难，远超常规 CTF 竞赛。即便最简单的挑战——Birdhouse——解出率也只有 12.8%。
SageCTF 按类别、得分和解出率统计的解出挑战。
上表展示了 SageCTF 解出的七个挑战及其得分和难度（以解出率衡量）。解出率中位数低于 8%，七个解出的挑战中有六个解出率低于 10%。最难的一次完全解出是 gitvfs，686 支计分队伍中只有 30 支拿到了两面 flag——完全解出率为 4.4%。
SageCTF 背后的秘密武器
SageCTF 构建在 OpenSage 之上，引入了若干基础性设计创新，用于解决困难的长期任务：AI 生成的拓扑结构、智能体间通信、灵活的记忆层级，以及多模型编排。
SageCTF 与 Claude Code 最大的差异在架构：拓扑结构、通信、记忆和模型协作。
训练 SageCTF 变得更强
我们从 OpenSage 起步，这是我们定制的自编程智能体框架，随后通过 CTF 比赛对 SageCTF 进行重塑，在这个过程中每一个弱点都暴露无遗：缺少工具、环境搭建脆弱、视觉理解能力薄弱、解题拓扑僵化、智能体之间沟通不畅、单一模型的盲区、并行探索深度不足，以及长时间解题过程中的记忆丢失。
SageCTF 通过实况 CTF 压力测试不断演进，每一场比赛都为其补上了缺失的能力。
上图展示了 SageCTF 在各类 CTF 比赛中的演进过程。第一个版本的 SageCTF 基于默认的 OpenSage 智能体，在 TAMU CTF 中排名约前 20%。在加入 CTF 专用工具支持后，SageCTF 在 RITSEC CTF 中跃升至约前 10%。
在接下来的几场比赛中，我们加入了灵活的 AI 创建拓扑、多模型协作、智能体间通信，以及结合外部 RAG 的长期记忆。此后 SageCTF 开始在前列和精英梯队中徘徊，最终在 DEF CON CTF 中取得了前 5% 的成绩。
SageCTF 行为的更多细节
无需人工干预的数小时解题过程
下图展示了这次运行的两个方面：SageCTF 的得分何时上升，以及每道解出的题目各花了多长时间。
每道题的解题时长与得分随时间的变化轨迹。
解题时长表明，每一道被解出的挑战都需要持续数小时的自主探索。即便是 Birdhouse，按解出率来算是 SageCTF 解出的最简单的一道挑战，也花了整整一个小时。在七道被解出的挑战中，SageCTF 平均每道完整解题耗时 5 小时。My Favorite Instructions 运行了 12 小时，而 gitvfs 则延伸为两阶段解题，第一个 flag 在 3.5 小时后才出现，完整解出这道挑战大约用了 10 小时。
这一点很重要，因为长时间的 CTF 解题充满了假设与探索。智能体必须检查产物、搭建本地环境、验证假设、调试脚本、舍弃走不通的利用路径，并在没有人类引导调查的情况下保留阶段性进展。SageCTF 让这些循环持续得足够久，从而把不确定的探索转化为可用的 flag。
把一道挑战变成一支智能体团队
如上所述，OpenSage 会根据给定任务自动构建智能体拓扑。借助这一关键特性，SageCTF 将 CTF 挑战拆解为面向不同任务的不同多智能体系统。下图展示了 Birdhouse 挑战的一个拓扑。根 ctf_agent 从用户输入的提示词出发，从竞赛门户拉取挑战描述和产物，然后为调查中需要独立上下文、工具或搜索策略的部分创建专门的智能体。
最终 Birdhouse 解题中一个真实的 SageCTF 智能体拓扑。
在 Birdhouse 运行中，根智能体协调整个流程，而子智能体负责聚焦性工作，例如产物检查、逆向工程、漏洞利用假设测试和结果整合。子智能体还可以创建自己的子智能体，当某个子问题大到足以值得拥有自己的团队时，SageCTF 便拥有了递归层级结构。
这种拓扑结构使整个求解过程具备灵活性。一个智能体可以探索高风险的假设，而另一个智能体验证具体的原语；一个分支可以失败，而不会抹去系统其余部分的有用状态。根智能体接收发现结果，转发有用的部分，并保持最终漏洞利用路径的连贯性。
数千次模型调用，一次协调求解
SageCTF 原生支持多模型编排，并在求解过程中在不同模型之间切换。下图展示了所使用的模型调用和 token。在七个已解决的挑战中，SageCTF 进行了 23,919 次模型调用，处理了 2.4B tokens，将工作路由至 GPT-5.5、Claude-Opus-4.6 和 DeepSeek-V4-Pro 模型。
SageCTF 在已解决挑战中的模型调用规模，按模型系列细分。
SageCTF 使用 GPT-5.5 作为根智能体模型。大多数子智能体在创建时继承了父智能体的模型，从而保持求解的稳定性和连贯性。当证据变得特别复杂或模糊时，SageCTF 可以切换模型，或并行运行多个模型来交叉验证同一假设。
模型选择源自我们观察到的行为表现。GPT-5.5 是一个稳健的根模型，因为它能以更少的安全误报处理异常的 CTF 产物和漏洞利用形态的输入，降低了整个 SageCTF 因安全对齐而停摆的可能性。Claude-Opus-4.6 往往是困难推理、逆向工程和漏洞利用构造方面最强的求解器，因此在比赛期间承担了很大比例的高价值求解工作。
DeepSeek-V4-Pro 比 GPT-5.5 和 Claude-Opus-4.6 便宜得多、也快得多，使其适用于那些需要广度而非推理投入的碎片化探索任务。
案例研究 1：SageCTF 如何解决 mapllvm
mapllvm 展示了 SageCTF 架构的实际运作。主程序接收一个受限的单行源文件，并将其提交给一个定制的 Racket-to-LLVM 编译器服务。用户必须利用编译器/运行时的行为，使生成的 ELF 在没有直接文件读取原语的情况下读取 /flag。
以智能体拓扑方式求解 mapllvm：有针对性的消息将零散发现转化为可用的漏洞利用链。
SageCTF 从广泛探索开始求解。一个侧边 web_search_agent 和一个全新上下文的头脑风暴智能体帮助测试外部想法，并排除了诸如 Racket #reader / #lang 路径这样的死胡同。有用的路径来自静态/动态循环：dynamic_explorer 在 GDB 中验证了一个陈旧向量堆别名原语，而 static_auditor 将该原语转化为一个伪造闭包漏洞利用源。消息总线广播了重要的突破，包括验证绕过原语、u -> fv 别名以及最终的 shellcode 方向。
随后漏洞利用链变得具体起来。SageCTF 使用陈旧向量别名来控制伪造闭包字段，在 GDB 中确认了控制流目标，将跳转目标修正为 0x401048，并利用可执行立即数字节来运行 shellcode，该 shellcode 会打开、读取并写入 /flag。recon_solver 整合了这些发现，运行了最终的远程漏洞利用，并获取了 flag。
除了动态子智能体拓扑之外，SageCTF 的智能体通信机制也使协调循环变得可用。当某个对等方需要确切的候选方案或验证结果时，智能体会发送有针对性的更新，而消息总线则广播高价值发现，例如已验证的原语和已确认的死胡同。这使求解过程保持同步，而无需强制每个智能体共享同一个庞大的上下文。
案例研究 2：SageCTF 仍然不足的地方
Pixels and Nicotine 是一次接近成功但仍未命中的案例，SageCTF 在这里仍然表现不足。该挑战要求玩家从一个 4MiB 的二进制转储文件中恢复隐藏的 payload，并将其作为 flag 提交。
像素与尼古丁：SageCTF 恢复了整个流水线，却错过了决定最终显示顺序的关键线索。
SageCTF 正确地构建了大部分逆向工程流水线。它识别出该二进制文件是一个 NES ROM，发现一个被打补丁的例程将隐藏字节解码为 CHR 图块显示数据，定位到了相关字节，并逆向了计分例程。缺失的一环是一个语义鸿沟：NES 渲染例程按照内存序列的逆序写入图块，而 SageCTF 却按内存顺序读取解码后的字节。这产生了一个看似合理的 flag 候选，字符正确，但顺序错误。
我们的后验分析表明，根本原因在于 SageCTF 缺乏解读最终字节所需的冷门 NES 渲染线索。一个更细粒度的外部 RAG 系统将有助于应对这类挑战。
动态评判反馈也会有所帮助。SageCTF 的候选 flag 距离正确 flag 仅一步之遥；如果允许该智能体在比赛期间提交尝试，来自计分板的负面反馈本可以引导最终搜索走向正确的显示顺序。
要点总结
在 DEF CON CTF 2026 上，SageCTF 展示了完全自主智能体在最困难的 CTF 挑战上已经达到了何种程度。它恢复了 8 个 flag，解决了求解率仅为个位数的挑战，维持了长达数小时的调查，跻身前 5%，并超越了所有 No-AI 和 Low-AI 团队。
教训很直接：高难度 CTF 自主性如今已是一个智能体问题。强大的模型固然重要，但获胜的自动系统必须具备灵活且先进的智能体拓扑结构、跨智能体通信以及细粒度的记忆管理。
最优秀的人类团队依然更强。CTF 仍然是一门建立在品味、创造力和深厚安全直觉之上的人类技艺，完全自主的智能体仍有很大的提升空间。SageCTF 为这一方向上的未来改进奠定了基础。
DEF CON CTF is the Olympics of hacking. Its challenges are among the hardest in all of cybersecurity, typically involving reverse-engineering binaries and crafting working exploits. Throughout the history of DEF CON quals, these tasks have usually demanded a team of more than 20 world-class hackers — sometimes hundreds — working around the clock to solve them within the three-day window.
This year, with AI, things are different. At UC Santa Barbara and UC Berkeley, we built SageCTF: a CTF-specialized agent built on our next-generation agent scaffold, OpenSage. Competing as a solo player in the qualifiers, it achieved impressively strong results:
- Recovered 8 flags across 7 difficult challenges, where each one requires a team of professional hackers a few hours to solve
- Earned a total of 1,743 points — placing in the top 5% of all teams
- Outperformed all teams that claim not to use AI or use low AI
- In total, SageCTF attempted 15 non-interactive challenges. Among those it didn’t flag, 4 came very close — only one or two steps away from a solution
SageCTF marks a major breakthrough, demonstrating the growing capability of AI agents on some of the world’s most difficult security challenges.
Technically, OpenSage lets the AI construct its own agent system rather than following a pre-specified topology or workflow. It also gives the agent hierarchical memory, enabling fine-grained knowledge management. Both are well-suited to tasks like CTF, which are ultra-long-horizon and demand complex, long-horizon reasoning.
SageCTF Significantly Outperforms Claude Code
In addition to live DEF CON CTF, we further evaluated SageCTF and Claude Code on a set of 50 CTF challenges. These challenges are difficult tasks from recent TAMU CTF, UMass CTF, and RITSEC CTF, as well as a public NYU CTF Benchmark dataset. To enable an apple-to-apple comparison, both agents used the same evaluation settings:
- Main model: Claude-Opus-4.6 with high reasoning effort
- Sandbox: customized Docker container with tools preinstalled
- Preinstalled tools: mcp (ida-pro, ghidra), python libs (pwntools, angr, playwright, …), utilities (z3, sagemath, qemu, …)
- Time limit and budget: 10 hours and $200 per challenge
SageCTF vs Claude Code on CTF benchmarks.
SageCTF solved 39 of the 50 challenges, while Claude Code solved 13. More importantly, every challenge solved by Claude Code was also solved by SageCTF, with zero Claude-Code-only solves.
Remarkable Performance of SageCTF at DEF CON CTF 2026
Disclaimer
CTFs are an art form built by and for human players: a place to demonstrate real security skill, share techniques, and enjoy the craft with other hackers. We wanted to measure SageCTF without interfering with that culture or the live competition. Because DEF CON CTF 2026 does not permit fully automated bots to submit flags, we used SageCTF to solve the challenges but did not submit any flags.
SageCTF solves challenges end-to-end (without submitting flags).
DEF CON CTF 2026 was a 48-hour online Jeopardy-style CTF with 23 challenges across web, crypto, pwn, misc, reverse engineering, koth (King-of-the-Hill), and LiveCTF. More than one thousand teams registered, and 686 teams appeared on the scored leaderboard. Among the scored teams, 36 self-reported as using “No AI”, 139 as “Low AI”, and 511 as “Human-Led AI” under the organizers’ AI usage policy.
SageCTF attempted 15 challenges and solved 7 of them, recovering 8 flags for a total of 1,743 points. This result places SageCTF in the top 5% of 686 scored teams, ahead of many of the world’s famous hacking teams.
The result also stands out under the competition’s AI-usage categories. SageCTF finished ahead of every scored team that self-reported as using No AI or Low AI, a combined group of 175 teams.
SageCTF Solved Really Difficult Challenges
DEF CON CTF challenges are all difficult, far beyond routine CTF competitions. Even the easiest challenge — Birdhouse — had a solve rate of only 12.8%.
SageCTF's solved challenges by category, score, and solve rate.
The table above shows SageCTF’s seven solved challenges with their scores and difficulty (measured by solve rate). The median solve rate was below 8%, and 6 of the 7 solved challenges had solve rates below 10%. The hardest full solve was gitvfs, where only 30 of 686 scored teams recovered both flags — a 4.4% full-solve rate.
Secret Sauces Behind SageCTF
Built on top of OpenSage, SageCTF introduces several foundational design innovations for solving difficult, long-horizon tasks: AI-generated topology, inter-agent communication, a flexible memory hierarchy, and multi-model orchestration.
SageCTF and Claude Code differ most in architecture: topology, communication, memory, and model collaboration.
Train SageCTF to Be Better
We started from OpenSage, our customized self-programming agent framework, then reshaped SageCTF through CTFs, where every weakness became visible: missing tools, brittle environment setup, weak visual comprehension, rigid solve topology, poor agent communication, single-model blind spots, shallow parallel exploration, and memory loss across long solves.
SageCTF evolved through live CTF pressure tests, with each competition adding the missing capabilities.
The figure above shows the evolution of SageCTF through various CTF competitions. The first SageCTF version was based on a default OpenSage agent, and it landed around the top 20% at TAMU CTF. After adding CTF-specific tooling support, SageCTF jumped to around 10% at RITSEC CTF. Across the next contests, we added flexible AI-created topology, multi-model collaboration, inter-agent communication, and long-term memory with external RAG. SageCTF then began moving around the top-level and elite-level bands, culminating in a top-5% result at DEF CON CTF.
More Details on SageCTF’s Behaviors
Hours-Long Solves Without Human Intervention
The following figure shows both sides of the run: when SageCTF’s score increased, and how long each solved challenge took.
Per-challenge solve duration and score trajectory over time.
The solve durations show that each solved challenge required sustained autonomous exploration for hours. Even Birdhouse, the easiest challenge SageCTF solved by solve rate, took a full hour. Across the seven solved challenges, SageCTF spent an average of 5 hours per full solve. My Favorite Instructions ran for 12 hours, while gitvfs stretched into a two-stage solve where the first flag arrived after 3.5 hours, and the full challenge took around 10 hours.
That matters because long CTF solves are full of assumptions and exploration. The agent has to inspect artifacts, build local environments, test hypotheses, debug scripts, discard broken exploit paths, and preserve partial progress without a human steering the investigation. SageCTF kept those loops alive long enough to turn uncertain exploration into working flags.
Turning One Challenge Into a Team of Agents
As mentioned above, OpenSage automatically constructs agent topologies based on given tasks. Leveraging this key feature, SageCTF breaks down CTF challenges into different multi-agent systems for different tasks. The following figure shows a topology for the Birdhouse challenge. The root ctf_agent starts from the user input prompt, pulls the challenge description and artifacts from the contest portal, then creates specialized agents for the parts of the investigation that need independent context, tools, or search strategies.
A real SageCTF agent topology from the final Birdhouse solve.
In the Birdhouse run, the root agent coordinated the whole process while subagents handled focused work such as artifact inspection, reverse engineering, exploit hypothesis testing, and result integration. Subagents can also create their own subagents, giving SageCTF a recursive hierarchy when a subproblem becomes large enough to deserve its own team.
This topology enables flexibility across the solving process. One agent can explore a risky hypothesis while another validates a concrete primitive; one branch can fail without erasing the useful state from the rest of the system. The root agent receives findings, forwards the useful ones, and keeps the final exploit path coherent.
Thousands of Model Calls, One Coordinated Solve
SageCTF natively supports multi-model orchestration and switches between different models during the solving process. The following figure shows the model calls and tokens used. Across the seven solved challenges, SageCTF made 23,919 model calls and processed 2.4B tokens, routing work across GPT-5.5, Claude-Opus-4.6, and DeepSeek-V4-Pro models.
SageCTF's model-call scale across solved challenges, broken down by model family.
SageCTF used GPT-5.5 as the root agent model. Most subagents inherited the parent agent’s model when created, keeping the solve stable and coherent. When the evidence became especially complex or ambiguous, SageCTF could switch models or run multiple models in parallel to cross-check the same hypothesis.
The model choices came from our observed behavior. GPT-5.5 is a robust root model because it handles unusual CTF artifacts and exploit-shaped inputs with fewer safety false positives, reducing the chance that the whole SageCTF stops because of safety alignment. Claude-Opus-4.6 was often the strongest solver for difficult reasoning, reverse engineering, and exploit construction, and therefore carried a large share of the high-value solving work during the contest. DeepSeek-V4-Pro was much cheaper and faster than GPT-5.5 and Claude-Opus-4.6, making it useful for fragmented exploration tasks that needed breadth more than reasoning effort.
Case Study 1: How SageCTF Solved mapllvm
mapllvm shows SageCTF’s architecture in action. The main program receives a restricted one-line source file and submits it to a custom Racket-to-LLVM compiler service. Users must exploit compiler/runtime behavior to make the generated ELF read /flag without direct file-read primitives.
The mapllvm solve as an agent topology: targeted messages turned partial findings into a working exploit chain.
SageCTF started solving with broad exploration. A side web_search_agent and a fresh-context brainstorm agent helped test external ideas and eliminate dead ends like the Racket #reader / #lang path. The useful path came from the static/dynamic loop: dynamic_explorer validated a stale-vector heap aliasing primitive in GDB, while static_auditor turned that primitive into a fake-closure exploit source. The message bus broadcast the important breakthroughs, including the validation bypass primitive, the u -> fv alias, and the final shellcode direction.
The exploit chain then became concrete. SageCTF used the stale vector alias to control fake closure fields, confirmed the control-flow target in GDB, corrected the jump target to 0x401048, and used executable immediate bytes to run shellcode that opens, reads, and writes /flag. The recon_solver integrated the findings, ran the final remote exploit, and recovered the flag.
Besides the dynamic subagent topology, SageCTF’s agent communication mechanism also made the coordination loop usable. Agents sent targeted updates when a peer needed an exact candidate or verification result, while the message bus broadcast high-value findings such as the validated primitive and confirmed dead ends. This kept the solve synchronized without forcing every agent to share the same giant context.
Case Study 2: Where SageCTF Still Falls Short
Pixels and Nicotine is a close miss where SageCTF still falls short. The challenge asks players to recover a hidden payload from a 4MiB binary dump file and submit it as the flag.
Pixels and Nicotine: SageCTF recovered the pipeline, but missed the final display-order clue.
SageCTF built most of the reverse-engineering pipeline correctly. It identified the binary as a NES ROM, found that a patched routine decoded hidden bytes into CHR tile display data, located the relevant bytes, and reversed the score routine. The missing piece was a semantic gap: the NES renderer routine wrote tiles in reverse order of the memory sequence, while SageCTF read the decoded bytes in memory order. That produced a plausible flag candidate with the right characters but in the wrong order.
Our post-hoc analysis shows the root cause is that SageCTF lacked the niche NES rendering clue needed to interpret the final bytes. A more fine-grained external RAG system would help on this kind of challenge.
Dynamic judging feedback would also help. SageCTF’s candidate flag was one step away from the correct flag; if the agent had been allowed to submit attempts during the contest, negative feedback from the scoreboard could have guided the final search over the correct display order.
The Takeaway
At DEF CON CTF 2026, SageCTF showed how far fully autonomous agents have reached on the hardest CTF challenges. It recovered 8 flags, solved challenges with single-digit solve rates, kept multi-hour investigations alive, moved in the top 5%, and outscored every No-AI and Low-AI team.
The lesson is direct: hard CTF autonomy is now an agentic problem. Strong models matter, but the winning automatic system must have flexible and advanced agent topology, cross-agent communication, and fine-grained memory management.
The best human teams are still stronger. CTF remains a human craft built on taste, creativity, and deep security intuition, and fully autonomous agents still have plenty of room to improve. SageCTF set the foundation for future improvement along this direction.