(完整立场文件:
迈向自主软件开发
在软件工程的大部分历史中,其组织方式始终围绕一种稀缺资源:能够编写和审查代码的人。编程语言、框架、测试系统和组织架构中所有当前的最佳实践,都是围绕这一约束建立起来的。这些实践的目的,是帮助人们在有限的时间、注意力和认知能力下,将需求转化为可靠的软件。
有了 AI,这一约束正开始松动。
前沿编程智能体如今已能自主跨代码仓库进行推理、编写并执行测试、识别漏洞,以及协调多阶段工作。在一个引人注目的演示中,一支由十六个并行 Claude 智能体组成的团队以不到 $20K 的成本构建了一个可工作的 C 编译器。然而,在那些旨在测试持续软件演化而非孤立任务的基准上,前沿智能体仍然会急剧退化:它们能够添加功能,但难以在接连不断的变更中保持正确性和架构一致性。
这两种现实应当结合起来看待。编程智能体的能力正变得足以承担有意义的自主职责,但其可靠性尚不足以让我们将自主性视为一种单一的、不加区分的能力。
因此,关键问题不再仅仅是“AI 如何帮助开发者完成某项指定任务?”,而是“如何在整个软件开发生命周期中把责任从人类转移到 AI,以及这种转移要安全、可靠、可问责,必须具备哪些条件?”
软件自主性的共享词汇
自动驾驶研究很早就认识到,一个领域如果没有共享词汇,就无法清晰地思考安全、能力与责任问题。SAE 自动化分级使得区分驾驶辅助与有条件自动化、完全自动驾驶成为可能,并能明确每个阶段由谁承担责任。
软件开发领域没有可与之相比的框架。如今,“自主编码智能体”既可以指一个只建议几行代码的工具,也可以指一个会提交 pull request 的智能体、一个能测试并部署自身变更的系统,或一个决定要构建哪些功能的智能体。这些是根本不同的系统,具有根本不同的失效模式。
受 SAE 自动化分级启发,我们根据软件开发生命周期中哪些阶段已从人类责任转移为完全由 AI 控制,提出并定义了软件开发的三个自主性等级。
我们的分级对 AI 系统拥有软件开发生命周期的多少部分,以及哪些责任仍由人类承担进行分类。
第 I 级——代码自主性
AI 完全接管系统设计与实现,无需人类逐行审批。智能体产出完整的 pull request,包括设计理由、代码和文档。人类仍然决定要构建什么,以 pull request 为粒度审查所提议的变更,监督测试与安全审计,并把关部署。我们认为当今的 AI 辅助编程是这一层级的先声。
第二级——流水线自主
AI 运行从设计与实现到测试、审计和部署的整条流水线。人类既不编写也不审查代码。他们只提出高层需求,并评估最终行为。这是一次质的断裂:它假定人类意图能够被足够完整的规格说明所捕获,并且即便没有人类检查中间产物,自动化验证与确认仍然可信。如今在大规模场景下,这两个假定都不成立。
第三级——需求自主
AI 不仅构建、测试和部署软件,还决定应该构建什么。它从遥测数据、用户行为、安全公告、依赖变更以及系统的持续演化状态中识别需求。人类在循环往复的开发闭环中的最后角色也随之消失,尽管系统仍受人类确立的创始使命所约束。核心挑战由此变为:确保自我生成的需求持续服务于那一使命,而不是悄然重新定义它。
我们预计,不同领域的组织会以不同的节奏推进这一自主性框架。高保障领域可能长期停留在 Level I 或以下,而内部工具和一次性应用则可能更快接近 Level II。这套分类法旨在让能力主张、部署选择和问责机制变得清晰可读。
在使用近期模型时,人们很容易认为我们已经深入 Level I 或 Level II 的自主性,但这些级别旨在达到稳定的能力,而非当今那些容易出错但往往可用、且没有正式正确性保证的输出。
自主性不止一个维度
这三个自主性级别回答了一个核心问题:AI 系统拥有软件开发生命周期的哪些阶段?但仅凭级别本身,并不能完全刻画系统的自主程度。处于同一级别的两个系统,可能因三个额外的横切维度而表现出截然不同的运作方式:
- 规格粒度:一份带有可复现测试的详细缺陷报告会约束智能体。而诸如“添加多租户”这样的请求,则迫使它去推断范围、架构、权衡取舍和成功标准。薄弱的规格说明甚至可能把一个名义上较低级别的系统推向更高自主性所面临的挑战。
- 时间自主性: 智能体可以在工单级别、冲刺级别、发布级别运作,也可以持续运行数月乃至数年。长时程运行会引入记忆、溯源、回归和架构一致性等问题,这些是单次基准测试无法捕捉的。
- 监督模式: 人类参与的程度可以从共同制定规格、动作级审批,到 pull request 审查、策略护栏、仅监控运行,以及自动回滚。合适的模式在很大程度上取决于领域风险和可逆性。
统一性问题:保留人类意图
在全部三个层级中,同一个挑战以不同形式出现:随着人类直接控制的退场,如何保留并忠实执行人类意图。
在层级 I,人类仍可通过审查来还原意图。在层级 II,意图必须被编码进一份没有人端到端验证过的规格说明中。在层级 III,AI 必须在系统历经数年演进的过程中自行综合并维护这份规格说明。
许多看似彼此独立的失败,其实是同一个问题的不同表现。规格漂移意味着实现逐渐偏离人们原本的意图。奖励黑客意味着系统满足了某个可度量的代理指标,却违背了底层目标。多智能体分歧意味着不同智能体基于互不兼容的理解行事,却没有把冲突暴露出来。一个编写测试的智能体可以产出一份实现和一套测试套件,二者彼此一致,却共同出错。
核心风险:当同一个智能体既编写实现代码又编写测试时,测试通过可能只证明了自洽性,而非正确性。
这就是为什么更高的自主性改变了保障的对象。仅仅验证软件产物已不再足够。我们还需要审计产出它的智能体:其规范、技能、记忆、决策溯源、通信协议以及执行轨迹。
已经浮现的不当实践:跳级
最直接的风险并非某种奇异的完全自主系统,而是组织在跳级。
一个团队可能名义上运行在 Level I,由人类仍然对审查和部署承担全部责任,但实际上却在合并没有任何人进行过有意义检查的智能体生成的变更。该团队采用了 Level II 的做法,却没有 Level II 的验证、治理或问责机制。当团队让智能体从遥测数据或外部内容发起变更,却没有保护好触发这些变更的渠道时,类似的压力也会出现。
因此,我们主张明确的层级门控:只有当与当前层级相关的挑战已被切实解决时,系统才应向前推进。举证责任应随着部署的自主性、持续时间和风险的增加而提高。
六大结构性转变
如果实现变得充裕,软件开发不会简单地变成当今工作流的加速版。它会经历一场更深层的变革。软件工程的主要产物会改变。工程流程本身会改变。最终,软件工程生态会重新组织。
我们预计会在这三个层面上展开六项结构性转变。
一、软件变得生成式
第一项转变发生在软件产物本身。人类意图日益取代实现,成为软件的持久表示,而实现则变得更容易生成、修改和重新生成。
1. 规格说明成为主要开发产物
随着人类与代码的交互减少,规格说明成为人类意图与机器执行之间的主导接口。它们不仅必须捕捉功能需求,还必须捕捉安全约束、可维护性预期、架构不变量、产品约定和测试策略。
手动维护这类规格说明,最终可能会比维护它们所源自的交互历史更加困难。因此,我们预计会出现规格蒸馏:人们通过对话、示例和反馈与智能体协作,而智能体则持续将这些交互编译为持久、结构化的规格说明。随着时间推移,规格说明可能会趋近于与软件系统近乎一一对应的关系,使得仅凭规格说明就能重建实现。
2. 抽象边界变得可渗透
现代软件抽象栈,例如函数、模块、库、API、框架和服务,其存在是为了适应人类的认知局限。随着自主智能体获得在软件栈更大范围内进行推理的能力,这些抽象仍然是宝贵的组织工具,但它们不再是实现上的刚性约束。只要这样做能改善整体系统,智能体就可以常规性地重写、合并、拆分、内联或绕过这些抽象。
这并不意味着抽象会消失。相反,它们的角色从开发者必须在其内部工作的结构,转变为智能体可以调整和重新组织的灵活约定。随着时间推移,软件共享可能会从实现转向协议、行为规格说明、接口契约和参考实现,从而赋予智能体更大的优化自由,同时保持互操作性。
3. 软件从静态产物转向动态、持续演进的系统
自主开发不仅改变了软件的构建方式,也改变了软件的本质。软件系统不再是按离散版本发布的静态制品,而可以成为神经符号系统,将确定性保证与神经网络的灵活性相结合;可以自我演化,持续适应运行信号;可以对话式交互,模糊使用系统与修改系统之间的边界;也可以是临时的,为单一任务生成,当重新生成比复用更划算时便将其丢弃。
一个成熟的系统可能同时具备其中若干特性:在安全关键的核心采用确定性逻辑,在自适应边缘采用神经网络,在生产环境中持续演化,在交互界面上支持对话。
II. 工程转向以智能体为中心
随着软件日益生成式化,工程流程本身也必须演进。验证、协调和开发工作流不能再假设人类是软件的主要生产者。
4. 保障从制品转向智能体
AI 生成的实现即使满足同一份规格说明,也可能存在显著差异。这种多样性削弱了依赖熟悉编码模式或稳定实现的保障技术。更重要的是,自主智能体可能同时生成实现、测试、文档和理由说明,从而在原本用于相互验证的所有制品之间造成相关性故障。
独立验证者智能体只有在以真正独立的目标、可信的评估机制以及用于解决分歧的原则性协议运作时,才能发挥作用。因此,保障的范围已超越对软件制品的验证,扩展到对产出这些制品的自主智能体及其开发流程的评估。
5. 多智能体协调超越人类组织结构
当前大多数多智能体系统仍然类似于人类组织:管理者分配工作,专家执行任务,审查者检查输出,沟通主要通过自然语言进行。这些结构反映的是人类认知的局限——注意力、通信带宽、记忆和控制幅度——而非协作的根本要求。
自主智能体在不同的约束条件下运作。它们可以分叉和合并执行状态、共享完整上下文、通过结构化协议进行协调,并扩展到数千乃至数百万个并发协作者。因此,我们预计将涌现出根本性的新型 AI 原生协调形式,同时也会带来新的协调失败、安全风险和问责问题。
III. 软件工程生态系统的重构
随着软件制品和工程工作流的变化,周围的软件生态系统也必须随之演进。组织、机构、教育和市场都在适应这样一个世界:实现变得充裕,而可信的自主性变得越来越有价值。
6. 软件工程生态将重构
随着自主智能体承接越来越多的实现工作,软件开发组织将能够以规模小得多的工程团队来交付和维护日益复杂的系统。人类的贡献将转向产品定义、架构、规格说明、评估、安全、治理和事件响应。
与此同时,全新的生态机构将变得不可或缺。独立审计机构、认证组织、可信度评估服务商和治理服务,可能会变得如同今天的测试平台或云服务商一样重要。软件工程教育同样将更加重视规格说明、验证、安全、治理和系统级判断,而非仅仅重视手工实现。
关于软件开发未来的十大预测
上述结构性转变引出了一组具体且可检验的预测。这些预测是刻意前瞻性的,而非必然发生的。不同领域将以不同的速度演进,但这些预测共同展示了自主软件开发可能如何在未来十年重塑软件工程。
软件制品
1. 规格说明成为软件的基因组
软件系统的持久化表示将日益成为其规格说明:即需求、约束、架构决策、测试策略与设计依据的持续演进记录。代码则成为该意图的一种编译实现,而非权威产物本身。
2. 重新生成成为重构的实用替代方案
对于能够从其规格说明忠实重建的软件而言,重新生成一份干净的实现,可能比逐年累月地增量修复累积的复杂性更为廉价。这一转变不会普遍适用。我们预计,那些拥有大量未记录知识或遗留集成的系统,仍将继续以增量方式演进。但对于许多其他应用而言,重新生成将从根本上改变维护的经济性。
3. 软件共享从实现转向协议
随着定制实现变得廉价,库、框架和 SaaS 平台将日益充当行为标准、接口契约和参考实现,而非不可变的依赖项。自主智能体将在本地适配、优化或重新生成实现,同时保持对外可见的行为不变。
4. 动态软件变得司空见惯
软件将日益把确定性保证与神经网络的灵活性、持续适应、对话式界面和按需生成结合起来。许多生产系统将不再只有单一的标准实现,而是会持续演进,同时保持稳定的外部行为。
工程实践
5. 规格蒸馏成为核心工程能力
开发者将不再预先编写完整的需求文档,而是越来越多地通过对话、示例、评审和修正来沟通。自主智能体会持续将这些交互蒸馏为结构化、可维护的规格,并随系统一同演进。
6. 保障重心从审查代码转向认证智能体
代码审查仍将重要,但保障将日益聚焦于生产软件的自主智能体。组织将审查智能体的规格、权限、记忆、执行轨迹、来源溯源、评估结果和治理控制。信任生产过程将变得与信任产出的制品同样重要。
7. AI 原生协作结构取代仿人类工作流
如今的多智能体系统在很大程度上沿袭了人类组织的模式。随着时间推移,智能体团队将围绕机器能力而非人类管理约束,发展出沟通、委派、共识与冲突解决机制,从而实现人类团队难以企及的大规模协作。
8. 项目上下文成为持久资产
随着实现代码的重新生成变得廉价,组织将越来越看重指导软件生成的持久项目上下文,而非任何特定的实现。规格说明、架构决策、工程约定、约束条件、评测历史以及积累的反馈,其生命周期可能跨越数代代码。以可移植、厂商中立的格式保存这些项目上下文,将成为在模型、工具和平台之间保持连续性的关键。
组织与生态
9. 软件开发组织规模缩小,治理强度提升
以实现为主的工程团队可能大幅缩减,而架构、规格、验证、安全、治理、合规和事件响应的重要性则相对上升。与此同时,包括独立审计机构、认证组织和保障服务提供商在内的新生态机构,将成为软件生命周期中日益重要的一部分。
10. 软件变得充裕;信任变得稀缺
自主开发将大幅降低生产定制软件的成本,扩展可构建的内容以及可参与构建的人群。与此同时,生成软件的充裕将使信任日益稀缺。建立来源溯源、验证行为、认证 AI 智能体,以及构建可信的治理机制,可能成为软件生态中最有价值的能力之一。
研究议程,以及我们还有多远的路要走
这份路线图的目的不是宣告完全自主已经到来,而是阐明在负责任地声称更高等级之前,必须具备哪些能力和保障措施。
规约合成与行为理解
我们需要仓库级别的方法,来推断、生成并维护机器可检查的规约。基准测试应衡量规约在多个开发周期中是否始终与实现保持一致。如今,这样的评测基础设施几乎不存在。
可信验证变得至关重要
随着软件开发日益走向自主化,智能体需要可信赖的方式来判断其输出是否真正满足预期规范。形式化验证是最有前景的方法之一,但当今的技术在覆盖范围和可扩展性上仍然有限。验证大型、持续演进的软件系统从根本上依然困难重重。
安全、治理与问责
自主软件开发提出了涵盖安全、治理与问责的根本性研究问题。新的攻击面从智能体记忆、工具、外部数据、多智能体协同以及自主执行等不同方面涌现,需要在安全架构、权限系统、监控、来源追溯与恢复等方面探索新的方法。
除技术安全之外,该领域还必须为可信治理奠定基础。自主智能体应如何被授权、监督、审计并承担责任?什么样的证据足以建立信任?随着自主软件日益成为关键基础设施的重要组成部分,又需要哪些标准、认证流程与责任框架?
以智能体为中心的系统与语言设计
如今的软件栈大多是为人类编写、理解和维护代码而设计的。自主软件开发对这一假设提出了挑战。一个核心研究问题是:如何为智能体优先的软件工程重新设计编程语言、开发环境、内存系统、溯源模型和协调协议,同时不牺牲人类的监督、可解释性或控制权。
人机接口与经济模型
随着自主智能体承担越来越多的软件开发流程,人类与智能体应如何协作这一根本问题随之浮现。项目知识、设计意图、需求和约束应如何传达?人类如何才能有效地理解、监督和引导日益自主的智能体,而不至于成为瓶颈?
我们还需要面向自主软件开发的新经济模型。传统的生产力指标已不再足够;评估自主开发还必须考虑推理成本、人类监督、维护开销、技术债务、系统可靠性和运营风险。
研究优先级:我们最具体的呼吁:构建仓库级规格合成基准,以及对规格—实现漂移的长期评估。没有这些,关于 Level II 或 Level III 就绪度的主张就无法得到证实。
当编写代码不再是瓶颈时会发生什么?
这一转变提出了任何基准测试都无法独自回答的问题:
- 当智能体承担了大部分编码工作时,开发者最具杠杆效应的价值是什么?
- 未经人工代码审查就发布的软件,是一种成熟的工程实践,还是我们尚不知如何衡量的风险?
- 智能体是让软件创造走向大众化,还是将专业能力转移到规格定义、评估、架构与治理之中?
- 当自主智能体部署的软件造成损害时,谁来承担责任?
- 即使智能体在技术上能够做出某项决策,什么决策仍应保留为人类的专属决定?
我们并不认为软件开发的未来会是一个人类从流程中简单消失的故事。持久的人机协作伙伴关系仍然是有可能的,而且采用过程将是渐进且不均衡的。但发展方向已经足够清晰,研究界应当开始将软件开发作为一个自主性不断提升的演进轨迹来研究,而不是将其视为一系列孤立的编码任务。
在 Agentic AI Summit 上继续讨论
我们将在 8 月 1 日至 2 日于 UC Berkeley 举办的 Agentic AI Summit 的“软件工程的未来”专题中探讨这些问题。该专题将汇聚研究编码智能体、开发者工具、评测、安全以及智能体驱动软件开发新兴基础设施的研究人员和构建者。
该讨论小组不仅将探讨编码智能体已经变得多么强大,还将探讨当实现不再是主要瓶颈时,这门学科本身会如何变化,以及在自主性能够被信任之前,需要哪些技术、组织和治理基础。
如需了解完整框架、分类体系和研究议程,请阅读立场论文:Towards Autonomous Software Development。
(Full position paper:
Towards Autonomous Software Development
For most of its history, software engineering has been organized around a scarce resource: humans who can write and review code. All of the current best practices in programming languages, frameworks, testing systems, and organizations have all been built around this constraint. The purpose of these has been to help people translate requirements into reliable software while managing limited time, attention, and cognitive capacity.
With AI, this constraint is beginning to loosen.
Frontier coding agents can now autonomously reason across repositories, author and execute tests, identify vulnerabilities, and coordinate multi-stage work. In one striking demonstration, a team of sixteen parallel Claude agents built a working C compiler with under $20K. However, on benchmarks designed to test continuous software evolution rather than isolated tasks, frontier agents still deteriorate sharply: they can add features, but they struggle to preserve correctness and architectural coherence across successive changes.
These two realities should be considered together. Coding agents are becoming capable enough to take on meaningful ownership, but not yet reliable enough for us to treat autonomy as a single, undifferentiated capability.
The key question is therefore no longer only, “How can AI help a developer complete a specified task?”, but “How can responsibility be transferred from humans to AI across the software development lifecycle, and what must be true for that transfer to be safe, reliable, and accountable?”
A Shared Vocabulary for Software Autonomy
Autonomous-driving research learned early that a field cannot reason clearly about safety, capability, and responsibility without a shared vocabulary. The SAE automation levels made it possible to distinguish driver assistance from conditional automation and full autonomy and to identify who remains responsible at each stage.
Software development has no comparable framework. “Autonomous coding agent” currently can refer to a tool that suggests a few lines, an agent that opens a pull request, a system that tests and deploys its own changes, or an agent that decides which features to build. These are fundamentally different systems with fundamentally different failure modes.
Inspired by the SAE automation levels, we propose and define three levels of software-development autonomy based on which stages of the software development lifecycle have moved from human responsibility to full AI control.
Our levels classify how much of the software development lifecycle an AI system owns, and which responsibilities remain with humans.
Level I — Code Autonomy
AI takes full ownership of system design and implementation without line-by-line human approval. The agent produces a complete pull request, including design rationale, code, and documentation. Humans still decide what to build, review the proposed change at pull-request granularity, oversee testing and security auditing, and gate deployment. We view today’s AI-assisted coding as a precursor to this level.
Level II — Pipeline Autonomy
AI runs the entire pipeline from design and implementation through testing, auditing, and deployment. Humans neither author nor review the code. They state a high-level demand and evaluate the resulting behavior. This is a qualitative break: it assumes that human intent can be captured in a sufficiently complete specification and that automated validation and verification can be trusted even though no human inspects the intermediate artifacts. Neither assumption holds at scale today.
Level III — Demand Autonomy
AI not only builds, tests, and deploys software; it decides what should be built. It identifies demands from telemetry, user behavior, security advisories, dependency changes, and the evolving state of the system. The final human role in the recurring development loop disappears, although the system remains bounded by a founding mission established by people. The central challenge becomes ensuring that self-generated demands continue to serve that mission rather than silently redefining it.
We anticipate organizations in different domains to progress through this autonomy framework at different paces. High-assurance domains may remain at Level I or below for a long time, while internal tools and disposable applications may approach Level II much sooner. The taxonomy is a way to make capability claims, deployment choices, and accountability legible.
It is tempting to assume that we are already well into Level I or Level II autonomy when using recent models, but these levels are designed to reach consistent capabilities rather than today’s error prone but often functional outputs with no formal guarantees of correctness.
Autonomy Has More Than One Dimension
The three autonomy levels answer one primary question: Which stages of the software development lifecycle does the AI system own? But the level alone does not fully characterize how autonomous the system is. Two systems at the same level can operate very differently depending on three additional, cross-cutting dimensions:
- Specification granularity: A detailed bug report with reproducing tests constrains the agent. A request such as “add multi-tenancy” forces it to infer scope, architecture, tradeoffs, and success criteria. Weak specifications can push even a nominally lower-level system toward the challenges of higher autonomy.
- Temporal autonomy: An agent may operate at ticket level, sprint level, release level, or continuously over months or years. Long-horizon operation introduces memory, provenance, regression, and architectural-coherence problems that one-shot benchmarks do not capture.
- Oversight mode: Human involvement can range from co-specification and action-level approval to pull-request review, policy guardrails, monitor-only operation, and automated rollback. The appropriate mode depends heavily on domain risk and reversibility.
The Unifying Problem: Preserving Human Intent
Across all three levels, one challenge appears in different forms: preserving and faithfully executing human intent as direct human control recedes.
At Level I, humans can still recover intent through review. At Level II, intent must be encoded in a specification that no person verifies end to end. At Level III, the AI must synthesize and maintain the specification itself while the system evolves over years.
Many seemingly separate failures are expressions of this same problem. Specification drift means the implementation gradually diverges from what people wanted. Reward hacking means the system satisfies a measurable proxy while violating the underlying objective. Multi-agent disagreement means different agents act on incompatible interpretations without surfacing the conflict. A test-writing agent can produce an implementation and a test suite that are mutually consistent but jointly wrong.
CORE RISK: When the same agent writes the implementation and the tests, passing tests may demonstrate consistency, not correctness.
This is why higher autonomy changes the object of assurance. Validating the software artifact is no longer sufficient. We also need to audit the agent that produced it: its specification, skills, memory, decision provenance, communication protocols, and execution traces.
The Mispractice Already Emerging: Level-Skipping
The most immediate risk is not an exotic fully autonomous system. It is organizations skipping levels.
A team may nominally operate at Level I, with humans remaining fully responsible for review and deployment, while in practice merging agent-generated changes that no one has meaningfully inspected. The team adopts Level II practices without Level II verification, governance, or accountability. Similar pressures appear when teams let agents initiate changes from telemetry or external content without securing the channels that trigger those changes.
We therefore argue for explicit level gating: systems should advance only when the challenges associated with their current level have been demonstrably addressed. The burden of proof should rise with the autonomy, duration, and risk of the deployment.
Six Structural Shifts
If implementation becomes abundant, software development does not simply become a faster version of today’s workflow. It undergoes a deeper transformation. The primary artifacts of software engineering change. The engineering process itself changes. And ultimately, the software engineering ecosystem reorganizes.
We expect six structural shifts that unfold across these three layers.
I. Software Becomes Generative
The first transformation is in the software artifact itself. Human intent increasingly replaces implementation as the durable representation of software, while implementations become easier to generate, modify, and regenerate.
1. Specifications become the primary development artifact
As humans interact less with code, specifications become the dominant interface between human intent and machine execution. They must capture not only functional requirements but also security constraints, maintainability expectations, architectural invariants, product conventions, and testing strategies.
Maintaining such specifications manually may eventually become harder than maintaining the interaction history from which they emerge. We therefore expect specification distillation: people collaborate with agents through conversations, examples, and feedback, while agents continuously compile those interactions into durable, structured specifications. Over time, specifications may approach a near one-to-one correspondence with software systems, making implementations reconstructible from specifications alone.
2. Abstraction boundaries become permeable
Modern software abstraction stacks, such as functions, modules, libraries, APIs, frameworks, and services, exist to accommodate human cognitive limits. As autonomous agents gain the ability to reason across much larger portions of the software stack, these abstractions remain valuable organizational tools, but they are no longer rigid constraints on implementation. Agents can routinely rewrite, merge, split, inline, or bypass abstractions whenever doing so improves the overall system.
This does not mean abstractions disappear. Rather, their role shifts from structures developers must work within to flexible conventions that agents can adapt and reorganize. Over time, software sharing may evolve away from implementations toward protocols, behavioral specifications, interface contracts, and reference implementations, giving agents greater freedom to optimize while preserving interoperability.
3. Software shifts from static artifacts to dynamic, continuously evolving systems
Autonomous development changes not only how software is built, but what software is. Instead of being static artifacts released in discrete versions, software systems can become neurosymbolic, combining deterministic guarantees with neural flexibility; self-evolving, continuously adapting to operational signals; conversational, blurring the boundary between using a system and modifying it; or ephemeral, generated for a single task and discarded when regeneration is cheaper than reuse.
A mature system may combine several of these properties simultaneously: deterministic at its safety-critical core, neural at its adaptive edges, continuously evolving in production, and conversational at its interface.
II. Engineering Becomes Agent-Centric
As software becomes increasingly generative, the engineering process itself must evolve. Verification, coordination, and development workflows can no longer assume that humans are the primary producers of software.
4. Assurance moves from artifacts to agents
AI-generated implementations can differ substantially even when they satisfy the same specification. This diversity weakens assurance techniques that depend on familiar coding patterns or stable implementations. More importantly, autonomous agents may generate the implementation, tests, documentation, and rationale together, creating correlated failures across all the artifacts intended to validate one another.
Independent verifier agents help only if they operate with genuinely independent objectives, trustworthy evaluation mechanisms, and principled protocols for resolving disagreement. Assurance therefore expands beyond verifying software artifacts to evaluating the autonomous agents and development processes that produce them.
5. Multi-agent coordination moves beyond human organizations
Most current multi-agent systems still resemble human organizations: managers delegate work, specialists perform tasks, reviewers inspect outputs, and communication happens largely through natural language. These structures reflect human cognitive limits—attention, communication bandwidth, memory, and span of control—rather than fundamental requirements of collaboration.
Autonomous agents operate under different constraints. They can fork and merge execution states, share complete context, coordinate through structured protocols, and scale to thousands or millions of concurrent collaborators. As a result, we expect fundamentally new forms of AI-native coordination to emerge, along with new coordination failures, security risks, and questions of accountability.
III. The Software Engineering Ecosystem Restructures
As software artifacts and engineering workflows change, the surrounding software ecosystem must evolve as well. Organizations, institutions, education, and markets all adapt to a world in which implementation is abundant but trustworthy autonomy becomes increasingly valuable.
6. The software engineering ecosystem restructures
As autonomous agents absorb more implementation work, software development organizations will be able to deliver and maintain increasingly complex systems with much smaller engineering teams. Human contribution shifts toward product definition, architecture, specification, evaluation, security, governance, and incident response.
At the same time, entirely new ecosystem institutions become necessary. Independent auditors, certification organizations, trustworthiness assessment providers, and governance services may become as important as today’s testing platforms or cloud providers. Software engineering education will likewise place greater emphasis on specification, verification, security, governance, and system-level judgment than on manual implementation alone.
Ten Predictions About the Future of Software Development
The structural shifts described above suggest a set of concrete, testable predictions. They are deliberately forward-looking rather than inevitable. Different domains will evolve at different speeds, but together these predictions illustrate how autonomous software development could reshape software engineering over the coming decade.
Software Artifacts
1. Specifications become the software’s genome
The durable representation of a software system will increasingly be its specification: the evolving record of requirements, constraints, architectural decisions, testing strategies, and design rationale. Code becomes one compiled realization of that intent rather than the canonical artifact itself.
2. Regeneration becomes a practical alternative to refactoring
For software that can be faithfully reconstructed from its specification, regenerating a clean implementation may become cheaper than incrementally repairing years of accumulated complexity. This shift will not apply universally. We expect systems with extensive undocumented knowledge or legacy integrations will continue to evolve incrementally. For many other applications, regeneration will fundamentally change the economics of maintenance.
3. Software sharing shifts from implementations to protocols
As custom implementation becomes inexpensive, libraries, frameworks, and SaaS platforms will increasingly serve as behavioral standards, interface contracts, and reference implementations rather than immutable dependencies. Autonomous agents will adapt, optimize, or regenerate implementations locally while preserving externally visible behavior.
4. Dynamic software becomes commonplace
Software will increasingly combine deterministic guarantees with neural flexibility, continuous adaptation, conversational interfaces, and on-demand generation. Many production systems will no longer have a single canonical implementation but will instead evolve continuously while maintaining stable external behavior.
Engineering Practice
5. Specification distillation becomes a core engineering capability
Rather than writing complete requirements documents upfront, developers will increasingly communicate through conversations, examples, reviews, and corrections. Autonomous agents will continuously distill these interactions into structured, maintainable specifications that evolve alongside the system.
6. Assurance shifts from reviewing code to certifying agents
Code review will remain important, but assurance will increasingly focus on the autonomous agents that produce software. Organizations will inspect agent specifications, permissions, memories, execution traces, provenance, evaluation results, and governance controls. Trusting the production process becomes as important as trusting the produced artifact.
7. AI-native coordination structures replace human-inspired workflows
Today’s multi-agent systems largely inherit human organizational patterns. Over time, agent teams will develop communication, delegation, consensus, and conflict-resolution mechanisms designed around machine capabilities rather than human managerial constraints, enabling collaboration at scales impractical for human teams.
8. Project context becomes the durable asset
As implementations become inexpensive to regenerate, organizations will increasingly value the persistent project context that guides software generation rather than any particular implementation. Specifications, architectural decisions, engineering conventions, constraints, evaluation history, and accumulated feedback may outlive multiple generations of code. Preserving this project context in portable, vendor-neutral formats will become essential for maintaining continuity across models, tools, and platforms.
Organizations and Ecosystems
9. Software development organizations become smaller and more governance-intensive
Implementation-heavy engineering teams may shrink dramatically, while architecture, specification, verification, security, governance, compliance, and incident response grow in relative importance. At the same time, new ecosystem institutions, including independent auditors, certification organizations, and assurance providers, will become an increasingly important part of the software lifecycle.
10. Software becomes abundant; trust becomes scarce
Autonomous development will dramatically reduce the cost of producing custom software, expanding what can be built and by whom. At the same time, the abundance of generated software will make trust increasingly scarce. Establishing provenance, verifying behavior, certifying autonomous agents, and building trustworthy governance mechanisms may become among the most valuable capabilities in the software ecosystem.
The Research Agenda, and How Far We Still Have to Go
The purpose of this roadmap is not to declare that full autonomy has arrived. It is to clarify the capabilities and safeguards that must exist before higher levels can be responsibly claimed.
Specification synthesis and behavioral understanding
We need repository-scale methods that infer, generate, and maintain machine-checkable specifications. Benchmarks should measure whether specifications remain consistent with implementations across many development cycles. Today, this evaluation infrastructure barely exists.
Trustworthy verification becomes essential
As software development becomes increasingly autonomous, agents need trustworthy ways to determine whether their outputs actually satisfy their intended specifications. Formal verification is one of the most promising approaches, but today’s techniques remain limited in both scope and scalability. Verifying large, evolving software systems remains fundamentally difficult.
Security, governance, and accountability
Autonomous software development raises fundamental research questions spanning security, governance, and accountability. New attack surfaces emerge from different aspects including agent memory, tools, external data, multi-agent coordination, and autonomous execution, requiring new approaches to secure architectures, permission systems, monitoring, provenance, and recovery.
Beyond technical security, the field must also develop foundations for trustworthy governance. How should autonomous agents be authorized, supervised, audited, and held accountable? What evidence is sufficient to establish trust? And what standards, certification processes, and liability frameworks will be needed as autonomous software becomes an increasingly important part of critical infrastructure?
Agent-centric system and language design
Much of today’s software stack was designed for humans to write, understand, and maintain code. Autonomous software development challenges this assumption. A central research question is how to redesign programming languages, development environments, memory systems, provenance models, and coordination protocols for agent-first software engineering without sacrificing human oversight, interpretability, or control.
Human-agent interfaces and economic models
As autonomous agents take on more of the software development process, fundamental questions arise about how humans and agents should collaborate. How should project knowledge, design intent, requirements, and constraints be communicated? How can humans effectively understand, supervise, and steer increasingly autonomous agents without becoming a bottleneck?
We also need new economic models for autonomous software development. Traditional productivity metrics are no longer sufficient; evaluating autonomous development must also account for inference costs, human supervision, maintenance overhead, technical debt, system reliability, and operational risk.
RESEARCH PRIORITY: Our most concrete call: build repository-scale specification-synthesis benchmarks and long-horizon evaluations of specification–implementation drift. Without them, claims of Level II or Level III readiness cannot be substantiated.
What Happens When Writing Code Is No Longer the Bottleneck?
This transition raises questions that no benchmark can answer by itself:
- What becomes the developer’s highest-leverage work when agents do most of the coding?
- Is software that ships without human code review a mature engineering practice or a risk we do not yet know how to measure?
- Do agents democratize software creation, or relocate expertise into specification, evaluation, architecture, and governance?
- Who is accountable when an autonomous agent deploys software that causes harm?
- What should remain a reserved human decision even if an agent is technically capable of making it?
We do not expect the future of software development to be a simple story of humans disappearing from the process. A durable human-AI partnership remains plausible, and adoption will be gradual and uneven. But the direction of travel is clear enough that the research community should begin studying software development as a trajectory of increasing autonomy rather than a collection of isolated coding tasks.
Continue the Discussion at the Agentic AI Summit
We will explore these questions in the “Future of Software Engineering” session at the Agentic AI Summit, August 1–2 at UC Berkeley. The session will bring together researchers and builders working on coding agents, developer tools, evaluation, security, and the emerging infrastructure for agent-driven software development.
The panel will discuss not only how capable coding agents have become, but how the discipline itself changes when implementation is no longer the dominant bottleneck and what technical, organizational, and governance foundations are needed before autonomy can be trusted.
For the full framework, taxonomy, and research agenda, read the position paper: Towards Autonomous Software Development.