微软新论文:技能蒸馏让非推理模型追平推理模式

DAIR.AI · @dair_ai · X·2026-08-12 03:46·42天前
AI 导读

微软新论文提出用技能蒸馏摊薄推理成本:编码智能体从训练轨迹中提取简洁自然语言技能,注入非推理模型的系统提示词。

DAIR.AI@dair_ai
36AI 编辑部评分,满分 100

微软新论文:技能蒸馏让非推理模型追平推理模式

2026-08-12 03:46· 42天前
AI 导读

微软新论文提出用技能蒸馏摊薄推理成本:编码智能体从训练轨迹中提取简洁自然语言技能,注入非推理模型的系统提示词。

Impressive new paper from Microsoft.

(bookmark it)

Reasoning modes beat non-reasoning modes on multi-step agentic tasks, and they charge a 3 to 6x premium in output tokens on every single episode. Much of that spend goes into re-deriving procedures the model already worked out on earlier episodes in the same domain.

This work amortizes that cost.

A coding agent reads a small corpus of existing trajectories from a training split, compiles a compact natural-language skill, and injects it into the non-reasoning model's system prompt.

Across ALFWorld, tau-squared-bench telecom and retail, and SpreadsheetBench-Verified, skills recover 55% to over 100% of the reasoning gap for GPT-5.4-mini on held-out tasks. On two of the four benchmarks the skill-equipped non-reasoning model beats reasoning mode outright, while emitting 2.7 to 6x fewer output tokens and zero reasoning tokens.

Reasoning traces turn out to be optional. Skills distilled from non-reasoning trajectories alone stay competitive with skills distilled from paired corpora.

Paper: https://arxiv.org/abs/2608.07885

Track more trending AI papers in our academy: https://academy.dair.ai/