你的笔记本电脑现在能运行一个能力几乎媲美云端任何模型的模型。我把 Qwen3.8-27B 换进了我的智能体，它表现得极为出色。这只鸟的飞行方式与飞机不同。
这个小巧的 Qwen 模型在 135 个模型中排名第 1，在 Artificial Analysis 的 Intelligence Index 上得分 52，比 GLM-5.2 高出一分——后者是来自 Z.ai 的 SOTA 开源模型，拥有 753b 参数。1一个笔记本上的模型击败了一个可辨识的、前沿级别的云端同类，而后者规模约为它的 28 倍。
大黄蜂是如何实现与客机相同的飞行的？更大的模型能存储更多知识，因此它们可以直接跳到答案，就像许多不同领域的专家一样。较小的模型记忆的内容没那么多，因此它们必须更多地推理，几乎是从第一性原理出发，才能弥合这一差距。2
我在将 DeepSeek V4 云端模型与两个本地模型进行基准测试时亲眼看到了这一点。我在相同的工作上对它们进行了比较，共 25 项风险投资任务（研究初创公司、总结文章、转录播客），由评判模型打分。3
Qwen3.8-27B 是稠密模型：它在每个问题上都会用到书中的每一章。略读型模型 DeepSeek 和 Qwen 3.6 35b（我投入测试的另一个本地模型），则只翻到与问题相关的章节。4
| 模型 | 质量 /9 | tok/s | 平均 tokens | 平均延迟 |
|---|---|---|---|---|
| deepseek-v4-flash（plane） | 8.0 | 137.3 | 159 | 1.1s |
| qwen3.8-27b（bumblebee） | 8.0 | 51.9 | 369 | 7.2s |
| qwen3.6-35b-a3b（hummingbird） | 7.9 | 113.4 | 1,143 | 10.0s |
这些模型给出的答案同样出色。但速度各不相同。本地的 Qwen 35b 以最高速度飞驰，但需要比云端模型多思考约 7.2 倍，比 DeepSeek 晚 9 秒冲过终点线。最新的 Qwen 模型快了 3 秒，而云端模型还要再快 6 秒。
云端模型直接跳到正确答案；本地模型则以不同的速度和准确率在内部沉思与辩论。
例如：在一项分诊任务中，35B 花了 993 个 token 才产出六个词，“Classification: Scheduling / Action: Respond.” 在给出回答前进行 1000 个 token 的深思熟虑，就像蜂鸟冲向金银花的一次疾飞。而大黄蜂需要 369 个思考 token，以一半的速度嗡嗡前行。
本地模型可以达到与云端模型相同的结果，但它们会沿着不同的飞行路径抵达那里。
-
Artificial Analysis 在此将现任者 Qwen3.8-27B 排在 Intelligence Index 上 135 个模型中的第 1 名，得分 52，比 GLM-5.2 的 51 分高出一分，后者是 Z.ai 两个月前发布的 753b 参数前沿模型。同一页面将其排在每任务输出 token 数上 135 个模型中的第 23 名，为 160M 加权 token，而同类中位数为 43M。智能排名与冗长程度排名各自独立变动，而这正是整篇文章所讨论的取舍。↩︎
-
模仿差距的解释。较小的模型会生成流畅的思维链，但这种思维链更可能在逻辑上变得不一致，因为一旦被迫偏离通往答案的直接路径，它们可用来参照的邻近正确样本图谱更为稀疏。参见 Chain of Thought in Large Language Models: Elicited Reasoning or Constrained Imitation?。我在 When Models Learn 中讨论了这种权衡的一般形态，即用推理时计算换取能力。↩︎
-
方法。25 个风险投资任务（调研初创公司、总结文章、转录播客），取自于我自己的智能体队列。一个独立的评判模型 deepseek-v4-pro 对输出进行盲评，从完整性、准确性和简洁性三个维度打分，每项 3 分，总计 9 分。每次运行的 max_tokens 均为 4096。两个本地模型都通过 Ollama 在相同的 MLX 运行时上提供服务，因此这一比较不会因运行时差异而受到干扰。我通过对相同输出重新评分来确定评判模型的噪声下限，结果平均绝对差为 0.16。↩︎
-
Qwen3.6-35B-A3B 是一个 35b 参数模型，每个 token 有 3b 激活参数，采用稀疏混合专家架构，共 256 个专家，每个 token 激活 8 个路由专家和 1 个共享专家。依据 Qwen 在 Hugging Face 上的模型卡和 vLLM 的模型配方。↩︎
Your laptop can now run a model as capable as nearly anything in the cloud. I swapped Qwen3.8-27B into my agent & it works brilliantly. This bird flies differently than a plane.
This little Qwen model ranks #1 of 135 models, scoring 52 on Artificial Analysis’s Intelligence Index, a point above GLM-5.2, the state-of-the-art open-source model from Z.ai, at 753b parameters.1 A laptop model beats a recognizable, frontier-class cloud peer roughly 28 times its size.
How does a bumblebee achieve the same flight as an airliner? Bigger models can store more knowledge, so they can skip straight to an answer, like an expert in many different fields. Smaller models don’t have as much memorized, so they must reason more, almost from first principles, to close that gap.2
I saw this firsthand when benchmarking the DeepSeek V4 cloud model against two local models. I compared them on the same work, 25 venture-capital tasks (researching startups, summarizing articles, transcribing podcasts), scored by a judge model.3
Qwen3.8-27B is dense : it uses every chapter in the book on every question. Book skimmers DeepSeek & Qwen 3.6 35b (another local model I threw into the test), flips only to the relevant chapters for a question.4
| model | quality /9 | tok/s | avg tokens | avg latency |
|---|---|---|---|---|
| deepseek-v4-flash (plane) | 8.0 | 137.3 | 159 | 1.1s |
| qwen3.8-27b (bumblebee) | 8.0 | 51.9 | 369 | 7.2s |
| qwen3.6-35b-a3b (hummingbird) | 7.9 | 113.4 | 1,143 | 10.0s |
These models provide identically good answers. But the speed varies. The local Qwen 35b shreds at top speed, but needs to think about 7.2x more than the cloud model, crossing the line 9 seconds after DeepSeek. The newest Qwen model is three seconds faster, & the cloud is 6 seconds faster yet.
The cloud model jumps to the right answer ; the local models contemplate & debate internally at different rates of speed & accuracy.
For example : on one triage task, the 35B spent 993 tokens to produce six words, “Classification: Scheduling / Action: Respond.” 1000 tokens of deliberation before the response is a hummingbird’s sprint to a honeysuckle. The bumblebee needed 369 thinking tokens, buzzing along at half the speed.
Local models can achieve the same result as cloud models, but they’ll take a different flight path to get there.
-
Artificial Analysis ranks the incumbent here, Qwen3.8-27B, #1 of 135 models on the Intelligence Index, scoring 52, a point above GLM-5.2’s 51, a 753b-parameter frontier model Z.ai shipped two months earlier. The same page ranks it #23 of 135 on output tokens per task, 160M weighted tokens against a class median of 43M. Intelligence rank & verbosity rank move independently, & that’s the trade this whole post is about. ↩︎
-
The imitation-gap explanation. Smaller models produce fluent chain-of-thought that’s more likely to drift logically inconsistent, because they have a sparser map of nearby correct examples to draw on once forced off the direct path to an answer. See Chain of Thought in Large Language Models: Elicited Reasoning or Constrained Imitation? I covered the general shape of this tradeoff, trading inference-time compute for capability, in When Models Learn. ↩︎
-
Method. 25 venture-capital tasks (researching startups, summarizing articles, transcribing podcasts) drawn from my own agent queue. A separate judge model, deepseek-v4-pro, scored outputs blind on completeness, accuracy & conciseness, 3 points each for 9 total. max_tokens was 4096 for every run. Both local models were served through Ollama on the same MLX runtime, so the comparison isn’t confounded by runtime differences. I established the judge’s noise floor by re-scoring identical outputs, which returned a mean absolute difference of 0.16. ↩︎
-
Qwen3.6-35B-A3B is a 35b parameter model with 3b active parameters per token, a sparse mixture-of-experts architecture, 256 total experts with 8 routed & 1 shared active per token. Per Qwen’s model card on Hugging Face & vLLM’s model recipe. ↩︎