# 九套编程线束在 MacBook 上实测 Qwen 3.8 27B：prefill 与上下文占用差异悬殊

- 来源：Hacker News 热门（buzzing.cc 中文翻译）
- 作者：nasutton12
- 发布时间：2026-09-11 18:09
- AIHOT 分数：67
- AIHOT 链接：https://aihot.news/items/cmtwtclzy0b30row723l6jfx7
- 原文链接：https://nasutton.notion.site/Nine-coding-harnesses-vs-your-laptop-3d139990182b80d59fa3cf500f0450ba?pvs=74

## AI 摘要

作者用同一台 M4 MacBook Pro（24GB）和 llama.cpp 服务 Qwen 3.8 27B，对 9 套编程线束跑 8 道 Exercism 题目实测。

## 正文

Nine coding harnesses vs. your laptop

If you have ever considered swapping out the API calls your coding harness is making for a local model it probably ended in disappointment. You ran a quick llama-bench and thought you were going to see X tokens/second. What that benchmark doesn’t tell you is how it actually feels to use these harnesses. The development experience is extremely variable. You might have sat there for minutes before seeing any response. Maybe you were getting somewhere and halfway through it started stalling. ‘Who is this prefill, taking all my time?’ It’s not your fault: most coding harnesses weren’t built with a local model in mind.

This is not a hit job on the excellent work of other teams. I’m trying to reliably measure what happens when you swap out the data center for localhost. A certain level of wrong tool for the problem is to be expected. Please add salt: I have been tinkering with chad, a coding harness optimized specifically for Qwen 3.8 27B on Apple silicon.

Laptop Physics

Most harnesses conspire against localhost in three ways.

Large system prompts & tool schemas. Let’s assume your laptop reads at 90 tokens per second and writes about 10. These represent the prefill & generation parts of every loop in your coding harness. At these speeds every 1,000 tokens of reading a prompt translates to ~ 11 seconds of staring at the cursor before the model begins to write. Before your LLM does any work it will read the entire system prompt along with any loaded tool schemas. In the pi harness this combination was 2,008 tokens for Qwen 3.8 27B, but 18,046 all in for Opencode. You won’t notice much difference when you have a data center GPU with prefill rates averaging 10k+ tokens/second. This collapses to 0.2 versus 1.8 seconds. On your laptop? That is the difference between 22 and 226 seconds, measured. Unbearable!

Smaller context windows. After your LLM finishes the system prompt you have a finite context window left in memory to do work. It is smaller than you think, and your harness just spent part of it. How much context you have depends on how much memory you are starting with vs. how big the model’s weights are. There are a lot of variables here, but 32,000 tokens is a reasonable guess at how much room you’ll have left with a reasonably good model on a reasonably good laptop. How much of that 32,000 token budget is left for the pi harness? 94% - seems manageable. Opencode? With 18,046 tokens out the door already only 44% of your context is left for actually doing work. A habit of side requests. In the traditional local client / remote server pattern the harness can make as many side requests to the data center servers as it likes. Your laptop is both the client and the server. In the best case those side requests cause the local model to queue and wait. In the worst case they cause repeated long prefills. Over 24 tasks opencode fired 33 of them, crush 51 and dsh 24 (session titles and summaries), almost every one overlapping an agent turn. The model was “busy” 125% and 114% of wall clock for opencode and crush: two requests in flight on one GPU.

Harness Outcomes

I put 9 harnesses through a series of 8 Exercism exercises, each in its own auto-approve mode, with one identical one-sentence prompt. Each task leveraged the same M4 MacBook Pro (24GB, macOS 26.6.2) with a 3 bit quant of Qwen 3.8 27B model served via llama.cpp (build 10470). The same llama-server was shared by every harness and a common proxy enforced the same recommended sampling regime for Qwen (temperature=1.0, top_k=20, top_p=0.95, min_p=0.05). Every session had the same 32,768 unified cache served across four slots. Every number reported below is llama-server's own accounting as read through the proxy, never a harness self-report, except the two rows marked * (chad on its in-process MLX engine, where there is no server to observe, so they come from chad's own prefill trace with the same definitions).

wait before 1st token is the server's prefill time for the system prompt, tool schemas, and the first request.

wait / later turn is the same pause on every turn after the first, side requests excluded.

cache reuse is the share of each later turn served from the prefix cache.

experienced tokens/second is generated tokens over wall clock for the whole task, prefill and tool time included; it spreads up to 50% between nights, so nothing between the lean arms is a finding.

pass is a gate for the Exercism task. Please do not interpret as a ranking, as these are trivial python tasks. T counts timeouts at the 1,200 second cap.

harness version tools tax: turn-1 prompt (tok) wait before 1st token, turn 1 wait / later turn (med · p90) cache reuse exp. tok/s pass (gate)

mini-swe-agent 2.4.6 1 1,171 12.2 s 3.6 s · 21 s 96% 8.0 11/24 (14 T)

pi 0.80.3 4 2,008 21.6 s 1.3 s · 22 s 99% 8.1 19/24 (7 T)

cline 3.0.60–61 26 5,876 64.1 s 9.9 s · 52 s 94% 7.3 17/24

codex 0.151.0 10 7,804 87.8 s 9.6 s · 28 s 94% 6.9 19/24 (5 T)

dsh 0.1.1-rc.2 25 8,052 94.4 s 2.2 s · 34 s 99% 7.2 18/24

goose 1.50.0 18 9,617 110.3 s 1.0 s · 22 s 100% 8.0 22/24 (3 T)

crush 0.92.0 26 16,263 199.8 s 1.8 s · 40 s 100% 5.8 18/24 (8 T)

opencode 1.17.12 10 18,046 225.7 s 4.6 s · 44 s 99% 5.7 15/24 (13 T)

chad (llama.cpp) 2.0.3 5 2,563 25.6 s 0.8 s · 19 s 99% 7.9 24/24

chad (MLX, serial) * 2.0.3 5 2,566 4.7 s 1.0 s · 19 s 99% 12.4 21/24 (3 T)

chad (MLX, dflash2) * 2.0.3 5 2,562 4.6 s 0.9 s · 36 s 99% 17.4 22/24 (3 T)

In my experience the harnesses fell into three groups.

Lean and stable (pi, mini-swe-agent, chad). These harnesses have a trim system prompt and 96–99% cache reuse. They largely work out of the box with either a data center or local model backend. The only caveat here is that mini-swe-agent only passed 11/24 of the Exercism tasks and had the most timeouts.

Heavy but disciplined (dsh, cline, codex, goose). The combined effect of either long system prompts or a high tool schema count. Their prefix is byte stable so once you get started it feels tolerable. (goose earned this group at 1.50.0: earlier releases re-rendered a minute-resolution timestamp into the first user message every turn. This tanked cache reuse to 78%)

Heavy to start (crush, opencode). You’ll wait 3-4 minutes before you see any activity.

The case for chad

None of these harnesses are badly engineered. Opencode’s 18k system prompt is there because it helps frontier models deployed behind an API. The 26 tool schemas in crush are fine when you have 200k context and prefill is instantaneous. All of these choices were made in an environment where the prefill is virtually free. At local speeds this all falls apart.

chad is minimal on purpose. There are only 5 tools available, all shapes the model already learned during pretraining. A disk check-pointed system prompt, persistent prefix cache, and a fitted drafter are all extensions of this scarcity mindset. This discipline alone doesn’t explain the gap you’d actually feel. With the standard client/server model the chad/llama.cpp interface isn’t even the fastest of the lean three. You could probably write most of the loop in an afternoon.

chad’s real number shows when you have the liberty to couple the server and the agent loop in the same process. The harness now owns the cache. With the same harness and same prompts, the in-process MLX engine experienced 7.9 → 17.4 tokens/second (or 12.4 tokens/second with the drafter off). The only advantage of the MLX process was decode and not chad’s request pattern: the wait per later turn is the same one second on either engine. Paired against its own control inside generation over three more nights, the DFlash2 drafter reads 23.3 vs. 15.9 tokens/second, 1.47x pooled and a win on every one of the eight tasks.

Reproduce it

Everything runs from a checkout of chad: the runner, forcing proxy, and scorecard for the eight tasks are under benchmarks/matrix. Each arms raw rows (

grid.json

, one

turns.jsonl

row per request, the sampler audit, provenance) are committed beside the pooled tables with the intention that so every number above traces to a row.

uv run python benchmarks/matrix/run.py setup|smoke|llama|mlx|table

​

Thanks to the llama.cpp maintainers for a server-side prefix cache and a timings object good enough to build the whole instrument on, and to z-lab for DFlash2.