Stanford + MIT paper on Model Harnesses shows that AI performance depends not just on the model itself, but on the surrounding system code — the “harness”.

This is what decides what to store, retrieve, show to the model, and how the workflow runs. With the same underlying LLM, changing the harness can create up to a 6× performance gap on the same benchmark.

They conclude the harness around a model matters as much as the model itself.

The paper introduces Meta-Harness, an outer-loop system that automatically improves harness code. Instead of giving the optimizing agent only a score or a short summary of past attempts, it gives the agent rich access to prior code, logs, and execution traces through a filesystem-like setup.

The idea is that better diagnostic visibility lets the system improve the harness more intelligently.

What it found is pretty significant.

• On online text classification, a 7.7-point improvement over a strong SOTA context management, while using 4X fewer context tokens.

• On retrieval-augmented math reasoning, an average gain of 4.7 points across five held-out models on 200 IMO-level problems. On agentic coding, the discovered harnesses beat strong hand-engineered baselines on TerminalBench-2.

The paper shifts attention from “which model is best?” to “how is the whole AI system designed?”

For real deployments, harness design affects reliability, tool usage, context management, and failure recovery.

----

Paper – arxiv. org/abs/2603.28052

Paper Title: "Meta-Harness: End-to-End Optimization of Model Harnesses"