This is brilliant. Recommended reading.

It's a great example of how you could potentially build more effective harnesses using System One and System Two models.

It explores Jev's potential as a fuzzy linter. Before you dismiss it, check out the setup.

I like this post because it provides a few details of a good setup for testing these ideas. It scopes the problem well, like filtering out rules that are easy to judge (i.e., don't require extra context/resoning).

That's important because System One models like Jev aren't built for reasoning-heavy tasks. But it doesn't mean that they aren't more deterministic operations where Jev could help scale efficiency and performance.

You just have to look closer at the harness.

In the harness, a System Two model typically controls all the components and decisions that you could potentially offload to a System One model.

It's aligned with what ideas I shared here: https://academy.dair.ai/resources/jev-decisions-in-a-pi-sdk-harness

It's becoming extremely obvious that you don't need frontier models for everything.

You are likely paying a premium for something you don't need. I really hope we get more evals and benchmarks to measure these things because there is a thread worth pulling here.