Pay attention to this new wave of System One models if you are building custom harnesses.
First Jev. Now, Contrastive Language Model (CLM).
CLM is 9x faster than Jev.
CLM seems to be a better verifier than Jev, particularly at long-horizon tasks.
How do Jev and CLM differ?
CLM is contrastive, and Jev is trained with Reinforcement Learning for Calibrated Decisions (RLCD).
Jev receives a situation plus predefined questions, and returns typed decisions with probabilities. CLM embeds the situation and candidate actions, compares their similarity, then ranks or selects the best match.
The point is that there are several ways to attack this problem, which is exciting.
You can see my recent guide on combining System One and System Two models for building custom harnesses. https://academy.dair.ai/resources/jev-decisions-in-a-pi-sdk-harness