A System One Model for Fast and Generalizable Decision-Making

Jacky Kwok

${}^{†}$

,

Hangoo Kang

,

Tarun Suresh

,

Jon Saad-Falcon

,

Marco Pavone

Christopher Ré

,

Azalia Mirhoseini

Stanford University

NVIDIA Research

${}^{†}$

Project Lead

Posted: Sep 23, 2026

New Architecture, Data Recipe, and Scaling Laws

We introduce Contrastive Language Models (CLMs), a new class of

System One model

trained with a

contrastive learning

objective that connects

states and actions

.

We release

CLM-8B

, which is pre-trained on

60M Nemotron Q&A pairs

, mid-trained on

30M synthetic hard negatives

, and post-trained on

1M agentic trajectories

.

We build an ultra-efficient training and serving infra for CLM by

disaggregating states and actions

, allowing their embeddings to be cached and reused independently.

We establish

scaling laws for CLMs

and show that the test contrastive loss decreases predictably as a

power law

in training

compute

,

model size

, and

dataset size

.

Try

CLM on GitHub

:

Overview

CLM first trains a

state encoder

and an

action encoder

on a large-scale dataset with a

contrastive objective

(InfoNCE), so that each state is pulled toward the ground truth action that was taken and pushed away from all others. The two encoders then serve directly as a

zero-shot action classifier

.

At deployment, given the current state and a set of candidate actions, CLM scores each action by how well its embedding aligns with the state embedding and selects the highest-scoring action.

Dino Run (CLM vs. Jev)

Super Mario Demo

WikiRacing Demo

Zero-shot Evaluation

Across

computer-use, gaming, and tool-calling tasks

, CLM-8B performs on par with Jev while running

up to 9× faster

. The speedups are most pronounced when the number of candidate actions is large (e.g., WikiRacing) or when actions can be frequently reused across states (e.g., T-Rex Game).

Agentic Benchmarks

We find that

Jev fails to serve as a verifier

for long-horizon tasks, performing below the random-selection (Pass@1) baseline. In contrast, with lightweight fine-tuning, CLM achieves

SOTA performance

on challenging agentic benchmarks, including

DeepSWE (81.6%)

and

Terminal-Bench 2.1 (87.6%)

, while delivering

4–6× faster inference than Jev

.

Click to View More Experimental Details

Model Architecture

A CLM consists of a

state encoder

and an

action encoder

, as illustrated in Figure 1. Both encoders map their respective inputs into a shared embedding space, where the score of a state–action pair is computed as the

cosine similarity

between their embeddings.

Each encoder consists of a frozen LLM backbone followed by a trainable projection head. We take the hidden state of the final token, normalize it, and pass it through a MLP projection head. Only the 20M-parameter projection head is trained; the LLM remains frozen and never receives gradients. This design makes our scaling experiments inexpensive to run. We precompute the LLM embeddings once, then reuse them to train projection heads across different setups. A full

pre-training run

on the Nemotron DQA dataset takes about

an hour

on a single RTX 4090 GPU.

Most importantly, since

states and actions are disaggregated

, their

embeddings can be cached independently

. In settings where the state evolves continuously (e.g., Super Mario) while the action set remains fixed,

we only need to recompute the state embedding at each step and can reuse the cached action embeddings

. This substantially reduces inference cost, with the efficiency gains becoming increasingly significant as the number of candidate actions and context length grows. At ~1k candidates, CLM is

13x faster

than Jev.

Click to View More Experimental Details

How does “Action Caching” work for CLM in Super Mario?

The 4 action embeddings are precomputed before gameplay begins. At each step, only the new game state passes through the state encoder. Its embedding

$z_s$

is then scored against the cached action embeddings

$z_a$

, and the highest-scoring action is executed. This reduces the cost from

5

forward passes to just 1 per step.

Training Algorithm

CLM is trained with a bidirectional InfoNCE loss. Given a batch of

$B$

matched state-action pairs, we compute a

$B\times B$

similarity matrix. For each positive pair

$(s_i,a_i)$

, we optimize retrieval in both directions:

$s_i\to a_i$

and

$a_i\to s_i$

:

$L_{CLM}=-\frac{1}{2B}\sum _i[\log \frac{\exp \left(s_i^{\top }{a}_i/\tau \right)}{\sum _j\exp \left(s_i^{\top }{a}_j/\tau \right)}+\log \frac{\exp \left(a_i^{\top }{s}_i/\tau \right)}{\sum _j\exp \left(a_i^{\top }{s}_j/\tau \right)}]$

For mid-training, we extend the bidirectional InfoNCE objective with hard negatives. Let

$h_{ik}^{\left(a\right)}$

denote a hard negative action for state

$s_i$

. For the state-to-action direction, the loss is:

$L_{s\to a}^{hard}=-\frac{1}{B}\sum _i\log \frac{\exp \left(s_i^{\top }{a}_i/\tau \right)}{\exp \left(s_i^{\top }{a}_i/\tau \right)+\sum _k\exp \left(s_i^{\top }{h}_{ik}^{\left(a\right)}/\tau \right)}.$

Scaling Laws for Verification

We find that the test InfoNCE loss

$L$

scales as a power law with training compute

$C$

, dataset size

$D$

, projection-head size

$N$

, and encoder size

$N_{enc}$

. These dimensions must be scaled jointly to achieve the optimal verification performance. When each scale factor is not bottlenecked by the others, the dependence on each variable

$X\in C,D,N,N_{enc}$

can be described as:

$L\left(X\right)\approx {\left(\frac{X_c}{X}\right)}^{{\alpha }_X},$

where

$X_c$

is a fitted scale constant and

${\alpha }_X$

is the corresponding scaling exponent following Kaplan et al. Notably, scaling the encoder size yields the strongest gains. Experiments are conducted on the Nemotron DQA dataset and evaluated on a held-out dataset.

Data vs. Optimal Model Size

We fix a compute budget

$C$

and plot the test loss against the parameter count of the projection head

$N$

, with each curve corresponding to a different data budget

$D$

. In log-parameter space, each iso-FLOP curve is well approximated by a parabola, and its minimum identifies the optimal head size for that data budget. As the budget grows, the optimum shifts steadily toward larger heads. Specifically, the optimal size grows almost exactly linearly with the number of training tokens

$N^{\ast }\propto D^{1.02}$

, at roughly

310 tokens per parameter

.

Data Recipe

CLM is trained in three stages, with each stage introducing a progressively harder form of

state–action alignment

. Pre-training learns broad semantic representations, mid-training develops fine-grained discrimination, and post-training adapts the representation space for action classification.

Stage 1 — Pre-training.

We first pre-train CLM on an internet-scale question–answer dataset containing

~60M pairs from Nemotron DQA

. We treat each question as the

state

and its answer as the corresponding

action

. This stage learns broad semantic representations from a diverse corpus.

Stage 2 — Mid-training.

We then introduce

~30M synthetic hard negatives

generated by

Gemini 2.5 Flash-Lite

. For a subset of Nemotron DQA questions, we construct semantically similar but incorrect answers. These negatives are precomputed and incorporated into the

InfoNCE loss

using Equation 2, enabling more fine-grained discrimination between plausible actions.

Stage 3 — Post-training.

Finally, we post-train CLM on

~1M agent trajectories

from the

Agent Data Protocol (ADP)

dataset, supplemented by terminal traces from

Endless-Terminals

,

LiteCoder-Terminal-SFT

. Each trajectory step is represented as a

state–action pair

, where the state contains the agent’s current context and the action corresponds to the decision taken at that step. This adapts the learned representation space for action classification in agentic environments.

Evaluating CLM after Pre-training

We first evaluate

CLM-8B immediately after pre-training.

As illustrated above, when asked

“Who wrote the play Romeo and Juliet?”

, the Qwen3-8B embeddings assign the highest probability to an incorrect answer and ranks

“William Shakespeare”

third.

In contrast, CLM-8B correctly ranks

“William Shakespeare”

first with

54.2%.

This suggests that

pre-training reshapes the model’s representations into a useful decision space.

Data Mixture for Post-Training

To preserve the general representations learned during pre-training, we use

co-training with data replay

rather than fine-tuning exclusively on agentic traces. Specifically,

40%

of the post-training mixture consists of the Nemotron DQA examples, while the remaining

60%

are agentic trajectories.

This replay substantially mitigates catastrophic forgetting. With replay, the Nemotron hard-negative top-1 accuracy decreases slightly from

69% to 68.5%

. In contrast, when training on agentic data alone for the same number of agentic steps, the accuracy drops to 56.2%.

Why Not Train on Hard Negatives from the Start?

We compare two training strategies under a fixed compute budget:

Pre-train + Mid-train:

pre-train on the full Nemotron DQA corpus with bidirectional InfoNCE, then briefly mid-train on hard negatives using Equation 2.

Hard negatives from scratch:

train on Nemotron DQA and hard negatives jointly from the beginning.

We evaluate on ~100K held-out questions, each with one gold answer and 10 hard negatives. The top-1 accuracy measures whether the gold answer ranks highest. We find that pre-training alone reaches

52.1%

without seeing any hard negatives. A short mid-training stage then boosts accuracy to

69.2%

. In contrast, training with hard negatives from the start improves quickly but peaks at

62.4%

before overfitting. The two-stage recipe achieves

7% higher accuracy

at fixed budget.

Takeaway:

Hard negatives work best as a refinement signal on top of pre-training, rather than as a substitute for it.

CLM Playground

CLM comes with a playground on

GitHub

. Write a state, add typed questions, and see CLM's full probability distribution over the answers in milliseconds.

Join us!

We call on the community to join us in this effort, either by providing your feedback or contributing to the project!

Please don’t hesitate to get in touch:

Github Repo:

https://github.com/Contrastive-LM/CLM

Join Discord:

https://discord.gg/5dAQEDJBs

Contact:

jackykwok@stanford.edu

Conclusion

CLM opens up a new direction for scalable, reliable, and fast verification. Moving forward, we plan to explore several key directions:

Scaling Experiments:

Extend our scaling-law experiments to substantially larger backbones and study how verification performance can be further scaled

Vision and multimodal support:

Extend CLM to include images, video, and other modalities for robotics and computer-use tasks.

Scaling the data recipe:

Expand pre-training, hard-negative mining, and agentic post-training.

…and more to come.

CLM-8B

is part of our scaling ladder, where we train models across multiple scales to establish scaling laws and predict performance at larger scales. A

multimodal CLM-35B

is now in training with more data, compute, and parameters. Stay tuned for the release early next month ​

Citation

If you find CLM useful, please consider citing it:

@misc{kwok2026contrastivelanguagemodels, title={Contrastive Language Models: A System One Model for Fast and Generalizable Decision-Making}, author={Jacky Kwok and Hangoo Kang and Tarun Suresh and Jon Saad-Falcon and Marco Pavone and Christopher Ré and Azalia Mirhoseini}, year={2026}, note={Notion Blog} }

​