跳到正文
Sarvam AI·· 1 天前AI 评分58

Sarvam AI 发布决策模型 Sarvam Prism 并开放权重

Introducing Sarvam Prism

AI 导读

Sarvam AI 发布基于 Sarvam 105B 的 System One 决策模型 Sarvam Prism,在自测的 25 个决策模型中以 71.8 分、44 ms 响应(B200 GPU)成为 100 ms 内得分最高的模型。

正文 · 原文
  1. Blog
  2. Research

Introducing Sarvam Prism

Introducing Sarvam Prism

Nov 3, 2026·8 min read

Today we are releasing Sarvam Prism, a System One decision model built on Sarvam 105B.

Typeface created this category last month and named their Jev model after Kahneman’s System One thinking, the fast and intuitive mode of judgement that handles most of what people decide in a day. Much of work in an enterprise is a sequence of such decisions, and well calibrated decision models fit neatly into these workflows.

Best-in-class accuracy & speed: Prism sets a new point on the Pareto frontier of accuracy and speed among the 25 decision models we tested. The model scores 71.8 while responding in just 44 ms on B200 GPU, the highest-scoring model under 100 ms across the 25 models we tested, averaged over three indices. We believe System One models improve with scale and benefit from mixture-of-expert architecture trade-offs, hence Prism is trained on the Sarvam 105B-10B base.

Calibration enterprises can trust: For an enterprise to automate a workflow, it has to trust the model’s confidence on its output. Prism’s errors is always within its confidence bounds and its answer is not affected by the order in which options are presented.

Make it your own: Every enterprise has its own way of making decisions, shaped by its data and outcomes of those decisions. We are releasing Prism with open weights so that enterprises can train this calibrated base on their own data and run it on their own infrastructure.

Accurate and fast

We ran Prism and 24 other decision models through three indices ourselves, on the same benchmarks. We timed every model we could run on the same hardware (B200) using the bundled inference code.

Accurate and fast

Average score across three indices against median response time, 25 decision models

  • Sarvam Prism
  • Timed by us
  • API-only (reported time)
  • Best score at each response time

Average score ↑

58

62

66

70

74

78

0

50

100

150

200

250

300

Pareto frontier

Median response time per request (ms) →

Sarvam Prism: 71.8 at 44 ms

Average score across three indices and median response time, by model * Publicly reported response time; these models are API-only.
ModelAverageJevArtificial AnalysisIndian languagesResponse time (ms)
mammoth-175.275.370.280.1284
GLiDE 1.273.272.068.179.4116
Sarvam Prism71.867.065.982.544
elephant-171.372.267.674.0145
StartLux-27B70.872.961.578.1117
KnowLine-35B70.870.759.282.650
d3-ultra70.371.662.876.4229
KnowLine-26B69.168.257.681.463
AutoTrust-JEV-27B68.567.360.777.672
OneJev-27B68.469.658.377.227
H2O-Lightning-31B68.267.959.976.746
Perplexity Decider v1.268.067.761.275.0117
d366.965.560.375.0115
Torchcast-27B66.565.159.674.832
StartLux-9B66.270.058.869.860
VisionJEV65.366.457.372.3117
Jev 1.1464.560.761.970.8196*
Amos 1 large63.464.955.569.7121
StartLux-4B63.367.858.463.760
KnowLine-4B63.166.857.065.643
OneJev-9B62.865.353.469.610
Clef62.661.754.871.2222
Microsoft-Decision-161.157.855.670.085*
Perplexity Decider Flash60.360.158.462.462
Clef-Flash58.156.252.166.0162

Up and to the left is better. Response time is the median per request on one NVIDIA B200 GPU.

Scores on each index

Each index and their average, with median response time (higher is better)

  • Sarvam Prism (44 ms)
  • Perplexity Decider v1.2 (117 ms)
  • Jev 1.14 (196 ms*)
  • Clef (222 ms)
  • Microsoft-Decision-1 (85 ms*)

Jev Decision IndexSarvam Prism

67.0

Perplexity Decider v1.2

67.7

Jev 1.14

60.7

Clef

61.7

Microsoft-Decision-1

57.8

Artificial Analysis (ours)Sarvam Prism

65.9

Perplexity Decider v1.2

61.2

Jev 1.14

61.9

Clef

54.8

Microsoft-Decision-1

55.6

Indian languagesSarvam Prism

82.5

Perplexity Decider v1.2

75.0

Jev 1.14

70.8

Clef

71.2

Microsoft-Decision-1

70.0

AverageSarvam Prism

71.8

Perplexity Decider v1.2

68.0

Jev 1.14

64.5

Clef

62.6

Microsoft-Decision-1

61.1

405060708090

Score, axis starts at 40

Scores on each index and their average, by model
IndexSarvam PrismPerplexity Decider v1.2Jev 1.14ClefMicrosoft-Decision-1
Jev Decision Index67.067.760.761.757.8
Artificial Analysis (ours)65.961.261.954.855.6
Indian languages82.575.070.871.270.0
Average71.868.064.562.661.1

* Publicly reported response time; these models are API-only.

The score averages three indices. The first is the Jev Decision Index, 37 public benchmarks. The second is our recreation of the Artificial Analysis Decision Index, where we replaced its private knowledge test with curated hard questions that include an “I don’t know” option. The third is an internal Indian-language benchmark. Response time is the median over 1,000 requests sent one at a time, with each model running its developers’ own inference code. Jev 1.14 and Microsoft-Decision-1 are available only through an API, so we plot their reported times (196 ms and 85 ms). Measured from our cluster, they took 431 ms and 533 ms respectively.

Prism has the highest average score of any model that answers in under 100 ms. A few models score higher overall, but they take between two and six times as long to respond. Prism's strongest results are on the hard questions of our Artificial Analysis recreation, which reward a model for saying "I don't know" rather than guessing due to its calibrated decisions. On Indian languages it performs on par or better with the best models we tested.

Confidence you can trust

An enterprise automates a decision by acting on it only above a confidence threshold. This is safe only if the confidence output is calibrated which means its error rate should be lower than the model’s expected error rate. We evaluated calibration on public benchmarks from the Jev Decision Index and our internal Indian-language benchmark.

Confidence you can trust

Expected vs actual mistakes on answers given with at least 90% confidence (lower is better)

  • Mistakes the model expected
  • Actual: Sarvam Prism
  • Actual: no more than expected
  • Actual: more than expected (×N)

English

Sarvam Prism

3.3

2.2

Clef

2.8

2.3

Perplexity Decider

1.4

2.0×1.5

Microsoft-Decision-1

1.7

3.1×1.8

Jev 1.14

1.3

3.5×2.7

0%1%2%3%4%5%

Mistakes (% of confident answers)

Indian languages

Sarvam Prism

5.4

4.4

Clef

3.9

15.3×3.9

Perplexity Decider

3.0

8.1×2.7

Microsoft-Decision-1

3.0

10.1×3.3

Jev 1.14

3.4

16.1×4.8

0%5%10%15%20%

Mistakes (% of confident answers)

Expected vs actual mistakes at 90% confidence or more
ModelEnglish: expectedEnglish: actualIndian: expectedIndian: actual
Sarvam Prism3.3%2.2%5.4%4.4%
Clef2.8%2.3%3.9%15.3%
Perplexity Decider1.4%2.0%3.0%8.1%
Microsoft-Decision-11.7%3.1%3.0%10.1%
Jev 1.141.3%3.5%3.4%16.1%

English: 37 public benchmarks, 226,088 questions. Indian languages: the same benchmarks in 10 Indian languages and Hinglish, 20,108 questions.

Prism makes fewer errors than its confidence scores predict, across every threshold we tested from 50% to 95%, in English and Indian languages. For English answers at 90% confidence or higher, Prism predicts 3.3% errors and makes 2.2%. Clef is close, predicting 2.8% and making 2.3%. Perplexity Decider, Microsoft-Decision-1 and Jev 1.14 underestimate their errors by factors of 1.5, 1.8 and 2.7, respectively.

In Indian languages, Prism predicts 5.4% errors and makes 4.4%. The other models make 2.7 to 4.8 times as many errors as their confidence scores predict.

Same answer, any order

Changing the order of options can change a model’s answer. We tested whether this happened with Prism across 2,000 questions. We ran the questions four times with shuffled options and four times without shuffling. Prism’s answers stayed the same across every shuffle, with no extra computation or loss in accuracy.

Same answer, any order

Share of 2,000 questions whose answer changes (lower is better)

  • Options shuffled into four orders
  • Same request re-sent four times
  • Sarvam Prism

Sarvam Prism, fixed order

0.0%

0.0%

Clef

0.1%

0.0%

Sarvam Prism, options as sent

4.0%

0.0%

Microsoft-Decision-1

4.8%

4.8%

mammoth-1

8.8%

0.0%

GLiDE 1.2

8.9%

0.0%

Perplexity Decider v1.2

10.3%

0.0%

Jev 1.14

12.3%

5.6%

0%5%10%15%

Questions whose answer changes (%)

Share of questions whose answer changes, by model
ModelShuffled95% intervalRe-sent
Sarvam Prism, fixed order0.0%0 to 0.2%0.0%
Clef0.1%0 to 0.4%0.0%
Sarvam Prism, options as sent4.0%3.3 to 5%0.0%
Microsoft-Decision-14.8%3.9 to 5.8%4.8%
mammoth-18.8%7.6 to 10.1%0.0%
GLiDE 1.28.9%7.7 to 10.2%0.0%
Perplexity Decider v1.210.3%9 to 11.7%0.0%
Jev 1.1412.3%11 to 13.9%5.6%

Perplexity Decider, GLiDE 1.2 and mammoth-1 give the same answer to the same request, but they change 9-10% of answers when the options are shuffled. Jev 1.14 changes 5.6% of answers when nothing changes and 12.3% when shuffled. Prism, reading options in the order sent, gives the same answer to the same request and changes 4.0% when shuffled. Prism closes that gap by putting the options into one fixed order before scoring, whatever order they arrive in, and returning each probability under the original option name. With that step, it changes 0 of 2,000 answers.

We’ve tested Prism on real-world use-cases like speech to speech harness, trace evaluations, prompt injections guardrails, intent routing for customer support, model routing, and document intelligence.

We’re sharing some early benchmarks, and will be releasing more over time.

Agent trace evals

Agent work needs checking, both for whether the task succeeded and for whether the run needs a person to review it. We used three sets of recorded runs. AgentRewardBench contains web-agent runs on AssistantBench, VisualWebArena, WebArena and WorkArena, labelled by experts. τ-bench contains customer-service conversations, labelled by whether the task was completed. Who&When and TRAIL contain failed runs, labelled with the step and the agent responsible. All five models were scored on the same 700 runs (1,770 questions).

Each answer is checked against a known correct label, never another AI model's judgement. For whether a task succeeded or needs review, the labels come from human experts who reviewed each web-agent run (AgentRewardBench) or from the environment's own automatic check of the final result (τ-bench). For which agent caused a failure, human annotators marked the responsible agent in each failed run (Who&When, TRAIL).

Judging agent runs

Accuracy judging 700 recorded agent runs (higher is better)

  • Sarvam Prism
  • Clef
  • Jev 1.14
  • Microsoft-Decision-1
  • Claude Haiku 5.5

Did the task succeed?Sarvam Prism

75.0%

Clef

71.6%

Jev 1.14

71.8%

Microsoft-Decision-1

71.3%

Claude Haiku 5.5

66.0%

Does the run need human review?Sarvam Prism

86.4%

Clef

80.4%

Jev 1.14

81.9%

Microsoft-Decision-1

83.4%

Claude Haiku 5.5

77.9%

Which agent caused the failure?Sarvam Prism

69.6%

Clef

60.9%

Jev 1.14

65.2%

Microsoft-Decision-1

45.7%

Claude Haiku 5.5

37.0%

All questionsSarvam Prism

78.5%

Clef

76.4%

Jev 1.14

76.4%

Microsoft-Decision-1

75.7%

Claude Haiku 5.5

70.7%

0%25%50%75%100%

Accuracy (%)

Agent-run judging accuracy, by model
QuestionSarvam PrismClefJev 1.14Microsoft-Decision-1Claude Haiku 5.5
Did the task succeed?75.0%71.6%71.8%71.3%66.0%
Does the run need human review?86.4%80.4%81.9%83.4%77.9%
Which agent caused the failure?69.6%60.9%65.2%45.7%37.0%
All questions78.5%76.4%76.4%75.7%70.7%

All questions also includes repeated loops, side effects, the step where a run first went wrong, and the type of error.

Prism scores a full agent run in about half a second on a single GPU. It reads the trace once and scores all five questions together. Claude Haiku 5.5, called through its API, takes a median 2.5 seconds per run and costs $0.64 per 1,000 runs. That makes Prism faster and cheaper alternative. Prism can also handle up to 131,072 tokens per question, so long traces can be evaluated in one pass rather than split into chunks.

Agent security guardrails

This test uses an internal benchmark of 5,517 questions across nine tasks, built from vulnerabilities published since July 2026. Models are asked to classify whether the artifact is part of a phishing campaign, if an incoming prompt is trying to jailbreak and classifying attack types into MITRE frameworks.

We will expand this eval set and release it publicly in the coming weeks.

Security decisions

Accuracy on our internal cybersecurity benchmark, 5,517 questions (higher is better)

Sarvam Prism

89.0%

Clef

84.5%

Jev 1.14

78.6%

Microsoft-Decision-1

76.1%

50%60%70%80%90%100%

Accuracy (%), axis starts at 50%

Accuracy on the internal cybersecurity benchmark, by model
ModelAccuracy
Sarvam Prism89.0%
Clef84.5%
Jev 1.1478.6%
Microsoft-Decision-176.1%

Prism scores 89.0%, ahead of Clef (84.5%), Jev 1.14 (78.6%) and Microsoft-Decision-1 (76.1%).

How it works

Prism is a LoRA fine-tune of Sarvam 105B. It takes the state, question and options as input and processes them in a single forward pass. A small scoring head returns a probability for each option, with no token-by-token generation.

Training objective

Prism learns a probability distribution over the options through two equally weighted loss terms. The first is cross-entropy with 5% label smoothing, which discourages complete certainty. The second is a Brier loss, which measures squared probability error and makes confident mistakes costly. We attribute Prism’s calibration to these two training choices.

The attention mask

The state and question form one block that is read in both directions. Each option sees that block and itself but never the other options, and every option starts from the same position. Only the answer token sees everything, and that is where the options are compared. Isolating the options makes each judgement independent. Reading both directions within a block adds about 3 points on the Jev Decision Index.

Attention Visibility Across Tokens

Three attention masks for one question with two options

  • State
  • Question
  • Option A
  • Option B
  • Answer
  • Blocked

Standard: left to right

SQABAns

SQABAns

Block, left to right inside

SQABAns

SQABAns

Prism: block, both directions

SQABAns

SQABAns

Rows: the word doing the reading. Columns: the words it can see, coloured by what they are. S = state, Q = question, A and B = two options, Ans = answer token.

Middle to right: about +3 points on the Jev Decision Index.

The state and question block is fully bidirectional, so a standard unmasked kernel computes it exactly. Only the thin strip of option and answer rows needs the custom mask. Splitting the computation this way makes attention 2.6 times faster on requests over 2048 tokens. FP8 changed the Jev Decision Index score by 0.01 points. In total, these steps make Prism 15.1 times faster.

One attention mask, computed in two pieces

Prism's mask for one question with two options

  • Fast kernel, no mask
  • Masked kernel, only these rows
  • Blocked

StateQABAns

StateQABAns

State + question: a full squareFast kernel, no mask needed

Options + answer: a thin stripMasked kernel, only these rows

Rows: the word doing the reading. Columns: the words it can see. Q = question, A and B = two options, Ans = answer token.

Making it fast

Prism started at 664 ms per median request, running in 16-bit precision across two B200 GPUs. We converted the weights to FP8 so the model fits on a single GPU, ran all 128 experts in one grouped kernel, captured the forward pass as a CUDA Graph, folded the FP8 conversion into the expert kernels, and merged the fine-tuning adapter into the model weights. Together, these bring the median request to 44 ms, 15× faster than where we started, with no drop in accuracy.

Using Prism

Prism’s weights are on Hugging Face under the Apache 2.0 licence. Prism runs in FP8 on one GPU with at least 120 GB of memory. It is also available on the Sarvam API at Sarvam Indus. Each request takes a state, a question and any number of options that fit within the context window, and returns a probability for each option. The documentation is available here.

Curious what else we're building? Explore our APIs and start creating.

Curious what else we're building?

Explore our APIs and start creating.

来源:Sarvam AI · sarvam.ai