跳到正文
Aravind Srinivas· @AravSrinivas · X·· 2 小时前AI 评分61
AI 导读

Perplexity 推出 Orchestrator API,一个 API 提供 45+ 开源权重和专有模型,带自动 provider 故障切换和统一计费。与通用模型路由不同,该 API 兼容 KV cache,并由用户决定哪些任务使用哪些模型;例如把 fable 5.2 拒答的安全与生物学任务路由到 gpt-6.1 sol 后,terminal-bench 4.1 解题数从 38 提升到 41(共 63)。

正文 · 原文

KV-cache compatible decision-model based routing on Perplexity Orchestrator API

引用Denis Yarats@denisyarats

we are launching the Orchestrator API: 45+ open-weight and proprietary models in one api, with automatic provider failover and unified billing.

general model routing doesn't really work in practice: it breaks the kv cache and is usually overfit to a few benchmarks. so we designed our routing api differently: it's kv cache friendly, and you decide which models to use for which tasks.

one example: fable 5.2 often refuses benign security and biology tasks, so we route those (and anything after a refusal) to gpt-6.1 sol. on terminal-bench 4.1 this goes from 38 to 41 solved tasks out of 63.

在 X 查看被引用的帖子

来源:Aravind Srinivas · x.com