如果 Anthropic 在流量中途对你限流，你的应用不应该返回 500。正是这单一故障模式，成为团队一开始就寻求 LLM 路由器的重大原因。OpenRouter 将每个请求路由到 70+ 家提供商，而你可以控制路由方式，精细到提供商顺序、价格上限和回退链。
OpenRouter 中的路由是两个独立的决策：由哪个模型来回答请求，以及由哪个提供商来服务该模型。本指南中的每一个配置选项都对应这两个决策之一。将它们分开，正是让你在出问题时能够配置正确层级的关键。
OpenRouter Python SDK（pip install openrouter）只需几行代码就能让你上手，而切换模型只需改一个字符串。（已经在用 OpenAI SDK？把它的 base URL 指向 https://openrouter.ai/api/v1，同样可以工作。）
from openrouter import OpenRouter
client = OpenRouter(api_key="<OPENROUTER_API_KEY>")
resp = client.chat.send(
model="anthropic/claude-sonnet-4.6", # change this string to switch models
messages=[{"role": "user", "content": "Summarize this changelog."}],
)这就是全部的集成工作。路由发生在那个单一端点背后，本指南的其余部分将逐一讲解它所暴露的路由层级，并附上你可以直接复制使用的确切配置。
LLM 路由器究竟做什么
这两个决策属于一项更大的工作。路由器将一个请求送达一个端点，并处理将其移交给应当回答它的模型和提供商。每个路由器都要完成这 4 项工作，而 OpenRouter 全部包办：
| 工作 | 它决定什么 | OpenRouter 在哪里完成 |
|---|---|---|
| 模型选择 | 由哪个模型来回答提示词 | model 字段，或 openrouter/auto |
| 提供商选择 | 由哪个提供商来提供该模型的服务 | provider 对象（默认：基于价格） |
| 负载均衡 | 如何在多个稳定提供商之间分配流量 | 价格平方反比加权 |
| 故障转移 | 出现错误时该尝试什么 | models 数组 + 提供商回退 |
像 RouteLLM 和 LLMRouter 这样的开源项目，是你自行托管并接入自己基础设施的路由库。而 OpenRouter 是一个你直接调用的路由器：一项托管服务，通过单一端点对接 400+ 个模型。
当你需要决定是自己掌握路由逻辑还是将其外包时，这一区别至关重要，我们会在路由器与网关的对比章节中再次谈到它。
OpenRouter 中的两层路由
OpenRouter 在 2 个独立的层面上进行路由：模型路由（由哪个模型来回答）和提供商路由（由哪个提供商来服务该模型）。把这两者分开来看，下面每一项配置就都能对号入座了。
一个 兼容 OpenAI 的端点位于这两层之前：https://openrouter.ai/api/v1，一个 API key，覆盖 70+ 个提供商的 400+ 个模型。你调用 OpenRouter 的方式与调用 OpenAI 相同，它在幕后进行分发。
每个路由决策发生在哪里
这个最小请求在一个 payload 中展示了两个决策。
curl https://openrouter.ai/api/v1/chat/completions \
-H "Authorization: Bearer $OPENROUTER_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"model": "anthropic/claude-sonnet-4.6",
"messages": [{"role": "user", "content": "Hello"}]
}'model 字段是你的模型路由决策。如果某个给定模型由多个提供商服务，OpenRouter 的提供商路由会挑选由哪一个来处理这次具体的调用。你显式设置模型；提供商选择会自动运行，除非你将其覆盖。
一个 key，多个提供商
单 key 的设置意味着你使用一个 OpenRouter API key 就能访问多个提供商。如果你想改用自己提供商的 key，BYOK 指南涵盖了这条路径。它保留你现有的提供商协议，同时在其之上增加故障转移和路由。
哪种路由模式适合哪类任务
在讲具体机制之前，先给出这张地图。接下来的各节会说明每种模式如何工作。对于默认模式看不到的约束——合规、延迟预算、成本上限以及跨模型韧性——请使用覆盖配置。
| 你的情况 | 使用 | 原因 |
|---|---|---|
| 最便宜且可靠，不在乎是哪家提供商 | 默认负载均衡 | 平方反比加权会处理它 |
| 必须命中某一家提供商（合规、BYOK、区域） | provider.order + allow_fallbacks: false | 硬性停止，不静默回退 |
| 延迟敏感（面向用户的聊天） | :nitro | 为吞吐量而路由 |
| 有成本上限的批处理任务 | :floor / max_price | 为价格而路由，并强制执行上限 |
| 需要跨模型的韧性 | models 回退数组 | 能挺过整个模型宕机 |
| 不知道用户发送的是什么提示词 | openrouter/auto | NotDiamond 按提示词逐个挑选 |
本指南的其余部分将按顺序逐一讲解这些模式，从你什么都不设置时运行的那一种开始。
提供商路由的工作原理
默认情况下，OpenRouter 会把你的请求发送给最便宜且可靠的提供商，权重为价格的平方反比。这一层负责做出路由决策，并在每次请求时运行，除非你将其覆盖。
默认策略，逐步拆解
当一个模型有多个提供商，而你又没有设置 sort 或 order 时，OpenRouter 会执行 3 个步骤，依据提供商路由文档：
- 优先选择在过去 30 秒内没有出现重大故障的提供商（不稳定的提供商会排到队尾，但不会被移除）。
- 在稳定的提供商中，从成本最低的候选者里挑选，并按价格的平方反比进行加权。
- 将剩余的提供商用作回退。
设置 sort 或 order 会关闭负载均衡，改为按你的规则进行路由。不设置它们，你就会得到按价格加权的默认行为。
完整示例（为什么平方反比很重要）
以下是文档中关于该默认策略如何生效的完整示例。假设一个模型由 3 家提供商提供服务：提供商 A 为 $1/M tokens，提供商 B 为 $2/M，提供商 C 为 $3/M，且提供商 B 最近发生过服务中断。
平方反比加权意味着 A 被优先尝试的概率大约是 C 的 9 倍（1/3² 使 C 的权重仅为 A 的 1/9）。如果 A 失败，接下来是 C。B 刚刚经历中断，最后才会被尝试。
这种平方处理会把流量强力拉向最便宜的稳定选项，而不是均匀分散，因此当一家健康的 $1/M 提供商就在眼前时，你不会去支付 $3/M。
OpenRouter 在生产环境中对 70+ 家提供商运行这一策略，每月处理 100 trillion tokens，示例正是取自这一实际运营情况。需要记住的 2 条规则：30 秒中断窗口和平方反比价格加权。
关于该默认设置有一点需要注意：它管理的是标准请求。包含工具的请求会经由 Auto Exacto 路由——这是 OpenRouter 针对工具调用所采用的“质量优先”路由步骤，它会根据工具调用质量信号对提供商进行分层（将表现最差的提供商推到后面，同时在各层内保持价格排序），并且默认启用。若要在工具调用请求上重新启用价格加权，请用 :floor 或 provider.sort: "price" 强制启用。
使用 provider 对象控制路由
在请求体中添加一个 provider 对象，即可完全覆盖默认设置。该对象的完整说明见 此处文档；以下是开发者最先会用到的字段。
| 字段 | 作用 | 默认值 |
|---|---|---|
order | 按此确切顺序尝试各提供商 | 未设置 |
allow_fallbacks | 如果你选择的提供商失败，则回退到其他提供商 | true |
sort | 按 "price"、"throughput" 或 "latency" 路由 | 未设置 |
only | 限制为此提供商允许列表 | 未设置 |
ignore | 排除这些提供商 | 未设置 |
quantizations | 限制为特定的量化级别 | 未设置 |
data_collection | "allow" 或 "deny" 会用数据进行训练的提供商 | 未设置 |
zdr | 要求使用零数据保留（ZDR）的提供商 | 未设置 |
max_price | 设置你可接受的每 token 价格上限 | 未设置 |
preferred_min_throughput | 优先选择吞吐量高于此值的提供商 | 未设置 |
preferred_max_latency | 优先选择延迟低于此值的提供商 | 未设置 |
require_parameters | 仅使用支持你的请求参数的提供商 | 未设置 |
固定提供商顺序并禁用回退
当你必须命中某一个特定提供商（合规要求、BYOK 合同、区域限制）而不能使用其他任何提供商时，设置顺序并关闭回退。
{
"model": "openai/gpt-5.5",
"provider": { "order": ["openai", "azure"], "allow_fallbacks": false }
}OpenRouter 会先尝试 OpenAI，然后尝试 Azure，然后停止。它绝不会回退到你未批准的第三个提供商。
排除提供低质量变体的提供商
供应商质量参差不齐，OpenRouter 对此直言不讳。有些供应商提供的模型是量化程度更高的变体，其表现不如同一模型在其他地方托管的效果。你无法按质量分数进行排序或筛选，但可以直接排除某个供应商。
{
"model": "meta-llama/llama-4-maverick",
"provider": { "ignore": ["deepinfra"] }
}如需更精细的控制，quantizations 可将范围限制在你所信任的精度级别。这就是应对质量差异的可控手段。
定位特定供应商端点
供应商 slug 按基础名称匹配。像 "google-vertex" 这样的 slug 会匹配 OpenRouter 上该供应商的所有 Vertex 区域和端点。要锁定单个变体，请使用完整 slug，例如用 "deepinfra/turbo" 而非 "deepinfra"。
基础 slug 是大网撒捕，完整 slug 是手术刀精切。
:nitro 和 :floor 的作用
在模型名后追加 :nitro 可优化速度，追加 :floor 可优化成本。它们是快捷方式：:nitro 完全等同于 provider.sort: "throughput"，:floor 完全等同于 provider.sort: "price"。无需对象，无需额外配置，只需改动模型字符串。
| Slug | 优化目标 | 等同于 |
|---|---|---|
model:nitro | 吞吐量（速度） | provider.sort: "throughput" |
model:floor | 价格（成本） | provider.sort: "price" |
model（裸用） | 最便宜且可靠的平衡 | 默认按价格加权的路由 |
以下是只需改一行的做法：
model="meta-llama/llama-4-maverick:nitro" # route for speed
model="meta-llama/llama-4-maverick:floor" # route for cost在延迟能被用户感知的面向用户的聊天场景中，选用 :nitro；在成本是约束条件的批处理任务中，选用 :floor。对于有成本上限的工作，将 :floor 与 max_price 以及 BYOK 经济性搭配使用。
模型路由与故障转移如何协同工作
为了在提供商出错时保持可用，配置一个 models 回退数组，并依赖 OpenRouter 的自动提供商故障转移。这里叠加了 2 种机制，它们在不同层面运作。提供商路由让单个模型在多个提供商之间保持存活，而模型回退则处理该模型的所有提供商同时失败的情况。
这能覆盖大多数提供商错误，但无法让你免受所有服务级宕机的影响。
简而言之：列出备用模型，让提供商故障转移处理其余的事情，失败的请求不会向你收费。
模型回退（models 数组）
按优先级顺序传入 models。如果第一个出错，OpenRouter 会尝试下一个。
resp = client.chat.send(
model="anthropic/claude-sonnet-4.6",
models=["openai/gpt-5.4-mini"], # tried in order if the primary fails
messages=[{"role": "user", "content": "..."}],
)OpenRouter 的 SDK 将 models 作为一等字段。（在 OpenAI SDK 上，通过 extra_body 传入，因为它是 OpenRouter 的扩展。）回退会在上下文长度错误、审核标记、速率限制和宕机时触发。你只需为实际运行的模型付费，而不是那些先出错的模型。
提供商级故障转移（在单个模型内）
在模型回退之下，还有第二层安全网。在单个模型内，如果所选提供商返回 5xx 或对你进行速率限制，OpenRouter 会自动切换到为该模型提供服务的下一个提供商。此功能默认开启（allow_fallbacks: true），任何在过去 30 秒内发生宕机的提供商都会被降低优先级。
从经济角度来看，这让人可以放心依赖。失败的请求不计费；OpenRouter 的 零完成保险意味着你只需为完成的运行付费。
故障转移覆盖和不覆盖的内容
故障转移有一些值得了解的局限。中止流式输出并不会停止部分提供商的计费（根据流式文档，其中包括 Bedrock、Groq、Google 和 Mistral），因此被取消的流在这些提供商上仍可能产生费用。
而故障转移可以绕过提供商层面的故障，但它无法绕过 OpenRouter 本身：2025 年 8 月的宕机（一次约 50 分钟的数据库事故）导致整个服务瘫痪，回退机制也不例外。提供商故障转移是真实存在的，而且是自动的；它是一层韧性保障，而非 SLA 保证。两者都要做好预案。
Auto Router，以及何时使用它
向openrouter/auto发送请求，你就把模型选择权交给了 OpenRouter，而不是自己挑选。当某个提示词的最佳模型会变化时使用它，比如混合工作负载，其中一些请求需要强大的推理能力，另一些则需要快速补全。当你确切知道自己想要哪个模型时，就明确指定它。
Auto Router 由NotDiamond提供支持，它会针对每个提示词从一个精选模型池中选择一个模型。
Auto Router 如何选择
该模型池是一组轮换的强模型；Auto Router 文档列出了当前的阵容。调用之后，响应中的model字段会告诉你实际是哪一个模型做出了回答，所以你永远不用猜测。
你通过 auto-router 插件来引导这一选择。cost_quality_tradeoff 是一个 0-10 的刻度盘，默认值为 7。将其设为 0，路由器总是会选择能力最强的模型；将其设为 10，它则会选择最便宜的。
allowed_models 通过像 anthropic/* 这样的通配符模式，将选择限制在某个提供商家族内部。
{
"model": "openrouter/auto",
"plugins": [
{
"id": "auto-router",
"cost_quality_tradeoff": 3,
"allowed_models": ["anthropic/*", "openai/*"]
}
]
}在定价方面，没有 Auto Router 附加费。无论最终选中哪个模型，你都按标准费率付费，与直接调用该模型相同。
Auto Router 与手动 models 数组的对比
两者都是“路由”，但控制面正好相反。使用 openrouter/auto 时，由 OpenRouter 决定模型。使用 models 回退数组时，由你决定顺序，OpenRouter 只是在出错时按顺序依次尝试。
当你不知道用户会发送什么提示词时，使用 Auto Router；当你知道自己的模型偏好并希望在其背后获得韧性时，使用回退数组。
路由器与网关，以及 OpenRouter 所处的位置
网关是统一的访问入口（单一端点、认证、限流、可观测性）；路由器则做出逐请求的决策（用哪个模型、哪个提供商）。OpenRouter 就是你调用的网关，它内置了路由逻辑，决定由哪个模型和提供商处理每个请求。
| 能力 | 路由器 | 网关 | OpenRouter |
|---|---|---|---|
| 按请求决定模型/提供商 | 是 | 仅访问 | 是 |
| 自动故障转移 | 是 | 有时 | 是 |
| 单一统一端点 | 有时 | 是 | 是 |
| 托管 vs 自托管 | 两者皆可 | 两者皆可 | 托管 |
我们的 LLM 网关指南完整定义了网关层级；路由是其中的机制。如果你想要一个自己运行的自托管网关加路由器，LiteLLM 是自行托管的选择。OpenRouter 是直接调用的选择。
常见问题
LLM 路由是如何工作的？
路由器位于你的应用与多个模型和提供商之间，决定每个请求的去向。它处理 4 项工作：模型选择、提供商选择、负载均衡和故障转移。你向一个端点发送一个请求，路由器负责挑选目的地。
OpenRouter 如何选择使用哪个提供商？
默认情况下，它会将过去 30 秒内出现重大故障的任何提供商降级，然后在成本最低的提供商中进行选择，按价格的平方反比加权，其余提供商保留作为回退。设置 sort 或 order 会覆盖此默认行为。
LLM 路由器和 LLM 网关有什么区别？
网关是统一接入点（一个端点、认证、可观测性）。路由器则针对每个请求决定由哪个模型和提供商处理该调用。OpenRouter 两者兼具：它既是你调用的网关，同时也进行路由。
如何让 OpenRouter 故障转移到另一个模型或提供商？
按优先级顺序传入一个 models 数组，即可实现模型级回退（在上下文长度错误、审核标记、速率限制和宕机时触发）。对于同一模型内的提供商级故障转移，OpenRouter 默认会自动处理（allow_fallbacks: true）。失败的请求不计费。
:nitro 和 :floor 是做什么的？
将 :nitro 追加到模型 slug 后，可优化吞吐量（即 provider.sort: "throughput"）。:floor 则优化成本（provider.sort: "price"）。两者都只需对模型字符串做一行改动。
Auto Router 会额外收费吗？
不会。无论 openrouter/auto 选择哪个模型，你都按标准费率付费，没有额外的 Auto Router 费用。
If Anthropic rate-limits you mid-traffic, your app shouldn’t return a 500. That single failure mode is a big reason teams reach for an LLM router in the first place. OpenRouter routes every request across 70+ providers, and you control how, down to the provider order, the price ceiling, and the fallback chain.
Routing in OpenRouter is 2 independent decisions: which model answers the request and which provider serves that model. Every configuration option in this guide maps to one of those 2 decisions. Keeping them separate is what lets you configure the right layer when something breaks.
The OpenRouter Python SDK (pip install openrouter) gets you there in a few lines, and switching models is a one-string change. (Already on the OpenAI SDK? Point its base URL at https://openrouter.ai/api/v1 and it works too.)
from openrouter import OpenRouter
client = OpenRouter(api_key="<OPENROUTER_API_KEY>")
resp = client.chat.send(
model="anthropic/claude-sonnet-4.6", # change this string to switch models
messages=[{"role": "user", "content": "Summarize this changelog."}],
)That’s the whole integration. The routing happens behind that one endpoint, and the rest of this guide works through the routing layers it exposes, with the exact config you copy in.
What an LLM Router Actually Does
Those 2 decisions sit inside a bigger job. A router takes one request to one endpoint and handles the handoff to whatever model and provider should answer it. Every router does 4 of these jobs, and OpenRouter handles all 4:
| Job | What it decides | Where OpenRouter does it |
|---|---|---|
| Model selection | Which model answers the prompt | model field, or openrouter/auto |
| Provider selection | Which provider serves that model | provider object (default: price-based) |
| Load balancing | How to distribute across stable providers | Inverse-square price weighting |
| Failover | What to try when something errors | models array + provider fallback |
Open-source projects like RouteLLM and LLMRouter are routing libraries you self-host and wire into your own infrastructure. OpenRouter is a router you call: a managed service with a single endpoint in front of 400+ models.
The distinction matters when you’re deciding whether to own the routing logic or hand it off, and we come back to it in the router-vs-gateway section.
The Two Layers of Routing in OpenRouter
OpenRouter routes on 2 independent layers: model routing (which model answers) and provider routing (which provider serves that model). Hold those 2 apart, and every config below clicks into place.
One OpenAI-compatible endpoint sits in front of both layers: https://openrouter.ai/api/v1, one API key, 400+ models across 70+ providers. You call OpenRouter the same way you’d call OpenAI, and it fans out behind the scenes.
Where each routing decision happens
The minimal request shows both decisions in one payload.
curl https://openrouter.ai/api/v1/chat/completions \
-H "Authorization: Bearer $OPENROUTER_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"model": "anthropic/claude-sonnet-4.6",
"messages": [{"role": "user", "content": "Hello"}]
}'The model field is your model routing decision. If a given model is served by several providers, OpenRouter’s provider routing picks which one handles this specific call. You explicitly set the model; provider selection runs automatically unless you override it.
One key, many providers
The single-key setup means you use one OpenRouter API key to access multiple providers. If you want to use your own provider keys instead, the BYOK guide covers that path. It keeps your existing provider agreements while adding failover and routing on top.
Which Routing Mode Fits Which Job
Before the mechanics, here’s the map. The sections after this one show how each mode works. Use overrides for the constraints the default can’t see: compliance, latency budgets, cost caps, and cross-model resilience.
| Your situation | Use | Why |
|---|---|---|
| Cheapest reliable, don’t care which provider | Default load balancing | Inverse-square weighting handles it |
| Must hit one provider (compliance, BYOK, region) | provider.order + allow_fallbacks: false | Hard stop, no silent fallback |
| Latency-sensitive (user-facing chat) | :nitro | Routes for throughput |
| Cost-capped batch jobs | :floor / max_price | Routes for price, enforces a ceiling |
| Need cross-model resilience | models fallback array | Survives a whole model going down |
| Don’t know what prompts users send | openrouter/auto | NotDiamond picks per prompt |
The rest of this guide works through these modes in order, starting with the one that runs when you set nothing.
How Provider Routing Works
By default, OpenRouter sends your request to the cheapest reliable provider, weighted by the inverse square of price. This is the layer that makes routing decisions and runs on every request unless you override it.
The default strategy, step by step
When a model has multiple providers and you haven’t set sort or order, OpenRouter runs 3 steps, per the provider-routing docs:
- Prioritize providers with no significant outages in the last 30 seconds (unstable ones drop to the back of the line, they aren’t removed).
- Among the stable providers, pick from the lowest-cost candidates, weighted by the inverse square of price.
- Use the remaining providers as fallbacks.
Setting sort or order turns load balancing off and routes by your rule instead. Leave them unset and you get the price-weighted default.
The worked example (why inverse-square matters)
Here’s the docs’ worked example of how that default plays out. Say a model is served by 3 providers: Provider A at $1/M tokens, Provider B at $2/M, and Provider C at $3/M, and Provider B recently had outages.
Inverse-square weighting means A is roughly 9x more likely than C to be tried first (1/3² gives C 1/9 the weight of A). If A fails, C is next. B, fresh off outages, gets tried last.
The squaring pulls traffic hard toward the cheapest stable option instead of spreading it evenly, so you’re not paying $3/M when a healthy $1/M provider is sitting right there.
OpenRouter runs this strategy in production across 70+ providers at 100 trillion tokens a month, which is the operating reality the example is drawn from. The 2 rules to remember: the 30-second outage window and the inverse-square price weighting.
One caveat on that default: it governs standard requests. Requests that include tools route through Auto Exacto, OpenRouter’s quality-first routing step for tool calls, which tiers providers by tool-call quality signals (pushing the lowest performers to the back, with price order kept within each tier) and runs by default. To put price-weighting back on a tool-calling request, force it with :floor or provider.sort: "price".
Controlling Routing with the provider Object
Add a provider object to the request body and you override the default entirely. The full object is documented here; these are the fields developers reach for first.
| Field | What it does | Default |
|---|---|---|
order | Try providers in this exact order | unset |
allow_fallbacks | Fall through to other providers if your picks fail | true |
sort | Route by "price", "throughput", or "latency" | unset |
only | Restrict to this allowlist of providers | unset |
ignore | Exclude these providers | unset |
quantizations | Restrict to specific quantization levels | unset |
data_collection | "allow" or "deny" providers that train on data | unset |
zdr | Require Zero Data Retention (ZDR) providers | unset |
max_price | Cap the per-token price you’ll accept | unset |
preferred_min_throughput | Prefer providers above this throughput | unset |
preferred_max_latency | Prefer providers under this latency | unset |
require_parameters | Only use providers supporting your request params | unset |
Pin provider order and disable fallbacks
When you must hit one specific provider (a compliance requirement, a BYOK contract, a region constraint) and nothing else, set the order and turn fallbacks off.
{
"model": "openai/gpt-5.5",
"provider": { "order": ["openai", "azure"], "allow_fallbacks": false }
}OpenRouter tries OpenAI, then Azure, and stops. It never falls back to a third provider you didn’t approve.
Exclude a provider with lower-quality variants
Provider quality varies, and OpenRouter says so plainly. Some providers serve more heavily quantized variants of a model that underperform the same model hosted elsewhere. There’s no quality score you can sort or filter by, but you can exclude a provider outright.
{
"model": "meta-llama/llama-4-maverick",
"provider": { "ignore": ["deepinfra"] }
}For finer control, quantizations restricts to the precision levels you trust. This is the controllable surface for that quality variance.
Targeting a specific provider endpoint
Provider slugs match by base name. A slug like "google-vertex" matches every Vertex region and endpoint OpenRouter has for that provider. To pin a single variant, use the full slug, for example "deepinfra/turbo" instead of "deepinfra".
Base slug is the broad net; the full slug is the scalpel.
What :nitro and :floor Do
Append :nitro to optimize a model for speed, :floor to optimize for cost. They’re shortcuts: :nitro is exactly provider.sort: "throughput", and :floor is exactly provider.sort: "price". No object, no extra config, just a change to the model string.
| Slug | Optimizes for | Equivalent to |
|---|---|---|
model:nitro | Throughput (speed) | provider.sort: "throughput" |
model:floor | Price (cost) | provider.sort: "price" |
model (bare) | Cheapest-reliable balance | Default price-weighted routing |
Here it is as a one-line change:
model="meta-llama/llama-4-maverick:nitro" # route for speed
model="meta-llama/llama-4-maverick:floor" # route for costReach for :nitro on user-facing chat where latency is felt, and :floor on batch jobs where cost is the constraint. For cost-capped work, pair :floor with max_price and BYOK economics.
How Model Routing and Failover Work Together
To survive provider errors, configure a models fallback array and rely on OpenRouter’s automatic provider failover. 2 mechanisms stack here, and they operate at different layers. Provider routing keeps a single model alive across providers, and model fallbacks handle the case where all providers for that model fail at once.
That covers most provider errors, but it doesn’t protect you from every service-wide outage.
The short version: list backup models, let provider failover handle the rest, and you won’t pay for the requests that fail.
Model fallbacks (the models array)
Pass models in priority order. If the first errors, OpenRouter tries the next.
resp = client.chat.send(
model="anthropic/claude-sonnet-4.6",
models=["openai/gpt-5.4-mini"], # tried in order if the primary fails
messages=[{"role": "user", "content": "..."}],
)The OpenRouter SDKs take models as a first-class field. (On the OpenAI SDK, pass it through extra_body, since it’s an OpenRouter extension.) Fallbacks trigger on context-length errors, moderation flags, rate-limiting, and downtime. You’re billed for the model that actually runs, not the ones that errored out first.
Provider-level failover (within one model)
Underneath model fallbacks, there’s a second safety net. Within a single model, if the chosen provider returns a 5xx or rate-limits you, OpenRouter automatically falls through to the next provider serving that model. This is on by default (allow_fallbacks: true), and any provider with an outage in the last 30 seconds gets deprioritized.
The economics make this safe to lean on. Failed requests aren’t billed; OpenRouter’s zero-completion insurance means you pay only for the run that completes.
What failover does and doesn’t cover
Failover has limits worth knowing about. Aborting a stream doesn’t stop billing on a subset of providers (Bedrock, Groq, Google, and Mistral among them, per the streaming docs), so a cancelled stream can still cost you on those.
And failover routes around provider-level failures, but it can’t route around OpenRouter itself: the August 2025 outage (a roughly 50-minute database incident) took the whole service down, fallbacks included. Provider failover is real and it’s automatic; it’s a layer of resilience rather than an SLA guarantee. Plan for both.
The Auto Router, and When to Use It
Send a request to openrouter/auto and you hand model selection to OpenRouter instead of picking yourself. Use it when the best model for a prompt varies, such as a mixed workload where some requests need strong reasoning and others need fast completion. When you know exactly which model you want, set it explicitly.
The Auto Router is powered by NotDiamond, which selects a model per prompt from a curated pool.
How the Auto Router picks
The pool is a rotating set of strong models; the Auto Router docs list the current lineup. After the call, the response model field tells you which one actually answered, so you’re never guessing.
You steer the choice through the auto-router plugin. cost_quality_tradeoff is a 0-10 dial that defaults to 7. Set it to 0 and the router always reaches for the most capable model; set it to 10 and it goes cheapest.
allowed_models keeps selection inside a provider family with wildcard patterns like anthropic/*.
{
"model": "openrouter/auto",
"plugins": [
{
"id": "auto-router",
"cost_quality_tradeoff": 3,
"allowed_models": ["anthropic/*", "openai/*"]
}
]
}On pricing, there’s no Auto Router surcharge. You pay the standard rate for whichever model gets selected, same as calling that model directly.
Auto Router vs a manual models array
Both are “routing,” but the control surface flips. With openrouter/auto, OpenRouter decides the model. With a models fallback array, you decide the order and OpenRouter just walks it on error.
Use the Auto Router when you don’t know what prompts users will send; use a fallback array when you know your model preference and want resilience behind it.
Router vs Gateway, and Where OpenRouter Sits
A gateway is the unified access point (one endpoint, auth, rate limiting, observability); a router makes the per-request decision (which model, which provider). OpenRouter is the gateway you call, and it includes routing logic that decides which model and provider handle each request.
| Capability | Router | Gateway | OpenRouter |
|---|---|---|---|
| Per-request model/provider decision | Yes | Access only | Yes |
| Automatic failover | Yes | Sometimes | Yes |
| Single unified endpoint | Sometimes | Yes | Yes |
| Managed vs self-hosted | Either | Either | Managed |
Our LLM gateway guide defines the gateway tier in full; routing is the mechanism inside it. If you want a self-hosted gateway-plus-router you run yourself, LiteLLM is the host-it option. OpenRouter is the call-it option.
Frequently Asked Questions
How does LLM routing work?
A router sits between your app and multiple models and providers, and decides where each request goes. It handles 4 jobs: model selection, provider selection, load balancing, and failover. You send one request to one endpoint, and the router picks the destination.
How does OpenRouter choose which provider to use?
By default it deprioritizes any provider with a significant outage in the last 30 seconds, then picks among the lowest-cost providers, weighted by the inverse square of price, and keeps the rest as fallbacks. Setting sort or order overrides this default.
What’s the difference between an LLM router and an LLM gateway?
A gateway is the unified access point (one endpoint, auth, observability). A router makes the per-request decision about which model and provider handle the call. OpenRouter is both: a gateway you call that also routes.
How do I make OpenRouter fail over to another model or provider?
Pass a models array in priority order for model-level fallback (triggers on context-length errors, moderation flags, rate limits, and downtime). For provider-level failover within one model, OpenRouter does it automatically by default (allow_fallbacks: true). Failed requests aren’t billed.
What do :nitro and :floor do?
:nitro appended to a model slug optimizes for throughput (it’s provider.sort: "throughput"). :floor optimizes for cost (provider.sort: "price"). Both are one-line changes to the model string.
Does the Auto Router cost extra?
No. You pay the standard rate for whichever model openrouter/auto selects, with no additional Auto Router fee.