# Runway 发布 Model Router 成本与质量评测报告

- 来源：Runway：News（网页）
- 作者：Runway
- 发布时间：2026-09-25 02:58
- AIHOT 分数：44
- AIHOT 链接：https://aihot.news/items/cmufwaohb03qfro19k94k5lf7
- 原文链接：https://runwayml.com/news/developers/evaluating-runway-model-router

## AI 摘要

Runway 公布 Model Router 评测报告，在 250 条图生视频提示词上对比 SOTA 基线 Seedance 2.5 与三种路由配置。Quality + $1 Cap 配置将单条生成成本从 $1.80 降至 $0.61（-66%），可用率 74%，保留基线 95% 的可用质量；Cost 路由成本降 83% 但可用率仅 38%。

## 正文

Runway Dev’s Model Router selects the right model for every request, so builders don’t have to guess which media model is best when integrating into their products. Instead, define your preference for cost, quality or latency, and the router handles optimal selection at request time. Model Router evaluates incoming prompts and router configurations against Runway’s benchmarking dataset, which tracks qualitative and quantitative model performance.

This report is the first window into how Model Router configurations reduce cost and hold production usability rates in comparison to a state-of-the-art (SOTA) model baseline. Today, 64% of builders using Model Router on Runway Dev have configured their routers for cost optimization. Early feedback shows builders prefer maximum efficiency, with the cheapest model that still delivers production-ready outputs.

Despite this, many builders using generative media APIs default to using the latest SOTA model for every call to guarantee quality. This "better safe than sorry" strategy presents three key issues:

Over-provisioning: You often don't need an expensive SOTA model for straightforward generation tasks

Generalist vs. Specialist Trade-offs: A general SOTA model isn't always the top performer for niche or domain-specific use cases

Resource Constraints: Finding the cheapest model that meets production quality requires time-consuming evaluation loops that most teams don't have the bandwidth to run

Model Router resolves these issues on a per-request basis while continuously adapting as new models are deployed on Runway Dev.

Experimental Setup & Methodology

To test whether dynamic routing preserves quality while lowering costs, we conducted an experiment comparing a SOTA model as a baseline against three Model Router configurations. We generated videos for each setup across a benchmark dataset of 250 image-to-video prompts (text prompt + reference image).

Experimental configurations

We evaluated four total setups across the prompt dataset:

Baseline Control: Seedance 2.5 (a SOTA video model)

Quality-Optimized Router: Configured to select the highest-performing model regardless of cost

Quality-Optimized Router w/ $1.00 Cap: Configured to prioritize quality while enforcing a strict $1.00 per-generation limit

Cost-Optimized Router: Configured to select the cheapest model that supports the request

Dataset stratification

To ensure broad coverage across real-world production use cases, the 250 prompts were evenly distributed across 10 functional categories:

Human & character action

Animals & nature

Physics & object interactions

Camera movement & framing

Text & signage

Crowds & complex scenes

Stylized animation

Product & commercial

Dialogue & lip-sync

Abstract & VFX

Rigor & statistical controls

Blind Evaluation: Output comparisons were double-blinded to eliminate evaluator model bias

Statistical Significance: Preferences and pass rates were evaluated using paired two-sided t-tests with Benjamini-Hochberg FDR correction (controlling family-wise error rate at alpha = 0.05)

Example

Here are the prompt inputs for one of the scenarios in our dataset and the video outputs generated for each of our test configurations:

Inputs

Text Prompt: “A white storefront awning clearly displays the word OPEN in large black capital letters. A person walks past from left to right but does not enter. Daylight street.”

Outputs

媒体内容 · 前往原文查看

Baseline video (Seedance 2.5)

媒体内容 · 前往原文查看

Quality-optimized router video (Gemini Omni Flash 1.1)

媒体内容 · 前往原文查看

Quality-optimized with $1 cap router video (Gemini Omni Flash 1.0)

媒体内容 · 前往原文查看

Cost-optimized Router video (Gen-4 Turbo)

Evaluating Cost & Usable Quality

To measure real-world performance, we tracked production usability and cost on the complete 250-prompt dataset. The results were as follows:

媒体内容 · 前往原文查看

Figure 1: Usable Quality vs. Cost (n ≈ 250)

ConfigurationUsable RateCost / ClipSpend vs. BaselineSelected Model(s)

Baseline (Seedance 2.5)78%$1.80—Seedance 2.5 (100%)

Quality Router77%$1.28-29%Seedance 2.5 (56%), Gemini Omni Flash 1.1 (34%), Gemini Omni Flash 1.0 (9%), Happy Horse (1%)

Quality + $1 Cap74%$0.61-66%Gemini Omni Flash 1.1 (65%), Gemini Omni Flash 1.0 (34%), Happy Horse (1%)

Cost Router38%$0.30-83%Gen-4 Turbo (99%)

Performance Takeaways

Quality + $1 Cap captures a balance of quality and cost: Enforcing a $1.00 per-generation limit on the Quality-Optimized Router cuts cost by 66% while maintaining a 74% usable rate, retaining 95% of the baseline SOTA quality at a fraction of the price

Uncapped Quality Mode yields automatic savings without compromising output: The Quality-Optimized Router delivers near-identical usability to the fixed baseline (77% vs. 78%) while generating 29% cost savings by intelligently shifting lower-complexity prompts away from expensive SOTA calls

Cost-Optimized Mode is a targeted trade-off for early-stage workflows: While the Cost-Optimized Router slashes spend by 83%, usable output drops to 38%. This setup is best suited for high-volume brainstorming or drafting workflows that route final passes through a quality-optimized router

Rethinking Default SOTA Integrations

Defaulting to the latest SOTA model for every single generation request has become a costly and unnecessary habit in AI engineering. As our evaluation demonstrates, paying top-tier rates for every prompt isn't required to maintain production-ready quality.

Strategic Takeaways:

Over-provisioning is real: For a large percentage of everyday prompts, mid-tier specialized models pass production quality bars, without SOTA prices.

Constrained routers capture the sweet spot: Introducing simple operational guardrails, such as a $1.00 per-generation cap, retains 95% of SOTA usable quality (74% vs. 78%) while cutting generation spend by two-thirds.

Automated routing beats manual eval loops: Instead of spending bandwidth running ad-hoc evaluations every time a new model launches, dynamic routers continually match incoming prompts to the cheapest model that meets your defined quality bar.

Dynamic routing gives product builders granular control over unit economics without forcing a choice between overpaying or shipping broken outputs.

Ready to stop over-provisioning your media pipeline? Explore Model Router on Runway Dev or read our documentation to deploy your first cost-capped router in minutes.

Appendix: Category-Level Benchmark Data

You can see the breakdown of usability across domain-specific workloads represented by each of our 10 prompt categories in the tables below:

媒体内容 · 前往原文查看

Human & Character Action

ConfigurationUsable RateCost / ClipSpend vs. BaselineSelected Model(s)

Baseline84% (21/25)$1.80—Seedance 2.5 (100%)

Cost12% (3/25)$0.30−83%Gen-4 Turbo (100%)

Quality84% (21/25)$1.66−8%Seedance 2.5 (88%), Gemini Omni Flash 1.1 (12%)

Quality + $1 Cap68% (17/25)$0.61−66%Gemini Omni Flash 1.1 (52%), Gemini Omni Flash 1.0 (48%)

媒体内容 · 前往原文查看

Animals & Nature

ConfigurationUsable RateCost / ClipSpend vs. BaselineSelected Model(s)

Baseline76% (19/25)$1.80—Seedance 2.5 (100%)

Cost48% (12/25)$0.30−83%Gen-4 Turbo (100%)

Quality76% (19/25)$1.18−34%Gemini Omni Flash 1.1 (52%), Seedance 2.5 (48%)

Quality + $1 Cap72% (18/25)$0.61−66%Gemini Omni Flash 1.1 (60%), Gemini Omni Flash 1.0 (40%)

媒体内容 · 前往原文查看

Physics & Object Interactions

ConfigurationUsable RateCost / ClipSpend vs. BaselineSelected Model(s)

Baseline60% (15/25)$1.80—Seedance 2.5 (100%)

Cost24% (6/25)$0.30−83%Gen-4 Turbo (100%)

Quality56% (14/25)$0.66−63%Gemini Omni Flash 1.0 (52%), Gemini Omni Flash 1.1 (44%)

Quality + $1 Cap52% (13/25)$0.61−66%Gemini Omni Flash 1.0 (92%)

媒体内容 · 前往原文查看

Camera Movement & Framing

ConfigurationUsable RateCost / ClipSpend vs. BaselineSelected Model(s)

Baseline88% (22/25)$1.80—Seedance 2.5 (100%)

Cost40% (10/25)$0.31−83%Gen-4 Turbo (96%)

Quality84% (21/25)$1.32−26%Seedance 2.5 (60%), Gemini Omni Flash 1.1 (40%)

Quality + $1 Cap84% (21/25)$0.61−66%Gemini Omni Flash 1.1 (92%)

媒体内容 · 前往原文查看

Text & Signage

ConfigurationUsable RateCost / ClipSpend vs. BaselineSelected Model(s)

Baseline80% (20/25)$1.80—Seedance 2.5 (100%)

Cost64% (16/25)$0.30−83%Gen-4 Turbo (100%)

Quality80% (20/25)$0.94−48%Gemini Omni Flash 1.1 (72%), Seedance 2.5 (28%)

Quality + $1 Cap72% (18/25)$0.61−66%Gemini Omni Flash 1.1 (88%)

媒体内容 · 前往原文查看

Crowds & Complex Scenes

ConfigurationUsable RateCost / ClipSpend vs. BaselineSelected Model(s)

Baseline76% (19/25)$1.80—Seedance 2.5 (100%)

Cost56% (14/25)$0.31−83%Gen-4 Turbo (96%)

Quality76% (19/25)$1.51−16%Seedance 2.5 (76%), Gemini Omni Flash 1.1 (24%)

Quality + $1 Cap80% (20/25)$0.61−66%Gemini Omni Flash 1.1 (88%)

媒体内容 · 前往原文查看

Stylized & Animation

ConfigurationUsable RateCost / ClipSpend vs. BaselineSelected Model(s)

Baseline80% (20/25)$1.80—Seedance 2.5 (100%)

Cost39% (9/23)$0.31−83%Gen-4 Turbo (96%)

Quality84% (21/25)$1.51−16%Seedance 2.5 (76%), Gemini Omni Flash 1.1 (24%)

Quality + $1 Cap78% (18/23)$0.61−66%Gemini Omni Flash 1.0 (56%), Gemini Omni Flash 1.1 (44%)

媒体内容 · 前往原文查看

Product & Commercial

ConfigurationUsable RateCost / ClipSpend vs. BaselineSelected Model(s)

Baseline72% (18/25)$1.80—Seedance 2.5 (100%)

Cost48% (12/25)$0.30−83%Gen-4 Turbo (100%)

Quality64% (16/25)$1.28−29%Seedance 2.5 (56%), Gemini Omni Flash 1.1 (40%)

Quality + $1 Cap52% (13/25)$0.61−66%Gemini Omni Flash 1.1 (76%), Gemini Omni Flash 1.0 (24%)

媒体内容 · 前往原文查看

Dialogue & Lip-Sync

ConfigurationUsable RateCost / Clip Spend vs. BaselineSelected Model(s)

Baseline92% (22/24)$1.80—Seedance 2.5 (100%)

Cost16% (4/25)$0.30−83%Gen-4 Turbo (100%)

Quality92% (22/24)$1.76−2%Seedance 2.5 (96%)

Quality + $1 Cap96% (24/25)$0.62−65%Gemini Omni Flash 1.1 (92%)

媒体内容 · 前往原文查看

Abstract & VFX

ConfigurationUsable RateCost / ClipSpend vs. BaselineSelected Model(s)

Baseline68% (17/25)$1.80—Seedance 2.5 (100%)

Cost 32% (8/25)$0.30−83%Gen-4 Turbo (100%)

Quality 76% (19/25)$0.99−45%Gemini Omni Flash 1.1 (36%), Gemini Omni Flash 1.0 (32%), Seedance 2.5 (32%)

Quality + $1 Cap84% (21/25)$0.61−66%Gemini Omni Flash 1.1 (52%), Gemini Omni Flash 1.0 (48%)