我们全新的视频分析智能体功能可将模型 token 消耗最高降低 88%，成本最高降低 66%，同时将质量最高提升 7%。
Rohan Doshi
Mario Lučić
Google DeepMind 研究总监
Google 刚刚为 Gemini 3.7 Flash、3.6 Flash 和 3.5 Flash-Lite 推出了智能体视频理解功能。该功能让模型能够动态扫描视频片段，在提升准确率的同时，将模型 token 用量最高降低 88%、成本最高降低 66%。你现在就可以开始使用：在 Google AI Studio 或 Gemini Enterprise Agent Platform 中将 API 配置设为“agentic”即可。
今天，我们在最新模型中推出智能体视频理解功能，覆盖 Gemini 3.7 Flash、3.6 Flash 和 3.5 Flash-Lite。这项新能力在提升准确率的同时，大幅降低视频分析的模型 token 用量和成本。与智能体视觉类似——后者将代码执行与 Gemini 模型的原生图像理解能力相结合——智能体视频理解利用 Gemini 的原生视频工具来提升性能，并解锁视频处理的新能力，例如亚秒级时刻检索、更准确的异常检测、精确计数等。
该功能即日起可通过 Google AI Studio 和 Gemini Enterprise Agent Platform 中的 Gemini API，用于视频上传和 YouTube 视频。
基准测试
与当前“静态”处理方式不同——静态处理下，模型以固定的每秒帧率（默认 1 FPS，可通过 API 调整）摄取视频——智能体视频理解将模型的核心推理能力与原生视频工具相结合，动态地跨视觉帧、音频和转录文本搜索、扫描并检查目标视频片段。在标准视频分析基准测试中，采用智能体视频理解的 Gemini 模型可将分析成本降低最高 66%，token 消耗降低最高 88%，同时将准确率提升最高 7%。
这些效率提升在长视频上尤为显著（从 10 分钟的操作指南，到 90 分钟的讲座，再到数小时的录制内容），因为静态处理迫使开发者在高昂的 token 成本与可能丢失关键细节的技术之间做出取舍。
启用智能体视频理解后，Gemini 3.7 Flash 的 token 消耗可降低最高 88%，准确率可提升最高 7%。
虽然这些提升覆盖全部三款受支持的模型，但具备智能体理解的 Gemini 3.7 Flash 在整体上提供了最佳质量，同时也是质量与成本效率的最佳组合，使其在受测视频理解模型中处于准确率与成本之间的帕累托前沿。
使用智能体视频理解使 Gemini 3.7 Flash 在视频分析中处于准确率与成本的帕累托前沿。
工作原理
与模型以固定帧率摄取媒体流的静态处理方式不同，智能体视频理解使 Gemini 能够扮演主动的、目标导向的角色，自主决定看什么、以什么速度看、通过哪种模态（画面帧、音频或文字记录）来看，只获取所需的时刻和信号。此前开发者可以手动完成这项工作，而借助智能体视频理解，Gemini 可以通过智能体循环来完成，调用内部工具加载视频文件的相应部分，从而大幅降低开发开销。
能力与应用场景
智能体视频理解改变了开发者在各种高要求应用中处理长视频内容的方式。
- 亚秒级时刻检索：精准定位在 1 FPS 下极易被忽略的瞬间状态变化和紧凑剪辑边界，使精确的自动化视频剪辑成为可能。
- 长视频中的大海捞针式搜索：无需消耗数百万 token，即可回答跨数小时视频的复杂查询。
- 异常检测：以更高 FPS 重新采样感兴趣的时间窗口，以检查快速运动和细微的视觉伪影。
- 动作与物体计数：随时间准确追踪重复的肢体动作和不同的物体。
高 token 效率的长视频分析
了解 Gemini 3.7 Flash 在 LongVideoBench（一个长视频理解基准）上，启用与未启用智能体视频理解时的表现差异。请注意其显著的 token 消耗降低和准确率提升。
通过动态帧率实现精准快速的动态分析
借助智能体视频理解，3.7 Flash 能够根据需要以不同的每秒帧数扫描和重看视频，从而准确计数快速动作。
高 token 效率的“大海捞针”式搜索
通过智能体视频理解，Gemini 3.7 能够基于视频内容准确回答复杂问题，同时与静态分析相比消耗的 token 数量显著减少。
实际应用成果
我们的许多早期访问合作伙伴在测试智能体视频理解时都看到了强劲的性能表现。以下是他们的评价：
快速开始
智能体视频理解现已通过 Gemini API 在 Google AI Studio 和 Gemini Enterprise Agent Platform 中提供，覆盖 Gemini 3.7 Flash、3.6 Flash 和 3.5 Flash-Lite 模型。该功能采用标准 Gemini API token 定价，不额外收取功能费用。
要启用该功能，只需在 API 配置中将处理模式设置为“agentic”即可。阅读我们的 开发者指南，深入了解该功能及其入门方法。
from google import genai
client = genai.Client()
interaction = client.interactions.create(
model="gemini-3.7-flash",
input=[
{
"type": "video",
"uri": "https://youtu.be/7Z5Vy9JBANs",
"processing": "agentic"
},
{
"type": "text",
"text": "What are the 3 most important announcements in this keynote?",
},
],
)
print(interaction.output_text)
我们还正在将智能体视频理解在效率和画质方面的改进带给 Google 产品中数十亿用户。该功能将很快在 Gemini 应用中面向所有用户推出，覆盖 Flash 和 Flash-Lite 模型。此外，在未来几个月内，智能体视频理解还将为 YouTube 视频观看页面上的“Ask YouTube”功能提供支持，借助 Gemini 提供基于画面内容的高质量回答。
感谢以下人员对本工作的贡献：Sergi Caelles、Filip Pavetić、Ahmet Iscen、Suhas Yogin 以及 Agentic Vision 团队。
在您的收件箱中获取来自 Google 的最新资讯
Our new agentic feature for video analysis cuts token consumption by up to 88%, reduces costs by up to 66%, and boosts quality by up to 7%.
Rohan Doshi
Mario Lučić
Research Director, Google DeepMind
Google just launched agentic video understanding for Gemini 3.7 Flash, 3.6 Flash, and 3.5 Flash-Lite. This feature allows the model to dynamically scan video segments, which improves accuracy while cutting token usage by up to 88% and costs by up to 66%. You can start using this today by setting your API configuration to "agentic" in Google AI Studio or the Gemini Enterprise Agent Platform.
Today, we’re launching agentic video understanding across our latest models: Gemini 3.7 Flash, 3.6 Flash and 3.5 Flash-Lite. This new capability improves accuracy while dramatically reducing token usage and costs for video analysis. Similar to agentic vision, which combines code execution with Gemini models’ native image understanding, agentic video understanding uses Gemini’s native video tools to improve performance and unlock new capabilities for video processing like sub-second moment retrieval, more accurate anomaly detection, precise counting and more.
The feature is available today for video uploads and YouTube videos via the Gemini API in Google AI Studio and the Gemini Enterprise Agent Platform.
Benchmarks
Unlike current ‘static’ processing, where the model ingests the video at a fixed frames-per-second rate (default 1 FPS, adjustable via API), agentic video understanding pairs the model’s core reasoning with native video tools to dynamically search, scan, and inspect target video segments across visual frames, audio, and transcripts. Across standard video analysis benchmarks, Gemini models with agentic video understanding reduce analysis costs by up to 66% and token consumption by up to 88%, while improving accuracy by up to 7%.
These efficiency gains are especially pronounced on long-form video (from 10-minute how-to guides to 90-minute lectures and multi-hour recordings), where static processing forces developers to choose between high token costs or techniques that drop critical details.
Activating agentic video understanding drops token consumption by up to 88% and boosts accuracy by up to 7% with Gemini 3.7 Flash.
While these gains span all three supported models, Gemini 3.7 Flash with agentic understanding offers the best possible quality overall and the best combination of quality and cost efficiency, putting it at the accuracy-to-cost pareto frontier among tested models for video understanding.
Using agentic video understanding places Gemini 3.7 Flash at the accuracy-to-cost pareto frontier for video analysis.
How it works
Instead of static processing where the model ingests media streams at a fixed frame rate, agentic video understanding enables Gemini to take an active, goal-directed role in determining what to watch, at what speed, and through which modality (frames, audio, or transcript), fetching only the moments and signals needed. While developers could previously do this manually, with agentic video understanding, Gemini can accomplish it through an agentic loop, invoking an internal tool to load the relevant part of the video file, significantly reducing development overheads.
Capabilities and use cases
Agentic video understanding transforms how developers can process long-form video content across a variety of demanding applications.
- Sub-second moment retrieval: Pinpoint split-second state changes and tight cut boundaries that are easily missed at 1 FPS, making precise automated video editing possible.
- Long-form needle-in-a-haystack search: Answer complex queries across multi-hour videos without consuming millions of tokens.
- Anomaly detection: Resample interesting time windows at higher FPS to inspect rapid motion and subtle visual artifacts.
- Counting action & object: Accurately track repeated physical movements and distinct objects over time.
Token-efficient long-form video analysis
See how Gemini 3.7 Flash performs with and without agentic video understanding on LongVideoBench, a long-form video understanding benchmark. Notice the large token reductions and accuracy improvements.
Accurate fast action analysis with dynamic FPS
With agentic video understanding, 3.7 Flash is able to accurately count a fast-paced movement by scanning and rewatching the video at different frames per second, as needed.
Token-efficient needle-in-a-haystack search
Using agentic video understanding, Gemini 3.7 is able to accurately answer complex questions based on the content of the video while consuming a significantly lower number of tokens compared to static analysis.
Real-world results
Many of our early access partners saw strong performance while testing with agentic video understanding. Here’s what they have to say:
Getting started
Agentic video understanding is available via the Gemini API in Google AI Studio and Gemini Enterprise Agent Platform, launching across Gemini 3.7 Flash, 3.6 Flash, and 3.5 Flash-Lite. It uses standard Gemini API token pricing with no additional feature fee.
To enable it, simply set processing to "agentic" in the API configuration. Read our developer guide to get more insights into the feature and how to get started.
from google import genai
client = genai.Client()
interaction = client.interactions.create(
model="gemini-3.7-flash",
input=[
{
"type": "video",
"uri": "https://youtu.be/7Z5Vy9JBANs",
"processing": "agentic"
},
{
"type": "text",
"text": "What are the 3 most important announcements in this keynote?",
},
],
)
print(interaction.output_text)
We are also bringing the efficiency and quality improvements of agentic video understanding to billions of users across Google products. The feature will roll out to all users in the Gemini app across Flash and Flash-Lite models soon. And in the coming months, agentic video understanding will also power YouTube's ‘Ask YouTube’ feature on the video watch page, leveraging Gemini to deliver higher-quality answers grounded in the visuals.
Acknowledgement for their contribution to this work:Sergi Caelles, Filip Pavetić, Ahmet Iscen, Suhas Yogin, and the Agentic Vision team.