AlayaVista:从全景状态到透视视频的流式世界建模

HuggingFace Daily Papers(社区热门论文)·2026-09-13 08:00·3天前
AI 导读

AlayaVista 是一个相机可控的流式视频世界模型,将全景世界演化与透视观测合成解耦,给定单张透视图像即可构建 360 度场景先验并演化场景,输出透视视频。团队同时构建了 MUGEN 数据集,含 1,318 小时、分辨率至少 4K 的真实全景视频及语义与几何标注。

HuggingFace Daily Papers(社区热门论文)
45AI 编辑部评分,满分 100

AlayaVista:从全景状态到透视视频的流式世界建模

2026-09-13 08:00· 3天前
AI 导读

AlayaVista 是一个相机可控的流式视频世界模型,将全景世界演化与透视观测合成解耦,给定单张透视图像即可构建 360 度场景先验并演化场景,输出透视视频。团队同时构建了 MUGEN 数据集,含 1,318 小时、分辨率至少 4K 的真实全景视频及语义与几何标注。

Interactive video world models must maintain broad scene context under camera motion while producing high-fidelity observations with low latency. Existing approaches face a representation trade-off: perspective models operate on local views and must preserve off-screen content over long rollouts, whereas broader spatial coverage is typically obtained by synthesizing full-sphere videos or constructing explicit 3D representations. Motivated by the complementary roles of global context and selective local acuity in visual perception, we present AlayaVista, a camera-controllable streaming video world model that decouples panoramic world evolution from perspective observation synthesis.

Given a single perspective image, AlayaVista constructs a 360-degree scene prior using a pretrained panorama expansion model and then evolves the scene as a camera-conditioned panoramic latent state. A latent viewport renderer maps this state to the requested perspective video latents, while a perspective refiner restores details, suppresses artifacts, and performs super-resolution. To support efficient streaming, we adapt the panoramic generator to chunk-autoregressive generation and distill both panoramic generation and perspective refinement into few-step processes.

To provide the supervision required by this design, we construct MUGEN, a large-scale real-world panoramic video dataset containing 1,318 hours of videos at resolutions of at least 4K, together with rich semantic and geometric annotations.

来源:HuggingFace Daily Papers(社区热门论文)· arxiv.org