PhysBrain 1.5:从视觉语言模型到物理基础模型

HuggingFace Daily Papers(社区热门论文)·2026-09-14 08:00·1天前
AI 导读

PhysBrain 1.5 是一个统一理解物理环境、生成动作并预测未来状态的模型,从通用视觉语言模型出发,将语言响应、末端执行器运动和稠密视觉目标编码为离散序列,以自回归下一 token 预测联合优化。

HuggingFace Daily Papers(社区热门论文)
48AI 编辑部评分,满分 100

PhysBrain 1.5:从视觉语言模型到物理基础模型

2026-09-14 08:00· 1天前
AI 导读

PhysBrain 1.5 是一个统一理解物理环境、生成动作并预测未来状态的模型,从通用视觉语言模型出发,将语言响应、末端执行器运动和稠密视觉目标编码为离散序列,以自回归下一 token 预测联合优化。

We present PhysBrain 1.5, a unified model for understanding physical environments, generating actions, and predicting future states. Motivated by the physical loop of observation, interaction, and environmental change, we bring these capabilities into a common learning framework. Starting from a general vision--language model, we encode language responses, end-effector motion, and dense visual targets as discrete sequences and jointly optimize them with autoregressive next-token prediction. Pre-training draws its embodied supervision entirely from human interaction videos, using task-centered episodes to pair semantic and spatial context with recovered motion and subsequent observations.

We then adapt the model through supervised fine-tuning on a mixture of human demonstrations, robot trajectories, and simulated experience. Across 28 embodied understanding benchmarks, our 8B model achieves an average score of 72.5, setting a new open-source state of the art and performing on par with leading proprietary models such as GPT-6-Astra and Gemini 3.6 Flash. It achieves the best open-source results on 14 benchmarks while retaining general multimodal capabilities. Beyond these understanding evaluations, qualitative examples show the model's ability to produce end-effector trajectories and predict future scenes through spatially aligned RGB, depth, and robot-mask outputs.

来源:HuggingFace Daily Papers(社区热门论文)· arxiv.org