Qwen 发布 Qwen-Drive-1.0：面向自动驾驶的视觉语言基础模型
Qwen-Drive-1.0: An Initial Step towards a Vision-Language Foundation Model for Autonomous Driving
Qwen 团队发布 Qwen-Drive-1.0，称其为首个在预训练阶段统一 3D 感知与视觉问答、并延伸到运动规划的自动驾驶视觉语言基础模型，基于原生多模态 Qwen3.5-4B 且不改动预训练架构。
We introduce Qwen-Drive-1.0, the first vision-language foundation model for autonomous driving that unifies 3D perception and visual question answering at the pretraining stage and further extends to motion planning, while keeping the pretrained VLM architecture entirely untouched. Built on the natively multimodal Qwen3.5-4B, it attaches two external modules. A BEV perception head serves as an explicit, inspectable 3D probe, jointly performing 3D object detection, semantic occupancy prediction, and BEV map segmentation, and a Planning Expert generates future ego trajectories through flow matching. Through staged training, we substantially boost the autonomous driving capability of a general-purpose VLM and validate it on 3D perception, driving visual question answering, and motion planning tasks, forming a unified driving vision-language model that offers a new-generation VLM base for driving-scenario adaptation.
Highlights#
Qwen-Drive-1.0 is the first vision-language foundation model for autonomous driving that unifies 3D perception and visual question answering at the pretraining stage and further extends to motion planning, while keeping the pretrained VLM architecture entirely untouched.
An external BEV perception head serves as an explicit, inspectable 3D probe, jointly learning 3D detection, semantic occupancy prediction, and BEV map segmentation, equipping the same pretrained VLM with clear perception outputs while preserving highly competitive vision-language performance.
A staged training and data recipe unifies cross-dataset labels, rewrites responses, filters samples for consistency, and combines driving data with general-purpose vision-language supervision, supporting domain adaptation while mitigating catastrophic forgetting.
A Planning Expert tailored to pretrained VLM representations generates future ego trajectories with flow matching. Unified trajectory annotations enable joint training across multiple public driving datasets and yield highly competitive results across open-loop, pseudo-closed-loop, and closed-loop evaluations.
Model Architecture#
Qwen-Drive-1.0 builds on the natively multimodal Qwen3.5-4B. A shared vision encoder and VLM process single-view and multi-view driving images, temporal image sequences, and general images. Without changing the pretrained architecture, two external modules read from this shared pathway. The BEV perception head builds a BEV representation from multi-view single-frame inputs and jointly performs 3D object detection, semantic occupancy prediction, and BEV map segmentation. It acts as an explicit, inspectable 3D probe, and its losses provide an additional gradient path into the shared visual pathway during joint training. The Planning Expert is a diffusion transformer tailored to VLM representations. It generates 5-second ego trajectories through flow matching, with an optional textual planning reason as condition. Perception, question answering, and planning thus reside in one pretrained VLM.
Driving Scene Understanding without Losing General Capability#
Qwen-Drive-1.0-SFT reaches a driving QA average of 69.43, leading both general-purpose VLMs and driving or embodied specialists, and demonstrating its strong driving scene understanding capability.
InternVL3.5-8B-Inst.LLaVA-OV2-8BQwen3.5-4BCosmos-Reason2-8BCosmos3-nanoMiMo-Embodied-7BAlpamayo-1.5-10BQwen-Drive-1.0-SFT
Driving VQA
LingoQA46.4041.2070.4059.6065.0072.0064.0077.80
Ego3D RMSE ↓23.0124.9713.1712.6222.419.8525.317.78
VLAD54.4758.7165.3856.3757.7350.339.1366.52
SURDS32.8038.6052.9519.5439.7243.063.1066.13
WaymoQA Safety54.4749.6562.4657.6856.9366.5442.6170.70
WaymoQA All58.0955.2367.1057.9358.3669.5644.3774.47
CoC All--0.572.581.724.01--3.4441.26
Knowledge, Reasoning, and Recognition
MMBench80.0382.6687.0782.8279.57--7.5185.53
MMStar64.1364.9375.3365.2766.6722.4026.1375.87
MMMU62.0054.6773.4459.1160.89--27.4472.67
MMMU-Pro Std46.4236.3064.8636.0746.3627.4015.6162.72
MMMU-Pro Vis42.2525.9561.2743.5340.7528.0913.4759.71
CharXiv41.7040.1065.1042.5042.1057.501.5064.40
OCRBench83.2079.3086.9087.0085.2078.803.2086.40
RealWorldQA66.9371.7676.3467.4569.6728.5046.9378.95
SimpleVQA40.7736.6847.8445.2544.99----46.12
CountQA20.9422.5835.8622.3223.6322.644.7131.74
Spatial Understanding and Grounding
EmbSpatial74.2078.4375.9977.6177.8845.0520.5878.85
ERQA42.0042.2546.2543.2541.2539.7527.5048.50
RefSpatial----54.5151.81--2.17--50.78
Omni3D----47.4032.8532.26----45.79
ODinW13----40.7840.1935.87----45.87
- All benchmarks use the same high-certainty decoding settings (greedy=false, top-p=0.001, top-k=1, temperature=0.01, repetition_penalty=1.0, presence_penalty=0.0) to more directly reflect model capability.
- LingoQA is scored with Qwen-Plus as the judge instead of the official LingoJudge, which we found to score leniently and inconsistently across scenarios. Under the official LingoJudge protocol, Qwen-Drive-1.0-SFT obtains a LingoScore of 79.4.
- The same judge scores every method on each benchmark.
- -- marks an invalid or unparsable response.
Motion Planning on Open-Loop and Closed-Loop#
Qwen-Drive-1.0 demonstrates outstanding performance in motion planning, both in open-loop and closed-loop settings. The training is entirely based on publicly available data, comprising a total of 2.83 million samples. Due to differences in annotation styles across various datasets, we unified the trajectory format to achieve stable 5-second trajectory predictions at 10 Hz.
AutoVLASpanVLAMindVLA-U1Alpamayo-1.5SimWAMILQwen-Drive-1.0-SFTQwen-Drive-1.0-RL
Open-loop
WOD-E2E (RFS val/test ↑)--/7.56--8.20/7.87----7.95/7.788.45/7.91
WOD-E2E (ADE 5s val/test ↓)--/2.96--2.28/2.66----2.31/2.651.27/2.67
PAI-AV (Avg. ADE 3s ↓)------0.350.410.370.42
PAI-AV (Avg. ADE 5s ↓)------1.05--1.071.11
Pseudo-closed-loop
NAVSIM (PDMS ↑)89.690.3----90.388.290.7
NAVSIM best-of-6 (PDMS ↑)----------89.391.4
Closed-loop
AlpaSim (at-fault score ↑)------0.450.300.270.37
- AutoVLA and SimWAM train a separate model on each dataset.
- The SFT column reports Qwen-Drive-1.0-SFT conditioned on planning reasoning.
- IL denotes imitation learning.
- -- indicates that the method does not report a result on the corresponding benchmark.
What’s Next#
We consider Qwen-Drive-1.0 an initial step towards a vision-language foundation model for autonomous driving. Specifically, we introduce a BEV perception head as an explicit, inspectable 3D probe and a Planning Expert that generates future ego trajectories through flow matching. Through staged training, we substantially boost the autonomous driving capability of a general-purpose VLM and validate this route on 3D perception, driving visual question answering, and motion planning, forming a unified driving vision-language model that offers a new-generation VLM base for driving-scenario adaptation. Still, the consistency between textual reasoning and the generated trajectory remains to be strengthened, which we leave as a focus of future work.
@misc{zhou2026qwendrive10initialstepvisionlanguage, title={Qwen-Drive-1.0: An Initial Step towards a Vision-Language Foundation Model for Autonomous Driving}, author={Xin Zhou and Zongchuang Zhao and Zhibo Yang and Mingsheng Li and Humen Zhong and Shuai Bai and Du Chu and Ruizhe Chen and Zhaohai Li and Jun Tang and Qiuyue Wang and Mingkun Yang and Jiazhao Zhang and Dayiheng Liu and Dingkang Liang and Xiang Bai}, year={2026}, eprint={2609.00111}, archivePrefix={arXiv}, primaryClass={cs.CV}, url={https://arxiv.org/abs/2609.00111}, }
来源：Qwen：Blog Retrieval · qwen.ai