Alibaba Releases Open-Source Vision-Language Model for Autonomous Driving
Alibaba's Qwen team has released Qwen-Drive-1.0-4B, an open-source vision-language model for autonomous driving. This new model, built upon a Qwen3.5-4B base, integrates 3D perception and vehicle trajectory planning capabilities. Its components are available under an Apache 2.0 license, with variants accessible via Hugging Face and GitHub for developers.
The Qwen-Drive-1.0-4B system, developed in collaboration with Huazhong University of Science and Technology, maintains the existing Qwen3.5-4B multimodal backbone. The design incorporates specific modules for a perspective from above 3D perception and for generating the ego trajectory, without alterations to the underlying vision language architecture. This approach permits scene understanding to be paired with motion planning functions directly within the model.
Weights, along with inference code and demonstration data, are publicly available on platforms such as Hugging Face and GitHub. But the core vision language model continues to process free form questions related to driving scenes. An external perception head, providing a perspective from above, operates in conjunction, carrying out 3D object detection, semantic occupancy prediction, and map segmentation simultaneously. According to Pandaily, the Qwen team describes this head as an explicit, inspectable 3D probe of shared representations, rather than a separate specialist network that would either replace the VLM or obscure intermediate geometry from devs.
Motion planning functions are managed by a Planning Expert module. This expert processes features from the VLM and generates future ego trajectories through a flow matching process. The distributed package includes two distinct planner checkpoints. The planner-sft variant has undergone imitation training and supports both direct and reasoning style planning methodologies.
And the planner-rl variant receives further reward optimisation based on metrics such as NAVSIM PDMS and Waymo Open Dataset end to end scores. This variant is specifically recommended for reasoning planning mode. The team utilised unified trajectory annotations, which enabled training across multiple public driving datasets while ensuring consistent formats for evaluation.
A staged training programme combines driving supervision with general purpose vision language data. This methodology aims to enhance the model's domain competence whilst simultaneously limiting the potential for catastrophic forgetting of broader visual skills. Published data indicates the 4B class system performs well in open loop prediction, pseudo closed loop, and closed loop evaluations.
So the model shows strong driving visual question answering results compared to larger general VLMs. Its general knowledge and spatial benchmarks largely align with the foundational Qwen3.5-4B system, as detailed in the project card and technical report materials. The Big Tech organisation has made these resources widely accessible.
The repository layout places the VLM at the primary level, with the planner-sft, planner-rl, and perception heads situated in adjacent folders. This arrangement allows for selective download of specific components. Devs can execute visual question answering independently, integrate perception capabilities, or load a planner for bundled demonstration scenarios without requiring proprietary fleets or private logs.
But Qwen frames this project as an initial public contribution towards a unified vision language foundation for research in autonomous driving. The organisation encourages replication, ablation studies, and further extensions, rather than presenting a complete production stack. Future community contributions may assess the Planning Expert under varying sensor layouts, preserving the unaltered Qwen3.5-4B trunk as a stable base for instruction following and scene dialogue.
Alibaba's Qwen team has open-sourced Qwen-Drive-1.0-4B for autonomous driving.
The model integrates 3D perception and trajectory planning on a Qwen3.5-4B base.
Weights, code, and demonstration data are available under an Apache 2.0 license.
Two planner variants, planner-sft and planner-rl, are provided for motion planning.
The project aims to establish a foundational vision-language system for autonomous driving research.
Source: Pandaily
