Can vision-language models understand unstructured road scenes under degraded sensing conditions? Researchers developed the Unstructured Scene Benchmark (USB), a test and evaluation framework for autonomous driving models in unstructured road scenes and degraded visual conditions. By testing seven open-source vision-language models across six camera views, 20 corruption types, and 45,969 question-answer samples, the study reveals uneven robustness, weak planning ability, and signs of insufficient visual grounding.
The team published their study in Communications in Transportation Research ( https://doi.org/10.26599/COMMTR.2026.9640034 ).
Vision-language models (VLMs) are increasingly used to connect visual perception, language reasoning, and decision support in autonomous driving. However, many existing benchmarks focus on structured urban roads and relatively clean imagery. Real roads are often less orderly: lane boundaries may be unclear, temporary construction may change traffic rules, vulnerable road users may appear in unusual locations, and camera inputs may be degraded by weather, lighting, sensor faults, or occlusion.
"A model that performs well on clean images does not necessarily remain reliable when the road scene is unstructured or visually degraded," says Yuan Zheng, a researcher at Jiangsu Province Collaborative Innovation Center of Modern Urban Traffic Technologies, Southeast University. "Our goal was to build a test framework that makes these weaknesses visible under controlled and reproducible conditions."
USB reorganizes driving samples into synchronized six-camera inputs and expands each scene with 20 realistic corruption types, including backlight, glare, darkness, fog, rain, snow, sandstorm, smoke, motion blur, zoom blur, compression, quantization, bit errors, frame loss, lens obstruction, splash, and sensor failure. The benchmark then asks driving-related questions covering vulnerable road user perception, object motion prediction, traffic light recognition, environment description, causal explanation, and three-second ego-vehicle planning.
Unlike evaluations that rely only on simple accuracy, USB combines several scoring strategies. Multiple-choice questions are scored by exact matching and aggregated with expert-derived importance weights from 55 autonomous driving professionals. Free-text answers are evaluated with a GPT-4o-based scoring rubric that examines whether responses identify critical factors, remain consistent with visual evidence, explain causal relationships, and describe the road environment clearly.
Seven open-source VLMs with fewer than 8 billion parameters were tested: Qwen2.5-VL-7B-Instruct, Qwen3-VL-8B-Instruct, InternVL3.5-8B-Instruct, Keye-VL-1.5-8B, Phi-3.5-Vision-Instruct, LLaVA-v1.6-Mistral-7B, and LLaVA-v1.5-7B. Qwen3-VL achieved the highest unweighted multiple-choice accuracy at 0.6771 and the best causal explanation score at 8.24. InternVL3.5 achieved the highest weighted multiple-choice accuracy at 0.6660, while Qwen2.5-VL achieved the best road and environment description score at 9.44.
The results show that current VLMs still struggle with higher-level driving reasoning. Perception-related questions were generally easier, while object motion prediction and especially three-second ego planning were much harder. In the three-second planning task, all models scored low, indicating that recognizing scene elements does not automatically translate into reliable planning decisions.
The study also found that model rankings changed across corruption types. Some models remained strong under certain visual degradations but dropped sharply under others. More importantly, several models could still generate fluent and plausible answers when visual evidence was severely degraded or even absent. Under sensor failure, for example, some scores improved rather than declined, suggesting that models may sometimes rely on answer priors or general driving knowledge instead of the actual image content.
"High scores under missing or corrupted visual input should not be mistaken for genuine scene understanding," says Shen Li, a researcher at Department of Civil Engineering, Tsinghua University. "They can instead indicate weak visual grounding, which is a critical issue for safety-related applications."
The researchers argue that autonomous driving VLMs need evaluation settings that go beyond clean-image accuracy and include unstructured environments, visual corruption, task importance, and free-text reasoning quality. USB provides reproducible baselines and a diagnostic framework for identifying when models perceive, reason, or plan unreliably under challenging road conditions.
About Communications in Transportation Research
Communications in Transportation Research was launched in 2021, with academic support provided by Tsinghua University and China Intelligent Transportation Systems Association. The Editors-in-Chief are Professor Xiaobo Qu, a member of the Academia Europaea from Tsinghua University, and Professor Xiaopeng (Shaw) Li from University of Wisconsin–Madison. The journal mainly publishes high-quality, original research and review articles that are of significant importance to emerging transportation systems, aiming to serve as an international platform for showcasing and exchanging innovative achievements in transportation and related fields, fostering academic exchange and development between China and the global community.
It has been indexed in SCIE, SSCI, Ei Compendex, Scopus, CSTPCD, CSCD, OAJ, DOAJ, TRID and other databases. It was selected as Q1 Top Journal in the Engineering and Technology category of the Chinese Academy of Sciences (CAS) Journal Ranking List. In 2022, it was selected as a High-Starting-Point new journal project of the “China Science and Technology Journal Excellence Action Plan”. In 2024, it was selected as the Support the Development Project of “High-Level International Scientific and Technological Journals”. The same year, it was also chosen as an English Journal Tier Project of the “China Science and Technology Journal Excellence Action Plan PhaseⅡ”. In 2024, it received the first impact factor (2023 IF) of 12.5, ranking Top1 (1/58, Q1) among all journals in "TRANSPORTATION" category. In 2026, its 2025 IF was announced as 12.7, maintaining the Top1 position (1/66, Q1) in the same category.
From Volume 6 (2026), Communications in Transportation Research will be published by Tsinghua University Press on the SciOpen platform with the official journal website at https://www.sciopen.com/journal/2097-5023 . We kindly request that all new manuscript submissions be made through the journal’s submission system at https://mc03.manuscriptcentral.com/commtr . For any submission-related inquiries, please contact the Editorial Office at commtr_e@mail.tsinghua.edu.cn.
Communications in Transportation Research
Unstructured Scene Benchmark (USB): Which VLM Performs Better in Autonomous Driving?
30-Sep-2026