Autonomous vehicles in future intelligent transportation systems need to do more than navigate safely on their own. In complex scenarios such as unsignalized intersections, roundabouts, and bottleneck roads, they must also coordinate with surrounding vehicles to improve overall safety and efficiency. However, existing end-to-end driving methods mainly focus on individual navigation, while many multi-agent reinforcement learning methods only model local or neighborhood interactions. This makes it difficult to jointly optimize individual driving performance and system-level cooperation.
To address this challenge, researchers from the National University of Singapore, The Hong Kong Polytechnic University, and Tsinghua University developed COIN, a collaborative interaction-aware reinforcement learning framework for multi-agent self-driving systems. COIN follows a centralized training and decentralized execution paradigm. During training, vehicles learn cooperative strategies using global information; during execution, each vehicle makes decisions based only on its local observations, making the framework suitable for decentralized autonomous driving.
The core of COIN is counterfactual individual-global TD3, or CIG-TD3 algorithm, which jointly optimizes individual navigation and global collaboration objectives. A counterfactual baseline is introduced to estimate each vehicle’s contribution to the overall system performance, helping the learning algorithm assign credit more effectively. COIN also uses a dual-level interaction-aware centralized critic. Conditional variational autoencoders capture pairwise interactions between vehicles, while graph attention networks model system-level dependencies across the traffic scene.
The framework was evaluated in MetaDrive under three dense traffic scenarios: unsignalized intersections, roundabouts, and bottleneck roads. COIN outperformed IPPO, ITD3, CoPO, and TraCo in success rate, collision rate, safety, and efficiency. In the intersection scenario, COIN achieved an 88.78% success rate and reduced the collision rate to 3.94%. In the bottleneck scenario, it achieved a 96.33% success rate with only a 1.57% collision rate.
COIN was further validated on a hybrid real-simulation robotic platform. In a mock unsignalized intersection, eight Mecanum-wheeled mobile robots followed trajectories generated by COIN, while other vehicles were represented as virtual agents in simulation. This setup demonstrated that COIN can support safe and efficient cooperative navigation in complex multi-robot interaction scenarios.
By shifting multi-agent autonomous driving from independent decision-making toward cooperative optimization, COIN provides a promising direction toward safer and more efficient autonomous driving in dense urban traffic.
The team published their study in Communications in Transportation Research ( https://doi.org/10.26599/COMMTR.2026.9640031 ).
About Communications in Transportation Research
Communications in Transportation Research was launched in 2021, with academic support provided by Tsinghua University and China Intelligent Transportation Systems Association. The Editors-in-Chief are Professor Xiaobo Qu, a member of the Academia Europaea from Tsinghua University, and Professor Xiaopeng (Shaw) Li from University of Wisconsin–Madison. The journal mainly publishes high-quality, original research and review articles that are of significant importance to emerging transportation systems, aiming to serve as an international platform for showcasing and exchanging innovative achievements in transportation and related fields, fostering academic exchange and development between China and the global community.
It has been indexed in SCIE, SSCI, Ei Compendex, Scopus, CSTPCD, CSCD, OAJ, DOAJ, TRID and other databases. It was selected as Q1 Top Journal in the Engineering and Technology category of the Chinese Academy of Sciences (CAS) Journal Ranking List. In 2022, it was selected as a High-Starting-Point new journal project of the “China Science and Technology Journal Excellence Action Plan”. In 2024, it was selected as the Support the Development Project of “High-Level International Scientific and Technological Journals”. The same year, it was also chosen as an English Journal Tier Project of the “China Science and Technology Journal Excellence Action Plan PhaseⅡ”. In 2024, it received the first impact factor (2023 IF) of 12.5, ranking Top1 (1/58, Q1) among all journals in "TRANSPORTATION" category. In 2026, its 2025 IF was announced as 12.7, maintaining the Top1 position (1/66, Q1) in the same category.
From Volume 6 (2026), Communications in Transportation Research will be published by Tsinghua University Press on the SciOpen platform with the official journal website at https://www.sciopen.com/journal/2097-5023 . We kindly request that all new manuscript submissions be made through the journal’s submission system at https://mc03.manuscriptcentral.com/commtr . For any submission-related inquiries, please contact the Editorial Office at commtr_e@mail.tsinghua.edu.cn.
Communications in Transportation Research
COIN: Collaborative Interaction-Aware Multi-Agent Reinforcement Learning for Self-Driving Systems
30-Sep-2026