Unified Multi-Modal Multi-Agent Cooperative Perception Framework for Intelligent Transportation Systems University of California Los Angeles

Format:

Book

Conference/Event

Author/Creator:

Meng, Zonglin, author.

Contributor:

Gao, Letian

Liu, Wei

Ma, Jiaqi

Xia, Xin

Zheng, Zhaoliang

Zhu, Jiaqi

Conference Name:

SAE 2024 Intelligent and Connected Vehicles Symposium (2024-09-22 : Shanghai, China)

Language:

English

Physical Description:

1 online resource cm

Place of Publication:

Warrendale, PA SAE International 2024

Summary:

Cooperative perception has attracted wide attention given its capability to leverage shared information across connected automated vehicles (CAVs) and smart infrastructure to address the occlusion and sensing range limitation issues. To date, existing research is mainly focused on prototyping cooperative perception using only one type of sensor such as LiDAR and camera. In such cases, the performance of cooperative perception is constrained by individual sensor limitations. To exploit the multi-modality of sensors to further improve distant object detection accuracy, in this paper, we propose a unified multi-modal multi-agent cooperative perception framework that integrates camera and LiDAR data to enhance perception performance in intelligent transportation systems. By leveraging the complementary strengths of LiDAR and camera sensors, our framework utilizes the geometry information from LiDAR and the semantic information from cameras to achieve an accurate cooperative perception system. In order to fuse the multi-agent and multi-modal features, we use a bird's-eye view (BEV) space as the consistent and unified feature representations and employ a transformer-based network for effective multi-agent multi-modal BEV feature fusion. We validate our method on the OPV2V and V2XSet benchmarks, achieving state-of-the-art performance in 3D cooperative perception tasks. The proposed framework significantly improves object detection accuracy and robustness, especially in complex traffic scenarios with occlusions such as dense intersections

Notes:

Vendor supplied data

Publisher Number:

2024-01-7028

Access Restriction:

Restricted for use by site license