LDFA: Lightweight Dynamic Feature Aggregation for Multi-Modal Fusion Hangzhou City University

Format:

Book

Conference/Event

Author/Creator:

Feng, Xiaoyu, author.

Contributor:

Bian, Chen

Chu, Zheng

Duan, Linshuai

Wei, Lina

Zhang, Renhang

Conference Name:

SAE 2024 Intelligent Urban Air Mobility Symposium (2024-09-06 : Hangzhou, China)

Language:

English

Physical Description:

1 online resource cm

Place of Publication:

Warrendale, PA SAE International 2024

Summary:

Integrating 3D point cloud and image fusion into flying car detection systems is essential for enhancing both safety and operational efficiency. Accurate environmental mapping and obstacle detection enable flying cars to optimize flight paths, mitigate collision risks, and perform effectively in diverse and challenging conditions. The AutoAlignV2 paradigm recently introduced a learnable schema that unifies these data formats for 3D object detection. However, the computational expense of the dynamic attention alignment mechanism poses a significant challenge. To address this, we propose a Lightweight Cross-modal Feature Dynamic Aggregation Module, which utilizes a model-driven feature alignment strategy. This module dynamically realigns heterogeneous features and selectively emphasizes salient aspects within both point cloud and image datasets, enhancing the differentiation between objects and the background and improving detection accuracy. Additionally, we introduce the Lightweight Spatial-Reduction Attention (LSRA) layer to enhance the original attention mechanism. By employing spatial reduction and positional offset techniques, LSRA reduces computational complexity, accelerating the aggregation of cross-modal features while minimizing computational overhead. Furthermore, we implement a novel dropout scheme before extracting features from 2D images, enhancing the model's generalization capabilities and reducing computational costs. We present a new lightweight frameworkLightweight Dynamic Feature Aggregation for Multi-modal Fusion (LDFA)designed specifically for the harmonious fusion of 3D point cloud data and 2D image-derived information. The LDFA framework achieves a meticulous balance between computational efficiency and enhanced perceptual capabilities. Extensive experimental evaluations on the nuScenes benchmark dataset confirm the efficacy and efficiency of the LDFA fusion strategy, demonstrating its potential to redefine the state-of-the-art in multimodal 3D object detection. Code will be available at https://github.com/zishenjiucai/LDFA

Notes:

Vendor supplied data

Publisher Number:

2024-01-7008

Access Restriction:

Restricted for use by site license