Author(s)

Yiwen Jin ¹, Rong Zhang ¹, Yisu Hu ¹, Hongliang Luo ¹, Yongqiang Bai ¹

Affiliation(s)

¹ Zhejiang Wanli University, Ningbo, Zhejiang, 315000, China

Corresponding Author

Yiwen Jin

ABSTRACT

For the existing 3D small object detection is prone to false detection and missed detection and other deficiencies. A 3D object detection method based on multi-modal feature fusion is proposed. Firstly, a feature extraction module is designed. The input image data is down-sampled through the image feature extraction network, and the input point cloud data is sampled and grouped through the point cloud feature extraction network to obtain the feature information at different scales. Secondly, a multi-modal feature fusion module is constructed to realize the point correspondence between point cloud features and image features by projection operation, and then the image features and point cloud features are splicing and fused to generate the final point cloud features to compensate the deficiency of single modal feature information. The experimental results show that compared with the existing algorithms, the algorithm in this paper improves the average detection accuracy of small object by 2.03%.

KEYWORDS

Multi-modal; 3D Object Detection; Feature Fusion; point cloud; image

CITE THIS PAPER

Yiwen Jin, Rong Zhang, Yisu Hu, Hongliang Luo, Yongqiang Bai, Multi-modal Feature Fusion 3D Object Detection. Advances in Computer, Signals and Systems (2023) Vol. 7: 105-112. DOI: http://dx.doi.org/10.23977/acss.2023.070812.

REFERENCES

[1] Zhang Peng, Song Yifan, Zong Libo, et al. Advances in 3D Object Detection: A Brief Survey[J]. Computer Science, 2020, 47(4):94-102. 
[2] Huang Zhe, Wang Yongcai, Li Deying. A survey of 3D object detection algorithms [J]. Chinese Journal of Intelligent Science and Technology, 2023, 5(01):7-31. 
[3] Garrick Brazil, Xiaoming Liu. M3D-RPN: Monocular 3D Region Proposal Network for Object Detection[C]// IEEE/ CVF International Conference on Computer Vision (ICCV), 2019: 9286-9295. 
[4] Yunpeng Zhang, Jiwen Lu, Jie Zhou. Objects are Different: Flexible Monocular 3D Object Detection[C]//Proceedings of the IEEE Computer Vision and Pattern Recognition(CVPR), 2021: 3289-3298. 
[5] Charles R. Qi, Hao Su, Kaichun Mo, et al. PointNet:Deep Learning on Point Sets for 3D Classification and Segmentation [C]// Proceedings of the IEEE Computer Vision and Pattern Recognition(CVPR), 2017:652-660. 
[6] Charles R. Qi, Li Yi, Hao Su, et al. PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space [C]//Proceedings of the Advances in Neural Information Processing Systems(NIPS), 2017:5105-5114. 
[7] Yan Yan, Yuxing Mao, Bo Li. SECOND: Sparsely Embedded Convolutional Detection[J]. Sensors, 2018, 18(10): 3337. 
[8] Gao Yue, Dai Meng, Zhang Qing. RGB-D Salient Object Detection Based on Multi-modal Feature Interaction [J]. Computer Engineering and Applications, 2022:1-11. 
[9] Wei Liang, Pengfei Xu, Ling Guo. A survey of 3D object detection[J]. Multimedia Tools and Applications, 2021, 80(19): 29617-29641. 
[10] Alex H. Lang, Sourabh Vora, Holger Caesar, et al. PointPillars: Fast Encoders for Object Detection from Point Clouds [C] //Proceedings of the IEEE Computer Vision and Pattern Recognition(CVPR), 2019: 12697-12705. 
[11] Yin Zhou, Oncel Tuzel. VoxelNet: End-to-End Learning for Point Cloud Based 3D Object Detection[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2018:4490-4499. 
[12] Qiangeng Xu, Yiqi Zhong, Ulrich Neumann. Behind the Curtain: Learning Occluded Shapes for 3D Object Detection [C] //Association for the Advancement of Artificial Intelligence, 2021.

Sponsors, Associates, and Links

---

• Power Systems Computation
  
  
• Internet of Things (IoT) and Engineering Applications
  
  
• Computing, Performance and Communication Systems
  
  
• Journal of Artificial Intelligence Practice
  
  
• Journal of Network Computing and Applications
  
  
• Journal of Web Systems and Applications
  
  
• Journal of Electrotechnology, Electrical Engineering and Management
  
  
• Journal of Wireless Sensors and Sensor Networks
  
  
• Journal of Image Processing Theory and Applications
  
  
• Mobile Computing and Networking
  
  
• Vehicle Power and Propulsion
  
  
• Frontiers in Computer Vision and Pattern Recognition
  
  
• Knowledge Discovery and Data Mining Letters
  
  
• Big Data Analysis and Cloud Computing
  
  
• Electrical Insulation and Dielectrics
  
  
• Crypto and Information Security
  
  
• Journal of Neural Information Processing
  
  
• Collaborative and Social Computing
  
  
• International Journal of Network and Communication Technology
  
  
• File and Storage Technologies
  
  
• Frontiers in Genetic and Evolutionary Computation
  
  
• Optical Network Design and Modeling
  
  
• Journal of Virtual Reality and Artificial Intelligence
  
  
• Natural Language Processing and Speech Recognition
  
  
• Journal of High-Voltage
  
  
• Programming Languages and Operating Systems
  
  
• Visual Communications and Image Processing
  
  
• Journal of Systems Analysis and Integration
  
  
• Knowledge Representation and Automated Reasoning
  
  
• Review of Information Display Techniques
  
  
• Data and Knowledge Engineering
  
  
• Journal of Database Systems
  
  
• Journal of Cluster and Grid Computing
  
  
• Cloud and Service-Oriented Computing
  
  
• Journal of Networking, Architecture and Storage
  
  
• Journal of Software Engineering and Metrics
  
  
• Visualization Techniques
  
  
• Journal of Parallel and Distributed Processing
  
  
• Journal of Modeling, Analysis and Simulation
  
  
• Journal of Privacy, Trust and Security
  
  
• Journal of Cognitive Informatics and Cognitive Computing
  
  
• Lecture Notes on Wireless Networks and Communications
  
  
• International Journal of Computer and Communications Security
  
  
• Journal of Multimedia Techniques
  
  
• Automation and Machine Learning
  
  
• Computational Linguistics Letters
  
  
• Journal of Computer Architecture and Design
  
  
• Journal of Ubiquitous and Future Networks