IPFA-Net: Important Points Feature Aggregating Net for Point Cloud Classification and Segmentation

WANG Jingya; ZHANG Yu; ZHANG Bin; XIA Jinxiang; WANG Weidong

doi:10.23919/cje.2023.00.065

Article Contents

Article Navigation > Chinese Journal of Electronics > 2025 > Uncorrected proof

Jingya WANG, Yu ZHANG, Bin ZHANG, et al., “IPFA-Net: Important Points Feature Aggregating Net for Point Cloud Classification and Segmentation,” Chinese Journal of Electronics, vol. 34, no. 1, pp. 1–15, 2025 doi: 10.23919/cje.2023.00.065

Citation:

Jingya WANG, Yu ZHANG, Bin ZHANG, et al., “IPFA-Net: Important Points Feature Aggregating Net for Point Cloud Classification and Segmentation,” Chinese Journal of Electronics, vol. 34, no. 1, pp. 1–15, 2025 doi: 10.23919/cje.2023.00.065

Citation:

PDF( 6584 KB)

IPFA-Net: Important Points Feature Aggregating Net for Point Cloud Classification and Segmentation

doi: 10.23919/cje.2023.00.065

WANG Jingya^1
,,
ZHANG Yu^1
,,
ZHANG Bin^2
,,
XIA Jinxiang^{1
,
,},
WANG Weidong^1
,

1.
University of Electronic Science and Technology of China, Chengdu 610054, China
2.
Jilin University, Changchun 130012, China

More Information

Author Bio:
Jingya WANG received the B.S. degree in Computer Science and Technology from Shandong University of Finance and Economics, Jinan, China, in 2020. She is currently pursuing the Ph.D. degree in Software Engineering with University of Electronic Science and Technology of China, Chengdu, China. Her current research interests include deep learning, neural network control, and dynamic analysis. (Email: wjycindy@163.com)

Yu ZHANG received the B.S. degree in Software Engineering from University of Electronic Science and Technology of China, Chengdu, China, in 2022. He is currently pursuing the M.S. degree in Software Engineering with University of Electronic Science and Technology of China, Chengdu, China. His current research interests include computer vision, deep learning, and autonomous driving. (Email: 2857092231@qq.com)

Bin ZHANG received the B.S. degree in Software Engineering from University of Electronic Science and Technology of China, Chengdu, China, in 2022. He is currently pursuing the Ph.D. degree in Vehicle Engineering with Jilin University, Changchun, China. His current research interests include computer vision and vehicle control systems. (Email: zhangbin22@mails.jlu.edu.cn)

Jinxiang XIA received the M.S. degree in Mechanics and the Ph.D. degree in Communication and Information System from Sichuan University, Chengdu, China, and University of Electronic Science and Technology of China (UESTC), Chengdu, China, in 1998 and 2004, respectively. From 2005 to 2007, he held a postdoctoral position at Southwest Jiaotong University, Chengdu, China. In 2010, he became an Associate Professor of UESTC. Since July 1998, he has been working in UESTC. His current focus lies on deep learning, computer vision, and automatic driving with the Intelligent Software and Perceptual Computation Research Center of UESTC. (Email: jxxia@uestc.edu.cn)

Weidong WANG received the B.S. degree in Industrial Automation from Shenyang University of Technology, Shenyang, China, in 1996 and the M.S. and Ph.D. degrees in Control Science and Engineering and Computer Application from University of Electronic Science and Technology of China (UESTC), Chengdu, China, in 2002 and 2008, respectively. From 2017 to 2018, he was an academic visitor at the Institute for Data Analytics and Data Science, University of Essex, Colchester, UK. He is currently with the Research Center for Intelligent Software and Perceptive Computing, School of Information and Software Engineering, UESTC, Chengdu, China. His current research interests include distributed machine learning, Bayesian optimization, and multitask learning. (Email: wdwang@uestc.edu.cn)
Corresponding author: Email: jxxia@uestc.edu.cn
Available Online: 2024-04-28

Abstract

Abstract

This paper focuses on the problems of point cloud deep neural networks in classification and segmentation tasks, including losing important information during down-sampling, ignoring relationships among points when extracting features, and network performance being susceptible to the sparsity of point cloud. To begin with, this paper proposes a farthest point sampling (FPS)-important points sampling (F-IPS) method for down-sampling, which can preserve important information of point clouds and maintain the geometry of input data. Then, the local feature relation aggregating (LFRA) method is proposed for feature extraction, improving the network’s ability to learn contextual information and extract rich local region features. Based on these methods, the important points feature aggregating net (IPFA-Net) is designed for point cloud classification and segmentation tasks. Furthermore, this paper proposes the multi-scale multi-density feature connecting (MMFC) method to reduce the negative impact of point cloud data sparsity on network performance. Finally, the effectiveness of IPFA-Net is demonstrated through experiments on ModelNet40, ShapeNet part, and ScanNet v2 datasets. IPFA-Net is robust to reducing the number of point clouds, with only a 3.3% decrease in accuracy under a 16-fold reduction of point number. In the part segmentation experiments, our method achieves the best segmentation performance for five objects.
- Deep Learning,
- Point clouds,
- Down-sampling,
- Feature extraction

FullText(HTML)

References(58)

References

[1]	G. J. Wang, J. Wu, B. Tian, et al., “CenterNet3D: An anchor free object detector for point cloud,” IEEE Transactions on Intelligent Transportation Systems, vol. 23, no. 8, pp. 12953–12965, 2022. doi: 10.1109/TITS.2021.3118698
[2]	G. Wang, W. L. Li, C. Jiang, et al., “Trajectory planning and optimization for robotic machining based on measured point cloud,” IEEE Transactions on Robotics, vol. 38, no. 3, pp. 1621–1637, 2022. doi: 10.1109/TRO.2021.3108506
[3]	S. Vesal, M. X. Gu, R. Kosti, et al., “Adapt everywhere: Unsupervised adaptation of point-clouds and entropy minimization for multi-modal cardiac image segmentation,” IEEE Transactions on Medical Imaging, vol. 40, no. 7, pp. 1838–1851, 2021. doi: 10.1109/TMI.2021.3066683
[4]	Y. Eldar, M. Lindenbaum, M. Porat, et al., “The farthest point strategy for progressive image sampling,” IEEE Transactions on Image Processing, vol. 6, no. 9, pp. 1305–1315, 1997. doi: 10.1109/83.623193
[5]	R. Q. Charles, H. Su, M. Kaichun, et al., “PointNet: Deep learning on point sets for 3D classification and segmentation,” in Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA, pp. 652–660, 2017.
[6]	C. R. Qi, L. Yi, H. Su, et al., “PointNet++: Deep hierarchical feature learning on point sets in a metric space,” in Proceedings of the 31st International Conference on Neural Information Processing Systems, Long Beach, CA, USA, pp. 5105–5114, 2017.
[7]	A. Vaswani, N. Shazeer, N. Parmar, et al., “Attention is all you need,” in Proceedings of the 31st International Conference on Neural Information Processing Systems, Long Beach, CA, USA, pp. 6000–6010, 2017.
[8]	G. H. Wang, Q. Y. Zhai, and H. Liu, “Cross self-attention network for 3D point cloud,” Knowledge-Based Systems, vol. 247, article no. 108769, 2022. doi: 10.1016/j.knosys.2022.108769
[9]	X. F. Han, Z. Y. He, J. Chen, et al., “3CROSSNet: Cross-level cross-scale cross-attention network for point cloud representation,” IEEE Robotics and Automation Letters, vol. 7, no. 2, pp. 3718–3725, 2022. doi: 10.1109/LRA.2022.3147907
[10]	J. J. Chen, B. Kakillioglu, and S. Velipasalar, “Background-aware 3-D point cloud segmentation with dynamic point feature aggregation,” IEEE Transactions on Geoscience and Remote Sensing, vol. 60, article no. 5703112, 2022. doi: 10.1109/TGRS.2022.3168555
[11]	L. F. Ma, Y. Li, J. Li, et al., “Multi-scale point-wise convolutional neural networks for 3D object segmentation from LiDAR point clouds in large-scale environments,” IEEE Transactions on Intelligent Transportation Systems, vol. 22, no. 2, pp. 821–836, 2021. doi: 10.1109/TITS.2019.2961060
[12]	D. W. Li, G. L. Shi, Y. H. Wu, et al., “Multi-scale neighborhood feature extraction and aggregation for point cloud segmentation,” IEEE Transactions on Circuits and Systems for Video Technology, vol. 31, no. 6, pp. 2175–2191, 2021. doi: 10.1109/TCSVT.2020.3023051
[13]	D. Nie, R. Lan, L. Wang, et al., “Pyramid architecture for multi-scale processing in point cloud segmentation,” in Proceedings of 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition, New Orleans, LA, USA, pp. 17263–17273, 2022.
[14]	B. Graham, M. Engelcke, and L. van der Maaten, “3D semantic segmentation with submanifold sparse convolutional networks,” in Proceedings of 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA, pp. 9224–9232, 2018.
[15]	H. Y. Meng, L. Gao, Y. K. Lai, et al., “VV-Net: Voxel VAE net with group convolutions for point cloud segmentation,” in Proceedings of 2019 IEEE/CVF International Conference on Computer Vision, Seoul, Korea (South), pp. 8499–8507, 2019.
[16]	G. Riegler, A. O. Ulusoy, and A. Geiger, “OctNet: Learning deep 3D representations at high resolutions,” in Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA, pp. 6620–6629, 2017.
[17]	L. L. Huang, S. L. Wang, K. Wong, et al., “OctSqueeze: Octree-structured entropy model for LiDAR compression,” in Proceedings of 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA, pp. 1310–1320, 2020.
[18]	D. C. Garcia, T. A. Fonseca, R. U. Ferreira, et al., “Geometry coding for dynamic voxelized point clouds using octrees and multiple contexts,” IEEE Transactions on Image Processing, vol. 29, pp. 313–322, 2020. doi: 10.1109/TIP.2019.2931466
[19]	R. Klokov and V. Lempitsky, “Escape from cells: Deep Kd-networks for the recognition of 3D point cloud models,” in Proceedings of 2017 IEEE International Conference on Computer Vision, Venice, Italy, pp. 863–872, 2017.
[20]	Y. C. Liu, B. Fan, S. M. Xiang, et al., “Relation-shape convolutional neural network for point cloud analysis,” in Proceedings of 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Long Beach, CA, USA, pp. 8887–8896, 2019.
[21]	S. L. Wang, S. Suo, W. C. Ma, et al., “Deep parametric continuous convolutional neural networks,” in Proceedings of 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA, pp. 2589–2597, 2018.
[22]	W. X. Wu, Z. G. Qi, and F. X. Li, “PointConv: Deep convolutional networks on 3D point clouds,” in Proceedings of 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Long Beach, CA, USA, pp. 9613–9622, 2019.
[23]	A. Komarichev, Z. C. Zhong, and J. Hua, “A-CNN: Annularly convolutional neural networks on point clouds,” in Proceedings of 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Long Beach, CA, USA, pp. 7413–7422, 2019.
[24]	Z. Liu, H. Hu, Y. Cao, et al., “A closer look at local aggregation operators in point cloud analysis,” in Proceedings of the 16th European Conference on Computer Vision, Glasgow, UK, pp. 326–342, 2020.
[25]	M. T. Xu, R. Y. Ding, H. S. Zhao, et al., “PAConv: Position adaptive convolution with dynamic kernel assembling on point clouds,” in Proceedings of 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Nashville, TN, USA, pp. 3172–3181, 2021.
[26]	H. S. Zhao, L. Jiang, C. W. Fu, et al., “PointWeb: Enhancing local neighborhood features for point cloud processing,” in Proceedings of 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Long Beach, CA, USA, pp. 5560–5568, 2019.
[27]	M. T. Feng, L. Zhang, X. F. Lin, et al., “Point attention network for semantic segmentation of 3D point clouds,” Pattern Recognition, vol. 107, article no. 107446, 2020. doi: 10.1016/j.patcog.2020.107446
[28]	H. Q. Wang, D. Huang, and Y. H. Wang, “GridNet: Efficiently learning deep hierarchical representation for 3D point cloud understanding,” Frontiers of Computer Science, vol. 16, no. 1, article no. 161301, 2022. doi: 10.1007/s11704-020-9521-2
[29]	S. S. Mohammadi, N. F. Duarte, D. Dimou, et al., “3DSGrasp: 3D shape-completion for robotic grasp,” in Proceedings of 2023 IEEE International Conference on Robotics and Automation, London, United Kingdom, pp. 3815–3822, 2023.
[30]	Y. He, X. L. Ren, D. H. Tang, et al., “Density-preserving deep point cloud compression,” in Proceedings of 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition, New Orleans, LA, USA, pp. 2323–2332, 2022.
[31]	E. Nezhadarya, E. Taghavi, R. Razani, et al., “Adaptive hierarchical down-sampling for point cloud classification,” in Proceedings of 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA, pp. 12953–12961, 2020.
[32]	X. Yan, C. D. Zheng, Z. Li, et al., “PointASNL: Robust point clouds processing using nonlocal neural networks with adaptive sampling,” in Proceedings of 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA, pp. 5588–5597, 2020.
[33]	Y. N. Lin, Y. Huang, S. H. Zhou, et al., “DA-Net: Density-adaptive downsampling network for point cloud classification via end-to-end learning,” in Proceedings of 2021 4th International Conference on Pattern Recognition and Artificial Intelligence, Yibin, China, pp. 13–18, 2021.
[34]	I. Lang, A. Manor, and S. Avidan, “SampleNet: Differentiable point cloud sampling,” in Proceedings of 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA, pp. 7575–7585, 2020.
[35]	Z. R. Wu, S. R. Song, A. Khosla, et al., “3D ShapeNets: A deep representation for volumetric shapes,” in Proceedings of 2015 IEEE Conference on Computer Vision and Pattern Recognition, Boston, MA, USA, pp. 1912–1920, 2015.
[36]	L. Yi, V. G. Kim, D. Ceylan, et al., “A scalable active framework for region annotation in 3D shape collections,” ACM Transactions on Graphics, vol. 35, no. 6, article no. 210, 2016. doi: 10.1145/2980179.2980238
[37]	A. Dai, A. X. Chang, M. Savva, et al., “ScanNet: Richly-annotated 3D reconstructions of indoor scenes,” in Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA, pp. 2432–2443, 2017.
[38]	J. X. Li, B. M. Chen, and G. H. Lee, “SO-Net: Self-organizing network for point cloud analysis,” in Proceedings of 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA, pp. 9397–9406, 2018.
[39]	A. Boulch, “ConvPoint: Continuous convolutions for point cloud processing,” Computers & Graphics, vol. 88, pp. 24–34, 2020. doi: 10.1016/j.cag.2020.02.005
[40]	H. Thomas, C. R. Qi, J. E. Deschaud, et al., “KPConv: Flexible and deformable convolution for point clouds,” in Proceedings of 2019 IEEE/CVF International Conference on Computer Vision, Seoul, Korea (South), pp. 6410–6419, 2019.
[41]	D. Denipitiyage, V. Jayasundara, R. Rodrigo, et al., “PointCaps: Raw point cloud processing using capsule networks with Euclidean distance routing,” Journal of Visual Communication and Image Representation, vol. 88, article no. 103612, 2022. doi: 10.1016/j.jvcir.2022.103612
[42]	J. C. Yang, Q. Zhang, B. B. Ni, et al., “Modeling point clouds with self-attention and Gumbel subset sampling,” in Proceedings of 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Long Beach, CA, USA, pp. 3318–3327, 2019.
[43]	Y. Wang, Y. B. Sun, Z. W. Liu, et al., “Dynamic graph CNN for learning on point clouds,” ACM Transactions on Graphics, vol. 38, no. 5, article no. 146, 2019. doi: 10.1145/3326362
[44]	Y. Q. Lin, Z. Z. Yan, H. B. Huang, et al., “FPConv: Learning local flattening for point convolution,” in Proceedings of 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA, pp. 4292–4301, 2020.
[45]	Q. G. Xu, X. D. Sun, C. Y. Wu, et al., “Grid-GCN for fast and scalable point cloud learning,” in Proceedings of 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA, pp. 5660–5669, 2020.
[46]	Y. R. Shen, C. Feng, Y. Q. Yang, et al., “Mining point cloud local structures by kernel correlation and graph pooling,” in Proceedings of 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA, pp. 4548–4557, 2018.
[47]	Q. G. Huang, W. Y. Wang, and U. Neumann, “Recurrent slice networks for 3D segmentation of point clouds,” in Proceedings of 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA, pp. 2626–2635, 2018.
[48]	Z. H. Lin, S. Y. Huang, and Y. C. F. Wang, “Convolution in the cloud: Learning deformable kernels in 3D graph convolution networks for point cloud analysis,” in Proceedings of 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA, pp. 1797–1806, 2020.
[49]	K. Fujiwara and T. Hashimoto, “Neural implicit embedding for point cloud analysis,” in Proceedings of 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA, pp. 11731–11740, 2020.
[50]	X. F. Han, Y. F. Jin, H. X. Cheng, et al., “Dual transformer for point cloud analysis,” IEEE Transactions on Multimedia, vol. 25, pp. 5638–5648, 2023. doi: 10.1109/TMM.2022.3198318
[51]	N. Luo, H. Q. Yu, Z. F. Huo, et al., “KVGCN: A KNN searching and VLAD combined graph convolutional network for point cloud segmentation,” Remote Sensing, vol. 13, no. 5, article no. 1003, 2021. doi: 10.3390/rs13051003
[52]	N. Zhao, T. S. Chua, and G. H. Lee, “PS2-Net: A locally and globally aware network for point-based semantic segmentation,” in Proceedings of 2020 25th International Conference on Pattern Recognition, Milan, Italy, pp. 723–730, 2021.
[53]	H. Lei, N. Akhtar, and A. Mian, “SegGCN: Efficient 3D point cloud segmentation with fuzzy spherical kernel,” in Proceedings of 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA, pp. 11608–11617, 2020.
[54]	Y. N. Ma, Y. L. Guo, H. Liu, et al., “Global context reasoning for semantic segmentation of 3D point clouds,” in Proceedings of 2020 IEEE Winter Conference on Applications of Computer Vision, Snowmass, CO, USA, pp. 2920–2929, 2020.
[55]	H. J. Lin, Z. P. Luo, W. Li, et al., “Adaptive pyramid context fusion for point cloud perception,” IEEE Geoscience and Remote Sensing Letters, vol. 19, article no. 6500505, 2022. doi: 10.1109/LGRS.2020.3037509
[56]	J. Y. Gong, J. C. Xu, X. Tan, et al., “Boundary-aware geometric encoding for semantic segmentation of point clouds,” in Proceedings of the 35th AAAI Conference on Artificial Intelligence, Virtual Event, pp. 1424–1432, 2021.
[57]	Q. Y. Hu, B. Yang, L. H. Xie, et al., “RandLA-Net: Efficient semantic segmentation of large-scale point clouds,” in Proceedings of 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA, pp. 11105–11114, 2020.
[58]	M. H. Guo, J. X. Cai, Z. N. Liu, et al., “PCT: Point cloud transformer,” Computational Visual Media, vol. 7, no. 2, pp. 187–199.