Hyperspectral Image Classification Based on A Multi-Scale Weighted Kernel Network

SUN Le; XU Bin; LU Zhenyu

doi:10.1049/cje.2021.00.130

Volume 31 Issue 5

Sep. 2022

Turn off MathJax

Article Contents

Article Navigation > Chinese Journal of Electronics > 2022 > 31(5): 832-843

SUN Le, XU Bin, LU Zhenyu, “Hyperspectral Image Classification Based on A Multi-Scale Weighted Kernel Network,” Chinese Journal of Electronics, vol. 31, no. 5, pp. 832-843, 2022, doi: 10.1049/cje.2021.00.130

Citation:

SUN Le, XU Bin, LU Zhenyu, “Hyperspectral Image Classification Based on A Multi-Scale Weighted Kernel Network,” Chinese Journal of Electronics, vol. 31, no. 5, pp. 832-843, 2022, doi: 10.1049/cje.2021.00.130

Citation:

PDF( 3747 KB)

Hyperspectral Image Classification Based on A Multi-Scale Weighted Kernel Network

doi: 10.1049/cje.2021.00.130

SUN Le^{1, 3
,},
XU Bin²,
LU Zhenyu²

1.
School of Computer Science, Nanjing University of Information Science and Technology, Nanjing 210044, China
2.
School of Artificial Intelligence, Nanjing University of Information Science and Technology, Nanjing 210044, China
3.
Engineering Research Center of Digital Forensics, Ministry of Education, Nanjing University of Information Science and Technology, Nanjing 210044, China

Funds: This work was supported by the National Natural Science Foundation of China (61971233, U20B2061)

More Information

Author Bio:
was born in Jiangsu Province, China, in 1987. He received the B.S. degree from the School of Science, Nanjing University of Science and Technology (NJUST), Nanjing, China, in 2009, and the Ph.D. degree from the School of Computer Science and Engineering, NJUST, in 2014. From 2015 to 2018, he conducted research in the field of multiimages fusion based on sparse dictionary learning and compressive sensing as a Post-doctor at the School of Electronic and Electrical Engineering, Sungkyunkwan University, Korea. Since 2020, he has been an Associate Professor with the School of Computer and Science, Nanjing University of Information Science and Technology, Nanjing, China. His research interests include hyperspectral image processing (including unmixing, classification, restoration), sparse representation, compressive sensing, and deep learning. (Email: sunlecncom@163.com)

was born in Jiangsu Province, China, in 1994. He received the M.S. degree from the School of Artificial Intelligence, Nanjing University of Information Science and Technology, Nanjing, China. His research interests include hyperspectral image classification, machine learning, and deep learning

(corresponding author) was born in Guizhou Province, China, in 1976. He received the B.S. degree in electricity and the M.S. degree in information and communication from the Nanjing Institute of Meteorology, Nanjing, China, in 1999 and 2002, respectively, and the Ph.D. degree in optics engineering from the Nanjing University of Science and Technology, Nanjing, in 2008. He was a Research Associate with the Department of Mathematics and Statistics, University of Strathclyde, Glasgow, U.K., from 2012 to 2013. He is currently a Full Professor with the School of AI, Nanjing University of Information Science and Technology. His current research interests include neural networks, stochastic control, and artificial intelligence. (Email: luzhenyu@163.com)
Received Date: 2021-04-14
Accepted Date: 2021-12-14

Available Online: 2021-12-28

Publish Date: 2022-09-05

Abstract

Abstract

Recently, many deep learning models have shown excellent performance in hyperspectral image (HSI) classification. Among them, networks with multiple convolution kernels of different sizes have been proved to achieve richer receptive fields and extract more representative features than those with a single convolution kernel. However, in most networks, different-sized convolution kernels are usually used directly on multi-branch structures, and the image features extracted from them are fused directly and simply. In this paper, to fully and adaptively explore the multiscale information in both spectral and spatial domains of HSI, a novel multi-scale weighted kernel network (MSWKNet) based on an adaptive receptive field is proposed. First, the original HSI cubic patches are transformed to the input features by combining the principal component analysis and one-dimensional spectral convolution. Then, a three-branch network with different convolution kernels is designed to convolve the input features, and adaptively adjust the size of the receptive field through the attention mechanism of each branch. Finally, the features extracted from each branch are fused together for the task of classification. Experiments on three well-known hyperspectral data sets show that MSWKNet outperforms many deep learning networks in HSI classification.
- Hyperspectral image classification,
- Convolutional neural network,
- Kernel network,
- Multi-scale features,
- Attention mechanism

FullText(HTML)

References(45)

References

[1]	Q. Ma, X. Zhang, C. Zhang, and H. Zhou, “Hyperspectral image classification based on capsule network,” Chinese Journal of Electronics, vol.31, no.1, pp.146–154, 2022. doi: 10.1049/cje.2021.00.056
[2]	X. Wang, Y. Li, and Y. Cheng, “Hyperspectral image classification based on unsupervised heterogeneous domain adaptation cycleGan,” Chinese Journal of Electronics, vol.29, no.4, pp.608–614, 2020. doi: 10.1049/cje.2020.05.003
[3]	F. Luo, Z. Zou, J. Liu, et al., “Dimensionality reduction and classification of hyperspectral image via multi-structure unified discriminative embedding,” IEEE Transactions on Geoscience and Remote Sensing, vol.60, pp.1–16, 2022. doi: 10.1109/TGRS.2021.3128764
[4]	Q. Ye, Z. Li, L. Fu, et al., “Nonpeaked discriminant analysis,” IEEE Transactions on Neural Networks and Learning Systems, vol.30, no.12, pp.3818–3832, 2019. doi: 10.1109/TNNLS.2019.2944869
[5]	F. Melgani and L. Bruzzone, “Classification of hyperspectral remote sensing images with support vector machines,” IEEE Transactions on Geoscience and Remote Sensing, vol.42, no.8, pp.1778–1790, 2004. doi: 10.1109/TGRS.2004.831865
[6]	Q. Ye, H. Zhao, Z. Li, et al., “L1-norm distance minimization based fast robust twin support vector k-plane clustering,” IEEE Transactions on Neural Networks and Learning Systems, vol.29, no.9, pp.4494–4503, 2018. doi: 10.1109/TNNLS.2017.2749428
[7]	Q. Ye, P. Huang, Z. Zhang, et al., “Multiview learning with robust double-sided twin SVM,” IEEE Transactions on Cybernetics, Early access, DOI: 10.1109/TCYB.2021.3088519, 2021.
[8]	L. Fu, Z. Li, Q. Ye, et al., “Learning robust discriminant subspace based on joint L2, p- and L2, s-norm distance metrics,” IEEE Transactions on Neural Networks and Learning Systems, vol.33, no.1, pp.130–144, 2022. doi: 10.1109/TNNLS.2020.3027588
[9]	Q. Ye, J. Yang F. Liu, et al., “L1-norm distance linear discriminant analysis based on an effective iterative algorithm,” IEEE Transactions on Circuits Systems and Video Technology, vol.28, no.1, pp.114–129, 2018. doi: 10.1109/TCSVT.2016.2596158
[10]	Y. Duan, H. Huang, and T. Wang, “Semisupervised feature extraction of hyperspectral image using nonlinear geodesic sparse hypergraphs,” IEEE Transactions on Geoscience and Remote Sensing, vol.60, article no.5515115, 2022. doi: 10.1109/TGRS.2021.3110855
[11]	L. Sun, C. He, Y. Zheng, and S. Tang, “SLRL4D: Joint restoration of subspace low-rank learning and non-local 4-D transform filtering for hyperspectral image,” Remote Sensing, vol.12, no.18, article no.2979, 2020. doi: 10.3390/rs12182979
[12]	C. He, L. Sun, W. Huang, et al., “TSLRLN: Tensor subspace low-rank learning with non-local prior for hyperspectral image mixed denoising,” Signal Processing, vol.184, article no.108060, 2021. doi: 10.1016/j.sigpro.2021.108060
[13]	L. Sun, F. Wu, C. He, et al., “Weighted collaborative sparse and l_1/2 low-rank regularizations with superpixel segmentation for hyperspectral unmixing,” IEEE Geoscience and Remote Sensing Letters, vol.19, article no.5500405, 2022. doi: 10.1109/LGRS.2020.3019427
[14]	K. Kayabol, “Bayesian Gaussian mixture model for spatial-spectral classification of hyperspectral images,” in Proc. of 2015 23rd European Signal Processing Conference (EUSIPCO), Nice, France, pp.1805–1809, 2015.
[15]	Y. Hou, W. Zhu, and E. Wang, “Hyperspectral mineral target detection based on density peak,” Intelligent Automation and Soft Computing, vol.25, no.4, pp.805–814, 2019.
[16]	I. Bidari, S. Chickerur, H. Ranmale, S. Talawar, H. Ramadurg and R. Talikoti, “Hyperspectral imagery classification using deep learning,” in Proc. of 2020 Fourth World Conference on Smart Trends in Systems, Security and Sustainability (WorldS4), London, UK, pp. 672–676, 2020.
[17]	C. Yu, M. Zhao, M. Song, Y. Wang, F. Li, R. Han, and C. Chang, “Hyperspectral image classification method based on CNN architecture embedding with hashing semantic feature,” IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, vol.12, no.6, pp.1866–1881, 2019. doi: 10.1109/JSTARS.2019.2911987
[18]	Z. Gong, P. Zhong, Y. Yu, et al., “A CNN with multiscale convolution and diversified metric for hyperspectral image classification,” IEEE Transactions on Geoscience and Remote Sensing, vol.57, no.6, pp.3599–3618, 2019. doi: 10.1109/TGRS.2018.2886022
[19]	A. J. X. Guo and F. Zhu, “A CNN-based spatial feature fusion algorithm for hyperspectral imagery classification,” IEEE Transactions on Geoscience and Remote Sensing, vol.57, no.9, pp.7170–7181, 2019. doi: 10.1109/TGRS.2019.2911993
[20]	J. Xie, N. He, L. Fang, and P. Ghamisi, “Multiscale densely-connected fusion networks for hyperspectral images classification,” IEEE Transactions on Circuits and Systems for Video Technology, vol.31, no.1, pp.246–259, 2021. doi: 10.1109/TCSVT.2020.2975566
[21]	Q. Liu, L. Xiao, F. Liu, and J. Xu, “SSCDenseNet: A spectral-spatial convolutional dense network for hyperspectral image classification,” Acta Electronica Sinica, vol.48, no.4, pp.751–762, 2020.
[22]	J. Zhang, X. Jin, J. Sun, et al., “Spatial and semantic convolutional features for robust visual object tracking,” Multimedia Tools and Applications, vol.79, no.1, pp.15095–15115, 2020.
[23]	Qian Zhao, Li Ma, Fang Xu, Wei Cheng and Mao Xu, “Feature representation learning on multi-scale receptive fields for objection recognition,” in Proc. of 2015 11th International Conference on Natural Computation (ICNC), Zhangjiajie, China, pp.19–23, 2015.
[24]	R. Gabbaso and R. Paringer, “Influence of the receptive field size on accuracy and performance of a convolutional neural network,” in Proc. of 2020 International Conference on Information Technology and Nanotechnology (ITNT), Samara, Russia, pp.1–4, 2020.
[25]	K. Safari, S. Prasad, and D. Labate, “A multiscale deep learning approach for high-resolution hyperspectral image classification,” IEEE Transactions on Geoscience and Remote Sensing Letters, vol.18, no.1, pp.167–171, 2021. doi: 10.1109/LGRS.2020.2966987
[26]	S. Liu and Q. Shi, “Multitask deep learning with spectral knowledge for hyperspectral image classification,” IEEE Transactions on Geoscience and Remote Sensing Letters, vol.17, no.12, pp.2110–2114, 2020. doi: 10.1109/LGRS.2019.2962768
[27]	S. Wu, J. Zhang, and C. Zhong, “Multiscale spectral-spatial unified networks for hyperspectral image classification,” in Proc. of 2019 IEEE International Geoscience and Remote Sensing Symposium (IGARSS), Yokohama, Japan, pp.2706–2709, 2019.
[28]	W. Song, S. Li, L. Fang, and T. Lu, “Hyperspectral image classification with deep feature fusion network,” IEEE Transactions on Geoscience and Remote Sensing, vol.56, no.6, pp.3173–3184, 2018. doi: 10.1109/TGRS.2018.2794326
[29]	K. Pooja, R. R. Nidamanuri, and D. Mishra, “Multi-scale dilated residual convolutional neural network for hyperspectral image classification,” in Proc. of 2019 10th Workshop on Hyperspectral Imaging and Signal Processing: Evolution in Remote Sensing (WHISPERS), Amsterdam, Netherlands, pp.1–5, 2019.
[30]	M. E. Paoletti, J. M. Haut, R. Fernandez-Beltran, J. Plaza, A. J. Plaza, and F. Pla, “Deep pyramidal residual networks for spectral–spatial hyperspectral image classification,” IEEE Transactions on Geoscience and Remote Sensing, vol.57, no.2, pp.740–754, 2019. doi: 10.1109/TGRS.2018.2860125
[31]	K. Simonyan and A. Zisserman, “Very deep convolutional networks for large-scale image recognition,” in Proceedings of International Conference on Learning Representations, San Diego, CA, USA, pp.1–14, 2015.
[32]	J. M. Haut, M. E. Paoletti, J. Plaza, A. Plaza, and J. Li, “Visual attention-driven hyperspectral image classification,” IEEE Transactions on Geoscience and Remote Sensing, vol.57, no.10, pp.8065–8080, 2019. doi: 10.1109/TGRS.2019.2918080
[33]	K. He, X. Zhang, S. Ren, and, J. Sun, “Deep residual learning for image recognition,” in Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR), Boston, USA, pp.770–778, 2015.
[34]	J. M. Haut, M. E. Paoletti, J. Plaza, J. Li, and A. Plaza, “Active learning with convolutional neural networks for hyperspectral image classification using a new Bayesian approach,” IEEE Transactions on Geoscience and Remote Sensing, vol.56, no.11, pp.6440–6461, 2018. doi: 10.1109/TGRS.2018.2838665
[35]	X. Yang, Y. Ye, X. Li, R. Y. K. Lau, X. Zhang, and X. Huang, “Hyperspectral image classification with deep learning models,” IEEE Transactions on Geoscience and Remote Sensing, vol.56, no.9, pp.5408–5423, 2018. doi: 10.1109/TGRS.2018.2815613
[36]	G. Licciardi, P. R. Marpu, J. Chanussot, and J. A. Benediktsson, “Linear versus nonlinear PCA for the classification of hyperspectral data based on the extended morphological profiles,” IEEE Transactions on Geoscience and Remote Sensing Letters, vol.9, no.3, pp.447–451, 2012. doi: 10.1109/LGRS.2011.2172185
[37]	S. Prasad and L. M. Bruce, “Limitations of principal components analysis for hyperspectral target recognition,” IEEE Transactions on Geoscience and Remote Sensing Letters, vol.5, no.4, pp.625–629, 2008. doi: 10.1109/LGRS.2008.2001282
[38]	A. Krizhevsky, I. Sutskever, and G. E. Hinton, “ImageNet classification with deep convolutional neural networks,” Communications of the ACM, vol.60, no.6, pp.84–90, 2017. doi: 10.1145/3065386
[39]	Y. Li, H. Zhang, and Q. Shen, “Spectral-spatial classification of hyperspectral imagery with 3D convolutional neural network,” Remote Sensing, vol.9, no.1, pp.67–87, 2017. doi: 10.3390/rs9010067
[40]	Z. Zhong, J. Li, Z. Luo, and M. Chapman, “Spectral-spatial residuall network for hyperspectral image classification:A 3-D deep learning framework,” IEEE Transactions on Geoscience and Remote Sensing, vol.56, no.2, pp.847–856, 2018. doi: 10.1109/TGRS.2017.2755542
[41]	A. Ben Hamida, A. Benoit, P. Lambert, and C. Ben Amar, “3-D deep learning approach for remote sensing image classification,” IEEE Transactions on Geoscience and Remote Sensing, vol.56, no.8, pp.4420–4434, 2018. doi: 10.1109/TGRS.2018.2818945
[42]	G. Camps-Valls, L. Gomez-Chova, J. Muñoz-Marí, J. Vila-Francés, and J. Calpe-Maravilla, “Composite kernels for hyperspectral image classification,” IEEE Transactions on Geoscience and Remote sensing Letters, vol.3, no.1, pp.93–97, 2006. doi: 10.1109/LGRS.2005.857031
[43]	K. Makantasis, K. Karantzalos, A. Doulamis, and N. Doulamis, “Deep supervised learning for hyperspectral data classification through convolutional neural networks,” in Proc. of 2015 IEEE International Geoscience and Remote Sensing Symposium (IGARSS), Milan, Italy, pp. 4959-4962, 2015.
[44]	W. Wang, S. Dou and Z. Jiang, and L. Sun, “A fast dense spectral-spatial convolution network framework for hyperspectral images classification,” Remote Sensing, vol.10, no.7, article no.1068, 2018. doi: 10.3390/rs10071068
[45]	Z. Lu, B. Xu, L. Sun, T. Zhan, and S. Tang, “3-D channel and spatial attention based multiscale spatial–spectral residual network for hyperspectral image classification,” IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, vol.13, pp.4311–4324, 2020. doi: 10.1109/JSTARS.2020.3011992