Hyperspectral Image Classification Based on A Multi-scale Weighted Kernel Network

SUN Le; XU Bin; LU Zhenyu

doi:10.1049/cje.2021.00.130

Article Contents

Article Navigation > Chinese Journal of Electronics > 2021 > Accepted Manuscript

SUN Le, XU Bin, LU Zhenyu. Hyperspectral Image Classification Based on A Multi-scale Weighted Kernel Network[J]. Chinese Journal of Electronics. doi: 10.1049/cje.2021.00.130

Citation:

SUN Le, XU Bin, LU Zhenyu. Hyperspectral Image Classification Based on A Multi-scale Weighted Kernel Network[J]. Chinese Journal of Electronics. doi: 10.1049/cje.2021.00.130

SUN Le, XU Bin, LU Zhenyu. Hyperspectral Image Classification Based on A Multi-scale Weighted Kernel Network[J]. Chinese Journal of Electronics. doi: 10.1049/cje.2021.00.130

Citation:

SUN Le, XU Bin, LU Zhenyu. Hyperspectral Image Classification Based on A Multi-scale Weighted Kernel Network[J]. Chinese Journal of Electronics. doi: 10.1049/cje.2021.00.130

PDF( 3610 KB)

Hyperspectral Image Classification Based on A Multi-scale Weighted Kernel Network

doi: 10.1049/cje.2021.00.130

SUN Le^{1, 3
,},
XU Bin²,
LU Zhenyu²

1.
School of Computer Science, Nanjing University of Information Science and Technology, Nanjing 210044, China
2.
School of Artificial Intelligence, Nanjing University of Information Science and Technology, Nanjing 210044, China
3.
Engineering Research Center of Digital Forensics, Ministry of Education, Nanjing University of Information Science and Technology, Nanjing 210044, China

Funds: This work is supported by the National Natural Science Foundation of China (No.61971233, No.U20B2061)

More Information

Author Bio:
was born in Jiangsu, China, in 1987. He received the B.S. degree from the School of Science, Nanjing University of Science and Technology (NJUST), Nanjing, Jiangsu, China, in 2009, and the Ph.D. degree from the School of Computer Science and Engineering, NJUST, in 2014. From 2015 to 2018, he conducted research in the field of multiimages fusion based on sparse dictionary learning and compressive sensing as a Post-doctor at the School of Electronic and Electrical Engineering, Sungkyunkwan University, Korea. Since 2020, he has been an associate professor with the School of Computer and Science, Nanjing University of Information Science and Technology, Nanjing, China. His research interests include hyperspectral image processing (including unmixing, classification, restoration), sparse representation, compressive sensing, and deep learning.(Email: sunlecncom@163.com)

was born in Jiangsu, China, in 1994. He received the M.S. degree from the School of Artificial Intelligence, Nanjing University of Information Science and Technology, Nanjing, China. His research interests include hyperspectral image classification, machine learning, and deep learning

(corresponding author) was born in Guizhou, China, in 1976. He received the B.Sc. degree in electricity and the M.Sc. degree in information and communication from the Nanjing Institute of Meteorology, Nanjing, China, in 1999 and 2002, respectively, and the Ph.D. degree in optics engineering from the Nanjing University of Science and Technology, Nanjing, in 2008. He was a Research Associate with the Department of Mathematics and Statistics, University of Strathclyde, Glasgow, U.K., from 2012 to 2013. He is currently a full Professor with the School of AI, Nanjing University of Information Science and Technology. His current research interests include neural networks, stochastic control, and artificial intelligence
Received Date: 2021-04-14
Accepted Date: 2021-12-14

Available Online: 2021-12-28

Abstract

Abstract

Recently, many deep learning models have shown excellent performance in hyperspectral image classification. Among them, networks with multiple convolution kernels of different sizes have been proved to achieve richer receptive fields and extract more representative features than those with a single convolution kernel. However, in most networks, different-sized convolution kernels are usually used directly on multi-branch structures, and the image features extracted from them are fused directly and simply. In this paper, to fully and adaptively explore the multiscale information in both spectral and spatail domains of HSI, a novel multi-scale weighted kernel network (MSWKNet) based on an adaptive receptive field is proposed. First, the original HSI cubic patches are transformed to the input features by combining the principal component analysis and one-dimensional (1D) spectral convolution. Then, a three-branch network with different convolution kernels is designed to convolve the input features, and adaptively adjust the size of the receptive field through the attention mechanism of each branch. Finally, the features extracted from each branch are fused together for the task of classification. Experiments on three well-known hyperspectral data sets show that MSWKNet outperforms many deep learning networks in HSI classification.
- Hyperspectral image classification,
- Convolutional neural network,
- Kernel network,
- Multi-scale features,
- Attention mechanism

FullText(HTML)

References(46)

References

[1]	Q. Ma, X. Zhang, C. Zhang and H. Zhou, “Hyperspectral image classification based on capsule network,” Chinese Journal of Electronics, 2021. Online, doi: 10.1049/cje.2021.00.056.
[2]	X. Wang, Y. Li, and Y. Cheng, “Hyperspectral image classification based on unsupervised heterogeneous domain adaptation cycleGan,” Chinese Journal of Electronics, vol.29, no.4, pp.608–614, 2020. doi: 10.1049/cje.2020.05.003
[3]	F. Luo, Z. Zou, J. Liu and Z. Lin, “Dimensionality reduction and classification of hyperspectral image via multi-structure unified discriminative embedding,” IEEE Transactions on Geoscience and Remote Sensing, 2021. Online, doi: 10.1109/TGRS.2021.3128764.
[4]	Q. Ye, Z. Li, L. Fu, et al., “Nonpeaked discriminant analysis,” IEEE Transactions on Neural Networks and Learning Systems, vol.30, no.12, pp.3818–3832, 2019. doi: 10.1109/TNNLS.2019.2944869
[5]	F. Melgani and L. Bruzzone, “Classification of hyperspectral remote sensing images with support vector machines,” IEEE Transactions on Geoscience and Remote Sensing, vol.42, no.8, pp.1778–1790, 2004. doi: 10.1109/TGRS.2004.831865
[6]	Q. Ye, H. Zhao, Z. Li, et al., “L1-norm distance minimization based fast robust twin support vector k-plane clustering,” IEEE Transactions on Neural Networks and Learning Systems, vol.29, no.9, pp.4494–4503, 2018. doi: 10.1109/TNNLS.2017.2749428
[7]	Q. Ye, P. Huang, Z. Zhang, et al., “Multiview learning with robust double-sided twin SVM,” in IEEE Transactions on Cybernetics, 2021. Online, doi: 10.1109/TCYB.2021.3088519.
[8]	L. Fu, Z. Li, Q. Ye, et al., “Learning robust discriminant subspace based on joint L2, p- and L2, s-norm distance metrics,” IEEE Transactions on Neural Networks and Learning Systems, 2020. Online, doi: 10.1109/TNNLS.2020.3027588.
[9]	Q. Ye, J. Yang F. Liu, et al., “L1-norm distance linear discriminant analysis based on an effective iterative algorithm,” IEEE Transactions on Circuits Systems and Video Technology, vol.28, no.1, pp.114–129, 2018. doi: 10.1109/TCSVT.2016.2596158
[10]	Y. Duan, H. Huang and T. Wang, “Semisupervised feature extraction of hyperspectral image using nonlinear geodesic sparse hypergraphs,” IEEE Transactions on Geoscience and Remote Sensing, 2021. Online, doi: 10.1109/TGRS.2021.3110855.
[11]	L. Sun, C. He, Y. Zheng and S. Tang, “SLRL4D: joint restoration of subspace low-rank learning and non-local 4-D transform filtering for hyperspectral image,” Remote Sensing, vol.12, no.18, article no.2979, 2020. doi: 10.3390/rs12182979
[12]	C. He, L. Sun, W. Huang, J. Zhang, Y. Zheng and B. Jeon, “TSLRLN: tensor subspace low-rank learning with non-local prior for hyperspectral image mixed denoising,” Signal Processing, vol.184, article no.108060, 2021. doi: 10.1016/j.sigpro.2021.108060
[13]	L. Sun, F. Wu, C. He, et al., “Weighted collaborative sparse and l1/2 low-rank regularizations with superpixel segmentation for hyperspectral unmixing,” IEEE Geoscience and Remote Sensing Letters, 2020. Online, doi: 10.1109/LGRS.2020.3019427.
[14]	K. Kayabol, “Bayesian Gaussian mixture model for spatial-spectral classification of hyperspectral images,” Proc. of 2015 23rd European Signal Processing Conference (EUSIPCO), Nice, France, pp. 1805-1809, 2015.
[15]	Y. Hou, W. Zhu and E. Wang, “Hyperspectral mineral target detection based on density peak,” Intelligent Automation and Soft Computing, vol.25, no.4, pp.805–814, 2019.
[16]	I. Bidari, S. Chickerur, H. Ranmale, S. Talawar, H. Ramadurg and R. Talikoti, “Hyperspectral imagery classification using deep learning,” Proc. of 2020 Fourth World Conference on Smart Trends in Systems, Security and Sustainability (WorldS4), London, UK, pp. 672-676, 2020.
[17]	C. Yu, M. Zhao, M. Song, Y. Wang, F. Li, R. Han and C. Chang, “Hyperspectral image classification method based on CNN architecture embedding with hashing semantic feature,” IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, vol.12, no.6, pp.1866–1881, 2019. doi: 10.1109/JSTARS.2019.2911987
[18]	Z. Gong, P. Zhong, Y. Yu, W. Hu and S. Li, “A CNN with multiscale convolution and diversified metric for hyperspectral image classification,” IEEE Transactions on Geoscience and Remote Sensing, vol.57, no.6, pp.3599–3618, 2019. doi: 10.1109/TGRS.2018.2886022
[19]	A. J. X. Guo and F. Zhu, “A CNN-based spatial feature fusion algorithm for hyperspectral imagery classification,” IEEE Transactions on Geoscience and Remote Sensing, vol.57, no.9, pp.7170–7181, 2019. doi: 10.1109/TGRS.2019.2911993
[20]	J. Xie, N. He, L. Fang and P. Ghamisi, “Multiscale densely-connected fusion networks for hyperspectral images classification,” IEEE Transactions on Circuits and Systems for Video Technology, vol.31, no.1, pp.246–259, 2021. doi: 10.1109/TCSVT.2020.2975566
[21]	Q. Liu, L. Xiao, F. Liu and J. Xu, “SSCDenseNet: a spectral-spatial convolutional dense network for hyperspectral image classification,” Acta Electronica Sinica, vol.48, no.4, pp.751–762, 2020.
[22]	J. Zhang, X. Jin, J. Sun, et al., “Spatial and semantic convolutional features for robust visual object tracking,” Multimedia Tools and Applications, vol.79, no.1, pp.15095–15115, 2020.
[23]	Qian Zhao, Li Ma, Fang Xu, Wei Cheng and Mao Xu, “Feature representation learning on multi-scale receptive fields for objection recognition,” Proc. of 2015 11th International Conference on Natural Computation (ICNC), Zhangjiajie, China, pp. 19-23, 2015.
[24]	R. Gabbaso R. Paringer, “Influence of the receptive field size on accuracy and performance of a convolutional neural network,” Proc. of 2020 International Conference on Information Technology and Nanotechnology (ITNT), Samara, Russia, pp. 1-4, 2020.
[25]	K. Safari, S. Prasad and D. Labate, “A multiscale deep learning approach for high-resolution hyperspectral image classification,” IEEE Transactions on Geoscience and Remote Sensing Letters, vol.18, no.1, pp.167–171, 2021. doi: 10.1109/LGRS.2020.2966987
[26]	S. Liu and Q. Shi, “Multitask deep learning with spectral knowledge for hyperspectral image classification,” IEEE Transactions on Geoscience and Remote Sensing Letters, vol.17, no.12, pp.2110–2114, 2020. doi: 10.1109/LGRS.2019.2962768
[27]	S. Wu, J. Zhang and C. Zhong, “Multiscale spectral-spatial unified networks for hyperspectral image classification,” Proc. of 2019 - 2019 IEEE International Geoscience and Remote Sensing Symposium(IGARSS), Yokohama, Japan, pp. 2706-2709, 2019.
[28]	W. Song, S. Li, L. Fang and T. Lu, “Hyperspectral image classification with deep feature fusion network,” IEEE Transactions on Geoscience and Remote Sensing, vol.56, no.6, pp.3173–3184, 2018. doi: 10.1109/TGRS.2018.2794326
[29]	K. Pooja, R. R. Nidamanuri and D. Mishra, “Multi-scale dilated residual convolutional neural network for hyperspectral image classification,” Proc. of 2019 10th Workshop on Hyperspectral Imaging and Signal Processing: Evolution in Remote Sensing (WHISPERS), Amsterdam, Netherlands, pp. 1-5, 2019.
[30]	M. E. Paoletti, J. M. Haut, R. Fernandez-Beltran, J. Plaza, A. J. Plaza and F. Pla, “Deep pyramidal residual networks for spectral–spatial hyperspectral image classification,” IEEE Transactions on Geoscience and Remote Sensing, vol.57, no.2, pp.740–754, 2019. doi: 10.1109/TGRS.2018.2860125
[31]	K. Simonyan and A. Zisserman, “Very deep convolutional networks for large-scale image recognition,” Proceedings of International Conference on Learning Representations, San Diego, CA, USA, pp. 1-14, 2015.
[32]	J. M. Haut, M. E. Paoletti, J. Plaza, A. Plaza and J. Li, “Visual attention-driven hyperspectral image classification,” IEEE Transactions on Geoscience and Remote Sensing, vol.57, no.10, pp.8065–8080, 2019. doi: 10.1109/TGRS.2019.2918080
[33]	K. He, X. Zhang, S. Ren and, J. Sun, “Deep residual learning for image recognition,” Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR), Boston, USA, pp. 770-778, 2015.
[34]	J. M. Haut, M. E. Paoletti, J. Plaza, J. Li, and A. Plaza, “Active learning with convolutional neural networks for hyperspectral image classification using a new Bayesian approach,” IEEE Transactions on Geoscience and Remote Sensing, vol.56, no.11, pp.6440–6461, 2018. doi: 10.1109/TGRS.2018.2838665
[35]	X. Yang, Y. Ye, X. Li, R. Y. K. Lau, X. Zhang, and X. Huang, “Hyperspectral image classification with deep learning models,” IEEE Transactions on Geoscience and Remote Sensing, vol.56, no.9, pp.5408–5423, 2018. doi: 10.1109/TGRS.2018.2815613
[36]	G. Licciardi, P. R. Marpu, J. Chanussot, and J. A. Benediktsson, “Linear versus nonlinear PCA for the classification of hyperspectral data based on the extended morphological profiles,” IEEE Transactions on Geoscience and Remote Sensing Letters, vol.9, no.3, pp.447–451, 2012. doi: 10.1109/LGRS.2011.2172185
[37]	S. Prasad and L. M. Bruce, “Limitations of principal components analysis for hyperspectral target recognition,” IEEE Transactions on Geoscience and Remote Sensing Letters, vol.5, no.4, pp.625–629, 2008. doi: 10.1109/LGRS.2008.2001282
[38]	A. Krizhevsky, I. Sutskever, and G. E. Hinton, “ImageNet classification with deep convolutional neural networks,” Communications of the ACM, vol.60, no.6, pp.84–90, 2017. doi: 10.1145/3065386
[39]	Y. Li, H. Zhang, and Q. Shen, “Spectral-spatial classification of hyperspectral imagery with 3D convolutional neural network,” Remote Sensing, vol.9, no.1, pp.67–87, 2017. doi: 10.3390/rs9010067
[40]	Z. Zhong, J. Li, Z. Luo, and M. Chapman, “Spectral-spatial residuall network for hyperspectral image classification:A 3-D deep learning framework,” IEEE Transactions on Geoscience and Remote Sensing, vol.56, no.2, pp.847–856, 2018. doi: 10.1109/TGRS.2017.2755542
[41]	A. Ben Hamida, A. Benoit, P. Lambert and C. Ben Amar, “3-d deep learning approach for remote sensing image classification,” IEEE Transactions on Geoscience and Remote Sensing, vol.56, no.8, pp.4420–4434, 2018. doi: 10.1109/TGRS.2018.2818945
[42]	G. Camps-Valls, L. Gomez-Chova, J. Muñoz-Marí, J. Vila-Francés and J. Calpe-Maravilla, “Composite kernels for hyperspectral image classification,” IEEE Transactions on Geoscience and Remote sensing Letters, vol.3, no.1, pp.93–97, 2006. doi: 10.1109/LGRS.2005.857031
[43]	K. Makantasis, K. Karantzalos, A. Doulamis and N. Doulamis, “Deep supervised learning for hyperspectral data classification through convolutional neural networks,” Proc. of 2015 IEEE International Geoscience and Remote Sensing Symposium (IGARSS), Milan, Italy, pp. 4959-4962, 2015.
[44]	Z. Zhong, J. Li, Z. Luo and M. Chapman, “Spectral–spatial residual network for hyperspectral image classification: A 3-d deep learning framework,” IEEE Transactions on Geoscience and Remote Sensing, vol.56, no.2, pp.847–858, 2018. doi: 10.1109/TGRS.2017.2755542
[45]	W. Wang, S. Dou and Z. Jiang and, L. Sun, “A fast dense spectral-spatial convolution network framework for hyperspectral images classification,” Remote Sensing, vol.10, no.7, article no.1068, 2018. doi: 10.3390/rs10071068
[46]	Z. Lu, B. Xu, L. Sun, T. Zhan and S. Tang, “3-D channel and spatial attention based multiscale spatial–spectral residual network for hyperspectral image classification,” IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, vol.13, pp.4311–4324, 2020. doi: 10.1109/JSTARS.2020.3011992