ERSinAI2023+ Lightweight Steganography Detection Method Based on Multiple Residual Structures and Transformer

LI Hao; ZHANG Yi; WANG Jinwei; ZHANG Weiming; LUO Xiangyang

doi:10.23919/cje.2022.00.452

Article Contents

Article Navigation > Chinese Journal of Electronics > 2024 > Uncorrected proof

Hao LI, Yi ZHANG, Jinwei WANG, et al., “ERSinAI2023+ Lightweight Steganography Detection Method Based on Multiple Residual Structures and Transformer,” Chinese Journal of Electronics, vol. 33, no. 5, pp. 1–14, 2024 doi: 10.23919/cje.2022.00.452

Citation:

Hao LI, Yi ZHANG, Jinwei WANG, et al., “ERSinAI2023+ Lightweight Steganography Detection Method Based on Multiple Residual Structures and Transformer,” Chinese Journal of Electronics, vol. 33, no. 5, pp. 1–14, 2024 doi: 10.23919/cje.2022.00.452

Citation:

PDF( 9352 KB)

ERSinAI2023+ Lightweight Steganography Detection Method Based on Multiple Residual Structures and Transformer

doi: 10.23919/cje.2022.00.452

LI Hao^1
,,
ZHANG Yi^1
,,
WANG Jinwei^2
,,
ZHANG Weiming^3
,,
LUO Xiangyang^{1
,
,}

1.
Key Laboratory of Cyberspace Situation Awareness of Henan Province, Zhengzhou 450001, China
2.
Nanjing University of Information Science & Technology, Nanjing 210044, China
3.
University of Science and Technology of China, Hefei 230052, China

More Information

Author Bio:
Hao LI received his B.S. degree in Computer Science and Technology from Zhengzhou University in 2015 and his M.S. degree from Central South University of Forestry and Technology, China, in 2019. He is currently pursuing the Ph.D. degree at Henan Key Laboratory of Cyberspace Situation Awareness, Zhengzhou, China. His research interests are image steganography and steganalysis technique. (Email: li15575963101hao@163.com)

Yi ZHANG reveived her B.S., M.S. degrees, and Ph.D. degree from Henan Key Laboratory of Cyberspace Situation Awareness, Zhengzhou, China, in 2014, 2017, and 2021 respectively. Her research interests are image robust steganography and steganalysis technique. (Email: tzyy4001@sina.com)

Jinwei WANG is a professor from Nanjing University of Information Science & Technology. He received B.S. degree in Automatic Control from Inner Mongolia University of Technology in 2000. He received Ph.D. in Information Security at Nanjing University of Science & Technology in 2007 and was a visiting scholar in Service Anticipation Multimedia Innovation (SAMI) Lab of France Telecom R&D Center (Beijing) in 2006. He worked as a senior engineer at the 28th research institute, CETC from 2007 to 2010. He worked as a visiting scholar at New Jersey Institute of Technology, NJ, USA from 2014 to 2015. His research interests include multimedia copyright protection, multimedia forensics, multimedia encryption and data authentication. He has published more than 50 papers, hosted and participated in more than 10 projects. (Email: wjwei_2004@163.com)

Weiming ZHANG received the M.S. and Ph.D. degrees from Zhengzhou Information Science and Technology Institute, China in 2002 and 2005 respectively. Currently, he is a professor with the School of Information Science and Technology, University of Science and Technology of China. His research interests include information hiding and multimedia security. (Email: zhangwm@ustc.edu.cn)

Xiangyang LUO received his B.S., M.S., and Ph.D. degrees from the State Key Laboratory of Mathematical Engineering and Advanced Computing, University of information Engineering, Zhengzhou, China, in 2001, 2004, and 2010, respectively. He is the author or co-author of more than 200 refereed international journal and conference papers. He is currently a full professor of Henan Key Laboratory of Cyberspace Situation Awareness. His research interests are in network and information security. (Email: luoxy_ieu@sina.com)
Corresponding author: Email: luoxy_ieu@sina.com
Received Date: 2022-12-28
Accepted Date: 2024-01-12

Available Online: 2024-03-06

Abstract

Abstract

Existing deep learning-based steganography detection methods utilize convolution to automatically capture and learn steganographic features, yielding higher detection efficiency compared to manually designed steganography detection methods. Detection methods based on convolutional neural network frameworks can extract global features by increasing the network’s depth and width. These frameworks are not highly sensitive to global features and can lead to significant resource consumption. This manuscript proposes a lightweight steganography detection method based on multiple residual structures and Transformer (ResFormer). A multi-residuals block based on channel rearrangement is designed in the preprocessing layer. Multiple residuals are used to enrich the residual features and channel shuffle is used to enhance the feature representation capability. A lightweight convolutional and Transformer feature extraction backbone is constructed, which reduces the computational and parameter complexity of the network by employing depth-wise separable convolutions. This backbone integrates local and global image features through the fusion of convolutional layers and Transformer, enhancing the network’s ability to learn global features and effectively enriching feature diversity. An effective weighted loss function is introduced for learning both local and global features, BiasLoss loss function is used to give full play to the role of feature diversity in classification, and cross-entropy loss function and contrast loss function are organically combined to enhance the expression ability of features. Based on BossBase-1.01, BOWS2 and ALASKA#2, extensive experiments are conducted on the stego images generated by spatial and JPEG domain adaptive steganographic algorithms, employing both classical and state-of-the-art steganalysis techniques. The experimental results demonstrate that compared to the SRM, SRNet, SiaStegNet, CSANet, LWENet, and SiaIRNet methods, the proposed ResFormer method achieves the highest reduction in the parameter, up to 91.82%. It achieves the highest improvement in detection accuracy, up to 5.10%. Compared to the SRNet and EWNet methods, the proposed ResFormer method achieves an improvement in detection accuracy for the J-UNIWARD algorithm by 5.78% and 6.24%, respectively.
- Steganalysis,
- multiple residual blocks,
- transformer,
- channel shuffle

http://bows2.ec-lille.fr

FullText(HTML)

References(40)

References

[1]	J. Wen, Y. Q. Deng, W. L. Peng, et al., “Linguistic steganalysis via fusing multi-granularity attentional text features,” Chinese Journal of Electronics, vol. 32, no. 1, pp. 76–84, 2023. doi: 10.23919/cje.2022.00.009
[2]	J. W. Luo, M. J. Yu, X. L. Yin, et al., “Binary image steganalysis based on symmetrical local residual patterns,” Chinese Journal of Electronics, vol. 31, no. 4, pp. 752–763, 2022. doi: 10.1049/cje.2020.00.414
[3]	A. Westfeld and A. Pfitzmann, “Attacks on steganographic systems,” in Proceedings of 3rd International Workshop on Information Hiding, Dresden, Germany, pp. 61–76, 2000.
[4]	S. Dumitrescu, X. L. Wu, and Z. Wang, “Detection of LSB steganography via sample pair analysis,” IEEE Transactions on Signal Processing, vol. 51, no. 7, pp. 1995–2007, 2023. doi: 10.1109/TSP.2003.812753
[5]	K. Sullivan, U. Madhow, S. Chandrasekaran, et al., “Steganalysis for Markov cover data with applications to images,” IEEE Transactions on Information Forensics and Security, vol. 1, no. 2, pp. 275–287, 2006. doi: 10.1109/TIFS.2006.873595
[6]	Q. Z. Liu, A. H. Sung, and M. Y. Qiao, “Neighboring joint density-based jpeg steganalysis,” ACM Transactions on Intelligent Systems and Technology, vol. 2, no. 2, article no. 16, 2011. doi: 10.1145/1899412.1899420
[7]	J. Fridrich and J. Kodovsky, “Rich models for steganalysis of digital images,” IEEE Transactions on Information Forensics and Security, vol. 7, no. 3, pp. 868–882, 2012. doi: 10.1109/TIFS.2012.2190402
[8]	T. Denemark, V. Sedighi, V. Holub, et al., “Selection-channel-aware rich model for steganalysis of digital images,” in 2014 IEEE International Workshop on Information Forensics and Security, Atlanta, GA, USA, pp. 48–53, 2014.
[9]	T. Denemark, J. Fridrich, and P. Comesaña-Alfaro, “Improving selection-channel-aware steganalysis features,” Electronic Imaging, vol. 28, no. 8, article no. art00016, 2016. doi: 10.2352/ISSN.2470-1173.2016.8.MWSF-080
[10]	W. X. Tang, H. D. Li, W. Q. Luo, et al., “Adaptive steganalysis based on embedding probabilities of pixels,” IEEE Transactions on Information Forensics and Security, vol. 11, no. 4, pp. 734–745, 2016. doi: 10.1109/TIFS.2015.2507159
[11]	Y. Y. Ma, X. Y. Luo, X. L. Li, et al., “Selection of rich model steganalysis features based on decision rough set α-positive region reduction,” IEEE Transactions on Circuits and Systems for Video Technology, vol. 29, no. 2, pp. 336–350, 2019. doi: 10.1109/TCSVT.2018.2799243
[12]	K. Simonyan and A. Zisserman, “Very deep convolutional networks for large-scale image recognition,” arXiv preprint, arXiv: 1409.1556, 2015.
[13]	K. M. He, X. Y. Zhang, S. Q. Ren, et al., “Deep residual learning for image recognition,” in Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA, pp. 770–778, 2016.
[14]	M. X. Tan and Q. Le, “Efficientnetv2: Smaller models and faster training,” in Proceedings of the 38th International Conference on Machine Learning, Virtual Event, pp. 10096–10106, 2021.
[15]	Y. L. Qian, J. Dong, W. Wang, et al., “Learning and transferring representations for image steganalysis using convolutional neural network,” in Proceedings of the 2016 IEEE International Conference on Image Processing, Phoenix, AZ, USA, pp. 2752–2756, 2016.
[16]	G. S. Xu, H. Z. Wu, and Y. Q. Shi, “Structural design of convolutional neural networks for steganalysis,” IEEE Signal Processing Letters, vol. 23, no. 5, pp. 708–712, 2016. doi: 10.1109/LSP.2016.2548421
[17]	G. S. Xu, “Deep convolutional neural network to detect J-UNIWARD,” in Proceedings of the 5th ACM Workshop on Information Hiding and Multimedia Security, Philadelphia, PA, USA, pp. 67–73, 2017.
[18]	M. Chen, V. Sedighi, M. Boroumand, et al., “JPEG-phase-aware convolutional neural network for steganalysis of JPEG images,” in Proceedings of the 5th ACM Workshop on Information Hiding and Multimedia Security, Philadelphia, PA, USA, pp. 75–84, 2017.
[19]	J. Ye, J. Q. Ni, and Y. Yi, “Deep learning hierarchical representations for image steganalysis,” IEEE Transactions on Information Forensics and Security, vol. 12, no. 11, pp. 2545–2557, 2017. doi: 10.1109/TIFS.2017.2710946
[20]	M. Yedroudj, F. Comby, and M. Chaumont, “Yedroudj-Net: An efficient CNN for spatial steganalysis,” in Proceedings of the 2018 IEEE International Conference on Acoustics, Speech and Signal Processing, Calgary, AB, Canada, pp. 2092–2096, 2018.
[21]	C. F. Tsang and J. Fridrich, “Steganalyzing images of arbitrary size with CNNs,” Electronic Imaging, vol. 30, no. 7, article no. art00005, 2018. doi: 10.2352/ISSN.2470-1173.2018.07.MWSF-121
[22]	M. Boroumand, M. Chen, and J. Fridrich, “Deep residual network for steganalysis of digital images,” IEEE Transactions on Information Forensics and Security, vol. 14, no. 5, pp. 1181–1193, 2019. doi: 10.1109/TIFS.2018.2871749
[23]	R. Zhang, F. Zhu, J. Y. Liu, et al., “Depth-wise separable convolutions and multi-level pooling for an efficient spatial CNN-based steganalysis,” IEEE Transactions on Information Forensics and Security, vol. 15 pp. 1138–1150, 2020. doi: 10.1109/TIFS.2019.2936913
[24]	W. K. You, H. Zhang, and X. F. Zhao, “A Siamese CNN for image steganalysis,” IEEE Transactions on Information Forensics and Security, vol. 16 pp. 291–306, 2021. doi: 10.1109/TIFS.2020.3013204
[25]	A. T. Su, X. F. Zhao, and X. L. He, “Arbitrary-sized JPEG steganalysis based on fully convolutional network,” in Proceedings of 20th International Workshop on Digital Watermarking, Beijing, China, pp. 197–211, 2021.
[26]	Q. L. Liu, J. Q. Ni, and M. X. Jian, “Effective JPEG steganalysis using non-linear pre-processing and residual channel-spatial attention,” in Proceedings of the 2022 IEEE International Conference on Multimedia and Expo, Taipei, China, pp. 1–6, 2022.
[27]	S. W. Weng, M. F. Chen, L. F. Yu, et al., “Lightweight and effective deep image steganalysis network,” IEEE Signal Processing Letters, vol. 29 pp. 1888–1892, 2022. doi: 10.1109/LSP.2022.3201727
[28]	H. Li, J. W. Wang, N. Xiong, et al., “A Siamese inverted residuals network image steganalysis scheme based on deep learning,” ACM Transactions on Multimedia Computing, Communications and Applications, vol. 19, no. 6, pp. 1–23, 2023. doi: 10.1145/3579166
[29]	Y. T. Chen, R. L. Xia, K. Zou, et al., “FFTI: Image inpainting algorithm via features fusion and two-steps inpainting,” Journal of Visual Communication and Image Representation, vol. 91, article no. 103776, 2023. doi: 10.1016/j.jvcir.2023.103776
[30]	Y. T. Chen, R. L. Xia, K. Yang, et al., “MFFN: Image super-resolution via multi-level features fusion network,” The Visual Computer, vol. 10, no. 2, pp. 489–504, 2024. doi: 10.1007/s00371-023-02795-0
[31]	Y. T. Chen, R. L. Xia, K. Yang, et al., “DGCA: High resolution image inpainting via DR-GAN and contextual attention,” Multimedia Tools and Applications, vol. 82, no. 30, pp. 47751–47771, 2023. doi: 10.1007/s11042-023-15313-0
[32]	Y. T. Chen, R. L. Xia, K. Yang, et al., “DARGS: Image inpainting algorithm via deep attention residuals group and semantics,” Journal of King Saud University-Computer and Information Sciences, vol. 35, no. 6, article no. 101567, 2023. doi: 10.1016/j.jksuci.2023.101567
[33]	V. Holub and J. Fridrich, “Designing steganographic distortion using directional filters,” in Proceedings of the 2012 IEEE International Workshop on Information Forensics and Security, Costa Adeje, Spain, pp. 234–239, 2012.
[34]	V. Holub and J. Fridrich, “Digital image steganography using universal distortion,” in Proceedings of the First ACM Workshop on Information Hiding and Multimedia Security, Montpellier, France, pp. 59–68, 2013.
[35]	B. Li, M. Wang, J. W. Huang, et al., “A new cost function for spatial image steganography,” in Proceedings of the 2014 IEEE International Conference on Image Processing, Paris, France, pp. 4206–4210, 2014.
[36]	L. Abrahamyan, V. Ziatchin, Y. M. Chen, et al., “Bias loss for mobile neural networks,” in Proceedings of the 2021 IEEE/CVF International Conference on Computer Vision, Montreal, QC, Canada, pp. 6536–6546, 2021.
[37]	R. R. Selvaraju, M. Cogswell, A. Das, et al., “Grad-CAM: Visual explanations from deep networks via gradient-based localization,” in Proceedings of the IEEE International Conference on Computer Vision, Venice, Italy, pp. 618–626, 2017.
[38]	P. Bas, T. Filler, and T. Pevný, ““Break our steganographic system”: The ins and outs of organizing boss,” in Proceedings of 13th International Workshop on Information Hiding, Prague, Czech Republic, pp. 59–70, 2011.
[39]	R. Cogranne, Q. Giboulot, and P. Bas, “The ALASKA steganalysis challenge: A first step towards steganalysis,” in Proceedings of the ACM Workshop on Information Hiding and Multimedia Security, Paris, France, pp. 125–137, 2019.
[40]	D. P. Kingma and J. L. Ba, “Adam: A method for stochastic optimization,” in Proceedings of 3rd International Conference on Learning Representations, San Diego, CA, USA, pp. 1–15, 2015.