Linguistic Steganalysis via Fusing Multi-Granularity Attentional Text Features

WEN Juan; DENG Yaqian; PENG Wanli; XUE Yiming

doi:10.23919/cje.2022.00.009

Volume 32 Issue 1

Jan. 2023

Turn off MathJax

Article Contents

Article Navigation > Chinese Journal of Electronics > 2023 > 32(1): 76-84

WEN Juan, DENG Yaqian, PENG Wanli, XUE Yiming. Linguistic Steganalysis via Fusing Multi-Granularity Attentional Text Features[J]. Chinese Journal of Electronics, 2023, 32(1): 76-84. doi: 10.23919/cje.2022.00.009

Citation:

WEN Juan, DENG Yaqian, PENG Wanli, XUE Yiming. Linguistic Steganalysis via Fusing Multi-Granularity Attentional Text Features[J]. Chinese Journal of Electronics, 2023, 32(1): 76-84. doi: 10.23919/cje.2022.00.009

Citation:

PDF( 2638 KB)

Linguistic Steganalysis via Fusing Multi-Granularity Attentional Text Features

doi: 10.23919/cje.2022.00.009

1.
College of Information and Electrical Engineering, China Agricultural University, Beijing 100094, China

Funds: This work was supported by the National Natural Science Foundation of China (61872368, 61802410)

More Information

Author Bio:
Juan WEN received the B.E. degree in information engineering and Ph.D. degree in signal and information processing from Beijing University of Posts and Telecommunications. She is now an Associate Professor at China Agricultural University. Her research interests include artificial intelligence, information hiding, and natural language processing. (Email: wenjuan@cau.edu.cn)

Yaqian DENG is currently pursuing the M.E. degree in computer technology with the College of Information and Electrical Engineering, China Agricultural University. Her research interest includes information hiding. (Email: dengyaqian@cau.edu.cn)

Wanli PENG received the B.E. degree from College of Physics and Electronic Engineering, Harbin Normal University, China, in 2018. He is currently pursuing the Ph.D. degree with the College of Information and Electrical Engineering, China Agricultural University, Beijing, China. His research interests include information hiding and natural language processing. (Email: hunanpwl@cau.edu.cn)

Yiming XUE (corresponding author) is currently a Professor in the College of Information and Electrical Engineering, China Agricultural University. His research interests include multimedia processing, multimedia security, and VLSI design. (Email: xueym@cau.edu.cn)
Received Date: 2022-01-17
Accepted Date: 2022-05-13

Available Online: 2022-07-04

Publish Date: 2023-01-05

Abstract

Abstract

Deep learning based language models have improved generation-based linguistic steganography, posing a huge challenge for linguistic steganalysis. The existing neural-network-based linguistic steganalysis methods are incompetent to deal with complicated text because they only extract single-granularity features such as global or local text features. To fuse multi-granularity text features, we present a novel linguistic steganalysis method based on attentional bidirectional long-short-term-memory (BiLSTM) and short-cut dense convolutional neural network (CNN). The BiLSTM equipped with the scaled dot-product attention mechanism is used to capture the long dependency representations of the input sentence. The CNN with the short-cut and dense connection is exploited to extract sufficient local semantic features from the word embedding matrix. We connect two structures in parallel, concatenate the long dependency representations and the local semantic features, and classify the stego and cover texts. The results of comparative experiments demonstrate that the proposed method is superior to the state-of-the-art linguistic steganalysis.
- Information hiding,
- Natural language processing,
- Linguistic steganalysis,
- Attentional BiLSTM,
- Dense connection

FullText(HTML)

References(42)

References

[1]	F. Petitcolas, R. Anderson, and M. Kuhn, “Information hiding - A survey,” Proceedings of the IEEE, vol.87, no.7, pp.1062–1078, 1999. doi: 10.1109/5.771065
[2]	V. Sedighi, R. Cogranne, and J. Fridrich, “Content-adaptive steganography by minimizing statistical detectability,” IEEE Transactions on Information Forensics and Security, vol.11, no.2, pp.221–234, 2016. doi: 10.1109/TIFS.2015.2486744
[3]	Y. Luo, J. Qin, Y. Tan, et al., “Coverless real-time image information hiding based on image block matching and dense convolutional network,” Journal of Real-Time Image Processing, vol.17, no.1, pp.125–135, 2020. doi: 10.1007/s11554-019-00917-3
[4]	Y. Tew and K. Wong, “An overview of information hiding in H. 264/AVC compressed video,” IEEE Transactions on Circuits and Systems for Video Technology, vol.24, no.2, pp.305–319, 2014. doi: 10.1109/TCSVT.2013.2276710
[5]	Y. Xue, J. Zhou, H. Zeng, et al., “An adaptive steganographic scheme for H. 264/AVC video with distortion optimization,” Signal Processing: Image Communication, vol.76, no.8, pp.22–30, 2019. doi: 10.1016/j.image.2019.04.012
[6]	Y. Luo and Y. Huang, “Text steganography with high embedding rate: Using recurrent neural networks to generate chinese classic poetry,” in Proceedings of the 5th ACM Workshop on Information Hiding and Multimedia Security (IH&MMSec), Philadelphia, PA, USA, pp.99–104, 2017.
[7]	J. Wen, X. Zhou, M. Li, et al., “A novel natural language steganographic framework based on image description neural network,” Journal of Visual Communication and Image Representation, vol.61, no.5, pp.157–169, 2019. doi: 10.1016/j.jvcir.2019.03.016
[8]	T. Fang, M. Jaggi, and K. J. Argyraki, “Generating steganographic text with lstms,” in Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (ACL), Vancouver, Canada, pp.100–106, 2017.
[9]	Z. Yang, X. Guo, Z. Chen, Y. Huang, et al., “RNN-stega: Linguistic steganography based on recurrent neural networks,” IEEE Transactions on Information Forensics and Security, vol.14, no.5, pp.1280–1295, 2018. doi: 10.1109/TIFS.2018.2871746
[10]	L. Xiang, X. Sun, G. Luo, and B. Xia, “Linguistic steganalysis using the features derived from synonym frequency,” Multimedia Tools and Applications, vol.71, no.3, pp.1893–1911, 2014. doi: 10.1007/s11042-012-1313-8
[11]	H. Yang and X. Cao, “Linguistic steganalysis based on meta features and immune mechanism,” Chinese Journal of Electronics, vol.19, no.4, pp.661–666, 2010.
[12]	J. Wen, X. Zhou, P. Zhong, et al., “Convolutional neural network based text steganalysis,” IEEE Signal Processing Letters, vol.26, no.3, pp.460–464, 2019. doi: 10.1109/LSP.2019.2895286
[13]	Z. Yang, K. Wang, J. Li, et al., “TS-RNN: Text steganalysis based on recurrent neural networks,” IEEE Signal Processing Letters, vol.26, no.12, pp.1743–1747, 2019. doi: 10.1109/LSP.2019.2920452
[14]	Y. Niu, J. Wen, P. Zhong, et al., “A hybrid R-BILSTM-C neural network based text steganalysis,” IEEE Signal Processing Letters, vol.26, no.12, pp.1907–1911, 2019. doi: 10.1109/LSP.2019.2953953
[15]	H. Yang, Y. Bao, Z. Yang, et al., “Linguistic steganalysis via densely connected LSTM with feature pyramid,” in Proceedings of IH&MMSec’20: ACM Workshop on Information Hiding and Multimedia Security, Denver, CO, USA, pp.5–10, 2020.
[16]	Z. Yang, Y. Huang, and Y. Zhang, “TS-CSW: Text steganalysis and hidden capacity estimation based on convolutional sliding windows,” Multimedia Tools and Applications, vol.79, no.25, pp.18293–18316, 2020. doi: 10.1007/s11042-020-08716-w
[17]	W. Peng, J. Zhang, Y. Xue, et al., “Real-time text steganalysis based on multi-stage transfer learning,” IEEE Signal Processing Letters, vol.28, no.12, pp.1510–1514, 2021. doi: 10.1109/LSP.2021.3097241
[18]	Y. Xue, L. Kong, W. Peng, et al., “An effective linguistic steganalysis framework based on hierarchical mutual learning,” Information Sciences, vol.586, no.2, pp.140–154, 2022. doi: 10.1016/j.ins.2021.11.086
[19]	A. Vaswani, N. Shazeer, N. Parmar, et al., “Attention is all you need,” in Proceedings of 31st International Conference on Neural Information Processing Systems, Long Beach, CA, USA, pp.5998–6008, 2017.
[20]	K. He, X. Zhang, S. Ren, et al., “Deep residual learning for image recognition,” in 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA, pp.770–778, 2016,
[21]	G. Huang, Z. Liu, L. van der Maaten, et al., “Densely connected convolutional networks,” in 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA, pp.2261–2269, 2017.
[22]	L. Xiang, X. Yang, J. Zhang, et al., “A word-frequency-preserving steganographic method based on synonym substitution,” International Journal of Computational Science and Engineering, vol.19, no.1, pp.132–139, 2019. doi: 10.1504/IJCSE.2019.099648
[23]	M. Li, K. Mu, P. Zhong, et al., “Generating steganographic image description by dynamic synonym substitution,” Signal Processing, vol.164, no.9, pp.193–201, 2019. doi: 10.1016/j.sigpro.2019.06.014
[24]	L. Xiang, J. Yu, C. Yang, et al., “A word-embedding-based steganalysis method for linguistic steganography via synonym substitution,” IEEE Access, vol.6, no.10, pp.64131–64141, 2018. doi: 10.1109/ACCESS.2018.2878273
[25]	H. M. Meral, B. Sankur, A. S. Özsoy, et al., “Natural language watermarking via morphosyntactic alterations,” Computer Speech & Language, vol.23, no.1, pp.107–125, 2009. doi: 10.1016/j.csl.2008.04.001
[26]	L. Xiang, W. Wu, X. Li, et al., “A linguistic steganography based on word indexing compression and candidate selection,” Multimedia Tools and Applications, vol.77, no.26, pp.28969–28989, 2018. doi: 10.1007/s11042-018-6072-8
[27]	F. Z. Dai and Z. Cai, “Towards near-imperceptible steganographic text,” in Proceedings of the 57th Conference of the Association for Computational Linguistics (ACL), Florence, Italy, pp.4303–4308, 2019..
[28]	Z. M. Ziegler, Y. Deng, and A. M. Rush, “Neural linguistic steganography,” in Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), Hong Kong, China, pp.1210–1215, 2019.
[29]	X. Zhou, W. Peng, B. Yang, et al., “Linguistic steganography based on adaptive probability distribution,” IEEE Transactions on Dependable and Secure Computing, vol.19, no.5, pp.2982–2997, 2022. doi: 10.1109/TDSC.2021.3079957
[30]	B. Yang, W. Peng, Y. Xue, et al., “A Generation-based Text Steganography by Maintaining Consistency of Probability Distribution,” KSII Transactions on Internet and Information Systems, vol.15, no.11, pp.4184–4202, 2021. doi: 10.3837/tiis.2021.11.017
[31]	B. Yi, H. Wu, G. Feng, et al., “ALiSa: Acrostic linguistic steganography based on BERT and Gibbs sampling,” IEEE Signal Processing Letters, vol.29, no.4, pp.687–691, 2022. doi: 10.1109/LSP.2022.3152126
[32]	G. Deepthi, N. V.SriLakshmi, P. Mounika, et al., “Linguistic steganography based on automatically generated paraphrases using recurrent neural networks,” Mobile Computing and Sustainable Informatics, vol.68, no.1, pp.723–732, 2022.
[33]	C. M. Taskiran, U. Topkara, M. Topkara, et al., “Attacks on lexical natural language steganography systems,” in Proceedings of SPIE 6072, Security, Steganography, and Watermarking of Multimedia Contents VⅢ, SPIE, article no.607209, 2006.
[34]	Z. Chen, L. Huang, H. Miao, et al., “Steganalysis against substitution-based linguistic steganography based on context clusters,” Computers Electrical Engineering, vol.37, no.6, pp.1071–1081, 2011. doi: 10.1016/j.compeleceng.2011.07.004
[35]	X. Glorot, A. Bordes, and Y. Bengio, “Deep sparse rectifier neural networks,” in Proceedings of the 14th International Conference on Artificial Intelligence and Statistics (AISTATS), Fort Lauderdale, USA, pp.315–323, 2011.
[36]	N. Srivastava, G. Hinton, A. Krizhevsky, et al., “Dropout: A simple way to prevent neural networks from overfitting,” The Journal of Machine Learning Research, vol.15, no.1, pp.1929–1958, 2014.
[37]	T. Lin, M. Maire, S. J. Belongie, et al., “Microsoft COCO: Common objects in context,” in Proceedings of 13th European Conference on Computer Vision (ECCV 2014), Zurich, Switzerland, pp.740–755, 2014.
[38]	A. Go, R. Bhayani, and L. Huang, “Twitter sentiment classification using distant supervision,” available at: https://cs.stanford.edu/people/alecmgo/papers/TwitterDistantSupervision09.pdf, 2009.
[39]	A. L. Maas, R. E. Daly, P. T. Pham, et al., “Learning word vectors for sentiment analysis,” in Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies, Portland, Oregon, USA, pp.142–150, 2011.
[40]	Kaggle, “All the news,” available at: https://www.kaggle.com/snapcrack/all-the-news/data, 2017-8-20.
[41]	D. P. Kingma and J. Ba, “Adam: A method for stochastic optimization,” in Proceedings of 3rd International Conference on Learning Representations (ICLR), San Diego, CA, USA, pp.1–15, 2015.
[42]	Z. Yang, S. Zhang, Y. Hu, et al., “VAE-Stega: Linguistic steganography based on variational auto-encoder,” IEEE Trans. Inf. Forensics Secur., vol.16, no.1, pp.880–895, 2021. doi: 10.1109/TIFS.2020.3023279