Fake Face Detection Based on Fusion of Spatial Texture and High-Frequency Noise

ZHANG Dengyong; QI Feifan; CHEN Jiahao; CHEN Jiaxin; GONG Rongrong; TIAN Yuehong; ZHANG Lebing

doi:10.23919/cje.2023.00.342

Article Contents

Article Navigation > Chinese Journal of Electronics > 2024 > Uncorrected proof

Dengyong ZHANG, Feifan QI, Jiahao CHEN, et al., “Fake Face Detection Based on Fusion of Spatial Texture and High-Frequency Noise,” Chinese Journal of Electronics, vol. x, no. x, pp. 1–10, xxxx doi: 10.23919/cje.2023.00.342

Citation:

Dengyong ZHANG, Feifan QI, Jiahao CHEN, et al., “Fake Face Detection Based on Fusion of Spatial Texture and High-Frequency Noise,” Chinese Journal of Electronics, vol. x, no. x, pp. 1–10, xxxx doi: 10.23919/cje.2023.00.342

Citation:

PDF( 3645 KB)

Fake Face Detection Based on Fusion of Spatial Texture and High-Frequency Noise

doi: 10.23919/cje.2023.00.342

ZHANG Dengyong^{1, 2
,
,},
QI Feifan^{1, 2
,},
CHEN Jiahao^{1, 2
,},
CHEN Jiaxin^{1, 2
,},
GONG Rongrong^3
,,
TIAN Yuehong^4
,,
ZHANG Lebing^5
,

1.
Hunan Provincial Key Laboratory of Intelligent Processing of Big Data on Transportation, School of Computer and Communication Engineering, Changsha University of Science and Technology, Changsha 410004, China
2.
School of Computer and Communication Engineering, Changsha University of Science and Technology, Changsha 410004, China
3.
Changsha Social Work College, Changsha 410004, China
4.
Changkuangao Beijing Technology Co., Ltd, Beijing 101100, China
5.
School of Computer and Artificial Intelligence, Huaihua University, Huaihua 418000,China

More Information

Author Bio:
Dengyong ZHANG received the B.S. and M.S. degree from Changsha University of Science and Technology, Changsha, China, in 2003, 2006 respectively. He received Ph.D. degree from Hunan University, Changsha, China, in 2018. Now, He is a Professor at Changsha University of Science and Technology. His current research interests include digital media forensics and image processing. (Email: zhdy@csust.edu.cn)

Feifan QI received the B.E. degree from Xi'an MingDe Institute of Technology, Xi'an, China, in 2021. He is currently pursuing the M.S. degree at School of Computer and Communication Engineering, Changsha University of Science and Technology, in Hunan. His research interests include image forensics and Deepfake face detection. (Email: 21208051609@stu.csust.edu.cn)

Jiahao CHEN received the B.E. degree from Hebei University of Engineering, Hebei, China, in 2020. He is currently pursuing the M.S. degree at School of Computer and Communication Engineering, Changsha University of Science and Technology, in Hunan. His research interest includes face forgery detection. (Email: cjh_160188@stu.csust.edu.cn)

Jiaxin CHEN received the B.S. degree from Central China Normal University, Wuhan, China, in 2017, and Ph.D. degree from Hunan University, Changsha, China in 2023. She is currently a Lecturer with Changsha University of Science and Technology, China. Her current research interests include multimedia forensics and Deepfake detection. She is a member of Technical Committee (TC) on Digital Forensics and Security of China Society of Image and Graphics. (Email: chenjiaxin@hnu.edu.cn)

Rongrong GONG received the B.S. and M.S. degree from Changsha University of Science and Technology, Changsha, China, in 2003, 2006 respectively.Now, He is an Assistant Professor at Changsha University of Science and Technology. His current research interests include digital media forensics and image processing. (Email: 1251031@csmzxy.edu.cn)

Yuehong TIAN received the M.S. degree from Changsha University of Science and Technology, Changsha, China, in 2003. He currently works at ChangKuanGao (Beijing) Technology Co., Ltd., mainly engaged in 3D computer graphics research. He is a senior member of the Chinese Graphics Society, a professional member of the China Computer Society, and ACM member. (Email: tianyhongcn@126.com)

Lebing ZHANG received the Ph.D. degrees in Computer Science and Technology from Hunan University, Changsha, China, in 2019. He is currently an Assistant Professor at the School of Computer and Artificial Intelligence, Huaihua University, China. His research interests include digital forensics, multimedia security, and deep learning. (Email: zhanglebing@hhtc.edu.cn)
Corresponding author: Email: zhdy@csust.edu.cn
Received Date: 2023-11-01
Accepted Date: 2024-04-02

Available Online: 2024-06-28

Abstract

Abstract

The rapid development of the Internet has led to the widespread dissemination of manipulated facial images, significantly impacting people’s daily lives. With the continuous advancement of Deepfake technology, the generated counterfeit facial images have become increasingly challenging to distinguish. There is an urgent need for a more robust and convincing detection method. Current detection methods mainly operate in the spatial domain and transform the spatial domain into other domains for analysis. With the emergence of Transformers, some researchers have also combined traditional convolutional networks with Transformers for detection. This paper explores the artifacts left by Deepfakes in various domains and, based on this exploration, proposes a detection method that utilizes the steganalysis rich model (SRM) to extract high-frequency noise to complement spatial features. We have designed two main modules to fully leverage the interaction between these two aspects based on traditional convolutional neural networks. The first is the multi-scale mixed feature attention module, which introduces artifacts from high-frequency noise into spatial textures, thereby enhancing the model’s learning of spatial texture features. The second is the multi-scale channel attention module, which reduces the impact of background noise by weighting the features. Our proposed method was experimentally evaluated on mainstream datasets, and a significant amount of experimental results demonstrate the effectiveness of our approach in detecting Deepfake forged faces, outperforming the majority of existing methods.
- Deepfakes detection,
- Steganalysis rich model,
- Image forensics,
- Mix attention

FullText(HTML)

References(30)

References

[1]	R. Tolosana, R. Vera-Rodriguez, J. Fierrez, et al., “Deepfakes and beyond: A survey of face manipulation and fake detection,” Information Fusion, vol. 64, pp. 131–148, 2020. doi: 10.1016/j.inffus.2020.06.014
[2]	Y. Z. Li, M. C. Chang, and S. W. Lyu, “In ictu oculi: Exposing AI created fake videos by detecting eye blinking,” in Proceedings of the 2018 IEEE International Workshop on Information Forensics and Security (WIFS), Hong Kong, China, pp. 1–7, 2018.
[3]	H. D. Li, W. Q. Luo, X. Q. Qiu, et al., “Identification of various image operations using residual-based features,” IEEE Transactions on Circuits and Systems for Video Technology, vol. 28, no. 1, pp. 31–45, 2018. doi: 10.1109/TCSVT.2016.2599849
[4]	X. Wu, Z. Xie, Y. T. Gao, et al., “SSTNet: Detecting manipulated faces through spatial, steganalysis and temporal features,” in Proceedings of the 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Barcelona, Spain, pp. 2952–2956, 2020.
[5]	J. W. Fei, Y. S. Dai, P. P. Yu, et al., “Learning second order local anomaly for general face forgery detection,” in Proceedings of the 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), New Orleans, LA, USA, pp. 20238–20248, 2022.
[6]	J. A. Stuchi, M. A. Angeloni, R. F. Pereira, et al., “Improving image classification with frequency domain layers for feature extraction,” in Proceedings of the IEEE 27th International Workshop on Machine Learning for Signal Processing (MLSP), Tokyo, Japan, pp. 1–6, 2017.
[7]	J. Fridrich and J. Kodovsky, “Rich models for steganalysis of digital images,” IEEE Transactions on Information Forensics and Security, vol. 7, no. 3, pp. 868–882, 2012. doi: 10.1109/TIFS.2012.2190402
[8]	F. Matern, C. Riess, and M. Stamminger, “Exploiting visual artifacts to expose deepfakes and face manipulations,” in Proceedings of the 2019 IEEE Winter Applications of Computer Vision Workshops (WACVW), Waikoloa, HI, USA, pp. 83–92, 2019.
[9]	A. Rössler, D. Cozzolino, L. Verdoliva, et al., “Faceforensics++: Learning to detect manipulated facial images,” in Proceedings of the 2019 IEEE/CVF International Conference on Computer Vision (ICCV), Seoul, Korea (South), pp. 1–11, 2019.
[10]	H. Dang, F. Liu, J. Stehouwer, et al., “On the detection of digital face manipulation,” in Proceedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA, pp. 5780–5789, 2020.
[11]	H. Q. Zhao, T. Y. Wei, W. B. Zhou, et al., “Multi-attentional deepfake detection,” in Proceedings of the 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Nashville, TN, USA, pp. 2185–2194, 2021.
[12]	L. Chen, Y. Zhang, Y. B. Song, et al., “Self-supervised learning of adversarial example: Towards good generalizations for deepfake detection,” in Proceedings of the 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition, New Orleans, LA, USA, pp. 18689–18698, 2022.
[13]	X. Y. Dong, J. M. Bao, D. D. Chen, et al., “Protecting celebrities from deepfake with identity consistency transformer,” in Proceedings of the 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition, New Orleans, LA, USA, pp. 9458–9468, 2022.
[14]	D. Cozzolino, G. Poggi, and L. Verdoliva, “Splicebuster: A new blind image splicing detector,” in Proceedings of the 2015 IEEE International Workshop on Information Forensics and Security (WIFS), Rome, Italy, pp. 1–6, 2015.
[15]	Y. Y. Qian, G. J. Yin, L. Sheng, et al., “Thinking in frequency: Face forgery detection by mining frequency-aware clues,” in Proceedings of the 16th European Conference on Computer Vision, Glasgow, UK, pp. 86–103, 2020.
[16]	Y. C. Luo, Y. Zhang, J. C. Yan, et al., “Generalizing face forgery detection with high-frequency features,” in Proceedings of the 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Nashville, TN, USA, pp. 16312–16321, 2021.
[17]	A. Vaswani, N. Shazeer, N. Parmar, et al., “Attention is all you need,” in Proceedings of the 31st International Conference on Neural Information Processing Systems, Long Beach, CA, USA, pp. 6000–6010, 2017.
[18]	B. V. Salim, Chyntia, J. O. Indrawan, et al., “Face shape classification using swin transformer model,” Procedia Computer Science, vol. 227, pp. 557–562, 2023. doi: 10.1016/j.procs.2023.10.558
[19]	Y. Z. Li, X. Yang, P. Sun, et al., “Celeb-DF: A large-scale challenging dataset for DeepFake forensics,” in Proceedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, WA, USA, pp. 3204–3213, 2020.
[20]	B. Dolhansky, J. Bitton, B. Pflaum, et al., “The DeepFake detection challenge (DFDC) dataset,” arXiv preprint, arXiv: 2006.07397, 2020.
[21]	B. J. Zi, M. H. Chang, J. J. Chen, et al., “WildDeepfake: A challenging real-world dataset for deepfake detection,” in Proceedings of the 28th ACM International Conference on Multimedia, Seattle, WA, USA, pp. 2382–2390, 2020.
[22]	M. X. Tan and Q. V. Le, “EfficientNet: Rethinking model scaling for convolutional neural networks,” in Proceedings of the 36th International Conference on Machine Learning, Long Beach, CA, USA, pp. 6105–6114, 2019.
[23]	J. Deng, W. Dong, R. Socher, et al., “ImageNet: A large-scale hierarchical image database,” in Proceedings of the 2009 IEEE Conference on Computer Vision and Pattern Recognition, Miami, FL, USA, pp. 248–255, 2009.
[24]	R. R. Selvaraju, M. Cogswell, A. Das, et al., “Grad-CAM: Visual explanations from deep networks via gradient-based localization,” in Proceedings of the 2017 IEEE International Conference on Computer Vision, Venice, Italy, pp. 618–626, 2017.
[25]	D. Afchar, V. Nozick, J. Yamagishi, et al., “MesoNet: A compact facial video forgery detection network,” in Proceedings of the 2018 IEEE International Workshop on Information Forensics and Security (WIFS), Hong Kong, China, pp. 1–7, 2018.
[26]	Z. Z. Liu, X. J. Qi, and P. H. S. Torr, “Global texture enhancement for fake face detection in the wild,” in Proceedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA, pp. 8057–8066, 2020.
[27]	C. R. Wang and W. H. Deng, “Representative forgery mining for fake face detection,” in Proceedings of the 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Nashville, TN, USA, pp. 14918–14927, 2021.
[28]	F. Chollet, “Xception: Deep learning with depthwise separable convolutions,” in Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA, pp. 1800–1807, 2017.
[29]	J. K. Wang, Z. X. Wu, W. H. Ouyang, et al., “M2TR: Multi-modal multi-scale transformers for deepfake detection,” in Proceedings of the 2022 International Conference on Multimedia Retrieval, Newark, NJ, USA, pp. 615–623, 2022.
[30]	Z. Q. Guo, G. B. Yang, D. Y. Zhang, et al., “Rethinking gradient operator for exposing AI-enabled face forgeries,” Expert Systems with Applications, vol. 215, article no. 119361, 2023. doi: 10.1016/j.eswa.2022.119361