Robust Regularization Design of Graph Neural Networks Against Adversarial Attacks Based on Lyapunov Theory

YAN Wenjie; LI Ziqi; QI Yongjun

doi:10.23919/cje.2022.00.342

Article Contents

Article Navigation > Chinese Journal of Electronics > 2024 > Uncorrected proof

Wenjie YAN, Ziqi LI, and Yongjun QI, “Robust Regularization Design of Graph Neural Networks Against Adversarial Attacks Based on Lyapunov Theory,” Chinese Journal of Electronics, vol. 33, no. 3, pp. 1–10, 2024 doi: 10.23919/cje.2022.00.342

Citation:

Wenjie YAN, Ziqi LI, and Yongjun QI, “Robust Regularization Design of Graph Neural Networks Against Adversarial Attacks Based on Lyapunov Theory,” Chinese Journal of Electronics, vol. 33, no. 3, pp. 1–10, 2024 doi: 10.23919/cje.2022.00.342

Citation:

PDF( 6625 KB)

Robust Regularization Design of Graph Neural Networks Against Adversarial Attacks Based on Lyapunov Theory

doi: 10.23919/cje.2022.00.342

YAN Wenjie^{1
,
,},
LI Ziqi¹,
QI Yongjun²

1.
School of Artificial Intelligence, Hebei University of Technology, Tianjin 300401, China
2.
School of Computer Science and Engineering of North China Institute of Aerospace Engineering, Langfang 065000, China

More Information

Author Bio:
Wenjie YAN is currently an Associate Professor with the School of Articial Intelligence, Hebei University of Technology. His current research interests include deep learning and data mining. (Email: wenjieyanhit@163.com)

Ziqi LI is an M.S. degree candidate at the School of Articial Intelligence, Hebei University of Technology. Her current research interests include deep learning and data mining

Yongjun QI is currently a Senior Engineer with the School of Computer Science and Engineering of North China Institute of Aerospace Engineering. His research interests include computer vision and deep learning
Corresponding author: Email: wenjieyanhit@163.com
Received Date: 2022-10-08
Accepted Date: 2023-04-04

Available Online: 2023-07-13

Abstract

Abstract

The robustness of graph neural networks (GNNs) is a critical research topic in deep learning. Many researchers have designed regularization methods to enhance the robustness of neural networks, but there is a lack of theoretical analysis on the principle of robustness. In order to tackle the weakness of current robustness designing methods, this paper gives new insights into how to guarantee the robustness of GNNs. A novel regularization strategy named Lya-Reg is designed to guarantee the robustness of GNNs by Lyapunov theory. Our results give new insights into how regularization can mitigate the various adversarial effects on different graph signals. Extensive experiments on various public datasets demonstrate that the proposed regularization method is more robust than the state-of-the-art methods such as $ L_1 $-norm, $ L_2 $-norm, $ L_{21} $-norm, Pro-GNN, PA-GNN and GARNET against various types of graph adversarial attacks.
- Deep learning,
- Graph neural network,
- Robustness,
- Lyapunov,
- Regularization

FullText(HTML)

References(26)

References

[1]	A. Ilyas, L. Engstrom, A. Athalye, et al., “Black-box adversarial attacks with limited queries and information,” in Proceedings of the 35th International Conference on Machine Learning, Stockholm, Sweden, pp.2137–2146, 2018.
[2]	N. Carlini and D. Wagner, “Towards evaluating the robustness of neural networks,” in Proceedings of 2017 IEEE Symposium on Security and Privacy, San Jose, CA, USA, pp. 39–57, 2017.
[3]	C. H. Deng, X. Y. Li, Z. Feng, et al., “GARNET: Reduced-rank topology learning for robust and scalable graph neural networks,” in Proceedings of the First Learning on Graphs Conference, Virtual Event, article no.3, pp.1-23, 2022.
[4]	X. F. Tang, Y. D. Li, Y. W. Sun, et al., “Transferring robustness for graph neural network against poisoning attacks,” in Proceedings of the 13th International Conference on Web Search and Data Mining, Houston, TX, USA, pp.600–608, 2020.
[5]	T. N. Kipf and M. Welling, “Semi-supervised classification with graph convolutional networks,” in Proceedings of the 5th International Conference on Learning Representations, Toulon, France, pp.1-14, 2017.
[6]	P. Veličković, G. Cucurull, A. Casanova, et al., “Graph attention networks,” in Proceedings of the 6th International Conference on Learning Representations, Vancouver, BC, Canada, pp.1-12, 2018.
[7]	J. Klicpera, A. Bojchevski and S. Günnemann, “Predict then propagate: Graph neural networks meet personalized PageRank,” in Proceedings of the 7th International Conference on Learning Representations, New Orleans, LA, USA, pp.1-15, 2019.
[8]	Y. Ma, X. R. Liu, T. Zhao, et al., “A unified view on graph neural networks as graph signal denoising,” in Proceedings of the 30th ACM International Conference on Information & Knowledge Management, Virtual Event, Australia, pp. 1202–1211, 2021.
[9]	W. Jin, Y. Ma, X. R. Liu, et al., “Graph structure learning for robust graph neural networks,” in Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, Virtual Event, CA, USA, pp. 66–74, 2020.
[10]	X. R. Liu, W. Jin, Y. Ma, et al., “Elastic graph neural networks,” in Proceedings of the 38th International Conference on Machine Learning, Virtual Event, pp.6837–6849, 2021.
[11]	F. Farnia, J. M. Zhang, and D. Tse, “Generalizable adversarial training via spectral normalization,” in Proceedings of the 7th International Conference on Learning Representations, New Orleans, LA, USA, pp.1-27, 2019.
[12]	T. Q. Zhao, J. H. Wang, X. N. Lu, et al., “Neural Lyapunov control for power system transient stability: A deep learning-based approach,” IEEE Transactions on Power Systems, vol. 37, no. 2, pp. 955–966, 2022. doi: 10.1109/TPWRS.2021.3102857
[13]	M. D. Si and Q. S. Li, “L2, 1-norm regularized matrix completion for attack detection in collaborative filtering recommender systems,” Chinese Journal of Electronics, vol. 28, no. 5, pp. 906–915, 2019. doi: 10.1049/cje.2019.06.010
[14]	I. Goodfellow, Y. Bengio, and A. Courville, Deep Learning. MIT Press, Cambridge, MA, USA, pp.230-237, 2016.
[15]	R. A. Horn and C. R. Johnson, Matrix Analysis. Cambridge University Press, New York, NY, USA, pp.340-371, 2012.
[16]	W. Jiang, T. T. Ma, X. T. Feng, et al., “Robust semi-nonnegative matrix factorization with adaptive graph regularization for gene representation,” Chinese Journal of Electronics, vol. 29, no. 1, pp. 122–131, 2020. doi: 10.1049/cje.2019.11.001
[17]	G. Zames, “On the input-output stability of time-varying nonlinear feedback systems part one: Conditions derived using concepts of loop gain, conicity, and positivity,” IEEE Transactions on Automatic Control, vol. 11, no. 2, pp. 228–238, 1966. doi: 10.1109/TAC.1966.1098316
[18]	A. Rahnama, A. T. Nguyen, and E. Raff, “Robust design of deep neural networks against adversarial attacks based on Lyapunov theory,” in Proceedings of 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA, pp.8175–8184, 2020.
[19]	H. K. Khalil, Nonlinear Systems, 3rd ed., Prentice Hall, Englewood Cliffs, NJ, USA, pp.1091-1093, 2002.
[20]	Y. X. Li, W. Jin, H. Xu, et al., “DeepRobust: A platform for adversarial attacks and defenses,” in Proceedings of the 35th AAAI Conference on Artificial Intelligence, Virtual Event, pp.16078–16080, 2021.
[21]	Z. Yang, W. Cohen and R. Salakhudinov, “Revisiting semi-supervised learning with graph embeddings,” in Proceedings of the 33rd International Conference on Machine Learning, New York, NY, USA, pp.40–48, 2016.
[22]	J. McAuley, C. Targett, Q. F. Shi, et al., “Image-based recommendations on styles and substitutes,” in Proceedings of the 38th International ACM SIGIR Conference on Research and Development in Information Retrieval, Santiago, Chile, pp.43–52, 2015.
[23]	L. A. Adamic and N. Glance, “The political blogosphere and the 2004 U. S. election: Divided they blog,” in Proceedings of the 3rd International Workshop on Link Discovery, Chicago, IL, USA, pp.36–43, 2005.
[24]	H. Yang, K. L. Ma and J. Cheng, “Rethinking graph regularization for graph neural networks,” in Proceedings of the 35th AAAI Conference on Artificial Intelligence, Virtual Event, pp.4573–4581, 2021.
[25]	D. Zügner and S. Günnemann, “Adversarial attacks on graph neural networks via meta learning,” in Proceedings of the 7th International Conference on Learning Representations, New Orleans, LA, USA, pp.1-15, 2019.
[26]	D. Zügner, A. Akbarnejad, and S. Günnemann, “Adversarial attacks on neural networks for graph data,” in Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, London, UK, pp.2847–2856, 2018.