FlowGANAnomaly: Flow-Based Anomaly Network Intrusion Detection with Adversarial Learning

LI Zeyi; WANG Pan; WANG Zixuan

doi:10.23919/cje.2022.00.173

Volume 33 Issue 1

Jan. 2024

Turn off MathJax

Article Contents

Article Navigation > Chinese Journal of Electronics > 2024 > 33(1): 58-71

Zeyi LI, Pan WANG, Zixuan WANG, “FlowGANAnomaly: Flow-Based Anomaly Network Intrusion Detection with Adversarial Learning,” Chinese Journal of Electronics, vol. 33, no. 1, pp. 58–71, 2024 doi: 10.23919/cje.2022.00.173

Citation:

Zeyi LI, Pan WANG, Zixuan WANG, “FlowGANAnomaly: Flow-Based Anomaly Network Intrusion Detection with Adversarial Learning,” Chinese Journal of Electronics, vol. 33, no. 1, pp. 58–71, 2024 doi: 10.23919/cje.2022.00.173

Citation:

PDF( 4008 KB)

FlowGANAnomaly: Flow-Based Anomaly Network Intrusion Detection with Adversarial Learning

doi: 10.23919/cje.2022.00.173

LI Zeyi^1
,,
WANG Pan^{2
,
,},
WANG Zixuan^3
,

1.
School of Computer Science, Nanjing University of Posts and Telecommunications, Nanjing 210003, China
2.
School of Modern Posts, Nanjing University of Posts and Telecommunications, Nanjing 210003, China
3.
School of Internet of Things, Nanjing University of Posts and Telecommunications, Nanjing 210003, China

More Information

Author Bio:
Zeyi LI was born in Soochow, China, in 1997. He received the B.S. degree in mathematics in 2019 and M.S. degree in computer science in 2022. He is currently pursuing the Ph.D. degree in cyberspace security at Nanjing University of Posts and Telecommunications, China. His research interests include network security, anomaly detection, and deep packet inspection. (Email: 2022040506@njupt.edu.cn)

Pan WANG received the B.S./M.S./Ph.D. degrees in electrical and computer engineering from Nanjing University of Posts and Telecommunications, Nanjing, China, in 2001, 2004, and 2013, respectively. From 2017 to 2018, he has been a Visiting Scholar at University of Dayton (UD) in the Department of Electrical and Computer Engineering, OH, USA. He is currently a Full Professor at Nanjing University of Posts and Telecommunications. His research interests include cyber security and communication network security in B5G/6G/IIoT/smart grid/metaverse, ML/AI-enabled big data analytics, and applications. (Email: wangpan@njupt.edu.cn)

Zixuan WANG was born in Nanjing, China, in 1994. He obtained the M.S. degree in logistics engineering at Nanjing University of Posts and Telecommunications in 2020. He is currently pursuing the Ph.D. degree at Nanjing University of Posts and Telecommunications. His research interests include encrypted traffic identification and data balancing. (Email: 2020070135@njupt.edu.cn)
Corresponding author: Email: wangpan@njupt.edu.cn
Received Date: 2022-06-16
Accepted Date: 2022-11-23

Available Online: 2023-01-07

Publish Date: 2024-01-05

Abstract

Abstract

In recent years, low recall rates and high dependencies on data labelling have become the biggest obstacle to developing deep anomaly detection (DAD) techniques. Inspired by the success of generative adversarial networks (GANs) in detecting anomalies in computer vision and imaging, we propose an anomaly detection model called FlowGANAnomaly for detecting anomalous traffic in network intrusion detection systems (NIDS). Unlike traditional GAN-based approaches, which are composed of a flow encoder, a convolutional encoder-decoder-encoder, a flow decoder and a convolutional encoder, the architecture of this model consists of a generator (G) and a discriminator (D). FlowGANAnomaly maps the different types of traffic feature data from separate datasets to a uniform feature space, thus can capture the normality of network traffic data more accurately in an adversarial manner to mitigate the problem of the high dependence on data labeling. Moreover, instead of simply detecting the anomalies by the output of D, we proposed a new anomaly scoring method that integrates the deviation between the output of two Gs’ convolutional encoders with the output of D as weighted scores to improve the low recall rate of anomaly detection. We conducted several experiments comparing existing machine learning algorithms and existing deep learning methods (AutoEncoder and VAE) on four public datasets (NSL-KDD, CIC-IDS2017, CIC-DDoS2019, and UNSW-NB15). The evaluation results show that FlowGANAnomaly can significantly improve the performance of anomaly-based NIDS.
- Anomaly detection,
- Unsupervised learning,
- Generative adversarial network,
- Intrusion detection system

FullText(HTML)

References(41)

References

[1]	O. Depren, M. Topallar, E. Anarim, et al., “An intelligent intrusion detection system (IDS) for anomaly and misuse detection in computer networks,” Expert systems with Applications, vol. 29, no. 4, pp. 713–722, 2005. doi: 10.1016/j.eswa.2005.05.002
[2]	S. Ramaswamy, R. Rastogi, and K. Shim, “Efficient algorithms for mining outliers from large data sets,” in Proceedings of 2000 ACM SIGMOD International Conference on Management of Data, Dallas, TX, USA, pp. 427–438, 2000.
[3]	G. S. Pang, C. H. Shen, L. B. Cao, et al., “Deep learning for anomaly detection: a review,” ACM Computing Surveys, vol. 54, no. 2, article no. 38, 2022. doi: 10.1145/3439950
[4]	V. Chandola, A. Banerjee, and V. Kumar, “Anomaly detection: A survey,” ACM Computing Surveys, vol. 41, no. 3, article no. 15, 2009. doi: 10.1145/1541880.1541882
[5]	M. Ahmed, A. N. Mahmood, and J. K. Hu, “A survey of network anomaly detection techniques,” Journal of Network and Computer Applications, vol. 60, pp. 19–31, 2016. doi: 10.1016/j.jnca.2015.11.016
[6]	A. Creswell, T. White, V. Dumoulin, et al., “Generative adversarial networks: an overview,” IEEE Signal Processing Magazine, vol. 35, no. 1, pp. 53–65, 2018. doi: 10.1109/MSP.2017.2765202
[7]	Z. P. Qiang, L. B. He, F. Dai, et al., “Image inpainting based on improved deep convolutional auto‐encoder network,” Chinese Journal of Electronics, vol. 29, no. 6, pp. 1074–1084, 2020. doi: 10.1049/cje.2020.09.008
[8]	C. Qin and X. G. Gao, “Spatio-temporal generative adversarial networks,” Chinese Journal of Electronics, vol. 29, no. 4, pp. 623–631, 2020. doi: 10.1049/cje.2020.04.001
[9]	F. Falcão, T. Zoppi, C. B. V. Silva, et al., “Quantitative comparison of unsupervised anomaly detection algorithms for intrusion detection,” in Proceedings of the 34th ACM/SIGAPP Symposium on Applied Computing, Limassol, Cyprus, pp. 318–327, 2019.
[10]	E. Schubert, A. Koos, T. Emrich, et al., “A framework for clustering uncertain data,” Proceedings of the VLDB Endowment, vol. 8, no. 12, pp. 1976–1979, 2015. doi: 10.14778/2824032.2824115
[11]	P. Cunningham and S. J. Delany, “k-nearest neighbour classifiers-a tutorial,” ACM Computing Surveys, vol. 54, no. 6, article no. 128, 2022. doi: 10.1145/3459665
[12]	M. M. Breunig, H. P. Kriegel, R. T. Ng, et al., “LOF: Identifying density-based local outliers,” in Proceedings of 2000 ACM SIGMOD International Conference on Management of Data, Dallas, TX, USA, pp. 93–104, 2000.
[13]	J. Camacho, A. Pérez-Villegas, P. García-Teodoro, et al., “PCA-based multivariate statistical network monitoring for anomaly detection,” Computers & Security, vol. 59, pp. 118–137, 2016. doi: 10.1016/j.cose.2016.02.008
[14]	R. Kwitt and U. Hofmann, “Unsupervised anomaly detection in network traffic by means of robust PCA,” in Proceedings of 2007 International Multi-Conference on Computing in the Global Information Technology, Guadeloupe, French Caribbean, pp. 37–37, 2007.
[15]	H. P. Kriegel, M. Schubert, and A. Zimek, “Angle-based outlier detection in high-dimensional data,” in Proceedings of the 14th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, Las Vegas, NV, USA, pp. 444–452, 2008.
[16]	M. Amer, M. Goldstein, and S. Abdennadher, “Enhancing one-class support vector machines for unsupervised anomaly detection,” in Proceedings of the ACM SIGKDD Workshop on Outlier Detection and Description, Chicago, IL, USA, pp. 8–15, 2013.
[17]	F. T. Liu, K. M. Ting, and Z. H. Zhou, “Isolation forest,” in Proceedings of the 8th IEEE International Conference on Data Mining, Pisa, Italy, pp. 413–422, 2008.
[18]	F. T. Liu, K. M. Ting, and Z. H. Zhou, “Isolation-based anomaly detection,” ACM Transactions on Knowledge Discovery from Data, vol. 6, no. 1, article no. 3, 2012. doi: 10.1145/2133360.2133363
[19]	X. S. Wei, H. J. Ye, X. Mu, et al., “Multi-instance learning with emerging novel class,” IEEE Transactions on Knowledge and Data Engineering, vol. 33, no. 5, pp. 2109–2120, 2021. doi: 10.1109/TKDE.2019.2952588
[20]	L. L. Wang, B. Q. Wang, P. P. Zhao, et al., “Malware detection algorithm based on the attention mechanism and ResNet,” Chinese Journal of Electronics, vol. 29, no. 6, pp. 1054–1060, 2020. doi: 10.1049/cje.2020.09.006
[21]	D. W. Zhou, H. J. Ye, and D. C. Zhan, “Learning placeholders for open-set recognition,” in Proceedings of 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Nashville, TN, USA, pp. 4401–4410, 2021.
[22]	L. Ruff, J. R. Kauffmann, R. A. Vandermeulen, et al., “A unifying review of deep and shallow anomaly detection,” Proceedings of the IEEE, vol. 109, no. 5, pp. 756–795, 2021. doi: 10.1109/JPROC.2021.3052449
[23]	D. W. Zhou, Y. Yang, and D. C. Zhan, “Learning to classify with incremental new class,” IEEE Transactions on Neural Networks and Learning Systems, vol. 33, no. 6, pp. 2429–2443, 2022. doi: 10.1109/TNNLS.2021.3104882
[24]	A. Haque, L. Khan, and M. Baron, “SAND: Semi-supervised adaptive novel class detection and classification over data stream,” in Proceedings of the Thirtieth AAAI Conference on Artificial Intelligence, Phoenix, AZ, USA, pp. 1652–1658, 2016.
[25]	M. Gharib, B. Mohammadi, S. H. Dastgerdi, et al., “AutoIDS: Auto-encoder based method for intrusion detection system,” arXiv preprint, arXiv: 1911.03306, 2019.
[26]	Y. Mirsky, T. Doitshman, et al., “Kitsune: An ensemble of autoencoders for online network intrusion detection,” in Proceedings of the 25th Annual Network and Distributed System Security Symposium, San Diego, CA, USA, 2018.
[27]	S. Zavrak and M. Iskefiyeli, “Anomaly-based intrusion detection from network flow features using variational autoencoder,” IEEE Access, vol. 8, pp. 108346–108358, 2020. doi: 10.1109/ACCESS.2020.3001350
[28]	B. Abolhasanzadeh, “Nonlinear dimensionality reduction for intrusion detection using auto-encoder bottleneck features,” in Proceedings of the 7th Conference on Information and Knowledge Technology, Urmia, Iran, pp. 1–5, 2015.
[29]	N. Shone, T. N. Ngoc, V. D. Phai, et al., “A deep learning approach to network intrusion detection,” IEEE Transactions on Emerging Topics in Computational Intelligence, vol. 2, no. 1, pp. 41–50, 2018. doi: 10.1109/TETCI.2017.2772792
[30]	S. Longari, D. H. N. Valcarcel, M. Zago, et al., “CANnolo: An anomaly detection system based on LSTM autoencoders for controller area network,” IEEE Transactions on Network and Service Management, vol. 18, no. 2, pp. 1913–1924, 2021. doi: 10.1109/TNSM.2020.3038991
[31]	T. Schlegl, P. Seeböck, S. M. Waldstein, et al., “Unsupervised anomaly detection with generative adversarial networks to guide marker discovery,” in Proceedings of the 25th International Conference on Information Processing in Medical Imaging, Boone, NC, USA, pp. 146–157, 2017.
[32]	H. Zenati, C. S. Foo, B. Lecouat, et al., “Efficient GAN-based anomaly detection,” arXiv preprint, arXiv: 1802.06222, 2018.
[33]	T. Schlegl, P. Seeböck, S. M. Waldstein, et al., “f-AnoGAN: Fast unsupervised anomaly detection with generative adversarial networks,” Medical Image Analysis, vol. 54, pp. 30–44, 2019. doi: 10.1016/j.media.2019.01.010
[34]	S. Akcay, A. Atapour-Abarghouei, and T. P. Breckon, “GANomaly: Semi-supervised anomaly detection via adversarial training,” in Proceedings of the 14th Asian Conference on Computer, Perth, Australia, pp. 622–637, 2019.
[35]	I. Siniosoglou, P. Radoglou-Grammatikis, G. Efstathopoulos, et al., “A unified deep learning anomaly detection and classification approach for smart grid environments,” IEEE Transactions on Network and Service Management, vol. 18, no. 2, pp. 1137–1151, 2021. doi: 10.1109/TNSM.2021.3078381
[36]	A. Radford, L. Metz, and S. Chintala, “Unsupervised representation learning with deep convolutional generative adversarial networks,” arXiv preprint, arXiv: 1511.06434, 2015.
[37]	L. Dhanabal and S. P. Shantharajah, “A study on NSL-KDD dataset for intrusion detection system based on classification algorithms,” International Journal of Advanced Research in Computer and Communication Engineering, vol. 4, no. 6, pp. 446–452, 2015.
[38]	R. Panigrahi and S. Borah, “A detailed analysis of CICIDS2017 dataset for designing intrusion detection systems,” International Journal of Engineering & Technology, vol. 7, no. 3, pp. 479–482, 2018.
[39]	N. Moustafa and J. Slay, “UNSW-NB15: A comprehensive data set for network intrusion detection systems (UNSW-NB15 network data set),” in Proceedings of 2015 Military Communications and Information Systems Conference, Canberra, Australia, pp. 1–6, 2015.
[40]	I. Sharafaldin, A. H. Lashkari, S. Hakak, et al., “Developing realistic distributed denial of service (DDoS) attack dataset and taxonomy,” in Proceedings of 2019 International Carnahan Conference on Security Technology, Chennai, India, pp. 1–8, 2019.
[41]	H. Z. Xu, Y. J. Wang, S. L. Jian, et al., “Beyond outlier detection: Outlier interpretation by attention-guided triplet deviation network,” in Proceedings of the Web Conference 2021, Ljubljana, Slovenia, pp. 1328–1339, 2021.