A Deep Deterministic Policy Gradient-based Method for Enforcing Service Fault-tolerance in MEC

LONG Tingyan; CHEN Peng; XIA Yunni; MA Yong; SUN Xiaoning; ZHAO Jiale; LYU Yifei

doi:10.23919/cje.2023.00.105

Article Contents

Article Navigation > Chinese Journal of Electronics > 2024 > Uncorrected proof

Tingyan LONG, Peng CHEN, Yunni XIA, et al., “A Deep Deterministic Policy Gradient-based Method for Enforcing Service Fault-tolerance in MEC,” Chinese Journal of Electronics, vol. 33, no. 5, pp. 1–11, 2024 doi: 10.23919/cje.2023.00.105

Citation:

Tingyan LONG, Peng CHEN, Yunni XIA, et al., “A Deep Deterministic Policy Gradient-based Method for Enforcing Service Fault-tolerance in MEC,” Chinese Journal of Electronics, vol. 33, no. 5, pp. 1–11, 2024 doi: 10.23919/cje.2023.00.105

Citation:

PDF( 12659 KB)

A Deep Deterministic Policy Gradient-based Method for Enforcing Service Fault-tolerance in MEC

doi: 10.23919/cje.2023.00.105

LONG Tingyan^1
,,
CHEN Peng^{2
,
,},
XIA Yunni^{1
,
,},
MA Yong^3
,,
SUN Xiaoning^4
,,
ZHAO Jiale^1
,,
LYU Yifei^1
,

1.
College of Computer Science, Chongqing University, Chongqing 400044, China
2.
School of Computer and Software Engineering, Xihua University, Chengdu 610039, China
3.
School of Computer and Information Engineering, Jiangxi Normal University, Nanchang 330022, China
4.
School of Computer and Information Science, Chongqing Normal university, Chongqing 401331, China

More Information

Author Bio:
Tingyan LONG received the M.S. degree in School of Computer Science & Technology from Guizhou University, in 2019, and the Ph.D. degree in College of Computer Science from Chongqing University in 2023. Currently, she works at the School of Computer Science and Technology, Guizhou University, Guiyang, China. Her research interests include fault-tolerant, edge computing and reinforcement learning method. (Email: l5469369@hotmail.com)

Peng CHEN IEEE member, received the B.E. degree in Computer Science and Technology from University of Electronic Science and Technology of China, Chengdu, China in 2001, the M.S. degree in Computer Software and Theory from Peking University, Beijing, China in 2004 and Ph.D. degree in Computer Science and Technology from Sichuan University, Chengdu, China in 2017. He is currently a professor of School of Computer and Software Engineering, Xihua University, Chengdu, China. His research interests include machine learning, service computing and time series analysis. (Email: chenpeng@mail.xhu.edu.cn)

Yunni XIA IEEE member, received the B.S. degree in Computer Science from Chongqing University in 2003, and the Ph.D. degrees in Computer Science from Peking University (PKU) in 2008. He is currently a professor in the College of Computer Science at Chongqing University, China. He is the author or co-author of more than 100 research publications. His research interests are in Petri nets, software quality, performance evaluation, edge computing and cloud computing system dependability. (Email: xiayunni@hotmail.com)

Yong MA received the M.S. degree in Computer Science from Xidian University, in 2003, and the Ph.D. degree in Computer Science from Wuhan University in 2006. In 2018, he worked on the integrated control and dispatching of energy in microgrid with Malardalens University, Sweden. He is now a professor in the School of Computer Information Engineering, Jiangxi Normal University. His current research focuses on cloud computing, edge computing, and data science. (Email: mywuda@126.com)

Xiaoning SUN received the B.S. degree in Computer Science from Chongqing University, China, in 2015, and the Ph.D. degree in Software Engineering from Chongqing University, China, in 2022. Since July 2022, she has been a lecturer with the School of Computer and Information Science, Chongqing Normal University. Her research interests are in service computing, performance evaluation, and edge computing. (Email: sxiaoning@hotmail.com)

Jiale ZHAO received the B.E. degree in Information Security from Huaibei Normal University, Huaibei, China in 2017, the M.S. degree in Computer Technology from Jiangxi Normal University, Nanchang, China in 2021. He is currently studying for a Ph.D. in Computer Science and Technology at Chongqing University, Chongqing, China. His research interests include fault-tolerance, edge computing and cloud computing. (Email: zhaojiale0415@163.com)

Yifei LYU is currently pursuing the master's degree with the College of Computer Science, Chongqing University, China. His main research interests are in the areas of cloud computing, edge computing and reinforcement learning method. (Email: yuzhe1334021@163.com)
Corresponding author: Email: chenpeng@mail.xhu.edu.cn; Email: xiayunni@hotmail.com
Received Date: 2022-03-22
Accepted Date: 2022-03-22

Available Online: 2024-02-02

Abstract

Abstract

Mobile edge computing (MEC) provides edge services to users in a distributed and on-demand way. Due to the heterogeneity of edge applications, it is a key challenge for service providers to deploy latency and resource-intensive applications on resource-constrained devices. This is especially true when underlying edge infrastructures are fault and error-prone. In this paper, we propose a fault tolerance approach named DFGP, for enforcing mobile service fault-tolerance in MEC. It synthesizes a generative optimization network (GON) model for predicting resource failure and a deep deterministic policy gradient (DDPG) model for yielding preemptive migration decisions. We show through extensive simulation experiments that DFGP is more effective in fault detection and guaranteeing quality of service (QoS), in terms of fault detection accuracy, migration efficiency, task migration time, task scheduling time, and energy consumption than other existing method.
- Mobile edge computing,
- service,
- generative optimization network,
- fault tolerance,
- deep deterministic policy gradient

FullText(HTML)

References(40)

References

[1]	Y. Chen, J. Zhao, X. K. Zhou, et al, “A distributed game theoretical approach for credibility-guaranteed multimedia data offloading in MEC,” Information Sciences, vol. 644, article no. 119306, 2023. doi: 10.1016/j.ins.2023.119306
[2]	Y. Chen, J. Zhao, J. T. Hu, et al, “Distributed task offloading and resource purchasing in NOMA-enabled mobile edge computing: Hierarchical game theoretical approaches,” ACM Transactions on Embedded Computing Systems, vol. 23, no. 1, article no. 2, 2024. doi: 10.1145/3597023
[3]	X. F. Cao, G. M. Tang, D. K. Guo, et al, “Edge federation: Towards an integrated service provisioning model,” IEEE/ACM Transactions on Networking, vol. 28, no. 3, pp. 1116–1129, 2020. doi: 10.1109/TNET.2020.2979361
[4]	C. Engelmann, G. R. Vallee, T. Naughton, et al., “Proactive fault tolerance using preemptive migration, ” in Proceedings of the 2009 17th Euromicro International Conference on Parallel, Distributed and Network-based Processing, Weimar, Germany, pp. 252–257, 2009.
[5]	F. Z. Liu, J. W. Huang, and X. B. Wang, “Joint task offloading and resource allocation for device-edge-cloud collaboration with subtask dependencies,” IEEE Transactions on Cloud Computing, vol. 11, no. 3, pp. 3027–3039, 2023. doi: 10.1109/TCC.2023.3251561
[6]	J. W. Huang, J. Y. Wan, B. F. Lv, et al, “Joint computation offloading and resource allocation for edge-cloud collaboration in internet of vehicles via deep reinforcement learning,” IEEE Systems Journal, vol. 17, no. 2, pp. 2500–2511, 2023. doi: 10.1109/JSYST.2023.3249217
[7]	Y. Chen, J. T. Hu, J. Zhao, et al., “QoS-aware computation offloading in LEO satellite edge computing for IoT: A game-theoretical approach, ” Chinese Journal of Electronics, 2023. doi: 10.23919/cje.2022.00.412
[8]	J. L. Liu, S. G. Wang, A. Zhou, et al, “Using proactive fault-tolerance approach to enhance cloud service reliability,” IEEE Transactions on Cloud Computing, vol. 6, no. 4, pp. 1191–1202, 2018. doi: 10.1109/TCC.2016.2567392
[9]	A. Rawat, R. Sushil, A. Agarwal, et al, “A new adaptive fault tolerant framework in the cloud,” IETE Journal of Research, vol. 69, no. 5, pp. 2897–2909, 2023. doi: 10.1080/03772063.2021.1907231
[10]	B. K. Ray, A. Saha, S. Khatua, et al, “Proactive fault-tolerance technique to enhance reliability of cloud service in cloud federation environment,” IEEE Transactions on Cloud Computing, vol. 10, no. 2, pp. 957–971, 2022. doi: 10.1109/TCC.2020.2968522
[11]	P. Y. Zhang, S. Shu, and M. C. Zhou, “An online fault detection model and strategies based on SVM-grid in clouds,” IEEE/CAA Journal of Automatica Sinica, vol. 5, no. 2, pp. 445–456, 2018. doi: 10.1109/JAS.2017.7510817
[12]	X. F. Hu, Y. B. Li, L. Jia, et al, “A novel two-stage unsupervised fault recognition framework combining feature extraction and fuzzy clustering for collaborative AIoT,” IEEE Transactions on Industrial Informatics, vol. 18, no. 2, pp. 1291–1300, 2022. doi: 10.1109/TII.2021.3076077
[13]	Z. L. He, P. F. Chen, X. Y. Li, et al, “A spatiotemporal deep learning approach for unsupervised anomaly detection in cloud systems,” IEEE Transactions on Neural Networks and Learning Systems, vol. 34, no. 4, pp. 1705–1719, 2023. doi: 10.1109/TNNLS.2020.3027736
[14]	S. Tuli, G. Casale, and N. R. Jennings, “PreGAN: Preemptive migration prediction network for proactive fault-tolerant edge computing, ” in Proceedings of 2022 IEEE Conference on Computer Communications, London, United Kingdom, pp. 670–679, 2022.
[15]	P. Dasgupta, R. C. Chen, S. Menon, et al, “Design and implementation of the clouds distributed operating system,” Computing Systems, vol. 3, no. 1, pp. 11–46, 1990.
[16]	H. Gupta, A. Vahid Dastjerdi, S. K. Ghosh, et al, “IFogSim: A toolkit for modeling and simulation of resource management techniques in the internet of things, edge and fog computing environments,” Software:Practice and Experience, vol. 47, no. 9, pp. 1275–1296, 2017. doi: 10.1002/spe.2509
[17]	A. Al-Shuwaili, O. Simeone, A. Bagheri, et al, “Joint uplink/downlink optimization for backhaul-limited mobile cloud computing with user scheduling,” IEEE Transactions on Signal and Information Processing over Networks, vol. 3, no. 4, pp. 787–802, 2017. doi: 10.1109/TSIPN.2017.2668142
[18]	X. Chen, L. Jiao, W. Z. Li, et al, “Efficient multi-user computation offloading for mobile-edge cloud computing,” IEEE/ACM Transactions on Networking, vol. 24, no. 5, pp. 2795–2808, 2016. doi: 10.1109/TNET.2015.2487344
[19]	D. Satria, D. Park, and M. Jo, “Recovery for overloaded mobile edge computing,” Future Generation Computer Systems, vol. 70 pp. 138–147, 2017. doi: 10.1016/j.future.2016.06.024
[20]	K. J. Ye, Y. Y. Liu, G. Y. Xu, et al., “Fault injection and detection for artificial intelligence applications in container-based clouds, ” in Proceedings of the 11th International Con ference on Cloud Computing, Seattle, WA, USA, pp. 112–127, 2018.
[21]	A. Beloglazov and R. Buyya, “Optimal online deterministic algorithms and adaptive heuristics for energy and performance efficient dynamic consolidation of virtual machines in cloud data centers,” Concurrency and Computation:Practice and Experience, vol. 24, no. 13, pp. 1397–1420, 2012. doi: 10.1002/cpe.1867
[22]	S. Tuli, S. Tuli, G. Casale, et al., “Generative optimization networks for memory efficient data generation, ” arXiv preprint, arXiv: 2110.02912, 2021.
[23]	G. L. Liu, K. J. Shih, T. C. Wang, et al. , “Partial convolution based padding, ” arXiv preprint, arXiv: 1811.11718, 2018.
[24]	P. Chen, H. Y. Liu, R. Y. Xin, et al, “Effectively detecting operational anomalies in large-scale IoT data infrastructures by using a GAN-based predictive model,” The Computer Journal, vol. 65, no. 11, pp. 2909–2925, 2022. doi: 10.1093/comjnl/bxac085
[25]	Z. W. Yao, A. Gholami, S. Shen, et al. , “ADAHESSIAN: An adaptive second order optimizer for machine learning, ” in Proceedings of the 35th AAAI Conference on Artificial In telligence, virtual event, pp. 10665–10673, 2021.
[26]	S. Zhang, H. H. Tong, J. J. Xu, et al, “Graph convolutional networks: A comprehensive review,” Computational Social Networks, vol. 6, no. 1, article no. 11, 2019. doi: 10.1186/s40649-019-0069-y
[27]	D. Silver, G. Lever, N. Heess, et al. , “Deterministic policy gradient algorithms, ” in Proceedings of the 31st Internatio nal Conference on Machine Learning, Beijing, China, pp. I-387–I-395, 2014.
[28]	A. Siffer, P. A. Fouque, A. Termier, et al., “Anomaly detection in streams with extreme value theory, ” in Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, Halifax, NS, Canada, pp. 1067–1075, 2017.
[29]	S. G. Wang, Y. L. Zhao, J. L. Xu, et al, “Edge server placement in mobile edge computing,” Journal of Parallel and Distributed Computing, vol. 127 pp. 160–168, 2019. doi: 10.1016/j.jpdc.2018.06.008
[30]	S. Y. Liu, Y. H. Liu, L. M. Ni, et al., “Towards mobility-based clustering, ” in Proceedings of the 16th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, Washington, DC, USA, pp. 919–928, 2010.
[31]	S. Q. Shen, V. Van Beek, and A. Iosup, “Statistical characterization of business-critical workloads hosted in cloud datacenters, ” in Proceedings of the 2015 15th IEEE/ACM Inter national Symposium on Cluster, Cloud and Grid Computing, Shenzhen, China, pp. 465–474, 2015.
[32]	A. Javed, J. Robert, K. Heljanko, et al, “IoTEF: A federated edge-cloud architecture for fault-tolerant IoT applications,” Journal of Grid Computing, vol. 18, no. 1, pp. 57–80, 2020. doi: 10.1007/s10723-019-09498-8
[33]	Y. Feng, Z. J. Liu, J. L. Chen, et al, “Make the rocket intelligent at IoT edge: Stepwise GAN for anomaly detection of LRE with multisource fusion,” IEEE Internet of Things Journal, vol. 9, no. 4, pp. 3135–3149, 2022. doi: 10.1109/JIOT.2021.3097437
[34]	S. Tuli, G. Casale, and N. R. Jennings, “DRAGON: Decentralized fault tolerance in edge federations,” IEEE Transactions on Network and Service Management, vol. 20, no. 1, pp. 276–291, 2023. doi: 10.1109/TNSM.2022.3199886
[35]	D. Li, D. C. Chen, B. H. Jin, et al. , “MAD-GAN: Multivariate anomaly detection for time series data with generative adversarial networks, ” in Proceedings of the 28th Interna tional Conference on Artificial Neural Networks, Munich, Germany, pp. 703–716, 2019.
[36]	L. Hou, Z. H. Yuan, L. Huang, et al., “Slimmable generative adversarial networks,” in Proceedings of the 35th AAAI Conference on Artificial Intelligence, virtual event, pp. 7746–7753, 2021.
[37]	J. Audibert, P. Michiardi, F. Guyard, et al., “USAD: UnSupervised anomaly detection on multivariate time series, ” in Proceedings of the 26th ACM SIGKDD International Con ference on Knowledge Discovery & Data Mining, New York, NY, USA, pp. 3395–3404, 2020.
[38]	Y. X. Zhang, Y. Q. Chen, J. D. Wang, et al, “Unsupervised deep anomaly detection for multi-sensor time-series signals,” IEEE Transactions on Knowledge and Data Engineering, vol. 35, no. 2, pp. 2118–2132, 2023. doi: 10.1109/TKDE.2021.3102110
[39]	Z. W. Ji, J. H. Gong, and J. R. Feng, “A novel deep learning approach for anomaly detection of time series data,” Scientific Programming, vol. 2021, article no. 6636270, 2021. doi: 10.1155/2021/6636270
[40]	Y. Chen, J. Zhao, Y. Wu, et al, “QoE-aware decentralized task offloading and resource allocation for end-edge-cloud systems: A game-theoretical approach,” IEEE Transactions on Mobile Computing, vol. 23, no. 1, pp. 769–784, 2024. doi: 10.1109/TMC.2022.3223119