Multi-Objective Coordinated Optimization for UAV Charging Scheduling in Intelligent Aerial-Ground Perception Networks

ZHOU Yi; CHENG Xiang; SHI Huaguang; JIN Zhanqi; NING Nianwen; LIU Fuqiang

doi:10.23919/cje.2022.00.334

Volume 32 Issue 6

Nov. 2023

Turn off MathJax

Article Contents

Article Navigation > Chinese Journal of Electronics > 2023 > 32(6): 1203-1217

ZHOU Yi, CHENG Xiang, SHI Huaguang, et al., “Multi-Objective Coordinated Optimization for UAV Charging Scheduling in Intelligent Aerial-Ground Perception Networks,” Chinese Journal of Electronics, vol. 32, no. 6, pp. 1203-1217, 2023, doi: 10.23919/cje.2022.00.334

Citation:

ZHOU Yi, CHENG Xiang, SHI Huaguang, et al., “Multi-Objective Coordinated Optimization for UAV Charging Scheduling in Intelligent Aerial-Ground Perception Networks,” Chinese Journal of Electronics, vol. 32, no. 6, pp. 1203-1217, 2023, doi: 10.23919/cje.2022.00.334

Citation:

PDF( 4673 KB)

Multi-Objective Coordinated Optimization for UAV Charging Scheduling in Intelligent Aerial-Ground Perception Networks

doi: 10.23919/cje.2022.00.334

ZHOU Yi^{1, 3
,},
CHENG Xiang^{1, 3
,},
SHI Huaguang^{1, 2
,},
JIN Zhanqi^{1, 3
,},
NING Nianwen^{1, 3
,},
LIU Fuqiang^4
,

1.
School of Artificial Intelligence, Henan University, Zhengzhou 450046, China
2.
Henan Engineering Research Center for Industrial Internet of Things, Zhengzhou 450046, China
3.
International Joint Research Laboratory for Cooperative Vehicular Networking, Zhengzhou 450046, China
4.
College of Electronic and Information Engineering, Tongji University, Shanghai 201804, China

Funds: This work was supported by the National Natural Science Foundation of China (62176088), the Program for Science & Technology Development of Henan Province (212102210412, 222102210067, 222102210022), and the Young Elite Scientist Sponsorship Program by Henan Association for Science and Technology (2022HYTP013)

More Information

Author Bio:
Yi ZHOU received the B.S. degree in electronic engineering from the First Aeronautic Institute of Air Force, China, in 2002, and the Ph.D. degree in control system and theory from Tongji University, China, in 2011. He is currently a Full Professor and Deputy Dean with the School of Artificial Intelligence, Henan University, China. He is also the Director of International Joint Research Laboratory for Cooperative Vehicular Networks, Henan, China. His research interests include vehicular cyber-physical systems and multi-agent collaboration. (Email: zhouyi@henu.edu.cn)

Xiang CHENG is currently a postgraduate of Henan University, Zhengzhou, China. His research interests include multi-agent cooperation and multi-UAV cooperation deployment. (Email: richard@henu.edu.cn)

Huaguang SHI (corresponding author) received the B.S. degree in electronic science and technology from Zhengzhou University, Zhengzhou, China, in 2014, and the Ph.D. degree in measurement technique and automation equipment from the University of Chinese Academy of Sciences, Beijing, China, in 2021. He is currently a Lecturer with the School of Artificial Intelligence, Henan University, Zhengzhou, China. His current research interests include industrial Internet of things, wireless networks, and multi-agent learning. (Email: shihuaguang@henu.edu.cn)

Zhanqi JIN is currently a Postgraduate of Henan University, China. His current research interests include UAV-assisted communications and Intelligent reflective surface. (Email: Jinzhanqi@henu.edu.cn)

Nianwen NING was born in 1991, Ph.D. He is currently a lecturer with the School of Artificial Intelligence, Henan University, Zhengzhou, China. His main research interests include intelligent traffic and graph neural network. (Email: nnw@henu.edu.cn)

Fuqiang LIU was born in 1965, Ph.D. candidate, professor. He is winner of National Natural Science Foundation (key project), and his main research direction include Internet of Vehicles and Intelligent Transportation. (Email: liufuqiang@tongji.edu.cn)
Received Date: 2022-11-16
Accepted Date: 2023-02-14

Available Online: 2023-04-23

Publish Date: 2023-11-05

Abstract

Abstract

The unmanned aerial vehicles (UAVs)-assisted intelligent traffic perception system can provide effective situation awareness. However, UAVs are required to be recharged before the energy is exhausted, which may cause task interruption. To address this concern, the charging UAV (CUAV) is employed to provide wireless charging for the mission UAVs (MUAVs). This paper studies the charging scheduling problem of the CUAV under the premise of optimizing the MUAVs deployment. We first model the MUAVs deployment problem considering the energy consumption and data transmission and establish the CUAV charging model. Then, the above problem is formulated as a multi-objective multi-agent stochastic game process to simplify the decisions-making of MUAVs and CUAV, based on which we propose the utility-based Pareto optimal deployment and charging algorithm, which reduces the computing complexity by equivalent utility of the MUAVs while using Kullback-Leibler divergence to constrain solutions. Next, to ensure the effectiveness of policy update, the multi-agent communication protocol is adopted to improve policy exploration efficiency. Simulation results show that the proposed algorithm outperforms existing works in terms of energy efficiency and charging by comparing with the Pareto front of different methods, endurance anxiety of the MUAVs, and charging utilization under different task modes.
- Wireless charging,
- UAVs deployment,
- Multi-objective optimization,
- Multi-objective reinforcement learning,
- Pareto optimal

FullText(HTML)

References(48)

References

[1]	G. X. Liu, H. Shi, K. Abbas, et al., “Smart traffic monitoring system using computer vision and edge computing,” IEEE Transactions on Intelligent Transportation Systems, vol.23, no.8, pp.12027–12038, 2022. doi: 10.1109/TITS.2021.3109481
[2]	A. V. Savkin and H. L. Huang, “Navigation of a UAV network for optimal surveillance of a group of ground targets moving along a road,” IEEE Transactions on Intelligent Transportation Systems, vol.23, no.7, pp.9281–9285, 2022. doi: 10.1109/TITS.2021.3077880
[3]	N. Dilshad, J. Hwang, J. Song, et al., “Applications and challenges in video surveillance via drone: A brief survey,” in Proceedings of 2020 International Conference on Information and Communication Technology Convergence, Jeju, Korea (South), pp.728–732, 2020.
[4]	Z. Liu, C. Zhan, Y. Cui, et al., “Robust edge computing in UAV systems via scalable computing and cooperative computing,” IEEE Wireless Communications, vol.28, no.5, pp.36–42, 2021. doi: 10.1109/MWC.121.2100041
[5]	X. W. Li, H. P. Yao, J. J. Wang, et al., “Rechargeable multi-UAV aided seamless coverage for QoS-guaranteed IoT networks,” IEEE Internet of Things Journal, vol.6, no.6, pp.10902–10914, 2019. doi: 10.1109/JIOT.2019.2943147
[6]	Q. Chen, H. Zhu, L. Yang, et al., “Edge computing assisted autonomous flight for UAV: Synergies between vision and communications,” IEEE Communications Magazine, vol.59, no.1, pp.28–33, 2021. doi: 10.1109/MCOM.001.2000501
[7]	M. Q. Li, L. Liu, Y. Gu, et al., “Minimizing energy consumption in wireless rechargeable UAV networks,” IEEE Internet of Things Journal, vol.9, no.5, pp.3522–3532, 2022. doi: 10.1109/JIOT.2021.3097918
[8]	Z. H. Yang, W. Xu, and M. Shikh-Bahaei, “Energy efficient UAV communication with energy harvesting,” IEEE Transactions on Vehicular Technology, vol.69, no.2, pp.1913–1927, 2020. doi: 10.1109/TVT.2019.2961993
[9]	Y. W. Nie, J. H. Zhao, J. Liu, et al., “Energy-efficient UAV trajectory design for backscatter communication: a deep reinforcement learning approach,” China Communications, vol.17, no.10, pp.129–141, 2020. doi: 10.23919/JCC.2020.10.009
[10]	X. Zhang, X. H. Wang, X. P. Xu, et al., “Demand learning and cooperative deployment of UAV networks,” Chinese Journal of Electronics, vol.31, no.3, pp.408–415, 2022. doi: 10.1049/cje.2021.00.278
[11]	C. W. Wang, Y. L. Cui, D. H. Deng, et al., “Trajectory optimization and power allocation scheme based on DRL in energy efficient UAV-aided communication networks,” Chinese Journal of Electronics, vol.31, no.3, pp.397–407, 2022. doi: 10.1049/cje.2021.00.314
[12]	M. Q. Li, L. Liu, J. Xi, et al., “ECTSA: An efficient charging time scheduling algorithm for wireless rechargeable UAV network, ” in Proceedings of 2021 IFIP Networking Conference, Espoo and Helsinki, Finland, pp.1–9, 2021.
[13]	Y. Jin, Z. J. Qian, S. R. Gong, et al., “Learning transferable driven and drone assisted sustainable and robust regional disease surveillance for smart healthcare,” IEEE/ACM Transactions on Computational Biology and Bioinformatics, vol.18, no.1, pp.114–125, 2021. doi: 10.1109/TCBB.2020.3017041
[14]	J. W. Xu, K. Zhu, and R. Wang, “RF aerially charging scheduling for UAV fleet: A Q-learning approach, ” in Proceedings of the 15th International Conference on Mobile Ad-Hoc and Sensor Networks, Shenzhen, China, pp.194–199, 2019.
[15]	L. L. Liu, A. M. Wang, G. Sun, et al., “Multiobjective optimization for improving throughput and energy efficiency in UAV-Enabled IoT,” IEEE Internet of Things Journal, vol.9, no.20, pp.20763–20777, 2022. doi: 10.1109/JIOT.2022.3175712
[16]	M. Mozaffari, W. Saad, M. Bennis, et al., “A tutorial on UAVs for wireless networks: applications, challenges, and open problems,” IEEE Communications Surveys & Tutorials, vol.21, no.3, pp.2334–2360, 2019. doi: 10.1109/COMST.2019.2902862
[17]	Y. Yu, J. Tang, J. Y. Huang, et al., “Multi-objective optimization for UAV-assisted wireless powered IoT networks based on extended DDPG algorithm,” IEEE Transactions on Communications, vol.69, no.9, pp.6361–6374, 2021. doi: 10.1109/TCOMM.2021.3089476
[18]	J. J. Wang, C. X. Jiang, H. J. Zhang, et al., “Thirty years of machine learning: The road to Pareto-optimal wireless networks,” IEEE Communications Surveys & Tutorials, vol.22, no.3, pp.1472–1514, 2020. doi: 10.1109/COMST.2020.2965856
[19]	V. François-Lavet, P. Henderson, R. Islam, et al., “An introduction to deep reinforcement learning,” Foundations and Trends, no.3-4, pp.219–354, 2018. doi: 10.1561/2200000071
[20]	Y. D. Yang and J. Wang, “An overview of multi-agent reinforcement learning from game theoretical perspective,” arXiv preprint, arXiv: 2011.00583, 2020.
[21]	Y. Li, S. Y. Xu, Y. P. Wu, et al., “Network energy-efficiency maximization in UAV-enabled air-ground-integrated deployment,” IEEE Internet of Things Journal, vol.9, no.15, pp.13209–13222, 2022. doi: 10.1109/JIOT.2022.3142073
[22]	X. F. Chen, C. Wu, T. Chen, et al., “Information freshness-aware task offloading in air-ground integrated edge computing systems,” IEEE Journal on Selected Areas in Communications, vol.40, no.1, pp.243–258, 2022. doi: 10.1109/JSAC.2021.3126075
[23]	X. W. Pang, N. Zhao, J. Tang, et al., “IRS-assisted secure UAV transmission via joint trajectory and beamforming design,” IEEE Transactions on Communications, vol.70, no.2, pp.1140–1152, 2022. doi: 10.1109/TCOMM.2021.3136563
[24]	M. Mozaffari, W. Saad, M. Bennis, et al., “Mobile unmanned aerial vehicles (UAVs) for energy-efficient internet of things communications,” IEEE Transactions on Wireless Communications, vol.16, no.11, pp.7574–7589, 2017. doi: 10.1109/TWC.2017.2751045
[25]	C. H. Liu, Z. Y. Chen, J. Tang, et al., “Energy-efficient UAV control for effective and fair communication coverage: A deep reinforcement learning approach,” IEEE Journal on Selected Areas in Communications, vol.36, no.9, pp.2059–2070, 2018. doi: 10.1109/JSAC.2018.2864373
[26]	M. Samir, D. Ebrahimi, C. Assi, et al., “Leveraging UAVs for coverage in cell-free vehicular networks: A deep reinforcement learning approach,” IEEE Transactions on Mobile Computing, vol.20, no.9, pp.2835–2847, 2021. doi: 10.1109/TMC.2020.2991326
[27]	X. Zhang and L. J. Duan, “Energy-saving deployment algorithms of UAV swarm for sustainable wireless coverage,” IEEE Transactions on Vehicular Technology, vol.69, no.9, pp.10320–10335, 2020. doi: 10.1109/TVT.2020.3004855
[28]	F. Huang, G. X. Li, H. C. Wang, et al., “Navigation for UAV pair-supported relaying in unknown IoT systems with deep reinforcement learning,” Chinese Journal of Electronics, vol.31, no.3, pp.416–429, 2022. doi: 10.1049/cje.2021.00.305
[29]	G. Y. Wu and J. C. Gu, “Remote interference source localization: A multi-UAV-based cooperative framework,” Chinese Journal of Electronics, vol.31, no.3, pp.442–455, 2022. doi: 10.1049/cje.2021.00.310
[30]	K. Zhu, J. Yang, Y. Zhang, et al., “Aerial refueling: Scheduling wireless energy charging for UAV enabled data collection,” IEEE Transactions on Green Communications and Networking, vol.6, no.3, pp.1494–1510, 2022. doi: 10.1109/TGCN.2022.3164602
[31]	S. Fu, Y. J. Tang, Y. Wu, et al., “Energy-efficient UAV-enabled data collection via wireless charging: a reinforcement learning approach,” IEEE Internet of Things Journal, vol.8, no.12, pp.10209–10219, 2021. doi: 10.1109/JIOT.2021.3051370
[32]	Z. H. Xiong, Y. Zhang, W. Y. B. Lim, et al., “UAV-assisted wireless energy and data transfer with deep reinforcement learning,” IEEE Transactions on Cognitive Communications and Networking, vol.7, no.1, pp.85–99, 2021. doi: 10.1109/TCCN.2020.3027696
[33]	Y. J. Fu, H. B. Mei, K. Z. Wang, et al., “Joint optimization of 3D trajectory and scheduling for solar-powered UAV systems,” IEEE Transactions on Vehicular Technology, vol.70, no.4, pp.3972–3977, 2021. doi: 10.1109/TVT.2021.3063310
[34]	M. Reymond and A. Nowe, “Pareto-DQN: Approximating the Pareto front in complex multi-objective decision problems, ” in Proceedings of the Adaptive and Learning Agents Workshop 2019, Montreal, Canada, 2019.
[35]	T. H. Wang, Y. G. Luo, J. X. Liu, et al., “Multi-objective end-to-end self-driving based on Pareto-optimal actor-critic approach, ” in Proceedings of 2021 IEEE International Intelligent Transportation Systems Conference, Indianapolis, IN, USA, pp.473–478, 2021.
[36]	Y. Zeng, J. Xu, and R. Zhang, “Energy minimization for wireless communication with rotary-wing UAV,” IEEE Transactions on Wireless Communications, vol.18, no.4, pp.2329–2345, 2019. doi: 10.1109/TWC.2019.2902559
[37]	A. Al-Hourani, S. Kandeepan, and A. Jamalipour, “Modeling air-to-ground path loss for low altitude platforms in urban environments, ” in Proceedings of 2014 IEEE Global Communications Conference, Austin, TX, USA, pp.2898–2904, 2014.
[38]	Q. Q. Zhang, W. Fang, Q. W. Liu, et al., “Distributed laser charging: A wireless power transfer approach,” IEEE Internet of Things Journal, vol.5, no.5, pp.3853–3864, 2018. doi: 10.1109/JIOT.2018.2851070
[39]	A. B. Sediq, R. H. Gohary, R. Schoenen, et al., “Optimal tradeoff between sum-rate efficiency and Jain’s fairness index in resource allocation,” IEEE Transactions on Wireless Communications, vol.12, no.7, pp.3496–3509, 2013. doi: 10.1109/TWC.2013.061413.121703
[40]	W. T. Wei, R. Y. Yang, H. X. Gu, et al., “Multi-objective optimization for resource allocation in vehicular cloud computing networks,” IEEE Transactions on Intelligent Transportation Systems, vol.23, no.12, pp.25536–25545, 2022. doi: 10.1109/TITS.2021.3091321
[41]	R. Rădulescu, P. Mannion, and D. M. Roijers, “Multi-objective multi-agent decision making: A utility-based analysis and survey,” Autonomous Agents and Multi-Agent Systems, vol.34, no.1, article no.10, 2020. doi: 10.1007/s10458-019-09433-x
[42]	C. F. Hayes, R. Rădulescu, E. Bargiacchi, et al., “A practical guide to multi-objective reinforcement learning and planning,” Autonomous Agents and Multi-Agent Systems, vol.36, no.1, article no.26, 2022. doi: 10.1007/S10458-022-09552-Y
[43]	J. G. Kuba, R. Q. Chen, M. N. Wen, et al., “Trust region policy optimisation in multi-agent reinforcement learning,” arXiv preprint, arXiv: 2109.11251, 2021.
[44]	M. Zhou, Z. Y. Wan, H. J. Wang, et al., “MALib: A parallel framework for population-based multi-agent reinforcement learning,” arXiv preprint, arXiv: 2106.07551, 2021.
[45]	J. Schulman, S. Levine, P. Moritz, et al., “Trust region policy optimization,” in Proceedings of the 32nd International Conference on International Conference on Machine Learning, Lille, France, pp.1889–1897, 2015.
[46]	W. J. Wang and M. Sebag, “Hypervolume indicator and dominance reward based multi-objective Monte-Carlo Tree Search,” Machine Learning, vol.92, no.2, pp.403–429, 2013. doi: 10.1007/s10994-013-5369-0
[47]	J. Xu, Y. S. Tian, P. C. Ma, et al., “Prediction-guided multi-objective reinforcement learning for continuous robot control,” in Proceedings of the 37th International Conference on Machine Learning, Vienna, Austria, pp. 10607–10616, 2020.
[48]	S. Q. Zhang, J. Y. Lin, and Q. Zhang, “Succinct and robust multi-agent communication with temporal message control,” in Proceedings of the 34th International Conference on Neural Information Processing Systems, Vancouver, Canada, pp. 17271–17282, 2020.