Citation: | WANG Chaowei, CUI Yuling, DENG Danhao, et al., “Trajectory Optimization and Power Allocation Scheme Based on DRL in Energy Efficient UAV-Aided Communication Networks,” Chinese Journal of Electronics, vol. 31, no. 3, pp. 397-407, 2022, doi: 10.1049/cje.2021.00.314 |
[1] |
Y. Du, H. Zhu, H. Yang, et al., “Review of 5G mobile communication technology standard,” Telecommunications Science, vol.34, no.8, pp.2–9, 2018.
|
[2] |
S. Jin, X. Zhang, and Q. Zhou, “A statistical model for the UAV communication channel,” Acta Aeronautica et Astronautica Sinica, vol.25, no.1, pp.62–65, 2004.
|
[3] |
Zeng Y, Lyu J, Zhang R, “Cellular-connected UAV: Potential, challenges, and promising technologies,” IEEE Wireless Communications, vol.26, no.1, pp.120–127, 2018. doi: 10.1109/MWC.2018.1800023
|
[4] |
Y. Zeng, R. Zhang, and T. J. Lim, “Wireless communications with unmanned aerial vehicles: Opportunities and challenges,” IEEE Commun. Mag., vol.54, no.5, pp.36–42, 2016.
|
[5] |
J. Zhao, F. Gao, G. Ding, et al., “Integrating communications and control for UAV systems: Opportunities and challenges,” IEEE Access, vol.6, pp.67519–67527, 2018. doi: 10.1109/ACCESS.2018.2879637
|
[6] |
M. Mozaffari, W. Saad, M. Bennis, et al., “Efficient deployment of multiple unmanned aerial vehicles for optimal wireless coverage,” IEEE Commun. Lett., vol.20, no.8, pp.1647–1650, 2016. doi: 10.1109/LCOMM.2016.2578312
|
[7] |
M. Alzenad, A. El-Keyi, F. Lagum, et al., “3-D placement of an unmanned aerial vehicle base station (UAV-BS) for energyefficient maximal coverage,” IEEE Wireless Communications Letters, vol.6, no.4, pp.434–437, 2017. doi: 10.1109/LWC.2017.2700840
|
[8] |
R. I. Bor-Yaliniz, A. El-Keyi, and H. Yanikomeroglu, “Efficient 3-D placement of an aerial base station in next generation cellular networks,” in Proc. of 2016 IEEE International Conference on Communications (ICC), Kuala Lumpur, Malaysia, pp.1–5, 2016.
|
[9] |
J. Lyu, Y. Zeng, R. Zhang, et al., “Placement optimization of UAV-mounted mobile base stations,” IEEE Commun. Lett., vol.21, no.3, pp.604–607, 2017. doi: 10.1109/LCOMM.2016.2633248
|
[10] |
J. Zhao, F. Gao, Q. Wu, S. Jin, et al., “Beam tracking for UAV mounted satcom on-the-move with massive antenna array,” IEEE Journal on Selected Areas in Communications, vol.36, no.2, pp.363–375, 2018.
|
[11] |
L. Xiao, X. Lu, D. Xu, Y. Tang, et al., “UAV relay in vanets against smart jamming with reinforcement learning,” IEEE Transactions on Vehicular Technology, vol.67, no.5, pp.4087–4097, 2018.
|
[12] |
D. Orfanus, E. P. de Freitas, and F. Eliassen, “Self-organization as a supporting paradigm for military uav relay networks,” IEEE Communications Letters, vol.20, no.4, pp.804–807, 2016.
|
[13] |
N. Cheng, F. Lyu, W. Quan, et al., “Space/aerial-assisted computing offloading for IoT applications: A learning-based approach,” IEEE Journal on Selected Areas in Communications, vol.37, no.5, pp.1117–1129, 2019. doi: 10.1109/JSAC.2019.2906789
|
[14] |
F. Cheng, G. Gui, N. Zhao, et al., “UAV-relayingassisted secure transmission with caching,” IEEE Transactions on Communications, vol.67, no.5, pp.3140–3153, 2019. doi: 10.1109/TCOMM.2019.2895088
|
[15] |
H. Wang, J. Wang, G. Ding, et al., “Spectrum sharing planning for full-duplex uav relaying systems with underlaid D2D communications,” IEEE Journal on Selected Areas in Communications, vol.36, no.9, pp.1986–1999, 2018. doi: 10.1109/JSAC.2018.2864375
|
[16] |
Y. Yu, X. Bu, K. Yang, et al., “UAV-aided low latency mobile edge computing with mmwave backhaul,” in Proc. of 2019 IEEE International Conference on Communications (ICC), Shanghai, China, pp.1–7, 2019.
|
[17] |
M. Erdelj, E. Natalizio, K. R. Chowdhury, and I. F. Akyildiz, “Help from the sky: Leveraging UAVs for disaster management,” IEEE Pervasive Computing, vol.16, no.1, pp.24–32, 2017. doi: 10.1109/MPRV.2017.11
|
[18] |
C. She, C. Liu, T. Q. S. Quek, et al., “UAV-assisted uplink transmission for ultra-reliable and low-latency communications,” in Proc. of 2018 IEEE International Conference on Communications Workshops (ICC Workshops), Kansas City, MO, USA, pp.1–6, 2018.
|
[19] |
M. Mozaffari, W. Saad, M. Bennis, et al., “Wireless communication using unmanned aerial vehicles (UAVs): Optimal transport theory for hover time optimization,” IEEE Transactions on Wireless Communications, vol.16, no.12, pp.8052–8066, 2017.
|
[20] |
W. Shi, J. Li, N. Cheng, et al., “Multi-drone 3-D trajectory planning and scheduling in drone-assisted radio access networks,” IEEE Transactions on Vehicular Technology, vol.68, no.8, pp.8145–8158, 2019.
|
[21] |
J. Wang, C. Jiang, Z. Wei, et al., “Joint UAV hovering altitude and power control for space-air-ground IoT networks,” IEEE Internet of Things Journal, vol.6, no.2, pp.1741–1753, 2019. doi: 10.1109/JIOT.2018.2875493
|
[22] |
W. Wang, J. Tang, N. Zhao, X. Liu, et al., “Joint precoding optimization for secure swipt in uav-aided noma networks,” IEEE Trans. Commun., vol.68, no.8, pp.5028–5040, 2020.
|
[23] |
X. Mao, Y. Gu, and W. Yin, “Walk proximal gradient: An energyefficient algorithm for consensus optimization,” IEEE Internet Things J., vol.6, no.2, pp.2048–2060, 2019.
|
[24] |
J. Zhao, X. Guan, X. Li, et al., “Power allocation based on genetic simulated annealing algorithm in cognitive radio networks,” Chin. J. Electron., vol.22, no.1, pp.177–180, 2013.
|
[25] |
F. Cheng, S. Zhang, Z. Li, et al., “UAV trajectory optimization for data offloading at the edge of multiple cells,” IEEE Transactions on Vehicular Technology, vol.67, no.7, pp.6732–6736, 2018.
|
[26] |
J. Zhao, Q. Li, Y. Gong, et al., “Computation offloading and resource allocation for cloud assisted mobile edge computing in vehicular networks,” IEEE Trans. Veh. Tech., vol.68, no.8, pp.7944–7956, 2019.
|
[27] |
X. Xi, X. Cao, P. Yang, J. Chen, et al., “Joint user association and UAV location optimization for UAV-aided communications,” IEEE Wireless Communications Letters, vol.8, no.6, pp.1688–1691, 2019.
|
[28] |
P. Yang, X. Xi, K. Guo, et al., “Proactive UAV network slicing for URLLC and mobile broadband service multiplexing,” IEEE Journal on Selected Areas in Communications, vol.39, no.10, pp.3225–3244, 2021. doi: 10.1109/JSAC.2021.3088624
|
[29] |
L. Zhu, J. Zhang, Z. Xiao, et al., “Millimeter-wave full-duplex UAV relay: Joint positioning, beamforming, and power control,” IEEE Journal on Selected Areas in Communications, vol.38, no.9, pp.2057–2073, 2020.
|
[30] |
Y. Wang, Z. Gao, J. Zhang, et al., “Trajectory design for UAV-based Internet-of-things data collection: A deep reinforcement learning approach,” IEEE Internet of Things Journal, vol.9, no.5, pp.3899–3912, 2022. doi: 10.1109/JIOT.2021.3102185
|
[31] |
X. Yuan, Z. Feng, W. Xu, et al., “Capacity analysis of UAV communications: Cases of random trajectories,” IEEE Transactions on Vehicular Technology, vol.67, no.8, pp.7564–7576, 2018.
|
[32] |
Q. Zhang, M. Jiang, Z. Feng, et al., “IoT enabled UAV: Network architecture and routing algorithm,” IEEE Internet of Things Journal, vol.6, no.2, pp.3727–3742, 2019.
|
[33] |
X. Yuan, Z. Feng, W. Ni, et al., “Connectivity of UAV swarms in 3D spherical spaces under (un)intentional ground interference,” IEEE Transactions on Vehicular Technology, vol.69, no.8, pp.8792–8804, 2020.
|
[34] |
C. Qiu, Z. Wei, X. Yuan, et al., “Multiple UAV-mounted base station placement and user association with joint fronthaul and backhaul optimization,” IEEE Transactions on Communications, vol.68, no.9, pp.5864–5877, 2020.
|
[35] |
J. Chen, P. Chen, Q. Wu, et al., “A game-theoretic perspective on resource management for large-scale UAV communication networks,” China Communications, vol.18, no.1, pp.70–87, 2021.
|
[36] |
Z. Wang, F. Zhou, Y. Wang, et al., “Joint 3D trajectory and resource optimization for a UAV relay-assisted cognitive radio network,” China Communications, vol.18, no.6, pp.184–200, 2021.
|
[37] |
B. Liu, Y. Wan, F. Zhou, et al., “Robust trajectory and beamforming design for cognitive MISO UAV networks,” IEEE Wireless Communications Letters, vol.10, no.2, pp.396–400, 2021.
|
[38] |
Z. Xiao, H. Dong, L. Bai, et al., “Unmanned aerial vehicle base station (UAV-BS) deployment with millimeter-wave beamforming,” IEEE Internet of Things Journal, vol.7, no.2, pp.1336–1349, 2020.
|
[39] |
L. Zhu, J. Zhang, Z. Xiao, et al., “3-D Beamforming for flexible coverage in millimeter-wave UAV communications,” IEEE Wireless Communications Letters, vol.8, no.3, pp.837–840, 2019.
|
[40] |
Z. Xiao, L. Zhu, and X. -G. Xia, “UAV communications with millimeter-wave beamforming: Potentials, scenarios, and challenges,” China Communications, vol.17, no.9, pp.147–166, 2020. doi: 10.23919/JCC.2020.09.012
|
[41] |
L. Zhu, J. Zhang, Z. Xiao, et al., “Millimeter-wave NOMA with user grouping, power allocation and hybrid beamforming,” IEEE Transactions on Wireless Communications, vol.18, no.11, pp.5065–5079, 2019. doi: 10.1109/TWC.2019.2932070
|
[42] |
J. Wang, C. Jiang, Z. Han, et al., “Taking drones to the next level: Cooperative distributed unmanned-aerial-vehicular networks for small and mini drones,” IEEE Vehicular Technology Magazine, vol.12, no.3, pp.73–82, 2017. doi: 10.1109/MVT.2016.2645481
|
[43] |
Y. Zeng and R. Zhang, “Energy-efficient UAV communication with trajectory optimization,” IEEE Trans. Wireless Commun., vol.16, no.6, pp.3747–3760, 2017.
|
[44] |
R. Ding, F. Gao, and X. S. Shen, “3D UAV trajectory design and frequency band allocation for energy-efficient and fair communication: A deep reinforcement learning approach,” IEEE Transactions on Wireless Communications, vol.19, no.12, pp.7796–7809, 2020. doi: 10.1109/TWC.2020.3016024
|
[45] |
X. Li, H. Yao, J. Wang, et al., “A near-optimal UAV-aided radio coverage strategy for dense urban areas,” IEEE Transactions on Vehicular Technology, vol.68, no.9, pp.9098–9109, 2019. doi: 10.1109/TVT.2019.2927425
|
[46] |
V. Mnih, K. Kavukcuoglu, D. Silver, et al., “Human-level control through deep reinforcement learning,” Nature, vol.518, pp.529–533, 2015.
|
[47] |
Z. Mou, Y. Zhang, F. Gao, et al., “Deep reinforcement learning based three-dimensional area coverage with UAV swarm,” IEEE Journal on Selected Areas in Communications, vol.39, no.10, pp.3160–3176, 2021. doi: 10.1109/JSAC.2021.3088718
|
[48] |
C. H. Liu, Z. Chen, J. Tang, et al., “Energy-efficient UAV control for effective and fair communication coverage: A deep reinforcement learning approach,” IEEE J. Selected Areas in Commun., vol.36, no.9, pp.2059–2070, 2018. doi: 10.1109/JSAC.2018.2864373
|
[49] |
R. Ding, Y. Xu, F. Gao, et al., “Trajectory design and access control for air-ground coordinated communications system with multi-agent deep reinforcement learning,” IEEE Internet of Things, vol.9, no.8, pp.5785–5798, 2022. doi: 10.1109/JIOT.2021.3062091
|
[50] |
Y. Zeng, R. Zhang, and T. J. Lim, “Throughput maximization for UAV-enabled mobile relaying systems,” IEEE Trans. Commun., vol.64, no.12, pp.4983–4996, 2016. doi: 10.1109/TCOMM.2016.2611512
|
[51] |
Y. Zeng, J. Xu, and R. Zhang, “Energy minimization for wireless communication with rotary-wing UAV,” IEEE Transactions on Wireless Communications, vol.18, no.4, pp.2329–2345, 2019. doi: 10.1109/TWC.2019.2902559
|