Q-Learning Based Interference-Aware Channel Handoff for Partially Observable Cognitive Radio Ad Hoc Networks

XU Ning; ZHANG Huyin; XU Fang; WANG Zhiyong

doi:10.1049/cje.2017.06.015

Volume 26 Issue 4

Turn off MathJax

Article Contents

Article Navigation > Chinese Journal of Electronics > 2017 > 26(4): 856-863

XU Ning, ZHANG Huyin, XU Fang, et al., “Q-Learning Based Interference-Aware Channel Handoff for Partially Observable Cognitive Radio Ad Hoc Networks,” Chinese Journal of Electronics, vol. 26, no. 4, pp. 856-863, 2017, doi: 10.1049/cje.2017.06.015

Citation:

XU Ning, ZHANG Huyin, XU Fang, et al., “Q-Learning Based Interference-Aware Channel Handoff for Partially Observable Cognitive Radio Ad Hoc Networks,” Chinese Journal of Electronics, vol. 26, no. 4, pp. 856-863, 2017, doi: 10.1049/cje.2017.06.015

Citation:

PDF( 707 KB)

Q-Learning Based Interference-Aware Channel Handoff for Partially Observable Cognitive Radio Ad Hoc Networks

doi: 10.1049/cje.2017.06.015

Department of Computer, Wuhan University, Wuhan 430072, China

Funds: This work is supported by the National Natural Science Foundation of China (No.61272454, No.61540059), the Specialized Research Fund for the Doctoral Program of Higher Education (No.20130141110022), and the Natural Science Foundation of Hubei Province in China (No.2016CFB652).

More Information

Corresponding author: ZHANG Huyin (corresponding author) received the B.S. degree in electronics from Wuhan College of Technology, China, in 1983. He received the M.S. degree in application of computer science from Wuhan Technical University of Surveying and Mapping, China in 2000 and Ph.D. degree in computer application technology from Wuhan University, China in 2006. He is currently a professor and a Ph.D. supervisor in the department of computer, Wuhan University. His research interests include computer network, QoS, network management and ad hoc network. (Email: zhy2536@whu.edu.cn)
Received Date: 2016-01-19
Rev Recd Date: 2016-11-29
Publish Date: 2017-07-10

Abstract

Abstract

Channel handoff is a crucial function for Cognitive radio ad hoc networks (CRAHNs). The absence of centralized infrastructures and the limited power make the handoff design more challenging. A learningbased interference-aware handoff scheme is proposed for distributed CRAHNs. We model the channel handoff process as a Partially observable Markov decision process (POMDP) and adopt a Q-learning algorithm to find an optimal handoff strategy in a long term. The proposed algorithm obtains an efficient transmission performance by considering the interferences among SUs and PUs. To achieve PU awareness, the handoff scheme predicts the PU activities by using the historical channel usage statistics. In addition, we also propose a refined channel selection rule to compromise between learning speed and cumulative transmission reward. The simulation results show that the proposed handoff scheme can adapt to the PU activities and achieves a better performance in terms of high throughput and low collisions. And the learning process keeps a considerable balance between convergence time and cumulative reward.
- Cognitive radio,
- Ad hoc networks,
- Partially observable Markov decision process (POMDP),
- Handoff,
- Q-Learning

FullText(HTML)

References(18)

References

F.C. Commission, et al., “ET docket no. 03-289”, Technical Report, Tech. Rep., FCC, 2003.

I.F. Akyildiz, W.Y. Lee, M.C. Vuran, et al., “NeXt generation/dynamic spectrum access/cognitive radio wireless networks: A survey”, Computer Networks, Vol.50, No.13, pp.2127-2159, 2006.

I.F. Akyildiz, W.Y. Lee and K.R. Chowdhury, “CRAHNs: Cognitive radio ad hoc networks”, Ad Hoc Networks, Vol.7, No.5, pp.810-836, 2009.

M. NoroozOliaee, B. Hamdaoui, X. Cheng, et al., “Analyzing cognitive network access efficiency under limited spectrum handoff agility”, IEEE Transactions on Vehicular Technology, Vol.63, No.3, pp.1402-1407, 2014.

Y. Song and J. Xie, “ProSpect: A proactive spectrum handoff framework for cognitive radio ad hoc networks without common control channel”, IEEE Transactions on Mobile Computing, Vol.11, No.7, pp.1127-1139, 2012.

K. Kumar, A. Prakash and R. Tripathi, “Spectrum handoff in cognitive radio networks: A classification and comprehensive survey”, Journal of Network and Computer Applications, Vol.61, No.2, pp.161-188, 2016.

X. Liu, J. Li and J. Jiang, “Optimal cooperative sensing order using inactive secondary users in cognitive radio networks”, Chinese Journal of Electronics, Vol.23, No.2, pp.394-398, 2014.

L. Zhang, T. Song, M. Wu, et al., “Traffic-adaptive proactive spectrum handoff strategy for graded secondary users in cognitive radio networks”, Chinese Journal of Electronics, Vol.24, No.4, pp.844-851, 2015.

I.F. Akyildiz, W.Y. Lee and K.R. Chowdhury, “Spectrum management in cognitive radio ad hoc networks”, IEEE Network, Vol.23, No.4, pp.6-12, 2009.

L.C. Wang, C.W. Wang and C.J. Chang, “Modeling and analysis for spectrum handoffs in cognitive radio networks”, IEEE Transactions on Mobile Computing, Vol.11, No.9, pp.1499-1513, 2012.

C. Zhang, X.Wang and J. Li, “Cooperative cognitive radio with priority queueing analysis”, Proc. of IEEE International Conference on Communications, Dresden, Germany, pp.1-5, 2009.

J.H. Chu, R.T. Ma and K.T. Feng, “Stochastic spectrum handoff protocols for partially observable cognitive radio networks”, Wireless Networks, Vol.20, No.5, pp.1003-1022, 2014.

G. Zhang, A. Huang, H. Shan, et al., “Design and analysis of distributed hopping-based channel access in multi-channel cognitive radio systems with delay constraints”, IEEE Journal on Selected Areas in Communications, Vol.32, No.11, pp.2026-2038, 2014.

Y. Wu, F. Hu, S. Kumar, et al., “A learning-based QoE-driven spectrum handoff scheme for multimedia transmissions over cognitive radio networks”, IEEE Journal on Selected Areas in Communications, Vol.32, No.11, pp.2134-2148, 2014.

N.C. Theis, R.W. Thomas, L. DaSilva, et al., “Rendezvous for cognitive radios”, IEEE Transactions on Mobile Computing, Vol.10, No.2, pp.216-227, 2011.

H.P. Shiang and M. Van der Schaar, “Online learning in autonomic multi-hop wireless networks for transmitting missioncritical applications”, IEEE Journal on Selected Areas in Communications, Vol.28, No.5, pp.728-741, 2010.

R.S. Sutton and A.G. Barto, Reinforcement Learning: An Introduction, MIT press, Cambridge, Massachusetts, USA, pp.144-147, 2012.

N. Baldo, A. Asterjadhi and M. Zorzi, “Dynamic spectrum access using a network coded cognitive control channel”, IEEE Transactions on Wireless Communications, Vol.9, No.8, pp.2575-2587, 2010.

Relative Articles

Supplements(0)

Cited By

Proportional views