MDP-Based Network Selection with Reward Optimization in HetNets

CHEN Xin; LI Zhuo; WANG Kai; XING Lei

doi:10.1049/cje.2017.11.017

Volume 27 Issue 1

Turn off MathJax

Article Contents

Article Navigation > Chinese Journal of Electronics > 2018 > 27(1): 183-190

CHEN Xin, LI Zhuo, WANG Kai, et al., “MDP-Based Network Selection with Reward Optimization in HetNets,” Chinese Journal of Electronics, vol. 27, no. 1, pp. 183-190, 2018, doi: 10.1049/cje.2017.11.017

Citation:

CHEN Xin, LI Zhuo, WANG Kai, et al., “MDP-Based Network Selection with Reward Optimization in HetNets,” Chinese Journal of Electronics, vol. 27, no. 1, pp. 183-190, 2018, doi: 10.1049/cje.2017.11.017

Citation:

PDF( 409 KB)

MDP-Based Network Selection with Reward Optimization in HetNets

doi: 10.1049/cje.2017.11.017

1.
School of Computer Science, Beijing Information Science and Technology University, Beijing 100101, China;
2.
Beijing Key Laboratory of Internet Culture and Digital Dissemination Research, Beijing Information Science and Technology University, Beijing 100101, China

Funds: This work is supported by the National Natural Science Foundation of China (No.61370065, No.61502040), National Key Technology Research and Development Program of the Ministry of Science and Technology of China (No.2015BAK12B03-03), and Beijing Municipal Program for Excellent Teacher Promotion (No.PXM2017-014224-000028).

Received Date: 2017-05-16
Rev Recd Date: 2017-08-02
Publish Date: 2018-01-10

Abstract

Abstract

We investigate the network selection problem over heterogeneous networks. We formulate the decision making problem as a Markov decision process (MDP). Two Poisson processes are introduced to model user arrival and departure, and the change of the number of user in a specific network is described by Skellam Distribution. We apply the Total order preference by similarity to the ideal solution (TOPSIS) method to address the issue of state space explosion in MDP model. To acquire the optimal network selection scheme, the value iteration algorithm is employed to solve the MDP model. We take simulations in Matlab to prove the effectiveness of the proposed scheme. From the experiment results, it is found that our proposed scheme can reduce handover probability by 7.5% and 40.4% at the early stage of connection, compared with the Infinite MDP-based (IMDP) strategy and TOPSISonly strategy. It can also reduce blocking probability by 6.1% compared with IMDP, while maintaining desirable throughput for the target user.
- Heterogeneous networks,
- Network selection,
- Markov decision process (MDP),
- Multiple attributes decision making

FullText(HTML)

References(27)

References

Cisco, "Cisco visual networking index:Global mobile data traffic forecast update, 2014-2019", Technical report, February 2015.

C.K. Ho, D. Yuan and S. Sun, "Data offloading in load coupled networks:A utility maximization framework", IEEE Transactions on Wireless Communications, Vol.13, No.4, pp.1921-1931, 2014.

N. Xu, H.Y. Zhang, F. Xu, et al., "Q-Learning based interference-aware channel handoff for partially observable cognitive radio Ad Hoc networks", Chinese Journal of Electronics, Vol.26, No.4, pp.856-863, 2017.

Q. Ye, B. Rong, Y. Chen, et al., "User association for load balancing in heterogeneous cellular networks", IEEE Transactions on Wireless Communications, Vol.12, No.6, pp.2706-2716, 2013.

I.Z. Kovács, D. Laselva, P.H. Michaelsen, et al., "Performance of SON for RSRP-based LTE/WLAN access network selection", 201411th International Symposium on Wireless Communications Systems, Barcelona, Spain, pp.360-364, 2014.

X.Y. Wu and Q.H. Du, "Utility-function-based radio-accesstechnology selection for heterogeneous wireless networks", Computers and Electrical Engineering, Vol.52, pp.171-182, 2016.

P. Kosmides, A. Rouskas, and M. Anagnostou. "Utilitybased rat selection optimization in heterogeneous wireless networks", Pervasive and Mobile Computing, Vol.12, No.10, pp.92-111,2014.

A. Awad, A. Mohamed and C.F. Chiasserini, "Dynamic network selection in heterogeneous wireless networks:A user-centric scheme for improved delivery", IEEE Consumer Electronics Magazine, Vol.6, No.1, pp.53-60, 2017.

N.N. Sui, D.M. Zhang, W. Zhong, et al., "Network selection for heterogeneous wireless networks based on multiple attribute decision making and evolutionary game theory", 201625th Wireless and Optical Communication Conference, Chengdu, China, pp.1-5, 2016.

N.N. Sui, D.M. Zhang, W. Zhong, et al., "Evolutionary game theory based network selection for constrained heterogeneous networks", 20152nd International Conference on Information Science and Control Engineering, Shanghai, China, pp.738-742, 2015.

B. Liu, H. Tian, B. Wang, et al., "AHP and game theory based approach for network selection in heterogeneous wireless networks", 2014 IEEE 11th Consumer Communications and Networking Conference, Las Vegas, NV, USA pp.501-506, 2014.

T. Goyal and S. Kaushal, "Optimized network selection during handover using analytic hierarchy process in 4G network", 2016 International Conference on Advances in Computing, Communication, & Automation (Spring), Dehradun, India, pp.1-4, 2016.

P. Tillapart, T. Thumthawatworn, P. Viriyaphol, et al., "Intelligent handover decision based on fuzzy logic for heterogeneous wireless networks", 201512th International Conference on Electrical Engineering/Electronics, Computer, Telecommunications and Information Technology, Hua Hin, Thailand, pp.1-6, 2015.

R. Trestian, O. Ormond and G.M. Muntean, "Performance evaluation of madm-based methods for network selection in a multimedia wireless environment", Wireless Networks, Vol.21, No.5, pp.1-19, 2014.

R. Sadeghian and M.R. Sadeghian, "A decision support system based on artificial neural network and fuzzy analytic network process for selection of machine tools in a flexible manufacturing system", The International Journal of Advanced Manufacturing Technology, Vol.82, No.9, pp.1-9, 2015.

K. Ahuja, B. Singh and R. Khanna, "Network selection algorithm based on link quality parameters for heterogeneous wireless networks", Optik-International Journal for Light and Electron Optics, Vol.125, No.14, pp.3657-3662, 2014.

M.A. Senouci, S. Hoceini, and A. Mellouk, "Utility functionbased TOPSIS for network interface selection in Heterogeneous Wireless Networks", 2016 IEEE International Conference on Communications, Kuala Lumpur, Malaysia, pp.1-6, 2016.

F.A. Tuzunkan, V.C. Gungor, E.Zeydan, et al., "Seamless mobile data offloading in heterogeneous wireless networks based on IEEE 802.21 and user experience", 2014 IEEE Wireless Communications and Networking Conference Workshops, Istanbul, Turkey, pp.178-183, 2014.

M. Drissi and M. Oumsis, "Performance evaluation of multicriteria vertical handover for heterogeneous wireless networks", 2015 Intelligent Systems and Computer Vision, Fez, Morocoo, pp.1-5, 2015.

N. Singh and B. Singh, "Vertical handoff decision in 4g wireless networks using multi attribute decision making approach", Wireless Networks, pp.1203-1211, 2014.

H. Wang, C. Rosa, and K. Pedersen, "Analysis of carrier deployment strategies for LTE-A HetNets with multicell cooperation", 2014 IEEE 80th Vehicular Technology Conference, Vancouver, BC, Canada, pp.1-5, 2014.

P.Y. Kong, "Optimal probabilistic policy for dynamic resource activation using markov decision process in green wireless networks", IEEE Transactions on Mobile Computing, Vol.13, No.10, pp.2357-2368, 2014.

D. Ma and M.D. Ma, "Network selection and resource allocation for multicast in hetnets", Journal of Network and Computer Applications, Vol.43, No.5, pp.17-26, 2014.

M.E. Helou, M. Ibrahim, S. Lahoud, et al., "A Network-Assisted approach for RAT selection in heterogeneous cellular networks", IEEE Journal on Selected Areas in Communications, Vol.33, No.6, pp.1055-1067, 2015.

M. Mezzavilla, S. Goyal, S. Panwar, et al., "An MDP model for optimal handover decisions in mmWave cellular networks", 2016 European Conference on Networks and Communications, Athens, Greece, pp.100-105, 2016.

N. Kiran, R. Raj and A. Khan, "Maximization of throughput in cognitive radio using non-persistent csma combined with reinforcement learning", International Journal of Innovative Research in Electrical, Electronics, Instrumentation and Control Engineering, Vol.3, No.5, pp.75-79, 2015.

H. Tabrizi, G. Farhadi and J. Cioffi, "A Learning-Based network selection method in heterogeneous wireless systems", 2011 IEEE Global Telecommunications Conference-GLOBECOM 2011, Houston, TX, USA, pp.1-5, 2011.

Relative Articles

Supplements(0)

Cited By

Proportional views