基于强化学习的异构网络垂直切换方法

Vertical handover for heterogeneous networksbased on deep reinforcement learning

摘要:

网络切换技术不仅可以保证用户的网络连接，同时能够以较强的信号传输网络数据。网络切换技术的性能对网络服务质量(quality of service, QoS)的影响至关重要。然而，现有的切换算法多数存在严重的乒乓效应，这不仅造成网络资源的严重浪费，还会损害用户的QoS。为此，提出了一种基于强化学习的异构网络垂直切换方案，主要从触发切换、网络选择及判决切换等方面进行优化。在触发切换时将垂直切换考虑成必要切换和择优切换，通过Q-Learning（QL）算法在选择网络时优化垂直切换；然后以QoS为条件，在判决切换时加入驻留定时器，从多角度减少用户切换次数，降低乒乓效应对异构网络垂直切换的影响。仿真结果表明，基于强化学习的异构网络垂直切换方法可以在保证QoS的条件下有效减少用户切换网络的次数，短时间内频繁切换的情况也有所改善，降低了乒乓效应的影响。

Abstract:

Network handover technology can not only ensure the network connection of users, but also transmit network data with strong signals. The performance of network handover has a critical impact on the quality of service (QoS). However, most of the existing handover algorithms have serious ping-pong effect, which may cause a waste of network resources and damage the QoS. Therefore, a vertical handover scheme for heterogeneous networks based on reinforcement learning was proposed, which was mainly optimized from the aspects of triggering handoff, network selection and decision handover. Specifically, the necessary and the preferred handover were considered for the vertical handover when the handover was triggered. Then, the vertical handover was optimized when the network through Q-Learning(QL) was selected. Based on QoS, the resident timer was also added when deciding the handover to reduce the number of user handover from multiple angles and reduced the impact of ping-pong effect on the proposed vertical handover of heterogeneous networks. Simulation results have shown that the proposed vertical handover scheme for heterogeneous networks could effectively reduce the number of times for handover, improve the situation of frequent handover in a short time, and reduce the impact of the ping-pong effect while ensuring the quality of service.