異質無線網路中以乏晰Q-Learning為基礎之網路選取機制

Abstract

隨著智慧型手機與行動裝置日益普及，透過無線網路行動上網的需求量增加，使得現有的基地台面臨數據頻寬不夠服務大量使用者的問題。於是基地台業者開始佈建許多比較平價低廉的小型基地台，以及在人口密集的定點佈建Wi-Fi熱點提供服務，期望結合LTE-A與Wi-Fi共存的異質網路系統，可以容納更多使用者連線，提供更快速且穩定的服務品質，滿足大量資料傳輸的需求。因此在異質網路中，如何根據使用者各自移動與通道狀況及服務品質需求選取適合的無線網路連線，提供更高速的傳輸速率且容納更多行動用戶，成為一項重要的議題。 在本篇論文中，我們提出了結合模糊理論(fuzzy logic theory)與Q-learning學習法之網路選取機制，考量同時有LTE-A大型與小型基地台及電信業者佈建之Wi-Fi覆蓋範圍的環境下，能夠選擇出最適當的網路連線，增加可容納的使用者個數，提升整體系統傳輸速率。此機制考慮了三種系統參數:通道品質(SINR) ，負載狀況(loading)，以及移動速度(mobility)，透過乏晰邏輯(fuzzy inference system)判斷每個候選網路狀況並且計算出相對應的Q-value值，最後選擇Q-value值最大的為目標網路，接著Q學習演算法會根據服務品質需求(QoS requirement) ，不斷地學習與修正乏晰系統規則，做出更佳的決策。模擬結果顯示我們提出的方法，不僅能提升系統傳輸速率(throughput)，同時能讓使用者體驗到良好的服務品質，且可以更適應網路環境的變化，經由學習機制做出最佳的網路選擇。

To cope with the explosive growth in traffic demands for data transmission with limited network capacity, the Internet operators tend to develop the heterogeneous networks by setting up many small cell base stations or providing Wi-Fi access points as hotspots for service. As a result, how to find an effective mechanism to select the suitable target network for new arrival user to access in the heterogeneous wireless environment becomes an important issue nowadays. In this thesis, a fuzzy Q-learning based network selection (FQNS) scheme is proposed to maximize system capacity, improve total throughput, and satisfy quality of service (QoS) requirements in LTE-A/Wi-Fi heterogeneous networks. Integrating the fuzzy logic with Q-learning algorithm, the FQNS scheme considers SINR factor, loading factor, and mobility factor to choose appropriate network. Through fuzzy inference system, we can know the candidate network states and calculate the corresponding Q-value. The target network, which has the maximum Q-value, could be selected for service. And then, the FQNS scheme will receive a reinforcement signal as a feedback from system to adaptively adjust the action decision and keep self-learning. Eventually, simulation results show that the FQNS scheme has better performance than the PVCS scheme, which means that system utilizes the FQNS scheme not only can achieve higher total throughput but also satisfy QoS requirements at the same time.