在高速下行封包擷取系統中採用乏晰Q-Learning技術之混合自動重傳機制

Abstract

第三代合作夥伴計畫（3rd generation partnership project, 3GPP）在現有的第三代通訊系統下，提出了一種高速下行封包擷取技術（high speed downlink packet access, HSDPA）來提供更高速且安全的下鏈路資料封包傳送。HSDPA採取混合自動重傳機制（hybrid automatic retransmission request, HARQ）程序，它有一個重要的服務品質要求：在根據通道狀況決定原始傳送的MCS時，必須使其封包錯誤率小於0.1。針對這個課題，我們在本篇論文裡利用乏晰Q 學習演算法來實現高速下行封包擷取技術之混合自動重傳機制（FQL based HARQ）。在同時考慮了通道動態調節以及重傳程序的交互影響的情況下，我們將HARQ程序模擬為離散時間馬可夫決策過程（Markov decision process, MDP），經由乏晰系統規則的設計，來實現BLER的服務品質需求，同時利用Q學習法，不斷的學習每一種MCS在不同的系統環境下之輸出表現，並修正乏晰系統規則。在學習收斂後，我們期望可以達到正確的選擇不違反QoS需求同時又能達到最高系統輸出的傳送方式之目的。由模擬結果可以看到，我們所提出的機制可以在有通道訊息延遲的情況下，對不同的行動用戶通道環境來選擇適當的MCS，既可滿足QoS的需求，又可比傳統只考慮通道動態調節的機制達到更高速的傳輸速率。相較於其他的比較系統，我們的FQL-HARQ可以在滿足QoS需求下，獲得最大之系統輸出。

In order to provide higher speed and more effective downlink packet data service in 3G, high speed downlink packet access (HSDPA) is proposed by 3rd generation partnership project (3GPP). An important QoS requirement defined in spec for the hybrid automatic retransmission request (HARQ) process is to choose a suitable MCS to maintain the initial block error rate (BLER) smaller than 0.1 based on the channel quality information. In this thesis, we proposed a fuzzy Q-learning based HARQ (FQL-HARQ) scheme for HSDPA to solve this problem. The HARQ scheme is modeled as a Markov decision process (MDP). On one hand, the fuzzy rule is designed to maintain the BLER requirement by separated to different parts based on a shot term BLER performance. On the other hand, by considering both link adaptation and HARQ version, the Q-learning algorithm is used to learn the performance of MCS under different environment. After learning, we want to choose the MCS with highest throughput while not going to violate the BLER requirement. The simulation results show that the proposed scheme can indeed choose a suitable MCS for the initial transmission with channel information delay consideration. Comparing to other traditional schemes, the FQL-HARQ scheme can achieve higher system throughput and maintain the BLER at the same time.