適用於UMTS高速下行封包擷取技術之Q-Learning式混合自動重傳機制

Abstract

為了在現有的WCDMA universal mobile telecommunications system (UMTS)下提供更高速、有效率、健全的下鏈路資料封包傳送，一種高速下行封包擷取技術 (high speed downlink packet access HSDPA) 被3rd generation partnership project (3GPP) 所提出，且已經在Release 5被標準化。針對不同的連結，為了作到更好的適應性，HSDPA除了動態的去調整使用不同的調變方式、不同的編碼速率 (AMC) 提供更多不同的傳輸速率；提供更大量的多碼 (multi-code) 使用運作；同時配合一種混合自動重傳機制 (H-ARQ)；以及大幅縮短成2ms的傳送時間間隔，希望去創造更有效率的資源分配。 在本篇論文中，我們提出了一個適用於UMTS高速下行封包擷取技術之Q-learning式混合自動重傳機制 (Q-HARQ)。我們先將整個H-ARQ的程序模擬成一種離散時間馬可夫決策過程 (Markov decision process MDP)，並將封包傳送所會付出的代價 (cost) 針對我們所希望去滿足的傳輸服務品質 (quality of service QoS)的封包傳送錯誤率 (BLER) 作設計；再利用一種名為 Q-learning 的即時加強型學習演算法去估計每一次的傳送代價，不斷的去學習，針對每次封包的第一次傳送過程，來達到最佳且符合一定傳輸品質 (QoS) 傳送決策。 模擬結果顯示，我們所提出的方法可以在滿足我們所要求BLER下去選擇最佳的傳送決策。這意味著，針對封包的第一次傳送，我們能提供最有效率的傳輸方法以對抗變化劇烈的通道環境。另一方面，我們也證實了我們所提出的Q-HARQ機制在收斂時間，及處理運算時間均符合實際系統的要求，適用於現行的通訊系統。

WCDMA Release 5 has been standardized for universal mobile telecommunications system (UMTS) in the 3rd generation partnership project (3GPP), where high speed downlink packet access (HSDPA) is proposed to provide efficient, robust, and high-speed packet data services for UMTS. In HSDPA, the adaptive modulation and coding (AMC) technique and extensive multi-code operation are adopted for the link adaptation. Also, an advanced retransmission strategy based on hybrid automatic repeat request (H-ARQ) is proposed to upgrade the robustness against link adaptation errors. In this thesis, a Q-learning-based hybrid automatic repeat request (Q-HARQ) scheme for HSDPA in UMTS system is proposed to achieve efficient resource utilization. The Hybrid ARQ procedure is modeled as a discrete-time Markov decision process, where the transmission cost is defined in terms of the QoS parameters of transport block error rate for enhancing spectrum utilization subject to QoS constraint. The Q-learning reinforcement algorithm is employed to accurately estimate the transmission cost to perform the most suitable decision of modulation and coding scheme for the packet initial transmission while the requirement of transport block error rate is guaranteed.

Simulation results show that the QoS requirement of BLER for Q-HARQ is indeed fulfilled. In addition, the performance of the Q-HARQ can be improved under the specific QoS constraint of BLER. It is verified finally that the Q-HARQ scheme is feasible in the practical system.