A Q-learning-based multi-rate transmission control scheme for RRC in WCDMA systems

Abstract

In this paper, a Q-learning-based multi-rate transmission control scheme (Q-MRTC) for radio resource control (RRC) in WCDMA systems is proposed. The RRC problem is modelled as a semi-Markov decision process (SMDP). And we successfully apply a real-time reinforcement learning algorithm, named Q-learning, to accurately estimate the transmission cost for the multi-rate transmission control. For the cost function approximation, we apply the feature extraction method to map the original state space into a more compact set which represents the resultant interference profile. Simulation results show that the Q MRTC can achieve higher system throughput and better users' satisfaction index, by an amount of 87% and 50%, respectively, than the interference-based multi-rate transmission control scheme, while keeping the QoS requirement.