Reinforcement learning for an ART-based fuzzy adaptive learning control network

Abstract

This paper proposes a reinforcement fuzzy adaptive learning control network (RFALCON) for solving various reinforcement learning problems, The proposed RFALCON is constructed by integrating two fuzzy adaptive learning control networks (FALCON's), each of which is a connectionist model with a feedforward multilayer network developed for the realization of a fuzzy controller, One FALCON performs as a critic network (fuzzy predictor), and the other as an action network (fuzzy controller), Using the temporal difference prediction method, the critic network can predict the external reinforcement signal and provide a more informative internal reinforcement signal to the action network, The action network performs a stochastic exploratory algorithm to adapt itself according to the internal reinforcement signal, An ART-based reinforcement structure/parameter-learning algorithm is developed for constructing the RFALCON dynamically, During the learning process, both structure learning and parameter learning are performed simultaneously in the two FALCON's, The proposed RFALCON can construct a fuzzy control system dynamically and automatically through a reward/penalty signal (i.e., a ''good'' or ''bad'' signal), It is best applied to the learning environment, where obtaining exact training data is expensive, The proposed RFALCON has two important features, First, it reduces the combinatorial demands placed by the standard methods for adaptive Linearization of a system, Second, the RFALCON is a highly autonomous system, Initially, there are no hidden nodes (i.e., no membership functions or Fuzzy rules), They are created and begin to grow as learning proceeds, The RFALCON can also dynamically partition the input-output spaces, tune activation (membership) functions, and find proper network connection types (fuzzy rules), Computer simulations have been conducted to illustrate the performance and applicability of the proposed learning scheme.