加強式類神經模糊推論網路及其應用

Abstract

本論文主要目的在針對加強式學習的問題，提出一類神經模糊推論網路之架構及其相關演算法來實現一傳統模糊邏輯控制器。而在探討加強式學習的問題前，我們需先建立一適當的類神經模糊推論網路。因此，一開始我們提出一五層連接網路，以利於結合傳統模糊邏輯控制器之基本元件及函數至此網路架構中。這裡若外界可提供專家知識(模糊法則)情況下，我們可輕易整合專家知識於此網路架構中。此外，配合著網路架構以倒傳遞演算法所推導出的參數學習方式，可進一步用於網路參數的調整，將網路效能調至最佳情況。 另外，基於原先的五層連接網路架構，我們進一步提出模糊適應性學習控制網路(FALCON)。FALCON以一具有線上學習能力的兩步驟演算法(FALCON-ART)來動態建構網路。與原先的五層網路比較，除了亦是用倒傳遞演算法來完成網路的參數學習外，其利用模糊適應性共振理論(fuzzy ART)演算法來完成架構學習。此FALCON-ART學習演算法具有自動線上分割輸入及輸出空間，調整歸屬函數，及找到適當的模糊邏輯法則。故在學習過程中，僅需由外界給予訓練資料即可完成架構及參數學習，而不需要任何事先的知識。此學習演算法又能將輸入及輸出空間分割成不規則狀，因此避免在複雜系統中因維數的增加，導致棋盤狀的分割數目亦隨之遽增。 當類神經模糊推論網路之架構與學習方式發展完整後，我們延伸上述所提出的網路架構於加強式學習的問題探討：(1)將基因演算法的原理帶入FALCON的架構學習中。由於基因演算法為一搜尋演算法，其不需使用導函數資訊，故可視為一加強式學習方式。另外因其具有全域最佳化能力，而已變成另一有用的工具來自動設計模糊控制系統。這裡一改良的架構及參數學習演算法 (FALCON-GA) 被提出來自動構成 FALCON，此學習方法是一具三階段混合學習演算法，除了是利用基因演算法來找到適當的模糊邏輯法則外，其餘皆與原來FALCON學習方式相同。經一些實驗結果比較，我們發現FALCON-GA在各種效能上均較原先的FALCON好。(2)提出具有加強式學習能力之類神經模糊結合器(Neuro-Fuzzy Combiner, NFC)，以解決多目標控制的問題。這裡NFC的主要元件為FALCON，不過為了解決多目標控制的複雜特性及外界資料獲得不易的問題，我們加入階層式控制及加強式學習兩種概念於網路架構中。再詳細來看，此NFC以一階層式方法結合n個已存在的低階控制器，完成一多目標模糊控制器。每個已存在的低階控制器是用來控制個別特殊的目標，而NFC扮演著融合n個低階控制器所產生動作，並在每個時刻決定一適當的動作於受控物體。因此，NFC能結合一群低階控制器而同時達到多重目標控制的效果。 以上針對所提出的加強式類神經模糊推論網路，其效能均已將透過電腦模擬來驗證和比較。其中，FALCON-GA已實現在渾沌時間序列預測及控制倒車入庫兩個問題上。至於應用在多目標控制問題，我們已實現在倒單擺系統及吊車系統上。這些應用均驗證了所提方法的效率與能力。

In this thesis, aiming at the problem of reinforcement learning, we propose the structure and associated learning algorithm of a neural fuzzy inference network for realizing the basic elements and functions of a traditional fuzzy logic controller. However, before we discuss the problem of reinforcement learning, we must construct a proper neural fuzzy inference network previously. Hence, at the beginning, we propose a basic five-layered connectionist network which could easily integrate the basic elements and functions of a traditional fuzzy logic controller into a connectionist structure. If expert knowledge (or fuzzy rules) is provided from the outside world, here we could easily integrate expert knowledge into a network structure. Additionally, we derive the parameter learning method according to the network structure and backpropagation learning scheme. The derived parameter learning method can be further used to adjust the network parameters for obtaining the best performance. In addition, based upon the structure of the original five-layered connectionist network, we further propose a Fuzzy Adaptive Learning COntrol Network (FALCON). FALCON uses an on-line two-step learning algorithm, called FALCON-ART, for constructing the network structure dynamically. Compared with the original five-layered connectionist network, FALCON uses the fuzzy ART algorithm for structure learning in addition to the backpropagation learning scheme for parameter learning. The FALCON-ART can partition the input/output spaces on-line, tune membership functions and find proper fuzzy logic rules. All things are done automatically and dynamically. More notably, in this learning method, only the training data need to be provided from the outside world. That is, the users need not give the initial fuzzy partitions, membership functions and fuzzy logic rules. Hence, there is no input/output term nodes and no rule nodes in the beginning of learning. The FALCON-ART partitions the pattern space into irregular hyperboxes and thus can avoid the problem of combinatorial growing of partitioned grids in some complex systems. When the development of the structure and associated learning algorithm of a neural fuzzy inference network is finished, we extend the above network structure to the area of reinforcement learning: (1) We bring the genetic algorithms (GAs) into the structure learning of FALCON. GAs belong to a kind of search algorithms. Since GAs do not require or use derivative information, the most appropriate applications are problems where gradient information is unavailable or costly to obtain. Reinforcement learning is just one example of such a domain. Therefore, we can regard GAs as a kind of reinforcement learning. In addition, due to its global optimization capability, GAs have become another useful tool to the automatic design of fuzzy control systems. Here an improved structure/parameter learning algorithm, called FALCON-GA, is proposed for constructing the FALCON automatically. The FALCON-GA is a three-phase hybrid learning algorithm. Except for using GAs to find proper fuzzy logic rules, its learning algorithms, for partitioning input/output spaces and tuning membership functions, are the same as FALCON. By comparing the simulated results, we find that the performance of FALCON-GA is better than that of FALCON. (2) We propose a reinforcement Neuro-Fuzzy Combiner (NFC) for multiobjective control. Here the key component of NFC could be still FALCON. However, for solving the problems of complex multiobjective control and no instructive teaching information available, we add the concepts of the hierarchical control and the reinforcement learning into NFC structure. In more detail, the structure of the multiobjective control system is composed of the NFC and n existing low-level controllers. It is assumed that each low-level (fuzzy or nonfuzzy) controller has been well designed to serve a particular objective. The role of the NFC is to fuse the $n$ actions decided by the n low-level controllers and determine a proper action through reinforcement learning method to act on the environment (plant) at each time step. Hence, the NFC can combine low-level controllers and achieve multiple objectives (goals) at once. Capabilities and performances of the proposed reinforcement neural fuzzy inference network have been verified and compared through various computer simulations. We have used FALCON-GA in solving the problems of the chaotic time-series prediction and the control of the truck backer-upper. In the application of multiobjective control, we have also realized in a cart-pole balancing system and a crane system. Capabilities and performances of the proposed methods are all verified from these applications. Abstract in English Chapter 1: Introduction Chapter 2: A Basic Five-layered Neural Fuzzy Inference Network Chapter 3: A Fuzzy Adaptive Learning Control Network Chapter 4: A GA-Based Fuzzy Adaptive Learning Control Network Chapter 5: A Reinforcement Neuro-Fuzzy Combiner for Multiobjective Control Chapter 6: Conclusion