具演化式結構學習能力之類神經網路及其預測之應用

Abstract

本論文提出一以前饋式類神經網路輔助之灰色模型及其相關的線上參數學習與結構學習演算法，此模型採用一階單變數灰色模型來預測訊號，再使用前饋式類神經網路補償灰色模型的預測誤差，此外，本論文提出一線上批次訓練法來即時更新類神經網路的權重值，於是，此模型可以執行預測且持續地適應動態的訊號變化。為了有效地設計此模型的結構，本論文提出一種以神經元為基礎的結構學習，稱為共生結構學習演算法，來建立類神經網路的拓墣結構，此演算法首先建構一神經元族群，再由神經元族群建立類神經網路族群，由於神經元族群裡的神經元包含雙曲線正切與線性活化函數，此演算法能任意且輕易地發展串聯式網路與前饋式網路，此演算法進一步根據共生進化的概念，在神經元族群裡執行神經元交配與突變，其所發展的前饋式類神經網路輔助之灰色模型將執行訊號預測且持續地以線上批次訓練法調適模型於環境中。另一方面，本論文提出一種以網路為基礎的結構學習，稱為演化式建構與修剪演算法，用演化的方式結合建構與修剪的概念，來設計類神經網路的拓墣結構。此演算法從一群具有最簡單結構的類神經網路開始，即一群只有一顆連接單一輸入單元的隱藏層神經元的類神經網路，此演算法採用網路交配與突變來增加隱藏層神經元以及鏈結，用以提升類神經網路的訊號處理能力，此外，本論文提出一以叢集為基礎之修剪法用隨機的方式來除去不重要的神經元，也提出一以年齡為基礎之生存者選擇法來移除較老且可能具有複雜結構的類神經網路，接著引進新的且具有最簡單結構的類神經網路。數值模擬與實驗結果將展現所提出的方法在預測問題上的有效及可行性。

This dissertation proposes a feedforward-neural-network-aided grey model (FNAGM) and its related on-line parameter learning and structure learning algorithms. The FNAGM uses a first-order single variable grey model (GM(1,1)) to predict signal and adopts a feedforward neural network (NN) to compensate the prediction error of GM(1,1). Furthermore, an on-line batch training is proposed to update the weights of NN in real-time. Thus, FNAGM can precisely predict and adapt itself to the dynamical change of the signal. To design the structure of FNAGM efficiently, a neuron-based structure learning, called symbiotic structure learning algorithm (SSLA), is proposed to establish the topology of NN. The SSLA constructs a neuron population and then builds a network population from the neuron population, and it can arbitrarily develop cascade NNs and feedforward NNs in an easy way. Further, SSLA carries out neuron crossover and mutation on the neuron population according to the idea of symbiotic evolution. The evolved FNAGM is applied to predict the signal and continuously adapt itself to the environment by the on-line batch training. On the other hand, a network-based structure learning, called evolutionary constructive and pruning algorithm (ECPA), is proposed to design the topology of NN by incorporating constructive and pruning methods in an evolutionary way. The ECPA starts from a set of NNs with the simplest possible structures, one hidden neuron connected to an input node. It then adds hidden neurons and connections by using the network crossover and mutation to increase the processing capabilities of NNs. Furthermore, a cluster-based pruning is proposed to prune insignificant neurons in a stochastic way. An age-based survival selection is proposed to delete old NNs with potentially complex structures and then introduce new NNs with the simplest possible structures. Numerical and experimental results of prediction problems show the effectiveness and feasibility of the proposed methods.