分割且克服之學習與模組化感知器神經網路之研究

Abstract

本篇論文提出了一個以模組化感知器網路（Modular Perceptron Network (MPN)）為基礎，整合分割與克服學習（Divide-and-Conquer Learning (DCL)）的方法與權值估測（Weight Estimation）技巧的模組化類神經網路架構之創新設計。最後再將感知器網路進行維度縮減（Dimensionality Reduction），在解決維度障礙（Curse of Dimensionality）有顯著效益。 當一個 multilayer perceptron 在訓練過程中，如果陷入到局部極小值區域或是平坦區域，則無法獲得一良好的學習性能，而本論文所建議的Divide-and-Conquer Learning (DCL)方法可被應用於劃分目前較難學習的訓練樣本區成為兩個較易學習的樣本區，可化解局部極小值區域或是平坦區域的困境。 當產生其中一個新的學習樣本區時 self-growing perceptron network 及其初始估測權值將被建構。 另外一個的樣本區將在原有的神經網路上重新學習。整個學習樣本區重複的進行學習與權值估測，直到MPN可以完全學習全部的訓練樣本為止。 本篇論文所建議的MPN已經利用大家所熟悉的Two-spirals problem及另選三組廣為人知實際的紀錄資料進行評估與比較。 在Two-spirals problem的訓練階段，MPN配以權值估測可利用非常少的data presentations (相對其他方法減少99.01%~ 87.86%) 達成較佳的學習性能、較佳的普遍性(相對其他方法高4.0%)及在retrieving階段花費較少的處理時間(相對其他方法減少2.0% ~81.3% )。在三組實際的紀錄資料訓練過程顯示，這個 MPNs架構再配合權值估算（Weight Estimation）與單一的MLP比較，除上述的優點外其過度學習（overfitting）現象可以減少或避免。 另外， 在解決高維度輸入資料在學習過程中所產生的維度障礙（Curse of Dimensionality），本文亦建立一評估演算法，以克服此一問題，由試驗結果顯示輸入資料維度可大幅降低（減少75%~88%），data presentations亦有明顯下降（約減少70%~95%），如此可獲得一小型化的學習網路，但是學習及測試性能仍能維持原有水準。由於此架構具有自我增生及快速局部學習之特質，所以此種類神經網路模組（MPN）對於一個快速改變的環境下很容易適用於 on-line and/or incremental learning的需求。

A novel Modular Perceptron Network (MPN) and Divide-and-Conquer Learning (DCL) schemes with Weight Estimation for the design of modular neural networks are proposed. Finally the perceptron networks is performed on Dimensionality reduction with obvious effectiveness for solution in Curse of Dimensionality. When a training process in a multilayer perceptron falls into a local minimum or stalls in a flat region, the proposed DCL scheme is applied to divide the current training data region (e.g., a hard to be learned training set) into two easier (hopely) to be learned regions. The learning process continues when a self-growing perceptron network and its initial weight estimation are constructed for one of the newly partitioned regions. Another partitioned region will resume the training process on the original perceptron network. Data region partitioning, weight estimating and learning are iteratively repeated until all the training data are completely learned by the MPN. We have evaluated and compared the proposed MPN with several representative neural networks on the Two-spirals problem and real-world database. On learning the the Two-spirals problem, the MPN achieves better weight learning performance by requiring much less data presentations ( less 99.01%~ 87.86%) during the network training phases, and better generalization performance (4.0% better), and less processing time (less 2.0% ~ 81.3%) during the retrieving phase. On learning the real-world data, the MPNs show the same performance as above problem learning results and less overfitting compared to single MLP. Otherwise, for solution in curse of dimensionality, induced from high dimensional input data during learning process. An evaluation algorithm is established in this paper to overcome this problem. It shows that from the experimental results the dimension of the input data can be largely decreased (less 75% ~ 88%) and the data presentations are also reduced (less 70% ~ 90%). So a small size MPNs can be procured with learning and testing performance maintained as the good level as before. In addition, due to its self-growing and fast local learning characteristics, the modular network MPN can easily adapt to on-line and/or incremental learning requirements for a rapid changing environment.