An on-line ICA-mixture-model-based self-constructing fuzzy neural network

Abstract

This paper proposes a new fuzzy neural network (FNN) capable of parameter self-adapting and structure self-constructing to acquire a small number of fuzzy rules for interpreting the embedded knowledge of a system from the given training data set. The proposed FNN is inherently a modified Takagi-Sugeno-Kang (TSK)-type fuzzy-rule-based model with neural network's learning ability. There are no rules initiated at the beginning and they are created and adapted through an on-line learning processing that performs simultaneous structure and parameter identification. In the structure identification of the precondition part, the input space is partitioned in a flexible way according to newly proposed on-line independent component analysis (ICA) mixture model. The input space is thus represented by linear combinations of independent, non-Gaussian densities. The first input training pattern is assigned to the first rule initially by the on-line ICA mixture model. Afterwards, some additional significant terms (input variables) selected by the on-line ICA mixture model will be added to the consequent part (forming a liner equation of input variables) incrementally or create a new rule in the learning processing. The combined precondition and consequent structure identification scheme can make the network grow dynamically and efficiently. In the parameter identification, the consequent parameters are tuned by the backpropagation rule and the precondition parameters are turned by the on-line ICA mixture model. Both the structure and parameter identifications are done simultaneously to form a fast learning scheme. The derived on-line ICA mixture model also provide a natural linear transformation for each input variable to enhance the knowledge representation ability of the proposed FNN and reduce the required rules and achieve higher accuracy efficiently. in order to demonstrate the performance of the proposed FNN, several experiments covering the areas of system identification, classification, and image segmentation are carried out. Our experiments show that the proposed FNN can achieve significant improvements in the convergence speed and prediction accuracy with simpler network structure.