Quadratic optimization method for multilayer neural networks with local error-backpropagation

Abstract

A fast new local error-backpropagation (LBP) algorithm is presented for the training of multilayer neural networks. This algorithm is based on the definition of a new local mean-squared error function. If the local desired outputs have been estimated, the multilayer neural networks can be decomposed into a set of adaptive linear elements (Adaline) that can be trained by quadratic optimization methods. Among a lot of quadratic optimization methods, the conjugate gradient (CG) method is one of the most famous methods that can find the global optimal solution of quadratic problems within finite steps. The iteration number and the computation time are significantly reduced because the stepsize is computed without line-search. Experimental results on the pattern recognition and memorizing of spatiotemporal patterns are provided.