Dynamic optimal training of a three layer neural network with sigmoid function

Abstract

This paper proposes a dynamical optimal training algorithm for a three layer neural network (NN) with sigmoid activation functions in the hidden and output layers. This three layer neural network can be used for classification problems, such as the classification of Iris data. The mathematical formulation of this three layer NN is rigorously derived first in this paper, so that the dynamical optimal training of it can be performed. The dynamical optimal training process for this three layer NN is therefore presented which guarantees the convergence of the training in a minimum number of epochs. This dynamical optimal training does not use fixed learning rate for training. Instead, the learning rates are updated for next iteration to guarantee the optimal convergence of the training result. Excellent results have been obtained for XOR and Iris data set.