Browsing by Author "Lu, Tsai-Wei"

Now showing 1 - 3 of 3

DEEP RECURRENT REGULARIZATION NEURAL NETWORK FOR SPEECH RECOGNITION
(2015-01-01)
Chien, Jen-Tzung
;
Lu, Tsai-Wei
;
電機工程學系
;
Department of Electrical and Computer Engineering
This paper presents a deep recurrent regularization neural network (DRRNN) for speech recognition. Our idea is to build a regularization neural network acoustic model by conducting the hybrid Tikhonov and weight-decay regularization which compensates the variations due to the input speech as well as the model parameters in the restricted Boltzmann machine as a pre-training stage for feature learning and structural modeling. In addition, a new backpropagation through time (BPTT) algorithm is developed by extending the truncated minibatch training for recurrent neural network where the minibatch BPTT is not only performed in recurrent layer but also in feedforward layer. The DRRNN acoustic model is accordingly established to capture the temporal correlation in a regularization neural network. Experimental results on the tasks of RM and Aurora4 show the effectiveness and robustness of using DRRNN for speech recognition.
DEEP RECURRENT REGULARIZATION NEURAL NETWORK FOR SPEECH RECOGNITION
(2015)
Chien, Jen-Tzung
;
Lu, Tsai-Wei
;
電機學院
;
College of Electrical and Computer Engineering
This paper presents a deep recurrent regularization neural network (DRRNN) for speech recognition. Our idea is to build a regularization neural network acoustic model by conducting the hybrid Tikhonov and weight-decay regularization which compensates the variations due to the input speech as well as the model parameters in the restricted Boltzmann machine as a pre-training stage for feature learning and structural modeling. In addition, a new backpropagation through time (BPTT) algorithm is developed by extending the truncated minibatch training for recurrent neural network where the minibatch BPTT is not only performed in recurrent layer but also in feedforward layer. The DRRNN acoustic model is accordingly established to capture the temporal correlation in a regularization neural network. Experimental results on the tasks of RM and Aurora4 show the effectiveness and robustness of using DRRNN for speech recognition.
TIKHONOV REGULARIZATION FOR DEEP NEURAL NETWORK ACOUSTIC MODELING
(2014)
Chien, Jen-Tzung
;
Lu, Tsai-Wei
;
電機學院
;
College of Electrical and Computer Engineering
Deep neural network (DNN) has been widely demonstrated to achieve high performance in different speech recognition tasks. This paper focuses on the issue of model regularization in DNN acoustic model. Our idea is to compensate for the perturbations over training samples in the restricted Boltzmann machine (RBM) which is applied as a pre-training stage for unsupervised feature learning and structural modeling. We introduce the Tikhonov regularization in pre-training procedure and pursue the invariance property of objective function over the variations in input samples. This Tikhonov regularization is further combined with the regularization based on weight decay. The error function in supervised cross-entropy training is accordingly reduced. Experimental results on using RM and Aurora4 tasks show that hybrid regularization in RBM pre-training improves the training condition in DNN acoustic model and the robustness in speech recognition performance.