<link rel="stylesheet" href="styles.cf390c6bfbe7555f.css">

學術出版;;Publications

Permanent URI for this communityhdl:11536/1

收集所有交通大學研究人員與學生個人或團體所發表之各項學術著作，涵蓋期刊論文、教師專書、學位論文等項。;;Collections of Publications of NCTU Researchers and Students Like Journal Papers and Thesis.

Browse

Filters

20162

Settings

🏛️國立陽明交通大學機構典藏

Author: search.filters.author.Huang, Pei-WenSubject: search.filters.subject.speech recognition

Search Results

Now showing 1 - 2 of 2
  • Thumbnail Image
    Item type:Item,
    Hybrid Accelerated Optimization for Speech Recognition
    (2016-01-01)
    Chien, Jen-Tzung
    ;
    Huang, Pei-Wen
    ;
    Lee, Tan
    ;
    電機工程學系
    ;
    Department of Electrical and Computer Engineering
    Optimization procedure is crucial to achieve desirable performance for speech recognition based on deep neural networks (DNNs). Conventionally, DNNs are trained by using mini-batch stochastic gradient descent (SGD) which is stable but prone to be trapped into local optimum. A recent work based on Nesterov's accelerated gradient descent (NAG) algorithm is developed by merging the current momentum information into correction of SGD updating. NAG less likely jumps into local minimum so that convergence rate is improved. In general, optimization based on SGD is more stable while that based on NAG is faster and more accurate. This study aims to boost the performance of speech recognition by combining complimentary SGD and NAG. A new hybrid optimization is proposed by integrating the SGD with momentum and the NAG by using an interpolation scheme which is continuously run in each mini-batch according to the change rate of cost function in consecutive two learning epochs. Tradeoff between two algorithms can be balanced for mini-batch optimization. Experiments on speech recognition using CUSENT and Aurora-4 show the effectiveness of the hybrid accelerated optimization in DNN acoustic model.
  • Thumbnail Image
    Item type:Item,
    VARIANCE REDUCTION FOR OPTIMIZATION IN SPEECH RECOGNITION
    (2016)
    Chien, Jen-Tzung
    ;
    Huang, Pei-Wen
    ;
    電機工程學系
    ;
    Department of Electrical and Computer Engineering
    Deep neural network (DNN) is trained according to a mini-batch optimization based on the stochastic gradient descent algorithm. Such a stochastic learning suffers from instability in parameter updating and may easily trap into local optimum. This study deals with the stability of stochastic learning by reducing the variance of gradients in optimization procedure. We upgrade the optimization from the stochastic dual coordinated ascent (SDCA) to the accelerated SDCA without duality (or dual-free ASDCA). This optimization incorporates the momentum method to accelerate the updating rule where the variance of gradients can be reduced. Using dual-free ASDCA, the optimization of dual function of SDCA in a form of convex loss is implemented by directly optimizing the primal function with respect to pseudo-dual parameters. The non-convex optimization in DNN training can be resolved and accelerated. Experimental results illustrate the reduction of training loss, variance of gradients and word error rate by using the proposed optimization for DNN speech recognition.