Rapid Bayesian Learning for Recurrent Neural Network Language Model
顯示簡易詮釋資料 統計資料| 欄位 | 內容 | 語言 |
|---|---|---|
dc.citation.epage |
38 | en_US |
dc.citation.spage |
34 | en_US |
dc.citation.woscount |
0 | en_US |
dc.contributor.author |
Chien, Jen-Tzung | en_US |
dc.contributor.author |
Ku, Yuan-Chu | en_US |
dc.contributor.author |
Huang, Mou-Yue | en_US |
dc.contributor.department |
電機資訊學士班 | zh_TW |
dc.contributor.department |
Undergraduate Honors Program of Electrical Engineering and Computer Science | en_US |
dc.date.accessioned |
2015-07-21T08:31:27Z | |
dc.date.available |
2015-07-21T08:31:27Z | |
dc.date.issued |
2014-01-01 | en_US |
dc.description.abstract |
This paper presents Bayesian learning for recurrent neural network language model (RNN-LM). Our goal is to regularize the RNN-LM by compensating for the randomness of the estimated model parameters which is characterized by a Gaussian prior. This model is not only constructed by training the synaptic weight parameters according to the maximum a posteriori criterion but also regularized by estimating the Gaussian hyperparameter through the type 2 maximum likelihood. However, a critical issue in Bayesian RNN-LM is the heavy computation of Hessian matrix which is formed as the sum of a large amount of outer-products of high-dimensional gradient vectors. We present a rapid approximation to reduce the redundancy due to the curse of dimensionality and speed up the calculation by summing up only the salient outer-products. Experiments on 1B-Word Benchmark, Penn Treebank and World Street Journal corpora show that rapid Bayesian RNN-LM consistently improves the perplexity and word error rate in comparison with standard RNN-LM. | en_US |
dc.identifier.isbn |
978-1-4799-4219-0 | en_US |
dc.identifier.issn |
en_US | |
dc.identifier.journal |
2014 9TH INTERNATIONAL SYMPOSIUM ON CHINESE SPOKEN LANGUAGE PROCESSING (ISCSLP) | en_US |
dc.identifier.uri |
https://ir.lib.nycu.edu.tw/handle/11536/125004 | |
dc.identifier.wosnumber |
WOS:000349765600008 | en_US |
dc.language.iso |
en_US | en_US |
dc.subject |
Hessian matrix | en_US |
dc.subject |
Bayesian learning | en_US |
dc.subject |
Recurrent neural network language model | en_US |
dc.subject |
speech recognition | en_US |
dc.title |
Rapid Bayesian Learning for Recurrent Neural Network Language Model | en_US |
dc.type |
Proceedings Paper | en_US |
檔案
本條目沒有附加檔案。
所屬典藏: 會議論文
引用此條目
Chien, J.-T., Ku, Y.-C., & Huang, M.-Y. (2014). Rapid Bayesian Learning for Recurrent Neural Network Language Model. 2014 9TH INTERNATIONAL SYMPOSIUM ON CHINESE SPOKEN LANGUAGE PROCESSING (ISCSLP), 34–38. https://ir.lib.nycu.edu.tw/handle/11536/125004
引用文字依 APA 第 7 版格式自動產生,使用前請再核對。
分享
永久網址:https://ir.lib.nycu.edu.tw/handle/11536/125004