Bayesian Recurrent Neural Network for Language Modeling
顯示簡易詮釋資料 統計資料| 欄位 | 內容 | 語言 |
|---|---|---|
dc.citation.epage |
374 | en_US |
dc.citation.issue |
2 | en_US |
dc.citation.spage |
361 | en_US |
dc.citation.volume |
27 | en_US |
dc.contributor.author |
Chien, Jen-Tzung | en_US |
dc.contributor.author |
Ku, Yuan-Chu | en_US |
dc.contributor.department |
電機學院 | zh_TW |
dc.contributor.department |
College of Electrical and Computer Engineering | en_US |
dc.date.accessioned |
2017-04-21T06:56:03Z | |
dc.date.available |
2017-04-21T06:56:03Z | |
dc.date.issued |
2016-02 | en_US |
dc.description.abstract |
A language model (LM) is calculated as the probability of a word sequence that provides the solution to word prediction for a variety of information systems. A recurrent neural network (RNN) is powerful to learn the large-span dynamics of a word sequence in the continuous space. However, the training of the RNN-LM is an ill-posed problem because of too many parameters from a large dictionary size and a high-dimensional hidden layer. This paper presents a Bayesian approach to regularize the RNN-LM and apply it for continuous speech recognition. We aim to penalize the too complicated RNN-LM by compensating for the uncertainty of the estimated model parameters, which is represented by a Gaussian prior. The objective function in a Bayesian classification network is formed as the regularized cross-entropy error function. The regularized model is constructed not only by calculating the regularized parameters according to the maximum a posteriori criterion but also by estimating the Gaussian hyperparameter by maximizing the marginal likelihood. A rapid approximation to a Hessian matrix is developed to implement the Bayesian RNN-LM (BRNN-LM) by selecting a small set of salient outer-products. The proposed BRNN-LM achieves a sparser model than the RNN-LM. Experiments on different corpora show the robustness of system performance by applying the rapid BRNN-LM under different conditions. | en_US |
dc.identifier.doi |
10.1109/TNNLS.2015.2499302 | en_US |
dc.identifier.issn |
2162-237X | en_US |
dc.identifier.journal |
IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS | en_US |
dc.identifier.uri |
http://dx.doi.org/10.1109/TNNLS.2015.2499302 | en_US |
dc.identifier.uri |
https://ir.lib.nycu.edu.tw/handle/11536/133534 | |
dc.identifier.wosnumber |
WOS:000372020500014 | en_US |
dc.language.iso |
en_US | en_US |
dc.subject |
Bayesian learning | en_US |
dc.subject |
Hessian matrix | en_US |
dc.subject |
language model | en_US |
dc.subject |
rapid approximation | en_US |
dc.subject |
recurrent neural network | en_US |
dc.title |
Bayesian Recurrent Neural Network for Language Modeling | en_US |
檔案
本條目沒有附加檔案。
所屬典藏: 期刊論文
引用此條目
Chien, J.-T., & Ku, Y.-C. (2016). Bayesian Recurrent Neural Network for Language Modeling. IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS, 27(2), 361–374. https://doi.org/10.1109/TNNLS.2015.2499302
引用文字依 APA 第 7 版格式自動產生,使用前請再核對。
分享
永久網址:https://ir.lib.nycu.edu.tw/handle/11536/133534