MARKOV RECURRENT NEURAL NETWORK LANGUAGE MODEL

檔案

本條目沒有附加檔案。

日期

2019-01-01

期刊名稱

2019 IEEE AUTOMATIC SPEECH RECOGNITION AND UNDERSTANDING WORKSHOP (ASRU 2019)

卷期與頁次

頁 807–813

ISBN

978-1-7281-0306-8

語言

英文

摘要

Recurrent neural network (RNN) has achieved a great success in language modeling where the temporal information based on deterministic state is continuously extracted and evolved through time. Such a simple deterministic transition function using input-to-hidden and hidden-to-hidden weights is usually insufficient to reflect the diversities and variations of latent variable structure behind the heterogeneous natural language. This paper presents a new stochastic Markov RNN (MRNN) to strengthen the learning capability in language model where the trajectory of word sequences is driven by a neural Markov process with Markov state transitions based on a K-state long short-term memory model. A latent state machine is constructed to characterize the complicated semantics in the structured lexical patterns. Gumbel-softmax is introduced to implement the stochastic backpropatation algorithm with discrete states. The parallel computation for rapid realization of MRNN is presented. The variational Bayesian learning procedure is implemented. Experiments demonstrate the merits of stochastic and diverse representation using MRNN language model where the overhead of parameters and computations is limited.

所屬典藏

會議論文
顯示完整詮釋資料 統計資料

引用此條目

Chien, J.-T., & Kuo, C.-Y. (2019). MARKOV RECURRENT NEURAL NETWORK LANGUAGE MODEL. 2019 IEEE AUTOMATIC SPEECH RECOGNITION AND UNDERSTANDING WORKSHOP (ASRU 2019), 807–813. https://ir.lib.nycu.edu.tw/handle/11536/154482

BibTeX RIS

引用文字依 APA 第 7 版格式自動產生,使用前請再核對。

分享

永久網址:https://ir.lib.nycu.edu.tw/handle/11536/154482