學術出版;;Publications

Permanent URI for this communityhdl:11536/1

收集所有交通大學研究人員與學生個人或團體所發表之各項學術著作，涵蓋期刊論文、教師專書、學位論文等項。;;Collections of Publications of NCTU Researchers and Students Like Journal Papers and Thesis.

Browse

🏛️國立陽明交通大學機構典藏

Search Results

Now showing 1 - 10 of 19

A Proximity Sensor Based No-Touch Mechanism for Mobile Applications on Smart Phones
(2012-01-01)
Lin, Chia-Yu
;
Chen, Yu-Jia
;
Wang, Li-Chun
;
Tseng, Yu-Chee
;
電機工程學系
Smart phones with touch screens have become very popular and have changed our behaviors of using handsets. However, using touch screens is not safe for mobile phone users especially when they are driving cars. Thus, many applications using smart phones cannot be initiated because users who are driving the cars cannot easily touch the small icons on the screens of smart phones. To overcome this issue, we propose a proximity sensors based "no-touch" mechanism for smart phones by applying proximity sensors to initiate mobile applications without the need of touching the screen. We will discuss how to implement the proximity-sensors driven "no-touch" mechanism in Android platform and investigate its performance issues regarding detection accuracy and power consumption. Speech-oriented applications using the proposed "no-touch" mechanism on Android is also demonstrated in this paper.
Deep Neural Factorization for Speech Recognition
(2017-01-01)
Chien, Jen-Tzung
;
Shen, Chen
;
電機工程學系
;
Department of Electrical and Computer Engineering
Conventional speech recognition system is constructed by unfolding the spectral-temporal input matrices into one-way vectors and using these vectors to estimate the affine parameters of neural network according to the vector-based error backpropagation algorithm. System performance is constrained because the contextual correlations in frequency and time horizons are disregarded and the spectral and temporal factors are excluded. This paper proposes a spectral-temporal factorized neural network (STFNN) to tackle this weakness. The spectral-temporal structure is preserved and factorized in hidden layers through two ways of factor matrices which are trained by using the factorized error backpropagation. Affine transformation in standard neural network is generalized to the spectro-temporal factorization in STFNN. The structural features or patterns are extracted and forwarded towards the softmax outputs. A deep neural factorization is built by cascading a number of factorization layers with fully-connected layers for speech recognition. An orthogonal constraint is imposed in factor matrices for redundancy reduction. Experimental results show the merit of integrating the factorized features in deep feedforward and recurrent neural networks for speech recognition.
Stochastic Recurrent Neural Network for Speech Recognition
(2017-01-01)
Chien, Jen-Tzung
;
Shen, Chen
;
電機工程學系
;
Department of Electrical and Computer Engineering
This paper presents a new stochastic learning approach to construct a latent variable model for recurrent neural network (RNN) based speech recognition. A hybrid generative and discriminative stochastic network is implemented to build a deep classification model. In the implementation, we conduct stochastic modeling for hidden states of recurrent neural network based on the variational auto-encoder. The randomness of hidden neurons is represented by the Gaussian distribution with mean and variance parameters driven by neural weights and learned from variational inference. Importantly, the class labels of input speech frames are incorporated to regularize this deep model to sample the informative and discriminative features for reconstruction of classification outputs. We accordingly propose the stochastic RNN (SRNN) to reflect the probabilistic property in RNN classification system. A stochastic error backpropagation algorithm is implemented. The experiments on speech recognition using TIMIT and Aurora4 show the merit of the proposed SRNN.
Hybrid Accelerated Optimization for Speech Recognition
(2016-01-01)
Chien, Jen-Tzung
;
Huang, Pei-Wen
;
Lee, Tan
;
電機工程學系
;
Department of Electrical and Computer Engineering
Optimization procedure is crucial to achieve desirable performance for speech recognition based on deep neural networks (DNNs). Conventionally, DNNs are trained by using mini-batch stochastic gradient descent (SGD) which is stable but prone to be trapped into local optimum. A recent work based on Nesterov's accelerated gradient descent (NAG) algorithm is developed by merging the current momentum information into correction of SGD updating. NAG less likely jumps into local minimum so that convergence rate is improved. In general, optimization based on SGD is more stable while that based on NAG is faster and more accurate. This study aims to boost the performance of speech recognition by combining complimentary SGD and NAG. A new hybrid optimization is proposed by integrating the SGD with momentum and the NAG by using an interpolation scheme which is continuously run in each mini-batch according to the change rate of cost function in consecutive two learning epochs. Tradeoff between two algorithms can be balanced for mini-batch optimization. Experiments on speech recognition using CUSENT and Aurora-4 show the effectiveness of the hybrid accelerated optimization in DNN acoustic model.
Deep Long Short-Term Memory Networks for Speech Recognition
(2016-01-01)
Chien, Jen-Tzung
;
Misbullah, Alim
;
電機工程學系
;
Department of Electrical and Computer Engineering
Speech recognition has been significantly improved by applying acoustic models based on deep neural network which could be realized as the feedforward NN (FNN) or the recurrent NN (RNN). In general, FNN is feasible to project the observations onto a deep invariant feature space while RNN is beneficial to capture the temporal information in a sequential data for speech recognition. RNN based on long short-term memory (LSTM) is capable of storing inputs over a long time period and thus exploiting a self-learned mechanism for long-range temporal context. Considering the complimentary FNN and RNN in their modeling capabilities, this paper presents a deep model which is constructed by stacking LSTM and FNN. Through the cascade of LSTM cells and fully-connected feedforward units, we explore the temporal patterns and summarize the long history of previous inputs in a deep learning machine. The experiments on 3rd CHiME challenge and Aurora-4 show that the stacks of hybrid model with FNN post-processor outperform stand-alone FNN and LSTM and the other hybrid models for robust speech recognition.
VARIANCE REDUCTION FOR OPTIMIZATION IN SPEECH RECOGNITION
(2016)
Chien, Jen-Tzung
;
Huang, Pei-Wen
;
電機工程學系
;
Department of Electrical and Computer Engineering
Deep neural network (DNN) is trained according to a mini-batch optimization based on the stochastic gradient descent algorithm. Such a stochastic learning suffers from instability in parameter updating and may easily trap into local optimum. This study deals with the stability of stochastic learning by reducing the variance of gradients in optimization procedure. We upgrade the optimization from the stochastic dual coordinated ascent (SDCA) to the accelerated SDCA without duality (or dual-free ASDCA). This optimization incorporates the momentum method to accelerate the updating rule where the variance of gradients can be reduced. Using dual-free ASDCA, the optimization of dual function of SDCA in a form of convex loss is implemented by directly optimizing the primal function with respect to pseudo-dual parameters. The non-convex optimization in DNN training can be resolved and accelerated. Experimental results illustrate the reduction of training loss, variance of gradients and word error rate by using the proposed optimization for DNN speech recognition.
TIKHONOV REGULARIZATION FOR DEEP NEURAL NETWORK ACOUSTIC MODELING
(2014)
Chien, Jen-Tzung
;
Lu, Tsai-Wei
;
電機學院
;
College of Electrical and Computer Engineering
Deep neural network (DNN) has been widely demonstrated to achieve high performance in different speech recognition tasks. This paper focuses on the issue of model regularization in DNN acoustic model. Our idea is to compensate for the perturbations over training samples in the restricted Boltzmann machine (RBM) which is applied as a pre-training stage for unsupervised feature learning and structural modeling. We introduce the Tikhonov regularization in pre-training procedure and pursue the invariance property of objective function over the variations in input samples. This Tikhonov regularization is further combined with the regularization based on weight decay. The error function in supervised cross-entropy training is accordingly reduced. Experimental results on using RM and Aurora4 tasks show that hybrid regularization in RBM pre-training improves the training condition in DNN acoustic model and the robustness in speech recognition performance.
Hierarchical Pitman-Yor-Dirichlet Language Model
(2015-08-01)
Chien, Jen-Tzung
;
電機資訊學士班
;
Undergraduate Honors Program of Electrical Engineering and Computer Science
Probabilistic models are often viewed as insufficiently expressive because of strong limitation and assumption on the probabilistic distribution and the fixed model complexity. Bayesian nonparametric learning pursues an expressive probabilistic representation based on the nonparametric prior and posterior distributions with less assumption-laden approach to inference. This paper presents a hierarchical Pitman-Yor-Dirichlet (HPYD) process as the nonparametric priors to infer the predictive probabilities of the smoothed n-grams with the integrated topic information. A metaphor of hierarchical Chinese restaurant process is proposed to infer the HPYD language model (HPYD-LM) via Gibbs sampling. This process is equivalent to implement the hierarchical Dirichlet process-latent Dirichlet allocation (HDP-LDA) with the twisted hierarchical Pitman-Yor LM (HPY-LM) as base measures. Accordingly, we produce the power-law distributions and extract the semantic topics to reflect the properties of natural language in the estimated HPYD-LM. The superiority of HPYD-LM to HPY-LM and other language models is demonstrated by the experiments on model perplexity and speech recognition.
Rapid Bayesian Learning for Recurrent Neural Network Language Model
(2014-01-01)
Chien, Jen-Tzung
;
Ku, Yuan-Chu
;
Huang, Mou-Yue
;
電機資訊學士班
;
Undergraduate Honors Program of Electrical Engineering and Computer Science
This paper presents Bayesian learning for recurrent neural network language model (RNN-LM). Our goal is to regularize the RNN-LM by compensating for the randomness of the estimated model parameters which is characterized by a Gaussian prior. This model is not only constructed by training the synaptic weight parameters according to the maximum a posteriori criterion but also regularized by estimating the Gaussian hyperparameter through the type 2 maximum likelihood. However, a critical issue in Bayesian RNN-LM is the heavy computation of Hessian matrix which is formed as the sum of a large amount of outer-products of high-dimensional gradient vectors. We present a rapid approximation to reduce the redundancy due to the curse of dimensionality and speed up the calculation by summing up only the salient outer-products. Experiments on 1B-Word Benchmark, Penn Treebank and World Street Journal corpora show that rapid Bayesian RNN-LM consistently improves the perplexity and word error rate in comparison with standard RNN-LM.
Laplace Group Sensing for Acoustic Models
(2015-05-01)
Chien, Jen-Tzung
;
電機資訊學士班
;
Undergraduate Honors Program of Electrical Engineering and Computer Science
This paper presents the group sparse learning for acoustic models where a sequence of acoustic features is driven by Markov chain and each feature vector is represented by groups of basis vectors. The group of common bases represents the features across Markov states within a regression class. The group of individual basis compensates the intra-state residual information. Laplace distribution is used as the sparse prior of sensing weights for group basis representation. Laplace parameter serves as regularization parameter or automatic relevance determination which controls the selection of relevant bases for acoustic modeling. The groups of regularization parameters and basis vectors are estimated from training data by maximizing the marginal likelihood over sensing weights which is implemented by Laplace approximation using the Hessian matrix and the maximum a posteriori parameters. Model uncertainty is compensated through full Bayesian treatment. The connection of Laplace group sensing to lasso regularization is illustrated. Experiments on noisy speech recognition show the robustness of group sparse acoustic models in presence of different noise types and SNRs.

學術出版;;Publications

Browse

Filters

Settings

Sort By

Results per page

🏛️國立陽明交通大學機構典藏

Search Results