Browsing by Author "Chien, Jen-Tzung"

Now showing 1 - 20 of 84

Adaptive Processing and Learning for Audio Source Separation
(2013)
Chien, Jen-Tzung
;
Sawada, Hiroshi
;
Makino, Shoji
;
電機資訊學士班
;
Undergraduate Honors Program of Electrical Engineering and Computer Science
This paper overviews a series of recent advances in adaptive processing and learning for audio source separation. In real world, speech and audio signal mixtures are observed in reverberant environments. Sources are usually more than mixtures. The mixing condition is occasionally changed due to the moving sources or when the sources are changed or abruptly present or absent. In this survey article, we investigate different issues in audio source separation including overdetermined/ underdetermined problems, permutation alignment, convolutive mixtures, contrast functions, nonstationary conditions and system robustness. We provide a systematic and comprehensive view for these issues and address new approaches to overdetermined/ underdetermined convolutive separation, sparse learning, nonnegative matrix factorization, information-theoretic learning, online learning and Bayesian approaches.
ADVERSARIAL DOMAIN SEPARATION AND ADAPTATION
(2017-01-01)
Tsai, Jen-Chieh
;
Chien, Jen-Tzung
;
電機工程學系
;
Department of Electrical and Computer Engineering
Traditional domain adaptation methods attempted to learn the shared representation for distribution matching between source domain and target domain where the individual information in both domains was not characterized. Such a solution suffers from the mixing problem of individual information with the shared features which considerably constrains the performance for domain adaptation. To relax this constraint, it is crucial to extract both shared information and individual information. This study captures both information via a new domain separation network where the shared features are extracted and purified via separate modeling of individual information in both domains. In particular, a hybrid adversarial learning is incorporated in a separation network as well as an adaptation network where the associated discriminators are jointly trained for domain separation and adaptation according to the minmax optimization over separation loss and domain discrepancy, respectively. Experiments on different tasks show the merit of using the proposed adversarial domain separation and adaptation.
ADVERSARIAL MANIFOLD LEARNING FOR SPEAKER RECOGNITION
(2017-01-01)
Chien, Jen-Tzung
;
Peng, Kang-Ting
;
電機工程學系
;
Department of Electrical and Computer Engineering
This paper presents an adversarial manifold learning (AML) for speaker recognition based on the probabilistic linear discriminant analysis (PLDA) using i-vectors. PLDA basically consists of an encoder for finding the latent variables and a decoder for reconstructing the i-vectors. AML is developed and incorporated in deep learning for a latent variable model. Low-dimensional latent space is therefore constructed according to an adversarial learning with neighbor embedding. This AML-PLDA is formulated to jointly optimize three learning objectives including a reconstruction error based on PLDA, a subspace learning for neighbor embedding and an adversarial loss caused by a discriminator and a generator. Using the deep neural networks, the generator is trained to fool the discriminator with its generated samples in latent space. The parameters in encoder, decoder and discriminator are jointly estimated by using the stochastic gradient descent algorithm. The experiments on speaker recognition show the merit of AML-PLDA in manifold learning and pattern classification.
AUTOMATIC SPEECH RECOGNITION FOR ACOUSTICAL ANALYSIS AND ASSESSMENT OF CANTONESE PATHOLOGICAL VOICE AND SPEECH
(2016)
Lee, Tan
;
Liu, Yuanyuan
;
Huang, Pei-Wen
;
Chien, Jen-Tzung
;
Lam, Wang Kong
This paper describes the application of state-of-the-art automatic speech recognition (ASR) systems to objective assessment of voice and speech disorders. Acoustical analysis of speech has long been considered a promising approach to non-invasive and objective assessment of people. In the past the types and amount of speech materials used for acoustical assessment were very limited. With the ASR technology, we are able to perform acoustical and linguistic analyses with a large amount of natural speech from impaired speakers. The present study is focused on Cantonese, which is a major Chinese dialect. Two representative disorders of speech production are investigated: dysphonia and aphasia. ASR experiments are carried out with continuous and spontaneous speech utterances from Cantonese-speaking patients. The results confirm the feasibility and potential of using natural speech for acoustical assessment of voice and speech disorders, and reveal the challenging issues in acoustic modeling and language modeling of pathological speech.
BALSON: BAYESIAN LEAST SQUARES OPTIMIZATION WITH NONNEGATIVE L1-NORM CONSTRAINT
(2018-01-01)
Xie, Jiyang
;
Ma, Zhanyu
;
Zhang, Guoqiang
;
Xue, Jing-Hao
;
Chien, Jen-Tzung
A Bayesian approach termed the BAyesian Least Squares Optimization with Nonnegative L-1-norm constraint (BALSON) is proposed. The error distribution of data fitting is described by Gaussian likelihood. The parameter distribution is assumed to be a Dirichlet distribution. With the Bayes rule, searching for the optimal parameters is equivalent to finding the mode of the posterior distribution. In order to explicitly characterize the nonnegative L-1-norm constraint of the parameters, we further approximate the true posterior distribution by a Dirichlet distribution. We estimate the moments of the approximated Dirichlet posterior distribution by sampling methods. Four sampling methods have been introduced and implemented. With the estimated posterior distributions, the original parameters can be effectively reconstructed in polynomial fitting problems, and the BALSON framework is found to perform better than conventional methods.
BAYESIAN ADVERSARIAL LEARNING FOR SPEAKER RECOGNITION
(2019-01-01)
Chien, Jen-Tzung
;
Kuo, Chun-Lin
;
電機工程學系
;
Department of Electrical and Computer Engineering
This paper presents a new generative adversarial network (GAN) which artificially generates the i-vectors to compensate the imbalanced or insufficient data in speaker recognition based on the probabilistic linear discriminant analysis. Theoretically, GAN is powerful to generate the artificial data which are misclassified as the real data. However, GAN suffers from the mode collapse problem in two-player optimization over generator and discriminator. This study deals with this challenge by improving the model regularization through characterizing the weight uncertainty in GAN. A new Bayesian GAN is implemented to learn a regularized model from diverse data where the strong modes are flattened via the marginalization. In particular, we present a variational GAN (VGAN) where the encoder, generator and discriminator are jointly estimated according to the variational inference. The computation cost is significantly reduced. To assure the preservation of gradient values, the learning objective based on Wasserstein distance is further introduced. The issues of model collapse and gradient vanishing are alleviated. Experiments on NIST i-vector Speaker Recognition Challenge demonstrate the superiority of the proposed VGAN to the variational autoencoder, the standard GAN and the Bayesian GAN based on the sampling method. The learning efficiency and generation performance are evaluated.
Bayesian Factorization and Learning for Monaural Source Separation
(2016-01-01)
Chien, Jen-Tzung
;
Yang, Po-Kai
;
電機資訊學士班
;
Undergraduate Honors Program of Electrical Engineering and Computer Science
This paper presents a new Bayesian nonnegative matrix factorization (NMF) for monaural source separation. Using this approach, the reconstruction error based on NMF is represented by a Poisson distribution, and the NMF parameters, consisting of the basis and weight matrices, are characterized by the exponential priors. A variational Bayesian inference procedure is developed to learn variational parameters and model parameters. The randomness in separation process is faithfully represented so that the system robustness to model variations in heterogeneous environments could be achieved. Importantly, the exponential prior parameters are used to impose sparseness in basis representation. The variational lower bound of log marginal likelihood is adopted as the objective to control model complexity. The dependencies of variational objective on model parameters are fully characterized in the derived closed-form solution. A clustering algorithm is performed to find the groups of bases for unsupervised source separation. The experiments on speech/music separation and singing voice separation show that the proposed Bayesian NMF (BNMF) with adaptive basis representation outperforms the NMF with fixed number of bases and the other BNMFs in terms of signal-to-distortion ratio and the global normalized source to distortion ratio.
Bayesian Factorization and Selection for Speech and Music Separation
(2014-01-01)
Yang, Po-Kai
;
Hsu, Chung-Chien
;
Chien, Jen-Tzung
;
電機工程學系
;
Department of Electrical and Computer Engineering
This paper proposes a new Bayesian nonnegative matrix factorization (NMF) for speech and music separation. We introduce the Poisson likelihood for NMF approximation and the exponential prior distributions for the factorized basis matrix and weight matrix. A variational Bayesian (VB) EM algorithm is developed to implement an efficient solution to variational parameters and model parameters for Bayesian NMF. Importantly, the exponential prior parameter is used to control the sparseness in basis representation. The variational lower bound in VB-EM procedure is derived as an objective to conduct adaptive basis selection for different mixed signals. The experiments on single-channel speech/music separation show that the adaptive basis representation in Bayesian NMF via model selection performs better than the NMF with the fixed number of bases in terms of signal-to-distortion ratio.
Bayesian group sparse learning for music source separation
(2013)
Chien, Jen-Tzung
;
Hsieh, Hsin-Lung
;
電機資訊學士班
;
Undergraduate Honors Program of Electrical Engineering and Computer Science
Nonnegative matrix factorization (NMF) is developed for parts-based representation of nonnegative signals with the sparseness constraint. The signals are adequately represented by a set of basis vectors and the corresponding weight parameters. NMF has been successfully applied for blind source separation and many other signal processing systems. Typically, controlling the degree of sparseness and characterizing the uncertainty of model parameters are two critical issues for model regularization using NMF. This paper presents the Bayesian group sparse learning for NMF and applies it for single-channel music source separation. This method reconstructs the rhythmic or repetitive signal from a common subspace spanned by the shared bases for the whole signal and simultaneously decodes the harmonic or residual signal from an individual subspace consisting of separate bases for different signal segments. A Laplacian scale mixture distribution is introduced for sparse coding given a sparseness control parameter. The relevance of basis vectors for reconstructing two groups of music signals is automatically determined. A Markov chain Monte Carlo procedure is presented to infer two sets of model parameters and hyperparameters through a sampling procedure based on the conditional posterior distributions. Experiments on separating single-channel audio signals into rhythmic and harmonic source signals show that the proposed method outperforms baseline NMF, Bayesian NMF, and other group-based NMF in terms of signal-to-interference ratio.
Bayesian Group Sparse Learning for Nonnegative Matrix Factorization
(2012)
Chien, Jen-Tzung
;
Hsieh, Hsin-Lung
;
電機資訊學士班
;
Undergraduate Honors Program of Electrical Engineering and Computer Science
Nonnegative matrix factorization (NMF) is developed for parts-based representation of nonnegative data with the sparseness constraint. The degree of sparseness plays an important role for model regularization. This paper presents Bayesian group sparse learning for NMF and applies it for single-channel source separation. This method establishes the common bases and individual bases to characterize the shared information and residual noise in observed signals, respectively. Laplacian scale mixture distribution is introduced for sparse coding given a sparseness control parameter. A Markov chain Monte Carlo procedure is presented to infer two groups of parameters and their hyperparameters through a sampling procedure based on the conditional posterior distributions. Experiments on separating the single-channel audio signals into rhythmic and harmonic source signals show that the proposed method outperforms baseline NMF, Bayesian NMF and other group-based NMF in terms of signal-to-interference ratio.
BAYESIAN LATENT VARIABLE MODELS FOR SPEECH RECOGNITION
(2013)
Chien, Jen-Tzung
;
Liu, Peng
;
電機資訊學士班
;
Undergraduate Honors Program of Electrical Engineering and Computer Science
We present a Bayesian framework to learn prior and posterior distributions for latent variable models. Our goal is to deal with model regularization and achieve desirable prediction using heterogeneous speech data. A variational Bayesian expectation-maximization algorithm is developed to establish a latent variable model based on the exponential family distributions. This algorithm does not only estimate model parameters but also their hyperparameters which reflect the model uncertainties. The uncertainty is compensated to construct a variety of regularized models. We realize this full Bayesian framework for uncertainty decoding of speech signals. Compared to maximum likelihood method and Bayesian approach with heuristically-selected hyperparameters, the proposed method achieves higher speech recognition accuracy especially in case of sparse and noisy training data.
BAYESIAN LEARNING FOR SPEECH DEREVERBERATION
(2016)
Chien, Jen-Tzung
;
Chang, You-Cheng
;
電機工程學系
;
Department of Electrical and Computer Engineering
This study presents a Bayesian approach to enhance the magnitude spectra of single-channel reverberant speech signals. Speech dereverberation model is constructed by using a non-negative convolutive transfer function (NCTF) and a nonnegative matrix factorization (NMF). NCTF is used to characterize the magnitude spectra of speech signal and room impulse response while NMF is applied to represent the fine structure of speech spectra. Importantly, we deal with the variations of dereverberation model by introducing the exponential priors for reverberation kernel and noise signal. A full Bayesian solution to speech dereverberation is obtained according to the variational Bayesian inference algorithm. Using this algorithm, the room configuration and the speaker characteristics are automatically learned from data. Such a general model can be reduced to the previous methods. Experimental results on both simulated data and real recordings from 2014 RE-VERB Challenge show the merit of the proposed method for single-channel speech dereverberation.
BAYESIAN NONPARAMETRIC LANGUAGE MODELS
(2012)
Chang, Ying-Lan
;
Chien, Jen-Tzung
;
電機資訊學士班
;
Undergraduate Honors Program of Electrical Engineering and Computer Science
Backoff smoothing and topic modeling are crucial issues in n-gram language model. This paper presents a Bayesian non-parametric learning approach to tackle these two issues. We develop a topic-based language model where the numbers of topics and n-grams are automatically determined from data. To cope with this model selection problem, we introduce the nonparametric priors for topics and backoff n-grams. The infinite language models are constructed through the hierarchical Dirichlet process compound Pitman-Yor (PY) process. We develop the topic-based hierarchical PY language model (THPY-LM) with power-law behavior. This model can be simplified to the hierarchical PY (HPY) LM by disregarding the topic information and also the modified Kneser-Ney (MKN) LM by further disregarding the Bayesian treatment. In the experiments, the proposed THPY-LM outperforms state-of-art methods using MKN-LM and HPY-LM.
Bayesian Recurrent Neural Network for Language Modeling
(2016-02)
Chien, Jen-Tzung
;
Ku, Yuan-Chu
;
電機學院
;
College of Electrical and Computer Engineering
A language model (LM) is calculated as the probability of a word sequence that provides the solution to word prediction for a variety of information systems. A recurrent neural network (RNN) is powerful to learn the large-span dynamics of a word sequence in the continuous space. However, the training of the RNN-LM is an ill-posed problem because of too many parameters from a large dictionary size and a high-dimensional hidden layer. This paper presents a Bayesian approach to regularize the RNN-LM and apply it for continuous speech recognition. We aim to penalize the too complicated RNN-LM by compensating for the uncertainty of the estimated model parameters, which is represented by a Gaussian prior. The objective function in a Bayesian classification network is formed as the regularized cross-entropy error function. The regularized model is constructed not only by calculating the regularized parameters according to the maximum a posteriori criterion but also by estimating the Gaussian hyperparameter by maximizing the marginal likelihood. A rapid approximation to a Hessian matrix is developed to implement the Bayesian RNN-LM (BRNN-LM) by selecting a small set of salient outer-products. The proposed BRNN-LM achieves a sparser model than the RNN-LM. Experiments on different corpora show the robustness of system performance by applying the rapid BRNN-LM under different conditions.
BAYESIAN RECURRENT NEURAL NETWORK LANGUAGE MODEL
(2014)
Chien, Jen-Tzung
;
Ku, Yuan-Chu
;
電機學院
;
College of Electrical and Computer Engineering
This paper presents a Bayesian approach to construct the recurrent neural network language model (RNN-LM) for speech recognition. Our idea is to regularize the RNN-LM by compensating the uncertainty of the estimated model parameters which is represented by a Gaussian prior. The objective function in Bayesian RNN (BRNN) is formed as the regularized cross entropy error function. The regularized model is not only constructed by training the regularized parameters according to the maximum a posteriori criterion but also estimating the Gaussian hyperparameter by maximizing the marginal likelihood. A rapid approximation to Hessian matrix is developed by selecting a small set of salient outer-products and illustrated to be effective for BRNN-LM. BRNN-LM achieves sparser model than RNN-LM. Experiments on different corpora show promising improvement by applying BRNN-LM using different amount of training data.
Bayesian Sparse Topic Model
(2014-03-01)
Chien, Jen-Tzung
;
Chang, Ying-Lan
;
電機資訊學士班
;
Undergraduate Honors Program of Electrical Engineering and Computer Science
This paper presents a new Bayesian sparse learning approach to select salient lexical features for sparse topic modeling. The Bayesian learning based on latent Dirichlet allocation (LDA) is performed by incorporating the spike-and-slab priors. According to this sparse LDA (sLDA), the spike distribution is used to select salient words while the slab distribution is applied to establish the latent topic model based on those selected relevant words. The variational inference procedure is developed to estimate prior parameters for sLDA. In the experiments on document modeling using LDA and sLDA, we find that the proposed sLDA does not only reduce the model perplexity but also reduce the memory and computation costs. Bayesian feature selection method does effectively identify relevant topic words for building sparse topic model.
Binary Mask Estimation Based on Frequency Modulations
(2014-01-01)
Hsu, Chung-Chien
;
Chien, Jen-Tzung
;
Chi, Tai-Shih
;
電機工程學系
;
Department of Electrical and Computer Engineering
In this paper, a binary mask estimation algorithm is proposed based on modulations of speech. A multi-resolution spectro-temporal analytical auditory model is utilized to extract modulation features to estimate the binary mask, which is often used in speech segregation applications. The proposed method estimates noise from the beginning of each test sentence, a common approach seen in many conventional speech enhancement algorithms, to further enhance the modulation features. Experimental results demonstrate that the proposed method outperforms the AMS-GMM system in terms of the HIT-FA rate when estimating the binary mask.
Convolutional Neural Turing Machine for Speech Separation
(2018-01-01)
Chien, Jen-Tzung
;
Tsou, Kai-Wei
;
電機工程學系
;
Department of Electrical and Computer Engineering
Long short-term memory (LSTM) has been successfully developed for monaural speech separation. Temporal information is learned by using dynamic states which are evolved through time and stored as an internal memory. The spectro-temporal data matrix of mixed signal is flattened as input vectors. There are twofold limitations. First, the internal memory in LSTM could not sufficiently characterize long-term information from different sources. Second, the temporal correlation and frequency neighboring in the flattened vectors were smeared. To deal with these limitations, this paper presents a convolutional neural Turing machine (ConvNTM) where the feature maps of spectro-temporal data are extracted and embedded in an external memory at each time step. ConvNTM aims to preserve the spectro-temporal structure in long sequential signals which is exploited to estimate the separated spectral signals. An addressing mechanism is introduced to continuously calculate the read and write heads to retrieve and update memory slots, respectively. The memory augmented source separation is implemented for single-channel speech enhancement. Experimental results illustrate the superiority of ConvNTM to LSTM, NTM and convolutional LSTM for speech enhancement in terms of short-term objective intelligibility measure.
Deep Bayesian Data Mining
(2020-01-01)
Chien, Jen-Tzung
;
交大名義發表
;
National Chiao Tung University
This tutorial addresses the fundamentals and advances in deep Bayesian mining and learning for natural language with ubiquitous applications ranging from speech recognition [7, 55] to document summarization [8], text classification [5, 75], text segmentation [18], information extraction [50], image caption generation [69, 72], sentence generation [25, 46], dialogue control [22, 76], sentiment classification, recommendation system, question answering [58] and machine translation [2], to name a few. Traditionally, "deep learning" is taken to be a learning process where the inference or optimization is based on the real-valued deterministic model. The "semantic structure" in words, sentences, entities, actions and documents drawn from a large vocabulary may not be well expressed or correctly optimized in mathematical logic or computer programs. The "distribution function" in discrete or continuous latent variable model for natural language may not be properly decomposed or estimated. This tutorial addresses the fundamentals of statistical models and neural networks, and focus on a series of advanced Bayesian models and deep models including hierarchical Dirichlet process [61], Chinese restaurant process [4], hierarchical Pitman-Yor process [60], Indian buffet process [35], recurrent neural network (RNN) [26, 41, 48, 65], long short-term memory, sequence-to-sequence model [59], variational auto-encoder (VAE) [44], generative adversarial network (GAN) [36], attention mechanism [27, 56], memory-augmented neural network [39, 58], skip neural network [6], temporal difference VAE [40], stochastic neural network [3, 47], stochastic temporal convolutional network [1], predictive state neural network [31], and policy neural network [49, 74]. Enhancing the prior/posterior representation is addressed [53, 62]. We present how these models are connected and why they work for a variety of applications on symbolic and complex patterns in natural language. The variational inference and sampling method are formulated to tackle the optimization for complicated models [54]. The word and sentence embeddings, clustering and co-clustering are merged with linguistic and semantic constraints. A series of case studies, tasks and applications are presented to tackle different issues in deep Bayesian mining, searching, learning and understanding. At last, we will point out a number of directions and outlooks for future studies. This tutorial serves the objectives to introduce novices to major topics within deep Bayesian learning, motivate and explain a topic of emerging importance for data mining and natural language understanding, and present a novel synthesis combining distinct lines of machine learning work.
Deep Bayesian Mining, Learning and Understanding
(2019-01-01)
Chien, Jen-Tzung
;
交大名義發表
;
National Chiao Tung University
This tutorial addresses the advances in deep Bayesian mining and learning for natural language with ubiquitous applications ranging from speech recognition to document summarization, text classification, text segmentation, information extraction, image caption generation, sentence generation, dialogue control, sentiment classification, recommendation system, question answering and machine translation, to name a few. Traditionally, "deep learning" is taken to be a learning process where the inference or optimization is based on the real-valued deterministic model. The "semantic structure" in words, sentences, entities, actions and documents drawn from a large vocabulary may not be well expressed or correctly optimized in mathematical logic or computer programs. The "distribution function" in discrete or continuous latent variable model for natural language may not be properly decomposed or estimated. This tutorial addresses the fundamentals of statistical models and neural networks, and focus on a series of advanced Bayesian models and deep models including hierarchical Dirichlet process, Chinese restaurant process, hierarchical Pitman-Yor process, Indian buffet process, recurrent neural network (RNN), long short-term memory, sequence-to-sequence model, variational auto-encoder (VAE), generative adversarial network (GAN), attention mechanism, memory-augmented neural network, skip neural network, stochastic neural network, predictive state neural network, policy neural network. We present how these models are connected and why they work for a variety of applications on symbolic and complex patterns in natural language. The variational inference and sampling method are formulated to tackle the optimization for complicated models. The word and sentence embeddings, clustering and co-clustering are merged with linguistic and semantic constraints. A series of case studies are presented to tackle different issues in deep Bayesian mining, learning and understanding. At last, we will point out a number of directions and outlooks for future studies.