<link rel="stylesheet" href="styles.cf390c6bfbe7555f.css">

學術出版;;Publications

Permanent URI for this communityhdl:11536/1

收集所有交通大學研究人員與學生個人或團體所發表之各項學術著作，涵蓋期刊論文、教師專書、學位論文等項。;;Collections of Publications of NCTU Researchers and Students Like Journal Papers and Thesis.

Browse

🏛️國立陽明交通大學機構典藏

Author: search.filters.author.Chien, Jen-TzungSubject: search.filters.subject.topic model

Search Results

Now showing 1 - 7 of 7
  • Thumbnail Image
    Item type:Item,
    Hierarchical Pitman-Yor and Dirichlet Process for Language Model
    (2013-01-01)
    Chien, Jen-Tzung
    ;
    Chang, Ying-Lan
    ;
    電機工程學系
    ;
    Department of Electrical and Computer Engineering
    This paper presents a nonparametric interpretation for modem language model based on the hierarchical Pitman-Yor and Dirichlet (HPYD) process. We propose the HPYD language model (HPYD-LM) which flexibly conducts backoff smoothing and topic clustering through Bayesian nonparametric learning. The nonparametric priors of backoff n-grams and latent topics are tightly coupled in a compound process. A hybrid probability measure is drawn to build the smoothed topic-based LM. The model structure is automatically determined from training data. A new Chinese restaurant scenario is proposed to implement HPYD-LM via Gibbs sampling. This process reflects the power-law property and extracts the semantic topics from natural language. The superiority of HPYD-LM to the related LMs is demonstrated by the experiments on different corpora in terms of perplexity and word error rate.
  • Thumbnail Image
    Item type:Item,
    The Nested Indian Buffet Process :for Flexible Topic Modeling
    (2014-01-01)
    Chien, Jen-Tzung
    ;
    Chang, Ying-Lan
    ;
    電機工程學系
    ;
    Department of Electrical and Computer Engineering
    This paper presents a flexible topic model based on the nested Indian buffet process (nIBP). The flexibility is achieved by relaxing three constraints: (1) number of topics is fixed, (2) topics are independent, and (3) topic hierarchy for a document is limited by a single tree path. Bayesian nonparametric learning is conducted to build a tree model where the number of topics and the topic hierarchies are automatically learnt from the given data. In particular, we propose the nIBP to construct the topic mixture model for representation of heterogeneous documents where the mixture components are flexibly selected from tree nodes or dishes that a document or customer chooses in Indian buffet process. The selection is performed in a nested and hierarchical manner. The experiments on document representation show the benefits of using the proposed nIBP.
  • Thumbnail Image
    Item type:Item,
    Deep Unfolding for Topic Models
    (2018-02-01)
    Chien, Jen-Tzung
    ;
    Lee, Chao-Hsi
    ;
    電機工程學系
    ;
    Department of Electrical and Computer Engineering
    Deep unfolding provides an approach to integrate the probabilistic generative models and the deterministic neural networks. Such an approach is benefited by deep representation, easy interpretation, flexible learning and stochastic modeling. This study develops the unsupervised and supervised learning of deep unfolded topic models for document representation and classification. Conventionally, the unsupervised and supervised topic models are inferred via the variational inference algorithm where the model parameters are estimated by maximizing the lower bound of logarithm of marginal likelihood using input documents without and with class labels, respectively. The representation capability or classification accuracy is constrained by the variational lower bound and the tied model parameters across inference procedure. This paper aims to relax these constraints by directly maximizing the end performance criterion and continuously untying the parameters in learning process via deep unfolding inference (DUI). The inference procedure is treated as the layer-wise learning in a deep neural network. The end performance is iteratively improved by using the estimated topic parameters according to the exponentiated updates. Deep learning of topic models is therefore implemented through a back-propagation procedure. Experimental results show the merits of DUI with increasing number of layers compared with variational inference in unsupervised as well as supervised topic models.
  • Thumbnail Image
    Item type:Item,
    Hierarchical Theme and Topic Modeling
    (2016-03)
    Chien, Jen-Tzung
    ;
    電機學院
    ;
    College of Electrical and Computer Engineering
    Considering the hierarchical data groupings in text corpus, e.g., words, sentences, and documents, we conduct the structural learning and infer the latent themes and topics for sentences and words from a collection of documents, respectively. The relation between themes and topics under different data groupings is explored through an unsupervised procedure without limiting the number of clusters. A tree stick-breaking process is presented to draw theme proportions for different sentences. We build a hierarchical theme and topic model, which flexibly represents the heterogeneous documents using Bayesian nonparametrics. Thematic sentences and topical words are extracted. In the experiments, the proposed method is evaluated to be effective to build semantic tree structure for sentences and the corresponding words. The superiority of using tree model for selection of expressive sentences for document summarization is illustrated.
  • Thumbnail Image
    Item type:Item,
    DIRICHLET MIXTURE ALLOCATION
    (2016)
    Chien, Jen-Tzung
    ;
    Lee, Chao-Hsi
    ;
    Tan, Zheng-Hua
    ;
    電機工程學系
    ;
    Department of Electrical and Computer Engineering
    The topic model based on latent Dirichlet allocation relies on the prior statistics of topic proportionals for multinomial words. The words in a document are modeled as a random mixture of latent topics which are drawn from a single Dirichlet prior. However, a single Dirichlet distribution may not sufficiently characterize the variations of topic proportionals estimated from the heterogeneous documents. To deal with this concern, we present a Dirichlet mixture allocation (DMA) model which learns latent topics and their proportionals for topic and document clustering by using the prior based on a Dirichlet mixture model. Multiple Dirichlets pave a way to capture the structure of latent variables in learning representation from real-world documents covering a variety of topics. This paper builds a new latent variable model and develops a variational Bayesian inference procedure to learn model parameters consisting of mixture weights, Dirichlet parameters and word multinomials. Experiments on document representation show the merit of the proposed structural learning by increasing the number of Dirichlets in a DMA topic model.
  • Thumbnail Image
    Item type:Item,
    Hierarchical Pitman-Yor-Dirichlet Language Model
    (2015-08-01)
    Chien, Jen-Tzung
    ;
    電機資訊學士班
    ;
    Undergraduate Honors Program of Electrical Engineering and Computer Science
    Probabilistic models are often viewed as insufficiently expressive because of strong limitation and assumption on the probabilistic distribution and the fixed model complexity. Bayesian nonparametric learning pursues an expressive probabilistic representation based on the nonparametric prior and posterior distributions with less assumption-laden approach to inference. This paper presents a hierarchical Pitman-Yor-Dirichlet (HPYD) process as the nonparametric priors to infer the predictive probabilities of the smoothed n-grams with the integrated topic information. A metaphor of hierarchical Chinese restaurant process is proposed to infer the HPYD language model (HPYD-LM) via Gibbs sampling. This process is equivalent to implement the hierarchical Dirichlet process-latent Dirichlet allocation (HDP-LDA) with the twisted hierarchical Pitman-Yor LM (HPY-LM) as base measures. Accordingly, we produce the power-law distributions and extract the semantic topics to reflect the properties of natural language in the estimated HPYD-LM. The superiority of HPYD-LM to HPY-LM and other language models is demonstrated by the experiments on model perplexity and speech recognition.
  • Thumbnail Image
    Item type:Item,
    BAYESIAN NONPARAMETRIC LANGUAGE MODELS
    (2012)
    Chang, Ying-Lan
    ;
    Chien, Jen-Tzung
    ;
    電機資訊學士班
    ;
    Undergraduate Honors Program of Electrical Engineering and Computer Science
    Backoff smoothing and topic modeling are crucial issues in n-gram language model. This paper presents a Bayesian non-parametric learning approach to tackle these two issues. We develop a topic-based language model where the numbers of topics and n-grams are automatically determined from data. To cope with this model selection problem, we introduce the nonparametric priors for topics and backoff n-grams. The infinite language models are constructed through the hierarchical Dirichlet process compound Pitman-Yor (PY) process. We develop the topic-based hierarchical PY language model (THPY-LM) with power-law behavior. This model can be simplified to the hierarchical PY (HPY) LM by disregarding the topic information and also the modified Kneser-Ney (MKN) LM by further disregarding the Bayesian treatment. In the experiments, the proposed THPY-LM outperforms state-of-art methods using MKN-LM and HPY-LM.