Browsing by Author "Hsaio, Wen-Hoar"

Now showing 1 - 12 of 12

An HMM-Based Algorithm for Content Ranking and Coherence-Feature Extraction
(2013-03-01)
Liu, Chien-Liang
;
Hsaio, Wen-Hoar
;
Lee, Chia-Hoang
;
Chi, Hsiao-Cheng
;
資訊工程學系
In this paper, we propose an algorithm called coherence hidden Markov model (HMM) to extract coherence features and rank content. Coherence HMM is a variant of HMM and is used to model the stochastic process of essay writing and identify topics as hidden states, given sequenced clauses as observations. This study uses probabilistic latent semantic analysis for parameter estimation of coherence HMM. In coherence-feature extraction, support vector regression (SVR) with surface features and coherence features is used for essay grading. The experimental results indicate that SVR can benefit from coherence features. The adjacent agreement rate and the exact agreement rate are 95.24% and 59.80%, respectively. Moreover, this study submits high-scoring essays to the same experiment and finds that the adjacent agreement rate and exact agreement rate are 98.33% and 64.50%, respectively. In content ranking, we design and implement an intelligent assisted blog writing system based on the coherence-HMM ranking model. Several corpora are employed to help users efficiently compose blog articles. When users finish composing a clause or sentence, the system provides candidate texts for their reference based on current clause or sentence content. The experimental results demonstrate that all participants can benefit from the system and save considerable time on writing articles.
Bayesian exploratory clustering with entropy Chinese restaurant process
(2018-01-01)
Liu, Chien-Liang
;
Hsaio, Wen-Hoar
;
Lin, Che-Yuan
;
資訊工程學系
;
工業工程與管理學系
Data exploration is essential to data analytics, especially when one is confronted with massive datasets. Clustering is a commonly used technique in data exploration, since it can automatically group data instances into a list of meaningful categories, and capture the natural structure of data. Traditional finite mixture model requires the number of clusters to be specified in advance of analyzing the data, and this parameter is crucial to the clustering performance. Chinese restaurant process (CRP) mixture model provides an alternative to this problem, allowing the model complexity to grow as more data instances are observed. Although CRP provides the flexibility to create a new cluster for subsequent data instances, one still has to determine the hyperparameter of the prior and the parameters for the base distribution in the likelihood part. This work proposes a non-parametric clustering algorithm based on CRP with two main differences. First, we propose to create a new cluster based on entropy of the posterior, whereas the CRP uses a hyperparameter to control the probability of creating a new cluster. Second, we propose to dynamically adjust the parameters of the base distribution according to the mean of the observed data owing to Chebyshev's inequality. Additionally, detailed derivation and update rules are provided to perform posterior inference with the proposed collapsed Gibbs sampling algorithm. The experimental results indicate that the proposed algorithm avoids to specify the number of clusters and works well on several datasets.
Clustering data with partial background information
(2019-05-01)
Liu, Chien-Liang
;
Hsaio, Wen-Hoar
;
Chang, Tao-Hsing
;
Li, Hsuan-Hsun
;
資訊工程學系
Clustering with partial supervision background information or semi-supervised clustering, learning from a combination of both labeled and unlabeled data, has received a lot of attention over the last decade. The supervisory information is usually used as the constraints to bias clustering towards a good region of search space. This paper proposes a semi-supervised algorithm, called constrained non-negative matrix factorization (Constrained-NMF), with a few labeled examples as constraints to improve performance. The proposed algorithm is a matrix factorization algorithm, in which initialization of matrices is required at the beginning. Although the benefits of good initialization are well-known, randomized seeding of basis matrix and coefficient matrix is still the standard approach for many non-negative matrix factorization (NMF) algorithms. This work devises an algorithm called entropy-based weighted semi-supervised fuzzy c-means (EWSS-FCM) algorithm to initialize the seeds. The experimental results indicate that the proposed Constrained-NMF can benefit from the initialization obtained from EWSS-FCM, which emphasizes the role of labeled examples and automatically weights them during the course of clustering. This work considers labeled examples in the objective functions to devise the two algorithms, in which the labeled information is propagated to unlabeled examples iteratively. We further analyze the proposed Constrained-NMF and give convergence justifications. The experiments are conducted on five real data sets, and experimental results indicate that the proposed algorithm generally outperforms the other alternatives.
Epileptic Seizure Prediction With Multi-View Convolutional Neural Networks
(2019-01-01)
Liu, Chien-Liang
;
Xiao, Bin
;
Hsaio, Wen-Hoar
;
Tseng, Vincent S.
;
交大名義發表
The unpredictability of seizures is often considered by patients to be the most problematic aspect of epilepsy, so this work aims to develop an accurate epilepsy seizure predictor, making it possible to enable devices to warn patients of impeding seizures. To develop a model for seizure prediction, most studies relied on Electroencephalograms (EEGs) to capture physiological measurements of epilepsy. This work uses the two domains of EEGs, including frequency domain and time domain, to provide two different views for the same data source. Subsequently, this work proposes a multi-view convolutional neural network framework to predict the occurrence of epilepsy seizures with the goal of acquiring a shared representation of time-domain and frequency-domain features. By conducting experiments on Kaggle data set, we demonstrated that the proposed method outperforms all methods listed in the Kaggle leader board. Additionally, our proposed model achieves average area under the curve (AUCs) of 0.82 and 0.89 on two subjects of CHB-MIT scalp EEG data set. This work serves as an effective paradigm for applying deep learning approaches to the crucial topic of risk prediction in health domains.
Epileptic Seizure Prediction With Multi-View Convolutional Neural Networks
(2019-01-01)
Liu, Chien-Liang
;
Xiao, Bin
;
Hsaio, Wen-Hoar
;
Tseng, Vincent S.
;
交大名義發表
The unpredictability of seizures is often considered by patients to be the most problematic aspect of epilepsy, so this work aims to develop an accurate epilepsy seizure predictor, making it possible to enable devices to warn patients of impeding seizures. To develop a model for seizure prediction, most studies relied on Electroencephalograms (EEGs) to capture physiological measurements of epilepsy. This work uses the two domains of EEGs, including frequency domain and time domain, to provide two different views for the same data source. Subsequently, this work proposes a multi-view convolutional neural network framework to predict the occurrence of epilepsy seizures with the goal of acquiring a shared representation of time-domain and frequency-domain features. By conducting experiments on Kaggle data set, we demonstrated that the proposed method outperforms all methods listed in the Kaggle leader board. Additionally, our proposed model achieves average area under the curve (AUCs) of 0.82 and 0.89 on two subjects of CHB-MIT scalp EEG data set. This work serves as an effective paradigm for applying deep learning approaches to the crucial topic of risk prediction in health domains.
Locality-constrained max-margin sparse coding
(2017-05-01)
Hsaio, Wen-Hoar
;
Liu, Chien-Liang
;
Wu, Wei-Liang
;
資訊工程學系
;
工業工程與管理學系
This work devises a locality-constrained max-margin sparse coding (LC-MMSC) framework, which jointly considers reconstruction loss and hinge loss simultaneously. Traditional sparse coding algorithms use l(1) constraint to force the representation to be sparse, leading to computational expensive process to optimize the objective function. This work uses locality constraint in the framework to preserve information of data locality and avoid the optimization of l(1). The obtained representation can achieve the goal of data locality and sparsity. Additionally, this work optimizes coefficients, dictionaries and classification parameters simultaneously, and uses block coordinate descent to learn all the components of the proposed model. This work uses semi supervised learning approach in the proposed framework, and the goal is to use both labeled data and unlabeled data to achieve accurate classification performance and improve the generalization of the model. We provide theoretical analysis on the convergence of the proposed LC-MMSC algorithm based on Zangwill's global convergence theorem. This work conducts experiments on three real datasets, including Extended YaleB dataset, AR face dataset and Caltech101 dataset. The experimental results indicate that the proposed algorithm outperforms other comparison algorithms.
Maximum-margin sparse coding
(2017-05-17)
Liu, Chien-Liang
;
Hsaio, Wen-Hoar
;
Xiao, Bin
;
Chen, Chun-Yu
;
Wu, Wei-Liang
This work devises a maximum-margin sparse coding algorithm, jointly considering reconstruction loss and hinge loss in the model. The sparse representation along with maximum-margin constraint is analogous to kernel trick and maximum-margin properties of support vector machine (SVM), giving a base for the proposed algorithm to perform well in classification tasks. The key idea behind the proposed method is to use labeled and unlabeled data to learn discriminative representations and model parameters simultaneously, making it easier to classify data in the new space. We propose to use block coordinate descent to learn all the components of the proposed model and give detailed derivation for the update rules of the model variables. Theoretical analysis on the convergence of the proposed MMSC algorithm is provided based on Zangwill's global convergence theorem. Additionally, most previous research studies on dictionary learning suggest to use an overcomplete dictionary to improve classification performance, but it is computationally intensive when the dimension of the input data is huge. We conduct experiments on several real data sets, including Extended YaleB, AR face, and Caltech101 data sets. The experimental results indicate that the proposed algorithm outperforms other comparison algorithms without an over-complete dictionary, providing flexibility to deal with high-dimensional data sets. (C)2017 Elsevier B.V. All rights reserved.
Movie Rating and Review Summarization in Mobile Environment
(2012-05-01)
Liu, Chien-Liang
;
Hsaio, Wen-Hoar
;
Lee, Chia-Hoang
;
Lu, Gen-Chi
;
Jou, Emery
In this paper, we design and develop a movie-rating and review-summarization system in a mobile environment. The movie-rating information is based on the sentiment-classification result. The condensed descriptions of movie reviews are generated from the feature-based summarization. We propose a novel approach based on latent semantic analysis (LSA) to identify product features. Furthermore, we find away to reduce the size of summary based on the product features obtained from LSA. We consider both sentiment-classification accuracy and system response time to design the system. The rating and review-summarization system can be extended to other product-review domains easily.
Nonparametric multi-assignment clustering
(2017-01-01)
Liu, Chien-Liang
;
Hsaio, Wen-Hoar
;
Chang, Tao-Hsing
;
Jou, Tzai-Min
;
資訊工程學系
Multi-label learning has attracted significant attention from machine learning and data mining over the last decade. Although many multi-label classification algorithms have been devised, few research studies focus on multi-assignment clustering (MAC), in which a data instance can be assigned to multiple clusters. The MAC problem is practical in many application domains, such as document clustering, customer segmentation and image clustering. Additionally, specifying the number of clusters is always a difficult but critical problem for a certain class of clustering algorithms. Hence, this work proposes a non-parametric multi-assignment clustering algorithm called multi-assignment Chinese restaurant process (MACRP), which allows the model complexity to grow as more data instances are observed. The proposed algorithm determines the number of clusters from data, so it provides a practical model to process massive data sets. In the proposed algorithm, we devise a novel prior distribution based on the similarity graph to achieve the goal of multi-assignment, and propose a Gibbs sampling algorithm to carry out posterior inference. The implementation in this work uses collapsed Gibbs sampling and compares with several methods. Additionally, previous evaluation metrics used by multi-label classification are inappropriate for MAC, since label information is unavailable. This work further devises an evaluation metric for MAC based on the characteristics of clustering and multi-assignment problems. We conduct experiments on two real data sets, and the experimental results indicate that the proposed method is competitive and outperforms the alternatives on most data sets.
Semi-supervised Linear Discriminant Clustering
(2014-07-01)
Liu, Chien-Liang
;
Hsaio, Wen-Hoar
;
Lee, Chia-Hoang
;
Gou, Fu-Sheng
;
交大名義發表
This paper devises a semi-supervised learning method called semi-supervised linear discriminant clustering (Semi-LDC). The proposed algorithm considers clustering and dimensionality reduction simultaneously by connecting K-means and linear discriminant analysis (LDA). The goal is to find a feature space where the K-means can perform well in the new space. To exploit the information brought by unlabeled examples, this paper proposes to use soft labels to denote the labels of unlabeled examples. The Semi-LDC uses the proposed algorithm, called constrained-PLSA, to estimate the soft labels of unlabeled examples. We use soft LDA with hard labels of labeled examples and soft labels of unlabeled examples to find a projection matrix. The clustering is then performed in the new feature space. We conduct experiments on three data sets. The experimental results indicate that the proposed method can generally outperform other semi-supervised methods. We further discuss and analyze the influence of soft labels on classification performance by conducting experiments with different percentages of labeled examples. The finding shows that using soft labels can improve performance particularly when the number of available labeled examples is insufficient to train a robust and accurate model. Additionally, the proposed method can be viewed as a framework, since different soft label estimation methods can be used in the proposed method according to application requirements.
Semi-Supervised Text Classification With Universum Learning
(2016-02)
Liu, Chien-Liang
;
Hsaio, Wen-Hoar
;
Lee, Chia-Hoang
;
Chang, Tao-Hsing
;
Kuo, Tsung-Hsun
Universum, a collection of nonexamples that do not belong to any class of interest, has become a new research topic in machine learning. This paper devises a semi-supervised learning with Universum algorithm based on boosting technique, and focuses on situations where only a few labeled examples are available. We also show that the training error of AdaBoost with Universum is bounded by the product of normalization factor, and the training error drops exponentially fast when each weak classifier is slightly better than random guessing. Finally, the experiments use four data sets with several combinations. Experimental results indicate that the proposed algorithm can benefit from Universum examples and outperform several alternative methods, particularly when insufficient labeled examples are available. When the number of labeled examples is insufficient to estimate the parameters of classification functions, the Universum can be used to approximate the prior distribution of the classification functions. The experimental results can be explained using the concept of Universum introduced by Vapnik, that is, Universum examples implicitly specify a prior distribution on the set of classification functions.
Time Series Classification With Multivariate Convolutional Neural Network
(2019-06-01)
Liu, Chien-Liang
;
Hsaio, Wen-Hoar
;
Tu, Yao-Chung
Time series classification is an important research topic in machine learning and data mining communities, since time series data exist in many application domains. Recent studies have shown that machine learning algorithms could benefit from good feature representation, explaining why deep learning has achieved breakthrough performance in many tasks. In deep learning, the convolutional neural network (CNN) is one of the most well-known approaches, since it incorporates feature learning and classification task in a unified network architecture. Although CNN has been successfully applied to image and text domains, it is still a challenge to apply CNN to time series data. This paper proposes a tensor scheme along with a novel deep learning architecture called multivariate convolutional neural network (MVCNN) for multivariate time series classification, in which the proposed architecture considers multivariate and lag-feature characteristics. We evaluate our proposed method with the prognostics and health management (PHM) 2015 challenge data, and compare with several algorithms. The experimental results indicate that the proposed method outperforms the other alternatives using the prediction score, which is the evaluation metric used by the PHM Society 2015 data challenge. Besides performance evaluation, we provide detailed analysis about the proposed method.