Bayesian exploratory clustering with entropy Chinese restaurant process

Abstract

Data exploration is essential to data analytics, especially when one is confronted with massive datasets. Clustering is a commonly used technique in data exploration, since it can automatically group data instances into a list of meaningful categories, and capture the natural structure of data. Traditional finite mixture model requires the number of clusters to be specified in advance of analyzing the data, and this parameter is crucial to the clustering performance. Chinese restaurant process (CRP) mixture model provides an alternative to this problem, allowing the model complexity to grow as more data instances are observed. Although CRP provides the flexibility to create a new cluster for subsequent data instances, one still has to determine the hyperparameter of the prior and the parameters for the base distribution in the likelihood part. This work proposes a non-parametric clustering algorithm based on CRP with two main differences. First, we propose to create a new cluster based on entropy of the posterior, whereas the CRP uses a hyperparameter to control the probability of creating a new cluster. Second, we propose to dynamically adjust the parameters of the base distribution according to the mean of the observed data owing to Chebyshev's inequality. Additionally, detailed derivation and update rules are provided to perform posterior inference with the proposed collapsed Gibbs sampling algorithm. The experimental results indicate that the proposed algorithm avoids to specify the number of clusters and works well on several datasets.