Enhancing Text Classification with the Universum

Abstract

The Universum is a data set that shares the same domain as the target problem, but does not comprise any category of interest. Recently, the concept of inference through contradictions has shown that the Universum provides a means for machine learning algorithms to encode prior knowledge into the model to improve performance. This work investigates whether text classification algorithms can benefit from the Universum when one has only a few labeled examples at hand. Additionally, this work proposes a confidence scheme to incorporate Universum into the learning process, and further devises a learning with Universum algorithm called Universum logistic regression (U-LR). The confidence scheme provides another means for machine learning algorithms to incorporate Universum into learning process. We conduct experiments on three data sets with several combinations. The experimental results indicate that the proposed method outperforms the other learning with Universum methods.