深層半監督式學習於領域調適之研究

Abstract

領域調適(domain adaptation)是透過學習一個可以使知識在領域之間轉移與共享的特徵表示(feature representation)來將分類器從來源領域(source domain)調適至目標領域(target domain)。先前大部分的研究都僅限於淺層的模型，並且分開地訓練特徵表示器(feature representation)與模型分類器(classifier)。在本論文中，我們提出一種結合深層半監督式學習與領域適應的方法。此方法是在深層類神經網絡(deep neural network)中協同訓練(co-train)特徵表示與分類模型。此外，我們不需要使用到目標領域的標籤(label)。我們將深層類神經網絡的隱藏層(hidden layers)視為特徵提取器，並採用分類與回歸建構輸出層(output layer)來實現多任務學習(multi-task learning)。具體來說，分類任務是被應用於提取來源領域的標籤信息，而回歸任務是用於尋找來源領域與目標領域的共享成分。除了上述兩個任務之外，我們還添加額外的目標函數(objective function)，以減少來源領域和目標領域在特徵空間中的差異。此構想是建立在基於特徵的領域適應方法(feature-based domain adaptation)上，希望能最小化兩個領域中有標籤數據的分布與未標籤數據的分布之間差異以及最小化兩套數據的回歸重構誤差(reconstruction error)，同時將來源領域的分類錯誤最小化。此方法的學習策略是基於多任務學習與領域適應關鍵技巧的匹配分布(distribution matching)，能在沒有目標領域標籤數據的情況下，有效地調適深層神經網絡。在圖像分類(image classification)和文句情感分類(sentiment classification)的實驗中，顯示了用於領域適應的深層類神經網絡協同訓練有卓越的能力。

Domain adaptation aims to adapt a classifier from source domain to target domain through learning a good feature representation that allows knowledge to be shared and transferred across domains. Most of previous studies are restricted to train features and classifier separately under a shallow model structure. In this thesis, we propose a semi-supervised domain adaptation method which co-trains the feature representation and pattern classification under deep neural network (DNN) framework. The labeling in target domain is not required. We treat the hidden layers in DNN as feature extraction and construct the output layer consisting of classification and regression to implement multi-task learning. Specifically, the classification task is applied to extract label information from source domain and we find the components which are shared by source and target domain through the regression task. Besides, we add extra cost to error function in order to reduce the difference between source domain and target domain in feature space. Our idea is to conduct the feature-based domain adaptation which jointly minimizes the divergence between distributions from labeled and unlabeled data in both domains and the classification errors due to the labeled data in source domain. The learning strategy based on multi-task learning and matching distributions can effectively adapt DNN without the labeled data in target domain. Experiments on image classification and sentiment classification show the superiority of DNN co-training for domain adaptation.