基於深度卷積信念網路之人臉視角轉換技術

Abstract

這篇論文主要是在探討如何從人臉影像中找到比較好的特徵表現，並且藉由連接特徵表現來建構有對應關係的人臉影像之連結，此連結被建立起來後將可以做到人臉視角的轉換。此一做法的初始想法是因為我們發現當一個物體移動或是轉動很緩慢時，這個物體的特徵表現往往只會緩慢的改變。如果我們能夠把物體特徵改變的方式給學習下來並用模型去加以模擬，就可以利用這個改變的機制去做到人臉視角的轉換。另一方面，我們使用深度捲積信念網路當成萃取影像特徵的原因是因為它具有兩個很重要的特性。第一，當物體移動時，用來代表此物體的特徵也會隨著物體移動而作空間上相對應的移動。第二，當物體在小範圍空間內有些微的移動時，深度捲積信念網路仍然可以用相同的特徵來描述此物體，並不會受這些細微變動的影響。在本論文中，我們使用的學習演算法是一種非督導式學習，這個方法叫做預先學習。經過預先學習後的模型會是一個生成模型，這個模型會是資料數據與隱藏的狀態的交互機率分布。因此給予模型一張人臉的影像，這個模型將可以從與這筆資料有關聯性之模型的隱藏狀態中去轉換出一個不同視角的人臉。

In this thesis, we aim at finding a better way of representing and connecting related human face images using the learning approach in convolutional deep belief networks (DBN). Since images are connecting with corresponding representations, it is possible for the convolutional DBN to infer a human face image with a view angle by a given image with the same human face from another view angle. The proposed methods are shown to work well due to the fact that the features detected on an image of an object in different movements are highly correlated. If patterns of feature changes could be modeled by the deep architecture, warping of human face view can be realized. Besides, the reason of using the convolutional deep belief nets as the feature extractor is that they have translated representation and translation invariant properties, which renders a more robust model to translated data. The proposed training algorithm is an unsupervised learning called pre-training. After pre-training, the model becomes a generative model specifying a joint distribution of all data and hidden states. Therefore, with a given image of human face, the model can infer a warping of the face from correlation in hidden states.