使用視訊模型從單一影像自動產生有動態嘴形動作的虛擬人臉之研究

Abstract

本論文提出一個由單一影像自動產生有動態嘴型動作的虛擬人臉之系統。此系統包含了三個流程：視訊模型分析、臉部特徵點追蹤、虛擬人臉產生。由本系統產生的動態虛擬人臉系列與所輸入單一影像中之人臉皆相同。為了產生虛擬人臉，我們提出一個含二十六個特徵點的嘴形模型。首先我們分離事先錄製的視訊模型之語音成分，再將視訊模型分解成多張連續影像。之後，再半自動地取得所輸入單一影像及視訊模型的第一張影像中的臉部特徵點。接著，我們提出了兩種嘴部狀態和三種閉嘴嘴形，以及一個影像對應技術，並用以分析視訊模型中的嘴型動作及追蹤其臉部特徵點。此技術使用相關係數求得各特徵點之最佳對應位置，並依不同嘴部狀態而動態改變視窗大小。為了取得正確的臉部特徵點，每當偵測到閉嘴嘴形時，我們即校正特徵點至正確位置。接著我們利用一個形變(morphing)技術讓所輸入單一影像及視訊模型中之嘴形動作同步化，使得虛擬人臉看起來像與視訊模型中的人一樣說出相同的話。而我們所指定的臉部控制點亦能調整虛擬人臉之嘴部大小及下巴位置，使得人臉在講話過程中看起來更加地自然。良好的實驗結果證明了本論文所提方法之可行性。

In this study, a system for automatic creation of virtual talking faces with dynamic mouth movement using a single image of a human face and a video model of a real talking face is proposed, which includes three processes: video model analysis, feature point tracking, and virtual face creation. The dynamic virtual face series created by the system is the same as the input image. First, a mouth model of 26 feature points is proposed for virtual face creation. Two mouth states and three closed-mouth shapes are proposed for video analysis to obtain mouth movements in the real-face video model. For feature point tracking, an image matching technique using correlation coefficients with dynamically changed window sizes is proposed. The window sizes are changed according to the mouth states. A technique for correction of the feature point locations of a closed mouth is proposed. A mouth shape morphing technique is used for synchronizing the mouth shapes of the input image with the video model, yielding the effect that the created virtual faces look like speaking the same words as the person in the video model. A concept of assigning facial control points is applied to create the virtual faces with scaled mouth sizes. Good experimental results show the feasibility and applicability of the proposed method.