由有複雜背景的連續影像中自動即時產生會說話卡通臉之研究及其應用

Abstract

本論文提出了一套自動即時產生會說話卡通臉的系統。這個系統包含了五個階段：環境學習、臉部特徵追蹤、影像特徵點轉換、錄音與動畫製作。在環境學習階段，系統會自動學習一些特徵值來輔助臉部特徵追蹤。在臉部特徵追蹤階段，系統會由連續影像追蹤眼睛和嘴形的變化，我們提出了一些臉部特徵追蹤的方法來達到此結果，除此之外更提出了一個能夠偵測轉頭的方法。在我們提出的錯誤校正方法輔助之下，臉部特徵追蹤的方法也可用於晃動的人臉上。在影像特徵點轉換階段，我們提出一個方法將追蹤到的特徵點參數轉換成相對應的人臉模型的控制點。在錄音階段，聲音將會持續被錄音及即時播放出來。在動畫製作階段，我們根據人臉模型控制點的位置產生卡通臉，並提出一個方法產生側面的二維卡通臉動畫。以這套系統為基礎，我們發展出另一套網路應用的自動即時產生會說話卡通臉的系統，並實作出二種有趣的應用。實驗結果證實本論文所提出方法之可行性。

A system for automatic real-time generation of talking cartoon faces is proposed, which includes five processes: environment learning, facial feature tracking, image feature point transformation, speech recording, and animation generation. In the environment learning process, some threshold values are learned to help tracking facial features. In the facial feature tracking process, the eyes and mouths are tracked from sequential facial images. An eye-pair tracking method and a mouth tracking method are used to reach the goal. Besides, a head turning detection method is proposed. Some error correction techniques are proposed to cause the facial tracking methods be applicable to shaking faces. In the image feature point transformation process, a method used to transform image feature points into 2D face model control points is proposed. In the speech recording process, speeches are recorded and played in real time. In the animation generation process, talking cartoon faces are rendered from the face model control points. A method for creation of lateral 2D cartoon faces is also proposed. Based on the proposed system, an automatic real-time talking cartoon face generation system for use on networks is designed. Two kinds of interesting applications on networks are also implemented. Experimental results show the feasibility of the proposed methods.