由連續影像自動產生說話卡通臉

Abstract

本研究主要在提供一個利用影像處理、嘴形合成、臉部特徵追蹤和語音同步等技術來產生說話卡通臉的方法。首先我們利用本論文中所提出的兩個方法自動偵測一張無表情臉之五官、臉部以及頭髮所在的位置和範圍，再細部偵測其外型給予特徵點。接著配合提出的卡通人臉模型，描繪出基本卡通臉型。同時我們挑選部分特徵點作為控制點，以簡單控制的的方式來變化產生有表情之卡通人臉。驅動卡通人臉的方式尚可分為兩種：一種是以輸入單張無表情臉，藉由分析輸入的語音和演講稿，利用本論文提出的九種基本嘴形來模擬說話卡通臉。每一個中文發音可由有一到四個基本嘴形組成，同時可配合表情使卡通更富變化；另一種則為輸入一段錄影，分離語音和多張連續影像後，藉由自動追蹤五官之變化來產生表情，再與語音合併產生說話卡通臉。最後藉由一可編輯並且開放之可擴展標記語言(XML)，亦即SVG，來達成繪圖並做同步語音輸出之效果。良好的實驗結果證明了我們所提方法確實可行。

In this study, an approach to automatic generation of talking cartoon faces by facial image analysis, moving-lip synthesis, facial feature tracking, and voice synchronization techniques is proposed. A method for creating a personal cartoon face automatically is proposed to extract the facial features from a given neutral facial image. A hierarchical bi-level thresholding method and a knowledge-based facial feature detection method are used to reach the goal. A face model of 72 facial feature points is proposed for cartoon face drawing. A concept of assigning control points is applied to animate the cartoon face more easily. Additionally, the personal cartoon face can be animated by a speech file and a script file. Nine basic mouth shapes for Mandarin speaking are proposed to synthesize the moving lips. Each syllabus may consist of one to four basic mouth shapes. Relations between each mouth shapes are analyzed in this study. Cartoon faces can also be animated by the use of sequential facial images through automatic tracking of facial features, and then the cartoon face will act like the user in the image sequence. Finally, an editable and opened vector-based XML language of W3C (World Wide Web consortium) standard-SVG (Scalable Vector Graphics) is used for rendering the cartoon face and synchronizing with speech. Good experimental results show the feasibility of the proposed methods.