人臉角度判斷應用於人臉置換系統之研究

Abstract

人臉置換系統在娛樂業是相當受到重視的工作，但現今大部分的工作都是利用手動與特殊的道具輔助，在本篇論文中提出一個新的人臉置換系統，目的在於能將人臉置換的工作純粹使用影像處理來完成。 對影像中要被置換的臉 (原始臉)，先利用膚色分析與形態學(morphology)分割膚色區，由邊緣偵測(edge detection)與連通量(connectivity)分析，取出臉部候選區，利用支持向量分類機(SVM)尋找眼睛挑選出原始臉。接著配合唇色分析，利用眼睛和嘴巴的相對關係，找到嘴巴的特徵點，由嘴巴以梯度(gradient)與熵(entropy)值找出下巴特徵點，計算出下巴曲線。人臉角度判斷是先計算臉部特徵點之間的幾何關係，利用類神經網路(neural network)去判斷，而後從已建好的人臉資料庫中選擇對應角度的臉 (目標臉)去做置換。將人臉區域中的膚色部分分離出來，對原始臉與目標臉的膚色區域做直方圖比對(histogram matching)，使兩者膚色近似。再借由影像混合(image blending)與平滑化消除接縫。本論文貢獻在於，第一，建立一個角度偵測系統，範圍包含左右90度、上下20度，還有歪斜30度的情況。第二，基於角度偵測之臉部置換系統，可以改善基於五官形變之臉部置換系統在大角度時會有形變的缺點;把頭髮視為臉部特徵一併置換，可以避免一般只換臉部方式在瀏海處會有不自然連接的情形。實驗結果顯示，本研究提出的人臉置換系統，當在人臉上沒有太大的光影變化與皮膚質感相差不大的情況下，都有不錯的效果，並且相較於一般利用幾何形變的方式去做臉部置換的方式，本論文的可適用角度範圍更大，且結果更加自然。

Face replacement system plays an important role in the entertainment industries. In this paper, a new face replacement system based on pose estimation is proposed, for automatically replacing a face merely by digital image processing techniques. In the system, skin region extraction using skin-tone color model and connectivity analysis are applied to find candidates face region. For all candidate regions, eye detection using Support Vector Machine (SVM) further decide the actual face region (the source face). Moreover, combined with mouth color analysis, eye detection helps us to locate mouth and obtain features of the mouth. Feature points of the chin can be obtained by information of gradient and entropy. Facial feature points are important to estimate head pose. Then, pose estimation can be accomplished by neural network (NN) and used to choose target face. The skin color can be adjusted to be more similar by histogram matching. Finally, image blending and smoothing is used to eliminate the seam. The contributions of this thesis may be summarized as follows. First, develop an algorithm to estimate head pose covering 90 degree yaw changes and 20 degree tilt changes and 30 degree roll changes. Second, face replacement system based on warping tends to distort the face, but face replacement system based on pose estimation can improve this defect. Besides, replacing whole head (including hair region) can avoid unnatural parts between hair and face. As can be seen, the experimental results show that this face replacement system has good performance as long as there are no large variations in illumination and skin tones.