多張校正影像視圖之三維人臉建構法

Abstract

本論文的目的在於使用多台已校正相機參數的相機影像，利用影像一致性(photo consistency)的概念以建構出三維人臉模型。其中photo consistency高低的評估函數(fitness function)使用的是影像區塊間的normalized cross correlation (NCC)。本論文建構三維平面區塊是以最佳化photo consistency為目標，使用particle swarm optimization (PSO)演算法尋找NCC值較高的三維平面區塊。本論文使用PSO演算法在平面區塊的二個角度及深度的三維空間做進行最佳化。為了使整個方法有效率的進行且重建得更正確，我們可以預先尋找多視圖的對應點以估計人臉與相機的深度，接著再讓PSO演算法在估出的深度範圍附近尋找能產生較高NCC值的平面區塊。 而由於建構出來的平面區塊可能含有錯誤的情況，此外建構出來的平面區塊很可能不完全連續在一起，所以需再使用Poisson surface reconstruction去掉雜訊並重建出一個完整連續的人臉mesh模型。模型完成後再加以貼圖，整個三維人臉模型就完成了。

This thesis proposes an algorithm using the concept of photo consistency for reconstructing 3D face model from multiple calibrated camera images. The fitness function for photo consistency is the normalized cross correlation (NCC) between the planar image patches across the multiple views. The reconstruction of the planar patches is by means of optimizing the photo consistency using the particle swarm optimization (PSO) algorithm. The optimization process is carried out in the 3 dimensional spaces defined by the two rotational angles of a planar patch and the depth value of the patch. We can train the matched points across the multiple views in advance, and estimate the initial depth of the matched points. After knowing the depth range of the face, the PSO optimization process can be initialized in a smaller and more accurate range. Since the reconstructed planar patches may have error, and the reconstructed patches may not be bounded, Poisson surface reconstruction is used to construct the final mesh model. After building the 3D bounded face mesh model, the face model is finished with the image texture mapping.