基於影像之3D物體重建

Abstract

在這篇論文中我們提出改善3D物體表面重建以及場景顏色重建的相關新技術。對於3D物體的表面重建，我們主要是採用photometric stereo的立體重建技術方法，這個方法是基於多張不同光源方向但是相同物體場景的影像來重建影像中物體的深度(depth)資訊，依據以往的研究，反射(specular)成分與散射(diffuse)成分的影響是必須被一起考慮，因此我們將改善的方法分成兩個階段來完成，第一階段我們先針對反射成分與散射成分的結合比例問題提出結合兩個對稱型的類神經網路來取代傳統利用固定比例結合的方法，透過我們提出的類神經網路學習機制，最後可以得到影像中每一個點對應到表面散射及反射成分最適當的混合比例及表面法線向量，進一步利用表面的法線向量，我們使用Enforcing Integrability方法重建出影像中物體表面的深度資訊，經實驗驗證我們所提出的類神經網路的方法比傳統固定比例結合的方法能更有效的重建3D表面的結果。 為了更有效的結合反射成分與散射成分，第二個階段我們提出一個單一非線性(nonlinear)混合模型同時用來表現物體表面的散射及反射成分，我們不需要從這個模型中將這兩個成分分開，透過非線性獨立成成分分析(independent component analysis)技術的計算，我們最後可以從這單一非線性混合模型中分解出對應到影像中每一點的表面法線向量，接著利用表面法線向量可以重建出影像中物體表面的深度資訊，經實驗驗證我們知道物體表面的反射成分與散射成分是可以用單一模型來表現，其次證實非線性獨立成分分析技術可以有效的應用在此一問題上。 對於3D物體的色彩重建，我們也提出一個基於類神經網路的影像中外在環境光源的估測方法，這個估測方法是依據影像上所有顏色在色域圖上的統計直條圖分佈來估測影像外在光源，我們以影像色域統計圖分佈的中心值當作類神經網路的輸入，透過BP學習法則學習過的類神經網路，我們可以得到相對應於光源參數的輸出，利用這些估測的光源參數，我們能完成色彩的重建，從實驗的結果可以驗證，不管是定量或定性的實驗顯示，我們所提出的方法確實能正確且有效快速的完成光源的估測及色彩的重建。

In this thesis, we propose three new techniques to improve the surface reconstruction and color reconstruction of 3D objects. For the surface reconstruction of 3D objects, photometric stereo is able to estimate local surface orientations by using several images of the same surface which are photographed from the same viewpoint but under the illuminations from different directions. According to previous researches, a successful reflectance model for surface reconstruction of 3D objects should combine two major components, the diffusion and specular components. As a result, in this thesis, we categorize the improvement by our methodology into two stages. In the first stage, a new neural-network-based adaptive hybrid-reflectance model is proposed for combining the diffusion and specular components automatically. The supervised learning algorithm is adopted and the hybrid ration for each point is updated in the learning iterations. After the learning process, the neural network can estimate the normal vector for each point on the surface of 3D objects in an image. The enforcing integrability method is applied to the reconstruction of 3D objects by using the obtained normal vectors. The experimental results demonstrate that the proposed network estimates the point-wisely adaptive combination ratio of the diffusion and specular intensities such that the different reflection properties of each point on the object surface are considered to achieve better performance on the surface reconstruction. In the second stage, we further propose a new nonlinear reflectance model consisting of diffusion and specular components for modeling the surface reflectance of 3D objects in an image. Unlike the previous approaches, these two components are not separated and processed individually in the proposed model. An unsupervised learning adaptation algorithm is developed to estimate the reflectance model based on image intensities. In this algorithm, the post-nonlinear independent component analysis (ICA) is used to obtain the surface normal on each point of an image. Then, the 3D surface model is reconstructed based on the estimated surface normal on each point of image by using the enforcing integrability method. The results clearly indicate the superiority of the proposed nonlinear reflectance model over the other linear hybrid reflectance model. The experimental results demonstrate that the post-nonlinear ICA method can be used in the problems of surface reconstruction. For color recovering of 3D objects, a new neural-network-based algorithm for surrounding illumination estimation of image scenes is proposed. This estimation is based upon the chromaticity histogram of a color image, which is obtained by the accumulation of CIE chromaticity values corresponding to all the colors in the image. A neural network with a BP learning algorithm is used to model the nonlinearly functional relationship between the central values of the chromaticity histogram and the coefficients of illuminant functions. The trained BP network can then be used to estimate the spectral power distribution of the surrounding illuminant. By substituting this illuminant estimates into the finite-dimensional linear model of surface reflectance, the colors of the image can be recovered with the standard illuminant (such as D65) for color constancy. The experimental results show that the new algorithm outperforms the existing popular compared algorithms, both in quantitative error indices and in qualitative visual perception.