多視圖之三維景物重建視覺法

Abstract

本研究計畫預計在二年期間內發展一系列的電腦視覺技術，來研發透過多張影像來 進行場景的三維稠密式幾何重建，而這些輸入影像可能是利用許多佈置於景物上方圓頂 形架上之眾多相機所拍攝，或是利用單一相機於場景中移動所拍攝。此外這些影像或視 圖可以是已校正過或是未校正過。 第一年的研究計畫將利用單一相機來拍攝，並進行多視圖稠密式戶外場景的重建。 在此假設相機的內部參數是已知且在拍攝時固定不變。然而因為相機的外部參數是未知 的，因此需要對外部參數進行估算。在估算之前，必須先得到足夠數量的可靠對應點。 經由我們對於具有仿射不變性所擷取的特徵點中，可以得到一些唯一對應的特徵點。至 於若需要其他更多的點對應，可以針對已得的唯一對應特徵點找最近的特徵點用。利用 這些特徵點可用多視圖矩陣分解法得到相機的外部參數與特徵點的三維座標。為了排除 可能在上述相機參數估測時發生只找到的區域性的最小值，將再利用模擬退火法搭配直 交實驗設計法，以前述方法的解當做初始解來進行統計搜尋，藉以得到全域的參數估測 最佳解。此時就能使用得到的可靠相機間極線幾何關係，解決戶外人工場景中大量的重 覆性圖樣間的可能錯誤匹配。最後則如同第一年計畫進行稠密式的物體幾何模型重建。 第二年提出的計畫則為利用未校正影像進行多視圖三維重建。以今日之手持數位攝 影機進行變焦拍攝十分普遍，取得的影像就是未校正過。我們嘗試分析相機內部參數無 變動與有變動兩種影像對重建工作精準度之影響。兩種影像都先用多視圖矩陣分解法求 得投射空間的三維場景模型，再試圖用兩種可能方法將三維模型由投射空間轉換至歐基 理德空間。其中一種是利用場景知識如三組以上的明顯互相垂直線條組合資訊來做，另 一種是用自動校正技術來做。此外我們也要研究解決在三維重建過程中會遇到的影像雜 訊、遺失點與外來點等問題，以進而提昇三維重建模型的精準度。

The two-year research project addresses a series of computer vision techniques for reconstructing 3D dense scenes from plenty of images taken by either a large number of fixed cameras scattering around the scene or a single camera moving around the scene. In addition, the images can be either calibrated or uncalibrated. In the first-year project a multi-view dense outdoor scene reconstruction is conducted using a single camera, assuming its intrinsic camera parameters are known and fixed throughout the picture shooting. Since the camera extrinsic parameters are unknown in this camera set-up, we need to estimate these parameters. Before the estimation a sufficient number of reliable corresponding point pairs have to be obtained first. This is made possible through our affine-invariant interest point detector in which some uniquely matched point pairs can be found. Besides, more point pairs are obtained through the use of proximity measure on the interest points with respect to the already uniquely matched point pairs. The multi-view factorization method is used to derive the camera extrinsic parameters and 3D interest points. To deal with possible local minimum traps in the parameter estimation a global optimal solution to the parameter estimation the above solution is used as an initial solution in a stochastic search method making use of the simulated annealing procedure with the aid of the orthogonal experimental design for the solution space. Consequently, the reliable eipolar geometry relationships among the cameras can be obtained and the relationships will be used to disambiguate the similar structure patterns abundant in the outdoor man-made scene. Finally, a dense reconstruction can be done in the same way as given in the first-year project. The second-year project will address the multi-view 3D reconstruction using uncalibrated images. The uncalibrated images are quite common in the nowadays hand-held digital camera shooting during which the lens zooming function is likely activated. We shall study the effect of the change in the camera intrinsic parameters on the reconstruction accuracy. For the group of pictures captured with or without the change in the intrinsic parameters we apply the multi-view factorization method to obtain a projective 3D scene model, and then two possible ways of transforming the projective reconstruction to a Euclidean reconstruction will be proposed: one using the scene prior knowledge and one using the autocalibration. At the same time, we shall study how to deal with the image noise, missing points, and outliers as a whole during the multi-view reconstruction process so that more accurate reconstruction can be achieved.