適用於三維場景重建之寬基線立體影像匹配

Abstract

在本篇論文中，我們提出了可用於三維場景重建的寬基線立體影像對應系統。在此系統中我們使用了三台未校準的相機，且這些相機被設置的很分散。由於大角度所造成的扭曲和遮蔽的現象，使我們的匹配任務變得更加困難。 為了得到一個準確的對應，我們採用了隨機森林來克服影像因大角度差而造成的扭曲，並使用修改過的Histogram of Oriented Gradients（HOG） 配合條件隨機場（CRF）來求解，結合了這兩種方法不僅可修正錯誤的對應關係還可處理一些大角度遮蔽的問題。 獲得匹配點以後，可經由Bundle Adjustment（BA）求出世界座標點雲及相機參數。接下來，我們加入了一個分割的方法(spectral matting)，讓我們可以根據像素空間和色彩之間的關係來重新定義點雲的世界座標。 最後再根據點雲來建立出一個立體的三維模型。

In this thesis, we present a wide-baseline stereo system for 3D scene reconstruction. We implement our system with multiple un-calibrated cameras which are set widely. The main challenge of the system lies on how to match image pairs at wide-baseline, in which there appear large perspective distortions and large occlusion areas between images. In this research, we attempt to tackle the problem based on machine learning and optimization techniques. In order to match image more accurately, we apply random forest to overcome large perspective distortions, and add Conditional Random Field (CRF) with modified Histogram of Oriented Gradients (HOG) to solve the matching problem. Combining conditional random field with random forest can not only correct error correspondences but handle some occlusions. After getting matched points, we use these correspondences to find a 3D point set and camera matric by bundle adjustment (BA) that minimizes re-projection error. Then, we use the idea of spectral matting to refine the 3D point set. Finally, we build a 3D model with the refined point set.