使用多重攝影機影像的立體視差估算

Abstract

Free viewpoint TV (FTV) 是一個很新的立體影像技術，它可以由已知的multiple images合成虛擬視點的影像。藉著校準過的攝影機陣列所截取到的空間中景物的影像，FTV系統幾乎能夠合成任何位置所能拍攝到的影像，因此它可以讓觀眾自由的選擇自己所喜歡的視角去欣賞影片。而在FTV系統的實務上，一種快速的方式是以ray-space representation來當做image-based rendering技術的基礎。 FTV系統一個很重要的基本原素就是估算multiple images中物體的視差值，而這也就是本論文所探討的焦點。在本論文中，我們的目標是發展準確的視差估算技術以增加image-based rendering的可靠度。我們發現，四台攝影機的架構比起傳統立體影像研究中常見的兩台攝影機架構，在視差估算以及image-based rendering方面提供了較多有用的資訊，因此我們將兩台攝影機的架構推廣為四台攝影機。 為了增進遮蔽區域以及無紋理區域的視差估算的準確度，我們提出了單邊視窗對立體匹配方法。以及採用了動態規劃演算法求得整體上較佳的視差值。我們在Middlebury College提供的包含了擁有視差圖的ground truth的立體影像資料上測試我們的演算法的視差估算能力。並且以blender合成的虛擬場景的multi-camera影像來實驗我們的四台攝影機架構的image-based rendering演算法，結果顯示這樣的做法有不錯的成效，值得進一步發展。

Free-viewpoint TV (FTV) is a novel technique that can render a virtual scene based on the given multiple images and thus it allows people to choose their favorite viewpoint freely. The 3-D objects are captured by a calibrated camera array. The suggested FTV system can synthesize the projected images of these 3-D objects from nearly any virtual camera position. In practical implementation, researchers use the image-based rendering technique based on the ray-space representation. One essential element in the FTV synthesis procedure is to identify the object disparity information, which is the focus of this thesis. We adopt the ray-space representation and develop accurate matching techniques to increase the reliability of image-based rendering. We find that, comparing to the traditional 2-carmea stereo set-up, a 4-camera structure provides a much better results on disparity estimation and thus image rendering. To improve the accuracy on estimating the correct disparity values in the occluded and non-textured regions, we propose the single-sided window pair method. Also, we employ the dynamic programming algorithm to find the global optimal disparity estimate. We test our algorithms on the Middlebury College stereo image data sets with ground-truth disparity values. Also, we examine the 4-camera algorithm on the synthesized multi-camera scenes. The results are very encouraging.