針對多視角視訊合成之景深修正演算法

Abstract

隨著近年來顯示科技、影像擷取及壓縮技術的發展，自由視角電視﹝Free Viewpoint TV﹞、自由視角視訊﹝Free Viewpoint Video﹞以及立體視訊﹝Stereoscopic Video﹞等多視角視訊的應用紛紛發表且受到關注。以上應用為了達到能夠自由變換視角的功能，除了原本的視訊資料之外必須加上景深的資訊。雖然已經有眾多演算法可以預估景深，但是對於預估正確的景深仍有許多挑戰。在這篇論文中，我們使用立體的攝影機設置進行景深估計以節省景深預估所使用的資源並且提出一個景深修正演算法針對景深錯誤的像素進行修正。我們提出的方法先將景深圖中的像素區分為可信的﹝reliable﹞及不可信的﹝unreliable﹞，再針對不可信的像素進行景深修正以得到品質更好的景深圖。除了景深修正演算法，我們也提出一個可信度加權視角合成演算法。最後，我們會以合成視角的品質評估修正過後的景深圖，並且會以主觀的及客觀的實驗結果進行比較。

With the recent progress of display, capture device, and coding technologies, multi-view video applications such as free viewpoint TV (FTV), free viewpoint video (FVV), and stereoscopic video have been introduced to the public with growing interest. To achieve free navigation of such applications, depth information is required along with the video data. There have been many research activities in the area of depth estimation; however it still poses us great challenge to estimate accurate depth map. In this paper, we use stereo camera setting to estimate depth map in order to save the resources to be used in depth estimation and propose a depth refinement algorithm for recovering bad depth pixels. The proposed algorithm classifies the pixel-wise depth map into two categories, one is reliable and the other is unreliable, followed by the depth refinement algorithm for those pixels with unreliable depth values. Except for the depth refinement algorithm, we also propose a reliable weighted view interpolation algorithm. At last, the refined depth map is evaluated by the quality of the synthesized views subjectively and objectively.