針對多視角視訊合成之遮蔽效應消除演算法

Abstract

在傳統上的視訊，其觀賞的視角是由導播來決定。由於隨著顯示科技與影像擷取技術的進步，使用多台攝影機同步的來紀錄同一場景，提供觀眾更多的觀賞視角，已經不是遙不可及。而在眾多的應用當中，具備了能讓觀眾來自由變換視角的自由視角電視（Free Viewpoint TV）和自由視角視訊（Free Viewpoint Video）較具代表性。然而，為了達到讓觀眾可以自由變換視角的目的，所需要傳輸的資料量勢必相當龐大。此外，觀眾的隨選視角也有相當大的可能是在未架設攝影機的地方。所以在實作上，通常利用少數有資料的視角來進行視角合成，降低傳送的資料量。而一般解決此問題的視角合成演算法，都是以一張一張的畫面進行視角合成。在這篇論文當中，我們提出一個演算法：當在進行某張畫面的視角合成時，利用遮蔽區域附近的資訊，參考同一視角的前後的影像，來達到：在不增加所傳送的資料量之情況下，減少因為遮蔽效應而產生的影響。此外，即便對於背景會移動的影片，或者是在攝影機會移動的多視角環境之下，本演算法亦能達到不錯的效果。

In tradition, the view point of a video scene is decided by the studio director. It’s not too far away that making people choose the view points of a scene which they like. With the current camera capture and display technologies, we can record a scene into multiple video streams. Free viewpoint TV (FTV) and free viewpoint video (FVV) are the most well-known among the relative applications. In order to provide the functionality of view switch, the amount of data, compared to the traditional video, increase dramatically. Besides, the view that a user decides may locate at the view angle that is not captured. In common implementations, view synthesis is usually used. Texture and depth information along with the camera parameters are used in the view synthesis to generate virtual views. In this paper, we proposed an occlusion elimination algorithm. As view synthesis is performed, the current virtualized frame will refer to forward and backward frames to reduce the residual data required. As a result, we can provide better quality without increasing any data. In addition, this algorithm also works well even if the background is not stationary.