於多視角視訊之平順自由視角之合成演算法

Abstract

由於科技的進步，近十年來在多視角視訊的領域有更進一步的發展，此領域的應用在TV方面，以FTV﹝Free Viewpoint TV﹞及3DTV 為主。本篇論文主要是研究FTV這個領域，根據不同視角，需要不同的視角影像，因為不可能準備如此多的影像，所以是採用即時合成影像的方式，合成所需的影像。本篇文章選擇幾個固定視角，利用合成的方式撥放出這幾個固定視角的影像，當一個視角轉換到另一個視角時，如果2個視角之間相距越大，2個影像內容差異也就越大，如此的差異會產生轉換不流暢的感覺，為了解決這個問題，因此在視角轉換的過程中，使用合成的方式內插了許多影像，轉換的過程中撥放內插的影像，使得轉換看起來很流暢。我們也發現在轉換的過程中，使用者注重轉換速度大於影像品質，所以在轉換的過程中，我們可以使用品質較低但是合成速度較快的演算法，去合成內插的影像。使用不同的合成方法，會有不同的合成速度跟品質，該使用哪一種的合成方式，該內插多少張影像，使得轉換的速度和內插影像的品質達到最好的平衡，是本篇文章探討的重點。

With current technology, many advances have been made in the area of multi-view video in the last decade. There are two important applications of multi-view video on TV; one is Free Viewpoint TV, and another is 3D TV. This paper will focus on Free Viewpoint TV. To provide videos at an arbitrary viewpoint, this requirement would lead to a large number of viewpoint video sequences transmitted in FTV. It could not prepare a large number of video at all possible angles; the application of Free Viewpoint TV is limited by viewpoint video sequence. Because of this reason above, the virtual view of arbitrary view angle might have to be synthesized in real time. In this study, it offers only fixed views; we play the virtual view by synthesis in real time. We adopt view synthesis techniques to extend the navigation function and realize virtual camera positions. When the view switched from one viewpoint to another, if the distance between viewpoints is farther, the difference of the content will be bigger in the view. The difference due to that it is not fluent during switching view. To overcome these shortcomings, we interpolated some frames by synthesis in real time. It will play those frames during switching view. Switching view will be more fluent. We find a special feature. Playing view is a behavior of long time; switching view is a behavior of short time. The user wishes the time is short during switching view, because they want to watch the view of the destination. They care the speed of switching view than the quality of switching view. The primary research questions to be addressed in this paper are as follows: How do make it more fluent during view switching, what method of synthesis is the best? How many frames to interpolate? In this work, it is the key point.