基於三角形轉換映射的任意視點合成演算法

Abstract

立體影像視覺顯影技術在多媒體通訊應用領域已經成為了非常熱門的研究主題，其中又以任意視點合成技術為最主要的關鍵議題。在近年來，有許多任意視點合成演算法已經被提出，通常都是利用多重影像以及其對應的深度資訊圖來產生虛擬視點的影像以達到任意視點的效果。 在本篇論文中，我們利用計算特徵對應點架構達到較佳的三角化歪斜映射後的結果。以及為了減少不自然的視覺感受，我們同時也利用多重頻段調合技術達到較好的影像品質。在計算特徵對應點架構中，我們計算影像亮度資訊以及相對應深度資訊圖上的梯度相關性來判定是否為特徵點。然而，三角化歪斜應射後的結果往往會在合成影像的物體邊緣處產生人造的不自然現象。因此，我們利用中值濾波器以及多重頻段調合技術的概念以達到較佳的主觀視覺品質。當我們合成影像的過程中，多重頻段調合技術幫助我們對於物體邊緣處有柔和化的效果。 我們首先利用影像紋理針對前景深的部份映射至目標影像，我們稱之為TBTW演算法。比起傳統的畫素對畫素映射演算法，TBTW演算法的優點在於可耗費較少的運算資源。然而，合成影像往往特別對於物體邊緣處都會產生瑕疵現象。因此我們採用相似的流程僅對於景深圖使用三角映射的方式，我們稱之為TBDMW演算法。對於紋理合成的部份我們仍然使用傳統的畫素對畫素映射方式。在本文獻的最後，我們也會針對我們提出的演算法顯示MPEG測試影像的合成結果，實驗結果顯示我們的演算法達到與MPEG所提供的VSRS類似的效果。

The free viewpoint video technology or free-viewpoint television (FTV) is an emerging research topic in multimedia processing. View synthesis is one key enabling technique of FTV. Several view synthesis schemes have been proposed in the past a few years. Typically, it uses multiple image views and depth maps to generate the intermediate view at nearly any arbitrary viewpoint. In this study, we adopt the triangular warping technique for either texture mapping or depth map mapping. In the proposed scheme, we suggest a new feature point extraction process and include a multiband blending technique for image artifacts reduction. The new feature point extraction scheme makes the use of the correlation between the gradients of the image luminance intensity and the depth map. Sometimes, texture mapping using triangular warping may cause unnatural artifacts on the image object boundaries due to the discontinuity and the impairment of depth map. Thus, the median filtering and the multi-band blending techniques are employed to achieve better subjective image quality. The multi-band blending scheme provides smooth transitions when mixing two (left and right) synthesized images. We first apply the afore-described image warping method to the image texture synthesis. It is called Triangular-based Texture Warping (TBTW) algorithm. It requires much fewer calculations when compared with the traditional pixel-by-pixel warping methods. However, the synthesized images may contain artifacts in the complicated texture regions particularly along object boundaries. Then, we apply a similar data processing flow to the depth map warping and call it, Triangular-based Warping for Depth maps (TBDMW) algorithm. The texture synthesized part is using the traditional pixel-by-pixel warping. Often, with a simpler processing structure the proposed algorithm can provide an image quality comparable with that produced by the more complex MPEG reference software. In this study, we also investigate the effect of the triangular number and the multi-band blending technique.