適用於高畫質立體電視之200MHz 270fps 9視角影像合成設計

Abstract

在現今的3D電視系統以及自由視點電視系統(FTV)中，視點影像合成技術(view synthesis)可以用內差的方式產生虛擬視角的影像。然而，因為視訊解析度的提升以及多視角影像的需求，使得在設計視點影像合成技術上，遇到大量的資料運算、儲存以及高頻寬等課題。 在這篇論文中，針對上述的課題，我們提出了一個前級列運算和後級區塊運算的管線化架構。在前級，我們重新定義了新的多視角視訊架構，且提出了動態參考點轉換(dynamic reference switch warping)的方式來達到多視角影像的平行運算。此外，我們提出了動態的記憶體設計，讓前級內部的記憶體能夠適用於任何解析度甚至更高解析度的相機影像。在後級，我們提出了畫面分割的技術，讓我們在處理空洞填補(hole filling)上能夠大量節省頻寬用量。在這樣的設計與台積電90奈米的製程下，我們的硬體設計共消耗了100.7k的邏輯閘以及11.4kB的記憶體使用量。此設計在時鐘200MHz的速度下，可以達到每秒輸出270張(共九個視角，每個視角每秒可輸出30張)影像的運算速度，且輸出的影像可以達到與MPEG VSRS演算法相近的視覺品質。

View synthesis interpolates the virtual view on arbitrary viewpoints for 3DTV and Free-Viewpoint (FTV) systems. However, owing to increasing video resolution and number of output views, design of view synthesis faces the problems of rapidly increasing computations, internal buffer size and high bandwidth access. To address the problems, this thesis proposes a two-stage architecture with row-based process at the front stage and block-based process at the back stage. At the front stage, we modify the multi-view configuration and propose the dynamic reference switch warping (DRS) for parallel computation of multi-views. Additionally, we propose the dynamic buffer design and make the internal buffers size independent of the video frame size. At the back stage, we propose the block segmentation and block-based hole filling to fill the remaining holes with low bandwidth access. The proposed design are implemented with the TSMC_90nm process, which consumes 100.7k gate count and 11.4kB of memory usage and support 270fps (9 views@30fps) processing at 200MHz with similar visual quality as that of MPEG VSRS.