在社群網路上進行2D/3D感知串流之多視角合成與編碼技術

Abstract

諸如自由視角電視及自由視角視訊之可在網路上傳輸的多視角視訊應用，隨著顯示 器、視訊擷取裝置以及編碼技術的演進與成熟而逐漸受到矚目。為了達成這些應用的自 由導覽，在有限的網路資源下通常需要景深圖來合成所需觀看之視角。為了增加傳輸和 儲存多視角視訊大量影像資料的效率，包含於MPEG-4 AVC Multiview High Profile 裡之 多視角視訊編碼也已標準化。雖然這些壓縮技術雖然提供了比單純做時間軸的預測編碼 為優的壓縮效果，但仍未將多視角視訊的特性完全利用。另外，3D Video 以及Free Viewpoint Television 在MPEG 組織裡標準化的準備動作正在活躍進行中。本子計畫的目 標，著重在同儕網路上之串流服務系統，研究發展在有限網路資源下讓使用者進行互動 立體視訊及自由視角立體視訊服務所需之關鍵技術。包含研究多視角視訊編碼技術，以 及因應異質網路資源所對應到之二維視訊至立體視訊之合成，和多視角視訊視角合成。 並針對同儕網路在多視角立體視訊串流下網路傳輸的架構及協定進行研究，結合網路編 碼的技術來使多視角立體視訊串流的傳輸更順暢。除此之外，增進使用者合作與避免惡 意使用者之攻擊的演算法也會進行研究及開發。

With the recent progress of display, capture device, and coding technologies, multiview video applications over the transmission networks such as free viewpoint TV (FTV), free viewpoint video (FVV), and stereoscopic video have been introduced to the public with growing interest. To achieve free navigation of such applications, depth information is usually required along with the video data to synthesize the desired view under the constraint of available network resource. In order to improve the efficiency of transmission and storage of huge amount of Multiview video streams, the multiview video coding in MPEG-4 AVC as the Multiview High Profile is standardized. Even though these coding techniques can offer improvements over pure temporal prediction structure, the essential features of multiview video streams can be further taken advantage of. At the same time, there are emergent standardization activities of 3D video and Free Viewpoint Television in MPEG organization that also show the great research and development interest from the experts in the related fields. The objectives of this project include the development and innovation of the key technologies that enable interactive stereoscopic video and free viewpoint television services over the peer-to-peer streaming networks, such as multiview video coding technologies and the projective transformation from two-dimensional videos to stereoscopic videos for the viewers with limited resource over the heterogeneous networks, and also Multiview video synthesis for the purposes of both rendering and compression. We will also investigate the network architectures and network protocols of peer-to-peer networks for the transmission of stereoscopic streaming of multiview videos in conjunction with the network coding to facilitate the delivery of multiview streams, while the algorithms to improve peer cooperation and to detect/prevent malicious users will also be investigated and developed.