具深度資訊的多視角、可適性之次世代視訊編碼其計算平行度與記憶體管理硬體關鍵技術研究

Abstract

邇來有三個由H.264衍伸的視訊編解碼重要議題，分別為如何產生具有立體感的影 片、如何產生流暢的即時視訊、以及如何產生可選擇觀賞視角的影片。相關的視訊技術 為：

1.多視角視訊編碼 MVC (Multi-view Video Coding)，

2.可適性視訊編碼 SVC (Scalable Video Coding)，

3.具深度資訊多視角視訊MVD (Multi-view plus Depth)

對應的硬體關鍵技術則為：一、發掘編解碼流程上的資料平行度以及獨立程序以利向量 及管線化平行計算；二、有效率地配置、管理記憶體；三、設計兼容多種編解碼格式的 硬體架構。

本團隊於圖形及視訊處理已累積六年經驗，提出本三年期研究計晝的目的，是基於 既有經驗，嘗試具實用需求的新興議題。研究内容如下：

【一】 MVC編解碼的平行計算與記憶體管理架構。研究項目有：

A1.降低motion compensation解碼計算相依度以利平行計算的prediction structure ；

A2.提高解碼效率的internal buffers架構；

A3.就不同cameras配置的相容平行計算及internal buffers架構。

【二】整合SVC與MVC的編解碼平行計算與記憶體管理架構。研究項目有：

B1.整合 inter-views 與各 inter-layers 的高效率 prediction structure ；

B2.降低 motion compensation 中 layers 間或 views 間計算相依度的 prediction structure ； B3.就不同cameras配置的相容平行計算及internal buffers架構。

【三】 整合MVC與MVD的編解碼平行計算與記憶體管理架構。研究項目有：

C1. MVD 中 layer extraction, layer projection 以及 hole filling & filtering 三個步驟的管線化 設計，

C2.將MVC加入MVD解碼的管線化設計，

C3.在layer projection後空洞不增加的條件下，減少original views個數以減少儲存需求 與MVC解碼計算量。

Recently, three issues extended from H.264 are widely discussed. These are: How to generate 3D video, How to generate streamlined real-time video, and How to generate view-point selectable video. Involved video coding techniques are:

1. Multi-view video coding MVC,

2. Scalable video coding SVC, and

3. Multi-view plus depth video coding MVD.

And related key hardware technologies are first, to exploit data parallelism and independent processes in the flow of video encoding/decoding, to enhance vector and pipelined parallel computing; Secondly, to effectively equip and manage the assorted memories; and lastly, to design hardware architecture that can cope with many codec formats.

This research team has six years of experience in studying graphics and video computing. The purpose of this three-year research proposal is to try to solve those emerging video codec issues with practicality using hardware techniques, based on the team’s research experience. Research directions include:

[1] MVC codec parallel computing and memory management architecture. Topics include:

A1. Prediction structure that can reduce dependencies in motion compensation decoding to facilitate parallel computing;

A2. Architecture of internal buffers to enhance efficiency if decoding;

A3. Adaptive parallel computing and internal buffers architecture for various camera arrangements.

[2] Parallel computing and memory management architecture for integrated MVC and MVD codec. Topics include:

B1. Effective prediction structure considering both inter-views and inter-layers;

B2. Prediction structure that can reduce dependencies between layers and views in motion compensation;

B3. Adaptive parallel computing and internal buffers architecture for various camera arrangements.

[3] Parallel computing and memory management architecture for integrated SVC and MVC codec. Topics include:

C1. Pipeline design of the three steps—layer extraction, layer projection, and hole filling and filtering—in MVD;

C2. Pipeline design of MVD integrating MVC decoding;

C3. Under the condition that the number of holes does not increase after layer projection, reduction of original views to relieve storage requirement and computation in MVC decoding.