具強韌細緻架構之可調視訊編碼演算法

Abstract

MPEG-4標準委員會制訂了細緻可調視訊編碼以進行視訊串流及廣播。細緻可調視訊編碼的加強層使用了幀內預測以及位元層編碼，使其位元流能被截斷於任意位置並提供了細緻的影像畫質調節。因為缺乏幀間預測，MPEG-4細緻可調視訊編碼之加強層有良好的容錯能力，卻有較差的編碼效率。本論文提出了新的技術以加強幀間預測及編碼效率，同時仍保有良好的錯誤容忍度。這些技術並已被正制定中的H.264/AVC可調視訊編碼標準所採用。 本論文首先提出了強韌細緻可調視訊編碼，利用加強層資訊以及滲漏式預測來改善幀間預測效果。這個方法利用兩個參數，位元層數目 □（介於0與最大位元層數目之間）及滲漏預測係數 □（介於0與1之間），來控制加強層參考幀的產生。□ 與 □ 可在不同幀之間調整，以在加強壓縮效率及降低錯誤飄移之間取得平衡。本方法提供了整體且彈性的架構以利更進一步的最佳化調整。加強層的資訊同時也能用來幀間預測基礎層，以進一步加強壓縮效率。實驗結果顯示，在MPEG-4的測試條件下，本方法最多提供了超過4dB的PSNR改善。 本論文更進一步提出了堆疊式強韌細緻可調視訊編碼來改善強韌細緻可調視訊編碼。堆疊式強韌細緻可調視訊編碼簡化了強韌細緻可調視訊編碼的架構，並將其擴展為多層堆疊架構。堆疊式強韌細緻可調視訊編碼可針對不同應用最佳化在多個操作點，並仍然保持強韌細緻可調視訊編碼的細緻性與容錯能力。我們同時提出一個以宏塊為基礎，最佳化選擇 □ 的方法來加強壓縮效率。我們並且提出一個單加強層迴圈解碼結構以簡化解碼端的複雜度。實驗結果顯示，相較於強韌細緻可調視訊編碼，堆疊式強韌細緻可調視訊編碼提供了 0.4至3.0dB 的PSNR改善。堆疊式強韌細緻可調視訊編碼已被MPEG委員會審閱過，並在可調視訊編碼的Call for Evidence 競賽中被評比為最佳技術之一。 基於前述提出之滲漏式預測以及堆疊式架構，我們再進一步提出強韌可調視訊編碼，以同時提供在影像大小、播放幀數、以及影像畫質三個方面的可調視訊編碼。在影像大小以及影像畫質的可調性上，我們提出一個只增加有限周邊資訊的彈性層間預測方式，以去除層間的冗餘資訊。在影像畫質的可調性上，我們同時支援非細緻可調及細緻可調視訊編碼。其中，在細緻可調視訊編碼的部分，我們擴展了H.264/AVC中的CABAC技術，使其能提供位元層編碼及細緻可調性。在播放幀數的可調性上，我們提出一個能減少解碼端影像暫存記憶體的實現方法。相較於H.264/AVC可調視訊編碼標準，強韌可調視訊編碼提供的PSNR改善在-0.7dB至+0.8dB之間。 總結，本論文提出的強韌細緻可調視訊編碼及堆疊式強韌細緻可調視訊編碼顯著的改善了MPEG-4細緻可調視訊編碼的壓縮效果，並仍保持良好的細緻性及容錯能力。此技術也被採用於H.264/AVC可調視訊編碼標準。基於前述提出之技術，我們進一步提出強韌可調視訊編碼以同時提供在影像大小、播放幀數、以及影像畫質三個方面的可調性。此技術並提供了能與H.264/AVC可調視訊編碼標準相抗衡的壓縮效率。最後，我們還建立了一個影像串流模擬架構以展示可調視訊編碼的應用。

The MPEG-4 committee has defined the MPEG-4 Fine Granularity Scalability (FGS) Profile as a streaming video tool. The MPEG-4 FGS enhancement layer is intra coded with bitplane coding. It can be truncated at any location to provide fine granularity of reconstructed video quality. The lack of temporal prediction at the MPEG-4 FGS enhancement layer leads to inherent robustness at the expense of coding efficiency. In this dissertation, we propose novel techniques to improve the temporal prediction at the enhancement layer so that coding efficiency is superior to the MPEG-4 FGS. The proposed techniques are also adopted in the developing H.264/AVC SVC. We propose the Robust FGS (RFGS) that utilize enhancement layer information and leaky prediction technique to improve the temporal prediction efficiency. Our approach utilizes two parameters, the number of bitplanes □ (0 □ □ □ Maximal number of bitplanes) and the amount of predictive leak □ (0 □ □ □ 1), to control the construction of the reference frame at the enhancement layer. These parameters □ and □ can be selected for each frame to provide tradeoffs between coding efficiency and error drift. Our approach offers a general and flexible framework that allows further optimization. The enhancement layer is also used to predict the base layer for further improvement. Experimental results show over 4 dB PSNR improvements in coding efficiency using the MPEG-4 testing conditions. We further present Stack Robust FGS (SRFGS) to improve the RFGS performance. SRFGS simplifies the RFGS architecture and extends it into multi-layer stack architecture. SRFGS can be optimized at several operating points to meet the requirement for various applications, while maintaining the fine granularity and error robustness of RFGS. An optimized macroblock-based alpha adaptation scheme is proposed to improve the coding efficiency. A single-loop enhancement layer decoding scheme is proposed to reduce the decoder complexity. Simulation results show that SRFGS improves the performance of RFGS by 0.4 to 3.0 dB in PSNR. SRFGS has been reviewed by the MPEG committee and ranked as one of the best algorithms in the Call for Evidence on Scalable Video Coding. Based on the proposed leaky prediction and stack structure, we further propose the Robust Scalable Video Coding (RSVC) to support spatio-temporal and SNR scalability simultaneously. To remove the inter-layer redundancy, a flexible inter-layer prediction with limited overhead is proposed for spatial and SNR scalability. For SNR scalability, both coarse granularity scalability (CGS) and FGS are supported. The H.264/AVC CABAC is extended to support the bitplane coding and FGS. A lower Decoded Picture Buffer (DPB) requirement method is used to implement the temporal scalability. The simulation results show that we have -0.7dB to +0.8dB PSNR difference comparing with the H.264/AVC SVC. In conclusion, the proposed RFGS and SRFGS architectures significantly improve the coding efficiency of MPEG-4 FGS, while still maintaining the fine granularity and error robustness. The proposed ideas have been adopted in H.264/AVC SVC. Based on leaky prediction and stack structure, we further propose RSVC to support spatio-temporal and SNR scalability simultaneously. RSVC provides comparable performance against H.264/AVC SVC. Finally, we develop a video streaming architecture for mobile WiMAX to show an application scenario of scalable video coding.