深度資訊失真對3D合成視訊的影響及品質評估

Abstract

在虛擬視角的3D視訊編碼系統中，彩色影像和深度圖都會被壓縮並且傳送到接收端，深度圖在壓縮的過程中會被破壞，進而導致合成視訊上也出現明顯的失真，我們想要研究壓縮後的深度圖對合成視訊的影響，並開發出有效的品質測定模型去估計合成視訊的品質。 我們使用ITU/ISO國際視訊標準HEVC測試模型(HTM)來壓縮深度圖，錯誤的深度值會在物體的邊緣造成鬼影並且使物體產生不自然移動，因此，我們提出一種新的3D品質估計模型，去估計因為深度圖的錯誤而對3D合成視訊的影響。在我們的品質估計模型中，我們使用SSIM來計算圖像的基本分數，再利用影像上的特徵(邊緣、速度及深度資訊)去計算整張影像上每一個區域的比重，進而提升該區域的敏感度，最後在使用雙眼估計模型結合左右視訊的分數，並且選擇適當百分比的區塊去計算最後的分數。 為了評估我們的品質測定模型的效能，我們做主觀測試實驗，總共有30組測試影像，這些影像的壓縮及合成使用HEVC-3D的標準軟體及視角合成演算法。總計26未受測者對這些測試影片進行評分，從我們的實驗結果中可以看出我們提出的模型相較於起其他現有的模型，能更有效地估計合成視訊的主觀分數。

In the virtual-view 3D video coding system, both the RGB image data and the depth maps are compressed and transmitted to the receivers. After compression, the depth maps are distorted and may cause visible artifacts on the synthesized video. We study the visual effect of compressed depth maps on the synthesized video and develop a quality assessment model that predicts the subjective quality. We use the ITU/ISO international video standard HEVC Test Model (HTM) to compress the depth maps. The distorted depth values may lead to ghost artifacts around object edges and unnatural object motions on the synthesized video. Thus, we propose a new 3D quality metric to evaluate the quality of stereo video that may contain artifacts introduced by the rendering process due to depth map errors. In our proposed quality assessment (QA) model, we use SSIM to compute the basic score of stereo image pair; we extract the edge, motion, and depth features of stereo pairs and combine them to form local weights to increase the sensitivity of the noticeable regions. We use the binocular perception model to merge the scores of stereo pairs. We also select proper percentage of image blocks in the final pooling stage.

To evaluate the performance of our QA model, we conduct our own subject evaluation experiments. In total, over 30 video sequences were constructed using the HEVC-3D standard software including its view synthesis tool. About 26 viewers gave subjective scores on the test sequences. Our experimental data show that our model has a better match to the subjective scores when it is compared with the other existing QA metrics.