對於 Kinect v2 RGB-D 影片之深度圖品質優化

Abstract

隨著越來越多科技需要應用到深度影像，如何取得品質好的深度影像已成為研究趨勢。Microsoft 在 2010 年發表的Kinect for Xbox 360 利用深度感測器 Kinect 偵測深度距離獲取深度資訊，應用在遊戲上，是深度圖應用的重大突破。然而，雖然 Kinect 產生的深度圖在遊戲上的應用是足夠的，但因為低解析度與精確度問題，使得在其它應用較為困難。像是3D 虛擬視點合成中，需要精確的深度資訊才可以產生高品質的合成影像。因此，如何精確的提升深度圖的品質成為一個重要議題。 在本實驗中，我們採用 Microsoft 在 2014 年發表的 Kinect for Xbox Onex 獲取深度資訊與彩色影像， Kinect v2 獲得的深度資訊，我們可以透過提供的軟體發展套件取得與捕捉到的彩色影像相同解析度的深度圖，雖然相較於其它深度相機，可以獲得高解析度的深度圖，但仍有許多缺點，例如：彩色相機與深度相機視角不同形成遮蔽區域造成的深度圖破洞、在物體邊緣處錯誤的深度資訊、反射物件的錯誤深度資訊、深度資訊不穩定造成的深度影片不連續。因此本篇論文針對這些問題提出一套演算法。 我們利用 Kinect v2 同時錄製彩色影片及深度影片，將捕捉到的彩色影像，在空間域和時間域做為參考，提升深度圖品質。我們提出的方法利用到背景建構、不可靠區域偵測、遮蔽區域偵測、統計模型將異常深度值修正，填補正確深度資訊，提高深度圖品質。最後實驗比較我們提出的方法與其它方法，實驗結果顯示，我們的方法可以產生品質較高的深度圖及深度影片。

Owing to the development of the technology, the application of depth map is becoming more and more popular. To apply the depth image a high-quality depth map is necessary. Hence, how to capture a high-quality depth map has become a major research topic. Kinect for Xbox 360 made by Microsoft Company in 2010 has depth sensors to capture the depth information by the infrared light to detect the distance between the camera and the scene. It makes a breakthrough on the application of depth map. However, the depth map generated by Kinect still has many problems such as low resolution and error depth pixel so that it is hard to use in other applications. For example, in the application of 3D virtual view synthesis, the quality of depth maps directly affect the quality of synthesized image. Having a high-quality depth map is necessary. Hence, how to enhance the quality of depth map precisely has become an important issue. In our experiments, we use Kinect for Xbox One (Kinect v2) made by Microsoft Company in 2014 to capture the color sequences and the depth sequences. We use the color sequence as the reference on the spatial domain and time domain to enhance the quality of the depth map. We obtain the depth information by the Kinect v2 and then we produce the depth map which has the same high resolution with the captured color image by software development package (SDK). Although the depth map from Kinect v2 has the enough high resolution, there are still some problems such as depth holes caused by the occlusion region due to the different view of the color camera and depth camera, the wrong depth pixels appearing in the edge of objects, the wrong depth information in the reflector, inconsistency in depth frame due to the unstable depth information. Hence, we propose an algorithm to solve these problems and improve the depth map. We use Kinect v2 to capture the color sequences and the depth sequences at the same time. We use the color sequence as the reference in spatial domain and time domain to improve the depth map. We propose many techniques such as background construction, unreliable region detection, occlusion region detection, the depth outlier refinement based on the statistical model, the correct depth filling to enhance the quality of the depth map. At the end, we compare our refined depth map with those produced by other previous methods. The experimental results show that the quality of our depth maps is better.