标题: 对于 Kinect v2 RGB-D 影片之深度图品质优化
Depth Map Enhancement on RGB-D video captured by Kinect v2
作者: 林科瑜
杭学鸣
Lin, Ke-Yu
Hang, Hsueh-Ming
电子研究所
关键字: 高解析度深度图;从彩色影片取得的影像特质;Kinect 相机;High resolution depth map;Image features from color video;Kinect camera
公开日期: 2017
摘要: 随着越来越多科技需要应用到深度影像,如何取得品质好的深度影像已成为研究趋势。Microsoft 在 2010 年发表的Kinect for Xbox 360 利用深度感测器 Kinect 侦测深度距离获取深度资讯,应用在游戏上,是深度图应用的重大突破。然而,虽然 Kinect 产生的深度图在游戏上的应用是足够的,但因为低解析度与精确度问题,使得在其它应用较为困难。像是3D 虚拟视点合成中,需要精确的深度资讯才可以产生高品质的合成影像。因此,如何精确的提升深度图的品质成为一个重要议题。
在本实验中,我们采用 Microsoft 在 2014 年发表的 Kinect for Xbox Onex 获取深度资讯与彩色影像, Kinect v2 获得的深度资讯,我们可以透过提供的软体发展套件取得与捕捉到的彩色影像相同解析度的深度图,虽然相较于其它深度相机,可以获得高解析度的深度图,但仍有许多缺点,例如:彩色相机与深度相机视角不同形成遮蔽区域造成的深度图破洞、在物体边缘处错误的深度资讯、反射物件的错误深度资讯、深度资讯不稳定造成的深度影片不连续。因此本篇论文针对这些问题提出一套演算法。
我们利用 Kinect v2 同时录制彩色影片及深度影片,将捕捉到的彩色影像,在空间域和时间域做为参考,提升深度图品质。我们提出的方法利用到背景建构、不可靠区域侦测、遮蔽区域侦测、统计模型将异常深度值修正,填补正确深度资讯,提高深度图品质。最后实验比较我们提出的方法与其它方法,实验结果显示,我们的方法可以产生品质较高的深度图及深度影片。
Owing to the development of the technology, the application of depth map is becoming more and more popular. To apply the depth image a high-quality depth map is necessary. Hence, how to capture a high-quality depth map has become a major research topic. Kinect for Xbox 360 made by Microsoft Company in 2010 has depth sensors to capture the depth information by the infrared light to detect the distance between the camera and the scene. It makes a breakthrough on the application of depth map. However, the depth map generated by Kinect still has many problems such as low resolution and error depth pixel so that it is hard to use in other applications. For example, in the application of 3D virtual view synthesis, the quality of depth maps directly affect the quality of synthesized image. Having a high-quality depth map is necessary. Hence, how to enhance the quality of depth map precisely has become an important issue.
In our experiments, we use Kinect for Xbox One (Kinect v2) made by Microsoft Company in 2014 to capture the color sequences and the depth sequences. We use the color sequence as the reference on the spatial domain and time domain to enhance the quality of the depth map. We obtain the depth information by the Kinect v2 and then we produce the depth map which has the same high resolution with the captured color image by software development package (SDK). Although the depth map from Kinect v2 has the enough high resolution, there are still some problems such as depth holes caused by the occlusion region due to the different view of the color camera and depth camera, the wrong depth pixels appearing in the edge of objects, the wrong depth information in the reflector, inconsistency in depth frame due to the unstable depth information. Hence, we propose an algorithm to solve these problems and improve the depth map.
We use Kinect v2 to capture the color sequences and the depth sequences at the same time. We use the color sequence as the reference in spatial domain and time domain to improve the depth map. We propose many techniques such as background construction, unreliable region detection, occlusion region detection, the depth outlier refinement based on the statistical model, the correct depth filling to enhance the quality of the depth map. At the end, we compare our refined depth map with those produced by other previous methods. The experimental results show that the quality of our depth maps is better.
URI: http://etd.lib.nctu.edu.tw/cdrfb3/record/nctu/#GT070350257
http://hdl.handle.net/11536/140950
显示于类别:Thesis