DEPTH MAP ENHANCEMENT ON RGB-D VIDEO CAPTURED BY KINECT V2

Abstract

The quality of depth map is one key factor contributing to the quality of 3D video and virtual reality (VR) rendering. In this study, we use RGB-D camera (Microsoft Kinect for Windows v2) to capture the color sequences and depth sequences as our system inputs. The captured depth map contains various noises and artifacts in addition to the occlusion regions. We use the color sequences in both spatial domain and time domain to improve the quality of the depth map. Our main contributions are alignment between color and depth images and reducing artifacts in the reflection regions. Several techniques are adopted, modified, and re-designed such as moving object compensation, unreliable depth pixel detection, and locally adaptive depth pixel refinement algorithm. The experimental results show that the quality of the depth map is significantly improved.