景深感應器的3D影像合成

Abstract

指定背景顏色(Chroma keying)與場景合成(Scene Composition) 在電視與電影製作是一種流行性的技術。通常情況下，它合併從一個場景的前景與來自其它場景的背景。前景是經常發生在一個虛擬攝影棚的地面與牆壁和天花板被繪製成特定綠色，使背景可以由另一個場景很容易的做更換。現在，我們想要做同一個場景合成的工作在兩張任意影像。在這個過程中，我們需要從前景場景中提取主要對象。然後，將在背景場景中做前景的主要對象。 在本論文中，我們使用微軟Kinect2的設備來擷取前景的場景。由Kinect的2產生的深度影像有利於主要對象選取做處理。然而，所擷取的深度圖包含遮擋範圍(Disocclusion Region)與雜訊並且缺少深度像素值。我們使用中值濾波方式去除破洞(Holes)並且減少像素的缺失。 在前景擷取，我們採用普遍的大津法(Otsu’s Method)直方區域。此外，我們採用對象是三元圖(Trimap)的說明。並且在深度圖被劃分為3個區域，背景、前景和未知區域（前景和背景之間關係）。未知區域確定採用索貝爾法(Sobel Method)，其檢測前景和背景之間關係並且找出邊界值。 我們更換使用阿爾法通道(Alpha Channel)的技術為背景。因為我們推導出一個三元圖(Trimap)為前景，對於最後合成影像中的未知區域是前景與背景影像的混合影像。由於我們發現視覺效果更加生動呈現。

Chroma keying (scene composition) is a popular technique in TV and movie production. Typically, it merges the foreground from one scene with the background from another scene. The foreground is often taken in a virtual studio whose floor, wall and ceiling are painted with a specific green color, so that the background can be easily replaced by another scene. Now, we like to do the same scene composition job on two arbitrary images. In this process, we need to extract objects from the foreground scene. Then, place the object (foreground) on the background scene.

In this thesis, we use Microsoft Kinect 2 device to capture the foreground scene. The depth image produced by Kinect 2 facilitates the object extraction process. However, the captured depth map contains occlusion region and noises (missing depth pixels). We use an iterative median filter to remove the holes (missing pixels). In the foreground extraction, we adopt the popular Otsu’s method in the histogram domain. In addition, we adopt a “trimap” description of the object. A depth map is partitioned into 3 areas: background, foreground and unknown (between foreground and background). The unknown area is identified by the Sobel operator, which detects the object boundaries. We replace the background using the alpha channel technique. Because we derive a trimap for the foreground object, the “unknown” area on the final composition is the blended pixels of the foreground and background images. We found the visual results are more vivid and appearing.