在基於Kinect的三維point cloud中進行交互遮蔽偵測及遮蔽缺陷之缺補

Abstract

隨著camera的進步，能在Camera加上IR來取得彩色影像及深度影像 來建構三維空間模型，以視覺化效果呈現於使用者面前。三維空間模型之研究，其相關發展應用於三維城市模型(3D city Model)、三維空間資訊系統(3D spatial Information System)、房屋模型等。然而，在三維建模過程中常受些物件遮蔽之影響，導致某些區域不易呈現，因此，有效地偵測及透過平面紋理修補遮蔽是一個值得研究的課題。 與現有影像修補相比，如果將影像修補技術直接用於深度、彩色影像上來修補，深度影像的遮蔽會產生不理想結果。本研究為了修補三維模型的遮蔽，希望自動偵測遮蔽區塊並且透過三維座標系與影像座標系的轉換關係，將在三維模型上的遮蔽放到影像座標系上修補。本研究藉由平面紋理上之明顯結構進行修補後，再將紋理敷貼還原到三維模型上，以讓整體三維建模的結果更寫實。

With the advancement of cameras, it is easy to construct three dimension model from camera with IR to simultaneously get color and depth images, enhancing visual effect for users. The study in three-dimensional space model and related development have applications in urban modeling, spatial information systems and building models. However, there are usually occlusions from unwanted object in three dimension modeling. Therefore, it is worthwhile to develop effective methods for efficiently detecting and repairing occlusions in plane texture. Compared with existing image inpainting approaches, directly applied to RGBD image, depth image with occlusions may not produce satisfactory results. In order to repair occlusions on three-dimensional models, our approach aims to automatically detect occluded regions. The repair is done in the image frame of the planar structure and the result is then transformed back to the world frame. Our approach also apply texture mapping so that the plane can be rendered with better visual results.