透過單一影像自動化重建場景之三維結構

Abstract

本篇論文提出了一個自動重建單張影像三維結構的方法，我們主要會將建築物表面的不同的朝向標籤出來，根據這些幾何標籤推論出影像可能的三維構造，現今的三維重建法大多數都是假設影像中的場景符合曼哈頓假設，而我們採取一個更為普遍做法，不同於曼哈頓假設，我們允許影像場景中可以擁有三個以上的消失點，如此ㄧ來可以適用於大多數的戶外場景。我們的方法主要分兩的階段，第一個階段會先由消失線初步估計影像的方向圖，接著我們會考慮影像中的資訊，如:線段、材質以及特徵點，對初估的方向圖進行最佳化，第二階段則是根據最佳化後的方向圖結果做分割，並基於物理特性的條件推論影像的三維結構，最後我們提出了一個由物理性啟發的目標函數評估我們建出來的所有結果，初步的實驗結果顯示我們的方法可以處理較為複雜的建築物表面。

This thesis presents an automatic method to label different orientations of building façades in a single image and infer its 3D structure from these geometric labels. While most of the existing works use Manhattan World assumption, our method takes a more general assumption which agrees more than three vanishing points in an image. There are two stages in the proposed algorithm. First, we estimate the coarse orientation map from vanishing lines. To find optimal orientation labels, we propose a multi-cue optimization and consider line segments, texture entropy, SIFT features of an image. Second, we segment the orientation map to irregular polygon patches and recover the 3D scene according to physics-based criteria. We propose a physics-inspired objective function to evaluate the results of 3D structures. The preliminary experiments demonstrate that the proposed method can deal with complicated façades of architectures.