虛擬物品置入真實場景的擴充實境技術

Abstract

近年來，擴充實境的研究開始受到重視，相關應用也相繼問世，但目前研究發表的論文，多需要知道相機的參數，也就是需要經過相機校正的程序，但這個步驟無疑大大地限制了擴充實境的應用；因此，在本論文中，設計了一個方法，適用在未知拍攝相機參數的情況下，將一虛擬幾何物件以半自動方式擺設進入連續拍攝的影像之中。 我們所使用的方法，是透過 fundamental matrix 和 epipolar geometry 等，經由圖檔點對應而求得的投影幾何資訊，來推導出影像兩兩間的關係，進而求得虛擬物件所應該展現的投影關係。其中，我們將計算出每一張影像，定義於同一 projective space下的投影矩陣，利用該投影矩陣來進行場景的重建、虛擬物件擺設以及遮蔽問題處理等後續動作。 在系統進行之先，我們需要先行以人工方式，將虛擬物件擺設進入參考圖中，傳統的方式通常要求使用者在兩張圖中擺設，以便利用3D空間定位。在我們的方法中，使用者只需透過調整一虛擬相機之相關參數，在第一張參考圖中擺設虛擬物體，我們便可自動在第二張圖中繪製出其所應該呈現的成像結果，使用者可以觀察是否合理並滿意，如果需要的話，在於第一張圖中，繼續調整相關的相機參數，直至滿意為止。擺設完成之後，便決定了虛擬物體與真實場景之相互關係。另外，對於遮蔽之問題，我們也有一個處理遮蔽的演算法，能夠自動解決部分的遮蔽問題，使得影像看起來更真實。 我們所提出的這套擴充實境方式，在操作上很簡單，儘量減少了需要人工介入的部份。

Augmented reality technique has been receiving a lot of attention in recent years. And it is a thriving research field that can find various applications. But in most of the current research, it is needed to acquire the related information of real camera. In other words, a camera-calibration process is required. It's no doubt that such a process limits the scope of applications. Therefore, in this thesis, we propose a semi-automatic way to combine a virtual object with an image sequence taken by a camera without calibration. We derive the relation between two images through the fundamental matrix, which is obtained from the point correspondences of these two images. Furthermore, we can acquire the projection matrixes of each image under a unique projective space. By using each projection matrix, we can proceed the real scene reconstruction, place the virtual object in the real scene, and solve the occlusion between virtual object and real objects. To place the virtual object into a video sequence, we need to know the relation between the real scene and the virtual object. Typically, it requires the user to describe the pose of the virtual object in two basic images, then the 3D position of virtual object in the real scene is determined. We use a virtual camera to render the virtual object in the first basic image, then the user use the proposed object placement constrains to decide the relation between real scene and virtual object. Then the virtual object can be automatically rendered in the second image. The user evaluates the goodness of the object placement and invoke, if necessary, an iterative modification of the virtual camera projection matrix in the first image. We also have an algorithm for resolving occlusion between the virtual objects and real objects automatically. It is quite simple to operate our augmented reality system. We just try to keep the user labor work to an amount as minimum as possible.