基於立體視覺環境下即時人機互動介面之實作

Abstract

在電腦視覺的領域中，利用二維影像資訊重建三維特定座標點的指向系統之研究已行之多年。隨著電腦計算能力的進步，我們已能夠在此種基於電腦視覺的環境下實踐出即時人機互動的應用介面。本論文利用指向物的已知外觀特性，計算出指向物在二維影像中的方向向量，並且在已知的運作空間下，利用投影轉換將方向向量轉換到真實世界座標系中，並且利用攝影機中心與此向量上任兩點共面的特性在三維空間中重建一平面。在系統環境中，我們利用兩重建平面與投影平面三面共點的特性，便可重建出指向點在真實世界中的位置。我們並且發展出一組基於有限狀態機與類神經網路的控制元件，接受並且辨識使用者輸入的軌跡，並且賦予每一個軌跡一個對應的互動行為，根據使用者輸入的軌跡樣式，系統會將不同的行為反饋在介面上，進而達成人機互動的目標。本論文亦提出一些可供實作的系統雛型，以探討本系統在現實生活中的應用性。

The research of vision-based pointing systems through reconstruction of 3-D features using 2-D image information has been studied for decades. With the improvements of computing power, in recent years, one can now implement a real-time human computer interaction application under such a vision-based environment. In this paper, we develop a pointing system which tracks the pointer in two images and calculates its 3-D directional vector. With prior knowledge of some 3-D reference planes, the pointed point can be found without knowing camera’s intrinsic parameters and orientations. We develop a control unit to accept and recognize the user inputs in forms of trajectories of the pointed point based on finite state machine and neural network. The goal of developing a human computer interaction application is achieved by responding to the user inputs with proper actions through the user interface. We also implement some useful application prototypes to show the applicability of our system.