前瞻性虛擬互動3-D人機介面的系統單晶片電路及系統軟體設計

Abstract

本計畫的主要目的是將過去立體電腦視覺所用的多通道視訊分析處理技術，結合 新一代的立體顯示螢幕來設計一套虛擬3-D 人機互動介面系統。這套系統可以用在消 費性電子產品中。使用者可以憑空觸按三度空間中的虛擬按鈕來啓動應用程式的功 能，或是憑空撥動旋轉三度空間中的繪圖模型。比起目前的單點或多點式觸控螢幕， 這套系統較有彈性，也較不易弄污或毀損螢幕。而比起類似的虛像投影式人機介面系 統，本系統可以做的比較輕薄。本計畫提出的人機介面系統設計需要採用許多立體電 腦視覺的演算法。然而過去電腦視覺演算法往往需要利用高速工作站進行複雜的多頻 道視訊分析，而且取像用的攝影機也要有較高的訊雜比及較低的光學失真。因此這些 昂貴技術無法直接用在消費性電子產品中做為虛擬3-D 人機互動介面系統的關鍵元 件。本子計畫的研究重點如下。首先是開發新的演算法，能利用雜訊較高、缺乏精確 同步取像機制的多通道低價小型CMOS 攝影機來進行3-D 物體定位、立體移動量估 測、移動物體切割辦識等工作。其次是設計高效率的系統單晶片架構來執行這些演算 法來配合虛擬3-D 人機互動介面系統的應用。本計劃預期開發的關鍵技術包含：適合 作成加速電路的多視點影像前處理演算法、多視點攝影機參數快速校正演算法、3-D 影像式虛擬觸控點偵測及追蹤演算法、及3-D 人機介面系統單晶片架構開發及設計。 在三年後我們可以在一個嵌入式平台進行即時系統展示。

The goal of this project is to design a virtual 3-D man-machine interface (MMI) based on computational stereo vision techniques and new generations of stereo display LCD panel. The target applications of the system are for consumer electronic devices. A user of the virtual 3-D MMI system can see virtual graphical interface components such as the buttons, menus, etc., floating in the air. The user can push or manipulate these 3-D virtual objects and obtain proper response from the system. Comparing with existing touch screen systems, the proposed system is more flexible (since we have 3-D objects instead of 2-D objects for UI) and is easier to maintain the clarity of the LCD screen. Comparing with the projection-based 3-D MMI systems, the depth resolution is much better and the size of the device can be much thinner. Hence, the proposed system can be used in a portable device. The interactivity of the proposed MMI system requires the detection and triangulation of human hand using a pair of stereo cameras. However, computational stereo vision algorithms are usually very computationally expensive. Therefore, in this project, we will develop accelerating circuits to increase the performance of an embedded device for the tasks. In addition, traditional algorithms requires the use of high quality stereo cameras with little optical and sensor distortions to acquire the video inputs. In this project, we will develop new algorithms that can calculation depth information based on the input from multiple inexpensive, noisy, unsynchronized CMOS video cameras so that the technology can be adopted by inexpensive consumers’electronic devices. In this project, we plan to develop the following technologies: multi-vew video processing accelerator circuit, fast multi-view camera calibration algorithm, detection and tracking of user input for image-based 3-D virtual touch screen, and SoC architecture for 3-D MMI. We expect to have a live demonstration of the proposed design using an embedded development board by the end of the three-year project.