應用於三維互動顯示器之手勢辨識與追蹤及深度感測系統

Abstract

本論文提出一結合深度感測之手勢追蹤及手勢辨識系統並將其應用於三維互動顯示器。整個研究逐步完成手勢追蹤、手勢辨識與深度感測系統之多樣性的結合演算法的實現。此感測系統包含兩個攝影機，分別用於深度感測及手勢追蹤與辨識，其目標在於不僅擁有一般二維式的手勢追蹤即辨識系統外，還增加了第三維(深度)的資訊，而提升應用的範圍及真實度。整個研究可以分為三個階段：（1）手勢追蹤；（2）手勢辨識；（3）深度感測互動。第一部分使用Haar-like特徵做特定的手勢偵測，其目的為用於告知電腦開始追蹤，接著搭配使用平均位移(Mean-Shift)演算法及卡爾曼濾波器(Kalman filter)已達成追蹤程序。第二部分於追蹤的子視窗中做手勢的辨識，首先將子視窗中的圖像做色彩模型的變換後找出膚色部分，再經由形態學的膨脹與侵蝕將雜訊濾除，搭配使用主要成份分析(Principal Component Analysis)演算法將圖像降維，並取出圖像之特徵向量(Eigenvector)與特徵值(Eigenvalue)建立低維度的特徵空間(Eigenspace)。最後將圖像與資料庫的樣本圖像進行歐式距離(Euclidean Distance)的比對找出最符合的手勢。但此部分有一特殊手勢需判斷其旋轉角度，因此透過空間轉換後取出膚色區域並使用雷達掃描的方式計算出旋轉角度。第三部分透過簡易的光學設計，利用雷射及網格投射出一特定圖案，本論文中設計一橫向式網格，並使用霍夫轉換(Hough transform)演算法找出投射之圖案於手勢前後移動時之網格間的變化，以得到追蹤中手勢與攝影機之相對距離。將以上三部分完成整合於系統中後，本論文將其應用於三維顯示器並完成互動。經實驗驗證後本論文特定手勢偵測及任意手勢追蹤皆可成功的完成預期目標並將其實現在即時系統上。手勢辨識率可達90%以上，深度之精準度可達到一公分以內。

This study proposes a novel 3D interactive display system, the functions of which hand include gesture tracking, recognition and depth detection in 3D interaction. The system consists of two cameras and optical elements. Camera 1 is devoted to hand gestures tracking with recognition, while camera 2 applied to process infrared images with optical components that are applied to project a particular pattern for depth detection in 3D interaction. The goals of this study are not only preserving the 2-dimensions hand gestures tracking system but also containing the information of third dimension (depth information) by the depth detection system that can increase the reality and the range of application. The works can be divided into three main parts: (1) hand gestures tracking, (2) hand gestures recognition, (3) depth measurement in 3D interaction. First, Haar-like features are employed to detect a specific hand gesture for inform PC to start tracking. The mean-shift algorithm is used to track the hand gesture after PC received tracking signal. When the hand is first localized, the Kalman filter is applied to track the hand by its prediction of hand position. In the second part, the skin color areas are found via changing the color model from sub-window, morphological algorithms are used for erosion and dilation of noise. Having processed, the principal component analysis algorithm is conducted to decrease the dimension of images, then extracting the eigenvalues and eigenvectors to build low-dimension eigenspace. Comparing the processed image with database by using Euclidean Distance, the most similar hand gesture is found. One of the hand gestures is required to compute the angle of rotation; hence the radar-like scan algorithm is applied to calculate. In the last part, a simple optical design is proposed based on Hough transform algorithm. In order to detect depth, a laser beam and a particular pattern are used in the novel system. A horizontal-type pattern is proposed to project the lines on the hand and the Hough transform algorithm is applied to find out the projected lines in this study. The projected lines can be captured when hand gesture moves, the distance between hand gesture and the camera can also be measured. Having conducted a series of experiments and verifications, the specific hand gesture detection and arbitrary hand gesture tracking can successfully implement. The accuracy rate of hand gesture recognition is higher than 90 percent and the resolution of depth measurement can be achieved to 1cm.