标题: 应用于三维互动显示器之手势辨识与追踪及深度感测系统
Recognition, Tracking and Distance-Detection of Hand Gestures for a 3D Interactive Display
作者: 黄子凡
Huang, Tzu-Fan
赵昌博
Paul C.-P. Chao
电控工程研究所
关键字: 三维互动;手势辨识;深度感测;手势追踪;a 3D interactive system;hand gesture tracking;hand gesture recognition;depth measurement
公开日期: 2010
摘要: 本论文提出一结合深度感测之手势追踪及手势辨识系统并将其应用于三维互动显示器。整个研究逐步完成手势追踪、手势辨识与深度感测系统之多样性的结合演算法的实现。此感测系统包含两个摄影机,分别用于深度感测及手势追踪与辨识,其目标在于不仅拥有一般二维式的手势追踪即辨识系统外,还增加了第三维(深度)的资讯,而提升应用的范围及真实度。整个研究可以分为三个阶段:(1)手势追踪;(2)手势辨识;(3)深度感测互动。第一部分使用Haar-like特征做特定的手势侦测,其目的为用于告知电脑开始追踪,接着搭配使用平均位移(Mean-Shift)演算法及卡尔曼滤波器(Kalman filter)已达成追踪程序。第二部分于追踪的子视窗中做手势的辨识,首先将子视窗中的图像做色彩模型的变换后找出肤色部分,再经由形态学的膨胀与侵蚀将杂讯滤除,搭配使用主要成份分析(Principal Component Analysis)演算法将图像降维,并取出图像之特征向量(Eigenvector)与特征值(Eigenvalue)建立低维度的特征空间(Eigenspace)。最后将图像与资料库的样本图像进行欧式距离(Euclidean Distance)的比对找出最符合的手势。但此部分有一特殊手势需判断其旋转角度,因此透过空间转换后取出肤色区域并使用雷达扫描的方式计算出旋转角度。第三部分透过简易的光学设计,利用雷射及网格投射出一特定图案,本论文中设计一横向式网格,并使用霍夫转换(Hough transform)演算法找出投射之图案于手势前后移动时之网格间的变化,以得到追踪中手势与摄影机之相对距离。将以上三部分完成整合于系统中后,本论文将其应用于三维显示器并完成互动。经实验验证后本论文特定手势侦测及任意手势追踪皆可成功的完成预期目标并将其实现在即时系统上。手势辨识率可达90%以上,深度之精准度可达到一公分以内。
This study proposes a novel 3D interactive display system, the functions of which hand include gesture tracking, recognition and depth detection in 3D interaction. The system consists of two cameras and optical elements. Camera 1 is devoted to hand gestures tracking with recognition, while camera 2 applied to process infrared images with optical components that are applied to project a particular pattern for depth detection in 3D interaction. The goals of this study are not only preserving the 2-dimensions hand gestures tracking system but also containing the information of third dimension (depth information) by the depth detection system that can increase the reality and the range of application. The works can be divided into three main parts: (1) hand gestures tracking, (2) hand gestures recognition, (3) depth measurement in 3D interaction. First, Haar-like features are employed to detect a specific hand gesture for inform PC to start tracking. The mean-shift algorithm is used to track the hand gesture after PC received tracking signal. When the hand is first localized, the Kalman filter is applied to track the hand by its prediction of hand position. In the second part, the skin color areas are found via changing the color model from sub-window, morphological algorithms are used for erosion and dilation of noise. Having processed, the principal component analysis algorithm is conducted to decrease the dimension of images, then extracting the eigenvalues and eigenvectors to build low-dimension eigenspace. Comparing the processed image with database by using Euclidean Distance, the most similar hand gesture is found. One of the hand gestures is required to compute the angle of rotation; hence the radar-like scan algorithm is applied to calculate. In the last part, a simple optical design is proposed based on Hough transform algorithm. In order to detect depth, a laser beam and a particular pattern are used in the novel system. A horizontal-type pattern is proposed to project the lines on the hand and the Hough transform algorithm is applied to find out the projected lines in this study. The projected lines can be captured when hand gesture moves, the distance between hand gesture and the camera can also be measured. Having conducted a series of experiments and verifications, the specific hand gesture detection and arbitrary hand gesture tracking can successfully implement. The accuracy rate of hand gesture recognition is higher than 90 percent and the resolution of depth measurement can be achieved to 1cm.
URI: http://140.113.39.130/cdrfb3/record/nctu/#GT079812520
http://hdl.handle.net/11536/46875
显示于类别:Thesis