基於影像之即時手勢辨識系統設計

Abstract

本篇論文主要是研究以影像為基礎的即時手勢辨識系統，總共有六個基本手勢，包含向左、向右、向上、向下、左右、上下。本論文所發展之手勢辨識系統以四個階段來執行，分別為目標追蹤、目標影像前處理、特徵擷取與手勢分類，在過程中系統將持續追蹤連續影像中的手掌，並進行目標影像前處理，除了將手掌從背景影像中隔離出來，也一併去除手腕以下的膚色部分，接著是特徵擷取階段，計算手掌影像的區塊平均值，以及相關於大小平移不變量的影像矩，作為手掌影像的特徵向量，再根據K-means演算法做不同狀態之歸類，而這些狀態的變化是有限的，為了解決相同手勢的不同序列長度之問題，狀態序列裡的重覆與暫態必須予以去除，再將結果送至類神經網路作辨識，進行最後的手勢分類階段。從實驗結果可知，本論文所提之手勢辨識系統確實可以達成即時的辨識功能，並且具有不錯的辨識效果。

This thesis focuses on the development of real-time hand gestures recognition system. There are six hand gestures, including turn left, turn right, upward, downward, horizontal swing and vertical swing. The system operates in four stages, which are object tracking, object image pre-processing, feature extraction and hand gesture classification. During the operation, the hand is always traced by the mean-shift tracking algorithm. Then, the hand image, with the arm cropped, is further extracted from the background in the pre-processing stage. After that, the lattice average and the scale and translation invariant moment are calculated in the feature extraction stage to form a feature vector, which will be classified into some finite states by the K-means algorithm. However, two gestures with the same meaning may be represented by different number of states since some of the states are repeated or generated transiently. In order to deal with such problem, a sequence of neural networks is developed to eliminate the repeat and transient states. Finally, the resulted state sequence is fed into the hand gesture classification neural network. From the experimental results, the proposed hand gesture recognition system can perform in real-time and possess good recognition rates.