基於深度卷積神經網路之手勢辨識技術研究

Abstract

在這篇論文中，我們提出了一個使用任意單一攝影機在不同角度下仍能遠距離辨識多重手勢的技術。此技術不需固定攝影機角度，不需要由特定使用者操作，且能分辨多種手勢。此技術在影片中自動找出使用者的手的位置，並判斷使用者想傳達的訊息，希望能進行遠距離的操作以及在任何背景之下皆能達到手勢辨識的效果。在此設定議題下，為了能在複雜背景與不同視角拍攝的情況下有效的找到手部出現的區域以及辨識使用者所傳達的訊息，我們不採用易被複雜背景所誤導膚色資訊且不使用事先設定之特徵萃取技術，而是利用卷積神經網路有效且準確的學習不同手勢所擁有的特徵，並結合不同形狀與大小的特徵，以找到能分離不同手勢最佳的特徵空間，再利用深度神經網路找出特徵之間的關係以及不同手勢與各特徵的連接，藉此達到手部偵測與多重手勢辨識的成果。此外，我們也利用影片中已經得到的手部位置與移動資訊加上手勢辨識結果，推測之後較有可能出現手部的區域以及最佳的手勢辨識結果。

In this thesis, we propose an algorithm which recognize hand gestures with a single camera under different view-points within a range remotely. The algorithm can recognize multi-gestures without fixing view-point of the camera or a particular user controlling. In order to find the hand position and recognize the gestures from the video automatically and efficiency in the clutter background under different view-points within a range, we don’t take the skin color as the information which is easily influence by the clutter background, nor do we use the specific feature extraction processing. Instead, we use the convolutional neural network to learn the features in the hand gesture image and combine the different kernel sizes to get the best feature space for separating different gestures. Then we use the deep neural network to find the relationship between the hand features and the gesture classes. Under this setting, we are able to locate the hand and recognize multi hand gestures. Furthermore, with the help of the temporal information for the hand position and motion getting from the video, we are able to infer the most possible area where the hand would appear and the best recognition result.