基於RGB-D影像資訊之即時指寫法數字辨識系統設計

Abstract

指寫法辨識可提供使用者自然、直覺的人機互動方式，因此近年來被廣泛應用於各種領域中，如電子遊戲、遠距遙控等。本論文提出一個以色彩深度影像為基礎的指寫法數字辨識系統，分為手掌與指尖位置偵測、特徵擷取及指寫法數字辨識三個部分。首先利用膚色偵測配合聯通物件法與距離轉換法，分別找出手掌與指尖位置；接著追蹤指尖書寫數字之動態軌跡，先擷取軌跡移動方向作為數字辨識用之特徵，其中因為0與6無法經由移動方向特徵予以辨識，所以再提出軌跡起始點、終止點與垂直方向移動距離三種特徵，以提高辨識率。最後，利用串接式最鄰近法進行指寫法數字辨識。從實驗結果可知，本論文所提之即時指寫法數字辨識系統確實具有成效，可達九成五以上之辨識率。

The finger writing recognition approach has been introduced to a diversity of fields, like video games and remote control systems, because it provides a natural and intuitive communication for Human-Computer Interaction (HCI). This thesis proposes a real-time finger writing digit recognition system with high accuracy rate based on the RGB-D information. The system is divided into three main parts, including ROI selection, feature extraction, and finger writing digit recognition. For the ROI selection, first detect the skin color regions, then determine the palm and the fingertips based on the connect component labeling (CCL) and the distance transform respectively. Further, track the fingertip to create the trajectory and extract its directional features for digit recognition. However, since it is often confused in recognizing 0 and 6, three extra features are added to increase their recognition rate. Finally, with series k-NN classifiers, the experimental results show that the accuracy rate is higher than 95% in finger writing digit recognition, which implies the proposed real-time recognition system is indeed effective and efficient.