Tracking, recognition, and distance detection of hand gestures for a 3-D interactive display

Abstract

This study proposes an interactive system for displays, the technologies of which consists of three main parts: hand-gesture tracking, recognition, and depth measurement. The proposed interactive system can be applied to a general 3-D display. In this interactive system, for hand-gesture tracking, Haar-like features are employed to detect a specific hand gesture to start tracking, while the mean-shift algorithm and Kalman filter are adopted for fast tracking. First, for recognizing hand gestures, a principal component analysis (PCA) algorithm is used to localize colored areas of skin, and then hand gestures are identified by comparison with a prepared database. Second, a simple optical system is set up with an infrared laser source and a grid mask in order to project a proposed horizontal stripe pattern. Third, the projected patterns are deciphered to extract the depth information using the Hough-transform algorithm. The system containing hand-gesture localization, recognition, and associated depth detection (the distance between the display and the hand), was included in a prototype of an interactive display. Demonstration of rotation recognition of a finger-pointing hand gesture was successful by using the algorithm of radar-like scanning.