複雜背景中具深度資訊之手勢追蹤與辨識

Abstract

本論文提出一低複雜度的手勢追蹤演算法，本演算法不但可以提供手勢的深度資訊，也能在嚴峻複雜的背景中正常執行。此外，本論文也提出了一些手勢辨識方法來支援消費性智慧電子產品應用，例如智慧電視。 目前大部分的手勢追蹤演算法採用膚色過濾做為前處理步驟，但僅僅使用膚色過濾無法在含有相近膚色的背景下維持系統的功能性，例如:木質地板。 在本論文中，所提出的設計採用適應性的膚色過濾，此過濾方法可以根據目前追蹤手勢的顏色來調整過濾器的參數。本論文也提出一有效方法將手部區塊從畫面中切割出來，以利後續即時性的深度計算。 本論文也對演算法進行優化與加速，包括多執行緒與其他降低計算量之技巧，我們最終將設計實作在個人電腦與嵌入式平台上，個人電腦上的每秒幀數平均可達到VGA 解析度視訊每秒24張，而嵌入式平台上則可達到QVGA 解析度視訊每秒8張。

This thesis proposes a low-complexity algorithm for hand tracking which can provide depth information and is able to work under critical backgrounds. Besides, some gesture controls are also proposed to support intelligent consumer electronics applications, like intelligent TV. Most methods of hand tracking apply skin color filter as one of pre-processing. However, only applying skin color filter as the segmentation step cannot maintain correct system functionality while the background containing pixel values which are close to skin color, like wooden floor. In the proposed design, we adopt adaptive skin color filter which can adaptively change its parameters according to the pixel values of the currently tracked object. This thesis also proposes an effective way to segment hands out of entire image and which can also facilitate depth estimation of tracked hands in real-time by dual-camera systems. We also apply multithreading and several techniques to reduce computational complexity in our design. The final algorithm has been implemented both on PCs and embedded systems. On PCs, we can reach the performance about VGA video at 24 frames per second. On the other hand, after reducing image size (i.e. QVGA video), we can achieve the performance about 8 frame per second on PandaBoard embedded system.