針對即時手部辨識及追蹤所設計以粒子濾波器為基礎之視窗旋轉及縮放演算法

Abstract

在這篇論文中，我們提出了一個使用單一攝影機就可以即時辨識出不只手的位置、還有手的大小和角度的多層級系統。這系統中包含了視窗定位層、視窗縮放層和視窗旋轉層以分別用來偵測手的位置、大小和角度。 從攝影機抓到的影像先透過預先處理以去除掉非膚色的背景。接著，每一個層級都利用方向梯度直方圖、支持向量機和粒子濾波器來偵測並追蹤手部視窗的不同特性。一個粒子在我們系統中被定義成一個可旋轉的長方形視窗，因為他可以同時表達手的位置、大小和角度這三個狀態。 不像傳統的多層級系統必須逐案地偵測出許多非手的情況然後再移除，我們的系統不只可以直接判斷出手的狀態、還可以在膚色背景之下正常運作。這是因為我們系統中每一個層級都只負責處理擁有相似特性的資料，如此有利於支持向量機訓練模型和預測資料。再者，一個層級可以透過我們所提出的「跨層級狀態傳遞」來和其他層級共享預測到的結果，以此來幫助其他層級限縮他們可能的狀態空間。「跨層級狀態傳遞」讓每個粒子可以不用在整個狀態空間中探索，如此可以大幅度地減低即時運作時的負擔。這樣的架構允許每個層級專注在自己負責的特性以至能在一個低多樣性的空間裡運作，並且可以分享預測到的狀態來幫助其他層級建立這種低多樣性的空間。最後，我們還提出了一個方法去重複利用每次計算出來的方向梯度直方圖，以至可以增加大量粒子以改善系統預測結果同時又不嚴重降低幀率。

In this thesis, a real-time multi-stage system that can perceive not only hand location but also hand size and hand angle using a single camera is proposed. The system is called PWRS in this thesis, and it is the abbreviation of Particle-based Window Rotation and Scaling. There are three stages, window-locating stage, window-scaling stage and window-rotating stage, recognizing the location, size and angle of a hand respectively. The frame captured by the webcam is preprocessed first to exclude those non-skin contexts. Then, each stage employs Histogram of Oriented Gradients (HOG), Support Vector Machines (SVM), and particle filter to detect and track different characteristics of the hand window. A particle defined in our system is a rotated rectangular window which can describe the location, size and angle of a hand at the same time. Unlike traditional multi-stage system which needs to detect and then remove lots of non-hand regions case by case, PWRS can not only directly recognize the hand states but also work normally in front of skin-like background. Because, each stage in PWRS is responsible for data with similar characteristic so as to benefit the SVM training and prediction. Also, a stage can share the predicted results with the others by proposed cross-stage propagation (CSP) to help them narrow down the possible region of states. With CSP, particles need not explore the entire state space, and this considerably alleviates the online loading. The architecture allows each stage to concentrate on its own target characteristic so as to work in a diversity-reduced space, and to share the results with other stages so as to help them construct the low-diversitied space. Finally, a proposed method called sub-HOG extraction (SHE) is designed to reuse HOG features so as to increase numerous particles for better prediction without collapsing frames per second (FPS).