利用Isomap學習及VLMM技術來分析人類之動作

Abstract

人類動作分析是一個很基本的研究議題，並且被廣泛地應用在許多不同的研究領域。本論文提出了兩種適於人類動作分析之相關應用的視訊處理技衛。首先，為了自動化分析一段冗長且尚未被切割過之人類動作視訊資料，我們提出一個以流形學習(manifold learning)技術為基礎之非監督式(unsupervised)人類動作分析架構。為了有效地分析人類動作，非監督式學習的方法比監督式學習的方法更為適合，主要是因為非監督式學習的方法事先不需要太多的人為介入。然而，複雜的人類動作使得非監式學習的方法更具挑戰性。在這項研究中，我們首先從一個訓練用的動作序列中取得一個成對的人類姿勢距離矩陣。接著再利用等構映圖(Isomap)演算法從此矩陣中建構出低維度的結構。因此，訓練用的動作序列可以被映射到等構映圖空間中的流形軌跡(manifold trajectory)。為了有效地找出連續兩個單元(atomic)動作軌跡中的中斷點，我們將等構映圖空間中的流形軌跡描述為低維度的時間序列。我們接著再利用時間分割的技術將此時間序列分割成許多次級序列，每個次級序列都代表一個單元動作。然後，我們利用動態時間校正(dynamic time warping)的技術來對這些單元動作序列分群。最後，我們依據分群結果來學習單元動作，並且再利用最近鄰算法(nearest neighbor rule)對單元動作做分類。假如介於輸入的動作序列與最相近的群平均單元動作序列的距離大於某個門檻值時，我們便將此輸入的動作序列視為未知的單元動作。 在第二項研究中，我們提出了一個利用可變長度馬可夫模型(variable-length Markov models)技術來學習及辨認單元人類動作的架構。本架再包含兩個主要模組：姿勢標記模組及可變長度馬可夫模型之單元動作學習及辨認模組。首先，我們修改外形上下文(shape context)的技術來發展一個姿勢樣板(posture template)選擇的演算法。被選取的姿勢樣板可形成一個碼本(codebook)，利用此碼本我們可以將輸入的姿勢序列轉變為離散的符號序列。接著，我們利用可變長度馬可夫模型技術來學習對應於訓練用的單元動作之符號序列。最後，我們可將被建構的可變長度馬可夫模型轉換成隱藏式馬可夫模型(HMM)，並且再利用它來辨認輸入的單元動作。這項研究主要是結合可變長度馬可夫模型在學習方面的傑出好處及隱藏式馬可夫模型在容錯辨識能力的好處。

Human Action Analysis is a fundamental issue that can be applied to different application domains. In this dissertation, we propose two video processing techniques for human action analysis. First, to automatically analyze a long and unsegmented human action video sequence, we propose a framework for unsupervised analysis of human action based on manifold learning. To analyze of human action, unsupervised learning is superior to supervised one because the former does not require much human intervention beforehand. However, the complex nature of human action analysis makes unsupervised learning a challenging task. In this work, a pairwise human posture distance matrix is derived from a training action sequence. Then, the isometric feature mapping (Isomap) algorithm is applied to construct a low-dimensional structure from the distance matrix. Consequently, the training action sequence is mapped into a manifold trajectory in the Isomap space. To identify the break points between any two successive atomic action trajectories, we represent the manifold trajectory in the Isomap space as a time series of low-dimensional points. A temporal segmentation technique is then applied to segment the time series into sub-series, each of which corresponds to an atomic action. Next, the dynamic time warping (DTW) approach is used to cluster atomic action sequences. Finally, we use the clustering results to learn and classify atomic actions according to the nearest neighbor rule. If the distance between the input sequence and the nearest mean sequence is greater than a threshold, it is regarded as an unknown atomic action. In our second work, we propose a framework for learning and recognizing atomic human actions using variable-length Markov models (VLMMs). The framework comprises two modules: a posture labeling module, and a VLMM atomic action learning and recognition module. In the first stage, a posture template selection algorithm is developed based on a modified shape context matching technique. The selected posture templates form a codebook which can be used to convert input posture sequences into discrete symbol sequences for subsequent processing. Then, the VLMM technique is applied to learn the training symbol sequences of atomic actions. Finally, the constructed VLMMs are transformed into hidden Markov models (HMMs) for recognizing input atomic actions. This approach combines the advantages of the excellent learning function of a VLMM and the fault-tolerant recognition ability of an HMM.