整合頻繁樣式分析與多模隱藏式馬可夫模型的棒球影片事件分類架構

Abstract

影片的高階語意事件辨識已發展成為多媒體搜尋與索引領域最有趣的研究議題之一。因為影片低階特徵與高階事件在語意上距離遙遠，需要提出一個階層式的影片分析架構，利用中階特徵連結低階的聲音視覺特徵和高階語意事件。因此這篇論文提出一個使用中階時間區間特徵相互間時間前後關係的影片事件分類架構。在此架構中，我們提出了一個共同出現符號轉換方法來探勘隱藏式馬可夫模型機率事件分類架構中多種來源特徵間的完整時間關係。另外，資料探勘和頻繁樣式分析已成為從資料中發現新知識很受歡迎的方法，但是卻幾乎沒有被應用到影片語意分析領域，因此這篇論文提出兩種方法整合頻繁樣式分析和多模隱藏式馬可夫模型棒球事件分類，包含利用頻繁樣式訓練的隱藏式馬可夫模型和依照頻繁樣式設計的隱藏式馬可夫模型。除此之外，不同的時間序列編碼方法也被提出來和共同出現符號編碼法比較分類效果。實驗結果證明我們提出來的方法在棒球影片事件分類的優越性以及整合頻繁樣式分析至事件分類架構有助於提升事件分類的效能。

Semantic high-level event recognition of videos is one of most interesting issues for multimedia searching and indexing. Since low-level features are semantically distinct from high-level events, a hierarchical video analysis framework is needed, i.e. using mid-level features to provide clear linkages between low-level audio-visual features and high-level semantics. Therefore, this thesis presents a framework for video event classification using temporal context of mid-level interval-based multimodal features. In the framework, a co-occurrence symbol transformation method is proposed to explore full temporal relations among multiple modalities in probabilistic HMM event classification. Besides, data mining and frequent pattern analysis have recently become a popular way of discovering new knowledge from a data set. However, it is rarely applied to video semantic analysis. Therefore, this thesis introduces two methods: frequent-pattern trained HMM and frequent-pattern tailored HMM to incorporate frequent pattern analysis into multimodal HMM event classification for baseball videos. Moreover, different symbol coding methods including temporal sequence coding and co-occurrence symbol coding for multimodal HMM classification are compared. The results of our experiments on baseball video event classification demonstrate the superiority of the proposed approach and demonstrate that integration of frequent pattern analysis could help to improve event classification performances.