利用叢集演算法與叢集整合分析視訊中人物與地點之研究

Abstract

本計畫的主要目標是利用叢集演算法進行視訊內容的分析，尤其是在視訊中辨認出主要的人物與地點。一個直接的應用是以所找到的人物與地點作為視訊索引的根據，讓使用者可以利用這些資訊快速的找到有興趣的視訊內容。本計畫的成果亦可對視訊內容分析的未來研究有許多幫助，包括分析人物關係，利用人物與地點資訊產生更語意的視訊內容描述，以及作為更進一步與語音分析或動作分析的結合的基礎。 在本計畫中，我們會建立一組經人工標記的測試資料組，包括電影、電視影集、以及使用者自行拍攝的視訊內容。針對人物與地點的分群，我們將分析許多因素對分群結果的影響，包括臉部姿態、衣物色彩、畫面的前景/背景分割、各種計算串列（人物或地點）間相似度的方法、以及不同的相關性資料叢集法。除此之外，我們也會針對叢集整合在人物與地點的分群的應用進行分析。叢集整合與一般叢集演算法相較，具有新穎性與強韌性等優點。我們將實作兩種不同的叢集整合演算法並比較其結果。

This project focuses on the use of clustering algorithms on video content analysis. Specifically, the goal is to identify unique persons and places in a video. A direct application of this is to use the identified persons and places for video indexing purpose so that users can find use this information to efficiently locate interested video content. This work will also benefit future research on video content analysis, including the analysis of relations among people in the video, more semantic descriptions of video content that include the information of people and location, and for future integration with information of speech or behavior analysis. In this project we will build a set of labeled data including movies, TV series, and home video. For person and place clustering, we will investigate the effect of multiple factors on the clustering accuracy, including facial pose identification, torso color, foreground/background separation, various ways of computing sequence-sequence similarity, and various relational data clustering algorithms. In addition, we will also focus on the use of cluster ensembles, which has been demonstrated to have various benefits, including novelty and robustness, over common clustering algorithms. We will implement two different cluster ensemble methods and compare the results.