雙階層視訊分析 – 由靜態背景模型到動態前景切割

Abstract

雙階層視訊分割 --- 即對視訊影片作前景層與背景層的區域切割 --- 是電腦視覺領域中一個極具挑戰性的問題，蓋因視訊內容的變化多樣，使得前景與背景的階層分割變得複雜。對於此一問題，我們在論文中分別以「背景模型初始化」、「背景模型維護」與「視訊階層遞移」三個研究主題來進行探討；其中，前兩個研究主題，是針對固定式攝影機所拍攝的影片，作靜態背景模型的建構與維護，使得前景階層，可透過與背景相減分割出來，而第三個研究主題，則是針對移動式攝影機所拍攝的動態影片，作前景與背景階層的遞移分割。

在背景模型初始化的研究主題探討中，我們開發一個以影像區塊為基礎的快速背景模型估計法，並提出新穎的背景模型完整度量測法則，使得一個完整的初始背景模型，可被快速建構出來。在背景模型維護的研究主題探討中，我們檢視了常用的高斯混合模型，發現在高斯混合模型中，需要兩種型態的學習速率控制，方可有效地平衡背景變化容忍度與前景偵測敏感度兩項拮抗因素，對此，我們提出一個基於高階資訊回饋的新式學習速率控制法，來改良高斯混合模型之背景模型維護方式。在視訊階層遞移的研究主題探討中，我們提出一個基於半監督式頻譜叢集法的動態階層分割架構，來對於移動式攝影機所拍攝的視訊影像，逐張作前景與背景的階層分割；其中，我們進一步擴充了半監督式頻譜叢集法的數學模型，以便在視訊階層分割過程中，調控階層標籤估計的可靠度，以增進階層分割的準確性，實驗結果顯示，此一視訊階層遞移分割架構，對動態影片的切割具有良好的效果。

Bi-layer video segmentation, i.e., the extraction of foreground regions from background ones for a video sequence, is a challenging research field in computer vision due to large content variation among video frames. To better address this bi-layer video segmentation problem, three research topics are investigated in this thesis including background model initialization, background model maintenance, and video layer propagation. While the first two topics concern static background modeling for analyzing videos obtained from static cameras, the third one pertains to dynamic foreground segmentation for videos captured by moving cameras.

For the problem of background model initialization, we propose an efficient background model estimation scheme based on image block classification, and develop novel criteria for measuring the completeness of a background model. For the problem of background model maintenance, we look into the formulations of Gaussian mixture modeling (GMM) and identify the needs of two types of learning rates for GMM to effectively deal with a trade-off between robustness to background changes and sensitivity to foreground abnormalities. A novel bivariate learning rate control scheme for GMM based on a feedback of high-level information is also proposed. For the problem of video layer propagation, a new framework based on semi-supervised spectral clustering is proposed for dynamic foreground segmentation of a video shot captured by a moving camera. The adopted formulation of semi-supervised spectral clustering is generalized to regularize the reliabilities of layer labels in sequential propagation. Experimental results show that satisfactory results of related bi-layer video analysis can indeed be obtained with the proposed approaches.