具空間感知的影片軌跡穩定系統

Abstract

有別於專業的電影拍攝器材，手持式攝影裝置受到價格與攜帶的限制，往往不具備穩定系統，拍攝時容易造成高頻率的晃動，產生視覺上的不舒適感。因此，影片軌跡穩定的目的在消除那些不自然的搖晃並且維持原本的拍攝路徑。一般來說，解決上述問題的做法可分為三個步驟：(1)估測攝影機鏡頭路徑、(2)平滑鏡頭路徑、(3)重新投影物體至新的攝影機平面。傳統的投影演算法將影片中的每一個影格視為單一平面，然後透過線性的空間轉換將影格重新排列。這個做法雖然簡單，卻無法處理視差的問題。由於不同深度的物體受到手震的影響程度不同，線性的轉換無法對它們做個別處理，大大地減少這些方法的實用性。近年來非線性的做法將影格分解成許多小區塊，讓各個區塊有各自的空間轉換，克服了不同景深所帶來的問題。然而這個方法因為具有較高的自由度，反而有可能造成更大的視覺失真。本研究計畫的重點在設計一個具空間感知的變形架構，讓影格中的物體受到空間轉換時能參考與周圍物體的相對位置，避免產生與原本場景不一致的排列關係。同時，由於近年來立體攝影機與具深度的攝影機漸漸普級，我們也將這些攝影器材納入研究重點，探討他們的深度資訊在移除影片晃動時所帶來的優點，與衍生的問題。

Video stabilization becomes a hot research topic in recent years due to the shaky artifacts captured by hand-held cameras. The goal of this application is to remove high frequency camera motions while retaining the principle camera paths. To achieve this, most techniques try to (1) estimate camera movement between consecutive frames, (2) smooth the camera path, and (3) re-project content to the newly obtained camera coordinates. The homogeneous approaches consider each video frame as a plane and apply a linear transformation to align each video frame based on the camera coordinate. Although this method is simple and efficient, it is not able to handle scenes with large parallax effect. The shaky artifacts are of different magnitudes due to various depths. Thus, transforming all content homogeneously decreases the capability of video stabilization. On the other hand, non-homogeneous methods partition each video frame into regular quads and each quad is handled respectively. However, the high degree of freedom potentially introduces distortions because relative positions of objects may be destroyed. Hence, in this project, we strive to design a spatial aware deformation framework to solve this problem. In addition, due to the popularity of stereo and 3D cameras, we will utilize the additional information they provided to achieve even better results and also solve the side-effects they introduced.