攝影機校準及影像轉換技術與其應用之研究

Abstract

在電腦視覺領域中，抽取與分析攝影機攝取之影像內所含資訊是由演算法則實作之電腦程式來完成，其中一種資訊為空間中的物體幾何(形狀或位置等)。此類應用之演算法則中一般包含一重要之程序稱為“攝影機校準(camera calibration)”，其目的是在建立影像平面與物體空間座標系統間之對應關係，此對應關係以“數學函數(mathematical function)”表示，並在校準後得到一組參數，用以代表此攝影機在物體空間座標系統中之“特性(characteristics)”表現。此特性對於不同攝影機的光學系統架構(光學焦距、元件擺置等)是唯一的。而不同的攝影機種類(光學系統設計不同)則需要用不同的“數學函數”模式來描述。在完成“攝影機校準”程序後，演算法則才能將攝取影像之資訊轉換至物體空間中，以符合人腦之理解模式。本博士論文內容在探討不同種類攝影機之校準及影像轉換技術與其應用，共提出三種有關全方位(omni-directional)攝影機之全新影像轉換方法及兩種傳統攝影機之全新校準方法及應用。 全方位攝影機因有接近(甚至超過)半球形之廣大視野而被廣泛的應用於視覺監控、機器人視覺或自動車導航，其擷取之全方位影像(omni-image)最後必須轉正為一般正常之透視影像(perspective-view image)或環場影像(panoramic image)，以利人眼觀察或影像證據保存。目前之研究注重單一視點(single view point, SVP)攝影機之影像轉正，對於非單一視點(non-SVP)全方位攝影機之影像轉正，因其困難度高而少有研究。但因非單一視點攝影機具有比單一視點較好之平均徑向解析度(radial resolution)及視野較大之優點，使其更適合應用於上述之領域。本博士論文冀能領先全球研究，發展出並探討適用於非單一視點全方位攝影機之影像轉正方法，補足其先天缺點，使其更適於實際之應用。我們總結提出的方法如下: (a) 提出一個針對非單一視點之雙曲下折攝影機(hypercatadioptric camera)的影像轉正(image unwarping)方法。此方法擴展了目前既存(單一視點)之方法，可容許透鏡/反射鏡(lens/mirror)之不精確組裝。此問題在大部份的實際應用中是難以克服的。 (b) 提出一個稱為“雙層八方向之邊緣保存權重式插補(edge-preserving 8-directional two-layered weighting interpolation)” 方法，可用以插補從一個非單一視點之全方位攝影機攝取之全方位影像(omni-image)轉正到透視(perspective-view)或環場(panoramic)影像之未填充像點(unfilled pixel)。此方法能解決內含許多不均勻分佈之未填充像點的影像插補問題。 (c) 提出一個全方位影像轉正到環場或透視影像之統合方法，此方法是基植於一個環場轉換表(pano-mapping table)之新觀念。此表由一針對任何型式之全方位攝影機做一次簡單之學習程序而產生，隱含了所有攝影機參數之資訊。 此外，針對傳統攝影機在指標系統應用上的缺點, 我們提出兩種方法： (d) 提出一個顯示螢幕與其影像間座標轉換之強靭且精確的校準方法，此方法經由消除接近影像邊界之位移誤差而改善了座標轉換之精確度。 (e) 提出一個以視訊為基礎之控制電腦游標的相機滑鼠(camera mouse)，此滑鼠使用一個視訊攝影機，可手持於空中操作。此方法之主要優點是不需要複雜之攝影機校準。 以上本論文提出之方法(a)至(e)，皆為創新之作。同時多方面的實驗結果顯示所提方法可行實用，與其他方法比較結果，亦具有相當的優越性。

In the field of computer vision, extracting and analyzing the information contained in the image captured by a camera are performed by a computer program implementing a certain algorithm. One kind of such information is the geometry (its shape or pose) about an object in the space. The algorithm to extract such a kind of information usually includes an important procedure called “camera calibration.” The purpose of camera calibration is to construct the relationship between the image plane of the camera and the coordinates system of the object space. The relationship is usually represented by a “mathematical function.” After calibration, a set of parameters representing the “characteristics” of the camera in the coordinate system of the object space is obtained. The characteristics are unique for each distinct optical structure (the focus length of optics, the component lay-out, etc.) of the camera. Different cameras with different optics designs need different “mathematical function” models to describe their features. After completing the “camera calibration” procedure, an algorithm is utilized to transform the information contained in a captured image into the object space to conform the realization model of the human brain. In this dissertation study, the investigation of camera calibration and image transformation techniques, as well as their applications is conducted. Three new methods are proposed for related topics of image transformations for the omni-directional camera (or just omni-camera) and two novel methods are proposed for the purpose of camera calibration. Because the field of view (FOV) of an omni-camera is almost near or even beyond a full hemisphere, it is popularly applied in the fields of visual surveillance, and vision-based robot or autonomous vehicle navigation. The captured omni-directional image (or just omni-image) should be rectified into a normal perspective-view or panoramic image for convenient human viewing or image-proof preservation. Current studies focus on the image rectification or image unwarping for a single-view-point (SVP) omni-camera. Studies on non-SVP ones are limited because of the difficulty to analyze their structures. But a non-SVP omni-camera is superior, compared with an SVP one in the aspects of possessing uniform radial resolutions and larger FOVs. These merits make it more suitable in the above application cases. In this study, we develop some suitable solutions to the issue of image unwarping for non-SVP omni-cameras to compensate their inherent deficiencies for fitting the requirement of practical applications. Proposed methods in this study are summarized in the following. (a) An analytic image unwarping method is proposed for a non-SVP hypercatadioptric camera. The method has extended the image unwarping capability of the existing methods for SVP omni-cameras to tolerate lens/mirror assembly imprecision, which is difficult to overcome in most real applications. (b) A new method called “edge-preserving 8-directional two-layered weighting interpolation” is proposed for interpolating unfilled pixels in a perspective-view or panoramic image resulting from unwarping an omni-image taken by a non-SVP omni-camera. This method can solve the problem of edge preserving in interpolating the input image which has many irregularly distributed unfilled pixels. (c) A unified approach to unwarping of omni-images into panoramic or perspective-view images is proposed. The approach is based on a new concept of pano-mapping table, which is created once forever by a simple learning process for an omni-camera of any kind as a summary of the information conveyed by all the camera parameters. Moreover, for resolving the deficiency of traditional camera applications in the pointing system, we propose two methods. (d) A robust and accurate calibration method for coordinate transformation between display screens and their images is proposed. The method improves the accuracy of the coordinate transformation to eliminate the shift errors near the image border. (e) A camera mouse with a vision-based method for computer cursor control using a video camera held in hand in the air is proposed. The main merit of this method is that it requires no complicated camera calibration. All the proposed methods described above are innovative. Meanwhile, experimental results show the feasibility of the proposed methods, and their effectiveness and superiority to other methods.