利用環場視覺作自動車應用之定位與影像分析新技術之研究

Abstract

許多自動車的應用中，定位是欲操控自動車所必備的功能。一個廣泛採用的方法是利用電腦視覺來作自動車定位，其方法是藉由電腦分析攝影機所攝得的影像來推算自動車本身的位置。近來新型態的環場攝影機漸漸被廣泛地應用在自動車定位上。相較於傳統攝影機，環場攝影機寬廣的視角可讓更大的景物範圍出現在視野中，更有助於利用視覺作定位的計算。然而，實際上環場攝影機擷取到的影像是扭曲的，造成影像處理上的困難，使定位工作變得難以進行，因此需要進行影像修正。

在本論文中，我們提出了一系列基於環場視覺的新的自動車定位與環場影像修正技術，並將其應用在各種自動車的用途上。我們使用了雙曲面反射鏡式與魚眼透鏡式的環場攝影機，而且，我們採用了路標式定位法，該法使用環境中明顯路標來作自動車的定位。在另一方面，我們在環場影像上，直接分析基本的路標幾何特徵（如直線與圓形等）的投影與投影之間的相互關係（如平行與垂直等）。並將前述所提出的方法應用在多種自動車的導航上，包含室內自動車導航、直升機降落、汽車輔助駕駛等。更進一步地，我們提出了解決影像定位技術應用在自動車上常見的兩個問題的方法，一個是所謂扭曲環場影像的轉正方法，可用以解決自動車震動所導致的影像扭曲，另一個是一所謂空間與影像間的對映方法，此法可解決應用場合中攝影機重新安裝所導致的定位失效問題。這兩個方法讓前述所提自動車定位方法在實際應用中變得更為有效。茲將前述所提各種方法分為兩類□新自動車定位技術與新影像修正技術□詳細說明如下。

(A)針對自動車新定位技術 ---

(a)我們提出了一個描述環場影像中的圓形路標投影的新方法。在此法中，我們證明了此一投影可用橢圓來加以逼近，其中我們用了泰勒展開的技巧。以此，我們得以提出一個新法則來抽取環場影像中的橢圓狀投影。 (b)我們提出了一個利用圓錐曲線來描述環場影像中的直線投影的新方法。在此方法中我們推導出簡單而且有公式解的直線投影方程式，並接著提出可用以從環場影像中抽取圓錐曲線的簡便方法，其原理乃基於赫夫轉換。 (c)我們提出了一個以環場影像中的天花板圓形圖案，作為路標的室內自動車定位與導航之方法。此利用此路標具有容易偵測與辨識，且不易被遮蔽等優點。 (d)我們提出了一個利用環場影像中的Y形屋角影像，作為路標的室內自動車定位與導航之方法。因為燈光或拍攝角度的關係，Y形直角並非每次都能完整的出現在環場影像中，所以我們分析了Y型屋角在環場影像中，所有可能出現的樣式，如點、垂直線、水平線及其可能組合樣式，並分別提出相對應的定位方法。 (e)我們提出了一個利用環場影像中的標準停機坪影像，作為路標的直升機降落之定位方法。在充分且有效的分析環場影像中停機坪圖案的幾何特徵，包含圓形、水平線，加以參考直升機距離停機坪的遠近，我們提出了包含接近、對正與觸地的三階段定位方法。其中，求出的定位資訊包含高度、方位、與距離。 (f)我們提出了一個利用環場影像中的小客車車輪影像，作車側車輛定位方法。只要利用本身車輛上的一台環場攝影機所拍到的鄰車影像，再經分析中車輪影像後，此方法即可推算出鄰車相對於本身車輛的位置及方向，並具有公式解。

(B)針對新影像修正技術 ---

(a)我們提出了一個針對非置中環場攝影機所拍攝的扭曲影像，將其轉正的方法。造成非置中環場攝影機的現象，乃因在置中環場攝影結構中，原先的透鏡/反射鏡相對位置改變。造成此一改變的原因，常常是因為自動車的震動或攝影機的重新安裝。我們的方法可以當場解決此問題，而不必送回當初的攝影機製造廠作校正。 (b)我們提出了一個可應用於物體定位或自動車定位的空間對映方法，並能在實際應用時，適應攝影機高度與角度的改變。此方法基於空間對應表，可求出於多種型攝影機的影像座標與空間座標的對應關係，使得面臨實際環境時，該方法更加具有應用價值。

實驗結果顯示本論文提出的所有方法，皆具有優越性及有效性。最後，討論與未來可能研究方向也附於本論文中。

Vehicle localization is essential for autonomous vehicle guidance in many applications. A widely adopted approach is the vision-based technique by which the locations of vehicles can be computed by analyzing the images captured by cameras. Omni-cameras have become more and more popular recently for their wider field of views (FOVs). Wider FOVs make the job of localization easier because a larger scene range can be taken in a single shot. However, due to the distortion in the images taken by omni-cameras, it is difficult to use omni-cameras in vehicle localization applications unless taken omni-images are properly adjusted.

In this study, investigation of new omni-vision based vehicle localization techniques and omni-image adjustment, as well as their applications is conducted. Two types of omni-camera are used, including hyperboloidal omni-camera and fish-eye camera. Also, the landmark based approach to localization is adopted, in which obvious landmarks in vehicle navigation environments are utilized. On the other hand, the projections of basic landmark features (lines, circles, etc.) in omni-images, and their relations (parallelism, perpendicularity, etc.) are analyzed mathematically. Accordingly, methods for varoius vehicle localization applications using the basic landmark features are proposed, including indoor vehicle guidance, helicopter landing, car driving assistance, etc. Furthermore, solutions to two vehicle localization problems frequently encountered with in real applications are also proposed, one being an omni-image unwarping method for dealing with image distortions caused by a misaligned omni-camera, and the other being a space-to-image mapping method which is adaptive to camera setup changes found in in-field environments. These two solutions make the proposed vehicle localization methods more effective in real environments. The above-mentioned proposed methods are summarized in the following, classified into two categories: new vehicle localization techniques and image adjustment techniques.

A. New vehicle localization techniques

(a) A new method for describing the projection of a circular-shaped landmark in omni-images is proposed. It is shown that such a projection may be approximated by an ellipse based on the application of Taylor expansion. In accordance, a new algorithm is designed for extracting the elliptical-shaped projections from omni-images of circular-shaped landmarks. (b) A new method for describing the projection of a line in an omni-image as a conic section is proposed. Equations of such a projection are derived to be simple and analytic, and consequently uncomplicated effective image analysis algorithms are designed for extracting such conic sections out of omni-images by the Hough transform. (c) A new method for vehicle localization by omni-vision for autonomous vehicle navigation in indoor environments using circular landmarks on ceilings is proposed. Such landmarks have several advantages can be identified, including ease to detect and recognize, and freedom from oclussion. (d) Systematic vision-based vehicle localization techniques by hyperboloidal omni-cameras using Y-shaped house corners in indoor environments as landmarks are proposed. All possible partial structures of a house corner consisting of a corner point, a horizontal line, and a vertical one are considered, facilitating flexible vehicle localization under various lighting, occlusion, and imaging posture conditions. (e) An omni-vision-based self-localization method for automatic helicopter landing on a helipad with a circled H-shape is proposed. The landing process includes three stages: approaching, alignment, and docking. Three types of image features, circle, line, and point, are used to derive skillfully analytic equations for computing the helicopter height, distance, and orientation with respect to the landing site. (f) A lateral vehicle localization method by omni-image analysis is proposed for car driving assistance. The method estimates analytically the position and orientation of a lateral vehicle by utilizing the geometric properties of a circular-shaped wheel image of the lateral car taken by a single omni-camera.

B. Image adjustment techniques

(a) A new method for solving the problem of unwarping a distorted omni-image taken by a lateral-directionally misaligned omni-camera is proposed. Such camera misalignment is a frequently encountered problem of vision-based localization in real applications due to vehicle vibrations or camera redeployments. The method can solve this problem without camera calibration which is usually done in advance in the factory. (b) A new space-mapping method for object location estimation or vehicle localization, which is adaptive to camera setup changes in application environments is proposed. The method, which is general for various types of cameras, estimates the location of an object appearing in an image by mapping the image coordinates of an object point to the real-world coordinates of the point using a space-mapping table. Such a method makes the space-mapping based approach to object localization more useful to real applications.

Good experimental results are shown to prove the feasibility and effectiveness of all the proposed methods. Discussions on possible future research directions are also included.