魚眼攝影機校正與多視點影像接合

Abstract

在行車時，駕駛會根據週遭障礙物的方位操控車輛的行進方向，但車輛周圍通常存在許多視覺上的死角，不僅讓駕駛難以了解是否有障礙物存在，也提高乘客及行人的危險性。在本論文中，我們建構一套駕駛輔助系統提供車輛周邊360°的景物俯視圖，從無死角的整合式俯視圖，駕駛可得知車輛周邊是否存在障礙物及了解其相對的方位與距離，即使在許多障礙物的複雜環境當中，像是停車位、窄巷，駕駛也可以輕鬆地操控其車輛。這套輔助系統包含數個架於車輛周邊的魚眼攝影機同時擷取車輛周邊360°景物的影像，並透過影像接合將不同視角的影像整合為單一俯視影像，並將其呈現於中央監視螢幕。 此車輛周邊監控系統主要包含兩項技術，魚眼攝影機校正與多視點影像接合。在魚眼攝影機校正中，我們提出基於攝影校正法 (photogrammetric calibration) 的創新的校正方法，其中利用平面校正物進行魚眼攝影校正，並以視野失真模型 (field-of-view distortion model) 作為魚眼失真模型，透過降低攝影機校正誤差進行非線性的攝影機參數最佳化，由已校正的魚眼攝影機所拍攝的影像皆可經由失真校正 (distortion correction) 將魚眼影像還原至透視投影影像。針對多視點影像接合我們提出一個三步驟的接合法，首先，將拍攝到的地面影像透過透視投影轉換 (perspective transformation) 至俯視座標系統，並沿著最佳接合線(optimal seam) 作初步的接合；接著，透過動態規劃 (dynamic programmic) 沿著最佳接合線進行一維影像對位 (image registration)，並由影像形變 (image deformation) 將接合線上的影像對位結果平順地擴散至整張影像；最後，以影像混合 (image blending) 將已對位的各角度影像無接縫地接合至單一車輛俯視圖。 我們所提出的整合式車輛周邊監視系統可整合不同視點影像的地理資訊，駕駛從第三人稱的俯視圖中可清楚的了解車輛周邊的狀況，並根據障礙物的相對位置與移動物體的相對行進方向決定車輛的操控方式，大幅提升行車的安全性。基於整合地理資訊的優點，我們不僅將此影像整合應用至車輛周邊監控，並應用於安全監控上進行跨攝影機的大範圍監控。

What surrounds a vehicle directly effects the maneuvering of the vehicle. Vehicles usually have blind spots which lead to difficult maneuvering task and endanger passengers and pedestrians. In this thesis, we develop an effective driving assistant system which provides the surrounding image of a vehicle in bird’s eye view. This view enables the driver not only to survey the surrounding area of the vehicle without blind spots but also to maneuver the vehicle easily when driving in the environment having quite a few obstacles, such as narrow streets or parking spaces. This driver support system consist a few fisheye cameras mounted on a vehicle. The fisheye images of the surrounding scenes in all orientations are combined into the bird’s eye view image, and this integrated image is displayed on the single central monitor. This vehicle surrounding monitoring system involves two major techniques, fisheye camera calibration and multiple-view image stitching. We propose novel methods for both techniques to monitor the surrounding area of a vehicle. For fisheye camera calibration, we proposed a novel photogrammetric calibration method using planar calibration objects together with a distortion model based on the design of the fisheye lens. By minimizing the residual error between observed and predicted image coordinates of feature points, a set of camera parameters that best describe the imaging process of the camera can be obtained. Once the fisheye camera has been calibrated, the distorted images it taken can be rectified into perspective images. For multiple-view image stitching, we proposed a three-step method to stitch images capture from different viewpoints. First, the images of objects coplanar with the ground are registered on a planar surface in bird’s eye view using perspective transformation. Then the images of 3D objects are aligned along the stitching seams using a proposed 1D image warping method. Finally, a seamless bird’s eye view image is created by blending all registered images. The proposed support system can integrate the geography information of image captured from different viewpoints. We can easily know the location and moving direction of objects relative to the monitoring environment. We apply this system not only for vehicle navigation but also for integrated surveillance system.