可超寬調壓之智慧視覺處理晶片系統平台---子計畫六：可超寬調壓系統晶片之智慧視覺處理加速器設計

Abstract

隨著智慧型手機產品的快速崛起，許多視訊相關技術紛紛在智慧型手機上找到新的應用方式。如最近在餐廳常出現的是：美味佳餚甫上桌，朋友們便爭相拿手機拍個過癮、立刻上傳到facebook與更多親朋好友一塊分享，之後才捨得動筷子；出門在外想起家人或情人，隨手拿起智慧型手機便可以直接面對面通話聊天；沒多久前專屬於專業攝影領域的高畫質錄影技術，今日已成為智慧型手機的基本配備，而嵌入式視訊早已滲透我們的日常生活，成為許多人生活中不可或缺的技術。為了因應目前智慧型手機更輕、更薄、更高效能與更省電之發展趨勢，世界知名手機鏡頭晶片廠商Omnivision 於近期推出 Camera Cube chip (OVM7690, OVM7692)，支援VGA解析度，但是其實體大小僅有2.8x3.2x2.5mm3，可以符合手機更輕、更薄的發展趨勢。同時，為了利用多個手機鏡頭晶片來達到更高解析度之視訊處理效能，Pelican Imaging 也提出鏡頭陣列之設計概念，利用多鏡頭之訊號處理，以達到高解析度視訊之目標。由以上技術趨勢，可以看出未來智慧型手機視訊技術，將由現階段單鏡頭視訊處理技術，轉而朝向多鏡頭視訊處理技術發展。 然而，功率消耗一直是智慧型手機技術所需面對的設計挑戰，從Apple iPhone 4S拆解報告所揭露的功耗量測結果中得知， iPhone 4S在壓縮每秒30張full HD之H.264視訊需要610~640 mW，降為每秒30張的VGA畫質亦需要大略140mW。換句話說，不論何種模式操作，視訊處理皆是智慧型手機耗電之大宗，故值得持續投入研究資源來研究低功耗之智慧視覺處理技術。 為因應此技術發展趨勢與設計挑戰，本計畫搭配總計畫所提出之全新智慧視覺系統應用整合計畫：「可超寬調壓之智慧視覺處理晶片系統平台」，進行子計畫六的研究，題目為「可超寬調壓系統晶片之智慧視覺處理加速器設計(Intelligent vision processing accelerators for UDVS SoC)」。希望於此整合型計畫目標 「可超寬調壓之智慧視覺處理晶片系統平台」之架構下，研發多鏡頭視訊處理方法與其硬體加速器實現，可支援寬視角全景視訊、高動態範圍視訊、高解析度視訊等功能(如圖11-1所示)，同時支援可超寬範圍調整電壓之設計技術，使得本計畫研發技術可應用於未來智慧型手機應用處理器晶片上，以衍生各種創新智慧型手機系統應用。同時我們也希望藉由此計畫，逐年完成數項具前瞻性之創新智慧視覺系統應用技術，以提昇國內學術界創新技術之研發能量。 搭配總計畫之設計目標，本計畫在三年規劃中預計將可產出 (1) 360度全景視訊合成技術與其支援UDVS硬體加速器設計； (2) 高動態範圍視訊處理技術與其支援UDVS硬體加速器設計； (3) 超高解析度視訊合成技術與其支援UDVS硬體加速器設計。

With the rapid growth of smart phones, there have been lots of video related technologies found in smart phones, like taking a delicious dish and uploading through smart phones when you are having dinners in a restaurant, calling a video phones with your family or lovers through smart phones, and shooting high definition videos with your smart phones. You can realize that video applications in smart phones have been easily found in our daily lives and have become inevitable to us. In order to pursue the technology trends of lighter, thinner, more power efficient, and more energy saving in smart phones, a major CMOS image sensor player, Omnivision, has released a camera cube chip (OVM7690, OVM7692) owning the physical size of 2.8x3.2x2.5mm3 and supporting VGA video resolution to enable the lighter and thinner smart phones. Moreover, another company, Pelican Imaging, exploits the design idea of camera array to produce ultra high resolution video by combining lots of low resolution videos captured by camera array. From the observation on the above mentioned technology trend, the video processing technology in smart phones will evolve from single-camera vision processing to multiple-camera vision processing to enable lots of innovated applications for smart phones. However, low power consumtion is always a design challenge for smart phone technology, especially on the vision processing. From the report to analyze the Apple iPhone 4S performance, we can realize that it requires 610~640mW to encode HD1080 video by iPhone 4S. Even reducing the video resolution to VGA, it still requires 140mW by iPhone 4S to encode VGA resolution videos. That is to say, no matter which operation modes, video processing consumes the major power in smart phones, which reveals that it is worth to do more research on ultra low-power vision processing technology for smart hand-held devices. Following this technology trend, this project is the sub-project 6 of the main project entitled “Ultra-Dynamic Voltage Scaling (UDVS) SoC System Platform for Intelligent Vision Processing” for bringing brand new applications for intellugent mobile vision processing in the coming future. The title of this sub-project 6 is “Intelligent vision processing accelerators for UDVS SoC”. Under the design concept of the main project, as shown in Fig. 11-1, this sub-project will develop the algorithms and hardware architectures of some intelligent vision processing technology, including 360 degree panoramic video synthesis, high dynamic range (HDR) video synthesis, and super high resolution video. The goal of this sub-project 6 is to enable the possibility of innovative intelligent vision processing technologies on smart phones into mass production so as to induce lots of brand new intelligent vision applications in the near future. Moreover, we will develop some advanced vision processing technology in this project to enhance the research capability of Taiwan in this area. 2 To support the main project, this sub-project 6 will deliver the following key technology in three years: (1) 360 degree panoramic video synthesis algorithm and the associated hardware accelerator design supporting UDVS technology; (2) High dynamic range (HDR) video synthesis algorithm and the associated hardware accelerator design supporting UDVS technology; (3) Super high resolution vision synthesis algorithm and the associated hardware accelerator design supporting UDVS technology.