多視點視訊產生、傳輸與分析---子計畫三：機器學習方法於視訊編碼與處理應用

Abstract

國立交通大學資訊工程學系 計畫名稱： 機器學習方法於視訊編碼與處理應用 計畫主持人： 彭文孝（主持人） 經費來源： 行政院國家科學委員會 關鍵字： 機器學習、高效能視訊編碼、精簡描述子應用於視覺化搜尋、編碼端 控制、視覺化搜尋

本子計晝利用機器學習（Machine Learning)為視訊編碼與處理的應用設計高效能演 算法。近年來由於大量影像及影片資料的產生與取得變得容易，以往模型導向 (Model-based)在視訊編碼與處理中存在的一些問題，可望用資料導向（Data-driven) 的方式來解決。又機器學習的方法非常適合處理資料導向的問題，因此本子計晝目標為 利用機器學習的方法，針對（1)高效能視訊編碼（High Efficiency Video Coding, HEVC) 與（2)精簡描述子應用在視覺化搜尋（Compact Descriptors for Visual Search, CDVS) 兩項ISO/IEC MPEG與ITU-T VCEG正在開發的標準，為視訊編碼及處理兩項重要的多媒 體應用，設計高效能之演算法。前兩年我們會參與HEVC及CDVS之標準制定會議。同時 計晝（1)第一年產生大量HEVC編碼訓練資料，以機器學習方法解決分群及參數估測問 題，完成HEVC編碼端快速模式決策、快速位元率控制及快速編碼工具參數產生之演算 法開發；（2)第二年解決目前CDVS資料庫決定代表點的問題，使用機器學習方式分析並 設計更好的代表點選擇演算法，以及使用機器學習的方法設計使用多種比對演算法之搭 配方式，達到更高的比對準確率；（3)第三年基於兩項完成的標準，設計增進原有的演 算法效能之方法，以及改進演算法使其適用硬體設計。由於兩項標準在兩年内就會完 成，本子計晝發展之研究與演算法趕上標準的時程，將在隨之而來的應用上具有重要的 影響力。

Department of Computer Science, National Chiao Tung University Proposal Title: Application of Machine Learning Approach to Video Coding

and Processing Principle Investigator: Wen-Hsiao Peng (PI)

Sponsor National Science Council

Keywords: Machine Learning, High Efficiency Video Coding (HEVC),

Compact Descriptors for Visual Search (CDVS), Encoder control, Visual Search This project aims at applying Machine Learning (ML) approaches to video coding and processing. Recently, production and acquisition of large amount of video/picture data become easier than before. Therefore some problems existing in the model-based video encoding and processing are expected to be solved by data-driven approaches. Since ML is good at dealing with data-driven problems, in this project we shall design highly efficient algorithms for video coding and processing via ML approaches. Our algorithms are based on two international standards being developed by ISO/IEC MPEG and ITU-T VCEG: (1) High Efficiency Video Coding (HEVC) and (2) Compact Descriptors for Visual Search (CDVS).

In this 3-year project, we shall accomplish the following works.⑴ In the 1styear the ML approaches are applied to a large amount of data from HEVC compressed videos, trying to solve some difficult classification and parameter estimation problems. Those solutions will lead to the development of fast mode decision, fast rate control and fast parameter estimation, for the HEVC encoder control. (2) For the 2nd year, we study the problem of keypoint selection in the current design of CDVS database construction. ML approaches will be applied to analyze and develop a better keypoint selection algorithm. Moreover, we shall use the ML approach to develop a novel descriptor matching algorithm by merging several well-known matching methods, hoping to achieve higher matching accuracy. (3) Finally in the 3rd year we shall refine our algorithms and enhance them to be hardware-friendly. Furthermore, since HEVC and CDVS standards will be finalized within two years, we shall keep following the standard activities in the first two years.