卷積神經網路於現代繪圖處理器架構之設計探索

Abstract

卷積類神經網路(CNNs)應用於複雜的機器學習作業上具有高準確度以及容忍輸入雜訊的能力，因此近年來十分受到矚目。卷積運算所需的龐大計算效能對軟體及硬體方面都造成了極大的挑戰，不論在展開平行度還是資料復用方面都必須經過精心設計才能獲得優越的效能。目前的設計並未針對各個設計技巧與決策做完整分析，也缺乏對平行化的維度等設計準則背後的原因的深入研究。本論文解釋了卷積的程式中所使用的設計技巧，並進行各設計技巧在GPU架構下之質化與量化的分析與其對效能的影響。最後，我們總結實驗結果之觀察，提出了卷積神經網路在GPU架構下的設計指南。

Convolutional Neural Networks (CNNs) have gained attention in recent years for their ability to perform complex machine learning tasks with high accuracy and resilient to noise in the inputs. The time-consuming convolution operations required by CNNs pose great challenges to both software as well as hardware designers. To achieve superior performance, a design involves careful concerns between exposing the massive computation parallelism and exploiting data reuse in complex data accesses. Existing designs lack comprehensive analysis on design techniques and decisions. The analytical discussion and quantitative proof behind the design criterion, such as choosing proper dimensions to parallelize, are not well studied. This thesis studies the design techniques of CNNs and performs a series of qualitative and quantitative studies on both the programming techniques and their implications on the GPU architecture. Finally, we summarize the observations from our work, and conclude the design guidelines of CNN algorithms on both parameter tuning and future GPU architectures.