同步分級神經網路在CUDA架構上的平行化研究

Abstract

在本篇論文中，我們將藉由NVDIA公司提出的一個利用圖形處理器(Graphic Processor Unit，GPU)的運算架構，名為計算統一設備架構(Compute Unified Device Architecture，CUDA)，模擬同步分級神經網路(Synchronous Ranked Neural-Network，SRNN)的平行分塊更新。另外，針對SRNN模型的運作特性，討論分塊更新及神經元等級分佈，對我們的SRNN模型最後達收斂態所需計算量造成的影響，希望最後能夠針對我們SRNN模型所要處理的問題，找出一些較佳的設定，使得我們能夠用較少的計算量，就讓我們的SRNN模型達收斂狀態。最後，我們利用分波多工光相位排列交換器互連系統(WDM OPAS-based Optical Interconnect System，WOPIS)內的封包排程問題，作為我們實作SRNN模型平行分塊更新的處理問題，並且從中驗證我們所提出的兩個影響因素，是否如我們所預料的方式，對收斂所需計算量造成影響。另外再針對CUDA在各個平行度下執行效率的觀察，找出一個更新區塊個數及平行度最理想的權衡比例。

In this thesis, we using the compute unified device architecture (CUDA) which NVIDIA announced a graphic processor unit (GPU) computing architecture to simulate the feature that synchronous ranked neural network (SRNN) can be updated synchronously. And aim at the SRNN operating feature, we discuss the effect of block update manner and neurons’ rank distribution on the amount of computation of SRNN convergence. We hope that we can aim at different SRNN handling problems to find some better setting for minimum computation. In the end, we use the packet scheduling of WDM OPAS-based optical interconnect system (WOPIS) problem as our SRNN model’s handling problem in block update manner. And prove the two effect factors that is as our expectation. In addition, we also want to find the best trade-off between the number of update block and execution parallelism by observing the execution efficiency in different parallelism.