以電阻式突觸元件為基礎之類神經網路硬體平台與圖型識別應用

Abstract

研究如何仿效大腦作為下一世代的電腦運算方式的趨勢，除了能克服傳統von Neumann運算架構的瓶頸(計算與記憶單元獨立)，此外人腦還有低功耗(~10W)、高密度和平行運算等特性，更能有效地提高能源效率。目前投入仿生大腦的研究團隊越來越多，也已經有許多傑出的成果，無論是軟體方面的成就(例如: IBM Waston, Facebook DeepFace, 和Google DeepMind AlphaGo)，或是硬體方面的成果(例如: IBM TrueNorth Stanford Neurogid, UHEI BrainscaleS和UoM SpiNNaker)。然而這些以純電晶體製作的仿生系統(neuromorphic system)研究終究無法達到模仿人腦所需的龐大計算量和高密度神經網路。因此有研究團隊提出利用電阻式記憶體(resistive random-access memory, RRAM)作為電子突觸元件以克服龐大計算與高密度神經網路的困難。電阻式記憶體為雙端點的元件，其結構簡單(金屬-氧化層-金屬)，具有利於微縮以及應用於高密度陣列的潛力、操作速度快與低功耗等優點；在仿生運算中，可利用其可調的阻值特性來仿效生物神經之間突觸權重的變化 (稱作突觸可塑性 synaptic plasticity)，此外因為結構上的優勢，更是利於平行運算與操作，能大幅減少仿生系統中龐大的運算量。由於上述種種優勢，使用電阻式記憶體作為電子突觸元件於仿生運算中是相當明確的方向。 本篇論文主要致力於研究電阻式記憶體與類神經電路之整合，依據電阻式記憶體元件特性與贏者全拿演算法，設計類神經網路硬體的測試流程，探討實際元件特性如何影響系統，最終展示一二元圖型識別系統的硬體量測結果。 論文內容分為五個章節，主要研究內容於第二至四章: 第二章中將會介紹類神經網路的硬體量測平台建立與整合，說明類神經網路硬體的基本功能單位: (1) 電子突觸元件 (以電阻式記憶體當作仿生突觸) 、(2)神經電路(以CMOS為基礎的板級電路)、與(3) 控制介面運行(以FPGA控制神經電路)，而我們建立的硬體量測與分析平台為利用示波器擷取欲觀測的訊號傳送至電腦，再以Matlab軟體做數據分析。我們可以藉由此硬體量測平台讀取陣列內元件阻態的變化，以利觀測神經網路訓練的過程。 第三章首先介紹基本贏者全拿演算法的硬體系統運作方式，以及如何利用此硬體架構來測量單顆元件的特性。在實際量測中需要考量電阻式記憶體元件的特性，說明元件特性如何影響我們系統的運作。接著依據元件的特性改良演算法的運行，設計硬體語言(verilog)完善訊號量測的流程，其中利用到硬體語言設計狀態機 (finite state machine, FSM)和參數傳遞 (parameters propagation)的概念。 第四章則是展示6×2圖型識別系統的硬體量測結果，我們從硬體量測結果與數值上的推導，分析元件特性如何影響整個系統的運行，其中元件特性主要探討元件的資料保存性(retention)、變異性(variation)和記憶體視窗(window)大小。

Brain-inspired computing (neuromorphic computing) is a prospective trend for future computing paradigm. Brain-inspired computation can not only conquer the bottleneck of the conventional von Neumann computation architecture but also has many excellent characteristics, such as low power consumption, excellent fault tolerance, massive parallelism, and dual functions of storage and computation. Today there have been many standout demonstrations in software-based neuromorphic computing, such as IBM Waston, Facebook DeepFace, and Google DeepMind AlphaGo, and also successful research programs in hardware-based neuromorphic computing, such as IBM TrueNorth, Stanford Neurogid, UHEI BrainscaleS, and UoM SpiNNaker. However, these purely CMOS-based approaches may not achieve required computation capability and density in the biological neural network of human brains. Therefore, some people proposed to investigate using RRAM (resistive random-access memory ) as the synaptic device. RRAM is a two-terminal device, with a simple M-I-M (metal-insulator-metal) structure. It has promising scaling potential for high-density applications, and it also shows the ability of fast operation speed and low operation power. Furthermore, in neuromorphic application, its adjustable resistance can be employed to mimic biological synaptic weight change (so-called synaptic plasticity). Also, its crossbar array structure is suitable for parallel computation, which can dramatically accelerate neuromorphic computing. Consequently, using RRAM as the synaptic device is a promising direction of realizing neuromorphic hardware systems. In this thesis, we implement RRAM synaptic devices in hardware neural networks (HNNs). We investigate both device characteristics and winner-take-all hardware neural system, design the testing flow of HNN, and then demonstrate a binary pattern recognition function. There are five chapters in this thesis, and the main content of the research is described in Chapter 2 to Chapter 4. In Chapter 2, we construct the HNN testing platform. The basic functional units of HNN are (1) RRAM synapse unit, (2) CMOS neuron unit, and (3) FPGA control interface. Through understanding the functions of these units, we establish the testing and analysis platform by acquiring output signals from an oscilloscope and analyzing signals by Matlab. In Chapter 3, we introduce the function of winner-take-all hardware neural network. When constructing the RRAM-based HNN, the practical device properties, such as device retention, variation and window size, should be carefully considered. Based on the hardware system and practical device properties, we modify the proposed algorithm, and design hardware description language (verilog) to improve the testing flow by using the concepts of finite state machine (FSM) and parameters propagation. In Chapter 4, a binary pattern recognition function is demonstrated. We also investigate how the device properties affect the HNN system operation and the final recognition results by analyzing the discrepancy between the experimental results and analytical derivation.