Development and Validation of an EEG-Based Real-Time Emotion Recognition System Using Edge AI Computing Platform With Convolutional Neural Network System-on-Chip Design

Abstract

This study proposed an electroencephalogram (EEG)-based real-time emotion recognition hardware system architecture based on multiphase convolutional neural network (CNN) algorithm implemented on a 28-nm technology chip and on field programmable gate array (FPGA) for binary and quaternary classification. Sample entropy, differential asymmetry, short-time Fourier transform, and a channel reconstruction method were used for emotion feature extraction. In this work, six EEG channels were selected (FP1, FP2, F3, F4, F7, and F8), and EEG images were generated from spectrogram fusions. The complete CNN architecture included training and acceleration for efficient artificial intelligence (AI) edge application, and we proposed a multiphase CNN execution method to accommodate hardware resource constraints. Datasets of 32 subjects from the DEAP database were used to validate the proposed design, exhibiting mean accuracies for valance binary classification and valance-arousal quaternary classification of 83.36% and 76.67%, respectively. The core area and total power consumption of the CNN chip were 1.83 x 1.83 mm(2), respectively, and 76.61 mW. The chip operation was validated using ADVANTEST V93000 PS1600, and the training process and real-time classification processing time took 0.12495 ms and 0.02634 ms for each EEG image, respectively. The proposed EEG-based realtime emotion recognition system included a dry electrode EEG headset, feature extraction processor, CNN chip platform, and graphical user interface, and the execution time costed 450 ms for each emotional state recognition.