A Novel Convolutional Neural Networks for
Emotion Recognition Based on EEG Signal
Zhiyuan Wen, Ruifeng Xu (), Jiachen Du
School of Computer Science and Technology,
Shenzhen Graduate School, Harbin Institute of Technology, Shenzhen, China
wenzhiyuan2012@gmail.com, xuruifeng@hit.edu.cn, dujiachen@stmail.hitsz.edu.cn
Abstract — Emotion recognition based on
electroencephalogram (EEG) signal is attracting more and more
attention. Many feature engineering based models have been
investigated. However, these models require a lot of effort for
manually designing feature set. And these features can be hardly
transformed among different problems. To reduce the manual
effort on features used in EEG-based recognition and improve the
performance, we propose an end-to-end model which is based on
Convolutional Neural Networks (CNNs). In order to represent the
EEG signals better, the original channels of EEG are firstly
rearranged by Pearson Correlation Coefficient and the
rearranged EEGs are fed into CNN. experiments were carried on
DEAP dataset. The experimental results on the DEAP dataset
show that the proposed method achieves 77.98% accuracy on the
Valence recognition and 72.98% on the Arousal recognition.
Keywords—emotion recognition; EEG signal; Pearson
correlation coefficient features; convolution neural network;
combination method
I. INTRODUCTION
Emotion recognition is attracting more and more attention as
a hot topic in various fields such as computational neuroscience,
human-computer interaction, affective computing and et al.
Emotional recognition aims at identifying the human emotion by
analyzing and processing emotion-related signals automatically.
The emotional-related signals include physiological and non-
physiological signals. The physiological signals can be camped
into peripheral physiological signals and central nervous signals.
As a typical kind of central nervous signal,
Electroencephalogram (EEG) signal directly reflects the
strength and position of the brain activity with high temporal
resolution. Therefore, the emotion recognition based on EEG
signal is an emerging topic in emotion recognition. Recently
most of EEG-based emotion recognition methods are based on
the features of single-channel EEG features, including the
features of time domain, frequency domain and so on. However
these methods takes less consideration of the relationship
between different channels.
EEG signal belongs to the physiological signal of the central
nervous signal, it is a direct response of brain activity and has a
good time resolution. Therefore more and more scholars begin
to study EEG signal recognition. The conventional procedure of
EEG signal recognition is EEG signal preprocessing, feature
extraction and recognition using machine learning methods.
CNNs is a kind of End-to-End models. The End-to-End
models based on deep neural network learn the mapping from
the original input to the expected output effectively through the
deep neural network. It avoids the complicated manually feature
design and selection. But using CNNs in EEG emotion
recognition directly can hardly achieve an ideal result. The
reason is that the order of channels of input which fed into CNNs
need to be meaningful. However, the original EEG channels are
not arranged their orders according to their characteristic. The
proximity of the channels do not reflect the value of the relevant
information between the channels. Therefore, the strategy of
increasing the amount of information on adjacent channels by
channel rearrangement will help CNNs to learn more effectively.
Meanwhile, Pearson correlation coefficient features can
represent the connectivity information between different EEG
signal channels.
In this paper, we propose a emotion recognition method
based on EEG signals. For improving the information of
adjacent channels, two different algorithm are proposed using
Pearson correlation coefficient to rearrange the original EEG
channels. The Pearson correlation coefficient features are
combined with CNNs, in which the Pearson correlation
coefficient imports the correlation information of Long-distance
channels into CNNs to enhance performance of emotion
recognition. The experimental results on DEAP dataset, an
international standard dataset for EEG-based emotion
recognition, show that the proposed method is effective.
The result of the paper is organized as follows. Section 2
introduces the relevant research on emotion recognition based
on EEG signals. Section 3 mainly introduces the method we
proposed. Section 4 mainly introduces the experiment and
discussions. Finally, Section 5 concludes.
II. RELATED WORKS
The features used in emotion recognition based on EEG can
be mainly classified into time domain features, frequency
domain features, time-frequency domain features, spatial
distribution features of EEG signals and brain network features.
The time domain features include simple signal statistics [1],
Hjorth index [2], non-stationary index [3]and etc. These time
domain features often have the characteristics of strong time-
sensitive but poor anti-noise. The frequency domain features
include different frequency spectrum power [4], short time
Fourier transform feature (STFT) [5], wavelet analysis feature
[6] and so on. The main characteristics of frequency domain