【神经科技】中国科学家发现:伽马波段脑电波对情绪分类的作用(英文版)

发布时间:2017-06-08 16:30  浏览次数:

31st  Annual lnternational Conference ofthe IEEE  EMBS

 Minneapolis,Minnesota, USA ,September 2-6,2009

 

Emotion  Classification Based  on Gamma-band EEG

Mu  Li  and  Bao-Liang  Lu* Senior  Member, IEEE

        Abstract—In this paper, we use EEG signals to classify two  emotions—happiness and sadness. These emotions are evoked  by showing subjects pictures of smile and cry facial expressions. We propose a frequency band searching method to choose an

optimal band into which the recorded EEG signal is filtered.We use common spatial patterns (CSP) and linear-SVM to  classify these two emotions. To investigate the time resolution  of classification, we explore two kinds of trials with lengths  of 3s and 1s. Classification accuracies of 93.5%±  6.7% and  93.0%±6.2% are achieved on 10 subjects for 3s-trials and 1strials,respectively. Our experimental results indicate that the

gamma band (roughly 30–100 Hz) is suitable for EEG-based  emotion classification.


I. INTRODUCTION

        Emotions play an essential role in many aspects of our  daily lives, including decision making, perception, learning,rational thinking and behavior. Assessing emotions is key to  understanding human nature. Emotion classification1 is a step

towards aiding people such as in care taking and designing  brain-computer interfaces.

        As a mental and physiological state, emotion is associated  with a wide variety of feelings, thoughts, and behaviors. The  modern study of emotions began in the 19-century. Various  models and theories have been proposed in psychology, cognition,

neuroscience and other disciplines. There is, however,much controversy concerning how emotions are to be defined  and discriminated. Whether emotions are cognitive or noncognitive  is one major question of interest. The former claims  that cognitive activities are necessary for an emotion to  occur [1], while the latter argues that emotional experience  is largely due to the experience of bodily changes.

        Another question is whether emotions are distinctive discrete  states or continuous ones. One opinion is to divide  emotions into basic and complex emotions, where the latter  are blended with the former [3]. Another opinion is to let  emotions vary along several scales with respect to the relations  between them. A well-known continuous model is the  valence-arousal model [4], in which the valence dimension  represents the scale from pleasant to unpleasant and the  arousal dimension indicates the intensity of excitement.


       Asterisk indicates corresponding author. This work was partially supported

by the National Natural Science Foundation of China (Grant No.60773090 and Grant No. 90820018), the National Basic Research Program  of China (Grant No. 2009CB320901), and the National High-Tech Research  Program of China (Grant No. 2008AA02Z315). M. Li and B. L. Lu are with  the Department of Computer Science and Engineering, Shanghai Jiao Tong  University, Shanghai 200240, China, and MOE-Microsoft Key Laboratory  for Intelligent Computing and Intelligent Systems, Shanghai Jiao Tong  University, Shanghai 200240, China. E-mail: bllu@sjtu.edu.cn

        1The term emotion classification is also used as the meaning of taxonomy  of emotions, but we refer it as the machine learning approach to classify  the emotions which the subject is experiencing using related signals.


        The EEG signals under different frequency bands have  gained much research interest. Typically, low frequencies  such as alpha and mu rhymes are related to vigilance and  motion while high EEG frequencies, like gamma, are relevant  to high cognitive processes. Researches continue to suggest  connections between gamma band activities and emotions [5][6]. Further, ERD/ERS responses to pictures of facial

expressions in the gamma band show that ERD decreased  150–350 ms after presenting the stimuli [7].


II. RELATED WORK

        In neuroscience and psychology, event related potential  (ERP) is popular in the research of the brains rapid processing  of affective stimuli [8]. In computer science, research  is focused on detecting human emotions from affective  displays or physiological signals. Several studies [9]  have utilized facial expressions, tone of voice, and body  movement to recognize emotions. However, those signals  share a disadvantage—they are not reliable affective displays.Emotions occur without corresponding facial emotional expressions  or tone changes and body movements, especially  when the emotion intensity is not very high. In addition, such

displays could easily faked, as when one is telling a lie.

        Many studies [10] utilized signals from peripheral nervous  system, e.g. electrocardiogram and skin impedance.Nevertheless, EEG–the signal directly recorded from central  nervous system–has not received much interest[11]. There  are only a few studies using EEG to classify emotions.Choppin [12] used neural networks to classify EEG signals  from three emotions and got 64% classification accuracy.Chanel et al. [13] also confirmed that EEG and other physiological  signals can be used to recognize emotions along  one arousal dimension. The classification results are around  70% using two classes and 60% using three classes. Bos  [14] classified arousal and valence emotions and obtained an  average accuracy of 70% for two classes.


III. EXPERIMENT


A. Subjects

        The study protocal conforms to local ethics guidelines.In total 10 subjects (2 females; mean age 25; all normal  sight and right handed) participated in our experiment, and  all were paid for their participation. Subjects were informed  about the purpose of this experiment.


B. Stimuli

        The stimuli, an excerpt is shown in Fig. 1, consisted of  two kinds of emotional facial expression pictures—smile and  cry. The smiling people were mainly Asian actors and the  others pictures were taken of people who recently lost family  members. Pictures were resized to be of similar size.



阿尤卡运动心理健康 【神经科技】中国科学家发现:伽马波段脑电波对情绪分类的作用(英文版)

Fig. 1. Excerpt of a sequence of stimuli. The first two are smile facial

pictures and the last two are cry facial pictures.



        This type of stimuli was chosen for two reasons. Facial  expressions are the main channels with which people use to  transmit emotions, and are universal recognized. Moreover, smiling and crying are the expressions most likely to evoke  empathy [3].


        The emotional contents of these pictures were measured  by a self-assessment manikin (SAM) [15] containing 9  scales for both valence and arousal dimensions. Each subject  was required to label every picture using SAM after the  experiments. The results of the valence-arousal scales were(2.51±0.91, 4.60±1.41) and (7.41±1.03,4.37±1.94) for  smiling and crying pictures, respectively.


C. Protocol

        The pictures were shown on a black background with  a visual angle of approximately 6× 6. Each picture was  presented for 6 seconds before a small horizontal bar was  presented for 1s to require the subject’s attention. Between  each trial, 3s of black screen was shown to allow subjects to  rest. We did not adopt a completely random stimuli sequence  to prevent subjects from feeling discomfort due to high

frequency change of different emotional pictures. Instead,we divided the pictures into groups that each group consists  of 5 randomly chosen pictures from the same class. Then,we randomly ordered 12 groups into a stimuli sequence as a session. Each experiment consists of 2 sessions, and between  each session was a 10 minute long rest to assure attention  during the whole trial.

        The experiment was carried out in an illuminated and  sound proof room. The temperature of the room was about 27  degrees and the humidity was between 40% and 60%. During  the experiment, subjects were asked to focus their attention  only on the facial expressions.They were also required to  keep their head and body steady during the presentation of  the pictures.


D. Data recording

        Subjects were fitted with a 62-channel electrode cap during  the experiment. The Ag/AgCl electrodes were mounted  inside the cap with bipolar references behind the ears. The  electrodes were arranged according to the international 10-20 system. The contact impedance between electrodes and  skin was kept to a value less than 10kΩ

. The EEG data was  recorded with 32-bit quantization level at a sampling rate of 1000Hz.


IV. METHOD

A. Artifact Detection

        The time wave and energy of each trial (the segment  of EEG when a single picture was present) were visually  checked. Trials seriously contaminated by electromyogram

(EMG) were manually removed. Trials that were removed  typically showed larger amplitude wave and energy (about  10 times), compared to normal ones. We removed an average  of 3 trials from each experiment.


B. Filter

        The EEG signal was filtered into a specific frequency band  after removing artifacts. We utilized Fourier transform (FT) to filter instead of using the widely used IIR or FIR filters.We firstly transformed the signal into frequency domain, then  set the unwanted frequency components to zero.

        Since we did not know the optimal band to filter, we  needed to search many bands. The IIR or FIR approach  requires a separate filtering every time for each band; and

thus has a high time complexity. For FT, however, we only  need to perform FFT once since we only need to calculate  the covariance matrix in the following steps [16].


C. Common Spatial Patterns

        Common Spatial Patterns (CSP) [17] is a surpervised  dimension reduction method that is suitable for extracting  ERD/ERS features. CSP searches directions to maximize

the variances of two kinds of signals projected to these  directions. Denote these two kinds of signals by阿尤卡运动心理健康 【神经科技】中国科学家发现:伽马波段脑电波对情绪分类的作用(英文版)where i1 = 1,...   .n1, i2 = 1,...  .n2, and n1 and n2  are the numbers of trials for each kind of signal. For each  trial阿尤卡运动心理健康 【神经科技】中国科学家发现:伽马波段脑电波对情绪分类的作用(英文版)which is a time  channel matrix, its covariance  matrix阿尤卡运动心理健康 【神经科技】中国科学家发现:伽马波段脑电波对情绪分类的作用(英文版)is calculated by considering channels (column)  as variables. The mean covariance matrix阿尤卡运动心理健康 【神经科技】中国科学家发现:伽马波段脑电波对情绪分类的作用(英文版)for each class  is

阿尤卡运动心理健康 【神经科技】中国科学家发现:伽马波段脑电波对情绪分类的作用(英文版)

Now, CSP finds the directions w, which is a channel  ×1  vector, to minimize or maximize阿尤卡运动心理健康 【神经科技】中国科学家发现:伽马波段脑电波对情绪分类的作用(英文版)  . This optimization  problem is equal to the generalized eigenvalue equation,

阿尤卡运动心理健康 【神经科技】中国科学家发现:伽马波段脑电波对情绪分类的作用(英文版)

The eigenvalue 入 stands for the ability of the direction w to  discriminate two classes trials—weak when 入 is near 1 and  strong when  入 is larger or smaller. Let w1,...    wc be the  directions according to the eigenvalues sorted in ascending  order, where c is the number of channels. Then, m  directions  are selected to deduce the dimension

阿尤卡运动心理健康 【神经科技】中国科学家发现:伽马波段脑电波对情绪分类的作用(英文版)

D. Classification

        After deducing the dimension using CSP, we fed the  logarithm variance of the dimension-deduced trials as the  features into a linear support vector machine (linear-SVM)  [18]. Let the feature of a trial D be f, then f was computed  as


阿尤卡运动心理健康 【神经科技】中国科学家发现:伽马波段脑电波对情绪分类的作用(英文版)

where Var() computed the variance of each column, and  diag() denoted the diagonals of a matrix.In order to obtain reliable classification result, we randomly  divided the trials into training set and testing set  with ratio 7 : 3. The parameters, frequency band and m,

were selected using 5-fold cross validation on the training  set. After that, we performed CSP on the training set and  calculated the features for both training set and testing set.

The former was fed into a linear-SVM and the latter was  used to test classification accuracy


阿尤卡运动心理健康 【神经科技】中国科学家发现:伽马波段脑电波对情绪分类的作用(英文版)

Fig. 2. Classification accuracies using different frequency bands for two

subjects. The low and high cut-offs are presented in X-axis and Y-axis,

respectively. The intensity represents the accuracy.

V. RESULTS

        We divided the original 6s length trials into two kinds of  short trials, 3s and 1s, to increase the number of classification  trials and demonstrate our ability to classify emotions with a  high time resolution. Each experiment consists of around 240  trials for 3s-trials and around 720 trials for 1s-trials (several  EMG contaminated ones were removed, Sec. IV-A) .


A. Frequency band selection

        The cross validation results on the training set of 3s-trials  for frequency bands under 200Hz are shown in Fig. 2. One  can observe five interesting facts from the figure. One, the  high performance areas are in the shape of a vertical strip.The optimal strips always reach the region whose band width  is at most 50Hz. Two, the low cut-offs of the optimal strips  in the figure are both around 40–50 Hz, despite the fact that  the highest accuracies are different. This fact does not hold  for other subjects. Three, the high cut-offs of bands with  acceptable accuracy are always above 30 Hz; this holds for  all subjects. Four, both low and high cut-off frequencies of  suitable bands are enough to enter into the 100Hz–150Hz  range. This was surprising. Five, one can clearly note that

accuracy varies much with the frequency band and the  suitable frequency band distribution varies across subjects.Therefore, searching the suitable band for each subject is  necessary.

        Inspired by these observations, we chose a band selection  method. The basic idea is that, if we have chosen a suitable  low cut-off, then we are limited to a few several high cutoffs  not far from the low cut-off. Since it is not practical to  search every low cut-off for each experiment, we only choose  several bands with the low cut-off of {31,36,...91 }Hz  and a width of{5,10,...,50 }Hz. Denote  r(i, j)the cross validation results on these bands, where i = 1;    ; 25 and  j = 1,..., 10. We calculate the mean result for each low  cut-off, that is, r(i) =阿尤卡运动心理健康 【神经科技】中国科学家发现:伽马波段脑电波对情绪分类的作用(英文版)Then we select the low

cut-off with maximum阿尤卡运动心理健康 【神经科技】中国科学家发现:伽马波段脑电波对情绪分类的作用(英文版)namely argmaxi阿尤卡运动心理健康 【神经科技】中国科学家发现:伽马波段脑电波对情绪分类的作用(英文版)At last we  select the band width such that argmaxjr(i; j) and we get  the optimal band.


B. Classifier parameters

        We need to choose the dimension reduction m for CSP,which is used to control the complexity of the classifier.We used the default settings of the linear LibSVM [18].

Though SVM can efficiently avoid over-fitting, considering  the number of trials, feature dimension, and the low signalnoise  ratio of EEG signal, the curse of dimension is still

a big problem. In our method, four different values, m =2; 4; 20; 40, were considered. We chose m with average good  cross validation performance.


C. Classification Accuracy

        Using the selected parameters, we performed CSP on the  filtered training set. Then, the features of the training set  were used to train a linear-SVM. We then obtained the testing  accuracy on the testing set features.


        The testing accuracy of 3s-trials, see Table I, is 93.5% ±   6.7%, with 5 subjects (1, 4, 5, 7, 8) above 95%; and of 1strials  is 93.0% ±  6.2%, with 6 subjects (1, 4, 5, 7, 8, 10)above 95%.


TABLE I

CLASSIFICATION RESULTS FOR 10 SUBJECTS. EACH EXPERIMENT CONTAINED AROUND 240 3S-TRIALS OR 720 1S-TRIALS, OF WHICH 70% WERE

USED TO SELECT PARAMETERS BY 5-FOLD CROSS VALIDATION AND THE REST WERE USED FOR TESTING. THE PARAMETERS, LOW AND HIGH CUT-OFF

FREQUENCY, NUMBER OF CSP FEATURES, AND TESTING ACCURACY WERE SHOWN IN ROWS FOR EACH SUBJECT.


阿尤卡运动心理健康 【神经科技】中国科学家发现:伽马波段脑电波对情绪分类的作用(英文版)


VI. DISCUSSION

        The subjects whose results are greater than 95% and  ones whose results less than 85% point to the diversity of  subjects and quality of experiments—some claimed that they  were emotional arosed by the stimuli while others said they  experienced little emotion.


        The average optimal frequency bands are 43.5–68.5 Hz  for 3s- and 63.5–94.5 Hz for 1s-trials. Most bands are in the  gamma band. The result confirms that GBA is related to the  emotions of happiness and sadness.


        When comparing the results of 3s- and 1s-trials, it is  interesting to see that using short length trials does not  reduce the classification accuracy by much, and even induces  improvement for several subjects. This means that 1s EEG  signals are enough to classify emotions.


VII. CONCLUSION

        These two different emotions—smiling and crying— were  classified based on EEG signals. We received 93:5%6:7%,and 93.0% ± 6.2% classification accuracies on 10 subjects for  3s length and 1s length trials using CSP, SVM and frequency  band selection strategies. Our experimental results indicate  that the ERD/ERS activities in gamma band EEG can be used  to classify happiness and sadness with high time resolution.




来自世界卫生组织WHO的重要事实:

  • 抑郁症是一种常见的精神疾患,全球有超过3亿人患有抑郁症(中国9千万),遍布各年龄组;预计到2020年抑郁症将成为仅次于冠心病的人类第二大疾患

  • 抑郁症是世界各地的首要致残原因,它是导致全球疾病负担的一个重大因素;最严重时,抑郁症可以引致自杀死亡(每年近80万人)

  • 可以有效治疗抑郁症(有规律的运动对于轻度和中度抑郁症患者等同于吃药的疗效)

阿尤卡运动心理健康 【神经科技】中国科学家发现:伽马波段脑电波对情绪分类的作用(英文版)

👇,更多有关运动提升心理健康的科学与故事

标签:
脑电波