Developing an EEG-Based Emotion Recognition Using Ensemble Deep Learning Methods and Fusion of Brain Effective Connectivity Maps
Article
Bagherzadeh, Sara, Shalbaf, Ahmad, Shoeibi, Afshin, Jafari, Mahboobeh, Tan, Ru-San and Acharya, U. Rajendra. 2024. "Developing an EEG-Based Emotion Recognition Using Ensemble Deep Learning Methods and Fusion of Brain Effective Connectivity Maps." IEEE Access. 12, pp. 50949-50965. https://doi.org/10.1109/ACCESS.2024.3384303
| Article Title | Developing an EEG-Based Emotion Recognition Using Ensemble Deep Learning Methods and Fusion of Brain Effective Connectivity Maps |
|---|---|
| Article Category | Article |
| Authors | Bagherzadeh, Sara, Shalbaf, Ahmad, Shoeibi, Afshin, Jafari, Mahboobeh, Tan, Ru-San and Acharya, U. Rajendra |
| Journal Title | IEEE Access |
| Journal Citation | 12, pp. 50949-50965 |
| Number of Pages | 17 |
| Year | 2024 |
| Place of Publication | United States |
| Digital Object Identifier (DOI) | https://doi.org/10.1109/ACCESS.2024.3384303 |
| Web Address (URL) | https://ieeexplore.ieee.org/document/10488403 |
| Abstract | The objective of this paper is to develop a novel emotion recognition system from electroencephalogram (EEG) signals using effective connectivity and deep learning methods. Emotion recognition is an important task for various applications such as human-computer interaction and, mental health diagnosis. The paper aims to improve the accuracy and robustness of emotion recognition by combining different effective connectivity (EC) methods and pre-trained convolutional neural networks (CNNs), as well as long short-term memory (LSTM). EC methods measure information flow in the brain during emotional states using EEG signals. We used three EC methods: transfer entropy (TE), partial directed coherence (PDC), and direct directed transfer function (dDTF). We estimated a fused image from these methods for each five-second window of 32-channel EEG signals. Then, we applied six pre-trained CNNs to classify the images into four emotion classes based on the two-dimensional valence-arousal model. We used the leave-one-subject-out cross-validation strategy to evaluate the classification results. We also used an ensemble model to select the best results from the best pre-trained CNNs using the majority voting approach. Moreover, we combined the CNNs with LSTM to improve recognition performance. We achieved the average accuracy and F-score of 98.76%, 98.86%, 98.66 and 98.88% for classifying emotions using DEAP and MAHNOB-HCI datasets, respectively. Our results show that fused images can increase the accuracy and that an ensemble and combination of pre-trained CNNs and LSTM can achieve high accuracy for automated emotion recognition. Our model outperformed other state-of-the-art systems using the same datasets for four-class emotion classification. © 2013 IEEE. |
| Keywords | Effective connectivity; electroencephalography; emotion recognition; long short-term memory; transfer learning |
| Contains Sensitive Content | Does not contain sensitive content |
| ANZSRC Field of Research 2020 | 461103. Deep learning |
| Byline Affiliations | Islamic Azad University, Iran |
| Shahid Beheshti University of Medical Sciences, Iran | |
| University of Granada, Spain | |
| School of Mathematics, Physics and Computing | |
| National Heart Centre, Singapore | |
| Duke-NUS Medical Centre, Singapore | |
| Centre for Health Research |
Permalink -
https://research.usq.edu.au/item/z8461/developing-an-eeg-based-emotion-recognition-using-ensemble-deep-learning-methods-and-fusion-of-brain-effective-connectivity-maps
Download files
Published Version
44
total views19
total downloads1
views this month3
downloads this month