Self Supervised Adversarial Domain Adaptation for Cross-Corpus and Cross-Language Speech Emotion Recognition
Article
Article Title | Self Supervised Adversarial Domain Adaptation for Cross-Corpus and Cross-Language Speech Emotion Recognition |
---|---|
ERA Journal ID | 200608 |
Article Category | Article |
Authors | Latif, Siddique (Author), Rana, Rajib (Author), Khalifa, Sara (Author), Jurdak, Raja (Author) and Schuller, Bjorn (Author) |
Journal Title | IEEE Transactions on Affective Computing |
Journal Citation | 14 (3), pp. 1912-1926 |
Number of Pages | 15 |
Year | 2023 |
Publisher | IEEE (Institute of Electrical and Electronics Engineers) |
Place of Publication | United States |
ISSN | 1949-3045 |
Digital Object Identifier (DOI) | https://doi.org/10.1109/TAFFC.2022.3167013 |
Web Address (URL) | https://ieeexplore.ieee.org/document/9756868 |
Abstract | Despite the recent advancement in speech emotion recognition (SER) within a single corpus setting, the performance of these SER systems degrades significantly for cross-corpus and cross-language scenarios. The key reason is the lack of generalisation in SER systems towards unseen conditions, which causes them to perform poorly in cross-corpus and cross-language settings. Recent studies focus on utilising adversarial methods to learn domain generalised representation for improving cross-corpus and cross-language SER to address this issue. However, many of these methods only focus on cross-corpus SER without addressing the cross-language SER performance degradation due to a larger domain gap between source and target language data. This contribution proposes an adversarial dual discriminator (ADDi) network that uses the three-players adversarial game to learn generalised representations without requiring any target data labels. We also introduce a self-supervised ADDi (sADDi) network that utilises self-supervised pre-training with unlabelled data. We propose synthetic data generation as a pretext task in sADDi, enabling the network to produce emotionally discriminative and domain invariant representations and providing complementary synthetic data to augment the system. The proposed model is rigorously evaluated using five publicly available datasets in three languages and compared with multiple studies on cross-corpus and cross-language SER. Experimental results demonstrate that the proposed model achieves improved performance. |
Keywords | Adaptation models; adversarial learning; Australia; domain adaptation; Emotion recognition; Generators; self-supervised learning; Speech emotion recognition; Speech recognition; Task analysis; Training |
Related Output | |
Is part of | Deep Representation Learning for Speech Emotion Recognition |
Contains Sensitive Content | Does not contain sensitive content |
ANZSRC Field of Research 2020 | 461101. Adversarial machine learning |
460802. Affective computing | |
461104. Neural networks | |
461103. Deep learning | |
Public Notes | Files associated with this item cannot be displayed due to copyright restrictions. |
This article is part of a UniSQ Thesis by publication. See Related Output. | |
Byline Affiliations | School of Mathematics, Physics and Computing |
Commonwealth Scientific and Industrial Research Organisation (CSIRO), Australia | |
Queensland University of Technology | |
Imperial College London, United Kingdom | |
Institution of Origin | University of Southern Queensland |
https://research.usq.edu.au/item/q742q/self-supervised-adversarial-domain-adaptation-for-cross-corpus-and-cross-language-speech-emotion-recognition
172
total views3
total downloads0
views this month0
downloads this month