Cookies on this website

We use cookies to ensure that we give you the best experience on our website. If you click 'Accept all cookies' we'll assume that you are happy to receive all cookies and you won't see this message again. If you click 'Reject all non-essential cookies' only necessary cookies providing core functionality such as security, network management, and accessibility will be enabled. Click 'Find out more' for information on how to change your cookie settings.

Emotion recognition in conversations using artificial intelligence (AI) has recently gained a lot of attention, as it can provide additional emotion cues that can be correlated with human social behavior. An extension towards an AI-based emotional climate (EC) recognition, i.e., the recognition of the joint emotional atmosphere dynamically created and perceived by the peers throughout a conversation, is proposed here. In our approach, namely MLBispeC (Machine Learning Based Bispectral Classification), the peers' speech signals during their conversation are subjected to time-windowed bispectral analysis, allowing for feature extraction related to dynamic harmonics nonlinear interactions. In addition, peers' affect dynamics, derived from their same time-windowed emotion labeling, are combined to form an extended feature vector, inputted into two well-known machine learning classifiers (Support Vector Machine, K-Nearest Neighbor). MLBispeC was evaluated on the Interactive Emotional Dyadic Motion Capture (IEMOCAP) open access dataset, which contains 2D emotions, i.e., Arousal (A) and valence (V) that are divided into (low/high) classes. The experimental results have shown that MLBispeC outperforms previous state-of-the-art techniques, achieving an accuracy of 0.826A/0.754V, sensitivity of 0.864A/0.774V, and area under the curve (AUC) of 0.821A/0.799V. This demonstrates the effectiveness of MLBispeC to objectively recognize peers' EC during their conversation, allowing for insights into their emotional and social interactions.Clinical relevance-Unobtrusive, objective and dynamic recognition of the EC built during peers' conversation can scaffold effective assessment of patients with physiological, psychological, and mental diseases, at various age ranges (children, adults, and older adults).

Original publication

DOI

10.1109/EMBC40787.2023.10340028

Type

Conference paper

Publication Date

07/2023

Volume

2023

Pages

1 - 5

Keywords

Child, Humans, Aged, Speech, Artificial Intelligence, Emotions, Recognition, Psychology, Arousal