Classifying emotions and engagement in online learning based on a single facial expression recognition neural network

AV Savchenko, LV Savchenko… - IEEE Transactions on …, 2022 - ieeexplore.ieee.org
In this article, behaviour of students in the e-learning environment is analyzed. The novel
pipeline is proposed based on video facial processing. At first, face detection, tracking and …

Facial expression and attributes recognition based on multi-task learning of lightweight neural networks

AV Savchenko - 2021 IEEE 19th International Symposium on …, 2021 - ieeexplore.ieee.org
In this paper, the multi-task learning of lightweight convolutional neural networks is studied
for face identification and classification of facial attributes (age, gender, ethnicity) trained on …

Multimodal emotion recognition using cross modal audio-video fusion with attention and deep metric learning

B Mocanu, R Tapu, T Zaharia - Image and Vision Computing, 2023 - Elsevier
In the last few years, the multi-modal emotion recognition has become an important research
issue in the affective computing community due to its wide range of applications that include …

Audio–visual fusion for emotion recognition in the valence–arousal space using joint cross-attention

RG Praveen, P Cardinal… - IEEE Transactions on …, 2023 - ieeexplore.ieee.org
Automatic emotion recognition (ER) has recently gained much interest due to its potential in
many real-world applications. In this context, multimodal approaches have been shown to …

Cross attentional audio-visual fusion for dimensional emotion recognition

RG Praveen, E Granger… - 2021 16th IEEE …, 2021 - ieeexplore.ieee.org
Multimodal analysis has recently drawn much interest in affective computing, since it can
improve the overall accuracy of emotion recognition over isolated uni-modal approaches …

A transformer-based model with self-distillation for multimodal emotion recognition in conversations

H Ma, J Wang, H Lin, B Zhang… - IEEE Transactions on …, 2023 - ieeexplore.ieee.org
Emotion recognition in conversations (ERC), the task of recognizing the emotion of each
utterance in a conversation, is crucial for building empathetic machines. Existing studies …

Multi-grained fusion network with self-distillation for aspect-based multimodal sentiment analysis

J Yang, Y Xiao, X Du - Knowledge-Based Systems, 2024 - Elsevier
Aspect-based multimodal sentiment analysis (ABMSA) is an important branch of multimodal
sentiment analysis. The goal of ABMSA is to use multimodal information to infer users' …

Videoadviser: Video knowledge distillation for multimodal transfer learning

Y Wang, D Zeng, S Wada, S Kurihara - IEEE Access, 2023 - ieeexplore.ieee.org
Multimodal transfer learning aims to transform pretrained representations of diverse
modalities into a common domain space for effective multimodal fusion. However …

A Survey of Deep Learning for Group-level Emotion Recognition

X Huang, J Xu, W Zheng, Q Mao, A Dhall - arXiv preprint arXiv:2408.15276, 2024 - arxiv.org
With the advancement of artificial intelligence (AI) technology, group-level emotion
recognition (GER) has emerged as an important area in analyzing human behavior. Early …

Incongruity-Aware Cross-Modal Attention for Audio-Visual Fusion in Dimensional Emotion Recognition

RG Praveen, J Alam - IEEE Journal of Selected Topics in Signal …, 2024 - ieeexplore.ieee.org
Multimodal emotion recognition has immense potential for the comprehensive assessment
of human emotions, utilizing multiple modalities that often exhibit complementary …