A Hybrid Attention-Based Residual Neural Network for Multimodal Emotional Quadrant Classification Integrating Physiological and Behavioral Metrics for High-Precision Affective Computing

Source abstract

This study introduces a novel hybrid attention-based Residual Neural Network (ResNet) to address the challenge of high-precision emotional quadrant classification in affective computing by introducing a novel hybrid attention-based Residual Neural Network (ResNet) that integrates physiological and behavioral metrics. Moving beyond discrete categorization, the study targets the nuanced valence-arousal quadrant model to better represent complex affective states, yet current systems are limited by ineffective multimodal fusion and poor generalization. Our proposed architecture dynamically fuses temporal physiological signals with spatial-temporal behavioral data through a dedicated cross-modal attention mechanism, which adaptively weights the most discriminative features for each quadrant. Employing rigorous subject-independent validation, the model demonstrates exceptional performance, achieving 97.00% accuracy, a 97.02% macro F1-score, and a near-perfect ROC-AUC of 0.9982. These results establish a new state-of-the-art, confirming that synergistic, attention-guided fusion is essential for robust and granular emotion recognition. This work significantly advances the field by providing a powerful, interpretable framework for developing reliable emotion-aware systems with direct applications in personalized mental health monitoring and responsive human-computer interaction.