AFCLNet: An Attention and Feature-Consistency-Loss-Based Multi-Task Learning Network for Affective Matching Prediction in Music–Video Clips
Sensors · 2025
DOI: 10.3390/s26010123 ↗Kurz gefasst
Fachartikel in der Zeitschrift Sensors, 2025. Verfasst von Zhibin Su und 4 weiteren Autor:innen. Beteiligte Einrichtung: State Key Laboratory of Media Convergence and Communication, Beijing 100024, China. Hauptthemen laut MEDLINE: Music, Attention, Emotions, Machine Learning. Die Arbeit wurde bisher 1-mal zitiert (Crossref) und nennt 45 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: the National Key Research and Development Program of China, the National Natural Science Foundation of China.
Abstract (Originalsprache)
Emotion matching prediction between music and video segments is essential for intelligent mobile sensing systems, where multimodal affective cues collected from smart devices must be jointly analyzed for context-aware media understanding. However, traditional approaches relying on single-modality feature extraction struggle to capture complex cross-modal dependencies, resulting in a gap between low-level audiovisual signals and high-level affective semantics. To address these challenges, a dual-driven framework that integrates perceptual characteristics with objective feature representations is proposed for audiovisual affective matching prediction. The framework incorporates fine-grained affective states of audiovisual data to better characterize cross-modal correlations from an emotional distribution perspective. Moreover, a decoupled Deep Canonical Correlation Analysis approach is developed, incorporating discriminative sample-pairing criteria (matched/mismatched data discrimination) and separate modality-specific component extractors, which dynamically refine the feature projection space. To further enhance multimodal feature interaction, an Attention and Feature-Consistency-Loss-Based Multi-Task Learning Network is proposed. In addition, a feature-consistency loss function is introduced to impose joint constraints across dual semantic embeddings, ensuring both affective consistency and matching accuracy. Experiments on a self-collected benchmark dataset demonstrate that the proposed method achieves a mean absolute error of 0.109 in music–video matching score prediction, significantly outperforming existing approaches.
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Stichwörter der Autor:innen
Autor:innen und Institutionen
- State Key Laboratory of Media Convergence and Communication, Beijing 100024, China.
- Key Laboratory of Acoustic Visual Technology and Intelligent Control System, Ministry of Culture and Tourism, Beijing 100024, China.
- School of Information and Communication Engineering, Communication University of China, Beijing 100024, China.
Förderung
- the National Key Research and Development Program of China
- the National Natural Science Foundation of China
Zitieren
APA
Su, Z., Liu, J., Zhang, L., Feng, Y., & Ren, H. (2025). AFCLNet: An Attention and Feature-Consistency-Loss-Based Multi-Task Learning Network for Affective Matching Prediction in Music–Video Clips. Sensors. https://doi.org/10.3390/s26010123
Vancouver
Su Z, Liu J, Zhang L, Feng Y, Ren H. AFCLNet: An Attention and Feature-Consistency-Loss-Based Multi-Task Learning Network for Affective Matching Prediction in Music–Video Clips. Sensors. 2025. doi:10.3390/s26010123
BibTeX
@article{Su2025,
author = {Su, Zhibin and Liu, Jinyu and Zhang, Luyue and Feng, Yiming and Ren, Hui},
title = {AFCLNet: An Attention and Feature-Consistency-Loss-Based Multi-Task Learning Network for Affective Matching Prediction in Music–Video Clips},
journal = {Sensors},
year = {2025},
doi = {10.3390/s26010123}
}
EUVIMED