EUVIMEDEUVIMED entdecken
Crossref

FusionMind: Reliability-Aware Adaptive Fusion for Multimodal Sentiment Classification

Gundeti Mounika, Vimula Shwetha, Dr K Sridhar

ETDT · 2026

DOI: 10.65713/etdtv2i160 ↗

Kurz gefasst

Fachartikel in der Zeitschrift ETDT, 2026. Verfasst von Gundeti Mounika und 2 weiteren Autor:innen. Die Arbeit nennt 15 Literaturangaben.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Multimodal sentiment classification combines textual information, acoustic features, and visual expressions to identify the sentiment conveyed by a speaker. However, the reliability of these systems can decrease when one or more modalities contain noise, are unavailable, or provide conflicting information. To address these challenges, this paper proposes FusionMind, a reliability-aware hybrid deep learning framework for text, audio, and visual sentiment classification. The framework uses separate encoders for each modality, a shared representation space, and an adaptive gating mechanism that adjusts the contribution of each modality according to the input. A classification head generates sentiment predictions, while probability calibration is incorporated to improve the reliability of the predicted confidence scores. Modality dropout is included to examine the model's ability to handle incomplete inputs, and confidence-based selective prediction enables uncertain cases to be referred for further review. The proposed framework will be evaluated primarily on the CMU-MOSEI dataset, with CMU-MOSI used for additional evaluation. Comparisons will be conducted against unimodal classifiers, early-fusion methods, fixed-weight ensembles, and adaptive-fusion approaches. The evaluation will consider accuracy, macro-F1 score, negative log-likelihood, expected calibration error, selective risk, and inference latency. Since training and testing have not yet been conducted, no empirical performance improvements are claimed in this study.

Zitieren

APA

Mounika, G., Shwetha, V., & Sridhar, D. K. (2026). FusionMind: Reliability-Aware Adaptive Fusion for Multimodal Sentiment Classification. ETDT. https://doi.org/10.65713/etdtv2i160

Vancouver

Mounika G, Shwetha V, Sridhar DK. FusionMind: Reliability-Aware Adaptive Fusion for Multimodal Sentiment Classification. ETDT. 2026. doi:10.65713/etdtv2i160

BibTeX
@article{Mounika2026,
  author = {Mounika, Gundeti and Shwetha, Vimula and Sridhar, Dr K},
  title = {FusionMind: Reliability-Aware Adaptive Fusion for Multimodal Sentiment Classification},
  journal = {ETDT},
  year = {2026},
  doi = {10.65713/etdtv2i160}
}
Kostenlos starten →