EUVIMEDEUVIMED entdecken
Crossref

One-Versus-Others Attention: Scalable Multimodal Integration for Biomedical Data

Michal Golovanevsky, Eva Schiller, Akira Nair, Eric Han, Ritambhara Singh, Carsten Eickhoff

Biocomputing 2025 · 2024

DOI: 10.1142/9789819807024_0041 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Biocomputing 2025, 2024. Verfasst von Michal Golovanevsky und 5 weiteren Autor:innen. Beteiligte Einrichtung: §Department of Computer Science, Brown University, Providence, RI 02912, USA. Hauptthemen laut MEDLINE: Computational Biology. Die Arbeit wurde bisher 4-mal zitiert (Crossref).

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Multimodal models have become increasingly important as they surpass single-modality approaches on diverse tasks ranging from question-answering to disease diagnosis. Despite the importance of multimodal learning, existing efforts focus on vision-language applications, where the number of modalities rarely exceeds four (images, text, audio, video). However, data in healthcare domain, may include many more modalities like X-rays, PET scans, MRIs, genetic screening, genomic data, and clinical notes, creating a need for both efficient and accurate data integration. Many state-of-the-art multimodal models rely on cross-attention or self-attention for effective data integration, which do not scale well for applications with more than two modalities. The complexity per layer of computing attention in either paradigm is, at best, quadratic with respect to the number of modalities, posing a computational bottleneck that impedes broad adoption. To address this, we propose a new attention mechanism, One-Versus-Others (OvO) attention, that scales linearly with the number of modalities, thus offering a significant reduction in computational complexity compared to existing multimodal attention methods. Using three clinical datasets with multiple diverse modalities, we show that our method decreases computation costs while maintaining or increasing performance compared to popular integration techniques. Across all clinical datasets, OvO reduced the number of required floating point operations (FLOPs) by at least 91.98%, demonstrating its significant impact on efficiency and enabling multi-modal predictions in healthcare.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

Computational BiologyHumansAlgorithmsDatabases, Factual

Autor:innen und Institutionen

  • §Department of Computer Science, Brown University, Providence, RI 02912, USA.
  • Department of Computer Science, Brown University, Providence, RI 02912, USA.
  • Department of Computer Science, Brown University, Providence, RI 02912, USA, ritambhara@brown.edu.
  • School of Medicine, Institute for Bioinformatics and Medical Informatics, University of Tübingen, Tübingen, 72074, Germany, c.eickhoff@acm.org.

Zitieren

APA

Golovanevsky, M., Schiller, E., Nair, A., Han, E., Singh, R., & Eickhoff, C. (2024). One-Versus-Others Attention: Scalable Multimodal Integration for Biomedical Data. Biocomputing 2025. https://doi.org/10.1142/9789819807024_0041

Vancouver

Golovanevsky M, Schiller E, Nair A, Han E, Singh R, Eickhoff C. One-Versus-Others Attention: Scalable Multimodal Integration for Biomedical Data. Biocomputing 2025. 2024. doi:10.1142/9789819807024_0041

BibTeX
@article{Golovanevsky2024,
  author = {Golovanevsky, Michal and Schiller, Eva and Nair, Akira and Han, Eric and Singh, Ritambhara and Eickhoff, Carsten},
  title = {One-Versus-Others Attention: Scalable Multimodal Integration for Biomedical Data},
  journal = {Biocomputing 2025},
  year = {2024},
  doi = {10.1142/9789819807024_0041}
}
Kostenlos starten →