EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Multi-Translator FL: Modality-Agnostic Knowledge Construction via Modality Translation in Federated Learning

Sookyeong Kim, Hyungjune Lee

IEEE Access · 2026 · 14 · 81010-81022

DOI: 10.1109/access.2026.3697638 ↗

Kurz gefasst

Fachartikel in der Zeitschrift IEEE Access, 2026. Verfasst von Sookyeong Kim und 1 weiteren Autor:innen. Die Arbeit nennt 57 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: National Research Foundation (NRF), South Korea.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

The paradigm of foundation models has shifted the technology trends through guiding a well-established model towards intelligence across a variety of modalities similar to human intelligence. At the same time, the growth of utilizing edge devices grows rapidly, acquiring multi-modal data at the client sides. However, despite this abundance of data and the unparalleled potential of development in artificial intelligence, the majority of current research has not taken into account effective multi-modal knowledge construction in the distributed resource-scarce device environments. This work aims to bridge the gap between the core ideas of multi-modal research (for yielding general knowledge from additional modalities) and the characteristics of distributed-settings (under limited devices with single modality). We propose a multi-translation federated learning architecture called MTFL, by making the server infuse modality-specific knowledge into a general modality-agnostic knowledge space via one-to-all translation as a bridge and reservoir, expanding the knowledge space across modalities. A single device is able to learn uncovered knowledge derived from a unified knowledge space at the server, while the server with a global model consisting of VAE encoder-decoders, performs modality-agnostic translation. Further, after constructing general modality-agnostic knowledge, MTFL translates the unified model back into modality-specific knowledge tailored to individual clients, facilitating the accommodation of various downstream tasks. The experimental results validate its effectiveness across multiple baselines using the CUB-200-2011, AwA2, Food-101, and MS-COCO datasets. Notably, on the CUB-200-2011, our MTFL performs 84.14% and 44.12% for image and text classification tasks, respectively; in the FL settings, with 82.42% and 42.77%.

Förderung

  • National Research Foundation (NRF), South Korea

Zitieren

APA

Kim, S., & Lee, H. (2026). Multi-Translator FL: Modality-Agnostic Knowledge Construction via Modality Translation in Federated Learning. IEEE Access, 14, 81010-81022. https://doi.org/10.1109/access.2026.3697638

Vancouver

Kim S, Lee H. Multi-Translator FL: Modality-Agnostic Knowledge Construction via Modality Translation in Federated Learning. IEEE Access. 2026;14:81010-81022. doi:10.1109/access.2026.3697638

BibTeX
@article{Kim2026,
  author = {Kim, Sookyeong and Lee, Hyungjune},
  title = {Multi-Translator FL: Modality-Agnostic Knowledge Construction via Modality Translation in Federated Learning},
  journal = {IEEE Access},
  year = {2026},
  volume = {14},
  pages = {81010--81022},
  doi = {10.1109/access.2026.3697638}
}
Kostenlos starten →