Multi-Translator FL: Modality-Agnostic Knowledge Construction via Modality Translation in Federated Learning
IEEE Access · 2026 · 14 · 81010-81022
DOI: 10.1109/access.2026.3697638 ↗Kurz gefasst
Fachartikel in der Zeitschrift IEEE Access, 2026. Verfasst von Sookyeong Kim und 1 weiteren Autor:innen. Die Arbeit nennt 57 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: National Research Foundation (NRF), South Korea.
Abstract (Originalsprache)
The paradigm of foundation models has shifted the technology trends through guiding a well-established model towards intelligence across a variety of modalities similar to human intelligence. At the same time, the growth of utilizing edge devices grows rapidly, acquiring multi-modal data at the client sides. However, despite this abundance of data and the unparalleled potential of development in artificial intelligence, the majority of current research has not taken into account effective multi-modal knowledge construction in the distributed resource-scarce device environments. This work aims to bridge the gap between the core ideas of multi-modal research (for yielding general knowledge from additional modalities) and the characteristics of distributed-settings (under limited devices with single modality). We propose a multi-translation federated learning architecture called MTFL, by making the server infuse modality-specific knowledge into a general modality-agnostic knowledge space via one-to-all translation as a bridge and reservoir, expanding the knowledge space across modalities. A single device is able to learn uncovered knowledge derived from a unified knowledge space at the server, while the server with a global model consisting of VAE encoder-decoders, performs modality-agnostic translation. Further, after constructing general modality-agnostic knowledge, MTFL translates the unified model back into modality-specific knowledge tailored to individual clients, facilitating the accommodation of various downstream tasks. The experimental results validate its effectiveness across multiple baselines using the CUB-200-2011, AwA2, Food-101, and MS-COCO datasets. Notably, on the CUB-200-2011, our MTFL performs 84.14% and 44.12% for image and text classification tasks, respectively; in the FL settings, with 82.42% and 42.77%.
Förderung
- National Research Foundation (NRF), South Korea
Zitieren
APA
Kim, S., & Lee, H. (2026). Multi-Translator FL: Modality-Agnostic Knowledge Construction via Modality Translation in Federated Learning. IEEE Access, 14, 81010-81022. https://doi.org/10.1109/access.2026.3697638
Vancouver
Kim S, Lee H. Multi-Translator FL: Modality-Agnostic Knowledge Construction via Modality Translation in Federated Learning. IEEE Access. 2026;14:81010-81022. doi:10.1109/access.2026.3697638
BibTeX
@article{Kim2026,
author = {Kim, Sookyeong and Lee, Hyungjune},
title = {Multi-Translator FL: Modality-Agnostic Knowledge Construction via Modality Translation in Federated Learning},
journal = {IEEE Access},
year = {2026},
volume = {14},
pages = {81010--81022},
doi = {10.1109/access.2026.3697638}
}
EUVIMED