EUVIMEDEUVIMED entdecken
Crossref

Mechanism Matters: Modality Informativeness and Fusion Routing Determine Missing-Modality Robustness in Multi-Modal Federated Learning

Atul Rawal, Caleb Cooper

Proceedings of the AAAI Symposium Series · 06.11.2026 · 10 (1) · 411-419

DOI: 10.1609/aaaiss.v10i1.43433 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Proceedings of the AAAI Symposium Series, online erschienen am 06.11.2026. Verfasst von Atul Rawal und 1 weiteren Autor:innen.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Vertical Federated Learning (VFL) is increasingly used in tactical multi-sensor systems where camera feeds, inertial telemetry, acoustic streams, and radio-frequency (RF) spectra are held by physically separate clients. In real deployments, any modality can fail for physical reasons, including glare or overexposure in vision, IMU saturation, ambient masking in audio, and RF jamming or interference. These failures are often self-masking, meaning that the chance of losing an observation depends on the value the sensor would have reported. However, federated robustness is still usually tested with missing completely at random (MCAR) corruption, which cannot represent this behavior. It is therefore natural to assume that the more realistic missing not at random MNAR) mechanism is always harder. We show that this is not generally true. Using frozen ImageBind encoders and a controlled per-modality MCAR and MNAR sparsity injector, we evaluate four experimental setups: a Vision+IMU bi-modal model on the U-AIR UAV dataset, two tri-modal AU-AIR variants, and a tri-modal Vision+Audio+RF model on a three-class drone-identification task with an independent held-out RF validation split. The relative damage caused by MCAR and MNAR depends on which modality carries the predictive signal and whether the server-side gated fusion can route around a degraded client. When one informative modality is self-masked and the only alternative is uninformative, MNAR is harder. When a healthy alternative exists, CAR can be harder because independent dropouts create joint-missing events where every modality is absent on the same sample and the gate has nowhere to route, while MNAR's self-selection tends to leave at least one modality available. On the external drone-identification task, the ordering changes with the sparsity rate, and the held-out RF split shows that the learned gate collapses onto a single modality. As a result, a strong aggregate score can hide an RF client that contributes almost nothing. We argue that trustworthy VFL benchmarks should report results by missingness mechanism and by modality, and should treat MNAR as a standard corruption mechanism alongside MCAR.

Zitieren

APA

Rawal, A., & Cooper, C. (2026). Mechanism Matters: Modality Informativeness and Fusion Routing Determine Missing-Modality Robustness in Multi-Modal Federated Learning. Proceedings of the AAAI Symposium Series, 10(1), 411-419. https://doi.org/10.1609/aaaiss.v10i1.43433

Vancouver

Rawal A, Cooper C. Mechanism Matters: Modality Informativeness and Fusion Routing Determine Missing-Modality Robustness in Multi-Modal Federated Learning. Proceedings of the AAAI Symposium Series. 2026;10(1):411-419. doi:10.1609/aaaiss.v10i1.43433

BibTeX
@article{Rawal2026,
  author = {Rawal, Atul and Cooper, Caleb},
  title = {Mechanism Matters: Modality Informativeness and Fusion Routing Determine Missing-Modality Robustness in Multi-Modal Federated Learning},
  journal = {Proceedings of the AAAI Symposium Series},
  year = {2026},
  volume = {10},
  number = {1},
  pages = {411--419},
  doi = {10.1609/aaaiss.v10i1.43433}
}
Kostenlos starten →