MM-Mixing: Multi-Modal Mixing Alignment for 3D Understanding
Proceedings of the AAAI Conference on Artificial Intelligence · 11.04.2025 · 39 (7) · 7744-7752
DOI: 10.1609/aaai.v39i7.32834 ↗Kurz gefasst
Fachartikel in der Zeitschrift Proceedings of the AAAI Conference on Artificial Intelligence, online erschienen am 11.04.2025. Verfasst von Jiaze Wang und 7 weiteren Autor:innen. Die Arbeit wurde bisher 1-mal zitiert (Crossref).
Abstract (Originalsprache)
We introduce MM-Mixing, a multi-modal mixing alignment framework for 3D understanding. MM-Mixing applies mixing-based methods to multi-modal data, preserving and optimizing cross-modal connections while enhancing diversity and improving alignment across modalities. Our proposed two-stage training pipeline combines feature-level and input-level mixing to optimize the 3D encoder. The first stage employs feature-level mixing with contrastive learning to align 3D features with their corresponding modalities. The second stage incorporates both feature-level and input-level mixing, introducing mixed point cloud inputs to further refine 3D feature representations. MM-Mixing enhances intermodality relationships, promotes generalization, and ensures feature consistency while providing diverse and realistic training samples. We demonstrate that MM-Mixing significantly improves baseline performance across various learning scenarios, including zero-shot 3D classification, linear probing 3D classification, and cross-modal 3D shape retrieval. Notably, we improved the zero-shot classification accuracy on ScanObjectNN from 51.3% to 61.9%, and on Objaverse-LVIS from 46.8% to 51.4%. Our findings highlight the potential of multi-modal mixing-based alignment to significantly advance 3D object recognition and understanding while remaining straightforward to implement and integrate into existing frameworks.
Zitieren
APA
Wang, J., Wang, Y., Guo, Z., Zhang, R., Zhou, D., Chen, G., Liu, A., & Heng, P. (2025). MM-Mixing: Multi-Modal Mixing Alignment for 3D Understanding. Proceedings of the AAAI Conference on Artificial Intelligence, 39(7), 7744-7752. https://doi.org/10.1609/aaai.v39i7.32834
Vancouver
Wang J, Wang Y, Guo Z, Zhang R, Zhou D, Chen G, et al. MM-Mixing: Multi-Modal Mixing Alignment for 3D Understanding. Proceedings of the AAAI Conference on Artificial Intelligence. 2025;39(7):7744-7752. doi:10.1609/aaai.v39i7.32834
BibTeX
@article{Wang2025,
author = {Wang, Jiaze and Wang, Yi and Guo, Ziyu and Zhang, Renrui and Zhou, Donghao and Chen, Guangyong and Liu, Anfeng and Heng, Pheng-Ann},
title = {MM-Mixing: Multi-Modal Mixing Alignment for 3D Understanding},
journal = {Proceedings of the AAAI Conference on Artificial Intelligence},
year = {2025},
volume = {39},
number = {7},
pages = {7744--7752},
doi = {10.1609/aaai.v39i7.32834}
}
EUVIMED