EUVIMEDEUVIMED entdecken
Crossref

MM-Mixing: Multi-Modal Mixing Alignment for 3D Understanding

Jiaze Wang, Yi Wang, Ziyu Guo, Renrui Zhang, Donghao Zhou, Guangyong Chen, Anfeng Liu, Pheng-Ann Heng

Proceedings of the AAAI Conference on Artificial Intelligence · 11.04.2025 · 39 (7) · 7744-7752

DOI: 10.1609/aaai.v39i7.32834 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Proceedings of the AAAI Conference on Artificial Intelligence, online erschienen am 11.04.2025. Verfasst von Jiaze Wang und 7 weiteren Autor:innen. Die Arbeit wurde bisher 1-mal zitiert (Crossref).

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

We introduce MM-Mixing, a multi-modal mixing alignment framework for 3D understanding. MM-Mixing applies mixing-based methods to multi-modal data, preserving and optimizing cross-modal connections while enhancing diversity and improving alignment across modalities. Our proposed two-stage training pipeline combines feature-level and input-level mixing to optimize the 3D encoder. The first stage employs feature-level mixing with contrastive learning to align 3D features with their corresponding modalities. The second stage incorporates both feature-level and input-level mixing, introducing mixed point cloud inputs to further refine 3D feature representations. MM-Mixing enhances intermodality relationships, promotes generalization, and ensures feature consistency while providing diverse and realistic training samples. We demonstrate that MM-Mixing significantly improves baseline performance across various learning scenarios, including zero-shot 3D classification, linear probing 3D classification, and cross-modal 3D shape retrieval. Notably, we improved the zero-shot classification accuracy on ScanObjectNN from 51.3% to 61.9%, and on Objaverse-LVIS from 46.8% to 51.4%. Our findings highlight the potential of multi-modal mixing-based alignment to significantly advance 3D object recognition and understanding while remaining straightforward to implement and integrate into existing frameworks.

Zitieren

APA

Wang, J., Wang, Y., Guo, Z., Zhang, R., Zhou, D., Chen, G., Liu, A., & Heng, P. (2025). MM-Mixing: Multi-Modal Mixing Alignment for 3D Understanding. Proceedings of the AAAI Conference on Artificial Intelligence, 39(7), 7744-7752. https://doi.org/10.1609/aaai.v39i7.32834

Vancouver

Wang J, Wang Y, Guo Z, Zhang R, Zhou D, Chen G, et al. MM-Mixing: Multi-Modal Mixing Alignment for 3D Understanding. Proceedings of the AAAI Conference on Artificial Intelligence. 2025;39(7):7744-7752. doi:10.1609/aaai.v39i7.32834

BibTeX
@article{Wang2025,
  author = {Wang, Jiaze and Wang, Yi and Guo, Ziyu and Zhang, Renrui and Zhou, Donghao and Chen, Guangyong and Liu, Anfeng and Heng, Pheng-Ann},
  title = {MM-Mixing: Multi-Modal Mixing Alignment for 3D Understanding},
  journal = {Proceedings of the AAAI Conference on Artificial Intelligence},
  year = {2025},
  volume = {39},
  number = {7},
  pages = {7744--7752},
  doi = {10.1609/aaai.v39i7.32834}
}
Kostenlos starten →