MM‐LIMA: Less Is More for Alignment in Multi‐Modal Datasets
Artificial Intelligence for Engineering · 2025
DOI: 10.1049/aie2.70001 ↗Kurz gefasst
Fachartikel in der Zeitschrift Artificial Intelligence for Engineering, 2025. Verfasst von Lai Wei und 4 weiteren Autor:innen. Die Arbeit nennt 40 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: National Natural Science Foundation of China.
Abstract (Originalsprache)
Multi‐modal large language models are typically trained in two stages: the pre‐training stage and the visual instruction tuning stage. Recent studies have observed that widely used datasets for instruction tuning may contain low‐quality instances with incorrect or irrelevant responses, leading to a negative impact on final performance. In this paper, we explore the idea of data selection to enhance the performance and robustness of visual instruction tuning. Specifically, we propose using five indicators to assess the quality of each multi‐modal instruction data sample from different dimensions and then aggregating their results to obtain a final score. Based on these scores, we can automatically identify and filter out low‐quality multi‐modal instruction data. Our experiments on various benchmarks show that models trained on less but higher‐quality instruction data achieve better performance. Moreover, our data selection method demonstrates superiority over existing data selection baselines.
Förderung
- National Natural Science Foundation of China
Zitieren
APA
Wei, L., Li, X., Jiang, Z., Huang, W., & Sun, L. (2025). MM‐LIMA: Less Is More for Alignment in Multi‐Modal Datasets. Artificial Intelligence for Engineering. https://doi.org/10.1049/aie2.70001
Vancouver
Wei L, Li X, Jiang Z, Huang W, Sun L. MM‐LIMA: Less Is More for Alignment in Multi‐Modal Datasets. Artificial Intelligence for Engineering. 2025. doi:10.1049/aie2.70001
BibTeX
@article{Wei2025,
author = {Wei, Lai and Li, Xiaozhe and Jiang, Zihao and Huang, Weiran and Sun, Lichao},
title = {MM‐LIMA: Less Is More for Alignment in Multi‐Modal Datasets},
journal = {Artificial Intelligence for Engineering},
year = {2025},
doi = {10.1049/aie2.70001}
}
EUVIMED