EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

MM‐LIMA: Less Is More for Alignment in Multi‐Modal Datasets

Lai Wei, Xiaozhe Li, Zihao Jiang, Weiran Huang, Lichao Sun

Artificial Intelligence for Engineering · 2025

DOI: 10.1049/aie2.70001 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Artificial Intelligence for Engineering, 2025. Verfasst von Lai Wei und 4 weiteren Autor:innen. Die Arbeit nennt 40 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: National Natural Science Foundation of China.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Multi‐modal large language models are typically trained in two stages: the pre‐training stage and the visual instruction tuning stage. Recent studies have observed that widely used datasets for instruction tuning may contain low‐quality instances with incorrect or irrelevant responses, leading to a negative impact on final performance. In this paper, we explore the idea of data selection to enhance the performance and robustness of visual instruction tuning. Specifically, we propose using five indicators to assess the quality of each multi‐modal instruction data sample from different dimensions and then aggregating their results to obtain a final score. Based on these scores, we can automatically identify and filter out low‐quality multi‐modal instruction data. Our experiments on various benchmarks show that models trained on less but higher‐quality instruction data achieve better performance. Moreover, our data selection method demonstrates superiority over existing data selection baselines.

Förderung

  • National Natural Science Foundation of China

Zitieren

APA

Wei, L., Li, X., Jiang, Z., Huang, W., & Sun, L. (2025). MM‐LIMA: Less Is More for Alignment in Multi‐Modal Datasets. Artificial Intelligence for Engineering. https://doi.org/10.1049/aie2.70001

Vancouver

Wei L, Li X, Jiang Z, Huang W, Sun L. MM‐LIMA: Less Is More for Alignment in Multi‐Modal Datasets. Artificial Intelligence for Engineering. 2025. doi:10.1049/aie2.70001

BibTeX
@article{Wei2025,
  author = {Wei, Lai and Li, Xiaozhe and Jiang, Zihao and Huang, Weiran and Sun, Lichao},
  title = {MM‐LIMA: Less Is More for Alignment in Multi‐Modal Datasets},
  journal = {Artificial Intelligence for Engineering},
  year = {2025},
  doi = {10.1049/aie2.70001}
}
Kostenlos starten →