EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Bridging Linguistic Gaps: Cross-Lingual Mapping in Pre-Training and Dataset for Enhanced Multilingual LLM Performance

Weihua Zheng, Chang Liu, Zhengyuan Liu, Xin Huang, Kui Wu, Muhammad Huzaifah, Ai Ti Aw, Roy Ka-Wei Lee

ACM Transactions on Asian and Low-Resource Language Information Processing · 30.06.2026 · 25 (6) · 1-18

DOI: 10.1145/3811819 ↗

Kurz gefasst

Fachartikel in der Zeitschrift ACM Transactions on Asian and Low-Resource Language Information Processing, online erschienen am 08.06.2026. Verfasst von Weihua Zheng und 7 weiteren Autor:innen. Die Arbeit nennt 45 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: National Research Foundation, Singapore, National LLMs Funding Initiative.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Multilingual Large Language Models (LLMs) struggle with cross-lingual tasks due to data imbalances between high-resource and low-resource languages and the monolingual bias in pre-training. Existing methods, such as bilingual fine-tuning and contrastive alignment, improve cross-lingual performance but often require extensive parallel data or suffer from instability. To address these challenges, we introduce a Cross-Lingual Mapping Task in the pre-training phase, which enhances cross-lingual alignment without compromising monolingual fluency. Our approach bi-directionally maps languages within the LLM’s embedding space, improving both language generation and comprehension. We further introduce a Language Alignment Coefficient to robustly quantify cross-lingual consistency, even in limited-data scenarios. Experimental results on machine translation (MT), cross-lingual natural language understanding (CLNLU), and cross-lingual question answering (CLQA) show that our model achieves up to 11.9 BLEU score gains in MT, an increase of 6.72 in CLQA BERTScore-Precision and more than a 5% increase in CLNLU accuracy over strong multilingual baselines. Our findings highlight the potential of embedding cross-lingual objectives into pre-training, improving multilingual LLMs.

Förderung

  • National Research Foundation, Singapore
  • National LLMs Funding Initiative

Zitieren

APA

Zheng, W., Liu, C., Liu, Z., Huang, X., Wu, K., Huzaifah, M., Aw, A. T., & Lee, R. K. (2026). Bridging Linguistic Gaps: Cross-Lingual Mapping in Pre-Training and Dataset for Enhanced Multilingual LLM Performance. ACM Transactions on Asian and Low-Resource Language Information Processing, 25(6), 1-18. https://doi.org/10.1145/3811819

Vancouver

Zheng W, Liu C, Liu Z, Huang X, Wu K, Huzaifah M, et al. Bridging Linguistic Gaps: Cross-Lingual Mapping in Pre-Training and Dataset for Enhanced Multilingual LLM Performance. ACM Transactions on Asian and Low-Resource Language Information Processing. 2026;25(6):1-18. doi:10.1145/3811819

BibTeX
@article{Zheng2026,
  author = {Zheng, Weihua and Liu, Chang and Liu, Zhengyuan and Huang, Xin and Wu, Kui and Huzaifah, Muhammad and Aw, Ai Ti and Lee, Roy Ka-Wei},
  title = {Bridging Linguistic Gaps: Cross-Lingual Mapping in Pre-Training and Dataset for Enhanced Multilingual LLM Performance},
  journal = {ACM Transactions on Asian and Low-Resource Language Information Processing},
  year = {2026},
  volume = {25},
  number = {6},
  pages = {1--18},
  doi = {10.1145/3811819}
}
Kostenlos starten →