Bridging Linguistic Gaps: Cross-Lingual Mapping in Pre-Training and Dataset for Enhanced Multilingual LLM Performance
ACM Transactions on Asian and Low-Resource Language Information Processing · 30.06.2026 · 25 (6) · 1-18
DOI: 10.1145/3811819 ↗Kurz gefasst
Fachartikel in der Zeitschrift ACM Transactions on Asian and Low-Resource Language Information Processing, online erschienen am 08.06.2026. Verfasst von Weihua Zheng und 7 weiteren Autor:innen. Die Arbeit nennt 45 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: National Research Foundation, Singapore, National LLMs Funding Initiative.
Abstract (Originalsprache)
Multilingual Large Language Models (LLMs) struggle with cross-lingual tasks due to data imbalances between high-resource and low-resource languages and the monolingual bias in pre-training. Existing methods, such as bilingual fine-tuning and contrastive alignment, improve cross-lingual performance but often require extensive parallel data or suffer from instability. To address these challenges, we introduce a Cross-Lingual Mapping Task in the pre-training phase, which enhances cross-lingual alignment without compromising monolingual fluency. Our approach bi-directionally maps languages within the LLM’s embedding space, improving both language generation and comprehension. We further introduce a Language Alignment Coefficient to robustly quantify cross-lingual consistency, even in limited-data scenarios. Experimental results on machine translation (MT), cross-lingual natural language understanding (CLNLU), and cross-lingual question answering (CLQA) show that our model achieves up to 11.9 BLEU score gains in MT, an increase of 6.72 in CLQA BERTScore-Precision and more than a 5% increase in CLNLU accuracy over strong multilingual baselines. Our findings highlight the potential of embedding cross-lingual objectives into pre-training, improving multilingual LLMs.
Förderung
- National Research Foundation, Singapore
- National LLMs Funding Initiative
Zitieren
APA
Zheng, W., Liu, C., Liu, Z., Huang, X., Wu, K., Huzaifah, M., Aw, A. T., & Lee, R. K. (2026). Bridging Linguistic Gaps: Cross-Lingual Mapping in Pre-Training and Dataset for Enhanced Multilingual LLM Performance. ACM Transactions on Asian and Low-Resource Language Information Processing, 25(6), 1-18. https://doi.org/10.1145/3811819
Vancouver
Zheng W, Liu C, Liu Z, Huang X, Wu K, Huzaifah M, et al. Bridging Linguistic Gaps: Cross-Lingual Mapping in Pre-Training and Dataset for Enhanced Multilingual LLM Performance. ACM Transactions on Asian and Low-Resource Language Information Processing. 2026;25(6):1-18. doi:10.1145/3811819
BibTeX
@article{Zheng2026,
author = {Zheng, Weihua and Liu, Chang and Liu, Zhengyuan and Huang, Xin and Wu, Kui and Huzaifah, Muhammad and Aw, Ai Ti and Lee, Roy Ka-Wei},
title = {Bridging Linguistic Gaps: Cross-Lingual Mapping in Pre-Training and Dataset for Enhanced Multilingual LLM Performance},
journal = {ACM Transactions on Asian and Low-Resource Language Information Processing},
year = {2026},
volume = {25},
number = {6},
pages = {1--18},
doi = {10.1145/3811819}
}
EUVIMED