EUVIMEDEUVIMED entdecken
Crossref

Do Language Models Associate Sound with Meaning? A Multimodal Study of Sound Symbolism

Jinhong Jeong, Sunghyun Lee, Jaeyoung Lee, Seonah Han, Youngjae Yu

Proceedings of the AAAI Conference on Artificial Intelligence · 14.03.2026 · 40 (37) · 31247-31255

DOI: 10.1609/aaai.v40i37.40387 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Proceedings of the AAAI Conference on Artificial Intelligence, online erschienen am 14.03.2026. Verfasst von Jinhong Jeong und 4 weiteren Autor:innen. Die Arbeit wurde bisher 1-mal zitiert (Crossref).

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Sound symbolism is a linguistic concept that refers to non-arbitrary associations between phonetic forms and their meanings. We suggest that this can be a compelling probe into how Multimodal Large Language Models (MLLMs) interpret auditory information in human languages. We investigate MLLMs' performance on phonetic iconicity across textual (orthographic and IPA) and auditory forms of inputs with up to 25 semantic dimensions (e.g., sharp vs. round), observing models' layer-wise information processing by measuring phoneme-level attention fraction scores. To this end, we present LEX-ICON, an extensive mimetic word dataset consisting of 8,052 words from four natural languages (English, French, Japanese, and Korean) and 2,930 systematically constructed pseudo-words, annotated with semantic features applied across both text and audio modalities. Our key findings demonstrate (1) MLLMs' phonetic intuitions that align with existing linguistic research across multiple semantic dimensions and (2) phonosemantic attention patterns that highlight models' focus on iconic phonemes. These results bridge domains of artificial intelligence and cognitive linguistics, providing the first large-scale, quantitative analyses of phonetic iconicity in terms of MLLMs' interpretability.

Zitieren

APA

Jeong, J., Lee, S., Lee, J., Han, S., & Yu, Y. (2026). Do Language Models Associate Sound with Meaning? A Multimodal Study of Sound Symbolism. Proceedings of the AAAI Conference on Artificial Intelligence, 40(37), 31247-31255. https://doi.org/10.1609/aaai.v40i37.40387

Vancouver

Jeong J, Lee S, Lee J, Han S, Yu Y. Do Language Models Associate Sound with Meaning? A Multimodal Study of Sound Symbolism. Proceedings of the AAAI Conference on Artificial Intelligence. 2026;40(37):31247-31255. doi:10.1609/aaai.v40i37.40387

BibTeX
@article{Jeong2026,
  author = {Jeong, Jinhong and Lee, Sunghyun and Lee, Jaeyoung and Han, Seonah and Yu, Youngjae},
  title = {Do Language Models Associate Sound with Meaning? A Multimodal Study of Sound Symbolism},
  journal = {Proceedings of the AAAI Conference on Artificial Intelligence},
  year = {2026},
  volume = {40},
  number = {37},
  pages = {31247--31255},
  doi = {10.1609/aaai.v40i37.40387}
}
Kostenlos starten →