EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Semantics of Multiword Expressions in Transformer-Based Models: A Survey

Filip Miletić, Sabine Schulte im Walde

Transactions of the Association for Computational Linguistics · 30.04.2024 · 12 · 593-612

DOI: 10.1162/tacl_a_00657 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Transactions of the Association for Computational Linguistics, online erschienen am 30.04.2024. Verfasst von Filip Miletić und 1 weiteren Autor:innen. Die Arbeit wurde bisher 7-mal zitiert (Crossref) und nennt 88 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Multiword expressions (MWEs) are composed of multiple words and exhibit variable degrees of compositionality. As such, their meanings are notoriously difficult to model, and it is unclear to what extent this issue affects transformer architectures. Addressing this gap, we provide the first in-depth survey of MWE processing with transformer models. We overall find that they capture MWE semantics inconsistently, as shown by reliance on surface patterns and memorized information. MWE meaning is also strongly localized, predominantly in early layers of the architecture. Representations benefit from specific linguistic properties, such as lower semantic idiosyncrasy and ambiguity of target expressions. Our findings overall question the ability of transformer models to robustly capture fine-grained semantics. Furthermore, we highlight the need for more directly comparable evaluation setups.

Zitieren

APA

Miletić, F., & Walde, S. S. I. (2024). Semantics of Multiword Expressions in Transformer-Based Models: A Survey. Transactions of the Association for Computational Linguistics, 12, 593-612. https://doi.org/10.1162/tacl_a_00657

Vancouver

Miletić F, Walde SSI. Semantics of Multiword Expressions in Transformer-Based Models: A Survey. Transactions of the Association for Computational Linguistics. 2024;12:593-612. doi:10.1162/tacl_a_00657

BibTeX
@article{Mileti2024,
  author = {Miletić, Filip and Walde, Sabine Schulte im},
  title = {Semantics of Multiword Expressions in Transformer-Based Models: A Survey},
  journal = {Transactions of the Association for Computational Linguistics},
  year = {2024},
  volume = {12},
  pages = {593--612},
  doi = {10.1162/tacl_a_00657}
}
Kostenlos starten →