EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Distributional hypothesis as isomorphism between word-word co-occurrence and analogical parallelograms

Takuma Torii iD, Akihiro Maeda, Shohei Hidaka

PLOS ONE · 2024

DOI: 10.1371/journal.pone.0312151 ↗

Kurz gefasst

Fachartikel in der Zeitschrift PLOS ONE, 2024. Verfasst von Takuma Torii und 2 weiteren Autor:innen. Beteiligte Einrichtung: Tokyo Denki University, Hatoyama, Saitama, Japan. Hauptthemen laut MEDLINE: Natural Language Processing, Semantics, Language. Die Arbeit wurde bisher 3-mal zitiert (Crossref) und nennt 19 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Japan Society for the Promotion of Science, Precursory Research for Embryonic Science and Technology.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Most of the modern natural language processing (NLP) techniques are based on the vector space models of language, in which each word is represented by a vector in a high dimensional space. One of the earliest successes was demonstrated by the four-term analogical reasoning task: what is to C as B is to A? The trained word vectors form “parallelograms” representing the quadruple of words in analogy. This discovery in NLP offers us insight into our understanding of human semantic representation of words via analogical reasoning. Despite successful applications of the large-scale language models, it has not been fully understood why such parallelograms emerge by learning through natural language data. As the vector space model is not optimized to form parallelograms, the key structure related to geometric shapes of word vectors is expected to be in the data, rather than the models. In the present article, we test our hypothesis that such parallelogram arrangement of word vectors readily exists in the co-occurrence statistics of language. Our approach focuses more on the data itself, and it is different from the existing theoretical approach trying to find the mechanism of parallelogram formation in the algorithms and/or vector arithmetic operations on word vectors. First, our analysis suggested that analogical reasoning is possible by decomposition of the bigram co-occurrence matrix. Second, we demonstrated the formation of a parallelepiped, a more structured geometric object than a parallelogram, by creating a small artificial corpus and its word vectors. With these results, we propose a refined form of distributional hypothesis pointing out an isomorphism between a sort of symmetry or exchangeability and word co-occurrence statistics.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

Natural Language ProcessingSemanticsLanguageHumansAlgorithmsModels, Theoretical

Autor:innen und Institutionen

  • Tokyo Denki University, Hatoyama, Saitama, Japan.
  • Japan Advanced Institute of Science and Technology, Nomi, Ishikawa, Japan.

ORCID-Profile: Takuma Torii

Förderung

  • Japan Society for the Promotion of Science
  • Precursory Research for Embryonic Science and Technology

Zitieren

APA

Torii, T., Maeda, A., & Hidaka, S. (2024). Distributional hypothesis as isomorphism between word-word co-occurrence and analogical parallelograms. PLOS ONE. https://doi.org/10.1371/journal.pone.0312151

Vancouver

Torii T, Maeda A, Hidaka S. Distributional hypothesis as isomorphism between word-word co-occurrence and analogical parallelograms. PLOS ONE. 2024. doi:10.1371/journal.pone.0312151

BibTeX
@article{Torii2024,
  author = {Torii, Takuma and Maeda, Akihiro and Hidaka, Shohei},
  title = {Distributional hypothesis as isomorphism between word-word co-occurrence and analogical parallelograms},
  journal = {PLOS ONE},
  year = {2024},
  doi = {10.1371/journal.pone.0312151}
}
Kostenlos starten →