EUVIMEDOpdag EUVIMED
Leserbrief

The Relationship Between Word Length and Average Information Content in Japanese

Yuki Tanida

Cognitive Science · 2023-06 · 47 (6)

DOI: 10.1111/cogs.13302 ↗

At a glance

Leserbrief in Cognitive Science, published online on 2023-06-12. Written by Yuki Tanida. Institution: Department of Clinical Psychology, Faculty of Social Welfare, Hanazono University. Major MEDLINE topics: Language. Cited 2 times so far (Crossref); 11 references.

AbstractSubjectsBibliografiske oplysningerCiteRelated studies

Abstract (original language)

Piantadosi, Tily, and Gibson analyzed a large‐scale web‐scraping corpus (the Google 1T dataset) and reported that word length is independently predicted from average information content (surprisal) calculated by a 2‐ to 4‐gram model (hereafter, longer‐span surprisal) across 11 Indo‐European languages, namely, Czech, Dutch, English, French, German, Italian, Polish, Spanish, Portuguese, Romanian, and Swedish. However, a recent article by Meylan and Griffiths suggested the importance of preprocessing for studies with large‐scale corpora and reanalyzed the same databases. After their preprocessing, the results in Piantadosi et al. were not replicated in Czech, Romanian, and Swedish. Additionally, a German‐specific study by Koplenig, Kupietz, and Wolfer showed that the strict analysis did not replicate the result in Piantadosi et al. for that language with the preprocessing suggested by Meylan and Griffiths in a large‐scale but less noisy database. These three studies provide evidence from 11 Indo‐European languages and one Afro‐Asiatic language, Hebrew, as relevant in this debate. However, we do not have evidence from other linguistic groups. This study provides evidence about Japanese based on a strict preprocessing of Google's web‐scraping database. The results show that Japanese word length can be predicted independently by 2‐ to 4‐gram surprisal.

Subjects

MeSH terms (MEDLINE)

LanguageLinguisticsJapan

Author keywords

Corpus analysisInformation theoryLarge-scale corporaN-gram modelZipf's law

Authors and institutions

  • Department of Clinical Psychology, Faculty of Social Welfare, Hanazono University.
  • Japan Society for the Promotion of Science.
  • Graduate School of Sustainable System Sciences, Osaka Metropolitan University.

Cite

APA

Tanida, Y. (2023). The Relationship Between Word Length and Average Information Content in Japanese. Cognitive Science, 47(6). https://doi.org/10.1111/cogs.13302

Vancouver

Tanida Y. The Relationship Between Word Length and Average Information Content in Japanese. Cognitive Science. 2023;47(6). doi:10.1111/cogs.13302

BibTeX
@article{Tanida2023,
  author = {Tanida, Yuki},
  title = {The Relationship Between Word Length and Average Information Content in Japanese},
  journal = {Cognitive Science},
  year = {2023},
  volume = {47},
  number = {6},
  doi = {10.1111/cogs.13302}
}
Start for free →