EUVIMEDObjevte EUVIMED
Open access

Consistency and reliability of automated language measures across expressive language samples in autism

Heather MacFarlane iD, Alexandra C Salem iD, Steven Bedrick iD, Jill K Dolata iD, Jack Wiedrick, Grace O Lawley iD, Lizbeth H Finestack iD, Sara T Kover iD, Angela John Thurman iD, Leonard Abbeduto iD, Eric Fombonne iD

Autism Research · 2023

DOI: 10.1002/aur.2897 ↗

At a glance

Journal article in Autism Research, 2023. Written by Heather MacFarlane and 10 co-authors. Institution: Department of Psychiatry, Oregon Health & Science University, Portland, Oregon, USA. Major MEDLINE topics: Autistic Disorder, Autism Spectrum Disorder. Cited 8 times so far (Crossref); 57 references. The full text is freely available in PubMed Central. Funded by: NCATS NIH HHS, NICHD NIH HHS, NIDCD NIH HHS.

AbstraktSubjectsBibliografické údajeCiteRelated studies

Abstract (original language)

Autism Spectrum Disorder (ASD) is a neurodevelopmental disorder with substantial clinical heterogeneity, especially in language and communication ability. There is a need for validated language outcome measures that show sensitivity to true change for this population. We used Natural Language Processing to analyze expressive language transcripts of 64 highly‐verbal children and young adults (age: 6–23 years, mean 12.8 years; 78.1% male) with ASD to examine the validity across language sampling context and test‐retest reliability of six previously validated Automated Language Measures (ALMs), including Mean Length of Utterance in Morphemes, Number of Distinct Word Roots, C‐units per minute, unintelligible proportion, um rate, and repetition proportion. Three expressive language samples were collected at baseline and again 4 weeks later. These samples comprised interview tasks from the Autism Diagnostic Observation Schedule (ADOS‐2) Modules 3 and 4, a conversation task, and a narration task. The influence of language sampling context on each ALM was estimated using either generalized linear mixed‐effects models or generalized linear models, adjusted for age, sex, and IQ. The 4 weeks test‐retest reliability was evaluated using Lin's Concordance Correlation Coefficient (CCC). The three different sampling contexts were associated with significantly ( P < 0.001) different distributions for each ALM. With one exception (repetition proportion), ALMs also showed good test‐retest reliability (median CCC: 0.73–0.88) when measured within the same context. Taken in conjunction with our previous work establishing their construct validity, this study demonstrates further critical psychometric properties of ALMs and their promising potential as language outcome measures for ASD research.

Subjects

MeSH terms (MEDLINE)

Autistic DisorderAutism Spectrum DisorderChildYoung AdultHumansMaleAdolescentAdultFemaleReproducibility of ResultsLanguageCommunication

Author keywords

autismautomated measurescommunicationexpressive languagenatural language processing

Authors and institutions

  • Department of Psychiatry, Oregon Health & Science University, Portland, Oregon, USA.
  • Department of Medical Informatics and Clinical Epidemiology, Oregon Health & Science University, Portland, Oregon, USA.
  • Department of Pediatrics, Oregon Health & Science University, Portland, Oregon, USA.
  • School of Communication Sciences and Disorders, Pacific University, Forest Grove, Oregon, USA.
  • Biostatistics & Design Program, Oregon Health & Science University, Portland, Oregon, USA.
  • Computer Science and Electrical Engineering, Oregon Health & Science University, Portland, Oregon, USA.

ORCID profiles: Heather MacFarlane, Alexandra C Salem, Steven Bedrick, Jill K Dolata, Grace O Lawley, Lizbeth H Finestack, Sara T Kover, Angela John Thurman, Leonard Abbeduto, Eric Fombonne

Funding

  • NCATS NIH HHS
  • NICHD NIH HHS
  • NIDCD NIH HHS
  • National Institutes of Health
  • Simons Foundation

Cite

APA

MacFarlane, H., Salem, A. C., Bedrick, S., Dolata, J. K., Wiedrick, J., Lawley, G. O., Finestack, L. H., Kover, S. T., Thurman, A. J., Abbeduto, L., & Fombonne, E. (2023). Consistency and reliability of automated language measures across expressive language samples in autism. Autism Research. https://doi.org/10.1002/aur.2897

Vancouver

MacFarlane H, Salem AC, Bedrick S, Dolata JK, Wiedrick J, Lawley GO, et al. Consistency and reliability of automated language measures across expressive language samples in autism. Autism Research. 2023. doi:10.1002/aur.2897

BibTeX
@article{MacFarlane2023,
  author = {MacFarlane, Heather and Salem, Alexandra C and Bedrick, Steven and Dolata, Jill K and Wiedrick, Jack and Lawley, Grace O and Finestack, Lizbeth H and Kover, Sara T and Thurman, Angela John and Abbeduto, Leonard and Fombonne, Eric},
  title = {Consistency and reliability of automated language measures across expressive language samples in autism},
  journal = {Autism Research},
  year = {2023},
  doi = {10.1002/aur.2897}
}
Start for free →