EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Mapping the movie-watching brain with AI-derived semantics

Muwei Li iD

Imaging Neuroscience · 10.07.2026 · 4

DOI: 10.1162/imag.a.1300 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Imaging Neuroscience, online erschienen am 10.07.2026. Verfasst von Muwei Li. Beteiligte Einrichtung: Vanderbilt University Institute of Imaging Science, Vanderbilt University Medical Center, Nashville, TN, United States. Die Arbeit wurde bisher 1-mal zitiert (Crossref) und nennt 64 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: NIMH NIH HHS.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Naturalistic paradigms offer a powerful tool to investigate human brain function, but it remains difficult to link rich, continuous movie content to distributed brain activity in an interpretable way. In this study, I use a multimodal large language model (Gemini) as an automated “semantic annotator” to bridge naturalistic movie stimuli, brain responses, and cognitive performance. Using the Human Connectome Project movie-watching dataset, I segmented the film into 293 overlapping clips, prompting Gemini to rate each clip on 11 psychologically interpretable dimensions. Simultaneously, I extracted clip-wise BOLD activation patterns from the fMR images in 360 cortical ROIs. In this way, the AI and the brain effectively “watch” the same movies in parallel. For each brain ROI, I then fit linear regression models to predict clip-to-clip variation in movie-evoked responses from these features. Gemini-derived features robustly predicted movie-evoked responses in temporal, medial parietal, and lateral frontal association cortex, but explained little variance in unimodal somatosensory, dorsal parietal, insular, and piriform regions. Feature-weight maps reflected known functional specializations, and features with the largest global influence overlapped with the most explainable ROIs. Partial least squares analysis revealed that individual differences in resting-state connectivity strength and semantic explainability covaried along an asymmetric intrinsic axis: strongly integrated sensory-opercular systems at rest were associated with poorer AI predictability, whereas a smaller set of dorsal and medial association regions showed enhanced alignment. Finally, regional AI explainability in medial parietal and left perisylvian association areas was positively related to specific cognitive abilities. Together, these findings demonstrate that interpretable features from AI models provide a simple and scalable framework for quantifying AI-derived semantic predictability in naturalistic settings, offering a practical framework for utilizing artificial models as semantic references to probe human neural processing and individual differences.

Schlagwörter

Stichwörter der Autor:innen

LLMsfMRInaturalistic stimulisemantic encoding

Autor:innen und Institutionen

  • Vanderbilt University Institute of Imaging Science, Vanderbilt University Medical Center, Nashville, TN, United States.
  • Department of Radiology and Radiological Sciences, Vanderbilt University Medical Center, Nashville, TN, United States.

ORCID-Profile: Muwei Li

Förderung

  • NIMH NIH HHS

Zitieren

APA

Li, M. (2026). Mapping the movie-watching brain with AI-derived semantics. Imaging Neuroscience, 4. https://doi.org/10.1162/imag.a.1300

Vancouver

Li M. Mapping the movie-watching brain with AI-derived semantics. Imaging Neuroscience. 2026;4. doi:10.1162/imag.a.1300

BibTeX
@article{Li2026,
  author = {Li, Muwei},
  title = {Mapping the movie-watching brain with AI-derived semantics},
  journal = {Imaging Neuroscience},
  year = {2026},
  volume = {4},
  doi = {10.1162/imag.a.1300}
}
Kostenlos starten →