EUVIMEDEUVIMED entdecken
Crossref

fMRI Encoding and Decoding with LLMs: Input Embeddings vs. Hidden State Representations

Muxuan Liu, Ichiro Kobayashi

Journal of Advanced Computational Intelligence and Intelligent Informatics · 20.07.2026 · 30 (4) · 1243-1257

DOI: 10.20965/jaciii.2026.p1243 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Journal of Advanced Computational Intelligence and Intelligent Informatics, online erschienen am 20.07.2026. Verfasst von Muxuan Liu und 1 weiteren Autor:innen. Die Arbeit nennt 44 Literaturangaben. Gefördert von: Japan Science and Technology Agency, Tateisi Science and Technology Foundation.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Systematically comparing how linguistic representations relate to brain activity has become an important topic in the field of computational neuroscience. Prior studies have mainly relied on contextual hidden states combined with linear regression, leaving open questions about the role of static input embeddings and the benefits of nonlinear mappings. In this study, we compare input embeddings and hidden states from multiple language model families (BERT, GPT-2, and LLaMA) within both encoding frameworks, which map text features to brain responses, and decoding frameworks, which reconstruct linguistic features from brain activity. We benchmarked voxel-wise ridge regression against bidirectional long short-term memory (BiLSTMs) models, using repeat-split cross-validation and explainable variance normalization on functional magnetic resonance imaging (fMRI) data from three subjects. Our analyses demonstrate that input embeddings, despite being context-invariant, remain competitive and, in some cases, outperform hidden states, while BiLSTMs provide modest but region-specific improvements over ridge regression. Fine-grained voxel-level results further revealed distinct cortical distributions of stable versus context-dependent features. Together, these findings clarify the trade-off between predictive performance and interpretability and highlight that input embeddings offer a strong and interpretable baseline for representational alignment between language models and brain activity.

Förderung

  • Japan Science and Technology Agency
  • Tateisi Science and Technology Foundation

Zitieren

APA

Liu, M., & Kobayashi, I. (2026). fMRI Encoding and Decoding with LLMs: Input Embeddings vs. Hidden State Representations. Journal of Advanced Computational Intelligence and Intelligent Informatics, 30(4), 1243-1257. https://doi.org/10.20965/jaciii.2026.p1243

Vancouver

Liu M, Kobayashi I. fMRI Encoding and Decoding with LLMs: Input Embeddings vs. Hidden State Representations. Journal of Advanced Computational Intelligence and Intelligent Informatics. 2026;30(4):1243-1257. doi:10.20965/jaciii.2026.p1243

BibTeX
@article{Liu2026,
  author = {Liu, Muxuan and Kobayashi, Ichiro},
  title = {fMRI Encoding and Decoding with LLMs: Input Embeddings vs. Hidden State Representations},
  journal = {Journal of Advanced Computational Intelligence and Intelligent Informatics},
  year = {2026},
  volume = {30},
  number = {4},
  pages = {1243--1257},
  doi = {10.20965/jaciii.2026.p1243}
}
Kostenlos starten →