A novel DRL-guided sparse voxel decoding model for reconstructing perceived images from brain activity
Journal of Neuroscience Methods · 2024
DOI: 10.1016/j.jneumeth.2024.110292 ↗Kurz gefasst
Fachartikel in der Zeitschrift Journal of Neuroscience Methods, 2024. Verfasst von Xu Yin und 2 weiteren Autor:innen. Beteiligte Einrichtung: Key Laboratory of Child Development and Learning Science of Ministry of Education, School of Biological Science & Medical Engineering, Southeast Univ…. Hauptthemen laut MEDLINE: Magnetic Resonance Imaging, Visual Cortex, Image Processing, Computer-Assisted. Die Arbeit nennt 63 Literaturangaben. Gefördert von: National Natural Science Foundation of China.
Abstract (Originalsprache)
BACKGROUND: Due to the sparse encoding character of the human visual cortex and the scarcity of paired training samples for {images, fMRIs}, voxel selection is an effective means of reconstructing perceived images from fMRI. However, the existing data-driven voxel selection methods have not achieved satisfactory results. NEW METHOD: Here, a novel deep reinforcement learning-guided sparse voxel (DRL-SV) decoding model is proposed to reconstruct perceived images from fMRI. We innovatively describe voxel selection as a Markov decision process (MDP), training agents to select voxels that are highly involved in specific visual encoding. RESULTS: Experimental results on two public datasets verify the effectiveness of the proposed DRL-SV, which can accurately select voxels highly involved in neural encoding, thereby improving the quality of visual image reconstruction. COMPARISON WITH EXISTING METHODS: We qualitatively and quantitatively compared our results with the state-of-the-art (SOTA) methods, getting better reconstruction results. We compared the proposed DRL-SV with traditional data-driven baseline methods, obtaining sparser voxel selection results, but better reconstruction performance. CONCLUSIONS: DRL-SV can accurately select voxels involved in visual encoding on few-shot, compared to data-driven voxel selection methods. The proposed decoding model provides a new avenue to improving the image reconstruction quality of the primary visual cortex.
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Stichwörter der Autor:innen
Autor:innen und Institutionen
- Key Laboratory of Child Development and Learning Science of Ministry of Education, School of Biological Science & Medical Engineering, Southeast University, Nanjing, Jiangsu 210096, China.
- School of Innovations, Sanjiang University, China; School of Electronic Science and Engineering, Nanjing University, China.
- Key Laboratory of Child Development and Learning Science of Ministry of Education, School of Biological Science & Medical Engineering, Southeast University, Nanjing, Jiangsu 210096, China. Electronic address: hxwang@seu…
Förderung
- National Natural Science Foundation of China
Zitieren
APA
Yin, X., Wu, Z., & Wang, H. (2024). A novel DRL-guided sparse voxel decoding model for reconstructing perceived images from brain activity. Journal of Neuroscience Methods. https://doi.org/10.1016/j.jneumeth.2024.110292
Vancouver
Yin X, Wu Z, Wang H. A novel DRL-guided sparse voxel decoding model for reconstructing perceived images from brain activity. Journal of Neuroscience Methods. 2024. doi:10.1016/j.jneumeth.2024.110292
BibTeX
@article{Yin2024,
author = {Yin, Xu and Wu, Zhengping and Wang, Haixian},
title = {A novel DRL-guided sparse voxel decoding model for reconstructing perceived images from brain activity},
journal = {Journal of Neuroscience Methods},
year = {2024},
doi = {10.1016/j.jneumeth.2024.110292}
}
EUVIMED