Video‐Based Automated Seizure Classification Using Multimodal Large Language Models
Annals of Clinical and Translational Neurology · 20.11.2026
DOI: 10.1002/acn3.70574 ↗Kurz gefasst
Fachartikel in der Zeitschrift Annals of Clinical and Translational Neurology, online erschienen am 20.11.2026. Verfasst von Lina Zhang und 12 weiteren Autor:innen. Die Arbeit nennt 23 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY-NC-ND 4.0). Gefördert von: Fogarty International Center.
Abstract (Originalsprache)
Objective
Accurately distinguishing between epileptic seizures (ES) and nonepileptic seizures (NES) remains a significant clinical challenge, typically requiring resource‐intensive inpatient video‐EEG monitoring. We developed a multimodal large language model (MLLM)‐based framework for automated extraction of semiological features from seizure videos, followed by classification of events as ES or NES.
Methods
We analyzed 90 seizure videos (45 ES, 45 NES) from 29 patients in an epilepsy monitoring unit. Each event was labeled using expert‐reviewed video‐EEG and annotated for 24 clinically relevant semiological features. An MLLM framework integrating open‐source vision‐language models (VLMs) and audio‐language models (ALMs) was used to extract these features automatically. The extracted features were then used to train classifiers (K‐Nearest Neighbors, XGBoost, and Deep Factorization Machine) to distinguish ES from NES. Model performance was evaluated using leave‐one‐patient‐out (LOPO) cross‐validation.
Results
Expert‐annotated features yielded near‐perfect classification using KNN (precision = 0.92, recall = 0.92, F1‐score = 0.92, AUC = 0.99). Compared to expert annotations, the MLLM pipeline achieved a mean recall of 0.71, accuracy of 0.58, and F1‐score of 0.51 for feature extraction. Using MLLM‐extracted features, the KNN classifier ( k = 7) achieved a precision of 0.77, recall of 0.76, F1‐score of 0.76, and AUC of 0.76, correctly identifying 68 out of 90 events.
Conclusion
Multimodal large language models can extract clinically relevant semiological features from seizure videos and support interpretable classification of ES versus NES. This approach offers a scalable, explainable tool to assist in video‐based epilepsy diagnosis.
Förderung
- Fogarty International Center
Zitieren
APA
Zhang, L., Jiang, R., Monsoor, T., Pasqua, J. N., McCrimmon, C. M., Sinha, P., Sharma, K., Alzuabi, M., Morales, V. E., Miranda, H., Manjeshwar, C., Roychowdhury, V., & Mazumder, R. (2026). Video‐Based Automated Seizure Classification Using Multimodal Large Language Models. Annals of Clinical and Translational Neurology. https://doi.org/10.1002/acn3.70574
Vancouver
Zhang L, Jiang R, Monsoor T, Pasqua JN, McCrimmon CM, Sinha P, et al. Video‐Based Automated Seizure Classification Using Multimodal Large Language Models. Annals of Clinical and Translational Neurology. 2026. doi:10.1002/acn3.70574
BibTeX
@article{Zhang2026,
author = {Zhang, Lina and Jiang, Richard and Monsoor, Tonmoy and Pasqua, Jessica N. and McCrimmon, Colin M. and Sinha, Prateik and Sharma, Kartik and Alzuabi, Muayad and Morales, Victor E. and Miranda, Hailey and Manjeshwar, Chaya and Roychowdhury, Vwani and Mazumder, Rajarshi},
title = {Video‐Based Automated Seizure Classification Using Multimodal Large Language Models},
journal = {Annals of Clinical and Translational Neurology},
year = {2026},
doi = {10.1002/acn3.70574}
}
EUVIMED