EUVIMEDEUVIMED entdecken
Open Access · CC BY-NC-ND 4.0

Video‐Based Automated Seizure Classification Using Multimodal Large Language Models

Lina Zhang, Richard Jiang, Tonmoy Monsoor, Jessica N. Pasqua, Colin M. McCrimmon, Prateik Sinha, Kartik Sharma, Muayad Alzuabi, Victor E. Morales, Hailey Miranda, Chaya Manjeshwar, Vwani Roychowdhury und 1 weitere

Annals of Clinical and Translational Neurology · 20.11.2026

DOI: 10.1002/acn3.70574 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Annals of Clinical and Translational Neurology, online erschienen am 20.11.2026. Verfasst von Lina Zhang und 12 weiteren Autor:innen. Die Arbeit nennt 23 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY-NC-ND 4.0). Gefördert von: Fogarty International Center.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Objective

Accurately distinguishing between epileptic seizures (ES) and nonepileptic seizures (NES) remains a significant clinical challenge, typically requiring resource‐intensive inpatient video‐EEG monitoring. We developed a multimodal large language model (MLLM)‐based framework for automated extraction of semiological features from seizure videos, followed by classification of events as ES or NES.

Methods

We analyzed 90 seizure videos (45 ES, 45 NES) from 29 patients in an epilepsy monitoring unit. Each event was labeled using expert‐reviewed video‐EEG and annotated for 24 clinically relevant semiological features. An MLLM framework integrating open‐source vision‐language models (VLMs) and audio‐language models (ALMs) was used to extract these features automatically. The extracted features were then used to train classifiers (K‐Nearest Neighbors, XGBoost, and Deep Factorization Machine) to distinguish ES from NES. Model performance was evaluated using leave‐one‐patient‐out (LOPO) cross‐validation.

Results

Expert‐annotated features yielded near‐perfect classification using KNN (precision = 0.92, recall = 0.92, F1‐score = 0.92, AUC = 0.99). Compared to expert annotations, the MLLM pipeline achieved a mean recall of 0.71, accuracy of 0.58, and F1‐score of 0.51 for feature extraction. Using MLLM‐extracted features, the KNN classifier ( k = 7) achieved a precision of 0.77, recall of 0.76, F1‐score of 0.76, and AUC of 0.76, correctly identifying 68 out of 90 events.

Conclusion

Multimodal large language models can extract clinically relevant semiological features from seizure videos and support interpretable classification of ES versus NES. This approach offers a scalable, explainable tool to assist in video‐based epilepsy diagnosis.

Förderung

  • Fogarty International Center

Zitieren

APA

Zhang, L., Jiang, R., Monsoor, T., Pasqua, J. N., McCrimmon, C. M., Sinha, P., Sharma, K., Alzuabi, M., Morales, V. E., Miranda, H., Manjeshwar, C., Roychowdhury, V., & Mazumder, R. (2026). Video‐Based Automated Seizure Classification Using Multimodal Large Language Models. Annals of Clinical and Translational Neurology. https://doi.org/10.1002/acn3.70574

Vancouver

Zhang L, Jiang R, Monsoor T, Pasqua JN, McCrimmon CM, Sinha P, et al. Video‐Based Automated Seizure Classification Using Multimodal Large Language Models. Annals of Clinical and Translational Neurology. 2026. doi:10.1002/acn3.70574

BibTeX
@article{Zhang2026,
  author = {Zhang, Lina and Jiang, Richard and Monsoor, Tonmoy and Pasqua, Jessica N. and McCrimmon, Colin M. and Sinha, Prateik and Sharma, Kartik and Alzuabi, Muayad and Morales, Victor E. and Miranda, Hailey and Manjeshwar, Chaya and Roychowdhury, Vwani and Mazumder, Rajarshi},
  title = {Video‐Based Automated Seizure Classification Using Multimodal Large Language Models},
  journal = {Annals of Clinical and Translational Neurology},
  year = {2026},
  doi = {10.1002/acn3.70574}
}
Kostenlos starten →