Exploring Generative Pre-Trained Transformer-4-Vision for Nystagmus Classification: Development and Validation of a Pupil-Tracking Process
JMIR Formative Research · 2025-06-06 · 9 · e70070-e70070
DOI: 10.2196/70070 ↗At a glance
Validierungsstudie in JMIR Formative Research, published online on 2025-06-06. Written by Masao Noda and 6 co-authors. Institution: Department of Otolaryngology, Mejiro University Ear Institute Clinic, 320 Ukiya, Iwatsuki-ku, Saitama-shi, Saitama, 339-8501, Japan, 81 48 797 3341. Major MEDLINE topics: Nystagmus, Pathologic, Nystagmus, Physiologic, Generative Artificial Intelligence, Pupil. Cited 5 times so far (Crossref); 37 references. The full text is freely available in PubMed Central.
Abstract (original language)
Background
Conventional nystagmus classification methods often rely on subjective observation by specialists, which is time-consuming and variable among clinicians. Recently, deep learning techniques have been used to automate nystagmus classification using convolutional and recurrent neural networks. These networks can accurately classify nystagmus patterns using video data. However, associated challenges including the need for large datasets when creating models, limited applicability to address specific image conditions, and the complexity associated with using these models.
Objective
This study aimed to evaluate a novel approach for nystagmus classification that used the Generative Pre-trained Transformer 4 Vision (GPT-4V) model, which is a state-of-the-art large-scale language model with powerful image recognition capabilities.
Methods
We developed a pupil-tracking process using a nystagmus-recording video and verified the optimization model’s accuracy using GPT-4V classification and nystagmus recording. We tested whether the created optimization model could be evaluated in six categories of nystagmus: right horizontal, left horizontal, upward, downward, right torsional, and left torsional. The traced trajectory was input as two-dimensional coordinate data or an image, and multiple in-context learning methods were evaluated.
Results
The developed model showed an overall classification accuracy of 37% when using pupil-traced images and a maximum accuracy of 24.6% when pupil coordinates were used as input. Regarding orientation, we achieved a maximum accuracy of 69% for the classification of horizontal nystagmus patterns but a lower accuracy for the vertical and torsional components.
Conclusions
We demonstrated the potential of versatile vertigo management in a generative artificial intelligence model that improves the accuracy and efficiency of nystagmus classification. We also highlighted areas for further improvement, such as expanding the dataset size and enhancing input modalities, to improve classification performance across all nystagmus types. The GPT-4V model validated only for recognizing still images can be linked to video classification and proposed as a novel method.
Subjects
MeSH terms (MEDLINE)
Author keywords
Authors and institutions
- Department of Otolaryngology, Mejiro University Ear Institute Clinic, 320 Ukiya, Iwatsuki-ku, Saitama-shi, Saitama, 339-8501, Japan, 81 48 797 3341.
- Department of Otolaryngology, Jichi Medical University, Shimotsuke, Japan.
ORCID profiles: Masao Noda, Ryota Koshu, Reiko Tsunoda, Hirofumi Ogihara, Tomohiko Kamo, Makoto Ito, Hiroaki Fushiki
Cite
APA
Noda, M., Koshu, R., Tsunoda, R., Ogihara, H., Kamo, T., Ito, M., & Fushiki, H. (2025). Exploring Generative Pre-Trained Transformer-4-Vision for Nystagmus Classification: Development and Validation of a Pupil-Tracking Process. JMIR Formative Research, 9, e70070-e70070. https://doi.org/10.2196/70070
Vancouver
Noda M, Koshu R, Tsunoda R, Ogihara H, Kamo T, Ito M, et al. Exploring Generative Pre-Trained Transformer-4-Vision for Nystagmus Classification: Development and Validation of a Pupil-Tracking Process. JMIR Formative Research. 2025;9:e70070-e70070. doi:10.2196/70070
BibTeX
@article{Noda2025,
author = {Noda, Masao and Koshu, Ryota and Tsunoda, Reiko and Ogihara, Hirofumi and Kamo, Tomohiko and Ito, Makoto and Fushiki, Hiroaki},
title = {Exploring Generative Pre-Trained Transformer-4-Vision for Nystagmus Classification: Development and Validation of a Pupil-Tracking Process},
journal = {JMIR Formative Research},
year = {2025},
volume = {9},
pages = {e70070--e70070},
doi = {10.2196/70070}
}
EUVIMED