EUVIMEDEUVIMED entdecken
Open Access · CC BY-SA 4.0

Multimodal Hybrid Deep Learning for the Detection of Autism Spectrum Disorder (ASD) Using Eye-Tracking Data

Citra Wanodya Prameswari, Erwin Budi Setiawan

Equivalent: Jurnal Ilmiah Sosial Teknik · 05.11.2026 · 8 (5) · 2159-2175

DOI: 10.59261/jequi.v8i5.441 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Equivalent: Jurnal Ilmiah Sosial Teknik, online erschienen am 05.11.2026. Verfasst von Citra Wanodya Prameswari und 1 weiteren Autor:innen. Der Volltext ist frei zugänglich (Open Access, CC BY-SA 4.0).

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Background

Autism spectrum disorder (ASD) assessment may benefit from objective measures of visual attention, although eye-tracking classifiers remain research tools rather than replacements for clinical diagnosis.

Objective

This study compared eye-tracking representations and sequence encoders for binary classification of ASD versus typical development using the Saliency4ASD dataset.

Methods

Eye-movement sequences, fixation maps, stimulus images, and tabular features combining gaze and stimulus content were investigated in a hybrid learning pipeline that combined sequence encoders, ResNet-18 image-processing branches, and a multilayer perceptron (MLP) through feature concatenation. The sequential experiments evaluated six data-partition ratios, 15 modality combinations, seven optimization configurations, and alternative sequence architectures. The selected model was also evaluated across five additional holdout partitions grouped by stimulus image.

Results

The combination of scanpath and fixation-map representations achieved a file-level F1 score of 0.866 during modality confirmation. The long short-term memory (LSTM) + gated recurrent unit (GRU) configuration achieved a file-level F1 score of 0.9333 and a receiver operating characteristic area under the curve (ROC-AUC) of 0.9733 on the originally selected test partition comprising 30 stimulus images. Five additional image-grouped partitions yielded a mean file-level F1 score of 0.839 ± 0.015. The random forest baseline achieved an F1 score of 0.8999 on the original partition; however, it was not evaluated across the additional partitions.

Conclusion

The results demonstrate the potential of combining temporal and spatial gaze representations while highlighting sensitivity to data-partition selection. Reuse of test data during model tuning, limited subject independence, and the absence of verified generalization to previously unseen children preclude claims of independently validated diagnostic performance.

Zitieren

APA

Prameswari, C. W., & Setiawan, E. B. (2026). Multimodal Hybrid Deep Learning for the Detection of Autism Spectrum Disorder (ASD) Using Eye-Tracking Data. Equivalent: Jurnal Ilmiah Sosial Teknik, 8(5), 2159-2175. https://doi.org/10.59261/jequi.v8i5.441

Vancouver

Prameswari CW, Setiawan EB. Multimodal Hybrid Deep Learning for the Detection of Autism Spectrum Disorder (ASD) Using Eye-Tracking Data. Equivalent: Jurnal Ilmiah Sosial Teknik. 2026;8(5):2159-2175. doi:10.59261/jequi.v8i5.441

BibTeX
@article{Prameswari2026,
  author = {Prameswari, Citra Wanodya and Setiawan, Erwin Budi},
  title = {Multimodal Hybrid Deep Learning for the Detection of Autism Spectrum Disorder (ASD) Using Eye-Tracking Data},
  journal = {Equivalent: Jurnal Ilmiah Sosial Teknik},
  year = {2026},
  volume = {8},
  number = {5},
  pages = {2159--2175},
  doi = {10.59261/jequi.v8i5.441}
}
Kostenlos starten →