EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Examining the Potential of Generative Language Models for Aviation Safety Analysis: Case Study and Insights Using the Aviation Safety Reporting System (ASRS)

Archana Tikayat Ray, Anirudh Prabhakara Bhat, Ryan T. White, Van Minh Nguyen, Olivia J. Pinon Fischer, Dimitri N. Mavris

Aerospace · 31.08.2023 · 10 (9) · 770

DOI: 10.3390/aerospace10090770 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Aerospace, online erschienen am 31.08.2023. Verfasst von Archana Tikayat Ray und 5 weiteren Autor:innen. Die Arbeit wurde bisher 47-mal zitiert (Crossref) und nennt 46 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

This research investigates the potential application of generative language models, especially ChatGPT, in aviation safety analysis as a means to enhance the efficiency of safety analyses and accelerate the time it takes to process incident reports. In particular, ChatGPT was leveraged to generate incident synopses from narratives, which were subsequently compared with ground-truth synopses from the Aviation Safety Reporting System (ASRS) dataset. The comparison was facilitated by using embeddings from Large Language Models (LLMs), with aeroBERT demonstrating the highest similarity due to its aerospace-specific fine-tuning. A positive correlation was observed between the synopsis length and its cosine similarity. In a subsequent phase, human factors issues involved in incidents, as identified by ChatGPT, were compared to human factors issues identified by safety analysts. The precision was found to be 0.61, with ChatGPT demonstrating a cautious approach toward attributing human factors issues. Finally, the model was utilized to execute an evaluation of accountability. As no dedicated ground-truth column existed for this task, a manual evaluation was conducted to compare the quality of outputs provided by ChatGPT to the ground truths provided by safety analysts. This study discusses the advantages and pitfalls of generative language models in the context of aviation safety analysis and proposes a human-in-the-loop system to ensure responsible and effective utilization of such models, leading to continuous improvement and fostering a collaborative approach in the aviation safety domain.

Zitieren

APA

Ray, A. T., Bhat, A. P., White, R. T., Nguyen, V. M., Fischer, O. J. P., & Mavris, D. N. (2023). Examining the Potential of Generative Language Models for Aviation Safety Analysis: Case Study and Insights Using the Aviation Safety Reporting System (ASRS). Aerospace, 10(9), 770. https://doi.org/10.3390/aerospace10090770

Vancouver

Ray AT, Bhat AP, White RT, Nguyen VM, Fischer OJP, Mavris DN. Examining the Potential of Generative Language Models for Aviation Safety Analysis: Case Study and Insights Using the Aviation Safety Reporting System (ASRS). Aerospace. 2023;10(9):770. doi:10.3390/aerospace10090770

BibTeX
@article{Ray2023,
  author = {Ray, Archana Tikayat and Bhat, Anirudh Prabhakara and White, Ryan T. and Nguyen, Van Minh and Fischer, Olivia J. Pinon and Mavris, Dimitri N.},
  title = {Examining the Potential of Generative Language Models for Aviation Safety Analysis: Case Study and Insights Using the Aviation Safety Reporting System (ASRS)},
  journal = {Aerospace},
  year = {2023},
  volume = {10},
  number = {9},
  pages = {770},
  doi = {10.3390/aerospace10090770}
}
Kostenlos starten →