EUVIMEDEUVIMED entdecken
Crossref

MetaWriter: Exploring the Potential and Perils of AI Writing Support in Scientific Peer Review

Lu Sun, Stone Tao, Junjie Hu, Steven P. Dow

Proceedings of the ACM on Human-Computer Interaction · 17.04.2024 · 8 (CSCW1) · 1-32

DOI: 10.1145/3637371 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Proceedings of the ACM on Human-Computer Interaction, online erschienen am 26.04.2024. Verfasst von Lu Sun und 3 weiteren Autor:innen. Die Arbeit wurde bisher 40-mal zitiert (Crossref) und nennt 99 Literaturangaben.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Recent advances in Large Language Models (LLMs) show the potential to significantly augment or even replace complex human writing activities. However, for complex tasks where people need to make decisions as well as write a justification, the trade offs between making work efficient and hindering decisions remain unclear. In this paper, we explore this question in the context of designing intelligent scaffolding for writing meta-reviews for an academic peer review process. We prototyped a system called "MetaWriter'' trained on five years of open peer review data to support meta-reviewing. The system highlights common topics in the original peer reviews, extracts key points by each reviewer, and on request, provides a preliminary draft of a meta-review that can be further edited. To understand how novice and experienced meta-reviewers use MetaWriter, we conducted a within-subject study with 32 participants. Each participant wrote meta-reviews for two papers: one with and one without MetaWriter. We found that MetaWriter significantly expedited the authoring process and improved the coverage of meta-reviews, as rated by experts, compared to the baseline. While participants recognized the efficiency benefits, they raised concerns around trust, over-reliance, and agency. We also interviewed six paper authors to understand their opinions of using machine intelligence to support the peer review process and reported critical reflections. We discuss implications for future interactive AI writing tools to support complex synthesis work.

Zitieren

APA

Sun, L., Tao, S., Hu, J., & Dow, S. P. (2024). MetaWriter: Exploring the Potential and Perils of AI Writing Support in Scientific Peer Review. Proceedings of the ACM on Human-Computer Interaction, 8(CSCW1), 1-32. https://doi.org/10.1145/3637371

Vancouver

Sun L, Tao S, Hu J, Dow SP. MetaWriter: Exploring the Potential and Perils of AI Writing Support in Scientific Peer Review. Proceedings of the ACM on Human-Computer Interaction. 2024;8(CSCW1):1-32. doi:10.1145/3637371

BibTeX
@article{Sun2024,
  author = {Sun, Lu and Tao, Stone and Hu, Junjie and Dow, Steven P.},
  title = {MetaWriter: Exploring the Potential and Perils of AI Writing Support in Scientific Peer Review},
  journal = {Proceedings of the ACM on Human-Computer Interaction},
  year = {2024},
  volume = {8},
  number = {CSCW1},
  pages = {1--32},
  doi = {10.1145/3637371}
}
Kostenlos starten →