Controlling risks of AI in chemical science with agents
AI for Science · 2025
DOI: 10.1088/3050-287x/adfee5 ↗Kurz gefasst
Fachartikel in der Zeitschrift AI for Science, 2025. Verfasst von Jiyan He und 13 weiteren Autor:innen. Die Arbeit wurde bisher 3-mal zitiert (Crossref) und nennt 75 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).
Abstract (Originalsprache)
Artificial intelligence is rapidly advancing scientific discovery, but this progress carries risks of misuse, such as the creation of harmful substances, or circumvention of established regulations. In this paper, we first demonstrate the risks by highlighting real-world examples of AI misuse in chemical science, which underscore the need for effective safety alignment for these AI models. In response, we propose SciGuard, an agent-based guardrail that employs large language models, tools and external knowledge to assess and control risks in scientific AI interactions. For a fair comparison, we introduce a benchmark SciMT (Scientific Multi-Task) to assess both the safety and utility of different AI systems. SciGuard achieves a state-of-the-art harmlessness score on red-teaming queries, while maintaining high performance on benign tasks, without sacrificing scientific knowledge. Finally, we call for continued research and dialogue to ensure the safe deployment of AI in science.
Zitieren
APA
He, J., Guan, H., Feng, W., Min, Y., Yi, J., Tang, K., Li, S., Zhang, J., Chen, K., Zhou, W., Xie, X., Zhang, W., Yu, N., & Zheng, S. (2025). Controlling risks of AI in chemical science with agents. AI for Science. https://doi.org/10.1088/3050-287x/adfee5
Vancouver
He J, Guan H, Feng W, Min Y, Yi J, Tang K, et al. Controlling risks of AI in chemical science with agents. AI for Science. 2025. doi:10.1088/3050-287x/adfee5
BibTeX
@article{He2025,
author = {He, Jiyan and Guan, Haoxiang and Feng, Weitao and Min, Yaosen and Yi, Jingwei and Tang, Kunsheng and Li, Shuai and Zhang, Jie and Chen, Kejiang and Zhou, Wenbo and Xie, Xing and Zhang, Weiming and Yu, Nenghai and Zheng, Shuxin},
title = {Controlling risks of AI in chemical science with agents},
journal = {AI for Science},
year = {2025},
doi = {10.1088/3050-287x/adfee5}
}
EUVIMED