EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Controlling risks of AI in chemical science with agents

Jiyan He, Haoxiang Guan, Weitao Feng, Yaosen Min, Jingwei Yi, Kunsheng Tang, Shuai Li, Jie Zhang, Kejiang Chen, Wenbo Zhou, Xing Xie, Weiming Zhang und 2 weitere

AI for Science · 2025

DOI: 10.1088/3050-287x/adfee5 ↗

Kurz gefasst

Fachartikel in der Zeitschrift AI for Science, 2025. Verfasst von Jiyan He und 13 weiteren Autor:innen. Die Arbeit wurde bisher 3-mal zitiert (Crossref) und nennt 75 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Artificial intelligence is rapidly advancing scientific discovery, but this progress carries risks of misuse, such as the creation of harmful substances, or circumvention of established regulations. In this paper, we first demonstrate the risks by highlighting real-world examples of AI misuse in chemical science, which underscore the need for effective safety alignment for these AI models. In response, we propose SciGuard, an agent-based guardrail that employs large language models, tools and external knowledge to assess and control risks in scientific AI interactions. For a fair comparison, we introduce a benchmark SciMT (Scientific Multi-Task) to assess both the safety and utility of different AI systems. SciGuard achieves a state-of-the-art harmlessness score on red-teaming queries, while maintaining high performance on benign tasks, without sacrificing scientific knowledge. Finally, we call for continued research and dialogue to ensure the safe deployment of AI in science.

Zitieren

APA

He, J., Guan, H., Feng, W., Min, Y., Yi, J., Tang, K., Li, S., Zhang, J., Chen, K., Zhou, W., Xie, X., Zhang, W., Yu, N., & Zheng, S. (2025). Controlling risks of AI in chemical science with agents. AI for Science. https://doi.org/10.1088/3050-287x/adfee5

Vancouver

He J, Guan H, Feng W, Min Y, Yi J, Tang K, et al. Controlling risks of AI in chemical science with agents. AI for Science. 2025. doi:10.1088/3050-287x/adfee5

BibTeX
@article{He2025,
  author = {He, Jiyan and Guan, Haoxiang and Feng, Weitao and Min, Yaosen and Yi, Jingwei and Tang, Kunsheng and Li, Shuai and Zhang, Jie and Chen, Kejiang and Zhou, Wenbo and Xie, Xing and Zhang, Weiming and Yu, Nenghai and Zheng, Shuxin},
  title = {Controlling risks of AI in chemical science with agents},
  journal = {AI for Science},
  year = {2025},
  doi = {10.1088/3050-287x/adfee5}
}
Kostenlos starten →