Scalable watermarking for identifying large language model outputs
Nature · 24.10.2024 · 634 (8035) · 818-823
DOI: 10.1038/s41586-024-08025-4 ↗Kurz gefasst
Fachartikel in der Zeitschrift Nature, online erschienen am 23.10.2024. Verfasst von Sumanth Dathathri und 23 weiteren Autor:innen. Beteiligte Einrichtung: Google DeepMind, London, UK. sdathath@google.com. Die Arbeit wurde bisher 157-mal zitiert (Crossref) und nennt 36 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).
Abstract (Originalsprache)
Large language models (LLMs) have enabled the generation of high-quality synthetic text, often indistinguishable from human-written content, at a scale that can markedly affect the nature of the information ecosystem 1–3 . Watermarking can help identify synthetic text and limit accidental or deliberate misuse 4 , but has not been adopted in production systems owing to stringent quality, detectability and computational efficiency requirements. Here we describe SynthID-Text, a production-ready text watermarking scheme that preserves text quality and enables high detection accuracy, with minimal latency overhead. SynthID-Text does not affect LLM training and modifies only the sampling procedure; watermark detection is computationally efficient, without using the underlying LLM. To enable watermarking at scale, we develop an algorithm integrating watermarking with speculative sampling, an efficiency technique frequently used in production systems 5 . Evaluations across multiple LLMs empirically show that SynthID-Text provides improved detectability over comparable methods, and standard benchmarks and human side-by-side ratings indicate no change in LLM capabilities. To demonstrate the feasibility of watermarking in large-scale-production systems, we conducted a live experiment that assessed feedback from nearly 20 million Gemini 6 responses, again confirming the preservation of text quality. We hope that the availability of SynthID-Text 7 will facilitate further development of watermarking and responsible use of LLM systems.
Autor:innen und Institutionen
- Google DeepMind, London, UK. sdathath@google.com.
- Google DeepMind, London, UK.
- Google, Mountain View, CA, USA.
- Google DeepMind, London, UK. pushmeet@google.com.
Zitieren
APA
Dathathri, S., See, A., Ghaisas, S., Huang, P. S., McAdam, R., Welbl, J., Bachani, V., Kaskasoli, A., Stanforth, R., Matejovicova, T., Hayes, J., Vyas, N., Merey, M. A., Brown-Cohen, J., Bunel, R., Balle, B., Cemgil, T., Ahmed, Z., Stacpoole, K., …, & Kohli, P. (2024). Scalable watermarking for identifying large language model outputs. Nature, 634(8035), 818-823. https://doi.org/10.1038/s41586-024-08025-4
Vancouver
Dathathri S, See A, Ghaisas S, Huang PS, McAdam R, Welbl J, et al. Scalable watermarking for identifying large language model outputs. Nature. 2024;634(8035):818-823. doi:10.1038/s41586-024-08025-4
BibTeX
@article{Dathathri2024,
author = {Dathathri, Sumanth and See, Abigail and Ghaisas, Sumedh and Huang, Po-Sen and McAdam, Rob and Welbl, Johannes and Bachani, Vandana and Kaskasoli, Alex and Stanforth, Robert and Matejovicova, Tatiana and Hayes, Jamie and Vyas, Nidhi and Merey, Majd Al and Brown-Cohen, Jonah and Bunel, Rudy and Balle, Borja and Cemgil, Taylan and Ahmed, Zahra and Stacpoole, Kitty and Shumailov, Ilia and Baetu, Ciprian and Gowal, Sven and Hassabis, Demis and Kohli, Pushmeet},
title = {Scalable watermarking for identifying large language model outputs},
journal = {Nature},
year = {2024},
volume = {634},
number = {8035},
pages = {818--823},
doi = {10.1038/s41586-024-08025-4}
}
EUVIMED