Scalable watermarking for identifying large language model outputs

The rise of large language models (LLMs) has led to a surge in high-quality synthetic text generation, often indistinguishable from human-written content. This poses significant challenges for various sectors, including education, journalism, and software development, where distinguishing between human and AI-generated text is crucial. The ability to accurately identify the source of text is essential for maintaining trust, combating misinformation, and ensuring responsible use of this powerful technology. Several strategies have been proposed to address this issue. Retrieval-based approaches involve maintaining a database of all generated texts and comparing new text against this database. While effective, this method raises significant scalability, privacy, and coordination issues. Post-hoc detection methods use statistical features of text or machine learning classifiers to discriminate between human and AI-generated text. However, these methods are computationally expensive, suffer from inconsistent performance (particularly on out-of-domain data), and are prone to high false-positive rates. Moreover, their effectiveness diminishes as LLMs improve, requiring continuous retraining and recalibration. Text watermarking presents a more promising approach. By embedding imperceptible marks within the generated text during the generation process (generative watermarking), it allows for the identification of AI-generated text without relying on extensive databases or potentially biased classifiers. Other watermarking approaches, such as edit-based or data-driven watermarking, exist but have limitations, often involving noticeable artifacts or restricted watermarking contexts. This research focuses on generative watermarking, aiming to create a system that is both effective and minimally disruptive to the quality of the generated text.

Existing literature on LLM-generated text detection highlights the limitations of retrieval-based and post-hoc detection methods. Retrieval-based methods, while effective in principle, suffer from scalability challenges and raise privacy concerns due to the need to store and compare vast quantities of text data. Post-hoc detection methods, relying on statistical differences between human and AI-generated text, are prone to errors and require continuous updates to adapt to improving LLM capabilities. Prior work on text watermarking has explored various approaches, including edit-based and data-driven methods. However, these techniques often introduce noticeable artifacts in the text or only apply in specific scenarios, limiting their widespread applicability. Generative watermarking offers a promising alternative, but existing methods often struggle to balance detectability, text quality preservation, and computational efficiency. This paper aims to address these shortcomings by developing a novel generative watermarking approach.

This paper proposes SynthID-Text, a generative watermarking scheme that leverages a novel sampling algorithm called Tournament sampling. This algorithm introduces subtle, context-specific modifications into the generated text distribution during the text generation process. The key innovation lies in its tournament-like structure, where multiple watermarking functions compete to select the next token. This approach allows for a controlled balance between watermark detectability and text quality preservation. The SynthID-Text system consists of three core components: a random seed generator, a sampling algorithm (Tournament sampling), and a scoring function. The random seed generator creates a random seed based on the preceding text and a watermarking key. Tournament sampling utilizes this seed and multiple watermarking functions to iteratively select the next token from the LLM's probability distribution, embedding the watermark in a statistically detectable way. The scoring function measures the strength of the watermark's signature in a given text by evaluating correlations between the generated tokens and the random seeds. SynthID-Text is designed to be configurable to operate in either non-distortionary or distortionary modes. The non-distortionary mode prioritizes maintaining text quality, while the distortionary mode prioritizes watermark detectability, even at the cost of minor quality reduction. This flexibility allows for adaptation to various applications and risk tolerance levels. To address computational scalability, the paper integrates SynthID-Text with speculative sampling, a common technique for speeding up LLM text generation in production systems. Two algorithms are proposed: high-detectability watermarked speculative sampling and fast watermarked speculative sampling. These algorithms strategically balance watermarking with speculative sampling to ensure both efficiency and detectability. The evaluation of SynthID-Text involves comparisons with state-of-the-art non-distortionary and distortionary generative watermarking methods, using various LLMs and metrics. Human evaluation, including a large-scale live experiment involving nearly 20 million responses from the Gemini production system, is conducted to assess the impact of watermarking on perceived text quality.

The key findings of the research are as follows: 1. **Improved Detectability:** SynthID-Text demonstrates superior watermark detectability compared to existing methods (Gumbel sampling for non-distortionary and Soft Red List for distortionary) across multiple LLMs, maintaining high true-positive rates while keeping false-positive rates low. The improvement is particularly noticeable in lower-entropy settings (e.g., lower temperatures during LLM generation). 2. **Text Quality Preservation:** Comprehensive evaluations, including a large-scale live user study with approximately 20 million Gemini responses, and a controlled human preference test, confirm that the non-distortionary version of SynthID-Text does not noticeably impact perceived text quality. This is further corroborated by automatic evaluations using perplexity and standard model capability benchmarks. 3. **Scalability and Efficiency:** The integration of SynthID-Text with speculative sampling ensures negligible computational overhead, maintaining the efficiency of large-scale LLM production systems. The proposed 'fast watermarked speculative sampling' algorithm demonstrates minimal impact on the speculative sampling acceptance rate. 4. **Robustness:** SynthID-Text demonstrates robustness across different languages, exhibiting consistent performance across linguistic contexts, unlike some post-hoc detectors that are highly sensitive to language variations. 5. **Real-world Deployment:** The non-distortionary version of SynthID-Text has been successfully deployed in the production environments of Gemini and Gemini Advanced chatbots, marking the first large-scale deployment of generative text watermarking serving millions of users.

The results demonstrate the feasibility and effectiveness of SynthID-Text as a production-ready solution for watermarking LLM outputs. The improved detectability, coupled with the negligible impact on text quality and computational resources, addresses critical limitations of existing watermarking techniques. The successful deployment in a real-world, large-scale setting underscores the practical significance of this work. The findings significantly advance the field of LLM accountability and responsible AI development, providing a tangible tool for identifying and managing AI-generated content. The ability to watermark LLM outputs without compromising text quality offers a crucial mechanism for ensuring transparency and accountability in the rapidly evolving landscape of AI-driven text generation.

This research presents SynthID-Text, a novel and scalable watermarking technique for LLMs. Its superior detectability, negligible impact on text quality, and seamless integration with existing production infrastructure mark a significant advancement in the responsible deployment of AI. The successful deployment in Gemini and Gemini Advanced provides compelling evidence of its real-world viability. Future research could focus on further enhancing the robustness of SynthID-Text against watermark attacks, exploring its adaptability to various LLM architectures, and investigating its potential applications in other modalities beyond text.

While SynthID-Text offers substantial advantages over existing methods, certain limitations exist. The system requires cooperation among LLM providers to effectively apply the watermark; detecting AI-generated text from unwatermarked sources necessitates complementary techniques such as post-hoc detection. The decentralized nature of open-source models presents challenges for enforcing widespread adoption. Furthermore, SynthID-Text, like other generative watermarks, remains vulnerable to sophisticated attacks such as watermark stealing, spoofing, and scrubbing; this warrants ongoing research.