A Unified Attack Framework against Statistical Generative Watermarks for Large Language Models
-cross Abstract: Watermarking has emerged as a key tool for identifying text generated by large language models (LLMs), with current schemes…
Arxiv 2 versions
-
1 new sentence, beginning: “-cross Abstract: Watermarking has emerged as a key tool for identifying text generated by large language models (LLMs), with…”
A Unified Attack Framework against Statistical Generative Watermarks for Large Language Models
-cross Abstract: Watermarking has emerged as a key tool for identifying text generated by large language models (LLMs), with current schemes embedding statistical signals into the generation process to enable later detection. In this work, we identify a shared weakness across a broad class of popular LLM watermarking methods, including KGW (and its variants), GumbelSoft, DipMark, ITS, and SynthID. We show that these schemes exhibit a fundamental link between their detection scores and the token probability shifts they introduce during generation. This link, which we call score-ratio alignment, allows an attacker with only black-box query access to the model--and no knowledge of secret keys or model weights--to produce text that evades detection while remaining fluent and natural. We prove that the resulting attack is optimal across all covered watermark families, and show that the weakness is intrinsic: any watermark designed to maximize statistical detection power necessarily induces the very structure that attackers can exploit. We evaluate our attack on seven popular generative LLM watermarking schemes, three LLMs, and tasks, which reduce detection rates to at most 0.05 across all settings while preserving text quality. Our findings show that current watermarking methods can be reliably defeated under realistic attack conditions, highlighting the need for fundamentally different watermarking designs that decouple detection scores from token probability shifts.
-
A Unified Attack Framework against Statistical Generative Watermarks for Large Language Models
Watermarking has emerged as a key tool for identifying text generated by large language models (LLMs), with current schemes embedding statistical signals into the generation process to enable later detection. In this work, we identify a shared weakness across a broad class of popular LLM watermarking methods, including KGW (and its variants), GumbelSoft, DipMark, ITS, and SynthID. We show that these schemes exhibit a fundamental link between their detection scores and the token probability shifts they introduce during generation. This link, which we call score-ratio alignment, allows an attacker with only black-box query access to the model--and no knowledge of secret keys or model weights--to produce text that evades detection while remaining fluent and natural. We prove that the resulting attack is optimal across all covered watermark families, and show that the weakness is intrinsic: any watermark designed to maximize statistical detection power necessarily induces the very structure that attackers can exploit. We evaluate our attack on seven popular generative LLM watermarking schemes, three LLMs, and tasks, which reduce detection rates to at most 0.05 across all settings while preserving text quality. Our findings show that current watermarking methods can be reliably defeated under realistic attack conditions, highlighting the need for fundamentally different watermarking designs that decouple detection scores from token probability shifts.
Versions are compared on the headline and summary the publisher puts in their feed. An edit to the body of an article that leaves both untouched will not appear here.
Why am I seeing this Ranked on source trust — arXiv
It ranks mainly on source trust: arXiv is the most reliable outlet we track on this subject, and is the only one on the story so far.
Link-outLink-out, because it did not clear the bar for a write-up. Link-out means we point at the publisher and say nothing of our own.
| Factor | Weight | Score | Contribution | Where it came from |
|---|---|---|---|---|
| Corroboration | 0.35 | 0.39 | +0.135 25% | 1 independent org on the story. Tier-3 aggregators never corroborate — they can show something is circulating, never that it is true. |
| Source trustleads | 0.25 | 0.85 | +0.212 39% | arXiv is the highest-trust source on this story and is first-party — the organisation announcing its own news. Trust is taken from the best source, not averaged. |
| Pickup rate | 0.20 | 0.00 | +0.000 0% | One counted organisation, so there is no spread to measure — nothing has picked this up to set a rate. |
| Freshness | 0.20 | 0.99 | +0.198 36% | Halves every 10 hours from the newest item on the story. This is the only factor that rewards a story for nothing more than being recent. |
Corroboration counts distinct organisations, once each, and only from tiers 1 and 2. Freshness halves every 10 hours, so this ranking is a snapshot and will differ at the next build.
What happened
-cross Abstract: Watermarking has emerged as a key tool for identifying text generated by large language models (LLMs), with current schemes embedding statistical signals into the generation process to enable later detection. In this work, we identify a shared weakness across a broad class of popular LLM watermarking methods, including KGW (and its variants), GumbelSoft, DipMark, ITS, and SynthID. We show that these schemes exhibit a fundamental link between their detection scores and the token probability shifts they introduce during generation.
How this story arrived
Ordered by when each source was first observed, which is what the velocity figure is computed from. Publishers backdate; observed order does not.
- 01 Arxivfirst-party first seen A Unified Attack Framework against Statistical Generative Watermarks for Large Language Mode
Overclock clusters coverage from independent sources and grades it automatically. The figures above are computed, not editorial. This page summarises and links to reporting by the outlets named — follow the links for the original work.