AI Watermark Detection Is Theater

I spent a little time looking into whether a browser plugin could let someone highlight any block of text and check it for an AI watermark. The idea seemed straightforward enough: if the labs are embedding detectable signals in their outputs, a user should be able to surface them. It didn’t take long to hit the wall.

The public story and why it fails

Detection for these keyed statistical watermarks requires the secret keys, the exact tokenizer, and the generation configuration used at sampling time. Those stay with the platform that applied the mark. OpenAI can only detect OpenAI-watermarked text (to the extent it watermarks at all). Google can only detect Gemini’s. Anthropic can only detect Claude’s. There is no shared detector and no practical way for a third party to run one. Google’s own Responsible GenAI Toolkit documentation makes the dependency plain. That ended the plugin idea. It also made the larger point hard to miss: AI watermark-based detection is useless for anyone except the providers themselves.

The public story around these systems is that they give the rest of us a way to identify machine-generated text. Labs have deployed or announced versions under regulatory pressure, including EU AI Act transparency rules. Google’s approach is described in the Nature paper on SynthID-Text and the open-source notes. Anthropic has outlined a similar method for Claude. The mechanism is a subtle, key-dependent bias introduced during sampling. A detector holding the matching key can later measure it. Readers are not supposed to notice.

What does our search overlord say about AI watermarks?

Google’s ranking guidance has never treated pure AI detection as the signal that matters. Its position since early 2023, restated in the October 2026 updates, is that appropriate use of AI is not against the guidelines. What it targets under the scaled content abuse policy is mass production of unoriginal pages whose main purpose is manipulating rankings rather than helping users - no matter how the pages were made. The same updates said it is critical to manually fact-check AI output before publishing because models can hallucinate. Quality rater guidelines tell raters to score low-effort, low-originality AI content as Lowest quality. There is no public evidence that commercial detector scores are used as ranking factors. See the generative AI content guidance, coverage of the October 2026 changes, and explanations of scaled content abuse.

Even for the platform that holds the keys, the signal is easy to break. It is a statistical bias in token probabilities, not a durable signature. Light rewording through another model, commercial humanizers, multi-pass paraphrasing, translation, or ordinary editing all disrupt it. The humanizer market exists because the patterns are fragile. Benchmarks such as HumanizerBench track the back-and-forth, and write-ups on detector limits (for example this one) show how quickly scores collapse under rewriting. Open-sourcing the algorithm does not help outsiders when the production keys stay private.

Nobody builds serious public systems around this because the secrets are not shared and the mark does not survive normal editing. That leaves the providers as the only parties who can actually use the mechanism - and only for purposes that do not require anyone else to detect it.

The AI watermark internal customer: radioactivity

The research literature has looked at one such purpose for years not under the name AI watermark, but under the name watermark radioactivity. When a model is trained or distilled on a substantial volume of watermarked text, residual bias from the original sampling can remain detectable in the resulting model’s outputs, or in some accounts as a key-specific geometric signal in parameter space that sits apart from ordinary task fine-tuning.

Meta’s 2024 paper showed that fine-tuning on watermarked synthetic instructions could be detected at high confidence even when only about 5% of the training data carried the mark (“Watermarking Makes Language Models Radioactive”). Later work treats this as a way to spot unauthorized knowledge distillation: if a competitor scrapes large volumes of API outputs and trains on them, the original lab can later look for the residual. Papers such as “Can LLM Watermarks Robustly Prevent Unauthorized Knowledge Distillation?”, work on Watermark Basins, and schemes like TextSeal explore the same idea. The original keys are still required, which is fine when the only party that needs to run the test is the one that applied the AI watermark.

This does not require third-party access, a cross-platform scan, or the mark surviving public rewriting of individual documents. A platform can AI watermark its own outputs and later check whether a competitor’s model shows the corresponding residual. It functions as a private tracer for competitive monitoring. The public detection story does not hold; the private one can.

Limitations that remain

The internal application is still statistical and imperfect. Targeted paraphrasing of the training data, filtering, dilution with non-watermarked text, or certain inference-time neutralization methods can scrub the inherited signal while still transferring capability. Power depends on volume, proportion, and the fine-tuning process. It is evidence, not proof. Attack papers exist (see unified spoofing and scrubbing work), and more robust variants have been proposed in response.

What this means

For everyone else, these AI watermarks were never going to solve detection for ranking, moderation, or labeling. Private keys, separate platforms, and easy disruption make public use non-viable. That is why third parties ignore it and why humanizers are common. For the labs, the radioactivity property supplies a monitoring tool that does not require anyone else to be able to see the mark. It is not a hard barrier against distillation, but it is a detectable trace when outputs are used at scale without laundering. That capability is available only to the party that controls the keys.

The public story and the technical constraints do not match. The mechanism that fails for everyone else can still be useful to the platform that applied it. Public detection is theater. The private tracer is the part that was never meant for the audience.