AI safety has a friendly-fire problem. The technology that proves where AI text came from is weakening the technology that checks whether it is true. Watermarking, or embedding a hidden signature in generated text so that its origin can be proven later, is used to increase transparency on AI applications (Europe\u2019s EU AI Act Article 50 [1] ; China\u2019s AI labeling regulation by the Cyberspace Administration of China [2]). Besides, hallucination detection tries to flag the parts of an answer that the model made up. We use it to check if model outputs are reasonably predictable. Regulators are pushing both edges. But there is a hidden collision inside AI models between both transparency and liability. The collision is hidden in the model\u2019s internals but can have real consequences in AI systems’ safety.<\/p>\n
How watermarking works<\/p>\n
An LLM does not write sentences. It computes, at every step, a probability distribution over the next token. Sometimes that distribution is sharp. After \u201cThe capital of France is\u201d, the token \u201cParis\u201d carries almost all the probability mass, and there is no real choice to make. Sometimes it is flat. After \u201cShe left the party because\u201d, multiple continuations are equally plausible, and the model flips a weighted coin. The technical name for the flatness of that distribution is entropy or semantic entropy to be more specific [1]. High entropy means the model is choosing among many comparable options. Low entropy means the next token is essentially forced. Keep this distinction in mind, because both technologies in this story live at the high-entropy positions.<\/p>\n
A text watermark biases the coin flips using a secret key. The best known scheme for watermarking comes from the work of Kirchenbauer and colleagues in 2023 [4]. This approach splits the vocabulary at each step into a pseudorandom \u201cgreen\u201d and \u201cred\u201d list derived from the key and the recent context, then adds a small bonus to the green tokens (Figure 1). The text still reads naturally, but it contains statistically more green tokens than chance would produce. A detector holding the key reconstructs the lists, counts the green tokens, and computes how improbable that count would be in unwatermarked text.<\/p>\n
![\"\"](\"https:\/\/www.europesays.com\/ai\/wp-content\/uploads\/2026\/08\/1*3k74_oJbkvxSdd7U-_waKQ.png\")
Figure 1: The same prompt answered twice: without a watermark (top) and with one (bottom). Green marks the words the secret key favors. Human text of this length would contain about 9 green words without any watermark at work. The watermarked answer contains 28. The odds of that happening naturally are about 1 in 10 trillion, so the detector concludes the text is machine-made. Adapted from Kirchenbauer et al., 2023 [4]<\/p>\n
Google DeepMind\u2019s SynthID-Text [5], deployed in all its Gemini family models, refines the idea with a mechanism called tournament sampling (the model drafts several candidate words and the secret key referees a knockout between them; the champion gets written), applied as a processor on the model\u2019s output probabilities. The scheme is open sourced in the Hugging Face Transformers library.<\/p>\n
A third family, the distortion-free schemes inspired by Aaronson\u2019s Gumbel trick, arranges the bias so that the output distribution of a single generation is provably unchanged on average, while the text remains correlated with the key [6].<\/p>\n
A watermark can only push where there is room to push. At a low-entropy position, biasing the choice would mean writing the wrong word, and quality would visibly collapse. Take an invoice summary. In the sentence \u201cThe total due is 4,320 dollars,\u201d everything after \u201cis\u201d is constrained. There is only one correct continuation, and a watermark that nudged the model toward any other number would not be hiding a signature, it would be corrupting the document. Now take the sentence \u201cPayment was received immediately\u201d The model could have written \u201cquickly\u201d, \u201cswiftly\u201d, or \u201cwithout delay\u201d, and the reader could never say which word it would have picked on its own. That second slot is where the watermark lives. A signature can only hide inside choices that make no difference, which is exactly why it is invisible, and exactly why it cannot exist where the choice matters. So every scheme concentrates its signal at high-entropy positions, the moments of doubt. This signature is statistical and average-case. Detection is reliable \u201cgiven enough text\u201d, which is not the same as a per-output certificate.<\/p>\n
Currently Google marks all their Gemini family models. OpenAI built a scheme and chose not to deploy it. Anthropic has just announced they are going to add watermarks to their Claude family models. Open-weight models leave decoding in the user\u2019s hands, so their text is unmarked whenever the user prefers.<\/p>\n
How hallucination detectors work<\/p>\n
A hallucination is a fluent statement that happens to be wrong: an invented statistic, a fabricated citation, a number that appears in no source document. Current detectors can be divided into two mechanistically different families.<\/p>\n
Grounding-based detectors read the answer against a trusted source and ask whether each claim is supported. Token-support models such as LettuceDetect classify which spans of the answer lack backing in the context. Entailment models such as MiniCheck ask whether the source logically implies the claim. LLM judges do the same with a prompted model. What is common in these approaches is the input: the output text and the evidence, nothing else.<\/p>\n
Uncertainty-based detectors exploit a behavioral signature instead: a model that knows the answer produces it consistently, while a model that is guessing drifts across resamples. Semantic entropy[1] formalizes this by sampling several answers, clustering them by meaning, and measuring the entropy over clusters. SelfCheckGPT and the logprob-calibration methods are variations on the the same idea. In this family of detectors also the input is a common feature. the model\u2019s uncertainty, read either from repeated sampling or from the probabilities directly.<\/p>\n
The collision<\/p>\n
The watermark operates at high-entropy positions because those are the only positions where the choice can be steered invisibly. Uncertainty-based detectors reads those positions because hesitation is its \u201chere\u201d ignal. The watermark steps in at the moment of doubt, and resolves it with the secret key (a private number that fixes how every doubtful choice tips). The detector\u2019s trick depends on the resamples being fresh, independent tries. The watermark schema breaks that. Every retry uses the same secret key, so the coin flips inside the model land the same biased way each time, and in the strictest watermark schemes a fixed key produces exactly the same answer, word for word (Figure 2). If we ask a guessing model the same question five times we should see five slightly different guesses. That variation is the warning sign. With the watermark the five tries agree, not because the model stopped guessing, but because the same loaded coin decided all five. The steadiness we observe belongs to the sampler, not to the model\u2019s knowledge.<\/p>\n
![\"\"](\"https:\/\/www.europesays.com\/ai\/wp-content\/uploads\/2026\/08\/1*a8z0ItKtaDTnYtfSqzLAQg.png\")
Figure 2: The watermark removes the variation the detector was reading. Image by author.<\/p>\n
As a consequence, consistency-based hallucination detectors overestimate agreement and underestimate uncertainty on watermarked text.<\/p>\n
Not every confidence check works by asking the question again. A second family skips the retries and reads the model\u2019s internal probability numbers directly, the running record of how sure it was about each word. But the watermark has already edited those numbers before any check gets to read them, so this family inherits the distortion as well. A 2025 study across seven instruction-tuned models [7] found that watermarking measurably changes how models behave on downstream tasks, and that the changes remain after accounting for the loss in text quality. So, for any check that reads retries or probability numbers, the interference follows necessarily from how the watermark works.<\/p>\n
![\"\"](\"https:\/\/www.europesays.com\/ai\/wp-content\/uploads\/2026\/08\/1*BxranZG3aX1_lSkbQH7cZQ.png\")
Figure 3. One moment of doubt, one distribution over the next word. The check on the left is reading the model. The check on the right is reading the key. Image by author.<\/p>\n
Grounding-based detectors sit on the other side. They read only the output text against the evidence (for example in RAG systems), and the watermark selects among semantically equivalent continuations. So, in principle, they are not affected by watermarks as much as operate in a different space. If we need to run reliability checks on watermarked text, this is the family to run.<\/p>\n
The watermark removers<\/p>\n
The removal tools arrived on schedule, and nearly all of them are paraphrasers: a second model reads the watermarked text and says it again in different words. We are currently seing three approaches:<\/p>\n
Watermark stealing [8] showed that for under fifty dollars of API queries an attacker can learn enough of a scheme\u2019s hidden green-list rule to strip the mark from schemes previously considered safe, with success rates above 80 percent, and also to forge it.<\/p>\n
BIRA [9] (September 2025) rewrites text while steering the rewriter away from the words the watermark favored, and reports over 99 percent evasion without knowing which scheme was used.<\/p>\n
WASH [10] (May 2026) does not even rewrite: it averages the outputs of three ordinary models, the independent watermark biases cancel out, and detection scores fall from far above the alarm threshold to below it.<\/p>\n
The question is, do they work? The response is yes, and this is settled and it has a theoretical explanation. There is a mathematical proof of this with a fitting title: \u201cWatermarks in the sand: Impossibility of strong watermarking for generative models\u201d [11]. The argument is simple (Figure 4). Suppose the attacker can do two things: tell whether a small edit made the text worse, and keep making small edits that don\u2019t. Then they can wander step by step through thousands of rewrites that are all equally good, and somewhere along that walk the watermark\u2019s pattern gets left behind, because the pattern lived in the specific word choices and the words are no longer the ones the model chose. No current watermark scheme can survive that, whatever key it uses. Google, in its own description of SynthID-Text presents it as a marker for good-faith use, not as protection against someone determined to remove it.<\/p>\n
![\"\"](\"https:\/\/www.europesays.com\/ai\/wp-content\/uploads\/2026\/08\/1*XDqkX0hKmMY7HAO-r61XAg.png\")
Figure 4. A walk through equally good rewrites leaves the pattern behind. Image by authorSqueezing a balloon<\/p>\n
Remember which detectors the watermark could not hurt: the ones that never ask how confident the model felt. They take the output answer and hold it up against your source documents, checking word by word and phrase by phrase whether each claim is grounded in the provided context. The watermark changed which words got picked, but the claims still matched the sources, so these checks kept working. But the washing watermark-removal approach breaks exactly that safety. A washed text says the same thing in different words. Different words no longer line up with the source documents, so the matching scores fall, and a correct answer starts to look unsupported.<\/p>\n
Moreover, the washing tool is itself an \u201cAI rewriting the text\u201d, and nobody is checking its output. Rewriting is known to change things at the edges: a number shifts, a \u201cprobably\u201d disappears, a name gets swapped. The cleanup step can create the very mistakes the safety checks were built to catch. And since washing happens after the text is finished, it cannot bring back the natural variation the watermark removed. Conclusion: the confidence-based checks stay broken.<\/p>\n
The distortion only moves![\"\"](\"https:\/\/www.europesays.com\/ai\/wp-content\/uploads\/2026\/08\/1*MAoDZ3BoTlcZwNFG2oH96g.png\")
Figure 5: The distortion never disappears. Image by author.<\/p>\n
The distortion produced by the watermark and its removal process never disappears. It moves, like squeezing a balloon. The watermark presses on one side at writing time and distorts the checks that read doubt. The remover presses on the other side at cleanup time and distorts the checks that read evidence. Text that has been through both steps has weakened both kinds of hallucination-detection approaches. As far as I know, nobody has measured that second effect directly yet. The experiment is sitting there waiting for someone to run it.<\/p>\n
Forgery closes the loop from the other side. The same tricks that strip a watermark off can stamp one onto text the model never wrote. So the label \u201cwatermarked\u201d can fail both ways: missing from machine text that was laundered, present on false text that was faked.<\/p>\n
\ud83d\udcac Comments and suggestions are wellcome.\u2709\ufe0f You can contact me javier@jmarin.infoReferences<\/p>\n
[1] Regulation (EU) 2024\/1689 of the European Parliament and of the Council of 13 June 2024 laying down harmonised rules on artificial intelligence and amending Regulations (EC) No 300\/2008, (EU) No 167\/2013, (EU) No 168\/2013, (EU) 2018\/858, (EU) 2018\/1139 and (EU) 2019\/2144 and Directives 2014\/90\/EU, (EU) 2016\/797 and (EU) 2020\/1828 (Artificial Intelligence Act) (Text with EEA relevance) [https:\/\/eur-lex.europa.eu\/eli\/reg\/2024\/1689\/oj<\/a>]<\/p>\n [2] Measures for Labeling of AI-Generated Synthetic Content Document. State Information Office Tongzi [2025] \u21162. Cybersecurity Administration, Ministry of Industry and Information Technology, Ministry of Public Security, State Administration of Radio and Television [https:\/\/www.cac.gov.cn\/2025-03\/14\/c_1743654684782215.htm<\/a>]<\/p>\n [3] Farquhar, S., Kossen, J., Kuhn, L.\u00a0et al.\u00a0Detecting hallucinations in large language models using semantic entropy.\u00a0Nature\u00a0630, 625\u2013630 (2024).\u00a0https:\/\/doi.org\/10.1038\/s41586-024-07421-0<\/a><\/p>\n [4] Kirchenbauer, J., Geiping, J., Wen, Y., Katz, J., Miers, I., & Goldstein, T. (2023, July). A watermark for large language models. In\u00a0International conference on machine learning\u00a0(pp. 17061\u201317084). PMLR.<\/p>\n [5] Google DeepMind, SynthID documentation and the Hugging Face Transformers integration (SynthIDTextWatermarkLogitsProcessor, BayesianDetectorModel).\u00a0https:\/\/deepmind.google\/technologies\/synthid\/<\/a>\u00a0and\u00a0https:\/\/huggingface.co\/docs\/transformers<\/a><\/p>\n [6] Fu, J., Zhao, X., Yang, R., Zhang, Y., Chen, J., & Xiao, Y. (2024, August). Gumbelsoft: Diversified language model watermarking via the gumbelmax-trick. In\u00a0Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)\u00a0(pp. 5791\u20135808).<\/p>\n [7] Verma, A., Phan, N., & Trivedi, S. (2025). Watermarking degrades alignment in language models: Analysis and mitigation.\u00a0arXiv preprint arXiv:2506.04462.<\/p>\n [8] Jovanovi\u0107, N., Staab, R., & Vechev, M. (2024). Watermark stealing in large language models.\u00a0arXiv preprint arXiv:2402.19361.<\/p>\n [9] Hwang, J., Park, S., & Ok, J. (2025). LLM Watermark Evasion via Bias Inversion.\u00a0arXiv preprint arXiv:2509.23019.<\/p>\n [10] Wu, Z., Gong, G., Zhu, Q., Chen, Y., & Zhao, R. (2026). Linear Ensembles Wash Away Watermarks: On the Fragility of Distributional Perturbations in LLMs.\u00a0arXiv preprint arXiv:2605.30501.<\/p>\n [11] Zhang, H., Edelman, B. L., Francati, D., Venturi, D., Ateniese, G., & Barak, B. (2023). Watermarks in the sand: Impossibility of strong watermarking for generative models.\u00a0arXiv preprint arXiv:2311.04378.<\/p>\n","protected":false},"excerpt":{"rendered":"AI safety has a friendly-fire problem. The technology that proves where AI text came from is weakening the…\n","protected":false},"author":2,"featured_media":150709,"comment_status":"","ping_status":"","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[2],"tags":[24,4989,25,38873,73650,415,72404],"class_list":["post-150708","post","type-post","status-publish","format-standard","has-post-thumbnail","category-ai","tag-ai","tag-ai-safety","tag-artificial-intelligence","tag-editors-picks","tag-hallucination-detection","tag-llm","tag-watermarking"],"_links":{"self":[{"href":"https:\/\/www.europesays.com\/ai\/wp-json\/wp\/v2\/posts\/150708","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.europesays.com\/ai\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.europesays.com\/ai\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.europesays.com\/ai\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/www.europesays.com\/ai\/wp-json\/wp\/v2\/comments?post=150708"}],"version-history":[{"count":0,"href":"https:\/\/www.europesays.com\/ai\/wp-json\/wp\/v2\/posts\/150708\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.europesays.com\/ai\/wp-json\/wp\/v2\/media\/150709"}],"wp:attachment":[{"href":"https:\/\/www.europesays.com\/ai\/wp-json\/wp\/v2\/media?parent=150708"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.europesays.com\/ai\/wp-json\/wp\/v2\/categories?post=150708"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.europesays.com\/ai\/wp-json\/wp\/v2\/tags?post=150708"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}