Since Anthropic’s announcement a couple of weeks ago that they (along with most other LLM providers) would start adding an “imperceptible” watermark to Claude outputs, there’s been a lot of consternation online. Does it degrade the quality of the model’s generated text? Some assert it absolutely does not, and others assert it absolutely does.

Hm!

Well, I guess I have to go figure the darn thing out so I can make up my own mind.

The actual implementation is more in-the-weeds than I was hoping, at least for someone with a good decade between them and their last statistics class and no LLM familiarity beyond the basics. Still, after spending a few hours going back and forth with the papers and articles, not only do I think I have a more definitive answer for whether the process degrades the output, but I also believe it’s not so hard that only an academic can understand it.

So, I’ve taken that intuition and tried to distill it into a short explainer here, broken down into 4 stages with each stage building one small new thing upon the last, starting with normal LLM token sampling and ending with full-on SynthID.

Let’s see if we can understand it together!

Stage 1: Sample One Token

We’ll start with the normal way an un-watermarked LLM emits text.

In short, the token-generation process is to:

1. Look at all of the text that has preceded the token you want to emit
2. Do a lot of math to generate a probability distribution of what the next token could be
3. Sample from this distribution, emit that token – then do it all again

So, for a situation where the LLM’s next-token probability distribution looks like this:

Then you can try out the sampling process yourself to get a feel for it:

LLM knobs you may have heard of, like temperature and min-p, can control aspects of this distribution (e.g. how “sharp” it is, or how many distinct options there are), but the process we’re talking about here is always the same.

Stage 2: Sample Two Tokens

Now, let’s make a small modification to the sampling process above.

Instead of just sampling one token from the LLM’s distribution and emitting that, we’re going to sample two tokens, and just flip a coin to determine which of the two we’ll emit.

You can try out this modified sampling process yourself:

The important thing to note here is that this is perfectly equivalent to Stage 1 in terms of the outcome. Despite the extra steps here, we have not changed the model’s outputs in any way yet, as you can see when we take “mango” as an example:

# Stage 1
P("mango") = P("mango" is sampled)
           = 65%

# Stage 2
P("mango") = P("mango" is sampled first) * P(coin is heads) +
             P("mango" is sampled second) * P(coin is tails)
           = (65% * 50%) + (65% * 50%)
           = 65%

So yes we’ve just made it convoluted for no reason but BEAR WITH ME!

Stage 3: The ‘g’ Function

Now, the real watermarking begins here in Stage 3.

We’re going to change just one thing from Stage 2, which is the coin flip – we’ll replace it with a simple function called g, which, sometimes, has a preference between the two candidates in the contest.

Other than that, the process is exactly the same as in Stage 2, as you can see:

So, the question is – what is this g function, and when does it prefer one token over another?

Fortunately it’s pretty simple! Essentially:

• g calculates hash(previous_4_tokens ++ secret_key ++ candidate_token)
• g likes a candidate when that hash comes out as an even number (i.e. ends with 0)

And when g is given two tokens to decide between, it just opts for a token it likes, if there is one. If it likes both candidates (or it dislikes both), it selects one at random. This preference is what makes up the watermark!

You can put your own secret in below to see how g works:

You might guess that, in our toy example here, there are 16 distinct like/dislike assignments that g could have, 16 different “moods” it could be in, depending on the secret key and the 4 prior tokens, shown as 16 columns below:

M: 💚 │ 💚 │ 💚 │ 💚 │ 💚 │ 💚 │ 💚 │ 💚 │ 👎 │ 👎 │ 👎 │ 👎 │ 👎 │ 👎 │ 👎 │ 👎
B: 💚 │ 💚 │ 💚 │ 💚 │ 👎 │ 👎 │ 👎 │ 👎 │ 💚 │ 💚 │ 💚 │ 💚 │ 👎 │ 👎 │ 👎 │ 👎
C: 💚 │ 💚 │ 👎 │ 👎 │ 💚 │ 💚 │ 👎 │ 👎 │ 💚 │ 💚 │ 👎 │ 👎 │ 💚 │ 💚 │ 👎 │ 👎
P: 💚 │ 👎 │ 💚 │ 👎 │ 💚 │ 👎 │ 💚 │ 👎 │ 💚 │ 👎 │ 💚 │ 👎 │ 💚 │ 👎 │ 💚 │ 👎

And you’d be right!

Each of the elements of the hash serves an important purpose:

• previous_4_tokens is there so that g’s preference for a token is dependent on the tokens that come before it. Without this, g might (for example) just universally dislike “banana”, which would be noticeable to users
• secret_key being included in the hash means that only the owner of the LLM can apply, or detect, their own watermark
• candidate_token is, of course, the subject of our watermarking

An interesting thing is, without all three of these parameters, the hash is totally unpredictable, i.e. indistinguishable from random noise. This means, without the secret key, g’s choices seem random – there’s no statistical test you can run on a piece of text to tell whether it came from Stage 2 or Stage 3.

However, that doesn’t mean g has no impact on token probabilities. That is next!

Stage 3: Distortion

Let’s focus for a moment on the least probable token the LLM could generate: “papaya”, at just 5% likelihood. In a contest between “papaya” and any other token, there are 4 possible scenarios of which ones the g function will have a preference for:

• g likes both of them
• g likes neither of them
• g likes “papaya” but not the other one
• g likes the other one but not “papaya”

In the first two situations, g acts the same as the coin flip – no watermarking business occurs. But in the latter situations, things get weird – g no longer acts like a random coin. In the third scenario for instance, g will always choose “papaya” over the other candidate.

So, remember the “probability distribution” created by the LLM in Stage 1? This can distort it pretty substantially. If you try the buttons below, you can see how “papaya” is affected by 2 of the 16 different g moods: the one liking “papaya” and no other tokens, and the one liking all tokens except for “papaya”:

In the “g likes only papaya” case, “papaya” will automatically win any contest it’s in. This nearly doubles the chances that the LLM will emit “papaya” as the next token.

But, the “g likes everything except papaya” case is even more extreme – it means that the only time “papaya” can be emitted is when it appears as both candidates in a contest. This almost never happens, and its real chances drop to almost 0%.

Stage 3: Detection

Notice that crucially, g only relies on 3 parameters – two of which you can plainly see in the generated text itself, and secret_key, which is just a fixed string that lives on a post-it note in somebody at Anthropic’s desk.

What this means is that we don’t need the LLM around to determine if text was watermarked, we just need the text itself and the secret, and we can run the same calculation g did. This part was really non-intuitive to me – I thought I knew how watermarking worked (seeding the LLM’s PRNG), but when I dug in I realized that detection would have more or less required the entire context + re-running the LLM 😅

Anyways, calculating g ourselves is important because that’s how detection works! You can see the process right here:

We walk through the text, calculating g for each token using the 4 tokens before it and our secret key, tallying up how many g liked and how many it didn’t. Then, we use those counts to find the ratio, the “percent of tokens g preferred”.

If that ratio is suspiciously high, then the text is watermarked!

Stage 3: A Suspicious Ratio?

For me, the hand-waving above leaves a bit to be desired – what should we expect this ratio to be for watermarked and non-watermarked text, exactly? How do we know when it’s high enough to be “suspicious”?

To understand, let’s revisit the little contests that happen during generation, and look at cases separately – when g is deciding between two different tokens, and when it gets duplicates.

First, let’s consider the case where g is handed two different tokens. Remember from earlier the 4 situations, each equally likely on average, and in particular, let’s see what kind of token is output in each:

g likes both candidates           =>   💚 g picks a token it likes
g likes neither candidate         =>   👎 g picks a token it does not like
g likes the 1st but not the 2nd   =>   💚 g picks a token it likes
g likes the 2nd but not the 1st   =>   💚 g picks a token it likes

Do you see how, in 3 of the 4 situations, the LLM ends up emitting a token g preferred? Thus the expected ratio is pushed up towards 75% – pretty high!

But, now suppose g sees the same token twice to “decide between”. Well, it’s not really much of a decision, right? It returns the token regardless of whether it liked it or not, and moves on. In our running example, that’s gonna happen almost half the time in fact:

P(duplicate) = P("mango" & "mango") + P("banana" & "banana") + P("coconut" & "coconut") + P("papaya" & "papaya")
             = (65% ^ 2) + (20% ^ 2) + (10% ^ 2) + (5% ^ 2)
             = 42.25% + 4% + 1% + 0.25%
             = 47.5%

Any g function will like and dislike tokens in equal measure, thus the expected ratio is pushed the opposite way, down towards 50%.

So – “suspicious” means “somewhere between 50% and 75%”, depending on the text!

Stage 3: Predictable Text

You might have noticed that the latter case above – when g is handed two copies of the same token – is kind of problematic. The more often that situation crops up, the closer the expected ratio is pushed down towards 50%, and the smaller the difference becomes between normal text and watermarked text.

To make this really clear, let’s see what happens when an LLM is asked to regurgitate a piece from Shakespeare’s As You Like It:

In this situation, the two candidates for g to decide between will both be “stage” a whopping 98% of the time. So even if g doesn’t like “stage”, that is still almost always going to be what the LLM emits.

And, if we generate a lot of tokens like that, you can see how it affects the “ratio of tokens g likes” as we run our detection process:

So you can see how the process relies on creativity/freedom in token generation to embed its watermark; when the text being generated is so constrained, it’s pretty much impossible.

Stage 4: SynthID

Phew, we made it!

If you thought this was going to be the most complicated section, sorry to disappoint – in fact we’ve already done all the hard work.

Google’s SynthID is just the process we detailed above, repeated a bunch of times.

There are no problems with the approach from Stage 3 per se – as we saw, it does effectively watermark text, and we can detect it after the fact – but in practice researchers found that at detection-time, it takes way too many tokens to build up good confidence about whether text bears the watermark or not.

So, “higher signal” is what SynthID was built for! It’s hugely scaled up from what we discussed so far – instead of 1 g function, there are 30, and instead of 2 candidates, there are (literally) a billion, all competing in a March Madness style single-elimination bracket.

A simple 3-round example is below, with its corresponding 2 ^ 3 = 8 candidates:

The logic here is – if a token being preferred by one g-function gave us a little signal, it being preferred by all 30 functions gives us a lot more signal, since it’s so much less likely for that to happen by chance.

This does meddle further with the token probabilities, though.

Remember how in Stage 3: Distortion, we saw that g could take “papaya” from its original 5% likelihood to anywhere in [0.25%, 9.75%] depending on its mood? Here each additional round takes the mood of the previous round and distorts it again, causing that range to stretch wider and wider. Crucially, though, if you average papaya’s chances across all 16^30 possible moods of the 30-round tournament… it comes out to 5% still!

And, SynthID is extremely successful at what it set out to do – detection is much easier. From Section 4.1 of this paper:

When trying the watermark + detection process on 100-token-long bits of text, they saw the TPR (“true positive rate”, the percent of actually-watermarked texts that they could correctly identify) increase from a paltry 4% for the 1-round tournament, up to 88% for the 28-round tournament, which is transformative!

Fin

So, I learned something that I might have been able to see from the start – that watermarking does distort the probability that a certain token is generated at a certain moment (and I mean, of course it does, since that distortion is the exact thing that we can detect afterwards).

But, does it degrade the quality of the text overall? I don’t believe so.

Yes, it definitely sounds bad that “papaya” having a 5% chance is turned into a 0.25% chance (…sometimes, and an 9.75% chance just as often).

But, remember my “watermarking is easy, just seed the PRNG for the LLM” misunderstanding from earlier? In the end, this isn’t actually any different – after all, a seeded PRNG is maximally “distorted” too, with every probability collapsing to either 0% or 100%, and nobody considers that degradation! The decision for “papaya-or-not-papaya” may come from /dev/urandom, or a seed, or hash(previous_4_tokens ++ secret_key ++ token), but regardless the outcome is the same – “papaya” still shows up in an unbiased 5% of situations.

Besides, empirically, it doesn’t end up mattering much. Part of the Nature article details an A/B test that Google ran between vanilla vs watermarked Gemini, where they found no significant difference in user ratings. This is referenced in Anthropic’s later post too, where they ran their own internal tests and found “no impact of watermarking on the content, level of creativity, or readability of Claude’s text”.

So, maybe this is all tilting at windmills, but – at least they’re pretty interesting windmills.

¯\_(ツ)_/¯