> “By definition it must make text worse … because the nature of the watermarking algorithm requires it to sometimes increase the probability of selecting a worse word choice and decrease the probability of selecting the model’s best choice.”
Gruber made an effort to but doesn't fully understand how SynthID works. LLMs select the next word randomly from a set probability distribution, so there is no "best choice" unless you run the LLM with a temperature of 0 which would give out terrible results. Anthropic runs a non-distorting version of SynthID that doesn't change the probabilities of the underlying distribution of tokens. It makes the watermark less likely to work over smaller samples but preserves text quality. I encourage the mathematically inclined to read the paper:
> LLMs select the next word randomly from a set probability distribution, so there is no "best choice" unless you run the LLM with a temperature of 0 which would give out terrible results.
I'm not understanding how the word with the highest probability isn't the "best choice"?
Because it's pure exploit on the explore/exploit tradeoff. The best outputs come when the LLM comes up with lots of different ideas, considers them, and selects the best one. If you sample at low temperature it tends to regenerate the same ideas over and over.
You will understand if you run some LLM models with a greedy sampler that does that. The text quality begins to deteriorate rapidly. This is a very counter intuitive result so I don't blame you for not understanding until you actually tried it and experienced it for yourself.
> You will understand if you run some LLM models with a greedy sampler that does that. The text quality begins to deteriorate rapidly.
Right, I've done this, and this makes sense to me, but I'm not following how that falsifies the top probability word being the best choice in any particular instance.
"Picking only the best word at each decision point results in a worse final result" seems like an imminently reasonable hypothesis.
I think you're using different definitions of best. If best = leads to a correct answer overall then by definition anything that leads to a bad outcome can't be best
I came here to quote the same sentence. Here's another way to look at it:
Suppose there actually is a best word choice. The LLM doesn't know what it is but makes a guess. Maybe it's the best one, maybe it isn't. The probability that SynthID changes the best choice to a worse one is equal to the probability that it changes a worse choice to the best one.
I think that depends on the distribution of good choices and bad ones. There may be 10 choices and maybe 8 of them could be appropriate given a context, and 2 are absolutely nonsensical. Or it could be vice versa. And its a spectrum as well.
But the choices are weighted based on those probabilities. This doesn't affect the weightings, only how the final weighted pseudo-random selection is made.
That's not how LLMs work. If you're talking about number of parameters, you wouldn't be able to reduce the size much by "removing" support for other languages.
Imagine our LLM as this one-dimensional matrix (instead of unfathomably many, many more dimensions):
0.728201017
-1.9293839303
2.93020202929
5.82290101728
-0.2844920917
…
And then, all of these values are multiplied by some scaling factor, essentially vectors pointing in some direction (in our case, only on a one-dimensional line instead of in some higher-dimensional space).
Now imagine it’s possible to “remove” the portion of the weights’ directions that’s responsible for some undesired property like history knowledge, or another programming language:
0.428201017
-1.6293839303
2.63020202929
5.52290101728
-0.5844920917
…
(For demonstration and faster-to-type-out purposes, I’ve changed each vector by deducting a constant amount of 0.3, which doesn’t make much sense. In reality, it’d modify probably all digits in some direction.)
Now we can see: the number of rows has stayed the same, but the value has changed. This means the required storage (or computational) space remains the same.
Please do correct me if this simplified too much or I had a mistake in my understanding.
This is oversimplified. The proposed question is whether a model with fewer parameters could achieve performance on one language similar to that of a larger model that’s been trained more broadly, which isn’t straightforward to do.
Oh, I didn’t interpret the above question as asking in that direction; but yeah, that’s of course something I didn’t attempt to answer with my comment.
Although I’d be intrigued in the answer to that small-narrow vs. large-broad model question, too!
The exception being the token embeddings and lm head (which scale with the number of tokens the model knows and presumably you need a smaller number in the tokenizer for only English and python). But those are a pretty small % of the total model weights on most LLM sizes
For a set parameter size, if you benchmark an LLM on a single language, you'd get better results by training it on multiple languages rather than just the one. Scaling quality training data almost always increases performance as the neural network picks up general rules and an understanding of code that are language agnostic.
Only a tiny, tiny fraction of the parameters are encoding information that's specific to a particular programming language. Even if you could remove those without degrading performance, it would have a negligible effect on the model size.
n=56 doesn't give you much information regarding the margin of error, unless you practice Tarot Reader's cat science. The standard deviation of outcomes and the difference between both outcomes matter just as much.
If I flip a coin 56 times and it always falls on head, I can be pretty much certain that it's not a fair coin. I wouldn't need to flip it 1000 times. We are all someone else's "know-nothing hack"...
Can we know for sure that the price drop is accompanied by a change in the model such as quantization?
On twitter, some people say that some models perform better at night when there is a less demand which allows them to serve a non-quantized model.
Since the models are only available through API and there is no test to check which version of the model is served, it's hard to know what we're buying...
You can have growing deficits without actually "selling the nation" when you account for growth and inflation.
Say foreigners own 50% of US assets. When someone creates in the US a company ex nihilo and sells 10% of it to foreign investors, even though foreigners bought more US assets, this new wealth creation causes the % of foreign-owned US assets to diminish.
When foreign investors own US bonds, as inflation increases and the money supply increases, they end up owning less in real terms.
This doesn't account for investments that are limited in quantity such as real estate.
Also just because those are opposing forces, doesn't mean that foreign ownership wouldn't increase. As a matter of fact, when looking at the foreign ownership of US stocks, it has been steadily increasing.
Does this support the Selfish Gene theory of Richard Dawkins? They look like the smallest self-replicating molecules that he mentions in the beginning of the book.
Absolutely, reading Selfish Gene in high school set me on the path to this type of exploration. Genes, in the pure abstract sense, are the unit by which we interrogate understand evolutionary change. There's a large grey area about the boundaries of genes, but after a certain point, genes assemble into operational units larger than themselves, a genome. Obelisks are some of the simplest, most rudimentary genomes described thus far.
Gruber made an effort to but doesn't fully understand how SynthID works. LLMs select the next word randomly from a set probability distribution, so there is no "best choice" unless you run the LLM with a temperature of 0 which would give out terrible results. Anthropic runs a non-distorting version of SynthID that doesn't change the probabilities of the underlying distribution of tokens. It makes the watermark less likely to work over smaller samples but preserves text quality. I encourage the mathematically inclined to read the paper:
https://www.nature.com/articles/s41586-024-08025-4