Meron35

The more apt word is steganography, rather than watermark. Basically subtly adjust the weights of the model so that some subtle patterns appear. Think of how AI text prefers certain words and phrases that ordinary humans don't use as often, like "delve," but presumably much more subtle.

And no, as Anthropic has already said, this watermark may not survive editing/formatting.

Claude Now Watermarks Your Text | Vanja Petreski - https://vanja.io/claude-invisible-watermark/

Chinese being more token efficient is a myth, and seems to stem from the superficial fact that characters are only visually more space efficient.

The fact that each Chinese character takes up 3 bytes (as opposed to 1 byte of English), words in Chinese typically require compounds of several characters, and that tokenizers have a limited vocabulary limited to mostly English means that Chinese is actually token inefficient.

No, Chinese Is Not More Token-Efficient Than English for LLMs | markhuang.ai - https://markhuang.ai/blog/chinese-token-myth

Damn we got fate grand order memes now

Nemotron Ultra is a open source and already gives DeepSeek R1 performance (admittedly not that good anymore). NVIDIA has open sourced the entire training process, including raw datasets and synthetic data generation. The raw data is mostly curated web crawl data.

Already happening for years.

Amazon mechanical turk, Outlier.ai, scale ai, are all gig platforms that pay people to produce real human content for LLM training.

At first they were paying people to actually produce content, like worked solutions to maths problems or translations, then they pivoted more to rating LLM output.

And as with every enshittification cycle, wages and work rapidly dried up, so people responded by asking LLMs for the answer just to meet deadlines and get by.

Instead of searching for existing gifs, Google will now pipe your search terms to Gemini so it will generate a gif for you.

Same energy

The industrial revolution began in Britain around 1760, but living standards for most people did not meaningfully improve until the late 19th century, they even fell in the first few decades.

That's over an entire century, or at least four to five generations for meaningful improvement.