Technology

I know exactly how it works. I read the article, and I knew of it beforehand, hence I explained it to someone else in a comment three days ago:

https://lemmy.world/post/50533770/25234461

I’m sorry to jab back, but you hit a button of mine.

Lemmy commenters keep jabbing me with comments like “Clueless. Read the article and get back to me.”

Like yesterday:

https://lemmy.world/post/50595546/25269317

But I'm aware of how sampling works. I knew all about LLM fingerprinting ~two years ago, and I’ve been tinkering with samplers myself for years. I’ve messed with local LLMs trying to make them “aware” of their own sampling many times, and even hacked out a (unsuccessful) experiment where a tiny LLM picks tokens for a larger one.

I’m not trying to be pretentious, I’m not a researcher or expert or anything, but you shouldn’t assume everyone on Lemmy is clueless.

And back on topic… to be clear, I have tried what you are proposing, and even with local LLMs I have more control over, it doesn’t work. They have extremely poor “awareness” of their own logit spread and tokenization, which is why they perform so poorly on any tasks that depends on that.

You can’t tell them “don’t pick the top word” or “give more options in your logit spread” because that part of the process is completely invisible, from their perspective.