this post was submitted on 08 Aug 2026
518 points (98.5% liked)
Technology
87598 readers
4129 users here now
This is a most excellent place for technology news and articles.
Our Rules
- Follow the lemmy.world rules.
- Only tech related news or articles.
- Be excellent to each other!
- Mod approved content bots can post up to 10 articles per day.
- Threads asking for personal tech support may be deleted.
- Politics threads may be removed.
- No memes allowed as posts, OK to post as comments.
- Only approved bots from the list below, this includes using AI responses and summaries. To ask if your bot can be added please contact a mod.
- Check for duplicates before posting, duplicates may be removed
- Accounts 7 days and younger will have their posts automatically removed.
Approved Bots
founded 3 years ago
MODERATORS
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments
Well, the current trajectory points to:
LLM capabilities topping out.
All the tooling around them not keeping up anyway.
Actual “AGI” being distant and completely unrelated to contemporary models.
Inference costs plummeting.
The last one is critical.
Right this second, you can run Minimax H3 on a desktop for the tiny fraction of the compute/RAM OpenAI Sora took. And it’s better.
In a month, it will be ~8X faster.
You can run DeepseekV4 flash, dirt cheap, and get what Claude was less than a year ago. And it’s gonna spread to every host out there, to systems like Cerebas ASICs that don’t even need HBM.
So… Even if you’re an AI acolyte. And we go with that for the sake of argument…
We don’t actually need all that RAM for hosting generative models?
I’m very interested to see what happens to all these datacenters over the next two-three years.
They spent all this money on something that’s gonna be cheap as dirt to run, largely run locally, and that won’t need GPUs once bitnet takes off, sooo… they can’t make money off that.
What happens then?
What happens to all those Stargate RAM wafers, and excess datacenters?
You are 100% correct IMO for the AI stuff, just a question, what's bitnet? Is it some supposed bitcoin-killer (because that's not going to happen, IMO).
I also think we're on the cusp of "having enough" personal computing power with the usual exceptions, take a bit, give a bit.
Our mobile phones are close to what's needed for most people (8GB, 4k, gaming, ...). Or so I think!
Bitnet is a catch-all title for ML models that use a specific mathematical trick.
If elements of a matrix are composed of only 1, 0, or -1, multiplying them is the same as as adding them.
That’s huge. LLM computation is basically all matrix multiplication, so if you replace that with simple addition, you reduce the computational requirements by orders of magnitude.
The catch is such models are hard to train effectively; its proven that it works, but research to get the technique usable and practical is still being done.
Personally, I suspect it’s unviable for many “dense” parts of models, but sparse hybrid bitnet models would be really cool.
I mention it because, if it takes off, suddenly the massive matrix multiplier accelerators we have for LLMs aren’t as useful. Chips with simpler architectures could get the job done, at least for parts of models that are bitnet.
Thank you! Ah yes that makes sense somewhat I guess. Trading cpu usage for memory usage, an old classic 😁!
Nibble * nibble look up tables could probably be fast too, it all depends on where you want the complexity I guess.
There are indeed some Bitnet LUT implementations, already.
It’s both smaller and faster though; the models are drastically smaller than FP16 at the same parameter count, as they’re just 1.5 bits instead of 16.
How much is lost from the precision drop is an open question, though, and the answer seems to be complex and finicky. I think it’s why big labs are hesitant to pursue it; they want to try safer, lower hanging fruit first. Also, it’s not particularly efficient on hardware (like AI GPUs) where most of the silicon is dedicated to matrix multiply performance anyway.