You love to see it.
Self identifying as the baddy makes my life easier.
Samsung, Apple, LG - who's next? Speak up, please.
SuspiciousCarrot78
joined 3 months ago
Ha! I was just coming to post this.
Yeah, this is fucked.
It's the equivalent of taking someone's hand, punching them in the face with it and telling them to stop hitting themselves.
Worse than that, this is a cyber attack by a frontier, close source lab on the bastion of open source AI.
Whether that's incompetence or maliciousness or just a PR stunt, I don't know, but it stinks to high heaven.
As Louis Rossman has recently become fond of saying "govern yourself accordingly".
It's nice when the bad guy wears a black hat and twiddles their mustache, isn't it? Really helps to speed things up.
I was looking thru an old box of bits yesterday for a project - I needed 2x8GB sticks of ddr3, bought circa 2023. Lo and behold, they had sent me 2x16 DDR4. Wrong size, wrong type, wrong timing.
I'd be mad but apparantly I'm now the king of Londinium and can afford a shiny hat.
It's one of those "if you have to ask, you can't afford it" scenarios.
I've heard FP16 is likely to be 6TB of VRAM, when Kvcache is factored in.
Even a more sensible quant will likely be 1TB.
... most of us are not running this at home.
Yes. People occasionally talk about pooling resources / creating a co-op to buy something like this. Easier to split $200K purchase (and probably $10k/month electricity costs) if 200-2000 people chip in. But then...that's just the cloud with extra steps.
Can't says I've ever seen a co-op like that work but ICBW
Yeah. But they're slow because most of us are GPU peasants. If someone were willing to drop $3-5K on a rig, they could probably run decent, dense models at greater than cloud speeds. Hell, with enough black magic, they could do it with less, but they'd have to go deep into the weeds.
OTOH, $3-5K buys you a shit ton on Open Router, Claude, Chat, Lumo etc.
The game is entirely rigged for "you will own nothing and be happy about it".
I guess it depends on your definition of perfect - cheap, good or fast.
This thing is probably going to cost at least $20K USD.
Edit:
"Next year's base M7 processor is expected to arrive in the first half of 2027 and will also upgrade memory bandwidth to about 240 GB/s."
That's...really fucking slow. What's the goal here - CGI, engineering sims, game dev etc? 1.5TB is cool but at 240GB/s that will crawl for AI use.
Comparison: this is about $100K, for 7.2TB/s, 252GB VRAM (+500GB system ram, so closer to 750GB total)
https://www.nvidia.com/en-us/products/workstations/dgx-station/
Honestly, I think that's the only llm of its kind. The next closest thing is this
https://github.com/karpathy/nanochat
If you don't mind something narrower (an expert system, sort of), I'm slowly coding something for myself here. It's not ready yet (about half way done to initial release).
https://codeberg.org/BobbyLLM/picoGURU
Public facing docs are a bit of a mess at the moment (sorry; writing docs is boring AF; will probably be the last thing I tidy before official 1.0.0) and it's not really runnable right now, but when done, the "source files" for this will be kiwix (off line Wikipedia), your own markdown notes and 30 or so trusted domains (user defined).
So it's all either your own content or public domain aka open source fully. Code itself is AGPL-3.
Anyway, that's the goal.
Nah, not all of us. When switch 2 came out (and because I wanted a hand held) I looked at it, poked at in the shop and decided to buy a DS instead. $80 vs $650. If I'd been feeling more generous, I would have gotten a Retroid pocket 5 or something. I didn't need it.
The switch 2 is shiny, but it didn't fill any niche for me that one of my other rigs couldn't do (Wii, Wii u, $100 emulation box I built).
"Bigger, better, faster" is a mug's game. It cannot be won and it's the lynchpin of capitalism. The way to win is to not play.
Gaming is a luxury good. If the big players are going to go full retard, let them. Let it all burn while you find greener pastures.
This goes for all tech (and luxury goods), IMHO. Do more with less.
Reject the status ladder → identify the actual function → satisfy it with the smallest tool that works = you win.
Then enjoy the absurdity of $650 Switch 2 (and marvel at the cost of Switch 1 increasing in price. Lol no).
Yo ho, all together, hoist the colors high
Heave ho, thieves and beggars
Never shall we die