MangoCats

A lot has been made about the tokens being subsidized, and at the frontier models I believe that's very true. I also believe that we're getting to where the frontier may be better, but not necessary, in order to get value out of using the LLM - and a couple of steps back from the frontier is becoming quite affordable now.

I'd be very interested to try out a GLM-5.2 instance on about a $100K server (8 bit quantitized) - which should cost under $20K per year to operate and maintain (although with component prices inflating the way they are that number continues to climb)... if that setup is as useful as, say, Claude Opus 4.5, that's a reasonable price level for a system that should be able to serve a department of 6-10 users pretty well. If GLM-5.2 isn't "there yet" - it's likely just a matter of time before one gets there.

Sad reality, I'm using LLMs to review the work of our offshore programmers, and as you say: some are good, some are not - and sadly, the ones who are not are also not learning to leverage LLMs to improve their work products. Without LLM review, I'd be advocating to find some of our offshore programmers "more productive ways to apply their skills." With four rounds of LLM review, I'm effectively rewriting (the bad ones') code for them... every... single... pull request.

My best offshore coder figured out how to use the LLMs to review his own code, we have architecture discussions about the best things to do and never have issues with how they are done.

My worst offshore coder "doesn't believe in using AI" and continues to submit code for merge to master branch with obvious race conditions, panic crashes, etc.

I realize the subscription rates are running at a loss for the LLM providers, but... costs are also dropping. Anyway, the Claude $200 per month subscription rate is more tokens than I could practically use doing heavy software engineering 40 hours a week. If I saturated it with jobs around the clock I could use up all the tokens in the $200/month subscription in about a week, but that's 168 hours. Working a 40 hour schedule, even at 4.33 weeks per month that's only 173 hours... I'd hope people spend at least 5 hours a month doing something besides babysitting their LLM console.

Now, one way to burn tokens at an epic rate is to launch multiple projects in parallel, three or even four sessions open at one time working on different things, or possibly different aspects of the same thing. That's also a good way to completely lose the picture of what the LLM is building and have no idea how a thing works when it gets done. I'm sure there are plenty of software development houses trying to "optimize" their workforce this way - I suspect their risks are... significant.

I think Ford and the rest are using AI as an excuse to do a rank and yank - everybody is let go, then some are invited back. This isn't anything new - Florida Power and Light (which is about 1/10th the size of Ford in terms of engineers) did this "fire everybody and make them reapply for jobs" thing back in the early 1990s. It's a way of "cleaning house" without singling out the bad apples.

a tool for a human to use

Agree

the tasks it’s good at weren’t large bottlenecks to begin with.

Disagree. There are plenty of tasks it is being tried with which it's not good at, but to an extent you don't know that until you've tried.

I use LLMs for code review, they help - a lot. Like so many new technology applications, we just didn't do code review at anything like the level that LLMs enable doing it at, with LLMs we're doing more of what we "always wished we had the time / attention span for" but never actually did, in practice. LLMs are lowering the total cost of finding issues in code, making the reward of doing them in depth worth the cost. In the past we'd do a much higher level review, and miss a lot of detail issues, but the cost of finding those detail issues without LLMs was very high and for the most part we deemed the detail issues "not worth the cost to find." Using LLMs for code review is actually making more work than we used to put into code reviews, but the returns are there - worthwhile.

You might think of it like finite element analysis for mechanical design. Sure, that was possible with paper drawings and slide rules, but it wasn't worth the cost using those tools. Now, there are areas of mechanical design where FEA modeling is the standard practice - if you don't do it you're considered to be slacking, not applying state of the art tools to the job.

There are some things that I find LLMs to be horrible at - like 3D drawings in Blender. However, they are actually really good at making specialized Blender usage tutorials to make it easier / faster for human artists to learn how to use the tool themselves - and once in awhile they can whip up a very helpful python script to automate something that would have been tedious / time consuming to do over and over but not quite worth learning python and the APIs to automate.

Never underestimate the capabilities of idiots. An idiot can render a bowling ball useless with a plastic spoon. Matter of fact, last time I got a ride to a bowling alley an idiot had rendered the seat belt in the car useless with a plastic spoon.

20 years ago, after 20 years of watching computers get faster and cheaper, I felt like they were "fast enough" - I mean, sure, more faster is more better, but for everything I had used computers for up to that point, they were fast enough - hell, they were already streaming DVD quality video by then on "normal" laptops. Certainly computers today are much faster still, but so much of that performance feels wasted on bloat rather than enhancing actual user experience.

LLM models seem to be evolving faster. A year ago, they were nowhere near good enough, but you could see the potential, much like desktop computers in the mid 1980s. Just make them faster, more powerful, more storage, higher resolution, you'll really have something. Today, I feel about the LLMs (for code) almost like I felt about computers in 2006 - they're good enough. Of course they could always get better, but if I were stuck with what we've got today for the next 5 years, I wouldn't be too disappointed. The interesting question (that nobody seems to have a real answer for) is: how much better will they get. A year ago there were obvious rough edges that have quickly been smoothed off... how smooth can they actually get?

LLMs for graphic arts? Yeah, that feels like MS paint levels of performance at the moment, they definitely have room for improvement.

At 10 years lifetime, it's sounding like the hardware costs as much to buy as it does to run - not factoring in time value of money...

I just asked Gemini to estimate run costs for a local GLM-5.2 instance, something that a team of a few software engineers might use the way they are using Cursor today... power budget is 6KW, which around here - after facility cooling costs - works out around $1000 per month. Our Cursor subscriptions have $100 per month price tags on them for the developers who use them most extensively, and this $100K to buy in $1K per month to run local instance isn't likely to serve more than a dozen engineers efficiently. Even if you can lease it out at full utilization 24 hours a day, it doesn't sound like much of a money printing machine to me, yet.

My $20/month home subscription to Claude? Even less so.

GLM-5.2 ? Gemini says you can run a competent 8 bit local cluster for around $100K purchase and under $20K/yr run costs.

largely trained with Reddit shitposts.

How else do you expect the world to end...?

And those Redditors thought they'd never amount to anything.

a much higher risk of harming other people by using chainsaws.

That all depends on where you let them work.

If an idiot with a chainsaw drops a tree on himself in the woods and nobody is around to hear him scream...