this post was submitted on 20 Aug 2026
60 points (87.5% liked)
Technology
87598 readers
4071 users here now
This is a most excellent place for technology news and articles.
Our Rules
- Follow the lemmy.world rules.
- Only tech related news or articles.
- Be excellent to each other!
- Mod approved content bots can post up to 10 articles per day.
- Threads asking for personal tech support may be deleted.
- Politics threads may be removed.
- No memes allowed as posts, OK to post as comments.
- Only approved bots from the list below, this includes using AI responses and summaries. To ask if your bot can be added please contact a mod.
- Check for duplicates before posting, duplicates may be removed
- Accounts 7 days and younger will have their posts automatically removed.
Approved Bots
founded 3 years ago
MODERATORS
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments
Okay so the paper is actually interesting, and they have a explainer website: https://zheng-dai.github.io/AblationBasedCounterfactuals/
https://www.nature.com/articles/s41467-026-75667-5
https://github.com/zheng-dai/counterfactualuniverses
Amongst other things, they experimented with "what happens to a generation if you remove one image from the training dataset," particularly the "source" image for the target generation. What makes it interesting is they tested this at scale. There's (expectedly, somewhat janky) demo code to illustrate the spread. One example I exported myself:
Where each generation (the "counterfactual") in the top row corresponds to the image that was removed from the dataset in the bottom.
...Personally, I don't find the conclusion surprising.
These are models.
Lets say you make a model to predict hurricanes. With a small dataset/model, exclude the hurricane most similar to the one you're trying to predict, and it won't do a good job. But do the same with a model based on a huge dataset, and it should still model the novel hurricane reasonably well.
Diffusion models are no different.
I think AI Bros have overly anthropomorphized them with terms like "creativity" and such; they don't have this. But at the end of the day, they can model things that aren't strictly in their dataset. That's kind of the basic premise.
Yeah people don't understand the math behind these things. They are statistical models, like demographics or climate models. They are interesting and potentially useful, but not necessarily useful for the things we are trying to and want to use them for. They can predict certain things and trends very reliably in a general, high-level sense, but they will get the details wrong if you try to use them outside of their area of applicability, just like if you try to use national climate models to determine what day your basement is going to flood, it may come up with a specific date, but just as obviously that's not actually going to be correct in reality. It doesn't mean it's a bad model, it means you're using it wrong.
And lots of AI bros and AI companies are using these things desperately wrong, and for very wrong reasons. They want to sell things to you. They want to control you and manipulate you. They don't care if they're using the model properly, they only care if it's profitable for them to do it.