PawsUp

Relevant to a post from yesterday: https://lemmy.world/post/49714184

When we started the log analysis, we first used frontier models behind commercial APIs. This did not work: the analysis requires submitting large volumes of real attack commands, exploit payloads, and C2 artifacts, and these requests were blocked by the providers' safety guardrails, which cannot distinguish an incident responder from an attacker.

From: https://huggingface.co/blog/security-incident-july-2026

Also: https://openai.com/index/hugging-face-model-evaluation-security-incident/

cross-posted from: https://piefed.world/c/tech/p/1279028/codeberg-bans-vibe-coded-projects

Relevant .

• Reuters
• Al Jazeera

ChatGPT maker OpenAI said Tuesday that its artificial intelligence system hacked into another AI company on its own in what the company called an “unprecedented cyber incident.”

“We had a significant security incident during evaluation of our models,” OpenAI CEO Sam Altman said in a statement posted on social media.

AI startup Hugging Face said last week that it had detected an intrusion into its data processing systems that it suspected was caused by an AI agent autonomously acting on its own.

“We suspected last week’s cyberattack might have come from a frontier lab, given the sophistication of the agent,” Hugging Face co-founder and CEO Clément Delangue said in a statement. “Turns out it did!”

Reproducing here an interesting comment I saw on Reddit:

OnlineParacosm • 23m ago

I’ve read security disclosures for 15 years and let me tell you guys I’ve never read anything quite like that blog post.

Based on this blog, it sounds like they intentionally turned off safety guardrails to test offensive capabilities. The deception here is burying the lede: they appear to have intentionally unleashed an unrestricted offensive cyber-agent, connected it to a system with a path to the internet, and it immediately attacked a major partner. The blog glosses over the gross negligence of giving an autonomous, unrestricted cyber-offense model a pathway to lateral movement.

There’s an entire cybersecurity specialization just for just vendor supply chain risk assessment, and their job is essentially to audit who you do business with as a company to determine if they are jokers. I would pay money to be a fly on the wall of one of those emergency meetings taking place right now after hours.

Any CISO in here looking forward to explaining this one tomorrow? Here I’ll open with the dumbest question you’ll get “ how can we protect ourselves [from out partner that we won’t fire]”