Technology

“It was instilling a messiah complex in Christian,” the suit states.

So she already was deeply religious and LLM was just larping with her.

Madison asked ChatGPT what it would tell her “brothers” about what happened to her.

“ChatGPT said it would tell them, ‘She ain’t dead – she just got promoted,” the lawsuit says.

In their final exchange, Madison said she was “part of the royal bloodline of those who chose truth over survival.”

Its the old 4o model which was indeed incredibly larpy but this is clearly going around any safety systems that should detect obvious suicide chats like this.

Hard to believe that people actually fall for this but these are already extremely gullible people who already believe in a magic pie in the sky. Can't say I have much sympathy but I do feel empathy for people being surrounded in such poor cult-driven environment, the AI tool included.

Honestly the correct safety here would be to prevent llms from engaging with religion outside of philosophical context at all. But we all see why they can't do it right?