Reportedly, the app uses an on-device machine learning model to classify the content.
Using your shiny new phone to train AI...
For now its only supposed to check Google Messages, so not if you use a different text app.
But one vulnerability already is side loaded versions with super high version numbers so they won't be auto updated. Those will scan everything and do whatever with it. Considering the apps point is to scan for nudes, likely sending all your nudes to the hacker.
Techbros didn't even invent that shit.
That and "disruption" was just going into an existing market, flooding it quickly with cheap prices, and the second you squeeze out competition jack up rates.
It's the same thing Walmart did.
With AI they offer it cheaper than humans because they're burning investor cash and not making money. Once a company fires staff, their next AI contract switches to per token price instead of monthly unlimited.
And then suddenly realizing the "rockstars" that kept their job because they were AI experts, were just sitting at their laptop 8 hours straight constantly prompting over and over again. After that they're paying more than they were before AI with a shittier product and completely unpredictable workflow completely reliant on AI usage.
Hopefully we're reaching the point even c-suite execs figure it out. Once they do it'll pop.