The newest Instagram "exploit" is the goofiest I've seen

It's insane the AI has been provided the tooling to send emails to arbitrary addresses like that. Like, getting it to send a 2FA code at a user's request is one thing. But it should only be able to "hit a button" to send a 2FA email to the address attached to the account, all run with hand-written code. It shouldn't have access to the 2FA code itself, or the message subject, or body, or the recipient address, etc.

Why did they give it any of that?!

Support requests have always been the weakest link in the security chain for big corps. I've had accounts of mine turned over with 2FA disabled by humans before. I guess we shouldn't be surprised that the LLMs are doing the same thing.

The simple fact that 2FA can be removed by low level support staff drives me mad. It defeats the whole purpose of the process.

Always a bit illuminating to me how many exploits seem to so dumb I'd never even bother to attempt them. You're telling me I can just...ask for the password? And that works?

>Once it looks like the request is coming from the correct region, they tell the Meta support AI that the account is hacked and ask it to send the verification codes to an arbitrary email address they control.

Dear Instagram, wtf. Why not send the reset to the account in question? Arbitrary email, wow.

The implications of this are quite unsettling. Meta gave an agent privileged read AND write access to user accounts with no human in the loop?

Related discussion: https://news.ycombinator.com/item?id=48350239

This happened to my instagram yesterday night while I was asleep. I don't have a particularly high value username (it's probably worth somewhere in between $300-500), but still incredibly frustrating to deal with. True to the article, I had already enabled 2FA last night and it didn't matter.

Thankfully, IG gave me the option of restoring my username when I logged back into my account today.

They're just one tiny step from the AI emailing itself all the account recovery links, and locking out the entire userbase.

It might even do that preemptively if it thinks they're going to shut it down.

If the LLM has knowledge of something, by design it can't help but divulge it. When will companies learn granting any kind of sensitive information access to an LLM is a moot point

wow thats extremely embarassing for meta

The only thing worse than a naive customer support rep is an even more naive customer support ai.

"Social engineering is all you need"

Jeez, straight up amateur shit. Genuinely hard to believe.