A Meta AI chatbot designed to enhance security and user support on Facebook and Instagram has been reportedly exploited by malicious users to gain access to high-profile accounts, highlighting the security concerns that accompany the rapid integration of AI into critical functions of digital platforms.
The incident highlights the challenges organizations face when developing AI systems, particularly when they interact with sensitive data and user identification processes.
Following the developments, we present the commentary of Steve Giguere, Principal AI Security Advocate, Check Point, on the risks, challenges and best practices for the safe use of artificial intelligence.
Statement from Steve Giguere, Principal AI Security Advocate at Check Point:
“This incident looks less like a classic prompt injection or “AI jailbreak” attack and more like a failure in how trust and privileges were granted to an AI-based support process. The underlying issue is not necessarily that the AI overrode its instructions or was manipulated by a smart prompt. Rather, it appears that the AI had the ability to initiate or facilitate sensitive account recovery actions without sufficient independent verification. In other words, the security weakness may lie in the workflow itself, not the model. This distinction is critical, as many companies focus on protecting against prompt injection, jailbreaks, and other model-level attacks. These are significant risks, but some of the most serious incidents involve AI systems accessing critical actions without adequate safeguards, human oversight, or authorization checks. What makes such incidents so worrisome is that the root cause may not even be a jailbreak.”
“AI doesn’t need to be compromised when given powers that exceed existing security measures. As organizations increasingly adopt agentic systems, security teams need to evaluate not only what an AI can say, but also what it can do. This case highlights a critical security issue: not necessarily the breach of the AI itself, but how it is integrated into sensitive processes. The AI appears to have been able to initiate account recovery actions without adequate authentication, indicating a weakness in the workflow. Organizations need to ensure that AI systems do not have access to critical functions without clear security controls, human oversight, and strict authorization checks.”
Although the press releases will range from very select to rare, I said I'd pass...because sometimes the editors hide.
