Attacks on GenAI see sensitive data leaked

New research reveals that 90 percent of successful attacks against GenAI have resulted in the leakage of sensitive data.

The report from Pillar Security, based on real-world analysis of more than 2,000 AI applications, shows 20 percent of jailbreak attack attempts successfully bypassed GenAI application guardrails and adversaries needed an average of just 42 seconds to execute an attack.

Attackers needed only five interactions on average with GenAI applications to complete a successful attack. The attacks exploited vulnerabilities at every stage of interaction with GenAI systems, underscoring the critical need for comprehensive security measures.

The analyzed attacks also reveal a clear increase in both the frequency and complexity of prompt injection attacks, with users employing more sophisticated techniques and making persistent attempts to bypass safeguards as time progresses.

"The widespread adoption of GenAI in organizations has opened a new frontier in cybersecurity," says Dor Sarig, CEO and co-founder of Pillar Security. "Our report goes beyond theoretical risks and, for the first time, shines a light on the actual attacks occurring in the wild, offering organizations actionable insights to fortify their GenAI security posture."

Primary attacker motivations are stealing sensitive data, proprietary business information and PII and circumventing content filters to produce disinformation, hate speech, phishing messages and malicious code, among others.

"As we move towards AI agents capable of performing complex tasks and making decisions, the security landscape becomes increasingly complex," adds Sarig. "Organizations must prepare for a surge in AI-targeted attacks by implementing tailored red-teaming exercises and adopting a 'secure by design' approach in their GenAI development process."

You can get the full report from the Pillar site.

Image credit: ValeryBrozhinsky/depositphotos.com