Most AI detectors can't tell if a phishing email was written by a bot

The latest Phishing Threat Trends Report from Egress, based on data from its Egress Defend email security tool, reveals that nearly three-quarters of AI detectors can't tell if a phishing email has been written by a chatbot.

Because they utilize large language models (LLMs), the accuracy of most detector tools increases with longer sample sizes, often requiring a minimum of 250 characters to work. With 44.9 percent of phishing emails not meeting the 250-character limit, and a further 26.5 percent falling below 500, currently AI detectors either won't work reliably or won't work at all on 71.4 percent of attacks.

"Without a doubt chatbots or large language models lower the barrier for entry to cybercrime, making it possible to create well-written phishing campaigns and generate malware that less capable coders could not produce alone," says Jack Chapman, VP of threat intelligence at Egress. "However, one of the most concerning, but least talked about applications of LLMs is reconnaissance for highly targeted attacks. Within seconds a chatbot can scrape the internet for open-source information about a chosen target that can be leveraged as a pretext for social engineering campaigns, which are growing increasingly common. I'm often asked if LLM really changes the game, but ultimately it comes down to the defense you have in place. If you're relying on traditional perimeter detection that uses signature-based and reputation-based detection, then you urgently need to evaluate integrated cloud email security solutions that don't rely on definition libraries and domain checks to determine whether an email is legitimate or not!"

The report also shows the proportion of phishing emails employing obfuscation techniques has jumped by 24.4 percent in 2023 to 55.2 percent. Obfuscation enables cybercriminals to hide their attacks from certain detection mechanisms. Egress finds that almost half (47 percent) of phishing emails that use obfuscation contain two layers to increase the chances of bypassing email security defenses to ensure successful delivery to the target recipient. Less than a third (31 percent) use only one technique. HTML smuggling has proven the most popular obfuscation technique, accounting for 34 percent of instances.

More phishing emails are getting through traditional perimeter detection too, so while overall volume hasn't increased attacks are increasing in sophistication and cybercriminals use a multitude of tactics to successfully get through perimeter email security. The percentage of emails that get through Microsoft defenses has increased by 25 percent from 2022 to 2023. Likewise, the percentage of emails that get through secure email gateways (SEGs) increased by 29 percent from 2022 to 2023.

Additionally, there's been an 11 percent increase in phishing attacks sent from compromised accounts in 2023. Compromised accounts are trusted domains, so these attacks usually get through traditional perimeter detection. Almost half (47.7 percent) of the phishing attacks that Microsoft's detection missed were sent from compromised accounts. The most common type of payload is phishing links to websites (45 percent), up from 35 percent in 2022. And all payloads bypassed signature-based detection to some degree.

The full report is available from the Egress site.

Image credit: limbi007/depositphotos.com