Top websites struggle to guard against sophisticated bot attacks

A report released today shows that, while an average of 16 percent of websites across all industries can thwart simple bot attacks, only five percent are able to properly protect against sophisticated attacks.

The study from bot detection specialist Distil Networks, in conjunction with the Online Trust Alliance (OTA), evaluated the top 1,000 websites in retail, banking, consumer services, government, news media, internet service providers and OTA members.

The report divides bots into four categories:

• Sophisticated Bots -- coming in slowly from dozens of IP addresses, using browser automation tools that can hold cookies and maintain state
• Moderate Bots -- using normal browser user agents and headers, coming in slowly from one IP
• Simple Bots -- non-browser user agents and headers, coming in fast from one IP
• Crude Bots -- basic scripts that behave like a bot, coming fast from one IP address

The findings show that while most industries tested can adequately protect against crude bots, they struggle to effectively block the simple, moderate, and sophisticated ones. For example, federal websites block 22 percent of simple bots, but only protect against one percent of sophisticated bots, performing below any other industry tested.

Despite poor performance, this year's findings reveal a marked improvement from Distil's 2016 study, which found that websites tested could protect against only 0.7 percent of sophisticated bots. Such improvement can be attributed to gradual movement toward greater awareness and adoption of more advanced bot detection and mitigation solutions.

Looked at by industry, banks and ISPs are most effective at detecting sophisticated bots, followed by retailers. Against simple and moderate bots though banks’ detection rates lag behind those of retailers.

"While top websites do a better job protecting against simple bots, they continue to miss the mark in more sophisticated bots that can mimic human behavior," says Rami Essaid, CEO and co-founder of Distil Networks. "Our annual Bad Bot Report found that 75 percent of today’s bad bots are advanced persistent bots that can either load JavaScript, hold onto cookies, and load up external resources, or randomize their IP address, headers and user agents. These new findings show that no industry is immune to such attacks and, along with the OTA, we are committed to raising awareness about the risks posed by bad bots."

You can read more about the findings of the report on the Distil website and there will be an OTA webinar to discuss the findings on June 29th.

Image credit: Gajus-Images / depositphotos.com