CrowdStrike -- what went wrong?

This time last week businesses around the world were rocked by major disruption as a faulty update to the CrowdStrike security software brought down Windows systems.

The company has now issued a preliminary report into the incident which reveals that a 'Rapid Response Content' configuration update caused the problem.

This content is used to perform behavioral pattern matching in the security engine and is stored in a proprietary binary file. Last Friday's update followed from the introduction of a new IPC (Inter Process Communication) Template Type to detect novel attack techniques that abuse Named Pipes -- this happened back in February.

On July 19, two additional IPC Template Instances were deployed. Due to a bug in the Content Validator, one of the two Template Instances passed validation despite containing problematic content data.

George Kurtz, CrowdStrike founder and CEO, said following the problem, "I want to sincerely apologize directly to all of you for the outage. All of CrowdStrike understands the gravity and impact of the situation. We quickly identified the issue and deployed a fix, allowing us to focus diligently on restoring customer systems as our highest priority."

In order to prevent a similar issue arising again CrowdStrike is introducing additional testing and adding more validation checks to the Content Validator for Rapid Response Content.

Richard Ford, CTO at Integrity360, says:

There are still some questions that need to be answered, and I'm sure will come out once the full RCA is released. One of the core questions is not how the Content Validator missed the invalid file but how did that file become invalid in the first place?

As we get closer to the end of this incident, I think it's clear that we will look back on it, and the way it was handled by CrowdStrike, as an example of what good can look like in the face of adversity. They've been transparent, they've quickly implemented the immediate fix and identified the long-term solution to prevent it from happening again and they actively engaged with customer and partners to recover. There are valuable lessons to learn and implement across the industry.

You can read CrowdStrike's preliminary report on the company's site.

Image credit: rafapress/depositphotos.com