AI platform better predicts cyber attacks thanks to human experts

Researchers from MIT’s Computer Science and Artificial Intelligence Laboratory (CSAIL) have collaborated alongside the machine learning startup PatternEx to demonstrate how an artificial intelligence platform that makes use of continuous input from human experts would be able to predict cyber attacks better than the systems that exist today.

CSAIL and PatternEx are calling their new AI platform AI² due to how it combines the intuition of analysts with artificial intelligence.

Previously, "analyst-driven solutions" relied on the rules created by security experts which resulted in attacks that did not correspond to their rules slipping through the cracks. The machine learning approach to cyber-security relies on "anomaly detection" to discover possible attacks but often results in false positives. CSAIL and PatternEx created AI² by combining machine learning’s ability to detect anomalies with the knowledge of analysts to flag possible threats.

So far the new platform has been able to detect 85 percent of attacks and has reduced the number of false positives by a factor of five. To test AI², CSAIL and PatternEx used 3.6 billion pieces of data called "log lines" that took millions of users a period of three months to generate.

In order to predict attacks, the platform searches through data and detects suspicious activity using unsupervised machine learning. AI² then presents its findings to human analysts that take the necessary time to confirm which attacks are real and which are false positives. This feedback is then put back into the platforms models and is used to comb through the next set of data it is fed.

The CSAIL research scientist Kalyan Veeramachaneni who helped develop AI² likens the platform to a virtual analyst. Through its work the platform is able to generate new models which the human analysts then refine helping to improve its detection rates in a significant way over a short amount of time.

What sets AI² apart from other machine learning systems is that it combines three different unsupervised-learning methods to reach its top events which are then labelled by analysts. This allows it to build a supervised model that is constantly being refined which has lead to its developers referring to it as a "continuous active learning system".

AI² is a huge breakthrough that shows what can be accomplished when human experts and AI work together to solve problems.

Published under license from ITProPortal.com, a Net Communities Ltd Publication. All rights reserved.

Image Credit: Mopic / Shutterstock