Prelert cuts big data down to size for real-time analysis

Identifying complex anomalies that show up the presence of a persistent threat or pinpoint the cause of network and IT performance problems is the sort of thing big data analysis was made for.

The problem though is that big data is often just too big. This means it's usually necessary to transfer the information before doing any work on it which makes real-time analysis impossible and means valuable insights aren't available when they're most needed.

Prelert, a specialist in anomaly detection, has a solution in the form of its new Stats Reduce feature that can dramatically shrink data transfer sizes by up to 40 times. This cuts the time it takes to transfer data but retains its integrity to ensure accurate results.

It works by using the statistical aggregation functions already available in platforms like Splunk and Elasticsearch. The software has been rigorously tested against real world data sets and proven to deliver the same results whether working on the raw or aggregated data.

"Prelert is committed to providing the most accurate and robust insight into data in real-time, no matter how large or complex," says Stephen Dodson Ph.D, Prelert's CTO. "The scale of modern environments presents challenges that require careful selection of methods and techniques, and we built our technology to align with these environments from the start. With Stats Reduce, the aggregation techniques we developed allow massive volumes of data to be analyzed in a distributed manner, enabling real-time multidimensional anomaly detection on Big Data".

Stats Reduce is available from today in the latest version of Prelert's Anomaly Detective and will be rolled out to other big data platforms later this year. The company will be demonstrating it at the Splunk Worldwide Users' Conference this week in Las Vegas.

Photo Credit: Arkady/Shutterstock