How to measure the value of cybersecurity [Q&A]

With high profile cyber attacks and data breaches continuing to make the news, security is at the top of the priority list for businesses.

But how do you know that the resources you put into cybersecurity are providing a good return on the investment? We spoke with Oliver Rochford, security evangelist at Securonix to find out.

BN: Historically, what are the most common ways that companies have measured the value of their security programs and are they effective?

OR: Historically, companies have used low-level operational metrics such as KPIs and key risk indicators to measure security effectiveness. But using these types of metrics creates a misplaced sense of control, because they often lack relevance and actionability. In reality, they're measuring operational performance instead of security effectiveness or risk. There is also often a disconnect between the numbers and true value. For example, is it value if we process more false positives and alerts?

The biggest issue from the perspective of the board however with this method of measuring the value of your security program is that it lacks business relevance.

While we can say we're spending X amount on our security program, and that is typical for an enterprise of our size in this industry, it doesn't express how well we'll respond to ransomware or where we should invest our limited security budget for the most reduction in risk.

This method doesn't provide companies with the information they need to understand how to best reduce liability. And without having a baseline determining what is good or bad, the KPIs that are typically used are just numbers and cannot give a great indication of the true value of the program.

BN: Why is it so difficult to measure the value of security programs, whether or not a company has been breached?

OR: The first step in being able to properly measure the value of security starts with a change in the perceived value of security. The industry needs to move away from the idea that security is a cost and transition to the idea that it is an enabler for business growth and transformation. This will help organizations to move away from tying security value solely to how much money is spent, and instead to focus on how security enables the business to operate safely and securely.

The second step is moving away from the belief that the success of their program revolves only around the probability of an organization getting breached. The reality is that companies can't be 100 percent secure. This is not the concern because we know organizations will be breached. Instead, organizations should be asking themselves how they can achieve resilience -- which includes recovering from a breach, ensuring data is not exfiltrated when breached, and other processes that go beyond just preventing the attacker getting in. Prevention still has a place, but not everything is preventable, and only a diversified strategy based on prevention, detection, response and recovery will be effective and more importantly due to diminishing returns, affordable. It's really about understanding that security is a series of rounds, rather than a single event to achieve resilience. Instead of focusing on getting breached or not, we have to measure security, and value it accordingly. Security professionals can't get worked up when losing a round or two, and must focus on winning the match, or sometimes even just aiming for a draw if we’re working with a smaller budget or playing catch-up.

The solution lies in making security objectives measurable and actionable. If you want to measure your effectiveness against phishing for example, you shouldn’t wait for an attack. You can run phishing simulations, measure the percentage of employees that failed, and work to reduce the percent that fail. This also has the added advantage of obtaining an idea of how great the risk is of a phishing attack succeeding, and where to deploy additional controls to compensate.

BN: How has the adoption of new technologies in security operations -- like machine learning and behavior analytics -- impacted organizations' ability to measure the value of their security programs?

OR: On the one hand, we have seen rapid advances in technology to allow us to collect, analyze and model more data than ever. At the same time, we also generate more data than ever, and a lot of the data is not structured or easily processed. When it comes down to it, measurement is fundamentally a data problem, and data is the problem. Because we are collecting a lot of data that is not of a high quality, we can't always rely on what the data tells us. The average salary of security analysts, or what the average ransomware attack costs, are great examples where you can find hundreds of different answers, but most of them are based off of too few data points and on unfounded assumptions.

We also collect a lot of data without understanding what questions we want it to answer, and usually end up getting so overwhelmed that we use none of it. Today we have better tools to measure and quantify security, but we're not there yet in terms of how to best use them.

A part of the issue stems from the fact that as an industry, we still haven't reached the maturity where we have universally agreed on standards around metrics, although there are some great green shoots, for example the FAIR framework. I am expecting progress in quantifying security over the next few years, but right now it’s still more art than science. And that is hard to codify in a machine learning model.

BN: How does the overall 'immaturity' of enterprise security programs contribute to challenges in measuring security value?

OR: At a low maturity there will be few formalized processes to measure, and little tooling to collect and analyze data, especially in volume and over time. At that point, you have to choose your battles carefully, and identify a few key areas to begin quantifying, preferably based on prioritization and increasing your maturity. It's possible to do this using a spreadsheet to start, but you will need better tools quickly. Of course, if you happen to begin with a blank slate or inherit a low maturity, you should plan the quantification into your roadmap and ensure you mature your ability to measure in sync with maturing the program.

At greater maturity, the risk lies in getting what you wish for, and being inundated with too much data, especially lots of noise and only weak signals. Once again, the solution is to decide on a solid set of metrics to measure and to optimize the quantification methodologies for these, before adding the next set.

BN: What are some best practices for CISOs and security teams to measure the value of their security programs more effectively?

OR: The number one thing for CISOs and security teams when it comes to measuring the value of their security program is to understand the business goals and objectives to develop a risk profile and determine risk appetite and tolerance. This means to truly understand the company strategy. Effectively reducing risk with limited resources means being targeted and investing where it matters most. It needs to be clear how something that is measured fits into the security strategy, and how it can be improved.

Gartner says that your metrics should be:

• Consistent over time
• Adequate and controllable
• Reasonable in terms of balancing risk versus the business need
• Effective in producing the desired level of security

That's a solid checklist for ensuring your metrics are sensible and usable.

The CISO role is evolving and it's becoming increasingly more important for those in the role to understand and tie back risks and options to the broader business. Therefore, the most important thing is for CISOs to have a strong understanding of how the business understands its risks.

Image credit: Norebbo/Shutterstock