How BSIMM improves security by letting developers compare security practices [Q&A]

Every organization believes that it's making its systems secure, but because they usually work in isolation from competitors and other businesses they have no way of knowing how they compare.

The Building Security In Maturity Model (BSIMM) aims to quantify security practices and present them in a measurable way to allow companies to compare their performance.

We spoke to Paco Hope Principal Security Evangelist of application security specialist Cigital to find out more about BSIMM and how it works.

BN: Where did the idea for BSIMM come from?

PH: Around seven years ago Cigital decided that rather than preaching to people about security we should take a step back and ask them what they do. So we went to big companies like Adobe and asked them how they spent their security budgets. We then developed a model that described all of the things they told us they were doing. Since then we’ve gone on to measure over 100 firms, however, the model only contains data from those who’ve been measured recently.

BN: How does it work?

PH: What we're trying to do is observational science around software security; we’re not interested in the desktop IT, anti-virus, network operations areas. We're talking about people who build software and apply it around the enterprise. Over time we've settled on around 112 activities that we've seen these people doing. From this we’ve built a scientific model that lets us score companies based on what they do. There’s no perfect score, it’s about what stuff companies are doing and whether that’s right for them. The activities are broken into five categories, including penetration testing, version control, risk and compliance.

BN: How is the information used?

PH: We publish a report every year which is all creative commons and out in the open. We do charge people to be measured so there is a commercial aspect, but anyone can look at the BSIMM report and self-assess. The scores for any particular firm are kept secret, we don’t share what we found at particular businesses.

BN: So it's a bit like an audit?

PH: No really, it's typically done over a few days of interviews, we'll sit down with the heads of development, support, operations and testing. We have a pretty good set of questions having done this hundreds of times, but it’s not an accounting audit where we ask to see deliverables that demonstrate what’s being done.

BN: What advantage does it have for businesses and does it scale to smaller and medium enterprises?

It overcomes the difficulty of getting to grips with security. You don't have to go it alone and make it up, you can learn from what other people have done. We have interviewed some smaller companies but below a certain size -- say you have a mobile apps company with only 20 people -- this doesn't make sense for them. But any decent sized enterprise, especially one that's pulling software from lots of vendors and building many different apps, BSIMM works really well.

BN: Do some sectors benefit more than others?

PH: We don't talk about specific sectors very much until we have enough people in that sector to show results without revealing details about a particular member. So, we have 20 plus financial services companies for example so we can show an average for that sector without revealing anything about a particular company.

Financial services, independent software vendors, consumer electronics and healthcare are the areas where we have most data. What surprised us at first was how similar some sectors are -- financial services and software vendors for example. It's only as we’ve started to get into other verticals that we've started to see more differences. Healthcare for instance tends to have a lower score than other industries, this is because some industries tend to think of themselves as hardware based when today it's not true. Car makers too these days are increasingly in the software business because a modern car is a big computer with a petrol-based power supply.

The advantage BSIMM brings is that companies adapting to a changing business model can learn from others who have already analyzed what they do.

BN: What could companies that have suffered major data breaches learn from the BSIMM model?

PH: That list of 112 activities hasn't been the same over the years; it's changed. A couple of years ago we stared to see an activity called 'simulating a software crisis' where mature firms will test their procedures for dealing with a problem like a software breach. They'll involve all of the relevant departments including legal, PR and so on and work through how they’d deal with the problem.

A good example is Saleforce.com which is a member of BSIMM, a couple of months ago a cross-site scripting bug was discovered in a Saleforce admin panel. 30 days after this was reported the whole thing was fixed and Saleforce acknowledged the problem, they delivered the kind of response you’d expect from a mature organization. It’s not about being perfect and producing perfectly secure software every time, it’s about knowing what to do and how to handle the problem. By contrast the recent TalkTalk breach was handled quite badly as they took a long time to get out a coherent message.

BN: Is there a cost involved?

PH: In a sense it's a bit like ISO9001, you can go and read the standard and do everything yourself, but at some point you need a third-party to come along and verify what you've done. So yes, there is a nominal cost if someone wants to be measured by us, but it's not expensive.

A further advantage of joining is that we've created a community that brings together people from companies who've been measured and they can compare notes and realize that often they have the same problems.

Image Credit: Manczurov / Shutterstock