DevOps -- sorting the leaders from the laggards
What characterizes companies that do DevOps well compared to those that don't? A new study by Forrester for software testing company Tricentis aims to find out.
The research looked at 94 quality metrics that were classified into four categories. Key findings are that an understanding of business risk is the most important determining factor of DevOps and Agile maturity.
DevOps experts focus primarily on contextual metrics (e.g. requirements coverage) while others focused on 'counting' metrics (e.g. number of tests). Also experts are more likely to measure the user experience across an end-to-end transaction while others rely on application-specific or team-specific metrics.
Among other traits of the most successful DevOps and Agile practitioners are their transforming of software testing into Continuous Testing by adopting core practices, such as automating end-to-end functional testing and integrating testers into cross-functional teams. Leaders are almost twice as likely to consider automating the software quality process to be a 'critical business differentiator' (53 percent compared to 27 percent). They are also significantly more likely (by 23 percentage points) to have high levels of automation for key testing and QA processes.
"Organizations adopt DevOps to accelerate the delivery of innovative software that createspositive user experiences," says Clark Golestani president, emerging businesses and global CIO (retired) of healthcare technology group Merck. "If you start accelerating delivery without also rethinking your approach to quality, you will end up rapidly delivering updates that drive your customers to competitors. Automated, continuous testing is the gate that allows you to protect your organization from significant risk as you innovate at the speed modern business demands."
Most firms (80 percent) believe they deliver projects within acceptable business risk, but fewer than a quarter say that their QA and testing processes completely cover risk. Not only that, just 15 percent say that their test suites reliably provide a good indication of business risk. Agile and DevOps experts measured risk metrics significantly more frequently than the non-experts did.
Among other findings are that high-level decision makers are vastly overestimating the maturity of their companies' Agile and DevOps practices. Firms that rely on manual testing are also more than twice as likely to report that testing presents a bottleneck (59 percent), compared to those with more automated quality processes (26 percent).
You can find out more in the full report, available from the Tricentis website.
Image credit: tkemot / depositphotos.com