Why enterprises are struggling with their digital transformation projects [Q&A]

We recently reported on research from value stream platform Digital.ai into digital transformation projects, which should make worrying reading for many enterprises.

The study showed that many organizations feel they're not getting the most from their transformation projects. We talked to Gaurav Rewari, chief technology officer at Digital.ai to find out why this is happening and what can be done about it.

BN: What's going wrong with digital transformation projects?

GR: On the one hand you have IDC showing that something in the order of $6.8 trillion is being spent on digital transformation, but on the other what you have is this mounting dissatisfaction with those investments. Something in the order of 70 percent of digital transformations are not delivering the expected profound consequences. And as many as 90 percent of digital transformation efforts are running into trouble.

Large enterprises have the inertia of big ships, it's hard to drive change in these companies. You have silos, you have organizational process silos, you have closed procurement… Companies get started with tools that really promoted a lot of agility in the development organization, but they often don't scale to the enterprise. What happens as a consequence is that you then have these data silos that emerge that prevent you from getting end to end visibility and understanding whether, in fact, much of the development organization is pulling in the same direction. You lose that crucial feedback loop if you will.

We have seen that endorsed by recent research from Gartner and Forrester that talks about this discipline called value stream management. VSM really addresses this pain point as the next logical step in the evolution of Agile and DevOps. This means working in smaller teams and making sure that Dev and Ops automate as much of the pipeline as possible. But ultimately, the ability to match the work activities in the development organization with the needs and business outcomes that your company desires, that's the missing link and that's what the value stream management discipline is all about.

BN: The figures on projects not delivering are quite scary. Do you think businesses have maybe rushed into these methodologies like Agile and DevOps without understanding the cultural changes that they need to make?

GR: That's a fabulous question. It's not just about the product, it's about thinking about it as a holistic practice. What I'm really trying to say is that there are people dimensions to it, there are process dimensions to the platform or product dimensions. And unless you have organizational support for industry management and structures you're not going to be successful due purely to software.

You really need to think about how you organize the company in value streams. So the idea is to get consensus, through a workshop so methodologies that allow us to say, if these business metrics are met, then this feature is deemed to be a successful one. This involves project managers bringing together the business and development stakeholders.

A lot of companies may not have actually thought about these disciplines in that holistic fashion, therefore a lot of their digital transformation initiatives are likely struggling.

BN: Does this mean these issues need to be addressed at a higher level in the organization?

GR: It does need support in the higher echelons, but ultimately I think it's also about understanding the success of others and looking at companies and individuals that are leading the charge in being able to successfully transform their businesses.

Often times companies end up optimizing their existing processes and existing business models, and they're effectively investing in the pursuit of operational excellence. Whereas the needs of the market in that particular industry might actually call for a sea change in their business model, a complete overhaul to realize the benefits.

BN: Is there a better way to measure the success of digital transformation?

GR: In simpler times if you were at a manufacturing plant and you had a sense of what your sales were like, you sort of planned your output to make sure that supply was a good match to demand. But in the world of intangibles like software it's much trickier. So your R&D metrics might be measuring things like, releases per unit of time, amount of code shipped or written per day, and that could be painting a very rosy picture. Whereas your business metrics around the customer journey, or new customer acquisitions might be presenting a more grim view.

What's really needed is a tight feedback loop between these business outcomes and what's actually happening in the digital world with the work that's happening in the ‘software factory’ if you will. That feedback loop brings business relevant application data into the same environment where development metrics and development data have been gathered and gives you a measure of visibility and of control over costs.

If you run in isolation you'll end up potentially optimizing one thing at the expense of the other. So you might drive down your lead time, only to increase your change requests. So, what you really need to do is to pull that data together into a unified form you can look across so you get an end to end visibility and insight.

BN: Can technology like AI and machine learning help deliver this visibility?

GR: The sheer volume of data that's being thrown up by these monitoring processes is far beyond the capacities of humans to analyze by themselves. So you need a high degree of automation to make sense of that data and then the surface insights and warnings and outliers from the deluge of information.

Also about 70 percent of IT budgets are still tied to keeping the lights on, and yet the business is clamoring for growth and innovation by the development team or IT team. The budget needs to move from that running the lights on site to the development side instead. The ability to do that in an intelligent fashion, using data as a guide to understand where true opportunities for cost optimization exist, is another place where AI technologies can help. The third driver for AI is just the competitive advantage you get from being able to get smarter decisions across the entire life cycle by all the participants in the mix.

Image credit: Olivier26/depositphotos.com