IBM Turns to Open Source Development

BetaNews: So IBM feels it is taking the best of the open source culture. How will IBM keep its source code secure and guarantee the code's safety and integrity inside of this system?

Heintzman: Interesting question. You know that question cuts two ways. There is considerable merit in open source when you've got many eyeballs you can drive out bugs and security holes and flaws and fix them more efficiently. I think that that's true. I think that it is also true that commercial software companies like IBM have very rigid quality control best practices, reviews and scans - disciplines that they have developed over decades.

We are sharing with the open source community to try to help them mature. So we find ourselves borrowing, as you say, the best of this new culture and merging it with a lot of the lessons, tools, techniques and practices that we have developed over many, many years to come up with this productive hybrid. That's what Community Source is really all about.

BN: Could Community Source assist in breaking down language barriers between IBM's locations throughout the world since programming code is a universal language? IBM is a company with many laboratories outside of the United States so there must be linguistic challenges.

Heintzman: Absolutely. We've got a whole bunch of laboratories on every corner of the planet. I've never really thought about it. Certainly in all of the various different programming reviews that we have looked at, linguistic diversity has never been one of our prohibitors that I am aware of.

Will this help? Sure. Is it "the" reason to do it? I would say it's probably a "nice to have."

BN: So is linguistic diversity now somewhat easier to have than in the past?

Heintzman: I wouldn't quite phrase it that way. The frank reality is that in the world of coding there really is kind of one language. Programmers work with certain programming tools and we build our reference materials and our documentations in reference languages and then we massively translate things. Our laboratories have been working from common linguistic code bases for ages. It's too complex to try to integrate all of that componentry from different laboratories unless you have some kind of normalization.

BN:In a press release IBM said, "Community Source has made it easier for IBM developers to create and trade Lego-like components of code that can be plucked from a library, quickly assembled and re-used later."

Considering how many ongoing projects IBM has, how will these 'legos' be organized? We touched on this briefly when we discussed the full time IBM employees that are dedicated to making sure that these modules of code are used appropriately.

Heintzman:We've got a division whose role is to work with the core architects for all the software groups to get the big, big, big picture. They drill down what the big platform picture looks like, they identify targets that would lend themselves to componentization and high degrees of reuse.

This is a journey for us. We are starting the modules that will give us the best payback and we will move to other secondary modules. They sit down and kind of look from an architectural standpoint and they define lists. Then they go to the laboratories that have responsibilities in those areas and they sit down and say, "This is what it means to be defined as a component. This is how we are going to normalize things to allow for high degrees of reuse."

That's an interesting dialog because when you take on the responsibility of owning a component, you are actually taking on a higher degree of financial burden. It is more expensive to develop to the high degree of specification which components require and to subsequently support them and provide support to all of the groups that consume them.

Be that as it may, we have made a decision that says, "This is our strategic direction," and each of the laboratories understand that they have responsibilities to live up to, and yes they may be spending more money on one particular component, but they will also get benefits because they are now consuming components that they don't have to fund themselves.

BN: IBM states that development is 30 percent faster using an open source model: I am just curious do you have any empirical data to back that up and how did you come to that conclusion?

Heintzman: That's a good question, and unfortunately I am not the most skilled person to answer the specifics of the methodology behind those kinds of numbers.

At the high level there are all different kinds of code and there are all kinds of ways of measuring the productivity of programming. Say you are building a product and you need an administration console. The fact that there is one in the library that you can plug in is almost infinitely productive relative to hiring a bunch of coders and going out and building the thing. So how much extra productivity do you get from that now? A lot.

If you use these Community Source approaches, the more quickly and more cost effectively you get to the right answer. You could get that console built by taking those programmers and locking them in a room and then off they go, but will they produce a piece of code at the end of the day that really is highly productive for reuse across the company?

There are all sorts of ways of measuring it and 30 percent is kind of our rough averaging of how quickly the projects that we have brought into production have succeeded versus the percentage of them that have been deemed failures, and how many developers its was required to code a particular function versus how we have done it before. We actually do have specific metric we measure for everything and if you take a step back and put your thumb out we are getting about a 30 percent improvement.

BN: It struck me that IBM said, "at companies like Microsoft (just think of Longhorn, taking years to deliver at the risk of obsolescence upon delivery)." That's a pretty bold statement. How can IBM say that considering IBM itself has for a very long time used the same hierarchical approach?

Heintzman: I think there are actually two issues that we are commingling here, so let's make them a little distinct. So one of the issues is code that is highly integrated code that was built originally for a specific purpose. These code bases are very expensive to modernize, to add functionality to develop and support.

Certainly the Microsoft Windows platform, and we've witnessed the continual delays of the platform, is an incredibly complex engineering problem they have. They've got millions and millions of lines of code and they are all spaghetti'ed together and as soon as they go and change or fix something it can ripple all the way through the code.

We had exactly the same challenges in our Lotus Notes platform. It is a very mature, very complex highly functional platform, but it is very expensive to develop it and to maintain it. And so what we've decided to do is work to rip the Notes code out and replace it with components to kind of systematically move towards componentization.

Our hope is -- and our early experience with it is actually very encouraging -- that we will be able to bring products to the marketplace very, very quickly. If you look at the rate of innovation, feature addition and quality enhancement in something like our portal platform, they are versioning at unbelievable rates. It went from almost nothing to an industry leading incredibly comprehensive platform very quickly.

You couldn't do that with the same rate of innovation using the highly structured models of the code bases. So componentization gives you some extra flexibility there.

The second question: Now, if you buy the first argument that says that integrated platforms become huge and go into multi-million lines of code, the engineering problem becomes incredibly difficult to scale. The question is what kind of programming culture do you want to put in place using this componentization approach.

BN:So has Microsoft put itself at a disadvantage?

Heintzman: Microsoft has a unique challenge. In their position in marketplace and dealing with the compatibility issues, they have an enormously complex engineering problem that with every release just gets worse and worse and worse. At some point that system really breaks down and you get diminishing returns on your investment.

We kind of took a review and projected out where our various different platforms were going years ago and we projected that they were going to hit a knee in the curve that made them incredibly unproductive because of the engineering complexity. So we decided on this new development strategy and this new philosophy, and we turned to Community Source as one of the enabling technologies to allow us to obtain engineering efficiencies.

BN:One last question Doug: What is the future of the program?

Heintzman: You don't do these things over night. There are a lot of good things in the open source development process, the community development and the culture. There are also some deficiencies with that process and having some architectural discipline to make sure that you have the most efficient deployment of resource makes very good sense.

We are really trying to harvest the programming culture and practices that we have honed, developed and optimized over many, many years, along with some of these new techniques that we have learned through our exposure to the open source communities and have tailored to our purposes internally.

We will merge these things, and certain projects that have been done in highly traditional ways will adopt some of the practices. New projects that grew up on these new processes will also adapt to many of the traditional disciplines that we have matured over the years. We hope we end up with the appropriate balance of the best of both of these worlds.