IBM Turns to Open Source Development

INTERVIEW Is open source changing the way that software is made? It is at IBM. BetaNews sat down with Doug Heintzman, IBM Software Group's VP of Strategy and Technology, to discuss the adoption of a hybrid development model called Community Source that combines the best elements of the open source model with decades of IBM programming practice.

This componentization, says IBM, will liberate the creativity of programmers, drive efficiency, and bring products to the marketplace at a faster rate then was previously possible - avoiding a top down approach that IBM says could make Microsoft's Longhorn obsolete upon arrival.

BetaNews: Let's begin with an introduction to Community Source for our readers.

Doug Heintzman: The quick gist of it is really quite simple. We run a very large software development company and we have laboratories around the world. And due to a number of technological factors as well as some efficiency enablers like the Internet, we have decided to move to a new development methodology.

We are systematically decomposing our technologies into a number of components and a lot more reuse than was previous possible. Because of that strategic decision we need to move to a development system that allows us much greater transparency, and a much greater awareness and cross pollination of expertise, ideas and requirements between all of the various different laboratories.

Increasingly our products are assembled from components instead of architected from the top down. We basically leveraged our rather extensive experience with the open source communities and we have borrowed many of their philosophies, strategies, tools and a lot of their culture to transform IBM's internal development practices to support global component development and promote collaboration and reuse of technology.

That's the high level synopsis.

BetaNews: What inspired IBM to begin the Community Source program?

Doug Heintzman: The community source approach at IBM is a means to an end. We believe in an Internet connected world with the business requirements that the on-demand era of information technology is suggesting. There is going to be an important shift and to deliver technology that addresses that shift in both customer requirements as well as our technological capacity, we have decided to systematically componentize and modularize our software. That is allowing us to get to the market much more quickly to address these requirements in a much more time and cost effective manner.

So with that recognition that this is the way we are going to develop software going forward it is clear to us that the way we traditionally develop applications is sub-optimal to achieve this goal. It's a very ambitious goal; no one has tried to do it on the scale and scope that we are doing it.

We very much believe that the software industry is moving through the same kind of componentization transition that many other industries ranging from the automotive industry to the disk drive industry and chip industry have all gone through. And the companies that emerge from this transition and have successfully broken their products down into sub-assemblies to reusable components will have tremendous advantage in the marketplace. So that's the driving motivator. Community Source is a way to get there.

BN: Would you consider it to be managed code or something different? To elaborate, Microsoft is attempting to push its developers to use modules of managed code within Visual Studio to create more reliable Windows applications more efficiently.

Heintzman: I think that managed code is a way of referring to a structured component and we've got a number of names for them ourselves. We believe a few things: First of all, we have a systematic architectural view of our entire product portfolio. And we have a group of people dedicated to identifying what groups of technologies should be componentized and providing the heavy architectural framework to the development team so that they can do that componentization more effectively.

From that sense there is a managed code approach in that there is a group of people whose full time job it is to worry about what ought be componentized, how should that componentization be done, what standards need to be adhered to.

Now that being said, there is a second part of this, and this is really borrowing from the culture of the open source community. There is a very important role in a software company like IBM for top down managed code architecture and all that kind of good stuff. But there's also a tremendous amount of potential innovation that is locked up in the heads of the front line programmers and we try to liberate that creativity and the innovative potential of all of those people.

We look at the open source communities out there and we are witnessing this kind of fascinating bottoms up grassroots innovation where great people have an idea and collaborate with other people and get together to make those ideas into something real. That's a very exciting phenomenon. So certainly we have a structured approach to community source, but we also have an eye to promoting this bottom up, collaborative, creative process. This is part of borrowing the culture from the open source community.

BN: Speaking of scale and scope, when will Community Source be fully implemented companywide?

Heintzman: It's already been implemented companywide. We have a hundred projects that are currently up using the Community Source. We have over 2,000 users that are currently registered and using the Community Source tools and processes. There are nine or so new projects that have been registered just in February, which is the last time I saw the numbers on the progress we are making.

We actually have 34 or so projects that we call "in active production." They are not just kind of hobbies; these are active development projects that run directly into our product line.

BN: Is Community Source phased in throughout every product group, or is it targeted more towards improving core IBM software technologies that are more modular in nature?

Heintzman: Good question. There are two ways to answer that. Certainly a lot of our componentization efforts, and subsequently the tooling to support those componentization efforts, are focused on the core enabling. But the real key here is that we are looking at a lot of the new emerging applications especially in some of the development areas where we are seeing the first real projects. We don't want to be overly disruptive; we want to phase these things in responsibly.

A lot of projects in Rational are coming online using these kinds of techniques. And of course the Rational people have exposure to the Eclipse open source world. We are learning a lot from that experience. So it's not as systematic; it's the first area to really go at it. The first area that the open source community went out and really attacked was the development phase, so in many ways I suppose it makes some sense for that to be the place where we actually start ourselves. But increasingly, just like the open source community has gone into other areas, that is also true of the community source efforts inside of IBM.

BN:You mentioned grassroots Doug. Is it conceivable that IBM developers could be inspired to contribute projects they are not assigned to in their own spare time?

Heintzman: We would be very happy for programmers to program in their spare time, assuming of course, they have an appropriate degree of life/work balance and all of that. You actually raise a very interesting question. One of the most important reasons we are doing this is because of transparency issues.

It's because the code, the design specs, the documentation are really out in the open that any laboratory working on any product properly registered with the proper authority to do so can go in and make comments. We may have interesting techniques that we have developed in one group that they recognize have not been implemented in the product that they are consuming.

We want them to go out and say, "You know what guys we figured out how to do this six months ago, try it this way." Not coaching them, but to help them along. At the end of the day we want the whole engine to be running as efficiently as possible, and if that means different people working on projects that may need help and need work then so be it.

BN: Moving down another path in this discussion - what is the backend to Community Source? Is it similar to the SourceForge community Web site?

Heintzman: It is similar; in fact we use a lot of the same tools that a lot of these projects use. We have tools that store the actual source logic using repositories very much like SourceForge, documentation and specs, news and bulletins, patches and fixes, and educational material – just the way you would see it in SourceForge.

We have a bunch of other technologies and community tools that we surround it with. In many ways, because our developers aren't just working on one particular project, we have recently implemented a developers' intranet that provides a centralized umbrella for best practices and new tools that are available.

There's a broader community set that in many ways is a superset of what you would see in something just like SourceForge. It's a more comprehensive approach, but we have the resources and the requirement to do that. We also want high degrees of mobility of our programming staff.

That's a degree of discipline that you really don't see in the open source world per se. It's not that we are just copying the open source world, we are borrowing its culture and we are applying our own twist on it to be sure that it runs the way that we need it to run.

BN: So IBM feels it is taking the best of the open source culture. How will IBM keep its source code secure and guarantee the code's safety and integrity inside of this system?