Interview: The Future in Grid Computing

INTERVIEW Computing grids are software engines that pool together and manage resources from isolated systems to form a new type of low-cost supercomputer. In spite of their usefulness, grids remained the plaything of researchers for many years. But now, in 2005, grids have finally come of age and are becoming increasingly commercialized.

Sun Microsystems recently unveiled a new grid computing offering that promises to make purchasing computer time over a network as easy as buying electricity and water. Even Microsoft is said to be investing in grids and Sony has grid-enabled its PlayStation 3 for movie-like graphics.

As interest in these distributed technologies grow, so does the probability for disinformation. With that in mind, BetaNews sat down with some of the world's leading grid guru's, Dr. Ian Foster and Steve Tuecke, to set the record straight and divorce grid hype from grid reality.

BetaNews: Since we last spoke in 2001, what significant developments have there been in the commercialization of grid technologies?

Dr. Ian Foster: Back then we were just seeing earlier interest in grid technologies from companies like IBM etc. Since then we have seen tremendous growth and enthusiasm. And a lot of things are being labeled as grid that perhaps one could argue they are not. Perhaps they are more, in some cases, computing cluster management solutions, but also some substantial early deployments in the industry from companies like IBM and Sun, and others like HP and so forth.

Then more recently we have seen Univa being created, which I am involved as founder and advisor. I primarily focus on the open source software side. And that is significant because it represents a step forward on the transfer to the commercial side of this open source grid technology, which is going to be important as a means of building the standards-based systems that are required for broad deployment.

The other thing that you may have come across just recently is the Globus Consortium and this is a set of four major participants: HP, Intel, IBM and Sun; focused on advanced commercial use of Globus software.

BetaNews: Speaking of IBM, it claims to have grid-enabled much of its product line.

Foster: I think things are not quite as simple as that. IBM has a product called the IBM Grid Toolbox, which is basically Globus with some IBM enhancements or add-ons. They have done a fair bit of business based on that, using that technology to federate storage and computing resources within various industries. At the same time, when they say they grid-enabled WebSphere, for example, what they mean more is that they modified WebSphere to run on a cluster, I believe. So I do not think that the grid-enabled WebSphere at present is using Globus technologies.

BetaNews: Is that what you meant about mislabeling clustering as grid?

Foster: The key thing we are trying to achieve with grid is federating resources, breaking down the silo boundaries that are making computing more expensive or less reliable when it shouldn't be. And there are of course a number of ways of skinning the cat. So at the moment, in the absence of some of the necessary standards, first of all you see low ambition solutions that are already focused on clusters; other solutions that may extend across clusters that are more proprietary in nature; and lastly where we are moving -- with recent work on standards with the open source software basis -- towards larger scale federation and standards base federation. And those are the two things that the Globus software is about.

BetaNews: Do you consider Sun's grid engine to be an example of a proprietary solution?

Foster: Yes, the Sun grid engine is a system that can be used to manage a single cluster. I think it can be used to federate clusters that are running Sun grid engine software. It's a proprietary solution in the sense that it won't interoperate with other resource managers.

But you know it is a grid technology, and I think Sun would like to take it, and where we'd like to see it go, is toward something that would use open protocols probably based on Globus implementations as a basis for federating clusters of different sorts or computing resources of different sorts. So again, it's a technology that is useful today in sort of a closed environment that can become something more powerful as it integrates with the right open protocols.

BetaNews: Let's move on to Globus. Tell us where you are at.

Foster: So, when we would have talked last, Globus toolkit version 2 had come out probably about that time and that was the first version to really get large-scale deployment in the science space. Since then we have made a lot of progress in quite a few dimensions and one primary one is integration with Web services technologies.

Globus Toolkit version 4, which will come out in April, makes heavy use of web services technologies and implements a set of specifications that are relevant to distributed resource management and federation coming out of places like OASIS and W3C and the Global Grid Forum. And so in that sense it represents an alignment -- a convergence -- of what is going on in the Web service space and I think that's pretty significant.

BetaNews: What Web service technologies is Globus supporting?

Foster: What we provide is primarily an implementation of Web services standards to allow people to build services, and the primary goal is also for us to provide a set of pre-defined services that allow you to use Web services protocols to interact to request the allocation of compute resources, the creation of computational services and moving the data from one place to another and so forth.

The actual hosting environment on which we implement our services today is primarily Apache access, we don't currently have a .NET implementation but that is something that could be provided.

BetaNews: Will the upcoming Globus toolkit include pre-coded modules to build grid-based solutions?

Foster: The Globus toolkit version 4 itself provides a set of modules that address various issues that arise when you are building systems that federate computing resources. So, these include components that implement WS-Security standards, others that provide resource management services, data management services and so forth.

The Globus consortium is going to pull resources to add additional modules to the open source code base, perhaps in some cases they may devote resources to porting Globus components to new platforms, perhaps to integrate with new classes of enterprise software. There is a range of things that we would expect them to do that they haven't yet announced such as their plans for exactly where they'll spend their resources.

BetaNews: Will any of the modules be industry-specific modules such as finance or perhaps engineering? What types of applications would these systems actually be used for?

Foster: That's a really good question. Will some of this work that I am talking about the consortium doing focus on particular verticals, financial services for example? I would answer as follows:

What is going to get people using grid technology on a large scale is going to be vertically integrated solutions. The consortium, at least in its initial phases is probably going to be more likely to focus on infrastructure elements because that's where the different pediments will have common requirements. But as the consortium grows in membership, I could certainly imagine interest groups forming that would want to support more vertically focused components.

One thing of somewhat relevance that I would mention at this point, is a neat demo that was done by SAP at its SAP TechEd conference late last year that sort of suggests some of the ways in which Globus and grid technology might evolve. At this conference they showed three of their analysis packages that had been modified to use Globus to do dynamic provisioning so that as incoming load increased they would dynamically acquire and deploy their applications on more computers, and as load was reduced they would dynamically release those resources for other purposes.

BetaNews: What industries are Univa partnering with at the moment?

Steve Tuecke: At this point we are not talking about any particular customers. One partner that we are working with for example is building up to apply the technology to government markets. We have various discussions going on in financial services in big engineering type settings. Examples of those are automotive and aerospace. There's discussions going with other industries, the traditional markets of the big technical computing for grid. Where we see a lot of opportunities moving forward is out in those non-traditional technical computing markets such as the ones indicated by SAP type applications.

Foster: I think we see ISVs as a potentially big market for grid computing.

BetaNews: There has been a lot of buzz about Sony using grid in its PlayStation 3 gaming console. What are some non-enterprise applications of grid and how do you feel grid can perhaps revolutionize or change gaming? Will grid technologies affect consumer electronics such as devices that store photos or music?

Foster: I am not familiar in details of what Sony is doing. But I'll just lead to a couple of things. One is, I am not sure where they are at the moment, but at least a year ago I spoke to an interesting company called Butterfly.net that was proposing to provide grid-based infrastructure for hosting multi-player online games. The assumption being that as people development new games they needed the central infrastructure to host the actual gaming activities and that this was a competency that was better outsourced to a specialist and that is what they proposed to do. But I understand that is not exactly what you are asking about here.

So, there is a whole class of interesting things that people sometimes called peer-to-peer (P2P) that relate to the loose and informal federation of computing and storage resources among individual participants. This is not something that I think we have put a lot of thought into but clearly I think it can become very interesting and I can see that some of the same technologies that are being developed for grid being applied.

Especially if you want to do something more than the current P2P systems that are based on loose notions of federation and trust and perhaps think about systems that, for example, federate storage among members of a family or different participants in some distributed organization that have some degree of trust and are therefore more likely to be people that you rely on to do things for you.

BetaNews: How important are identity management standards to the future of grid computing? There is no firm common ID management standard as of yet. Could that fact impede grid technologies from reaching the market?

Tuecke: I do not think that lack of any particular one set of standards that everyone agrees to will substantially delay grid tech to the market, but it does speak to a larger question about standards in general. This is why at the end of the day grid will not be adopted in its full glory; grid will not see it full potential, until there really are a common set of standards. Not just for ID management, but for all sorts of bits for technology like the Web.

A lot of potential comes out when you get those ubiquitous standards. That being said, there are already getting to be very large grid deployments in enterprises and even across enterprises and those are based on some combination and or Web open standards that exist to underlie them combined with technologies like Globus.

BetaNews: In so far as grids and Web services are now converging, do you feel that grid computing can be used to create alternative to common desktop environments such as Windows?

Foster: So this is one take on things: One perhaps narrow view on grid is that it is part of this trend of moving away from the mainframe where computing and storage and data processing are now done on this loosely coupled fabric of computing systems which may be geographically distributed.

There are various reasons to move into that direction and one is to take advantage of commodity systems. Now as you make that move, you start to need to replicate functions that used to be provided by these nice mainframe operating systems so you need resource management, security, monitoring, and accounting - all of these. This machinery needs to be reconstructed in this loosely coupled web services environment.

As we start to do that -- and we are making good process on those things -- I think we start to create something that has some flavor of a distributed operating system. And perhaps if it is successful starts to commoditize the OS on an individual system.

I think that is part of the excitement and also challenge of Web services for people like Microsoft.

BetaNews: Thank you both for your time.

Dr. Ian Foster is associate director of the mathematics and computer science division of Argonne National Laboratory and the Arthur Holly Compton Professor of Computer Science at the University of Chicago. He created the Distributed Systems Lab at both institutions, which has pioneered key Grid concepts, developed Globus software, the most widely deployed Grid software, and led the development of successful Grid applications across the sciences. Foster is considered one of the founders of the international Grid community, and has written many influential documents on Grid architecture and principles.

Steve Tuecke cofounded the Globus Alliance, originally known as the Globus Project, with Dr. Ian Foster and Dr. Carl Kesselman. He was responsible for managing the architecture, design, and development of the Globus software, as well as the Grid and Web Services standards that underlie it.