Yahoo Debuts Open-Source Distributed Computing

Yahoo is launching an open-source project that it hopes will advance the science of distributed computing, which will also leverage its work in Hadoop. The project is an open-source distributed computing effort of the Apache Software Foundation, of which the Sunnyvale, Calif. search company has lead development on. Yahoo said it also plans to allow use of its 4,000 processor supercomputer for work on the project.

The first customer of the project will be Carnegie Mellon University, which will do various types of diagnostics on the system including performance evaluations, information retrieval tests, and testing its use for large-scale computer projects. Yahoo plans to open up the supercomputer for use by other groups in the future, it said. The company says the offering of its supercomputer for such users is important as many academic researchers do not have the hardware or software necessary for Internet-scale systems software research.