IBM, Sun Spar Over Server Performance

Dwindling economic prospects have spawned a spirited battle for market share between IBM and Sun Microsystems. According to an IDC report published in September, worldwide server factory revenues plummeted 16 percent from $14.4 billion in second quarter of 2000 to $12.1 billion in Q2 2001. Upon the release of IBM's Regatta (p690) lines of servers -- a product IBM dubbed "groundbreaking" -- the stage was set for a no holds barred confrontation between the world's largest server manufacturers.

Initially, a war of words enveloped each company as rhetoric flourished in the absence of hard facts. IBM and Sun each claimed to have superior product offerings and a leadership position in the market; not hesitating to dish out stern criticism of each other.

The p690 line of servers, formerly known solely by the codename Regatta, introduced a flurry of innovation from Big Blue. According to IBM, the Power 4 processor, the heart of the machine, was a "server on a chip" that allowed fewer processors to complete sophisticated tasks than while outperforming servers with twice as many chips. ISV's such as Oracle have adopted processor based pricing schemes, making fewer processors desirable for cost savings.

Additional features include: low power consumption, a consolidated usage of parts, virtualization, ultra-dense building blocks, and the company's proactive "self-healing" architecture known as eLiza. When all was said and done, IBM claimed to have created a "game changing" product for a fraction of the price of competitors.

A War of Words

Sun's Chris Kruell, Group Marketing Manager for Computing Systems quickly drew a comparison to his firm's mid-range Star Fire Servers, and pointed out that top of the line Star Cat servers contain 106 processors in comparison to the IBM's 32. He also downplayed IBM's assertions that they had developed anything "game changing," stating, "It might be for IBM, but it is not from Sun's perspective."

Kruell told BetaNews that he suspected the Regatta contained imbalanced architecture - asserting that IBM's new server may possess an IO bottleneck. To prove his accusations he submitted JBB benchmark data to BetaNews outlining what Sun believes to be weaknesses in the p690's design that date back to its predecessor.

"On the SpecJBB benchmark, IBM used the 16-way p690 HPC; this HPC configuration has only one CPU for each L2 cache, as opposed to sharing L2 cache between two CPUs as the commercial version of the p690 does," Kruell noted. "It seems only reason for IBM to offer such a configuration is to reduce contention and to increase cache bandwidth. Lots of questions get raised...1) Will it be hard for them to get a great scaling to 32 CPUs on SpecJBB? 2) Do they have hardware problems with 32-way Turbo's right now? 3) Will the p690 get good scaling when they have two CPUs per L2 cache on SpecJBB?"

In response, IBM's Director of Technology, Joel Tendler bluntly dismissed Sun's claims telling BetaNews, "They (Sun) don't know what they are talking about," and concluded that there was no basis for such bold accusations. Tendler went on, "It (p960) is a brand new machine that runs rings around our old machine, and theirs."

Tendler also pointed out that compared to IBM's 32 way server, Sun's 64 processor model contained five times as many parts. Something that he noted would lead to poor reliability and added cost. IBM also called into question Sun's scalability as the amount of processors on their servers increased incrementally. Kruell told BetaNews that Sun stands behind the SPARC III architecture given that Sun pours considerable resources into its CPU development team making it second largest only to
Intel.

Continuing on, Tendler maintained that IBM servers carry an abundant amount of IO bandwidth fit for any application as well as superior virtualization, reliability, and power savings. In his opinion IBM has a solid foundation for years to some. He bore down on Sun accusing the company of taking old technologies -- apparently in use by IBM for many years -- and acting as if they were newly discovered innovations.

Sun's Kruell continued his salvo by saying that Sun did not see anything new in the p690 - maintaining that Sun has excelled in server consolidation and dynamic partitioning for years. "The industry votes with its dollars," said Kruell, citing IDC market share data from September's Server Tracker report.

The report outlines that Sun has obtained 42 percent of the Unix server market - enough to hold on to the number one position. Dataquest reports that Sun has maintained its dominance for nine consecutive quarters. However, IDC data from the 7th of September shows that IBM is the momentum player in the market, illustrating gains of 4.1 points in Unix market share while Sun lost some ground.

Overall, IBM holds 26 percent of the total server market in comparison to Sun's 16.5 percent share, according to the same report.

A Benchmarking Brawl

Few direct comparisons existed between IBM's p960 and high end Sun servers with both companies accepting the adage that it was "our word against theirs" until IBM and Sun engaged in what amounts to a Fluent benchmarking war. Fluent is a utility that both companies acknowledge gauges "real world" application performance - something that IBM's Tendler pointed on his own accord. Giving even more credence to the benchmark, Sun recently abandoned the TPC-C transaction processing benchmark -- a measure used by many customers who buy servers -- in favor or Fluent.

The first shot was fired by Sun in a press release issued last Thursday entitled, "72-way Sun Fire[tm] 15K Server Outperforms 128-way IBM Server by 23 Percent." The release came out alongside the 15k's launch.

IBM fired back by issuing its own findings to BetaNews via e-mail and by subsequently publishing a press release the day following Sun's announcement.

When asked for comment, Illuminata analyst Jonathan Eunice remarked, "IBM's numbers are impressive." Although he cautioned against equating the Fluent results to each an every real world scenario. Eunice told BetaNews that however impressive, the results will not necessarily transfer over to a commercial environment such as a database. He recommended that interested parties benchmark their own applications.

The Fluent test measures deep technical computing - an aspect of IBM's design that is bolstered by well renowned floating point performance and its available memory bandwidth. Eunice commented that it was no surprise that IBM did well in the Fluent tests given its decade long track record of focusing on floating point performance. Tasks normally associated with technical computing include designing aircraft and automobile aerodynamics.

A Game of Leap Frog

David W. Gelard IBM Director of E-Server Benchmarks made clear that the recent set of benchmarks were directed toward informing industrial customers of the p690's price performance attributes versus Sun. Gelard denied that the benchmarks were meant to go head to head with Sun's own Fluent test results, but did promise more data to confirm the performance of IBM's products in the minds of potential customers.

Sun spokesperson Chris Kruell agreed that IBM's benchmarks were based on a solid foundation. However, he said that Sun's scores would improve once engineers executed code optimized for the SPARC III. Apparently the company was more focused on calming fears over the transition from SPARC II to SPARC III rather than on performance and ran the same binary code from a previous test - untouched. Sun is also waiting for version 6 of Fluent to arrive before proceeding into what Kruell referred to as a "leap frog game" of reactionary benchmarking.

"We are number one in the UNIX server market and IBM is going to take every chance they can to knock us off the top. We hold a leadership position in terms of mindshare with ISV's and customers in the UNIX market space," said Kruell.

As each side remains focused on decisively winning market share and squeezing out the competition, there is sure to be no love loss between Sun and IBM.