Firefox 4 beta loses to IE9 beta in browser speed, efficiency tests

But at what cost?

As is the case with any other kind of machine ever built throughout history, when you build it to perform better, there's a cost in terms of efficiency. The Chevy Chevette may have been the single ugliest product to emerge from Detroit (way uglier than the Edsel), but it did have that fuel economy thing going for it.

So one of the most important questions we can ask when testing Web browsers, is this: In making Internet Explorer 9 a far better performer than its predecessors, did its architects sacrifice "fuel economy?" Or more specifically, does IE9 chew up more memory and processor cycles?

Say what you will about IE's performance through the years, but one advantage of working next door to the Windows architects that its own architects can use to their advantage, is building a more efficient browser. Even Firefox's most fervent supporters have historically conceded that it's a memory hog. Indeed, Firefox 3.0 (my new index browser) was a virtual black hole into which memory appeared to fall with no chance of escape.

If we score that black hole a 1.000 for CPU and memory efficiency, then by comparison, IE8 - which in several respects is more like a Chevette than a Corvette - scores a 2.845, judging from the SunSpider battery, SlickSpeed CSS selector processing test battery, and DFScale battery combined. Against most other browsers, including the stable versions of Firefox, that's actually very high indeed. This is the one scoring department where we can expect the latest, high-performance betas to fall, not rise.

Overall, IE9 beta does pay a price for its computational superiority, scoring a 2.014 in the efficiency department. But that average doesn't tell the complete story: The current stable IE8 sips memory cycles at low workloads. My latest addition to the complete test suite, adapted from The Man in Blue's animation benchmarks, coupled with processor cycle and memory consumption monitoring using Microsoft's Windows 7 Performance Monitor, shows that in low-workload animation processing, IE8 can consume less than 0.2% of available processor cycles, while Firefox 3.6.10 chomps away at 21%. The IE9 beta does consume more CPU cycles at low workloads than its predecessor, maxing out at about 4%, but that's still impressively low.

As workload increases, however, IE8 opens its throttle and starts chomping down gas like it was water. The Firefox 4 beta posts some very impressive numbers in middleweight workloads, with efficiency scores on individual Man-in-Blue heats approaching 14 (reflecting, for example, about 0.7% overall CPU cycle consumption when plotting 100 simultaneously moving objects on the HTML 5 Canvas element). But at high workloads, where IE8 can actually crash (I had to repeat some tests several times until it didn't crash) and Firefox 4 falls through the floor, the IE9 beta moves steadily along, posting numbers that are still above that 1.000 index mark.

Google Chrome's recent claim to fame has been its slick graphics and high frames-per-second numbers. My latest test confirms that claim, but it also shows the price Google pays: In the "beauty" department, to borrow Microsoft's new term for it, IE9 beta scored a 3.122 in performance and 1.832 in processing efficiency. Google Chrome 6, meanwhile, scored only 2.892 in performance and 1.461 in efficiency. So think of IE9 as a more fuel-efficient performance engine. If you do the math, this suggests that IE9 makes about 26% better use of its CPU cycles and allocated memory for overall graphics production (which, for the first time, includes SVG) than does Chrome 6.

However, Firefox 4 Beta 5 did beat the IE9 beta in the efficiency department, scoring a 2.295 overall. Memory and CPU utilization efficiency have never been Firefox's strong suits, so FF4's high scores here are very encouraging. Particularly, FF4 is learning to scale down its CPU cycle consumption with lower workloads; for example, reducing its low-workload consumption averages from 9.1% to under 1%. Firefox 4 could very well be the best behaving Firefox that Mozilla has produced to date. The problem is, it'll have more company.

The complete test suite results, including an extensive analysis of every browser in the field including Opera and Safari, is the subject of the first Ingenus Report, available soon.

This article originally appeared in Net1News.

©Copyright 2010 BetaNews, Inc.