Fighting back with fire: Firefox 4 closes the gap, Chrome threatens Opera's lead

On the other side of the pond, Google has been ramping up Chrome to enable GPU acceleration as well. Late last week, Google premiered its first acceleration code with build number 552.0 - which the company has decided to dub, for the first time, Chrome 8. (The distinctions between Chrome version numbers aren't always as discrete as everyone else's; Chrome 8 is not at all an overhaul of Chrome 7, which itself has yet to be released in a stable edition.)

Google operates two development channels, dubbed "beta" for more general public participation, and "dev" for more experimental code. Chrome has often been the fastest brand of browser in Ingenus' tests, but it achieves many of its speed gains by distributing its tasks over two processes. (I've seen it fork a third process but not use it yet.) As a result, CPU and memory utilization are often quite high, just like for any sports car that achieves greater horsepower by lowering its gas mileage.

The latest beta build of Chrome 7 cranks out 84 FPS in the 100-sphere Canvas test, with 1260 samples in the 15-second interval. Because Chrome uses two processes when the going gets tough, quad-core CPU utilization ramps all the way up to 152.4% on Performance Monitor, or about 38% of real capacity. But compared to Firefox, memory utilization for Chrome is actually quite low, with 36.4 MB distributed over both processes.

With the dev build of Chrome 8, Google begins its own acceleration experiments. The early results. . . are mixed. For now, frames per second actually slows down versus Chrome 7 beta, to 56 FPS with 839 samples. CPU utilization for process #1 definitely declines to an imperceptible 1.366%, showing GPU offloading is clearly working. But that's just for process #1. Process #2 drops utilization only to 57.2%, leaving Chrome overall with a noticeable footprint, and memory consumption that's still slightly higher than for Chrome 7 beta (41.6 MB).

This is why Chrome's efficiency score on this test increases only marginally, from 0.803 (relative to Firefox 3.0.19 in the same test) to 1.083; whereas Firefox 4's efficiency catapults from 0.977 for Beta 6 to 20.666 for Beta 8. Keep in mind, that number is but one factor in a wide-ranging test suite of nine different batteries, accounting for computational speed, rendering speed for text and graphics, "scalability" (the ability to improve with higher workloads), and resource efficiency.

It was IE9 that increased developers' focus on GPU acceleration in the first place; and until this week, IE9 had the highest efficiency scores on the 100-sphere test. IE9 cranks out 59 FPS with 885 samples, with average quad-core CPU utilization of 24.6%, and memory utilization of 36.9 MB, for a relative score of 1.957. (IE also splits its workload over two processes, but tends not to use process #2 unless in extreme conditions.)

For now, Firefox's GPU offloading efficiency gains only show up during the 25- and 100-sphere tests; at 500 and 2500, the latest Beta 8 build appears to revert to standard CPU utilization above the 90% mark, comparable to Beta 6. There appears to be a kind of virtual rocker switch somewhere, at least in the current beta builds, limiting offloaded workloads to lighter amounts only. Theoretically, if the GPU offloading code is stabilized and the same performance gains end up applying proportionally to the 500 and 2500 tests, Firefox 4's overall score could surge forward all the way to the 5.0 mark, potentially eclipsing Opera and Chrome.

There's a reason that Firefox 4's score now threatens that of Apple Safari 5 as well, and it has to do with a scoring change decision. Both the stable Safari and the test build with the WebKit development snapshot, are capable of cranking out huge FPS numbers - for instance, 239 FPS in the 100-sphere Canvas test. At least, that's what the statistics read. But Safari gets away with this by turning off screen refreshes during the high-processing period, often during the entire 15-second interval. Often the final screen with the FPS and deviation count, is the first moment you'll actually see the spheres.

Now, being able to calculate that fast is awfully nice, but if the task is graphics and you're not seeing the graphics, it all means nothing. Imagine if this task were instead a complex game being rendered on the Canvas element; how could the player be expected to win without a touch of clairvoyance? So I made a decision: I would treat each test run where Safari (or any other browser) turned off the screen as a fail, just like a teacher who realized his student turned in empty pages for homework. That's why I've scored the latest stable Safari 5.0.2 down overall to 3.520, and 3.666 for the latest Safari with WebKit dev build 69699.

Opera's snapshot build remains the overall performance leader, but not by much. Chrome 8's marginal efficiency gains, coupled with very impressive computational speed gains, have narrowed the gap to just over a hundredth of a point: 4.992 for Chrome 10.70 build 9071, versus 4.980 for Chrome 8.0.552.0. The overall computational speed leader in Ingenus' tests is now Chrome 8, with 9.760 versus 7.965 for Opera 10.70 (the future Opera 11), 7.870 for Firefox 4 Beta 8, and 7.645 for IE9. Chrome 8 is currently the undisputed champion of the JSBenchmark battery, and staying in the top three for computational speed elsewhere in the suite.

Here's another important note: With multi-test batteries like SunSpider and SlickSpeed, Ingenus compares the scores in each individual test against those of the index browser (Firefox 3.0.19), and then computes the average of those scores (or, in the case of SlickSpeed, the geometric mean since each test in that battery is exactly the same job, just using different libraries). Most testing focuses on the final time elapsed, especially for SunSpider. But there are individual elements of SunSpider where IE9 blows away the field - for example, the CORDIC algorithm test, where IE9 scores 1 ms versus 8 ms for Opera 10.70 and 10 ms for Chrome 8. If we were examining cumulative final speed only, that 1 ms posting would get completely overlooked. That's one big reason why IE9's SunSpider score remains so high (14.268 versus 12.118 for Opera 10.70 and 11.177 for Chrome 8).

Where Opera 10.70 excels is in the rendering department, with a 5.223 score in that department, and its nearest competitor - the Chrome 7 beta build - only at 4.314.

Efficiency has been the great equalizer in Ingenus' latest test suite; it's where the current stable Internet Explorer 8 sets the high watermark at 3.899 (compared against the old, low-efficiency Firefox 3.0.19), with everyone else's scores tending to go down from there. Today, the browser with the most comparable efficiency score to IE8 is the latest stable release of Opera 10.63, with a 3.844. The last build of Opera 10.70 before the numbers change, follows up in the efficiency department at 3.670. And then in third, thanks to its successful Canvas experiments (when they are successful), is Firefox 4 Beta 8 with a 3.146. That's a huge jump over Beta 6's efficiency score of 1.728.

This article originally appeared in Net1News.

©Copyright 2010 BetaNews, Inc.