Google Chrome in a runaway lead for browser performance supremacy
If Apple's Safari is going to make any kind of a challenge for best performing Windows-based Web browser moving into next year, it needs to be now. In Betanews' most extensive testing to date, involving tests that by anyone's guess should not have given it any special advantages, the latest stable edition of Google Chrome runs away with a three point lead over the latest stable Safari -- a lead that now grows by one-half point with each point release.
Mozilla's team has been making efforts to better Firefox' control flow scores, evidently knowing how much they influence test results like ours. The evidence comes from the latest daily builds of Firefox 3.7 Alpha 1, on the "Minefield" track, whose control flow scores recently quantum-leaped down to 8 ms. That's almost a 5x improvement, but it will need another 3x blast to catch up with Chrome. The latest stable Apple Safari scores 3.4 ms on this element.
Another example: A factorial is the result of multiplying together all positive integers that are less than or equal to a number, and a new element of our testing battery includes a classic algorithm for obtaining the set of all factorials. On this heat, the higher number is better since the objective is to obtain as many factorials as possible over a set period of time, so the score is a relative index. While Firefox 3.6 Beta 1 scores a 17 on this heat, Chrome 3 scores a 150.4, and the latest dev build of Chrome 4 scores 164.5.
There are certain things that Chrome does where the score differences are factors of 10, where one might get the impression that Google is improving Chrome just to score better with Betanews (the company has been expressing its interest to us directly in recent days) or more likely, with its own internal test suite. Indeed, the company's V8 benchmark suite would have users thinking the browser is hundreds of times more capable than its competition -- a claim for which we just don't see the practical evidence just yet, which is why V8 isn't part of our tests.
That's why we've made an effort to pack our CRPI test suite with examinations of a multitude of real-world characteristics in various categories, now with 69 different "heats" in ten separate batteries, plus a multitude of derivative scores (e.g., average of 50 iterations, consistency between the fastest and slowest run, etc.) for each browser.
"Probably" pulling close? What, Betanews can't do better than that with respect to Safari? Yea, unfortunately there's still trouble with that: Apple's test builds of Safari come by way of grafts of its daily WebKit engine onto the existing 531.9.1 browser. Usually, after applying one of these grafts, the updated Safari displays better if not superior rendering performance than even Chrome. But its ability to serve as a full-scale browser for other tests vanishes. However, over the last week while Betanews has been trying to resolve this problem, it actually only got worse: Windows testers reported through Apple's forums that not even the grafting mechanism was working.
We validated those claims, discovering that the manifest which the replacement executables were being shipped with (the embedded XML files that point to proper COM components in the System Registry) were pointing instead to incompatible versions of Windows Common Controls, versions that may have worked back in the 1990s. Apple is apparently already aware of this, but as is the company's policy with regard to any kind of problem with its software or hardware, will not publicly comment.
Next: The latest test scores broken down...