Performance test: IE8 easily doubles IE7 speed

During his keynote address today at MIX 09, Internet Explorer 8 product manager Dean Hachamovitch repeatedly used the phrase "real-world" in talking about endowing his team's Web browser with performance that the "everyday" user can see and feel. (That sounds almost like it came from AMD's playbook.) Hachamovitch showed some videos demonstrating IE8's performance gains in loading and rendering, particularly versus Firefox 3.0.5 and the "latest" Google Chrome (no version number, so presumably version 1 and not the beta of version 2). And he conceded that some of these gains can't be seen without a good stopwatch, so they'll only make sense to the end user if they can be felt by him.

Can the end user feel the performance gain from IE8? If he's accustomed to IE7 day in and day out, "every day" in the "real world," then we believe the answer is an absolute yes. Our Betanews tests on the release-to-manufacturing build of IE8 indicate that, all around, it gives 238% the JavaScript performance of IE7.

Now, what are we qualifying here as "real-world?" For our recent battery of JavaScript benchmark tests involving the latest experimental builds of the five major Windows-based Web browser brands (there are five now, can you believe it?) we chose some standardized and some relatively obscure test suites that cover the gamut of activities that JavaScript interpreters tend to do every day, just done repetitively and rigorously. We involved IE7 in that test as a baseline, recording its performance, averaging the various suites' performance together, and declaring it with an index score of 1.00. As arguably a slow Web browser even by Microsoft's own admission, IE7 makes an appropriate baseline against which to judge almost any other browser.

Last week, the final release candidate of IE8 scored a 2.19 compared to IE7. With some of the "debug code" taken out, that score tweaked upward for the RTM version to 2.36.

All of the final IE8's improvements were in the loading and rendering departments -- the two areas that Hachamovitch's video focused on. The RTM's load time for the HowToCreate test was much, much faster than for IE8 RC1 -- 531 ms versus 1431 ms, for index scores of 1.03 and 0.38 versus IE7. But what that means is, in this particular department, IE8 caught up to IE7.

The Celtic Kane suite revealed that IE8 RTM got a speed boost in error handling, RegEx expression handling, and declaring variable arrays versus IE8 RC1. Otherwise, there were still some areas in which IE7 outperformed IE8 RTM, including arithmetic and handling of dates.

It was in the SunSpider suite where IE8 RTM actually showed some real lags over IE8 RC1 -- in some cases, as much as 15% slower. So the cumulative index score for RTM was 4.75 versus RC1's 5.05. RTM was generally slower in each of the nine tests in the SunSpider battery, except for RegEx expressions where RTM was slower by a negligible amount (only 4.4 ms).

Some readers have asked us, why include the Web Standards Project's Acid3 test in the suite, since it doesn't involve speed but rather compliance? The answer is, to help us maintain a balance between doing things fast and doing things right. If this were an automotive magazine, we might point out that a certain model of car appears to perform well in the corners, but if the driver feels a definite understeer, that's not good. It doesn't help a Web browser to be faster at rendering if it's not rendering the way the industry wants it to...so scoring a 20% on the Acid3, which IE8 RTM and IE8 RC1 both do, isn't much of a help.

Now, Internet Explorer has not become a speed demon overnight, at least not compared to other experimental and production browsers in the field. By our estimate, for example, the production version of Firefox 3.0.7 has almost precisely double IE8 RTM's JavaScript performance (4.68 versus 2.36), and scores a 71% on the Acid3. And Apple Safari 4 for Windows can still claim a huge performance lead, with an index score of 14.39 versus IE7.

Our testbed is a Windows Vista SP1-based virtual machine running in Microsoft Virtual PC 2007, on a 2.4 GHz Intel Core 2 Quad Q6600-based custom-made computer.