Firefox 3.5 Beta 4 beats IE8 in Microsoft's own load times test

Last month, Microsoft published a white paper suggesting that, as first adopters installed the final build of Internet Explorer 8, they judge the real performance of the new browser based on what they feel when they use it. Specifically, the authors of the paper (PDF available here) advised users to load their favorite Web sites and feel how much less time it takes to see results. But since many Web sites take only microseconds to load anyway -- perhaps beyond the range of everyday human feeling -- they went on to suggest that folks take videos of their Web sites loading, and use the time index cues to tell the story of whether they should feel pages loading faster or not.

Betanews took Microsoft up on its suggestion...kind of. No, we didn't set up a video camera. But we did set up a clean virtual machine with an adaptation of WebMonkey's Browser Load Time Stopwatch, to see just how many microseconds we should be feeling if our feelers felt that sensitively. At the time, our tests concluded that the latest edition of Mozilla's production release of Firefox available at that time, version 3.0.7, could load 25 selected pages from comScore's Top 50 Web sites (plus Betanews thrown in for good measure) 13% faster on average than the final IE8, and Firefox 3.1 Beta 3 loaded those pages 31% faster than IE8.

In a fresh round of Betanews tests this morning, however, we discovered the gap had widened considerably, at least with respect to Mozilla's latest public beta. Firefox 3.5 Beta 4, released yesterday (actually the next public beta after 3.1 Beta 3), blew away IE8 by loading pages 81% faster on average [an adjusted figure following a math correction by some of our readers, much obliged].

We used 23 of the 25 Web sites chosen by Microsoft for its own test -- which, incidentally, concluded IE8 was generally on a par with other Web browsers. Two sites -- Blogger.com and MySpace -- do not test accurately with the timer because of how they use JavaScript, so we replaced those with two sites in comScore's list of hot properties, TV.com and Weather.com. Then we added Betanews for good measure.

The Web sites that loaded faster in IE8 than in Firefox 3.5 Beta 4 were Google (certainly among the most used), YouTube by a small margin, eBay, and if you can believe it, Betanews. These results held up over multiple tries, and we gave IE8 the fastest time whenever we tried more than once. To ensure that we weren't hitting on the memory cache, we cleaned the cache completely for both browsers prior to loading each time trial.

In tests of JavaScript and CSS rendering performance yesterday, we found Firefox 3.5 Beta 4 to have 477% the speed of IE8 when performing calculations. Add to this the fact that the new beta renders fresh pages in about three-fourths the time, and we begin to wonder just what it is we're supposed to be feeling.