IE8 loads pages faster, but not fast enough, in Microsoft test recreation

Performance needs to be something the user feels. That's the explanation we hear from companies that have had difficulty of late demonstrating raw performance by the numbers (AMD comes readily to mind). Numbers may tell you a certain story, Microsoft's marketing team proclaims, but if the user doesn't perceive the speed increase, it may as well not be there.

So in a white paper released last week along with IE8, entitled "Measuring Browser Performance" (PDF available here), made the case that today's Web pages are comprised of so many components, any one of which may load faster or slower in one browser than another. And because of that, the only way to get a real sense of which browser is generally faster is to feel the overall speed.

"For the real world user, performance criteria are based on how quickly a page loads, not the performance of any individual subsystem," the white paper states. "For that reason, this paper will focus on how to measure complete webpage load times and discuss the related complexities."

If Microsoft's goal was to prove IE8 noticeably faster in the real world, it fell a little short. Its white paper ended up demonstrating, in Microsoft tests, that IE8 was 8.3% faster at loading popular Web sites' home pages than Firefox 3.0.5, and 0.023% faster than Google Chrome 1. If everyday users notice that, they're not only smart, but sensitive.

But how did Microsoft come up with these times. The paper suggests that the reader measure the various load times for individual Web pages in order to achieve comparable results. For its own results, the paper's authors chose 25 selections from comScore's Top 50 Web Properties partly because they tend to be more universally available and more reliable. The paper then suggests the reader deploy a video camera pointed at the monitor, which it says has the most reliable time code, and use that time code to assess the load time of each of the 25 pages in all of the browsers tested, then include a fudge factor of plus-or-minus 0.4 seconds.

Well, in reading the part about the "real world," we didn't think everyone would want to do this with a video camera. So Betanews came up with a more realistic alternative: For our tests, we used the WebMonkey Browser Load Time Stopwatch, a simple JavaScript tool made for the WebMonkey developers' site. Specifically, we found where one site had adapted this tool in such a way that it could render an accurate load time for any page, down to one thousandth of a second. Plus or minus zero.

The objective of our tests was to determine this: How much faster at just loading Web pages is IE8 from IE7, and is it faster than its biggest competitor and its future competitors at this specific task? Last week, Betanews tests showed IE8 to be more than twice as fast as IE7 for everyday tasks, including load times; but for JavaScript processing, IE8 remained the slowest of the current production browsers available. If we exclude JavaScript processing and concentrate just on loading, though -- the only part Microsoft says real users care about -- will IE8 demonstrate the speed gains with a real stopwatch?

Next: Is IE8's load time boost more of a tick or a boom?