IE9 Tech Preview beats latest Firefox alpha, as Chrome 5 clobbers King Opera

What scalability teaches us about browsers we didn't know before

The reason Opera 10.5 went from zero to hero as fast as it did was because of how well its JIT compiler breaks down instructions at greater and greater workloads. And the reason Internet Explorer 8 cannot catch a break is because it does not scale workloads as well as IE7. In fact, simply overcoming that problem is why IE9 has come so far.

As with the other tests in our suite, we compare our results against Internet Explorer 7 as a baseline, as a way of extracting the relative speed of the machine from consideration. When something scores a 2.00, for example, that's our way of saying it's double the performance of IE7, and it would be double no matter what machine you tested it on.

Everybody who has used IE7 knows what "the speed of IE7" means, generally speaking. But "the scalability of IE7," or any other browser, is not something you can easily see, so drawing a comparison between it and any other browser might not make immediate sense. So here's a general rundown of what I learned: In solving problems that tend to scale linearly as workload increases linearly, IE7's scalability is surprisingly not bad at all. For example, using the algorithm Microsoft used to fix its widely reported choice screen randomization bug last month (yes, the new "shuffler" is indeed one of the algorithms we decided to use), IE7 starts out by shuffling a 250-unit array at about 125,000 units per second. When we change the array to 250,000 units of unique values, the estimated speed is closer to 195,000 units per second. And that's actually very good.

For mathematical problems that become exponentially more complex as the workload grows linearly, IE7 struggles. There are two ways of expressing an algorithm for finding the first numbers in the Fibonacci sequence: one which is compact and easy on the programmer, and another which is more drawn out but easier on the processor. For the compact version, for the first 20 numbers in the sequence, IE7 runs at about 132 iterations per second. Increase the workload to just 30, and the run time slows to just 1.5 iterations per second. Make it 35, and IE7 hangs interminably.

IE8 is only marginally more scalable overall than IE7, partly because in some instances, it's actually a lot worse. The surprise here is, the simpler and more linear the algorithm, the poorer IE8 is at finding that extra gear when it needs it. For simple reiterative problems such as the Sieve of Eratosthenes and a wonderful discovery dubbed Euler's Problem #14, IE8 can actually become slower than IE7 over time -- its scalability goes way down, below the 1.0 mark. So if you can picture IE7's scalability as 1.0, IE8's is just 1.31.

By comparison, the IE9 Tech Preview issued last week posted a scalability score of 8.75. That means, by our estimate, the "Chakra" JavaScript engine does a nearly 9 times better job of scaling heavier workloads, than IE7. Compare that to a 4.47 scalability score for the daily private build of Firefox Alpha 4, and you see some evidence of Microsoft's claim that managing Windows' background/foreground process timing can be more efficient than using tracing and JIT compilation. (Of course, now that this particular cat is out of the bag, imagine what Mozilla could do with it.)

The new categorical breakdown

What our new test suite also enables us to do, that we couldn't do before, is give you a single graph (rather than ten) that breaks down the browsers' performance by category. Here, our final score is broken down into three groups: computational speed (33%), rendering speed (34%), and scalability (33%).
Just last week, the Chrome 5 development build was posting speed scores that were lower than those of the stable Chrome 4. The score from last Friday's dev build 356.2 (which we verified by running the whole suite two more times completely on a rebooted system) gained back about 14 points in speed alone. One has to wonder just what it was that Google was holding back.

The latest Opera 10.51 is the fastest rendering Web browser in the field, at 19.62; here Chrome 5's scores actually dive below not only Chrome 4 but Safari. While I was devising these tests, the Chrome 5 dev build at the time was posting scalability scores that were not as good as Chrome 4's. Now they're well ahead at 10.93 versus Chrome 4 at 8.25, and Opera 10.51 at 9.47.

Oh, yeah, Safari! Lost amid all this talk about everyone else is the fact that the new stable Safari from Apple is faster and more scalable, and also that it's a better platform for the daily WebKit development builds. WebKit scores had been suffering, but joined with the latest Safari 4.0.5, they've improved dramatically, now with better rendering scores than either the stable or dev build of Chrome.

Ladies and gentlemen, welcome to the five-way shootout. With Opera and now IE officially in the hunt for supremacy, the Web browser battle may as well follow Joe Bob Briggs' first rule of horror pictures: Anybody can die at any moment.