A real beta process at work: Mozilla fires up Firefox 3.6 Beta 2

After several weeks of delay for the release of Firefox 3.6 Beta 1, you might say the Mozilla team had some ground to make up. Flying squarely in the face of any commercial company that says it gets bogged down with so much user feedback, the organization accelerated the release of the public Beta 2, in response to 190 major issues with Beta 1 detected and reported by a multitude of users.

Betanews has only had the public Beta 2 up for the last hour, so we can't draw too many conclusions just yet. What we are noticing is that the browser's general responsiveness to the user seems smoother and snappier than the current stable version 3.5.4 (our tests on 3.5.5 are pending). While any 3.5 version is worlds more reliable than version 3.0, running ordinary tasks in Google Chrome appears noticeably faster, if not to a stopwatch than to our eyes. Firefox 3.6 Beta 1 appeared to have more of its game face on, and Beta 2 would appear to have its pants zipped up and its shirt tucked in.

For reasons we never quite discovered (and that we'd be willing to accept had more to do with Betanews.com than Firefox), the new Ctrl+Tab thumbnail preview feature had difficulty rendering the Betanews front page in Beta 1. That problem seems to have disappeared in Beta 2, so our thumbnail now shows up in miniature in the Windows 7 taskbar and in the new Ctrl+Tab selection pane.

We haven't yet run Beta 2 through every benchmark test on every platform, but we've just completed an early round of tests on Windows 7. So far, Beta 2 is the best performing Firefox we've ever tested, overall. On our new CRPI 2.2 scoring system, it registered a score of 14.63, compared to a 24.63 score for our most recent test of the stable Google Chrome 3 (not a typo there, the difference is exactly 10 points). Our last test of stable version 3.5.4 scored 13.88 on Win7 (we'll test stable 3.5.5 soon).

But not everything is faster with Beta 2; there are certain elements of its execution profile that are somewhat faster, and others that are somewhat slower. For example, our tests show that Beta 2 is faster at loading pages, and at accessing that first element of the page, but not necessarily at getting that critical onLoad() JavaScript event fired. That event has been a secret to Apple Safari's success. The faster that event loads, the faster the browser can start executing code even while the remainder of the page is still downloading and being rendered. Beta 2 scored an 8.69 in our CSS rendering test on Windows 7 versus 3.5.4's 10.48, which may be an indicator of the effect of bug fixes -- maybe more stable, but slower. But Beta 2 scored a 4.38 on loading everyday pages versus 3.5.4's 2.96; and Beta 2 scored 5.81 on rendering old-style HTML tables versus 3.5.4's 4.27.

Mozilla looks like it's concentrating on improving those elements that everyday users see and feel without a stopwatch (or without Betanews calculating everything to the n-th degree), and it's making headway there.

There are obvious changes to the calculation profile, suggesting changes in the JavaScript engine, perhaps for reliability and security. If you look at the speed of individual methods and instructions unto themselves, it's improved nicely: 43.50 in Windows 7 on the TestWorld instructions test versus a Firefox 3.5.4 score of 41.92. However, put those instructions to work with algorithms, and those faster instructions don't scale up the same way: Beta 2 scored a low 7.46 on the new JSBenchmark classic algorithmic test battery, while 3.5.4 scored a 9.14.

Still, the overall score for Beta 2 is a few tenths of a point higher than for Beta 1, suggesting that Mozilla's making the right tradeoffs. The organization has not publicly announced Beta 2's availability at the time of this writing, though a confirmed public Beta 2 build was downloadable from Mozilla's servers. Yes, we tested to make certain this wasn't a Release Candidate masquerading as a public beta (it's happened before). "Private" preview builds of 3.6 Beta 3 are already available.