Is Opera 10.5 ready for the March 1 'choice screen?'

The potential hazards of subdividing the ecosystem

Up until now, one of the strengths that have sustained Opera through the rough periods has been its customizability and extensibility. For the most part, Opera can be made to function the way that users want it to function, and that customizability comes as part of the package. By comparison, Firefox's strength to this point has been that a very strong community of developers, fostered and nurtured by Mozilla, have been able to craft extensions and add-ins to Firefox that enable professional users such as myself to practically reconstruct the browser into the tool they need it to be.

Opera has also nurtured a development community, but the model Opera has promoted revolves around widgets -- mini applications that are executed by Opera's JavaScript interpreter. Since Firefox is a JavaScript app in itself, extensions can run in the context of the browser and modify it. Now, Opera is becoming more strict about its development model. With version 10.5, widgets will be restricted to applets that run outside the browser context, such as weather applets, e-mail checkers, and sidebar-like games.

The new model for Opera 10.5 widgets is supposed to be based on stand-alone applets. But some of Opera's select widgets in its own gallery, are not.

Fair enough, especially since Opera doesn't need a lot of add-ons (like Firefox does) to be customizable. But when European Web browser users see the Opera brand for the first time, by virtue of its prominent placement on Microsoft's new browser "choice screen," whether 10.5 is ready for public consumption or not, Opera will likely lead new users to the place where they can download the fastest browser the company produces. After they do, users will want to experiment with how far they can stretch it...or, they may happen upon the Widgets panel while simply playing around with new and shiny buttons. Once they're taken to Opera's widgets gallery, they'll see add-ons like the homemade version of Google Toolbar (whose listing is shown in the screenshot above).

These users may not know old-style extension widgets won't work in the 10.5 context until installing them and seeing that nothing happens. Users may take that to be a bug in the program, when really it's a change in the architecture that hasn't been passed on to the proprietors of Opera's Widgets gallery. And that may become a problem.

Working with Opera in the performance department

In our report last Tuesday on Opera 10.5 Beta 1's relative performance, we noted that there appeared to be a continuing bug in the way the browser was executing one of our tests. Opera's engineers wondered whether the bug was in the test we were using -- which is actually fairly old, as benchmarks go: the Testnet.World JS JavaScript instruction test.

We've encountered situations before where Opera's and Safari's counters were synched with their respective JavaScript interpreters in such a way that they tended to report that no time was expended whatsoever ("0 ms"). Whether that's actually a defect or an architectural symptom may depend on where one stands (and who signs his paycheck while he's standing there). Nevertheless, two of Opera's architects suggested improvements to the Testnet.World battery that they said would avoid this perception problem.

Giving Opera the benefit of the doubt, we implemented their suggestions. What we discovered made us satisfied that we were given good advice. Essentially, the old benchmark tested the time expended, but when it came up 0, it assumed the test never started running, so it maintained the loop. Our new test battery no longer trusts the expended time indicator, but rather, triggers a binary "done" flag for each test. In order to better measure events that used to take tenths of seconds but now take fractions of milliseconds, we increased the workload for each test by a factor of 10, for a million iterations rather than 100,000.

It was in ten-tupling the workload that we made several interesting discoveries:

• Internet Explorer 7 wasn't such a slouch.. At 100,000 iterations, yes, IE7 on Vista SP2 (our slow index browser, for purposes of comparison) is pretty darned slow. Frankly, I was afraid that increasing the workload would make IE7 slow down exponentially. It did not; in fact, it scaled up the workload rather well, in some cases handling each iteration of the larger workload faster than for the smaller one. As a result, as the workload scales up, the comparative scores for other browsers measured against IE7 go down. That helped reset our overall scores by about 12% across the board.
• For some very repetitious workloads, Firefox is twice as fast as Chrome. No, that's not a typo. It could be argued that real-world Web pages are not going to repeat a million conditional statements. However, if we treat this test battery not in terms of how well browsers repeat statements but of how well they scale large (if artificial) workloads, Firefox is actually the champion here. The stable Firefox 3.6 on Windows 7 scored a 4.45 (445% the speed of IE7 in Vista SP2) on the corrected Testnet.World battery. By comparison, the latest daily build of WebKit/Safari 4 scored a 3.91, while Google Chrome 5 Dev build 335.0 scored a 2.26. The latest stable Chrome 4 scored 2.09. What difference does that make? On our old (uncorrected) Testnet.World battery, using one-tenth the workload, Firefox 3.6 scored a 31.88, while Chrome 5 scored 43.18.
• Opera's developers are pretty decent folk. Their suggested changes eradicated the appearance of the "0 ms" anomaly, but it did not result in Opera 10.5 Beta 2 receiving favorable treatment. It only scored a 3.32 on our corrected battery.

Compensating for the anomaly did, as we predicted, improve Opera 10.5 Beta 2's scores relative to Chrome. Though the revised Testnet.World numbers scaled everyone back (IE8 and Firefox least among them), there's now a three-point performance gap between Opera's and Google's most recent development builds. Chrome 5's scores on the other batteries in our Windows 7 test suite were generally lower once again. The latest dev build, among other adjustments, changed the way that browser interprets the source of locally stored HTML files -- each one is now considered within its own individual domain, rather than in the shared domain of the computer. That's a very important safety adjustment that security-minded users should welcome.

After resetting our numbers, Opera 10.5 is now the only browser that scores above a 20 in our Windows 7 index, with a 21.50 versus 18.51 for Chrome 5. In the latest builds, Chrome 5 opened up a bit more of a lead against Opera 10.5 in the all-important SunSpider battery. But Opera gained back quite a bit of performance in the table element rendering category, which has historically been a strong point for Opera anyway.

Does Opera feel ready?

As Microsoft would probably tell us at this point, performance gains only matter if users can feel them. "Splinters" such as skin elements that don't line up can lend to perception problems that the browser feels unfinished, and that perception can become a lens that magnifies any other problems the browser has.

One such problem we encountered with Opera 10.5 Beta 2 concerns hesitancy. From time to time, after clicking on a bookmark or a hyperlink, or even during our performance tests, we noticed that the browser can go into a state of limbo for as long as a second, before our network meter validated that it has started loading the page. And when a page contains a long <TABLE> element -- for instance, of items pulled up from an SQL database -- reloading or refreshing the page, or loading a page with a different <TABLE> element, causes the entire plotting area of the window to go white for a noticeable period of time.

There appear to be lots of little issues like this in the latest public beta of Opera 10.5 -- enough for us to suggest there should be at least a Beta 3 cycle before a final public release. If the aim is to move users off of Internet Explorer, it should be noted that IE8 does not have lots of little issues like this. It's slow as a jar of molasses buried under a glacier, but many users don't care because, to them, it feels finished, usable, drivable, like a '76 Cadillac Eldorado.

Opera has come a very long way in a very short period of time. But as Vista proved, perception in this business is everything. If its engineers take the time they need to sand off all the splinters, this browser has all the ingredients it needs to dethrone Firefox, Chrome, and even IE in their respective categories of strength. This is the one that could actually do it. But if it's rushed, and it premieres to the general public looking unfinished and cobbled together, those ingredients may go unnoticed.