Apple, supporters claim iPhone browser test flawed

A study on the loading times of websites on both Android and the iPhone came under fire late Thursday, with Apple supporters calling its results flawed and the company behind the study defending its findings amid questions on the methods used to test the iPhone's performance.

Blaze Software, a website optimization company found that on average Android loaded web pages 52 percent faster than the iPhone 4. It also said that it saw no noticeable performance increase due to the optimizations of the JavaScript engine included in iOS 4.3.

There was one problem, however -- the embedded browser in iOS 4.3 does not include the Nitro JavaScript engine that the standalone Safari browser has. The Loop's Jim Dalrymple was the first to jump on this error, calling the study 'flawed.'

Blaze's proprietary app in the test used the UIWebView API for the embedded browser. "The problem with using UIWebView is that, even though it's based on Safari, it didn't receive any of the updates that Safari did in iOS 4.3," Dalrymple wrote. "Using an embedded browser is not the same as using the official browser."

Apple later backed up and echoed his comments to reporters, adding that "despite this fundamental testing flaw, they still only found an average of a second difference in loading Web pages."

It is not exactly clear however if the issues with the embedded browser may make a difference in the final result. Blaze had said that only 15 percent of the page loading time had anything to do with JavaScript loading. Even if Nitro would speed up loading, it is likely not enough to make up the difference.

"We hope Apple will help us enable those optimizations and repeat the measurement," Blaze co-founder and CTO Guy Podjarny posted in a response to the growing criticism. "Until then, for all we know the missing optimizations may not make a big impact."

Podjarny added in response to those that may say the second difference didn't matter that in some cases the difference was much greater. For example, Android was loading the Wall Street Journal's website some five to ten seconds faster.

"The median gap was only one second, thanks in part to the great network conditions," he concluded.