Time for honesty -- Samsung seems to have stopped fiddling with benchmark figures
Benchmarks are important. With so much choice in the world of computers, smartphones and tablets, a key factor for potential buyers to bear in mind is raw performance. A few months back benchmarking stalwarts Futuremark took the unusual step of delisting a number of handsets produced by HTC and Samsung after tests appeared to show that the phone artificially boosted performance when they detected benchmarking software was running. Now it looks as though this apparent cheating has come to an end.
Back in October, results published on Anantech showed how a number of popular phones seemed to be cheating the system, giving consumers a false representation of real-world handset performance. Now, according to new tests carried out by Ars Technica it would appear that handsets are behaving in a far more reasonable fashion after being updated to KitKat.
When tests were performed on a Samsung Galaxy S4 and Samsung Galaxy Note 3 running Android 4.3, the questionable benchmark results were noticeable. But running the same tests under Android 4.4 yielded somewhat different results. Rather than detecting the presence of a benchmarking tool and putting the pedal to the metal with every single available core, processors were only used when it was necessary. It almost goes without saying that suspicious minds were still firing. John Poole from Primate Labs ran the tests and while the results appeared to show a lack of figure massaging, he was initially skeptical.
No artificial speed boost was found. He "did not detect a boost under 4.4.2, which leads me to believe that Samsung has disabled the benchmark boost in Android 4.4.2. I was concerned that the boost became more sophisticated and was able to evade the detector." But it does appear that benchmarking tools are now treated in just the same way as any other app, rather than being singled out for special treatment.
What do you make of this? Is it too little too late? Perhaps you don’t care whether benchmark results are inflated and are quite happy to judge performance for yourself. Do you think Samsung's reputation has been damaged by the whole debacle... and is this move enough to save face?