Image courtesy cicada
You've heard my express my concerns over the benchmark tool Quadrant recently.
Well, I decided to put it to the test, and in doing so, I've set some records. Read on past the break!
I've been spending a lot of time hanging out with various XDA-Developers in IRC recently, and I've seen them echo those frustrations. I've even watched a rather heated argument between two developers break out on a development thread over the fact that tweaks to improve Quadrant scores have no impact on actual performance.
One of those developers, Dameon87, decided to prove his point by enlisting the assistance of another well-known Epic developer, noobnl, to add to his latest ROM release one such tweak. He wanted to prove how simple it was to exploit Quadrant, and with noobnl's help, how dramatically those scores could be impacted.
I've been using Dameon's mod (a modded beta FroYo leak that has been a lot of fun to use) for a few days and decided to see what kind of Quadrant scores I could manage with his latest version. I upgraded his ROM after release this morning, and was able to hit a Quadrant score of 2597. I posted this score to SmartphoneBenchmarks.com, where it is the 8th highest score ever recorded and the very highest score for an Android phone with a stock clock speed. In addition, it is the highest Epic 4G score recorded, by more than double the previous record-holder.
I've posted about this score both on SmartphoneBenchmarks.com as well as XDA-Developer forums.
On SmartphoneBenchmarks, an admin replied to my post about my score (in which I expressed my concerns with Quadrant) with a candid response stating that they're working on their own benchmark tools to try to alleviate the problem by providing an alternate benchmark.
On XDA-Developer forums, noobnl replied to the thread I started there confirming that the hack he helped implement into the ROM does nothing to improve actual phone performance and is merely a way of inflating Quadrant scores.
So seriously everyone, don't use Quadrant. Use GLBenchmark instead...