It is a bit shocking to see manufacturers implementing benchmark cheating in light of how bad the backlash was last time it was attempted (with some benchmarks completely excluding cheating devices from their performance lists). Thankfully, manufacturers have become increasingly responsive to issues like this, and with the right attention being drawn to it, this can be fixed quickly. Unfortunately, we must report that some OEMs have started cheating again, meaning we should be on the lookout once more. You might still get a couple percent, but is it really worth it?" - John Poole "The problem is that once we have these large runtimes if you start gaming things by ramping up your clock speeds or disabling governors or something like that, you're going to start putting actual real danger in the phone. Reducing the benefits in order to ensure that the development costs of benchmark cheating aren't worth it. Primate Labs in particular made Geekbench 4 quite a bit longer than Geekbench 3, in part to reduce the effects of benchmark cheating. When we interviewed John Poole, the creator of Geekbench, the topic of benchmark cheating and what companies like Primate Labs can do to prevent it came up. Many benchmarks were made longer so that the thermal throttling from maximizing clock speeds would become immediately apparent. Most of the few that didn't stop at that point stopped soon after, as there were substantial changes made to how many benchmarks run, in an attempt to discourage benchmark cheating (by reducing the benefit from it). After a bit of a public berating (and some private conversations) from technology publications, industry leaders, and the general public, most manufacturers got the message that benchmark cheating was simply not acceptable, and stopped as a result. Most benchmarks aren't there to tell you the theoretical maximum performance of a phone in lab conditions that aren't reproducible in day to day use, but rather they are there to give you a point of reference for real world comparisons between phones. There was substantial outrage when it was discovered, as these attempts at benchmark cheating ran counter to the very point of the benchmarks themselves. These development efforts ran the whole gamut, from setting clock speed floors, to forcing the clock speeds to their maximum settings, to even creating special higher power states and special clock speeds that were only available when benchmarking, with these efforts often resulting in just a couple percentage point increases in benchmark. They were all investing time and money into attempts to eke a little bit extra performance out of their phones in benchmarks, in ways that wouldn't have any positive effect on everyday usage, in an attempt to fool users into thinking that their phones were faster than they actually were. At the time, the investigation found that almost every manufacturer except for Google/Motorola were engaging in benchmark cheating. OEMs of all sizes (including Samsung, HTC, Sony, and LG) took part in this arms race of attempting to fool users without getting caught, but thankfully they eventually stopped their benchmark cheating after some frank discussions with industry experts and journalists.īack in 2013, it was discovered that the Samsung was artificially boosting its GPU clock speeds in certain applications, sparking a series of investigations into benchmark cheating across the whole range of manufacturers. Read More.Ī few years ago there was a considerable uproar, when numerous major manufacturers were caught cheating on benchmarks. When you make a purchase using links on our site, we may earn an affiliate commission. Readers like you help support XDA Developers.
0 Comments
Leave a Reply. |
AuthorWrite something about yourself. No need to be fancy, just an overview. ArchivesCategories |