This site may earn affiliate commissions from the links on this page. Terms of use.

Huawei-Mate20-Feature

The Chinese manufacturer Huawei has been distributing devices with clandestine benchmark modes enabled, drastically misrepresenting the functioning of these devices and inflating their test results. It's far from the first time nosotros've been down this road, either with benchmarking or mobile devices, but what Huawei has done is a little unusual.

Anandtech stumbled on this upshot while testing multiple Huawei devices and confirmed the device behavior by using individual versions of the public benchmarks y'all tin download online. These private versions are code-identical to the standard variants, but use obfuscated names or hide other details to prevent a smartphone from detecting they are running and artificially boosting performance.

These tests focused on the Kirin 970 SoC, which powers the Huawei P20 Pro, the P20, and the Huawei Honor PlaySEEAMAZON_ET_135 See Amazon ET commerce. Every bit the chart shows, running the standard version of the test allowed the SoC to blow direct through its power envelope, with an average power consumption of 8.57W. That'south unsustainably loftier for even a high-end smartphone. Modern devices may flare-up up to that kind of power envelope for extremely brief periods, simply even the largest phones would quickly become also hot to concord if run at that TDP for whatsoever length of time.

As Anandtech explains, what Huawei does is a little dissimilar from what other SoC vendors have washed in the by. When we've seen these problems before, information technology's typically been because smartphone vendors were ignoring their own thermal guidelines and throttling beliefs in specific contexts and running the SoC total-tilt equally long as possible to produce better benchmark scores. But Huawei is approaching the concept from a different angle.

During standard operation, the device'south performance is sharply constrained to hold within its thermal envelope. The Kirin 970 never approaches its advertised clocks equally a result. The simply time the scrap performs at its claimed specifications is when its running certain benchmarks. The only way to do that is to blow out the SoC's TDP. Anandtech measured the SoC drawing an average of 4.39W when tested using benchmarks where the phone couldn't crook, versus 8.57W when the device was allowed to observe the criterion and run at top clock. This performance is completely unsustainable over the long term (if tests are left running, the devices all start displaying overheating notices).

PpV0evc

Technically, one tin can't argue that Huawei is overclocking their SoC. Later all, mobile manufacturers regularly annunciate heave clocks every bit if they were baseline frequencies and they provide no information on when or if users will see those clocks. But the trouble hither isn't whether the Kirin 970 runs at any particular frequency — it's whether the performance end-users come across when they run a graphics criterion has anything to practice with the operation they'll run across when they run a game.

Huawei-Performance

The most basic, primal test of any benchmark is whether information technology captures an aspect of device performance or capability that can guide the end-user in evaluating said device. By operating its smartphone in a different mode when these tests are run, Huawei has fabricated it incommunicable for tests like GFXBench to provide authentic data. Information technology ultimately doesn't matter whether the SoC is exceeding its published clock speeds or non. What matters, fundamentally, is that the device is operating in a different manner when and simply when cease users are attempting to approximate its relative performance next to other devices they might purchase instead.

Fighting Cheaters

According to Huawei, these changes are the result of decisions the visitor made to rein in power consumption on its devices and amend battery life. Implementing throttling and strict TDP limitations in almost all use cases tin can improve battery life while having a minimal affect on the user experience, provided the clocks are well-calibrated. Since upgrading to an iPhone SE before this spring, I actually keep the device in low power mode almost all the time. Yep, I take a small-scale performance hitting — but the phone battery lasts far longer, overall battery health should drop less quickly, and the bodily impact of the lowered CPU clock is almost unnoticeable.

The problem with Huawei'due south changes is that the company attempted to consume its cake and have it, besides. According to Huawei, it took these steps because benchmark cheating in Mainland china is simply rampant and the company feels it must appoint in some of this behavior as well to compete against other firms. The question of how to compete effectively in an industry when your competition lies regularly about the operation of its own products is a genuine one, and in that location are no easy answers to that problem. Only engaging in the aforementioned activeness ultimately does not serve the best interests of consumers.

Now Read: Samsung Goes Legit, Stops Cheating on Android Benchmarks, Tests Announced to Show the OnePlus 5 Cheating on Tests, and Lies, Damned Lies, and Benchmarks