It's that time of the year the new iPhones have dropped and our subscription boxes have been filled with reviews, camera comparisons and a whole host of other tech related videos. One of the genres of tech videos that I enjoy the most is the real world speed test in which two of the hottest new phones are pitted against each other. With them having to blaze through the same set of apps that you might expect the average Joe to sue over the course of his day. After all seeing how a phone handles the apps that you use almost daily would be a great way of telling how fast and responsive it is? Thats the intuitive answer it feels right but unfortunately thats not the case.
Speed Tests Are Inherently Flawed
The problems with the so called "Real World" test beings with the very moniker itself. How do you define the phrase "Real World Test"? There is no universal baseline that says that everyone in the world uses their phone in a particular manner. Leading to the conclusion that every one uses their phone differently. You might be into social media apps, your friend might be all about Netflix and your dad may be all about texting and calls. All valid use cases and all wildly different. So the tests themselves aren't testing the things the majority of the population are going to be using their phone for. You never see a speed test play a Netflix video, you never see them go into a game not do you see them type out a text. I fail to see how this is real world test when they omit all of the real world tasks associated with it.
Opening An App Does Not Constitute A Real World Test
In most speed test the first round is when you see how two phones open apps. Key word open. rarely in real life do you open an app and then instantly go the next and so on. You usually open the app and you stay there for a while. Besides opening an app barley taxes the CPU at all.You'd think that opening an app would put the Cpu through it's paces but opening an app just shows you how fast the phone can move data from it's memory into it's ram. Basically it's an I/O test. using the app how ever taxes the CPU and puts it through it's spaces. The only phase in a speed test that I think actually utilises the CPU is the video exporting test. In that case the CPU does have to work extra hard to render the video into the memory.
All Apps Aren't Created Equally
This is especially true for apps on two platforms, but despite having the same logo and the same name slapped on them it is very much likely that these two apps are made in entirely two different ways. They might have been developed by two different teams using two different methods. So if you were to compare Netflix on Android and iOS you're basically comparing two different apps.The underlying technologies might be so different, how they manage RAM, how they load dependencies, how they utilise memory, Its a world of change.You might as well compare Snapchat and Uber.
Hooking Them Up To The Same WiFi Is Useless
In an attempt to keep things fair hosts will often hook the devices up to the same wifi point. I mean of they're connected to the same wifi point then they're getting the exact same internet speed. Yes but there's a catch. After the data or to be technical the packets leave your router they might take wildly different routes to get to their destination and its all dependent upon chance. You might take the shortest route or you might take a circuitous route. This is further exacerbated by the fact that most network apps don't even load the same data, In most speed tests you see that a different version of the website loads up on both devices. This also introduces a level of uncertainty into the equation. A phone might have to load more or less content depending on the page it gets served. Despite trying to provide a level playing field the inherent structure of how our apps are built and the internet makes it hard to have a truly fair test on speed.But there is one method that is pretty good at measuring a phone's performance.
Sometimes Apps Are Dropped From The RAM For A Purpose
The utmost disappointment for any phone is when it fails to keep an app open in the background and often it is based on poor RAM management by the OEM. Sure it'd be nice if all apps stayed open all the time but thats being naive.It's possible that the software is dropping apps from memory or killing them on purposed and before you gasp know that sometimes its necessary. With all our phones being battery powered electrical power comes at a premium.you have to conserve every bit of power that you can and keeping a large app, lets say a game draws on that power that phones don't have a lot of to begin with. At the most basic level RAM is just a concoction of transistors and gates that are constantly being fed engird all the while they store data. Dropping an app that the OS thinks that you're not gonna use any time soon is great way to save power. in a world where we clamour for longer battery lives this I feel is an acceptable tradeoff.
The Venerable Benchmark
Benchmarks were originally used to compare phones and they fell out of favour and I can see why you had this app that did all of this cryptic stuff and out popped a number. Its not hard to se why people thought that benchmarks weren't indicative of real world performance. But behind the scenes a benchmarks puts a phones hardware to the test in way that speed tests rarely do. They make the phone decompress files, do facial recognition stuff and a whole suite of other tests. The best part is that this is platform independent.Benchmarks have the exact same execution environment across platforms. I guess you could say that benchmarks are the closest thing we have to comparing phones on an apple to apple basis. Pun intended.
Comments
Post a Comment