BenchmarkXPRT Blog banner

Category: Android 11

The XPRTs in 2020: a year to remember

Justin Greene
By Justin Greene
on December 17, 2020

As 2020 comes to a close, we want to take this opportunity to review another productive year for the XPRTs. Readers of our newsletter are familiar with the stats and updates we include each month, but for our blog readers who don’t receive the newsletter, we’ve compiled some highlights below.

Benchmarks
In the past year, we released CrXPRT 2 and updated MobileXPRT 3 for testing on Android 11 phones. The biggest XPRT benchmark news was the release of CloudXPRT v1.0 and v1.01. CloudXPRT, our newest  benchmark, can accurately measure the performance of cloud applications deployed on modern infrastructure-as-a-service (IaaS) platforms, whether those platforms are paired with on-premises, private cloud, or public cloud deployments. 

XPRTs in the media
Journalists, advertisers, and analysts referenced the XPRTs thousands of times in 2020, and it’s always rewarding to know that the XPRTs have proven to be useful and reliable assessment tools for technology publications such as AnandTech, ArsTechnica, Computer Base, Gizmodo, HardwareZone, Laptop Mag, Legit Reviews, Notebookcheck, PCMag, PCWorld, Popular Science, TechPowerUp, Tom’s Hardware, VentureBeat, and ZDNet.

Downloads and confirmed runs
So far in 2020, we’ve had more than 24,200 benchmark downloads and 164,600 confirmed runs. Our most popular benchmark, WebXPRT, just passed 675,000 runs since its debut in 2013! WebXPRT continues to be a go-to, industry-standard performance benchmark for OEM labs, vendors, and leading tech press outlets around the globe.

Media, publications, and interactive tools
Part of our mission with the XPRTs is to produce materials that help testers better understand the ins and outs of benchmarking in general and the XPRTs in particular. To help achieve this goal, we’ve published the following in 2020:

We’re thankful for everyone who has used the XPRTs, joined the community, and sent questions and suggestions throughout 2020. This will be our last blog post of the year, but there’s much more to come in 2021. Stay tuned in early January for updates!

Justin

Posted in AIXPRT, AnandTech, Android 11, benchmark, Benchmarking, BenchmarkXPRT, Cloud, CloudXPRT, CrXPRT, IaaS, Interactive tools, MobileXPRT, MobileXPRT 3, Performance benchmarking, Phones, WebXPRT | Tagged , , , , , , , , , , |

We’ve updated MobileXPRT 3 to address issues with Android 11

Justin Greene
By Justin Greene
on December 3, 2020

This week, we published an updated MobileXPRT 3 build, version 3.116.0.4, on MobileXPRT.com and in the Google Play Store. The new build addresses an issue we recently discovered, where MobileXPRT was crashing after installation on some Android 11 phones. Permissions requirements and a new storage strategy called scoped storage were causing the problem. By default, scoped storage restricts an app’s storage access to app-specific directories and media, and prohibits general access to external or public directories. It also prevents third-party apps such as email clients or file managers from accessing MobileXPRT 3 results files. This default setting requires an opt-in permissions prompt that MobileXPRT 3 did not have prior to this week’s release.

MobileXPRT 3.116.0.4 points all of the benchmark’s file references to its private directory and allows users to zip results files and attach them to results submission emails. Neither change affects the testing process or test scores. If you have any questions or comments about the new MobileXPRT 3 build, please let us know!

Justin

Posted in Android, Android 11, benchmark, Google, Performance benchmarking, Performance testing on tablets, Phones, Storage | Tagged , , , , , |

Following up with MobileXPRT 3 on Android 11

Justin Greene
By Justin Greene
on October 22, 2020

As we discussed last week, we’ve learned that MobileXPRT 3 is crashing after installation on some Android 11 phones. We now know what is causing this behavior: changes to the storage strategy and permissions requirements in Android 11. The new file storage strategy is called scoped storage. By default, scoped storage restricts an app’s storage access to app-specific directories and media, and prohibits general access to external or public directories. This default setting requires an opt-in permissions prompt that MobileXPRT 3 does not currently have. It also prevents third-party apps such as email clients or file managers from being accessing MobileXPRT 3 results files.

To fix this, we are planning to rebuild MobileXPRT 3 to (1) point all of the benchmark’s file references to its private directory, and (2) allow users to zip results files and attach them to results submission emails. While we do not expect that either of these changes will affect performance results, we’ll perform testing to confirm this before we publish the new build.

We don’t expect these changes to take too long, and will keep you updated here in the blog. If you have any questions about the update process or MobileXPRT 3, please let us know!

Justin

Posted in Android 11, benchmark, Mobile devices, MobileXPRT, MobileXPRT 3, Phones, Tablets | Tagged , , , , |

New MobileXPRT 3 installations may crash on Android 11

Justin Greene
By Justin Greene
on October 15, 2020

We recently received a tech support inquiry about problems with new MobileXPRT 3 installations on some Android 11 phones. The tester installed MobileXPRT 3 on a selection of phones running Android 11, and the app crashed immediately upon opening. We were able to reproduce the issue on multiple phones in our lab, and currently know that the issue may happen on the Google Pixel 3, Google Pixel 4a 5G, Google Pixel 4XL, Google Pixel 5, and the OnePlus 8T (running Android 11 with an Oxygen OS skin).

MobileXPRT 3 continues to run without issues on Android 9 and 10 phones. When we updated an Android 10 phone with an existing MobileXPRT 3 installation to Android 11, we found that the benchmark ran successfully. This suggests a lack of fundamental incompatibilities between MobileXPRT 3 and current versions of Android 11. Because some of our lab techs experienced crashes immediately after the app asked for permissions, we think it’s possible that new permissions-setting requirements in Android 11 are causing the problem.

We’re currently working to isolate the problem and identify a course of action. We’ll share more information here in the blog as soon as possible. If you’ve encountered this problem in your testing, we apologize for the inconvenience, and we’re thankful for your patience as we work towards a solution.

If you have any information you’d like to share about running MobileXPRT 3 on Android 11, please let us know!

Justin

Posted in Android, Android 10, Android 11, Android 9, benchmark, Benchmarking, Google, Mobile devices, MobileXPRT, MobileXPRT 3, Performance benchmarking, Phones, Tablets | Tagged , , , , , , , , , , |

Check out the other XPRTs:

Forgot your password?
© Principled Technologies, Inc. 2023   |   Privacy policy

Cookie Notice: Our website uses cookies to deliver a smooth experience by storing logins and saving user information. By continuing to use our site, you agree with our usage of cookies as our privacy policy outlines.

x