BenchmarkXPRT Blog banner

Category: Phones

The XPRTs in 2020: a year to remember

As 2020 comes to a close, we want to take this opportunity to review another productive year for the XPRTs. Readers of our newsletter are familiar with the stats and updates we include each month, but for our blog readers who don’t receive the newsletter, we’ve compiled some highlights below.

Benchmarks
In the past year, we released CrXPRT 2 and updated MobileXPRT 3 for testing on Android 11 phones. The biggest XPRT benchmark news was the release of CloudXPRT v1.0 and v1.01. CloudXPRT, our newest  benchmark, can accurately measure the performance of cloud applications deployed on modern infrastructure-as-a-service (IaaS) platforms, whether those platforms are paired with on-premises, private cloud, or public cloud deployments. 

XPRTs in the media
Journalists, advertisers, and analysts referenced the XPRTs thousands of times in 2020, and it’s always rewarding to know that the XPRTs have proven to be useful and reliable assessment tools for technology publications such as AnandTech, ArsTechnica, Computer Base, Gizmodo, HardwareZone, Laptop Mag, Legit Reviews, Notebookcheck, PCMag, PCWorld, Popular Science, TechPowerUp, Tom’s Hardware, VentureBeat, and ZDNet.

Downloads and confirmed runs
So far in 2020, we’ve had more than 24,200 benchmark downloads and 164,600 confirmed runs. Our most popular benchmark, WebXPRT, just passed 675,000 runs since its debut in 2013! WebXPRT continues to be a go-to, industry-standard performance benchmark for OEM labs, vendors, and leading tech press outlets around the globe.

Media, publications, and interactive tools
Part of our mission with the XPRTs is to produce materials that help testers better understand the ins and outs of benchmarking in general and the XPRTs in particular. To help achieve this goal, we’ve published the following in 2020:

We’re thankful for everyone who has used the XPRTs, joined the community, and sent questions and suggestions throughout 2020. This will be our last blog post of the year, but there’s much more to come in 2021. Stay tuned in early January for updates!

Justin

We’ve updated MobileXPRT 3 to address issues with Android 11

This week, we published an updated MobileXPRT 3 build, version 3.116.0.4, on MobileXPRT.com and in the Google Play Store. The new build addresses an issue we recently discovered, where MobileXPRT was crashing after installation on some Android 11 phones. Permissions requirements and a new storage strategy called scoped storage were causing the problem. By default, scoped storage restricts an app’s storage access to app-specific directories and media, and prohibits general access to external or public directories. It also prevents third-party apps such as email clients or file managers from accessing MobileXPRT 3 results files. This default setting requires an opt-in permissions prompt that MobileXPRT 3 did not have prior to this week’s release.

MobileXPRT 3.116.0.4 points all of the benchmark’s file references to its private directory and allows users to zip results files and attach them to results submission emails. Neither change affects the testing process or test scores. If you have any questions or comments about the new MobileXPRT 3 build, please let us know!

Justin

The XPRTs can help with your holiday shopping

The biggest shopping days of the year are fast approaching, and if you’re researching phones, tablets, Chromebooks, or laptops in preparation for Black Friday and Cyber Monday sales, the XPRTs can help! One of the core functions of the XPRTs is to help cut through all the marketing noise by providing objective, reliable measures of a device’s performance. For example, instead of trying to guess whether a new Chromebook is fast enough to handle the demands of remote learning, you can use its CrXPRT and WebXPRT performance scores to see how it stacks up against the competition when handling everyday tasks.

A good place to start your search for scores is our XPRT results browser. The browser is the most efficient way to access the XPRT results database, which currently holds more than 2,600 test results from over 100 sources, including major tech review publications around the world, OEMs, and independent testers. It offers a wealth of current and historical performance data across all the XPRT benchmarks and hundreds of devices. You can read more about how to use the results browser here.

Also, if you’re considering a popular device, chances are good that someone has already published an XPRT score for that device in a recent tech review. The quickest way to find these reviews is by searching for “XPRT” within your favorite tech review site, or by entering the device name and XPRT name (e.g. “Apple iPad” and “WebXPRT”) in a search engine. Here are a few recent tech reviews that use one or more of the XPRTs to evaluate a popular device:


The XPRTs can help consumers make better-informed and more confident tech purchases this holiday season, and we hope you’ll find the data you need on our site or in an XPRT-related tech review. If you have any questions about the XPRTs, XPRT scores, or the results database please feel free to ask!

Justin

Following up with MobileXPRT 3 on Android 11

As we discussed last week, we’ve learned that MobileXPRT 3 is crashing after installation on some Android 11 phones. We now know what is causing this behavior: changes to the storage strategy and permissions requirements in Android 11. The new file storage strategy is called scoped storage. By default, scoped storage restricts an app’s storage access to app-specific directories and media, and prohibits general access to external or public directories. This default setting requires an opt-in permissions prompt that MobileXPRT 3 does not currently have. It also prevents third-party apps such as email clients or file managers from being accessing MobileXPRT 3 results files.

To fix this, we are planning to rebuild MobileXPRT 3 to (1) point all of the benchmark’s file references to its private directory, and (2) allow users to zip results files and attach them to results submission emails. While we do not expect that either of these changes will affect performance results, we’ll perform testing to confirm this before we publish the new build.

We don’t expect these changes to take too long, and will keep you updated here in the blog. If you have any questions about the update process or MobileXPRT 3, please let us know!

Justin

New MobileXPRT 3 installations may crash on Android 11

We recently received a tech support inquiry about problems with new MobileXPRT 3 installations on some Android 11 phones. The tester installed MobileXPRT 3 on a selection of phones running Android 11, and the app crashed immediately upon opening. We were able to reproduce the issue on multiple phones in our lab, and currently know that the issue may happen on the Google Pixel 3, Google Pixel 4a 5G, Google Pixel 4XL, Google Pixel 5, and the OnePlus 8T (running Android 11 with an Oxygen OS skin).

MobileXPRT 3 continues to run without issues on Android 9 and 10 phones. When we updated an Android 10 phone with an existing MobileXPRT 3 installation to Android 11, we found that the benchmark ran successfully. This suggests a lack of fundamental incompatibilities between MobileXPRT 3 and current versions of Android 11. Because some of our lab techs experienced crashes immediately after the app asked for permissions, we think it’s possible that new permissions-setting requirements in Android 11 are causing the problem.

We’re currently working to isolate the problem and identify a course of action. We’ll share more information here in the blog as soon as possible. If you’ve encountered this problem in your testing, we apologize for the inconvenience, and we’re thankful for your patience as we work towards a solution.

If you have any information you’d like to share about running MobileXPRT 3 on Android 11, please let us know!

Justin

A new MobileXPRT 3 build is available

Today, we published an updated MobileXPRT 3 build, version 3.114.2.1, on MobileXPRT.com and in the Google Play Store. The new build fixes an issue that was causing crashes on Xiaomi phones. Xiaomi holds significant market share in China, so we wanted to address the issue as soon as possible.

Xiaomi phones use a proprietary Android-based firmware called MIUI, which requires apps to communicate with the system in a specific way. When we originally built MobileXPRT 3, Android allowed an app’s code to send implicit messages calling certain classes of actions. In MIUI, the code must broadcast explicit messages that call the exact action necessary (e.g., waking from sleep). The requirement can improve security by allowing more granular levels of user control, and save power by restricting the number of unseen tasks that apps can run in the background without a user’s knowledge. The new MobileXPRT 3 build code complies with MIUI’s requirements. Other Android-based platforms will likely require explicit messages in the near future, so we’re hoping the new build will be relatively future proof.

We also fixed a few small UI bugs and improved the accuracy of the system hardware information that the app reports when a user submits a set of results. None of these changes affect performance, so scores from prior MobileXPRT 3 builds are comparable to those from the new build. If you have any questions or comments about MobileXPRT 3, please let us know.

Justin

Check out the other XPRTs:

Forgot your password?