As we discussed last week, we’ve learned that MobileXPRT 3 is crashing after installation
on some Android 11 phones. We now know what is causing this behavior: changes
to the storage strategy and permissions requirements in Android 11. The new file
storage strategy is called scoped storage. By default, scoped storage restricts an app’s storage access
to app-specific directories and media, and prohibits general access to external
or public directories. This default setting requires an opt-in permissions
prompt that MobileXPRT 3 does not currently have. It also prevents third-party
apps such as email clients or file managers from being accessing MobileXPRT 3
To fix this, we are
planning to rebuild MobileXPRT 3 to (1) point all of the benchmark’s file
references to its private directory, and (2) allow users to zip results files
and attach them to results submission emails. While we do not expect that either
of these changes will affect performance results, we’ll perform testing to
confirm this before we publish the new build.
We don’t expect these changes to take too long, and will keep you updated here in the blog. If you have any questions about the update process or MobileXPRT 3, please let us know!
We recently received a tech support inquiry about problems with new MobileXPRT 3 installations on some Android 11 phones. The tester installed MobileXPRT 3 on a selection of phones running Android 11, and the app crashed immediately upon opening. We were able to reproduce the issue on multiple phones in our lab, and currently know that the issue may happen on the Google Pixel 3, Google Pixel 4a 5G, Google Pixel 4XL, Google Pixel 5, and the OnePlus 8T (running Android 11 with an Oxygen OS skin).
MobileXPRT 3 continues
to run without issues on Android 9 and 10 phones. When we updated an Android 10
phone with an existing MobileXPRT 3 installation to Android 11, we found that
the benchmark ran successfully. This suggests a lack of fundamental
incompatibilities between MobileXPRT 3 and current versions of Android 11. Because
some of our lab techs experienced crashes immediately after the app asked for
permissions, we think it’s possible that new permissions-setting requirements
in Android 11 are causing the problem.
working to isolate the problem and identify a course of action. We’ll share
more information here in the blog as soon as possible. If you’ve encountered
this problem in your testing, we apologize for the inconvenience, and we’re
thankful for your patience as we work towards a solution.
If you have any information you’d like to share about running MobileXPRT 3 on Android 11, please let us know!
We appreciate everyone’s
patience and feedback during the CrXPRT 2 development process. We’re excited to
say that we’re now wrapping up some final details and expect to release the
Community Preview (CP) within the next week.
Here is a summary of the key
differences between CrXPRT 2015 and CrXPRT 2:
we mentioned a few weeks ago, CrXPRT 2 has a completely
new UI in line with the functional and aesthetic themes we used for the latest
versions of WebXPRT, MobileXPRT, and HDXPRT, with a focus on intuitive
CrXPRT 2 performance test includes six of the seven workloads in CrXPRT 2015.
Newer versions of Chrome can’t run the Photo Collage workload without a workaround, so we removed it from CrXPRT
updated the images in the Photo Effects and Face Detection workloads to reflect
more contemporary file resolutions and sizes.
CrXPRT 2 battery life test requires a full rundown, so you’ll need charge your device
to 100 percent before you can start the test, and the length of battery life
tests will vary according to the battery life of the systems under test.
no longer require testers to enter luminance and audio measurements in order to
run a battery life test.
added a second video playback segment to each battery life iteration.
allow testers to publish CP test scores, but CrXPRT 2 overall performance test scores
and battery life measurements are not comparable to CrXPRT 2015 scores.
Only BenchmarkXPRT Development Community members will be able to download the CrXPRT 2 CP. Because the Chrome team stopped providing search and browse functions for hosted and packaged Chrome apps in the Chrome Web Store, members will need a direct link to access the app. Once the app is available, we’ll post that link, along with the CrXPRT 2 CP user manual, on the CrXPRT tab in the XPRT Members’ Area (login required). We’ll also send a message to the community and post a notice here in the blog.
If you have any questions about CrXPRT
2 or joining the community,
please let us know!
Black Friday and Cyber Monday are almost here, and you may be feeling overwhelmed by the sea of tech gifts to choose from. The XPRTs are here to help. We’ve gathered the product specs and performance facts for some of the hottest tech devices in one convenient place—the XPRT Spotlight Black Friday Showcase. The Showcase is a free shopping tool that provides side-by-side comparisons of some of the season’s most popular smartphones, laptops, Chromebooks, tablets, and PCs. It helps you make informed buying decisions so you can shop with confidence this holiday season.
Want to know how the Google Pixel 4 stacks up against the Apple iPhone 11 or Samsung Galaxy Note10 in web browsing performance or screen size? Simply select any two devices in the Showcase and click Compare. You can also search by device type if you’re interested in a specific form factor such as consoles or tablets.
The Showcase doesn’t go away after Black Friday. We’ll rename it the XPRT Holiday Showcase and continue to add devices such as the Microsoft Surface Pro X throughout the shopping season. Be sure to check back in and see how your tech gifts measure up.
If this is the first you’ve heard about the XPRT Tech Spotlight, here’s a little background. Our hands-on testing process equips consumers with accurate information about how devices function in the real world. We test devices using our industry-standard BenchmarkXPRT tools: WebXPRT, MobileXPRT, TouchXPRT, CrXPRT, BatteryXPRT, and HDXPRT. In addition to benchmark results, we include photographs, specs, and prices for all products. New devices come online weekly, and you can browse the full list of almost 200 that we’ve featured to date on the Spotlight page.
If you represent a device vendor and want us to feature your product in the XPRT Tech Spotlight, please visit the website for more details.
We’re excited to announce that the MobileXPRT 3 source code is now available to BenchmarkXPRT Development Community members!
Download the MobileXPRT 3 source here (login required).
We’ve also posted a download link on the MobileXPRT tab in the Members’ Area, where you will find instructions for setting up and configuring a local instance of MobileXPRT 3.
As part of our community model for software development, source code for each of the XPRTs is available to anyone who joins the community. If you’d like to review XPRT source code, but haven’t yet joined the community, we encourage you to join! Registration is quick and easy, and if you work for a company or organization with an interest in benchmarking, you can join the community for free. Simply fill out the form with your company e-mail address and select the option to be considered for a free membership. We’ll contact you to verify the address and then activate your membership.
If you have any other questions about community membership or XPRT source code, feel free to contact us. We look forward to hearing from you!
Today, we published an updated MobileXPRT 3 build, version 188.8.131.52, on MobileXPRT.com and in the Google Play Store. The new build fixes an issue that was causing crashes on Xiaomi phones. Xiaomi holds significant market share in China, so we wanted to address the issue as soon as possible.
Xiaomi phones use a proprietary Android-based firmware called MIUI, which requires apps to communicate with the system in a specific way. When we originally built MobileXPRT 3, Android allowed an app’s code to send implicit messages calling certain classes of actions. In MIUI, the code must broadcast explicit messages that call the exact action necessary (e.g., waking from sleep). The requirement can improve security by allowing more granular levels of user control, and save power by restricting the number of unseen tasks that apps can run in the background without a user’s knowledge. The new MobileXPRT 3 build code complies with MIUI’s requirements. Other Android-based platforms will likely require explicit messages in the near future, so we’re hoping the new build will be relatively future proof.
We also fixed a few small UI bugs and improved the accuracy of the system hardware information that the app reports when a user submits a set of results. None of these changes affect performance, so scores from prior MobileXPRT 3 builds are comparable to those from the new build. If you have any questions or comments about MobileXPRT 3, please let us know.