As we discussed last week, we’ve learned that MobileXPRT 3 is crashing after installation
on some Android 11 phones. We now know what is causing this behavior: changes
to the storage strategy and permissions requirements in Android 11. The new file
storage strategy is called scoped storage. By default, scoped storage restricts an app’s storage access
to app-specific directories and media, and prohibits general access to external
or public directories. This default setting requires an opt-in permissions
prompt that MobileXPRT 3 does not currently have. It also prevents third-party
apps such as email clients or file managers from being accessing MobileXPRT 3
To fix this, we are
planning to rebuild MobileXPRT 3 to (1) point all of the benchmark’s file
references to its private directory, and (2) allow users to zip results files
and attach them to results submission emails. While we do not expect that either
of these changes will affect performance results, we’ll perform testing to
confirm this before we publish the new build.
We don’t expect these changes to take too long, and will keep you updated here in the blog. If you have any questions about the update process or MobileXPRT 3, please let us know!
We recently received a tech support inquiry about problems with new MobileXPRT 3 installations on some Android 11 phones. The tester installed MobileXPRT 3 on a selection of phones running Android 11, and the app crashed immediately upon opening. We were able to reproduce the issue on multiple phones in our lab, and currently know that the issue may happen on the Google Pixel 3, Google Pixel 4a 5G, Google Pixel 4XL, Google Pixel 5, and the OnePlus 8T (running Android 11 with an Oxygen OS skin).
MobileXPRT 3 continues
to run without issues on Android 9 and 10 phones. When we updated an Android 10
phone with an existing MobileXPRT 3 installation to Android 11, we found that
the benchmark ran successfully. This suggests a lack of fundamental
incompatibilities between MobileXPRT 3 and current versions of Android 11. Because
some of our lab techs experienced crashes immediately after the app asked for
permissions, we think it’s possible that new permissions-setting requirements
in Android 11 are causing the problem.
working to isolate the problem and identify a course of action. We’ll share
more information here in the blog as soon as possible. If you’ve encountered
this problem in your testing, we apologize for the inconvenience, and we’re
thankful for your patience as we work towards a solution.
If you have any information you’d like to share about running MobileXPRT 3 on Android 11, please let us know!
Some time ago, we started to see unusual BatteryXPRT battery life estimates and high variance on some devices when running tests at the default length of 5.25 hours (seven 45-minute iterations). We suspected that the problem resulted from changes in how new OS versions report battery life on certain devices (e.g., charging past a reported level of 100 percent). In addition, the progress of battery technology in general means that the average phone battery lasts much longer than it did a few years ago. Together, these factors sometimes led to BatteryXPRT runs where the OS reported little to no battery decrease during the first few iterations of a test. We concluded that 5.25 hours wasn’t long enough to produce an accurate battery life estimate.
After extensive experimentation and testing, we’ve decided to release a new build that increases the default BatteryXPRT test length from 5.25 hours (seven iterations) to 45 hours (60 iterations) to allow enough time for a full rundown on most phones. Based on our testing, we consider full rundown tests to be the most accurate and will use those exclusively in our Spotlight testing and elsewhere. Testers will still have the option of choosing shorter test durations, but BatteryXPRT will flag the results with a qualifier that recommends performing a full rundown.
We plan to release the updated build by the end of next week and update BatteryXPRT documentation to reflect the changes. We have not changed any of the workloads and both performance results and full-rundown battery life estimates will be comparable to results from earlier builds.
BatteryXPRT continues to be a useful tool for gauging the performance and expected battery life of Android devices while simulating real-world tasks. If you have any questions about BatteryXPRT, please feel free to ask!
There’s a lot going on with the XPRTs, so we want to offer a quick update.
On the HDXPRT 4 front, we’re currently testing community preview candidate builds across a variety of laptops and desktops. Testing is going well, but as is often the case prior to a release, we’re still tweaking the code as necessary when we run into bugs. We’re excited about HDXPRT 4 and look forward to the community seeing how much faster and easier to use it is than previous versions. You can read more about what’s to come in HDXPRT 4 here.
On the MobileXPRT 3 front, proof-of-concept testing for the new and updated workloads went well, and we’re now working to implement the new UI. Below, you can see a mockup of the new MobileXPRT 3 start screen for phones. The aesthetic is completely different than MobileXPRT 2015, and is in line with the clean, bright look we used for WebXPRT 3 and HDXPRT 4. We’ve made it easy to select and deselect individual workloads by tapping the workload name (deselected workloads are grayed out), and we’ve consolidated common menu items into an Android-style taskbar at the bottom of the screen. Please note that this is an early view and some aspects of the screen will change. For instance, we’re certain that the final receipt-scanning workload won’t be called “Optical character recognition.”
We’ll share more information about HDXPRT 4 and MobileXPRT 3 in the coming weeks. If you have any questions about HDXPRT or MobileXPRT, or would like to share your ideas, please get in touch!