BenchmarkXPRT Blog banner

Category: Phones

Following up with MobileXPRT 3 on Android 11

As we discussed last week, we’ve learned that MobileXPRT 3 is crashing after installation on some Android 11 phones. We now know what is causing this behavior: changes to the storage strategy and permissions requirements in Android 11. The new file storage strategy is called scoped storage. By default, scoped storage restricts an app’s storage access to app-specific directories and media, and prohibits general access to external or public directories. This default setting requires an opt-in permissions prompt that MobileXPRT 3 does not currently have. It also prevents third-party apps such as email clients or file managers from being accessing MobileXPRT 3 results files.

To fix this, we are planning to rebuild MobileXPRT 3 to (1) point all of the benchmark’s file references to its private directory, and (2) allow users to zip results files and attach them to results submission emails. While we do not expect that either of these changes will affect performance results, we’ll perform testing to confirm this before we publish the new build.

We don’t expect these changes to take too long, and will keep you updated here in the blog. If you have any questions about the update process or MobileXPRT 3, please let us know!

Justin

New MobileXPRT 3 installations may crash on Android 11

We recently received a tech support inquiry about problems with new MobileXPRT 3 installations on some Android 11 phones. The tester installed MobileXPRT 3 on a selection of phones running Android 11, and the app crashed immediately upon opening. We were able to reproduce the issue on multiple phones in our lab, and currently know that the issue may happen on the Google Pixel 3, Google Pixel 4a 5G, Google Pixel 4XL, Google Pixel 5, and the OnePlus 8T (running Android 11 with an Oxygen OS skin).

MobileXPRT 3 continues to run without issues on Android 9 and 10 phones. When we updated an Android 10 phone with an existing MobileXPRT 3 installation to Android 11, we found that the benchmark ran successfully. This suggests a lack of fundamental incompatibilities between MobileXPRT 3 and current versions of Android 11. Because some of our lab techs experienced crashes immediately after the app asked for permissions, we think it’s possible that new permissions-setting requirements in Android 11 are causing the problem.

We’re currently working to isolate the problem and identify a course of action. We’ll share more information here in the blog as soon as possible. If you’ve encountered this problem in your testing, we apologize for the inconvenience, and we’re thankful for your patience as we work towards a solution.

If you have any information you’d like to share about running MobileXPRT 3 on Android 11, please let us know!

Justin

A new MobileXPRT 3 build is available

Today, we published an updated MobileXPRT 3 build, version 3.114.2.1, on MobileXPRT.com and in the Google Play Store. The new build fixes an issue that was causing crashes on Xiaomi phones. Xiaomi holds significant market share in China, so we wanted to address the issue as soon as possible.

Xiaomi phones use a proprietary Android-based firmware called MIUI, which requires apps to communicate with the system in a specific way. When we originally built MobileXPRT 3, Android allowed an app’s code to send implicit messages calling certain classes of actions. In MIUI, the code must broadcast explicit messages that call the exact action necessary (e.g., waking from sleep). The requirement can improve security by allowing more granular levels of user control, and save power by restricting the number of unseen tasks that apps can run in the background without a user’s knowledge. The new MobileXPRT 3 build code complies with MIUI’s requirements. Other Android-based platforms will likely require explicit messages in the near future, so we’re hoping the new build will be relatively future proof.

We also fixed a few small UI bugs and improved the accuracy of the system hardware information that the app reports when a user submits a set of results. None of these changes affect performance, so scores from prior MobileXPRT 3 builds are comparable to those from the new build. If you have any questions or comments about MobileXPRT 3, please let us know.

Justin

BatteryXPRT provides the objective battery life data that shoppers need

Over the last few weeks, we’ve discussed the capabilities and benefits of TouchXPRT and CrXPRT. This week, we’d like to reintroduce readers to BatteryXPRT, our app that evaluates the battery life and performance of Android devices.

Battery life for phones and tablets has improved dramatically over the last several years, to the point where many devices can support continuous use for well over a full work day on a single charge. This improvement is the result of advances in battery hardware technology, increased processor efficiency, and smarter utilization of software services by the operating system. Battery life has increased to some extent for most device categories and price points. However, enough of a range remains between devices at each level that access to objective battery life data is valuable for device shoppers.

Without BatteryXPRT, shoppers must rely on manufacturer estimates or full rundown tests that don’t resemble the types of things we do with our phones and tablets every day. A rundown test that surfs the web continuously for over 15 hours reveals which devices last the longest performing that specific task. It doesn’t tell you which devices last the longest over a full day performing a variety of common activities such as web browsing, watching videos, browsing and editing photos, playing music, and periodically sleeping. During BatteryXPRT’s battery life test, the app executes those same types of tasks and produces a performance score based on the speed with which a device completes each task.

BatteryXPRT provides an intuitive user interface in English and Simplified Chinese, and easy-to-understand results for both battery life and performance. Because your data connection can have a significant effect on battery life, BatteryXPRT runs in airplane mode, connected to the Internet via Wi-Fi, or connected to the Internet through a cellular data connection.

BatteryXPRT is easy to install and run, and is a great resource for anyone who wants to evaluate how well an Android device will meet their needs. If you’d like to see test results from a variety of Android devices, go to BatteryXPRT.com and click View Results, where you’ll find scores from many different Android devices.

If you’d like to run BatteryXPRT

Simply download BatteryXPRT from the Google Play store or BatteryXPRT.com. The BatteryXPRT installation instructions and user manual provide step-by-step instructions for configuring your device and kicking off a test. We designed BatteryXPRT to be compatible with a wide variety of Android devices, but because there are so many devices on the market, it is inevitable that users occasionally run into problems. In the Tips, tricks, and known issues document, we provide troubleshooting suggestions for issues we encountered during development testing.

If you’d like to learn more

The Exploring BatteryXPRT 2014 for Android white paper covers almost every aspect of the benchmark. In it, we explain the guiding concepts behind BatteryXPRT’s development, as well as the benchmark’s structure. We describe the component tests, the differences between the app’s Airplane and Network/Wi-Fi modes, and the statistical processes used to calculate expected battery life.

Justin

More, faster, better: The future according to Mobile World Congress 2019

More is more data, which the trillions of devices in the coming Internet of Things will be pumping through our air into our (computing) clouds in hitherto unseen quantities.

Faster is the speed at which tomorrow’s 5G networks will carry this data—and the responses and actions from our automated assistants (and possibly overlords).

Better is the quality of the data analysis and recommendations, thanks primarily to the vast army of AI-powered analytics engines that will be poring over everything digital the planet has to say.

Swimming through this perpetual data tsunami will be we humans and our many devices, our laptops and tablets and smartphones and smart watches and, ultimately, implants. If we are to believe the promise of this year’s Mobile World Congress in Barcelona—and of course I do want to believe it, who wouldn’t?—the result of all of this will be a better world for all humanity, no person left behind. As I walked the show floor, I could not help but feel and want to embrace its optimism.

The catch, of course, is that we have a tremendous amount of work to do between where we are today and this fabulous future.

We must, for example, make sure that every computing node that will contribute to these powerful AI programs is up to the task. From the smartphone to the datacenter, AI will end up being a very distributed and very demanding workload. That’s one of the reasons we’ve been developing AIXPRT. Without tools that let us accurately compare different devices, the industry won’t be able to keep delivering the levels of performance improvements that we need to realize these dreams.

We must also think a lot about how to accurately measure all other aspects of our devices’ performance, because the demands this future will place on them are going to be significant. Fortunately, the always evolving XPRT family of tools is up to the task.

The coming 5G revolution, like all tech leaps forward before it, will not come evenly. Different 5G devices will end up behaving differently, some better and some worse. That fact, plus our constant and growing reliance on bandwidth, suggests that maybe the XPRT community should turn its attention to the task of measuring bandwidth. What do you think?

One thing is certain: we at the Benchmark XPRT Development Community have a role to play in building the tools necessary to test the tech the world will need to deliver on the promise of this exciting trade show. We look forward to that work.

MobileXPRT 3 is here!

We’re excited to announce that MobileXPRT 3 is now available to the public! MobileXPRT 3 is the latest version of our popular tool for evaluating the performance of Android devices. The BenchmarkXPRT Development Community has been using a community preview for several weeks, but now anyone can run the tool and publish their results.

Compatible with systems running Android 5.0 and above, MobileXPRT 3 includes the same performance workloads as MobileXPRT 2015 (Apply Photo Effects, Create Photo Collages, Create Slideshow, Encrypt Personal Content, and Detect Faces to Organize Photos), plus a new optical character recognition-based workload called Scan Receipts for Spreadsheet.

MobileXPRT 3 is available at MobileXPRT.com and on the Google Play Store. Alternatively, you can download the app using either of the links below:



After trying out MobileXPRT 3, please submit your scores here and send any comments to BenchmarkXPRTsupport@principledtechnologies.com. To see test results from a variety of systems, go to MobileXPRT.com and click View Results, where you’ll find scores from a variety of Android devices. We look forward to seeing your results!

Justin

Check out the other XPRTs:

Forgot your password?