BenchmarkXPRT Blog banner

Month: April 2014

It’s all in the presentation

The comment period for BatteryXPRT CP2 ended on Monday. Now we are in the final sprint to release the benchmark.

The extensive testing we’ve been doing has meant that we’ve been staring at a lot of numbers. This has led us to make a change in how we present the results. As you would expect, the battery life when you’re running the test using Wi-Fi is different than when you’re running it using your cellular network. Although individual devices vary, the difference is in the vicinity of 10 percent, about the same as the difference between Airplane mode and using Wi-Fi.

BatteryXPRT has always captured a device’s Wi-Fi setting in its disclosure results, but had not included this information with the results. Because we found it so helpful to have the Wi-Fi setting alongside the results, we have changed the presentation of the results to recognize three modes: Airplane, Wi-Fi, and Cellular. We hope that this will avoid confusion as people are using BatteryXPRT.

Note that we have not changed the way the results are calculated. Results you generated during the preview are still valid. However, results from one mode should not be compared to results from another mode.

We’ve been talking a lot about BatteryXPRT, but TouchXPRT is also looking great! We’re looking forward to releasing both of them soon!

Eric

Comment on this post in the forums

Looking for the next big thing

We recently introduced a new Web form in the members’ area to make it easier for you to submit new benchmark ideas. We’ve already received some interesting suggestions:

  • A benchmark to assess performance, battery life, and Chrome-specific technologies on Chromebooks
  • A benchmark to evaluate camera features and photo quality on phones and tablets
  • A benchmark for measuring the performance of cloud services
  • A benchmark for measuring the performance and battery life of iOS-based devices

Are you interesting in seeing any of these? Or do you have an idea no one has mentioned yet? We know there’s more out there! We like finding new things to measure and new ways to measure them, so please don’t hesitate to share your ideas!

Also, remember that the comment period for BatteryXPRT 2014 for Android CP2 ends on Monday, April 21. CP2 is the first XPRT to feature a Simplified Chinese UI. Please send in your comments. We’ll be aiming for a BatteryXPRT general release soon.

By the way, if you have a language you’d like to see and you’re willing to help with the translation, we’d love to talk to you!

Let us know what you think about potential new benchmarks, language options, or anything else on your mind at BenchmarkXPRTSupport@principledtechnologies.com.

Eric

Comment on this post in the forums

Password reset issue

A couple of members had trouble resetting their passwords. Our technicians are working on a solution, but in the meantime, if you’ve encountered any problem with resetting your password, please contact us at BenchmarkXPRTsupport@principledtechnologies.com. We’ll be happy to manually reset your password and answer questions about any other concerns you may have.

BatteryXPRT 2014 for Android Community Preview 2 is available to community members!

As promised in Something new, we’re releasing the BatteryXPRT 2014 for Android Community Preview 2 (CP2) to demonstrate BatteryXPRT’s Simplified Chinese UI. See Community Preview 2 is available! for the details (login required).

The general release is coming soon, so this will be a really quick Community Preview. The comment period will last for a week, through Monday, April 21. Assuming there are no significant issues, we hope to release the final version shortly thereafter. Please let us know if you see any problems.

If you are not a community member, it’s easy to join!

Eric

Something new

We have seen significant interest from China in the XPRTs over the last year, with particular interest given to MobileXPRT. Given how important Android is in the Asian market, this not a surprise.

Our newest benchmark for Android devices is BatteryXPRT. While it’s still in Community Preview, we’re experimenting a bit with internationalization. On Monday, we’re going to release a second Community Preview with a UI in Simplified Chinese. Here’s what it looks like.

BattXPRT end test results Chinese                  BattXPRT detailed results Chinese

BatteryXPRT uses Android’s global language setting to determine whether to display in English or Simplified Chinese. Assuming your device is currently set to English, this is the setting you change by navigating to Settings → Language & input → Language. For those who are familiar with Android development, the new resources are in the res/drawable-zh-rCN and res/values-zh-rCN directories.

If your device is currently set up to use Simplified Chinese, and if everything is working correctly, BatteryXPRT will open with a Simplified Chinese UI. For any language other than Simplified Chinese, the application will default to English.

Because this is an experiment, the localization is not complete. For example, the User Manual is still in English.

We’re close to the general release, so this will be a really quick Community Preview. We’ll release CP2 to the community on Monday, April 14. We’ll take comments for a week, through Monday, April 21. Assuming the community does not find any significant issues, we hope to release the final version shortly thereafter. Please let us know if you see any problems.

We are very aware that this is a tiny first step on the road to internationalization. We’re currently researching what it would take to localize our various assets such as the documentation and Web sites. If you have any comments or ideas, please contact us at BenchmarkXPRTsupport@principledtechnologies.com.

And if you’d like to try your hand at translating the user manual, let us know!

Eric

Comment on this post in the forums

What is truth?

Last fall, we discussed AnandTech’s report on benchmark cheating, and why open and honest benchmark development is so important. This week, benchmark optimization is back in the news, as AnandTech says that the HTC One M8 boosts its performance in whitelisted benchmarks. CNET has quoted HTC admitting that they not only boosted performance, but promoted the boost as a feature.

However, HTC has gone a step further, giving users the option to manually set the phone to high-performance mode. Some of us at PT have been involved in developing benchmarks for over 20 years, and it’s always been true that one person’s cheat can be another person’s valid optimization. Whatever their motivation, HTC’s position – that some people will choose higher performance over longer battery life – is not necessarily wrong.

BatteryXPRT recognizes that there’s a tradeoff between performance and battery life, and that you shouldn’t penalize a fast system the same way you would a system that simply has poor battery life. That’s why it reports a performance score along with the estimated battery life.

Do you have thoughts on optimizations, cheating, or ways to make the benchmarks better? Please drop us a line at BenchmarkXPRTSupport@principledtechnologies.com.

Eric

Comment on this post in the forums

Check out the other XPRTs:

Forgot your password?