BenchmarkXPRT Blog banner

Category: Benchmarking

Time for a merger of equals?

As I said last week, we’re working on the design document for the new version of MobileXPRT, and we expect to have it out in the next couple of weeks. We have several ideas we’re pretty excited about.

One of the ideas we’ve been considering is merging BatteryXPRT and MobileXPRT into a single benchmark. This would be similar to what we’ve done with CrXPRT, which has tests for both battery life and performance. As with CrXPRT, you’d be able to run either test, and you could get performance and battery life for a device in a single day using a single benchmark.

If we as the community do decide to merge the benchmarks, there will be a lot to think about. For example, MobileXPRT is unaware whether it’s connected to the Internet, while BatteryXPRT not only detects how it’s connected, but selects the appropriate Airplane mode or Network-Wifi Mode test. And, of course, we’d have to figure out what to call it.

What do you think about merging the two benchmarks? Would it make your life simpler? What other features would you like to see in the new MobileXPRT? This is the time to speak up!

Eric

A couple of things

We’ve heard about a couple of issues this week. The HDXPRT 2014 Convert Videos test uses the MediaEspresso application and requires hardware acceleration be configured. On some systems, this setting is not configuring automatically, which may result in lower scores. We’re working on a solution, but in the meantime there’s an easy workaround.

We’ve also found that some Chromebooks report extremely low battery use, as low as 0%, for the first couple iterations of the CrXPRT battery life test. This can cause CrXPRT to report results with a very wide confidence interval, with an interval greater than 15%. We’re looking at ways to detect and compensate for this. However, if you see results with a very wide confidence interval, we recommend that you use the rundown test.

In other news, we’ve been talking about Intel Developer Forum 2015 – Shenzhen for weeks, and the time is finally here! Bill and Justin get on the plane tomorrow. If you’d like to talk about the XPRTs, or any of PT’s other offerings, Bill and the team would be happy to meet with you!

Eric

WebXPRT 2015 Community Preview is available!

As we said yesterday, we’re releasing the community preview of a WebXPRT 2015 Community Preview. Members of the BenchmarkXPRT community can now run the community preview and publish results.

Run the community preview here.

After trying out the community preview, please send in your comments. Either post them to the forum or mail them to benchmarkxprtsupport@principledtechnologies.com. If you mail information that’s of interest to the entire community, we may post an anonymous version of the comments to the forum.

Thanks for being part of the community,

Eric

Finally!

We’re releasing the community preview of WebXPRT 2015 tomorrow. We’re very excited that it’s finally here. In the past few weeks, we’ve discussed some of the new features in WebXPRT 2015, such as test automation, its new and improved tests, and its Chinese UI. We think you’re really going to enjoy the new WebXPRT.

The design document (login required) specifies that WebXPRT will contain an experimental workload. That workload is not in the community preview, although we plan for it to be in the general release. However, because any experimental workloads are not included in the overall score, this will not affect any results you generate.

We’re also investigating the use of the JavaScript navigator object to improve system disclosure, but we are still determining if we can get reliable enough information to display. So this information is not displayed in the community preview.

As with all the BenchmarkXPRT community previews, we’re not putting any publication restrictions on this preview release. Test at will, and publish your findings. We guarantee the results for the community preview will be comparable to results from the general release.

If you’re not a community member, join us and check out the new WebXPRT.

Eric

More power, more control

As I said last week, the community preview for WebXPRT 2015 is coming up soon. One of the changes that will be exciting to anyone who does a lot of testing is that we made it simpler to automate WebXPRT tests.

WebXPRT 2015 will let you automatically select any set of tests you want to run. However, as always, you must run the entire suite of tests to get an overall score. Although the community preview will not include any experimental tests, the automation includes control for those future tests as well.

You may choose from several output formats: HTML table, XML, and CSV, or you can download the results as a text file.

Using the automation is simple: you just append the desired test parameters to the end of the URL. The format allows you to mix and match a lot of options, while still being very concise. The details will be in the release notes.

As people who test a lot of devices, we are very excited about this new capability.

Eric

It’s been a show to watch!

Mobile World Congress 2015 (MWC) has been happening this week in Barcelona. There have been lots of cool phones and tablets, of course, but also there has also been a lot of other cool stuff. From smart watches, foldable keyboards, and 200 GB SD cards to “privacy glasses,” electric bicycles, and even furniture from IKEA, this has definitely been a show to watch.

As you know, Bill is our man in Barcelona. If you read Bill’s blog, you know he lives for this sort of stuff. He’s also been meeting with vendors and having great conversations about the XPRTs.

Last week I mentioned that Bill took a great-looking handout to MWC with him. For those who have been wondering what a megaphone had to do with anything, here’s what the front and back look like.

Let the world know blue Let the world know red

Even though MWC hasn’t ended yet, we’re already deep into preparations for Intel Developer Forum 2015 – Shenzhen. We’ll have more to say about that in the weeks to come.

In other news, we released a new build of BatteryXPRT 2014, v103. The build fixes a problem reported by a reviewer testing the LG G3. Scores from this build are comparable with previous BatteryXPRT scores. Click here for more details.

Eric

Check out the other XPRTs:

Forgot your password?