BenchmarkXPRT Blog banner

Category: CrXPRT

Five years later…

Five years ago this month, we started what we then called the HDXPRT Development Community. The first benchmark, HDXPRT 2011, appeared six months later. A LOT has happened since then.

We’ve grown to six benchmarks (HDXPRT, TouchXPRT, WebXPRT, MobileXPRT, BatteryXPRT, and CrXPRT) with plenty of updates. As a result, we had to change the name to the BenchmarkXPRT Development Community, though we’ve come to refer to the benchmarks themselves as the XPRTs.

To tell the world about the XPRTs and the BenchmarkXPRT Development Community, we’ve written over 200 blog entries. We’ve created videos, a training course, infographics, and white papers. We’ve met members of the community at their companies, via webinars, and at trade shows. We’ve quite literally traveled around the world to shows in Las Vegas, Barcelona, Taipei, and Shenzhen.

As a result, the community has grown to about 150 individuals at over 60 companies and organizations. People have downloaded or run the benchmarks over 100,000 times in over 44 countries. The XPRTs have been cited over 3,800 times in a wide variety of websites around the world.

Yes, a lot has happened over these five years.

On behalf of the BenchmarkXPRT team, I want to convey my sincere thanks to all of you for your involvement over these years. I’m really looking forward to what the next five years will look like. We’re just getting started!

Time for a merger of equals?

As I said last week, we’re working on the design document for the new version of MobileXPRT, and we expect to have it out in the next couple of weeks. We have several ideas we’re pretty excited about.

One of the ideas we’ve been considering is merging BatteryXPRT and MobileXPRT into a single benchmark. This would be similar to what we’ve done with CrXPRT, which has tests for both battery life and performance. As with CrXPRT, you’d be able to run either test, and you could get performance and battery life for a device in a single day using a single benchmark.

If we as the community do decide to merge the benchmarks, there will be a lot to think about. For example, MobileXPRT is unaware whether it’s connected to the Internet, while BatteryXPRT not only detects how it’s connected, but selects the appropriate Airplane mode or Network-Wifi Mode test. And, of course, we’d have to figure out what to call it.

What do you think about merging the two benchmarks? Would it make your life simpler? What other features would you like to see in the new MobileXPRT? This is the time to speak up!

Eric

Lots of things are happening!

The WebXPRT 2015 community preview hasn’t even been out two weeks yet, but there are already hundreds of runs in the database. If you haven’t checked it out yet, now is a good time! (login required)

As I mentioned last week, Bill and Justin are at Intel Developer Forum 2015 – Shenzhen. Here’s their home away from home this week.

booth

Bill and Justin have been having a lot of good conversations and have found a lot of interest in an open development community. We’re looking forward to having even more members in Asia soon!

On Monday we released the CrXPRT white paper. If you want to know more about the concepts behind CrXPRT 2015, how it was developed, how the results are calculated, or anything else about CrXPRT, the white paper is a great place to start.
Finally, the MobileXPRT 2015 design document is coming in the next couple of weeks. What would you like to see in the next version of MobileXPRT? 64-bit support? New types of tests? Improvements to the UI? Everything is on the table. This is the time to make your voice heard!
Eric

A couple of things

We’ve heard about a couple of issues this week. The HDXPRT 2014 Convert Videos test uses the MediaEspresso application and requires hardware acceleration be configured. On some systems, this setting is not configuring automatically, which may result in lower scores. We’re working on a solution, but in the meantime there’s an easy workaround.

We’ve also found that some Chromebooks report extremely low battery use, as low as 0%, for the first couple iterations of the CrXPRT battery life test. This can cause CrXPRT to report results with a very wide confidence interval, with an interval greater than 15%. We’re looking at ways to detect and compensate for this. However, if you see results with a very wide confidence interval, we recommend that you use the rundown test.

In other news, we’ve been talking about Intel Developer Forum 2015 – Shenzhen for weeks, and the time is finally here! Bill and Justin get on the plane tomorrow. If you’d like to talk about the XPRTs, or any of PT’s other offerings, Bill and the team would be happy to meet with you!

Eric

Back to Barcelona

It’s time for one of Bill’s favorite shows of the year. Mobile World Congress (MWC) starts Monday in Barcelona. Talking about technology in Barcelona, now that’s a plum assignment!

Bill’s got a lot to talk about. Since last year’s MWC, here’s some of what’s happened:

(more…)

A guest blog from Wilson Cheng at Acer Inc.

Two years ago, I got a job to evaluate cross-platform benchmarks for comparing Windows and Android.

Usually, Web-based benchmarks focus on only Java-script and HTML5 functionality. I found WebXPRT is easy to use and includes the tasks that you do every day. It reliably tests the user experience and gives a simple score for comparison. I suggested to our manager that we use it to evaluate our platform and he agreed.

Recently, Chromebooks have become very popular. Testing the battery life of Chromebooks has been difficult because you have needed to use the power_LoadTest image from the Chromium project or the test image from the Chrome OS Partners Web site to do power load testing. Without reimaging the system, you couldn’t test battery life. Because CrXPRT is designed to test battery life as well as performance on ChromeOS, this extra step is no longer needed. We can run CrXPRT without updating the test image, which is very useful in our battery life testing.

The advantage of XPRT benchmarks is that they are easy to use and give real-world results. The disadvantage is that they are not yet popular on review sites.

Check out the other XPRTs:

Forgot your password?