BenchmarkXPRT Blog banner

Tag Archives: Chromebooks

Persistent CrXPRT 2 battery life test error on Chrome v89 and later

A few weeks ago, we discussed an error that we’d recently started encountering during the CrXPRT 2 battery life test on systems running Chrome OS v89.x and later. The error prevents the test from completing and producing a battery life estimate. CrXPRT stops running its normal workload cycle and produces a “Test Error” page. The timing of the error can vary from run to run. Sometimes, CrXPRT stops running after only a few workload iterations, while other times, the battery life test almost reaches completion before producing the error.

We have seen the error on across multiple brands of Chromebooks running Chrome OS v89.x and later. To our knowledge, Chromebooks running Chrome OS v88.x and earlier versions complete the battery life test without issues. We are unaware of any problems with the CrXPRT 2 performance test.

We’re continuing to investigate this problem. Unfortunately, we have not yet identified the root cause. Without a solution, we are recommending that for now, testers not use the CrXPRT 2 battery life test. We will post this recommendation on CrXPRT.com.

We apologize for the inconvenience that this error is causing CrXPRT 2 testers. As soon as we identify a possible solution, we will share that information here in the blog. If you have any insight into recent Chrome OS changes or flag settings that could be causing this problem, please let us know!

Justin

CrXPRT 2 battery life error on Chrome 89 and 90

In recent lab tests, we’ve encountered an error during the CrXPRT 2 battery life test that prevents the test from completing and producing a battery life estimate. As the screenshot below shows, when the error occurs, CrXPRT stops running its normal workload cycle and produces a “Test Error” page. We have seen this behavior on systems running Chrome OS v89.x and v90.x, across multiple vendor platforms. In our testing, Chromebooks running  Chrome OS v88.x and earlier versions continue to complete the battery life test without any issues.

The error occurs consistently on every Chromebook running v89.x or v90.x that we’ve tested so far. However, the timing of the error varies from run to run on the same system. Sometimes, CrXPRT stops running after only a few workload iterations, while at other times, the battery life test runs almost to completion before producing the error.

We’re actively investigating this problem, but have not yet identified the root cause. We apologize for the inconvenience that this error may be causing CrXPRT 2 testers. As soon as we identify the root cause of the problem and have ideas about possible solutions, we will share that information here in the blog. If you have any insight into recent Chrome OS changes or flag settings that could be causing this problem, please let us know!

Justin

CrXPRT support through 2022

CrXPRT testers may remember that back around the time that we began the CrXPRT 2 development process, the Chrome team announced that they were phasing out support for Portable Native Client (PNaCL) in favor of WebAssembly (WASM). As a first step, they changed the Chrome OS setting that enabled PNaCL by default. At the time, this caused problems with the Photo Collage workload in CrXPRT 2015, and even though we identified a workaround, details in the Chrome team’s announcement led us to conclude that the workaround might stop working in June 2021. Because of this change, we decided that the best decision would be to remove the workload from CrXPRT 2, and keep existing CrXPRT 2015 testers informed of any changes with the workaround.

In 2020, the Chrome team also announced that they would be phasing out support for Chrome Apps altogether starting in June 2021, and would shift their focus to Chrome extensions. This change would have required us to reassess the viability of CrXPRT in anything like its current form.

We’re happy to report that the Chrome team has extended support for PNaCL and existing Chrome Apps through June 2022. Barring further changes, this means that CrXPRT 2015 (with the workaround) and CrXPRT 2 should continue to serve as reliable Chrome OS evaluation tools for some time.

If you have any questions about CrXPRT 2, please let us know!

Justin

The XPRTs can help with your holiday shopping

The biggest shopping days of the year are fast approaching, and if you’re researching phones, tablets, Chromebooks, or laptops in preparation for Black Friday and Cyber Monday sales, the XPRTs can help! One of the core functions of the XPRTs is to help cut through all the marketing noise by providing objective, reliable measures of a device’s performance. For example, instead of trying to guess whether a new Chromebook is fast enough to handle the demands of remote learning, you can use its CrXPRT and WebXPRT performance scores to see how it stacks up against the competition when handling everyday tasks.

A good place to start your search for scores is our XPRT results browser. The browser is the most efficient way to access the XPRT results database, which currently holds more than 2,600 test results from over 100 sources, including major tech review publications around the world, OEMs, and independent testers. It offers a wealth of current and historical performance data across all the XPRT benchmarks and hundreds of devices. You can read more about how to use the results browser here.

Also, if you’re considering a popular device, chances are good that someone has already published an XPRT score for that device in a recent tech review. The quickest way to find these reviews is by searching for “XPRT” within your favorite tech review site, or by entering the device name and XPRT name (e.g. “Apple iPad” and “WebXPRT”) in a search engine. Here are a few recent tech reviews that use one or more of the XPRTs to evaluate a popular device:


The XPRTs can help consumers make better-informed and more confident tech purchases this holiday season, and we hope you’ll find the data you need on our site or in an XPRT-related tech review. If you have any questions about the XPRTs, XPRT scores, or the results database please feel free to ask!

Justin

Results details and unexpected behavior with the CrXPRT 2 battery life test

It’s been two weeks since the CrXPRT 2 public release, and we’re happy to see widespread interest in the test right out of the gate!

This week, we received a couple of questions about its battery life test from Melissa Riofrio at PCWorld. First, she asked for clarification about the relationship between the rundown time and the 30-minute increments that appear in the iteration details table for each battery life run. Second, she asked what could be causing her to get “N/A” and “test error” battery life results at the end of what appeared to be successful tests. Both topics may be of interest to other CrXPRT 2 testers, so we’ve decided to address them here in the blog and invite our readers to provide any relevant feedback.

Rundown time vs. elapsed time

When you’re viewing previous CrXPRT 2 test results and click the Details link for a specific battery life test run, a window displaying additional test information appears (the screenshot below shows an example). The window first provides performance details, then presents a table with several data points for each iteration.

The data point in the far-right column, elapsed time, can be slightly confusing. Each test iteration runs for 30 minutes, and this column provides a cumulative total of these 30-minute increments. In some instances, these totals accurately reflect the actual time elapsed from the time that testing begins. However, if the test system shuts down for some reason before running the entire iteration, this table will still show the entire 30 minutes allotted for the that iteration. In these cases, the cumulative elapsed time value in the far-right column will not match the rundown time that the test reports for the system’s battery life. For that reason, testers should always consider rundown time as the definitive value for battery life.


 “N/A” and “test error” battery life results after apparently successful tests

We’re actively investigating this issue at present. We’ve tested a wide range of Chromebooks, both old and new, on several versions of Chrome OS, including the latest versions, and have been unable to reproduce the problem. Have you witnessed this behavior at the end of a CrXPRT 2 battery life test? If so, we’d love to get more information from you about the system under test and your testing procedures, so please contact us.

We’re grateful to Melissa for raising these questions, and we appreciate everyone’s feedback on CrXPRT 2. Hopefully, we’ll soon be able to determine the cause of the  “N/A” and “test error” results and find a solution. We’ll be sure to share that information here in the blog once we do.

Justin

CrXPRT 2 is here!

We’re excited to announce that CrXPRT 2 is now available to the public! Testers can install the app on Chromebooks by using this link to the Chrome Web Store listing, or by clicking the Chrome Web Store button on CrXPRT.com.

For those who may have missed our previous discussions about what’s new with CrXPRT 2, here is a recap of the key differences between CrXPRT 2015 and CrXPRT 2:

  • CrXPRT 2 has an all-new UI, with a focus on intuitive navigation.
  • The CrXPRT 2 performance test includes six of the seven workloads in CrXPRT 2015. Newer versions of Chrome can’t run the Photo Collage workload without a workaround, so we removed it from CrXPRT 2.
  • We updated the image resolutions and sizes for the Photo Effects and Face Detection workloads.
  • The battery life test now requires a full rundown, so the length of battery life tests will vary according to the battery life of the systems under test.
  • CrXPRT 2 no longer requires testers to enter luminance and audio measurements for battery life tests.
  • We added a second video playback segment to each battery life iteration.


As we’ve noted before, CrXPRT 2 overall performance test scores and battery life measurements are not comparable to CrXPRT 2015 scores. For testers that need to access CrXPRT 2015 for legacy comparison testing, we will continue to make CrXPRT 2015 available via a link on CrXPRT.com.

We appreciate everyone’s input during the CrXPRT 2 development process. If you have any further questions about CrXPRT 2, please let us know!

Justin

Check out the other XPRTs:

Forgot your password?