BenchmarkXPRT Blog banner

Tag Archives: battery life

Following up on CrXPRT 2 battery life errors

A few weeks ago, we discussed error messages that a tester received when starting up CrXPRT 2 after a battery life test. CrXPRT 2 battery life tests require a full battery rundown, after which the tester plugs in the Chromebook, turns it on, opens the CrXPRT 2 app, and sees the test results. In the reported cases, the tester opened the app after a battery life test that seemed successful, but saw “N/A” or “test error” messages instead of the results they expected.

During discussions about the end-of-test system environment, we realized that some testers might be unclear about how to tell that the battery has fully run down. During the system idle portion of CrXPRT 2 battery life test iterations, the Chromebook screen turns black and a small cursor appears somewhere on the screen to let testers know the test is still in progress. We believe that some testers, seeing the black screen but not the cursor, believe the system has shut down. Restarting CrXPRT 2 before the battery life test is complete could explain some of the “N/A” or “test error” messages users have reported.

If you see a black screen without a cursor, you can check to see whether the test is complete by looking for the small system power indicator light on the side or top of most Chromebooks. These are usually red, orange, or green, but if a light of any color is lit, the test is still underway. When the light goes out, the test has ended. You can plug the system in and power it on to see results.

Note that some Chromebooks provide low-battery warnings onscreen. During CrXPRT 2 battery life runs, testers should ignore these.

We hope this clears up any confusion about how to know when a CrXPRT 2 battery life test has ended. If you’ve received repeated “N/A” or “test error” messages during your CrXPRT 2 testing and the information above does not help, please let us know!

Justin

Results details and unexpected behavior with the CrXPRT 2 battery life test

It’s been two weeks since the CrXPRT 2 public release, and we’re happy to see widespread interest in the test right out of the gate!

This week, we received a couple of questions about its battery life test from Melissa Riofrio at PCWorld. First, she asked for clarification about the relationship between the rundown time and the 30-minute increments that appear in the iteration details table for each battery life run. Second, she asked what could be causing her to get “N/A” and “test error” battery life results at the end of what appeared to be successful tests. Both topics may be of interest to other CrXPRT 2 testers, so we’ve decided to address them here in the blog and invite our readers to provide any relevant feedback.

Rundown time vs. elapsed time

When you’re viewing previous CrXPRT 2 test results and click the Details link for a specific battery life test run, a window displaying additional test information appears (the screenshot below shows an example). The window first provides performance details, then presents a table with several data points for each iteration.

The data point in the far-right column, elapsed time, can be slightly confusing. Each test iteration runs for 30 minutes, and this column provides a cumulative total of these 30-minute increments. In some instances, these totals accurately reflect the actual time elapsed from the time that testing begins. However, if the test system shuts down for some reason before running the entire iteration, this table will still show the entire 30 minutes allotted for the that iteration. In these cases, the cumulative elapsed time value in the far-right column will not match the rundown time that the test reports for the system’s battery life. For that reason, testers should always consider rundown time as the definitive value for battery life.

 “N/A” and “test error” battery life results after apparently successful tests

We’re actively investigating this issue at present. We’ve tested a wide range of Chromebooks, both old and new, on several versions of Chrome OS, including the latest versions, and have been unable to reproduce the problem. Have you witnessed this behavior at the end of a CrXPRT 2 battery life test? If so, we’d love to get more information from you about the system under test and your testing procedures, so please contact us.

We’re grateful to Melissa for raising these questions, and we appreciate everyone’s feedback on CrXPRT 2. Hopefully, we’ll soon be able to determine the cause of the  “N/A” and “test error” results and find a solution. We’ll be sure to share that information here in the blog once we do.

Justin

CrXPRT 2 is here!

We’re excited to announce that CrXPRT 2 is now available to the public! Testers can install the app on Chromebooks by using this link to the Chrome Web Store listing, or by clicking the Chrome Web Store button on CrXPRT.com.

For those who may have missed our previous discussions about what’s new with CrXPRT 2, here is a recap of the key differences between CrXPRT 2015 and CrXPRT 2:

  • CrXPRT 2 has an all-new UI, with a focus on intuitive navigation.
  • The CrXPRT 2 performance test includes six of the seven workloads in CrXPRT 2015. Newer versions of Chrome can’t run the Photo Collage workload without a workaround, so we removed it from CrXPRT 2.
  • We updated the image resolutions and sizes for the Photo Effects and Face Detection workloads.
  • The battery life test now requires a full rundown, so the length of battery life tests will vary according to the battery life of the systems under test.
  • CrXPRT 2 no longer requires testers to enter luminance and audio measurements for battery life tests.
  • We added a second video playback segment to each battery life iteration.


As we’ve noted before, CrXPRT 2 overall performance test scores and battery life measurements are not comparable to CrXPRT 2015 scores. For testers that need to access CrXPRT 2015 for legacy comparison testing, we will continue to make CrXPRT 2015 available via a link on CrXPRT.com.

We appreciate everyone’s input during the CrXPRT 2 development process. If you have any further questions about CrXPRT 2, please let us know!

Justin

The CrXPRT 2 Community Preview is available!

We’re excited to announce that the CrXPRT 2 Community Preview (CP) is now available! BenchmarkXPRT Development Community members can access the preview using a direct link posted on the CrXPRT tab in the XPRT Members’ Area (login required), where they will also find the CrXPRT 2 CP user manual.

You can find more information about the key differences between CrXPRT 2015 and CrXPRT 2 in last week’s blog entry. During the preview period, we allow testers to publish CP test scores, but CrXPRT 2 overall performance test scores and battery life measurements are not comparable to CrXPRT 2015 scores.

If you have any questions about CrXPRT 2 or joining the community, please let us know!

Justin

A preview of the new CrXPRT 2 UI

As we get closer to the CrXPRT 2 Community Preview (CP), we want to provide readers with a glimpse of the new CrXPRT 2 UI. In line with the functional and aesthetic themes we used for the latest versions of WebXPRT, MobileXPRT, and HDXPRT, we’re implementing a clean, bright look with a focus on intuitive navigation. The screenshots below show how we’ve used that approach to rework the home, battery life test, performance test, and battery life test results screens. (We’re still tweaking the UI, so the screens you see in the CP may differ slightly.)

On the home screen, we kept the performance test and battery life test buttons, but made it clearer that you can choose only one. We also added a link to the user manual to the bottom ribbon for quick access.

If you choose to run a battery life test and click Next, the screen below appears. The CrXPRT 2 battery life test requires a full rundown, so you’ll need charge your device to 100 percent before you can start the test. Once you’ve done that, enter a name for the test run, unplug the system, and click Start. (Note that you no longer need to enter values for screen brightness and audio levels.)

The CrXPRT 2 performance test includes updated versions of six of the seven workloads in CrXPRT 2015. (As we discussed in a previous blog post, newer versions of Chrome can’t run the Photo Collage workload without a workaround, so we removed it from CrXPRT 2.)  To run the performance test, enter a name for the test run, customize the workloads if you wish, and click Start.

For the results screens, we wanted to highlight the most important end-of-test information while still offering clear paths for options such as getting additional details on the test, submitting results, and running the test again. Below, we show the results screen from a battery life test. Note the “Main menu” link in the upper-left corner, which we added to all screens to give users a quick way to navigate back to the home screen.

CrXPRT 2 development and testing are still underway. We don’t yet have an exact release date for the CP, but once we do, we’ll announce it here in the blog.

What do you think about the new CrXPRT 2 UI? Let us know!

Justin

Planning for the next CrXPRT

We’re currently planning the next version of CrXPRT, our benchmark that evaluates the performance and battery life of Chromebooks. If you’re unfamiliar with CrXPRT, you can find out more about how it works both here in the blog and at CrXPRT.com. If you’ve used CrXPRT, we’d love to hear any suggestions you may have. What do you like or dislike about CrXPRT? What features do you hope to see in a new version?

When we begin work on a new version of any benchmark, one of our first steps is to determine whether the workloads will provide value during the years ahead. As technology and user behavior evolve, we update test content to be more relevant. One example is when we replace photos with ones that use more contemporary file resolutions and sizes.

Sometimes the changing tech landscape prompts us to remove entire workloads and add new ones. The Photo Collage workload in CrXPRT uses Portable Native Client (PNaCl) technology, for which the Chrome team will soon end support. CrXPRT 2015 has a workaround for this issue, but the best course of action for the next version of CrXPRT will be to remove this workload altogether.

The battery life test will also change. Earlier this year, we started to see unusual battery life estimates and high variance when running tests at CrXPRT’s default battery life test length of 3.5 hours, so we’ve been recommending that users perform full rundowns instead. In the next CrXPRT, the battery life test will require full rundowns.

We’ll also be revamping the CrXPRT UI to improve the look of the benchmark and make it easier to use, as we’ve done with the other recent XPRT releases.

We really do want to hear your ideas, and any feedback you send has a chance to shape the future of the benchmark. Let us know what you think!

Justin

Check out the other XPRTs:

Forgot your password?