Recently, we had a discussion with a community member about why we no longer recommend specific screen brightness settings during CrXPRT 2 battery life tests. In the CrXPRT 2015 user manual, we recommended setting the test system’s screen brightness to 200 nits. Because the amount of power that a system directs to screen brightness can have a significant impact on battery life, we believed that pegging screen brightness to a common standard for all test systems would yield apple-to-apples comparisons.
After extensive experience with CrXPRT 2015 testing, we decided to not recommend a standard screen brightness with CrXPRT 2, for the following reasons:
A significant number of Chromebooks cannot produce a screen brightness of 200 nits. A few higher-end models can do so, but they are not representative of most Chromebooks. Some Chromebooks, especially those that many school districts and corporations purchase in bulk, cannot produce a brightness of even 100 nits.
Because of the point above, adjusting screen brightness would not represent real-life conditions for most Chromebooks, and the battery life results could mislead consumers who want to know the battery life they can expect with default out-of-box settings.
Most testers, and even some labs, do not have light meters, and the simple brightness percentages that the operating system reports produce different degrees of brightness on different systems. For testers without light meters, a standardized screen brightness recommendation could discourage them from running the test.
The brightness controls for some low-end Chromebooks lack the fine-tuning capability that is necessary to standardize brightness between systems. In those cases, an increase or decrease of one notch can swing brightness by 20 to 30 nits in either direction. This could also discourage testing by leading people to believe that they lack the capability to correctly run the test.
situations where testers want to compare battery life using standardized screen
brightness, we recommend using light meters to set the brightness levels as
closely as possible. If the brightness levels between systems vary by more than
few nits, and if the levels vary significantly from out-of-box settings, the
publication of any resulting battery life results should include a full
disclosure and explanation of test conditions.
the majority of testers without light meters, running the CrXPRT 2 battery life
test with default screen brightness settings on each system provides a reliable
and accurate estimate of the type of real-world, out-of-box battery life
consumers can expect.
If you have any questions or comments about the CrXPRT 2 battery life test, please feel free to contact us!
The holiday season is
fast approaching, and with widespread product shortages and supply chain
interruptions in the tech industry, it’s wise to start your holiday shopping
now. If you’re considering phones, tablets, Chromebooks, or laptops as gifts,
but are unsure where to get reliable information about the devices, the XPRTs can
One of the core
functions of the XPRTs is to cut through the marketing noise by providing
objective, reliable measures of a device’s performance. For example, instead of
trying to guess whether a new Chromebook is fast enough to handle the demands
of remote learning, you can use its CrXPRT and WebXPRT performance scores to see how it stacks up against the
competition on everyday tasks.
A good place to start looking
for device scores is our XPRT results browser, which lets you access our database of more than 2,800 test
results from over 110 sources, including major tech review publications around
the world, OEMs, and independent testers. You can find a wealth of current and
historical performance data across all the XPRT benchmarks and hundreds of
devices. Learn how to use the results browser here.
If you’re considering
a popular device, chances are good that a recent tech review includes an XPRT
score for that device. You can find these reviews by going to your favorite
tech review site and searching for “XPRT,” or entering the name of the device
and the appropriate XPRT (e.g., “Apple iPad” and “WebXPRT”) in a search engine.
Here are a few recent tech reviews that use one or more of the XPRTs to
evaluate popular devices:
The XPRTs can help
consumers make better-informed and more confident tech purchases this holiday
season, and we hope you’ll find the data you need on our site or in an
XPRT-related tech review. If you have any questions about the XPRTs, XPRT
scores, or the results database please feel free to ask!
Last month, we discussed a potential fix for the error that was preventing CrXPRT 2 testers from successfully completing battery life tests on systems running Chrome v89.x and later. Since then, we’ve been testing an updated, unpublished version of the app package across several Chromebook models to ensure that the new build is stable and produces consistent results. We’re happy to report that our testing was successful, and we’ve published the new CrXPRT build (v188.8.131.52) in the Chrome Web Store and it is live as of 12:45 PM EDT today.
that it might take some time for the update to appear on your Chromebook and,
once it does, you might have to manually approve the update notice.
the tests nor the method of calculating the overall score and battery-life
score in this new build have changed, so results are comparable with previous
CrXPRT 2 results.
We appreciate everyone’s patience while we found a solution to the error. If you have any questions or comments about the CrXPRT 2 battery life test, please feel free to contact us!
the past few months, we’ve been recommending that CrXPRT 2
testers not use the battery life test until we find a solution to a recurring
error on Chrome v89.x and later. The error prevents the test from completing
and producing a battery life estimate. Sometimes, the CrXPRT battery life test stops
running after only a few workload iterations, while at other times, it almost
reaches completion before producing the error.
We are cautiously optimistic that we’ve identified both the problem and a potential fix. We believe the problem stems from fluctuations in the time it takes the benchmark to communicate with Chrome to collect and store battery life information. While we haven’t identified the root cause of the fluctuations, adjusting the CrXPRT code to make it less sensitive to the fluctuations appears to be an effective fix. We have incorporated those adjustments into an updated, unpublished version of the app package, and we can now complete CrXPRT 2 battery life tests on Chrome v89.x and later with no failures.
are calling this a potential fix because we’re still testing across several
different Chromebook models to ensure consistency. In some testing, the
variance in estimated battery life results has been a little higher than we
like, so we’re taking time to determine whether that variance is present across
all systems or on only specific hardware.
We’d like to apologize once again for the inconvenience that this error is causing CrXPRT 2 testers. As soon as we better understand the viability of the current fix as a long-term update, we’ll let you know!
A few weeks ago, we discussed an error that we’d recently started encountering during the CrXPRT 2 battery life test on systems running Chrome OS v89.x and later.
The error prevents the test from completing and producing a battery life
estimate. CrXPRT stops running its normal workload cycle and produces a “Test
Error” page. The timing of the error can vary from run to run. Sometimes,
CrXPRT stops running after only a few workload iterations, while other times,
the battery life test almost reaches completion before producing the error.
We have seen the error on across multiple brands of Chromebooks running
Chrome OS v89.x and later. To our knowledge, Chromebooks running Chrome OS v88.x
and earlier versions complete the battery life test without issues. We are unaware
of any problems with the CrXPRT 2 performance test.
We’re continuing to investigate this problem. Unfortunately, we have not yet identified the root cause. Without a solution, we are recommending that for now, testers not use the CrXPRT 2 battery life test. We will post this recommendation on CrXPRT.com.
We apologize for the inconvenience that this error is causing CrXPRT 2 testers. As soon as we identify a possible solution, we will share that information here in the blog. If you have any insight into recent Chrome OS changes or flag settings that could be causing this problem, please let us know!
In recent lab tests, we’ve encountered an error during the CrXPRT 2 battery life test that prevents the test from completing and producing a battery life estimate. As the screenshot below shows, when the error occurs, CrXPRT stops running its normal workload cycle and produces a “Test Error” page. We have seen this behavior on systems running Chrome OS v89.x and v90.x, across multiple vendor platforms. In our testing, Chromebooks running Chrome OS v88.x and earlier versions continue to complete the battery life test without any issues.
The error occurs consistently on every Chromebook running v89.x or
v90.x that we’ve tested so far. However, the timing of the error varies from
run to run on the same system. Sometimes, CrXPRT stops running after only a few
workload iterations, while at other times, the battery life test runs almost to
completion before producing the error.
We’re actively investigating this problem, but have not yet
identified the root cause. We apologize for the inconvenience that this error
may be causing CrXPRT 2 testers. As soon as we identify the root cause of the
problem and have ideas about possible solutions, we will share that information
here in the blog. If you have any insight into recent Chrome OS changes or flag
settings that could be causing this problem, please let us know!