BenchmarkXPRT Blog banner

Tag Archives: CrXPRT

CrXPRT 2 on FydeOS!

Recently, a tester contacted us with details from a CrXPRT 2 performance test run that they’d successfully completed on… an Apple MacBook Pro! Because CrXPRT 2 is a Chrome Web App that we designed for Chrome OS, it was quite a surprise to hear that it is now possible to run CrXPRT 2 on non-Chrome OS platforms by using FydeOS.

FydeOS is an operating system based on a fork of the Chromium OS project. Developers originally intended FydeOS to be a Google-independent, Chrome-like alternative for the Chinese educational market, but FydeOS is now available to the English-speaking consumer and enterprise markets as well. FydeOS users can run a Chrome-like OS on something other than a Chromebook or a Chromebox, such as a PC, Mac, virtual machine, or even a Raspberry Pi device. Additionally, FydeOS supports Android, Chrome OS, and Linux apps, and users can run those apps at the same time on the same screen.

We have not yet conducted any testing with FydeOS in our lab, but we wanted to pass along this information to any readers who may be interested. If the OS operates as described, it may provide a way for us to experiment with using CrXPRT 2 in some interesting cross-platform tests.

Justin

Chrome OS support for CrXPRT apps ends in June 2022

Last March, we discussed the Chrome OS team’s original announcement that they would be phasing out support for Chrome Apps altogether in June 2021, and would shift their focus to Chrome extensions and Progressive Web Apps. The Chrome OS team eventually extended support for existing Chrome Apps through June 2022, but as of this week, we see no indication that they will further extend support for Chrome Apps published with general developer accounts. If the end-of-life schedule for Chrome Apps does not change in the next few months, both CrXPRT 2 and CrXPRT 2015 will stop working on new versions of Chrome OS at some point in June.

To maintain CrXPRT functionality past June, we would need to rebuild the app completely—either as a Progressive Web App or in some other form. For this reason, we want to reassess our approach to Chrome OS testing, and investigate which features and technologies to include in a new Chrome OS benchmark. Our current goal is to gather feedback and conduct exploratory research over the next few months, and begin developing an all-new Chrome OS benchmark for publication by the end of the year.

While we will discuss ideas for this new Chrome OS benchmark in future blog posts, we welcome ideas from CrXPRT users now. What features or workloads would you like the new benchmark to retain? Would you like us to remove any components from the existing benchmark? Does the battery life test in its current form suit your needs? If you have any thoughts about these questions or any other aspects of Chrome OS benchmarking, please let us know!

Justin

Updated system configuration recommendations for CrXPRT 2 battery life tests

Recently, we heard from a BenchmarkXPRT Development Community member who was testing Chromebooks in their lab. On a few of the Chromebooks, they saw sporadic CrXPRT 2 battery life test failures where CrXPRT 2 would successfully complete a battery life test and produce a result for the initial run, but then fail at the end of later runs.

After a considerable amount of troubleshooting, they determined that the issue seemed to be related to the way some systems automatically shut down before the battery is completely exhausted, and the way some systems will automatically boot up once the tester plugs in the power adapter for charging. This member found that when they added a few system configuration steps before battery life tests and made slight changes to their post-test routine, the systems that had previously experienced consistent failures would successfully complete battery life tests and produce results.

The added steps are quick and straightforward, and we decided to add them to the Configuring the test device and Running the tests sections of the CrXPRT 2 user manual. We hope this updated guidance will help to prevent future frustration for CrXPRT 2 testers.

If you have any questions or comments about the CrXPRT 2 battery life test, please feel free to contact us!

Justin

Why we don’t control screen brightness during CrXPRT 2 battery life tests

Recently, we had a discussion with a community member about why we no longer recommend specific screen brightness settings during CrXPRT 2 battery life tests. In the CrXPRT 2015 user manual, we recommended setting the test system’s screen brightness to 200 nits. Because the amount of power that a system directs to screen brightness can have a significant impact on battery life, we believed that pegging screen brightness to a common standard for all test systems would yield apple-to-apples comparisons.

After extensive experience with CrXPRT 2015 testing, we decided to not recommend a standard screen brightness with CrXPRT 2, for the following reasons:

  • A significant number of Chromebooks cannot produce a screen brightness of 200 nits. A few higher-end models can do so, but they are not representative of most Chromebooks. Some Chromebooks, especially those that many school districts and corporations purchase in bulk, cannot produce a brightness of even 100 nits.
  • Because of the point above, adjusting screen brightness would not represent real-life conditions for most Chromebooks, and the battery life results could mislead consumers who want to know the battery life they can expect with default out-of-box settings.
  • Most testers, and even some labs, do not have light meters, and the simple brightness percentages that the operating system reports produce different degrees of brightness on different systems. For testers without light meters, a standardized screen brightness recommendation could discourage them from running the test.
  • The brightness controls for some low-end Chromebooks lack the fine-tuning capability that is necessary to standardize brightness between systems. In those cases, an increase or decrease of one notch can swing brightness by 20 to 30 nits in either direction. This could also discourage testing by leading people to believe that they lack the capability to correctly run the test.

In situations where testers want to compare battery life using standardized screen brightness, we recommend using light meters to set the brightness levels as closely as possible. If the brightness levels between systems vary by more than few nits, and if the levels vary significantly from out-of-box settings, the publication of any resulting battery life results should include a full disclosure and explanation of test conditions.

For the majority of testers without light meters, running the CrXPRT 2 battery life test with default screen brightness settings on each system provides a reliable and accurate estimate of the type of real-world, out-of-box battery life consumers can expect.

If you have any questions or comments about the CrXPRT 2 battery life test, please feel free to contact us!

Justin

Looking back on 2021 with the XPRTs

As 2022 gets underway, we want to take this opportunity to look back on 2021 and review another productive year for the XPRTs. Readers of our newsletter are familiar with the stats and updates we include each month, but for our blog readers who don’t receive the newsletter, we’ve compiled some highlights below.

Benchmarks
In the past year, we released the WebXPRT 4 Preview, CloudXPRT v1.1, and an updated CrXPRT 2 build that included a fix for prior issues with the battery life test.

XPRTs in the media
Journalists, advertisers, and analysts referenced the XPRTs thousands of times in 2021. It’s always rewarding to know that the XPRTs have proven to be useful and reliable assessment tools for technology publications such as AnandTech, Expert Reviews, Gadgets 360, Gizmodo, Hot Hardware, Laptop Mag, Legit Reviews, Notebookcheck, PCMag, PCWorld, TechPowerUp, Tom’s Hardware, and ZDNet.

Downloads and confirmed runs
In 2021, we had more than 23,600 benchmark downloads and 228,900 confirmed runs. Our most popular benchmark, WebXPRT, just passed 909,800 runs since its debut in 2013! WebXPRT continues to be a go-to, industry-standard performance benchmark for OEM labs, vendors, and leading tech press outlets around the globe.

Media, publications, and interactive tools
Part of our mission with the XPRTs is to produce tools and materials that help testers better understand the ins and outs of benchmarking in general and the XPRTs in particular. To help achieve this goal, we published the following in 2021:

We’re thankful for everyone who has used the XPRTs, joined the community, and sent questions and suggestions throughout 2021. We look forward to an exciting 2022!

Justin

It’s time to shop for the holidays, and the XPRTs are here to help!

The holiday season is fast approaching, and with widespread product shortages and supply chain interruptions in the tech industry, it’s wise to start your holiday shopping now. If you’re considering phones, tablets, Chromebooks, or laptops as gifts, but are unsure where to get reliable information about the devices, the XPRTs can help!

One of the core functions of the XPRTs is to cut through the marketing noise by providing objective, reliable measures of a device’s performance. For example, instead of trying to guess whether a new Chromebook is fast enough to handle the demands of remote learning, you can use its CrXPRT and WebXPRT performance scores to see how it stacks up against the competition on everyday tasks.

A good place to start looking for device scores is our XPRT results browser, which lets you access our database of more than 2,800 test results from over 110 sources, including major tech review publications around the world, OEMs, and independent testers. You can find a wealth of current and historical performance data across all the XPRT benchmarks and hundreds of devices. Learn how to use the results browser here.

If you’re considering a popular device, chances are good that a recent tech review includes an XPRT score for that device. You can find these reviews by going to your favorite tech review site and searching for “XPRT,” or entering the name of the device and the appropriate XPRT (e.g., “Apple iPad” and “WebXPRT”) in a search engine. Here are a few recent tech reviews that use one or more of the XPRTs to evaluate popular devices:

The XPRTs can help consumers make better-informed and more confident tech purchases this holiday season, and we hope you’ll find the data you need on our site or in an XPRT-related tech review. If you have any questions about the XPRTs, XPRT scores, or the results database please feel free to ask!

Justin

Check out the other XPRTs:

Forgot your password?