Recent visitors to CrXPRT.com may have seen a notice that encourages visitors to use WebXPRT 4
instead of CrXPRT 2 for performance testing on high-end Chromebooks. The notice
reads as follows:
Chromebook technology has progressed rapidly since we released CrXPRT 2, and
we’ve received reports that some CrXPRT 2 workloads may not stress top-bin
Chromebook processors enough to give the necessary accuracy for users to
compare their performance. So, for the latest test to compare the performance
of high-end Chromebooks, we recommend using WebXPRT 4.
made this recommendation because of the evident limitations of the CrXPRT 2
performance workloads when testing newer high-end hardware. CrXPRT 2 itself is
not that old (2020), but when we created the CrXPRT 2 performance workloads, we
started with a core framework of CrXPRT 2015 performance workloads. In a
similar way, we built the CrXPRT 2015 workloads on a foundation of WebXPRT 2015
workloads. At the time, the harness and workload structures we used to ensure
WebXPRT 2015’s cross-browser capabilities provided an excellent foundation that
we could adapt for our new ChromeOS benchmark. Consequently, CrXPRT 2 is a close
developmental descendant of WebXPRT 2015. Some of the legacy WebXPRT
2015/CrXPRT 2 workloads do not stress current high-end processors—a limitation that
prevents effective performance testing differentiation—nor do they engage the
latest web technologies.
the past, the Chromebook market skewed heavily toward low-cost devices with down-bin,
inexpensive processors, making this limitation less of an issue. Now, however,
more Chromebooks offer top-bin processors on par with traditional laptops and
workstations. Because of the limitations of the CrXPRT 2 workloads, we now recommend
WebXPRT 4 for both cross-browser and ChromeOS performance testing on the latest
tools and libraries, modern WebAssembly workloads, and additional Web Workers
tasks that cover a wide range of performance requirements.
CrXPRT 2 continues to function as a capable performance and battery life
comparison test for many ChromeOS devices, WebXPRT 4 is a more appropriate tool
to use with new high-end devices. If you haven’t yet used WebXPRT 4 for
Chromebook comparison testing, we encourage you to give it a try!
If you have any questions or concerns about CrXPRT 2 or WebXPRT 4, please don’t hesitate to ask!
When we’ve released a
new version of an XPRT benchmark app, it’s been our practice for many years to
maintain a link to the previous version on the benchmark’s main page. For
example, visitors can start on the WebXPRT 4 homepage at WebXPRT.com and follow links to access
WebXPRT 3, WebXPRT 2015, and WebXPRT 2013. Historically, we’ve maintained these
links because labs and tech reviewers usually take a while to introduce a new
benchmark to their testing suite. Continued access to the older benchmarks also
allows users to quickly compare new devices to old devices without retesting
That being said, several of the XPRT pages currently contain links to benchmarks that we no longer actively support. Some of those benchmarks still function correctly, and testers occasionally use them, but a few no longer work on the latest versions of the operating systems or browsers that we designed them to test. While we want to continue to provide a way for longtime XPRT users to access legacy XPRTs, we also want to avoid potential confusion for new users. We believe our best way forward is to archive older tests in a separate part of the site.
In the coming weeks, we’ll
be moving several legacy XPRT benchmarks to an archive section of the site. Once
the new section is ready, we’ll link to it from the Extras drop-down menu at
the top of BenchmarkXPRT.com. The benchmarks will still be available in the
archive, but we will not actively support them or directly link to them from
the homepages of active XPRTs.
During this process,
we’ll move the following benchmarks to the archive section:
- WebXPRT 2015 and 2013
- CrXPRT 2015
- HDXPRT 2014
- TouchXPRT 2014
- MobileXPRT 2015 and 2013
If you have any questions or concerns about the archive process or access to legacy XPRTs, please let us know!
Last March, we discussed the Chrome OS team’s original announcement that they would be phasing out support for Chrome Apps altogether in June 2021, and would shift their focus to Chrome extensions and Progressive Web Apps. The Chrome OS team eventually extended support for existing Chrome Apps through June 2022, but as of this week, we see no indication that they will further extend support for Chrome Apps published with general developer accounts. If the end-of-life schedule for Chrome Apps does not change in the next few months, both CrXPRT 2 and CrXPRT 2015 will stop working on new versions of Chrome OS at some point in June.
To maintain CrXPRT
functionality past June, we would need to rebuild the app completely—either as
a Progressive Web App or in some other form. For this reason, we want to
reassess our approach to Chrome OS testing, and investigate which features and
technologies to include in a new Chrome OS benchmark. Our current goal is to
gather feedback and conduct exploratory research over the next few months, and begin
developing an all-new Chrome OS benchmark for publication by the end of the
While we will discuss ideas for this new Chrome OS benchmark in future blog posts, we welcome ideas from CrXPRT users now. What features or workloads would you like the new benchmark to retain? Would you like us to remove any components from the existing benchmark? Does the battery life test in its current form suit your needs? If you have any thoughts about these questions or any other aspects of Chrome OS benchmarking, please let us know!