In July, we discussed the Chrome OS team’s decision to end support for Chrome apps, and how that will prevent us from publishing any future fixes or updates for CrXPRT 2. We also announced our goal of beginning development of an all-new Chrome OS XPRT benchmark by the end of this year. While we are actively discussing this benchmark and researching workload technologies and scenarios, we don’t foresee releasing a preview build this year.
The good news is that,
in spite of a lack of formal support from the Chrome OS team, the CrXPRT 2
performance and battery life tests currently run without any known issues. We
continue to monitor the status of CrXPRT and will inform our blog readers of
any significant changes.
If you have any questions about CrXPRT, or ideas about the types of features or workloads you’d like to see in a new Chrome OS benchmark, please let us know!
Recently,
Tom’s Guide published an interesting article
about how they used ChromeOS Flex to turn a
ten-year-old Apple MacBook Pro into a functioning Chromebook by replacing the
laptop’s macOS operating system with ChromeOS. ChromeOS Flex is a free Google
tool that allows users to create a bootable USB drive that they can then use to
install ChromeOS on a wide variety
of hardware platforms that traditionally run other operating systems such as
macOS or Windows. Because ChromeOS is a cloud-first, relatively low-overhead
operating system, the ChromeOS Flex option could breathe new life into an old
laptop that you have lying around.
Never having encountered a MacBook Pro with ChromeOS, we were interested to learn about Tom’s experience running XPRT benchmarks in this new environment.WebXPRT 4, WebXPRT 3, and the CrXPRT 2 performance test apparently ran without any issues, but we have not yet seen a CrXPRT 2 battery life result from a ChromeOS Flex environment. We plan to experiment with this soon.
We
were happy to publish the results on our site, and will consider any ChromeOS
Flex results we receive for publication. If you submit results from ChromeOS
Flex testing, we ask that you use the “Additional information” field in the
results submission form to clarify that you ran the tests in a ChromeOS Flex
environment. This will prevent any possible confusion when we see a submission
that lists a traditional macOS or Windows hardware platform along with a
ChromeOS version number.
Do you have experience running CrXPRT or WebXPRT with ChromeOS Flex? We’d love to hear about it!
In March, we discussed the Chrome OS team’s plan to end support for Chrome apps in June and instead focus their
efforts on Chrome extensions and Progressive Web Apps. After receiving feedback
on their published timeline, the Chrome OS team decided to extend Chrome app support for Enterprise and Education account
customers through January 2025. Because we publish our Chrome app (CrXPRT) through a private BenchmarkXPRT developer account, and because
we have not seen any further updates to the support timeline, we don’t assume
that the support extension will apply to CrXPRT.
Since June has come and gone, and the support extension probably does not apply to our account, we do not expect to be able to publish any future fixes or updates for CrXPRT. As of now, and up through Chrome 105, the CrXPRT 2 performance and battery life tests are still working without a hitch. We will continue to run the benchmark on a regular basis to monitor functionality, and we will disclose any future issues here in the blog and on CrXPRT.com. We hope the app will continue to run both performance and battery life tests well into the future. However, given the frequency of Chrome updates, it’s difficult for us to predict how long the benchmark will remain viable.
As we mentioned back in March, we hope to begin development of an all-new Chrome OS XPRT benchmark by the end of this year. We’ll discuss that prospect in more detail in future blog posts, but if you have ideas about the types of features or workloads you’d like to see in a new Chrome OS benchmark, please let us know!
Recently, a tester contacted us with details from a CrXPRT 2 performance test run that they’d successfully completed on… an Apple MacBook Pro! Because CrXPRT 2 is a Chrome Web App that we designed for Chrome OS, it was quite a surprise to hear that it is now possible to run CrXPRT 2 on non-Chrome OS platforms by using FydeOS.
FydeOS is an operating system based on a fork of the Chromium OS project. Developers originally intended FydeOS to be a Google-independent, Chrome-like alternative for the Chinese educational market, but FydeOS is now available to the English-speaking consumer and enterprise markets as well. FydeOS users can run a Chrome-like OS on something other than a Chromebook or a Chromebox, such as a PC, Mac, virtual machine, or even a Raspberry Pi device. Additionally, FydeOS supports Android, Chrome OS, and Linux apps, and users can run those apps at the same time on the same screen.
We have not yet conducted any testing with FydeOS in our lab, but we wanted to pass along this information to any readers who may be interested. If the OS operates as described, it may provide a way for us to experiment with using CrXPRT 2 in some interesting cross-platform tests.
From time to time, we like to run a series of in-house WebXPRT comparison tests to see if recent updates have changed the performance rankings of popular web browsers. We published our most recent comparison last October, when we used WebXPRT 3 to compare Windows 10 and Windows 11 browser performance on the same system. Now that WebXPRT 4 is live, it’s time to update our comparison series with the newest member of the XPRT family.
For this round of tests, we used a Dell
XPS 13 7930, which features an Intel Core i3-10110U processor and 4 GB of RAM, running
Windows 11 Home updated to version 21H2 (22000.593). We installed all current
Windows updates and tested on a clean system image. After the update process
completed, we turned off updates to prevent them from interfering with test
runs. We ran WebXPRT 4 three times each across five browsers: Brave, Google
Chrome, Microsoft Edge, Mozilla Firefox, and Opera. The posted score for each
browser is the median of the three test runs.
In our previous round of tests with WebXPRT 3, Google Chrome narrowly beat out Firefox in Windows 10 and Windows 11 testing, but the scores among three of the Chromium-based browsers (Chrome, Edge, and Opera) were close enough that most users performing common daily tasks would be unlikely to notice a difference. Brave performance lagged by about 7 percent, a difference that may be noticeable to most users. This time, when testing updated versions of the browsers with WebXPRT 4 on Windows 11, the rankings changed. Edge was the clear winner, with a 2.2 percent performance advantage over Chrome. Firefox came in last, about 3 percent slower than Opera, which was in the middle of the pack. Performance from Brave improved to the point that it was no longer lagging the other Chromium-based browsers.
Do these results mean that Microsoft
Edge will always provide you with a speedier web experience? A device with a
higher WebXPRT score will probably feel faster during daily use than one with a
lower score. For comparisons on the same system, however, the answer depends in
part on the types of things you do on the web, how the extensions you’ve
installed affect performance, how frequently the browsers issue updates and
incorporate new web technologies, and how accurately each browser’s default
installation settings reflect how you would set up that browser for your daily
workflow.
In addition, browser speed can
increase or decrease significantly after an update, only to swing back in the
other direction shortly thereafter. OS-specific optimizations can also affect
performance, such as with Edge on Windows 11 and Chrome on Chrome OS. All these
variables are important to keep in mind when considering how WebXPRT results
translate to your everyday experience.
Do you have insights you’d like to share from using WebXPRT to compare browser performance? Let us know!
Last March, we discussed the Chrome OS team’s original announcement that they would be phasing out support for Chrome Apps altogether in June 2021, and would shift their focus to Chrome extensions and Progressive Web Apps. The Chrome OS team eventually extended support for existing Chrome Apps through June 2022, but as of this week, we see no indication that they will further extend support for Chrome Apps published with general developer accounts. If the end-of-life schedule for Chrome Apps does not change in the next few months, both CrXPRT 2 and CrXPRT 2015 will stop working on new versions of Chrome OS at some point in June.
To maintain CrXPRT
functionality past June, we would need to rebuild the app completely—either as
a Progressive Web App or in some other form. For this reason, we want to
reassess our approach to Chrome OS testing, and investigate which features and
technologies to include in a new Chrome OS benchmark. Our current goal is to
gather feedback and conduct exploratory research over the next few months, and begin
developing an all-new Chrome OS benchmark for publication by the end of the
year.
While we will discuss ideas for this new Chrome OS benchmark in future blog posts, we welcome ideas from CrXPRT users now. What features or workloads would you like the new benchmark to retain? Would you like us to remove any components from the existing benchmark? Does the battery life test in its current form suit your needs? If you have any thoughts about these questions or any other aspects of Chrome OS benchmarking, please let us know!
Cookie Notice: Our website uses cookies to deliver a smooth experience by storing logins and saving user information. By continuing to use our site, you agree with our usage of cookies as our privacy policy outlines.
