BenchmarkXPRT Blog banner

Category: Chrome

WebXPRT appearances in recent articles, reviews, and more!

Over the last few weeks, we’ve been excited to see steady growth in the number of people running the WebXPRT 5 Preview—as well as how widely distributed those runs are around the world. To get a picture of WebXPRT’s approximate global reach, we run the IP address data from our database through a program that provides general city- and country-level location information. By tallying that location data, we can better understand where people are running WebXPRT. By the way, this is the most detailed information we have, and all test data will remain completely anonymous unless you submit it for publication.

Based on that info—along with direct feedback from users—we’ve determined that OEM labs, tech press journalists, and individual consumers have already run the WebXPRT 5 Preview in more than 125 cities across 35 countries!

Test run data is just one of the ways that we track the progress we’re making toward our goal of providing all interested parties with benchmark tools that are reliable, relevant, and free to use. Another important way we track progress toward that goal is by paying attention to how much people use and discuss the XPRTs. When the name of one of our apps appears in an ad, article, or tech review, we call it a “mention.” Tracking those kinds of mentions helps us gauge our reach.

From time to time, we like to share a sample of recent XPRT mentions here in the blog. If you just started following the XPRTs, it may be surprising to see our global reach. Whether you’re a new visitor to the XPRT blog or a longtime reader, we hope you’ll enjoy exploring the links below!

Recent mentions include:

If you’d like to receive monthly updates on XPRT-related news and activity, we encourage you to sign up for our XPRT newsletter. It’s completely free, and all you need to do to join the newsletter mailing list is let us know! We won’t publish, share, or sell any of the contact information you provide, and we’ll only send you the monthly newsletter and occasional benchmark-related announcements, such as important news about patches or releases.

If you have any questions about the WebXPRT 5 Preview or the XPRTs in general, please feel free to contact us.

Justin

CrXPRT 2 functionality is ending with ChromeOS 139

Back in January, we discussed the ChromeOS team’s decision to eventually end support for all user-installed Chrome Apps—including CrXPRT 2—upon the release of Chrome 138 in July of this year. As best we can tell, the move is part of their overall strategy of transitioning all support to Chrome extensions and Progressive Web Apps. We knew that after the support cutoff date, we would not be able to publish any fixes or updates for CrXPRT 2, but we weren’t exactly sure how the transition would affect the app’s overall functionality.

We’ve now confirmed that while CrXPRT 2 still functions normally through Chrome 138.0.7204.255 (beta), the app does not launch at all on Chrome Canary 139. Consequently, we expect that stable channel system updates will disable CrXPRT 2 on most systems after Chrome 139 goes live on August 5th. We will initially leave CrXPRT 2 on our site for those who want to use it on older versions of Chrome, but over time we will archive it as an inactive benchmark.

We want to extend our heartfelt thanks to the many people around the world who used CrXPRT 2 for lab evaluations, product reviews, and individual testing over the past several years. We’re grateful for your support! We will update readers here in the blog if we decide to pursue new ChromeOS benchmark development work in the future.

Justin

February 2025 WebXPRT 4 browser performance comparisons

Once or twice per year, we refresh our ongoing series of WebXPRT comparison tests to see if software version updates have reordered the performance rankings of popular web browsers. We published our most recent comparison last June, when we used WebXPRT 4 to compare the performance of five browsers—Brave, Google Chrome, Microsoft Edge, Mozilla Firefox, and Opera—on a Lenovo ThinkPad T14s Gen 3. When assessing performance differences, it’s worth noting that all the browsers—except for Firefox—are built on a Chromium foundation. In the last round of tests, the scores were very tight, with a difference of only four percent between the last-place browser (Brave) and the winner (Chrome). Firefox’s score landed squarely in the middle of the pack.

Recently, we conducted a new set of tests to see how performance scores may have changed. To maintain continuity with our last comparison, we stuck with the same ThinkPad T14s as our reference system. That laptop is still in line with current mid-range laptops, so our comparison scores are likely to fall within the range of scores we would see from a typical user today. The ThinkPad is equipped with an Intel Core i7-1270P processor and 16 GB of RAM, and it’s running Windows 11 Pro, version 23H2 (22631.4890).

Before testing, we installed all current Windows updates, and we updated each of the browsers to the latest available stable version. After the update process was complete, we turned off updates to prevent any interference with test runs. We ran WebXPRT 4 five times on each of the five browsers. In Figure 1 below, each browser’s score is the median of the five test runs.

In this round of tests, the gap widened a bit between first and last place scores, with a difference of just over six percent between the lowest median score of 303 (Brave) and the highest median score of 322 (Firefox).

Figure 1: The median scores from running WebXPRT 4 five times with each browser on the Lenovo ThinkPad T14s Gen 3.

In this round of tests, the distribution of scores indicates that most users would not see a significant performance difference if they switched between the latest versions of these browsers. The one exception may be a change from the latest version of Brave to the latest version of Firefox. Even then, the quality of your browsing experience will often depend on other factors. The types of things you do on the web (e.g., gaming, media consumption, or multi-tab browsing), the type and number of extensions you’ve installed, and how frequently the browsers issue updates and integrate new technologies—among other things—can all affect browser performance over time. It’s important to keep such variables in mind when thinking about how browser performance comparison results may translate to your everyday web experience.

Have you tried using WebXPRT 4 in your own browser performance comparison? If so, we’d love to hear about it! Also, please let us know if there are other types of WebXPRT comparisons you’d like to see!

Justin

An update on CrXPRT support in ChromeOS

CrXPRT users may remember that back in 2022, we discussed the ChromeOS team’s decision to end formal support for Chrome Apps and instead focus on Chrome extensions and Progressive Web Apps. This decision meant that we would not be able to publish any future fixes or updates for CrXPRT 2, although moving forward, we weren’t sure how it would affect the app’s functionality.

After receiving a lot of feedback regarding their original timeline, the ChromeOS team decided to extend Chrome App support for Enterprise and Education account customers through January 2025. Because we publish CrXPRT through a private BenchmarkXPRT developer account, we assumed at the time that the support extension would not apply to CrXPRT.

Recently, the ChromeOS team released new information about their scheduled support timeline. Now, they plan to end formal support for all user-installed Chrome Apps in July 2025 (Chrome 138). In February 2028, the Chrome 168 release will mark the end of life for all Chrome Apps.

The good news is that—in spite of a lack of formal ChromeOS support over the past couple of years—the CrXPRT 2 performance and battery life tests have continued to run without any known issues. As of today, the app functions normally up through the Beta release of ChromeOS version 132.0.6834.52.

We will continue to run the benchmark on a regular basis to monitor functionality, and we will disclose any future issues here in the blog and on CrXPRT.com. We hope the app will continue to run both performance and battery life tests well into the future. However, given the frequency of Chrome updates, it’s difficult for us to predict how long the benchmark will remain viable.

If you have any questions about CrXPRT, please let us know!

Justin

Thinking through a potential WebXPRT 4 battery life test

In recent blog posts, we’ve discussed some of the technical considerations we’re working through on our path toward a future AI-focused WebXPRT 4 auxiliary workload. While we’re especially excited about adding to WebXPRT 4’s AI performance evaluation capabilities, AI is not the only area of potential WebXPRT 4 expansion that we’ve thought about. We’re always open to hearing suggestions for ways we can improve WebXPRT 4, including any workload proposals you may have. Several users have asked about the possibility of a WebXPRT 4 battery life test, so today we’ll discuss what one might look like and some of the challenges we’d have to overcome to make it a reality.

Battery life tests fall into two primary categories: simple rundown tests and performance-weighted tests. Simple rundown tests measure battery life during extreme idle periods and loops of movie playbacks, etc., but do not reflect the wide-ranging mix of activities that characterize a typical day for most users. While they can be useful for performing very specific apples-to-apples comparisons, these tests don’t always give consumers an accurate estimate of the battery life they would experience in daily use.

In contrast, performance-weighted battery life tests, such as the one in CrXPRT 2, attempt to reflect real-world usage. The CrXPRT battery life test simulates common daily usage patterns for Chromebooks by including all the productivity workloads from the performance test, plus video playback, audio playback, and gaming scenarios. It also includes periods of wait/idle time. We believe this mixture of diverse activity and idle time better represents typical real-life behavior patterns. This makes the resulting estimated battery life much more helpful for consumers who are trying to match a device’s capabilities with their real-world needs.

From a technical standpoint, WebXPRT’s cross-platform nature presents us with several challenges that we did not face while developing the CrXPRT battery life test for ChromeOS. While the WebXPRT performance tests run in almost any browser, cross-browser differences and limitations in battery life reporting may restrict any future battery life test to a single browser or browser family. For instance, with the W3C Battery Status API, we can currently query battery status data from non-mobile Chromium-based browsers (e.g., Chrome, Edge, Opera, etc.), but not from Firefox or Safari. If a WebXPRT 4 battery life test supported only a single browser family, such as Chromium-based browsers, would you still be interested in using it? Please let us know.

A browser-based battery life workflow also presents other challenges that we do not face in native client applications, such as CrXPRT:

  • A browser-based battery life test may require the user to check the starting and ending battery capacities, with no way for the app to independently verify data accuracy.
  • The battery life test could require more babysitting in the event of network issues. We can catch network failures and try to handle them by reporting periods of network disconnection, but those interruptions could influence the battery life duration.
  • The factors above could make it difficult to achieve repeatability. One way to address that problem would be to run the test in a standardized lab environment with a steady internet connection, but a long list of standardized environmental requirements would make the battery life test less attractive and less accessible to many testers.

We’re not sharing these thoughts to make a WebXPRT 4 battery life test seem like an impossibility. Rather, we want to offer our perspective on what the test might look like and describe some of the challenges and considerations in play. If you have thoughts about battery life testing, or experience with battery life APIs in one or more of the major browsers, we’d love to hear from you!

Justin

Web APIs: Possible paths for the AI-focused WebXPRT 4 auxiliary workload

In our last blog post, we discussed one of the major decision points we’re facing as we work on what we hope will be the first new AI-focused WebXPRT 4 auxiliary workload: choosing a Web AI framework. In today’s blog, we’re discussing another significant decision that we need to make for the future workload’s development path: choosing a web API.

Many of you are familiar with the concept of an application programming interface (API). Simply put, APIs implement sets of software rules, tools, and/or protocols that serve as intermediaries that make it possible for different computer programs or components to communicate with each other. APIs simplify many development tasks for programmers and provide standardized ways for applications to share data, functions, and system resources.

Web APIs fulfill the intermediary role of an API—through HTTP-based communication—for web servers (on the server side) or web browsers (on the client side). Client-side web APIs make it possible for browser-based applications to expand browser functionality. They execute the kinds of JavaScript, HTML5, and WebAssembly (Wasm) workloads—among other examples—that support the wide variety of browser extensions many of us use every day. WebXPRT uses those types of browser-based workloads to evaluate system performance. To lay a solid foundation for the first future browser-based AI workload, we need to choose a web API that will be compatible with WebXPRT and the Web AI framework and AI inference workload(s) we ultimately choose.

Currently, there are three main web API paths for running AI inference in a web browser: Web Neural Network (WebNN), Wasm, and WebGPU. These three web technologies are in various stages of development and standardization. Each has different levels of support within the major browsers. Here are basic overviews of each of the three options, as well as a few of our thoughts on the benefits and limitations that each may bring to the table for a future WebXPRT AI workload:

  • WebNN is a JavaScript API that enables developers to directly execute machine learning (ML) tasks on neural networks within web-based applications. WebNN makes it easier to integrate ML models into web apps, and it allows web apps to leverage the power of neural processing units (NPUs). WebNN has a lot going for it. It’s hardware-agnostic and works with various ML frameworks. It’s likely to be a major player in future browser-based inference applications. However, as a web standard, WebNN is still in the development stage and is only available in developer previews for Chromium-based browsers. Full default WebNN support could take a year or more.
  • Wasm is a binary instruction format that works across all modern browsers. Wasm provides a sandboxed environment that operates at near-native speeds and takes advantage of common hardware specs across platforms. Wasm’s capabilities offer web developers a great deal of flexibility for running complex client applications in the browser. Simply put, Wasm can help developers adapt their existing code for additional platforms and browser-based applications without requiring extensive code rewrites. Wasm’s flexibility and cross-platform compatibility is one of the reasons that we’ve already made use of Wasm in two existing WebXPRT 4 workloads that feature AI tasks: Organize Album using AI, and Encrypt Notes and OCR Scan. Wasm can also work together with other web APIs, such as WebGPU.
  • WebGPU enables web-based applications to directly access the graphics rendering and computational capabilities of a system’s GPU. The parallel computational abilities of GPUs make them especially well-suited to efficiently handle some of the demands of AI inference workloads, including image-based GenAI workloads or large language models. Google Chrome and Microsoft Edge currently support WebGPU, and it’s available in Safari through a tech preview.

Right now, we don’t think that WebNN will be fully out of the development phase in time to serve as our go-to web API for a new WebXPRT AI workload. Wasm and/or WebGPU appear to our best options for now. When WebNN is fully baked and available in mainstream browsers, it’s possible that we could port any existing Wasm- or WebGPU-based WebXPRT AI workloads to WebNN, which may open the possibility of cross-platform browser-based NPU performance comparisons.

All that said and as we mentioned in our previous post about Web AI frameworks, we have not made any final decisions about a web API or any aspect of the future workload. We’re still in the early stages of this project. We want your input.

If this discussion has sparked web AI ideas that you think would benefit the process, or if you have feedback you’d like to share, please feel free to contact us!

Justin

Check out the other XPRTs:

Forgot your password?