BenchmarkXPRT Blog banner

Tag Archives: WebXPRT 4

Best practices for WebXPRT testing

One of the strengths of WebXPRT is that it’s a remarkably easy benchmark to run. Its upfront simplicity attracts users with a wide range of technical skills—everyone from engineers in cutting-edge OEM labs to veteran tech journalists to everyday folks who simply want to test their gear’s browser performance. With so many different kinds of people running the test each day, it’s certain that at least some of them use very different approaches to testing. In today’s blog, we’re going to share some of the key benchmarking practices we follow in the XPRT lab—and encourage you to consider—in order to produce the most consistent and reliable WebXPRT scores.

We offer these best practices as tips you might find useful in your testing. Each step relates to evaluating browser performance with WebXPRT, but several of these practices will apply to other benchmarks as well.

  • Test with clean images: In the XPRT lab, we typically use an out-of-box (OOB) method for testing new devices. OOB testing means that other than running the initial OS and browser version updates that users are likely to run after first turning on the device, we change as little as possible before testing. We want to assess the performance that buyers are likely to see when they first purchase the device and before they install additional software. This approach is the best way to provide an accurate assessment of the performance retail buyers will experience from their new devices. That said, the OOB method is not appropriate for certain types of testing, such as when you want to compare largely identical systems or when you want to remove as much pre-loaded software as possible. The OOB method is less relevant to users who want to see how their device performs as it is.
  • Browser updates can have a significant impact: Most people know that different browsers often produce different performance scores on the same system. They may not know that there can be shifts in performance between different versions of the same browser. While most browser updates don’t have a large impact on performance, a few updates have increased (or even decreased) browser performance by a significant amount. For this reason, it’s always important to record and disclose the extended browser version number for each test run. The same principle applies to any other relevant software.
  • Turn off automatic updates: We do our best to eliminate or minimize app and system updates after initial setup. Some vendors are making it more difficult to turn off updates completely, but you should always double-check update settings before testing. On Windows systems, the same considerations apply to turning off User Account Control notifications.
  • Let the system settle: Depending on the system and the OS, a significant amount of system-level activity can be going on in the background after you turn it on. As much as possible, we like to wait for a stable baseline (idle time) of system activity before kicking off a test. If we start testing immediately after booting the system, we often see higher variance in the first run before the scores start to tighten up.
  • Run the test more than once: Because of natural variance, our standard practice in the XPRT lab is to publish a score that represents the median of three to five runs, if not more. If you run a benchmark only once and the score differs significantly from other published scores, your result could be an outlier that you would not see again under stable testing conditions or over the course of multiple runs.
  • Clear the cache: Browser caching can improve web page performance, including the loading of the types of JavaScript and HTML5 assets that WebXPRT uses in its workloads. Depending on the platform under test, browser caching may or may not significantly change WebXPRT scores, but clearing the cache before testing and between each run can help improve the accuracy and consistency of scores.

We hope these tips will serve as a good baseline methodology for your WebXPRT testing. If you have any questions about WebXPRT, the other XPRTs, or benchmarking in general, please let us know!

Justin

Recent XPRT mentions in articles, reviews, and more!

Here at the XPRTs, our primary goal is to provide free, easy-to-use benchmark tools that can help everyone—from OEM labs to tech press journalists to individual consumers—understand how well devices will perform while completing everyday computing tasks. We track progress toward that goal in several ways, but one of the most important is how much people use and discuss the XPRTs. When the name of one of our apps appears in an ad, article, or tech review, we call it a “mention.” Tracking mentions helps us gauge our reach.

We occasionally like to share a sample of recent XPRT mentions here in the blog. If you just started following the XPRTs, it may be surprising to see our program’s global reach. If you’re a longtime reader and you’re used to seeing WebXPRT or CrXPRT in major tech press articles, it may be surprising to learn more about overseas tech press publications or see how some government agencies use the XPRTs to make decisions. In any case, we hope you’ll enjoy exploring the links below!

Recent mentions include:

If you’d like to receive monthly updates on XPRT-related news and activity, we encourage you to sign up for the BenchmarkXPRT Development Community newsletter. It’s completely free, and all you need to do to join the newsletter mailing list is let us know! We won’t publish, share, or sell any of the contact information you provide, and we’ll only send you the monthly newsletter and occasional benchmark-related announcements, such as important news about patches or releases.

If you have any questions about the XPRTs, suggestions, or requests for future blog topics, please feel free to contact us.

Justin

February 2025 WebXPRT 4 browser performance comparisons

Once or twice per year, we refresh our ongoing series of WebXPRT comparison tests to see if software version updates have reordered the performance rankings of popular web browsers. We published our most recent comparison last June, when we used WebXPRT 4 to compare the performance of five browsers—Brave, Google Chrome, Microsoft Edge, Mozilla Firefox, and Opera—on a Lenovo ThinkPad T14s Gen 3. When assessing performance differences, it’s worth noting that all the browsers—except for Firefox—are built on a Chromium foundation. In the last round of tests, the scores were very tight, with a difference of only four percent between the last-place browser (Brave) and the winner (Chrome). Firefox’s score landed squarely in the middle of the pack.

Recently, we conducted a new set of tests to see how performance scores may have changed. To maintain continuity with our last comparison, we stuck with the same ThinkPad T14s as our reference system. That laptop is still in line with current mid-range laptops, so our comparison scores are likely to fall within the range of scores we would see from a typical user today. The ThinkPad is equipped with an Intel Core i7-1270P processor and 16 GB of RAM, and it’s running Windows 11 Pro, version 23H2 (22631.4890).

Before testing, we installed all current Windows updates, and we updated each of the browsers to the latest available stable version. After the update process was complete, we turned off updates to prevent any interference with test runs. We ran WebXPRT 4 five times on each of the five browsers. In Figure 1 below, each browser’s score is the median of the five test runs.

In this round of tests, the gap widened a bit between first and last place scores, with a difference of just over six percent between the lowest median score of 303 (Brave) and the highest median score of 322 (Firefox).

Figure 1: The median scores from running WebXPRT 4 five times with each browser on the Lenovo ThinkPad T14s Gen 3.

In this round of tests, the distribution of scores indicates that most users would not see a significant performance difference if they switched between the latest versions of these browsers. The one exception may be a change from the latest version of Brave to the latest version of Firefox. Even then, the quality of your browsing experience will often depend on other factors. The types of things you do on the web (e.g., gaming, media consumption, or multi-tab browsing), the type and number of extensions you’ve installed, and how frequently the browsers issue updates and integrate new technologies—among other things—can all affect browser performance over time. It’s important to keep such variables in mind when thinking about how browser performance comparison results may translate to your everyday web experience.

Have you tried using WebXPRT 4 in your own browser performance comparison? If so, we’d love to hear about it! Also, please let us know if there are other types of WebXPRT comparisons you’d like to see!

Justin

The XPRTs: What would you like to see in 2025?

If you’re a new follower of the XPRT family of benchmarks, you may not be aware of one of the characteristics of the XPRTs that sets them apart from many benchmarking efforts—our openness and commitment to valuing the feedback of tech journalists, lab engineers, and anyone else that uses the XPRTs on a regular basis. That feedback helps us to ensure that as the XPRTs grow and evolve, the resources we offer will continue to meet the needs of those that use them.

In the past, user feedback has influenced specific aspects of our benchmarks, such as the length of test runs, UI features, results presentation, and the addition or subtraction of specific workloads. More broadly, we have also received suggestions for entirely new XPRTs and ways we might target emerging technologies or industry use cases.

As we look forward to what’s in store for the XPRTs in 2025, we’d love to hear your ideas about new XPRTs—or new features for existing XPRTs. Are you aware of hardware form factors, software platforms, new technologies, or prominent applications that are difficult or impossible to evaluate using existing performance benchmarks? Should we incorporate additional or different technologies into existing XPRTs through new workloads? Do you have suggestions for ways to improve any of the XPRTs or XPRT-related tools, such as results viewers?

We’re especially interested in your thoughts about the next steps for WebXPRT. If our recent blog posts about the potential addition of an AI-focused auxiliary workload, what a WebXPRT battery life test would entail, or possible WebAssembly-based test scenarios have piqued your interest, we’d love to hear your thoughts!

We’re genuinely interested in your answers to these questions and any other ideas you have, so please feel free to contact us. We look forward to hearing your thoughts and working together to figure out how they could help shape the XPRTs in 2025!

Justin

Using WebXPRT 4’s language options

Sohu, a major Chinese site, recently published a tech review discussing their first impressions from Intel Core Ultra 5 245K and Intel Core Ultra 9 285K white box testing. In the article, they included screenshots of the WebXPRT 4 test results they produced during their evaluation. The screenshots showed that the testers had enabled WebXPRT 4’s Simplified Chinese UI. They’re not the first to use this option, and it’s one we are glad worked for them.

Though WebXPRT’s language settings menu has proven to be a popular feature for many users around the world, some folks may not even know the option is there. In today’s blog, we’ll go over the basics of this simple but helpful testing option.

On WebXPRT’s Start screen, you can choose from three language options in the WebXPRT 4 UI: Simplified Chinese, German, and English. We included Simplified Chinese and German because of the large number of tests we see from China and Central Europe. We wanted to make testing a little easier for users who prefer those languages and we’re glad to see people using the options.

Changing languages in the WebXPRT UI is quick and easy. Locate the “Change Language?” prompt under the WebXPRT 4 logo at the top of the Start screen, and click or tap the arrow beside it. After the drop-down menu appears, select the language you want. The language of the start screen will then change to the language you selected, and the in-test workload headers and end-of-test results screen will also appear in the language you selected.

Figures 1–3 below my sig show the “Change Language?” drop-down menu and how the Start screen appears when you select Simplified Chinese or German. It’s important to note that if you have a translation extension installed in your browser, it may override the WebXPRT UI by reverting the language back to your browser’s default. You can avoid this conflict by temporarily disabling the browser’s translation extension for the duration of WebXPRT testing.

We hope WebXPRT 4’s language options will help facilitate the testing process for many users around the world. If you’re a frequent WebXPRT user and would like to see us add support for another language, please contact us. And, of course, if you have any questions about WebXPRT 4 testing, please let us know!

Justin

Figure 1: A screenshot of the WebXPRT 4 Start screen showing the language options drop-down menu.
Figure 2: A screenshot of the WebXPRT 4 Start screen with a Simplified Chinese UI.
Figure 3: A screenshot of the WebXPRT 4 Start screen with a German UI.

Shop confidently this holiday season with the XPRTs!

The holiday shopping season is upon us, and trying to find the right tech gift for your friends or loved ones (or yourself!) can be a daunting task. If you’re considering new phones, tablets, Chromebooks, laptops, or desktops as gifts this year—and are unsure where to get reliable device information—the XPRTs can help!

The XPRTs provide industry-trusted and time-tested measures of a device’s performance that can help you cut through the fog of competing marketing claims. For example, instead of guessing whether the performance of a new gaming laptop justifies its price, you can use its WebXPRT performance score to see how it stacks up against both older models and competitors while tackling everyday tasks.

A great place to start looking for device scores is our XPRT results browser, which lets you access our database of more than 3,700 test results—across all the XPRT benchmarks and hundreds of devices—from over 155 sources, including major tech review publications around the world, OEMs, our own Principled Technologies (PT) testing, and independent submissions. For tips on how to use the XPRT results browser, check out this blog post.

Another way to view information in our results database is by using the WebXPRT 4 results viewer. The viewer provides an information-packed, interactive tool that we created to help people explore data from the set of almost 800 WebXPRT 4 results we’ve curated and published to date on our site. You’ll find detailed instructions in this blog post for how to use the WebXPRT 4 results viewer tool.

If you’re considering a popular device, it’s likely that a recent tech press review includes an XPRT score for it. To find those scores, go to your favorite tech review site and search for “XPRT,” or enter the name of the device and the appropriate XPRT (e.g., “iPhone” and “WebXPRT”) in a search engine. Here are a few recent tech reviews that used the XPRTs to evaluate popular devices:

In addition to XPRT-related resources in the tech press, here at PT we frequently publish reports that evaluate the performance of hot new consumer devices, and many of those reports include WebXPRT scores. For example, check out the results from our extensive testing of a Dell Latitude 7450 AI PC or our in-depth evaluation of three new Lenovo ThinkPad and ThinkBook laptops.

The XPRTs can help you make better-informed and more confident tech purchases this holiday season. We hope you’ll find the data you need on our site or in an XPRT-related tech review. If you have any questions about the XPRTs, XPRT scores, or the results database, please feel free to ask!

Justin

Check out the other XPRTs:

Forgot your password?