BenchmarkXPRT Blog banner

Category: Performance benchmarking

Comparing the WebXPRT 4 performance of five popular browsers

Every so often, we like to refresh a series of in-house WebXPRT comparison tests to see if recent updates have changed the performance rankings of popular web browsers. We published our most recent comparison last February, when we used WebXPRT 4 to compare the performance of five browsers on the same system.

For this round of tests, we used the same Dell XPS 13 7930 laptop as last time, which features an Intel Core i3-10110U processor and 4 GB of RAM, running Windows 11 Home updated to version 23H2 (22631.307). We installed all current Windows updates, and updated each of the browsers under test: Brave, Google Chrome, Microsoft Edge, Mozilla Firefox, and Opera.

After the update process completed, we turned off updates to prevent them from interfering with test runs. We ran WebXPRT 4 three times on each of the five browsers. The score we post for each browser is the median of the three test runs.

In our last round of tests, the range between high and low scores was tight, with an overall difference of only 4.3 percent. Edge squeaked out a win, with a 2.1 percent performance advantage over Chrome. Firefox came in last, but was only one overall score point behind the tied score of Brave and Opera.

In this round of testing, the rank order did not change, but we saw more differentiation in the range of scores. While the performance of each browser improved, the range between high and low scores widened to a 19.1 percent difference between first-place Edge and last-place Firefox. The scores of the four Chromium-based browsers (Brave, Opera, Chrome, and Edge) all improved by at least 21 points, while the Firefox score only improved by one point. 

Do these results mean that Microsoft Edge will always provide a faster web experience, or Firefox will always be slower than the others? Not necessarily. It’s true that a device with a higher WebXPRT score will probably feel faster during daily web activities than one with a much lower score, but your experience depends in part on the types of things you do on the web, along with your system’s privacy settings, memory load, ecosystem integration, extension activity, and web app capabilities.

In addition, browser speed can noticeably increase or decrease after an update, and OS-specific optimizations can affect performance, such as with Edge on Windows 11 and Chrome on Chrome OS. All these variables are important to keep in mind when considering how WebXPRT results may translate to your everyday experience.

Have you used WebXPRT 4 to compare browser performance on the same system? Let us know how it turned out!

Justin

Looking back on 2023 with the XPRTs

Around the beginning of each new year, we like to take the opportunity to look back and summarize the XPRT highlights from the previous year. Readers of our newsletter are familiar with the stats and updates we include each month, but for our blog readers who don’t receive the newsletter, we’ve compiled highlights from 2023 below.

Benchmarks
In March, we celebrated the 10-year anniversary of WebXPRT! WebXPRT 4 has now taken the lead as the most commonly-used version of WebXPRT, even as the overall number of runs has continued to grow.

XPRTs in the media
Journalists, advertisers, and analysts referenced the XPRTs thousands of times in 2023. It’s always rewarding to know that the XPRTs have proven to be useful and reliable assessment tools for technology publications around the world. Media sites that used the XPRTs in 2023 include 3DNews (Russia), AnandTech, Benchlife.info (China), CHIP.pl (Poland), ComputerBase (Germany), eTeknix, Expert Reviews, Gadgetrip (Japan), Gadgets 360, Gizmodo, Hardware.info, IT168.com (China), ITC.ua (Ukraine), ITWorld (Korea), iXBT.com (Russia), Lyd & Bilde (Norway), Notebookcheck, Onchrome (Germany), PCMag, PCWorld, QQ.com (China), Tech Advisor, TechPowerUp, TechRadar, Tom’s Guide, TweakTown, Yesky.com (China), and ZDNet.

Downloads and confirmed runs
In 2023, we had more than 16,800 benchmark downloads and 296,800 confirmed runs. Users have run our most popular benchmark, WebXPRT, more than 1,376,500 times since its debut in 2013! WebXPRT continues to be a go-to, industry-standard performance benchmark for OEM labs, vendors, and leading tech press outlets around the globe.

Trade shows
In January, Justin attended the 2023 Consumer Electronics Show (CES) Las Vegas. In March, Mark attended Mobile World Congress (MWC) 2023 in Barcelona. You can view Justin’s recap of CES here and Mark’s thoughts from MWC here.

We’re thankful for everyone who used the XPRTs and sent questions and suggestions throughout 2023. We’re excited to see what’s in store for the XPRTs in 2024!

Justin

WebXPRT 4 is good to go with the latest Apple software release

Last month, we reported the good news that our WebXPRT 4 tests successfully ran to completion on the beta releases of iOS 17.2, iPadOS 17.2, and macOS Sonoma 14.2 with Safari 17.2. When we tested with those beta builds, WebXPRT 4 did not encounter the issue of test runs getting stuck on iOS 17.1 while attempting to complete the receipt scanning task in the Encrypt Notes and OCR Scan subtest. Unfortunately, during the past several weeks, this fix was only available to Apple users running beta software through the Apple Developer Program.

We’re happy to report that Apple has now finalized and published the general releases of iOS 17.2, iPadOS 17.2, and Safari 17.2. WebXPRT 4 tests running on those platforms should now complete without any problems.

We do appreciate everyone’s patience as we worked to find a solution to this problem, and we look forward to seeing your WebXPRT 4 scores from all the latest Apple devices! If you have any questions or concerns about WebXPRT 4, or you encounter any additional issues when running the test on any platform, please let us know.

Justin

The evolving PC market brings new opportunities for WebXPRT

Here at the XPRTs, we have to spend time examining what’s next in the tech industry, because the XPRTs have to keep up with the pace of innovation. In our recent discussions about 2024, a major recurring topic has been the potential impact of Qualcomm’s upcoming line of SOCs designed for Windows on Arm PCs.

Now, Windows on Arm PCs are certainly not new. Since Windows RT launched on the Arm-based Microsoft Surface RT in 2012, various Windows on Arm devices have come and gone, but none of them—except for some Microsoft SQ-based Surface devices—have made much of a name for themselves in the consumer market.

The reasons for these struggles are straightforward. While Arm-based PCs have the potential to offer consumers the benefits of excellent battery life and “always-on” mobile communications, the platform has historically lagged Intel- and AMD-based PCs in performance. Windows on Arm devices have also faced the challenge of a lack of large-scale buy-in from app developers. So, despite the past involvement of device makers like ASUS, HP, Lenovo, and Microsoft, the major theme of the Windows on Arm story has been one of very limited market acceptance.

Next year, though, the theme of that story may change. If it does, WebXPRT 4 is well-positioned to play an important part.

At the recent Qualcomm Technology Summit, the company unveiled the new 4nm Snapdragon X Elite SOC, which includes an all-new 12-core Oryon CPU, an integrated Adreno GPU, and an integrated Hexagon NPU (neural processing unit) designed for AI-powered applications. Company officials presented performance numbers that showed the X Elite surpassing the performance of late-gen AMD, Apple, and Intel competitor platforms, all while using less power.

Those are massive claims, and of course the proof will come—or not—only when systems are available for test. (In the past, companies have made similar claims about Windows on Arm advantages, only to see those claims evaporate by the time production devices show up on store shelves.)

Will Snapdragon X Elite systems demonstrate unprecedented performance and battery life when they hit the market? How will the performance of those devices stack up to Intel’s Meteor Lake systems and Apple’s M3 offerings? We don’t yet know how these new devices may shake up the PC market, but we do know that it looks like 2024 will present us with many golden opportunities for benchmarking. Amid all the marketing buzz, buyers everywhere will want to know about potential trade-offs between price, power, and battery life. Tech reviewers will want to dive into the details and provide useful data points, but many traditional PC benchmarks simply won’t work with Windows on ARM systems. As a go-to, cross-platform favorite of many OEMs—that runs on just about anything with a browser—WebXPRT 4 is in a perfect position to provide reviewers and consumers with relevant performance comparison data.

It’s quite possible that 2024 may be the biggest year for WebXPRT yet!

Justin

Passing two important WebXPRT milestones

Over the past few months, we’ve been excited to see a substantial increase in the total number of completed WebXPRT runs. To put the increase in perspective, we had more total WebXPRT runs last month alone (40,453) than we had in the first two years WebXPRT was available (36,674)! This boost has helped us to reach two important milestones as we close in on the end of 2023.

The first milestone is that the number of WebXPRT 4 runs per month now exceeds the number of WebXPRT 3 runs per month. When we release a new version of an XPRT benchmark, it can take a while for users to transition from using the older version. For OEM labs and tech journalists, adding a new benchmark to their testing suite often involves a significant investment in back testing and gathering enough test data for meaningful comparisons. When the older version of the benchmark has been very successful, adoption of the new version can take longer. WebXPRT 3 has been remarkably popular around the world, so we’re excited to see WebXPRT 4 gain traction and take the lead even as the total number of WebXPRT runs increases each month. The chart below shows the number of WebXPRT runs per month for each version of WebXPRT over the past ten years. WebXPRT 4 usage first surpassed WebXPRT 3 in August of this year, and after looking at data for the last three months, we think its lead is here to stay.

The second important milestone is the cumulative number of WebXPRT runs, which recently passed 1.25 million, as the chart below shows. For us, this moment represents more than a numerical milestone. For a benchmark to succeed, developers need the trust and support of the benchmarking community. WebXPRT’s consistent year-over-year growth tells us that the benchmark continues to hold value for manufacturers, OEM labs, the tech press, and end users. We see it as a sign of trust that folks repeatedly return to the benchmark for reliable performance metrics. We’re grateful for that trust, and for everyone that has contributed to the WebXPRT development process over the years.

We look forward to seeing how far WebXPRT’s reach can extend in 2024! If you have any questions or comments about using WebXPRT, let us know!

Justin

Good news for WebXPRT 4 testing!

Over the past several weeks, we’ve been working to find a solution to a problem with WebXPRT 4 test failures on Apple devices running iOS 17/17.1, iPadOS 17/17.1, and macOS Sonoma with Safari 17/17.1. While we put significant effort into an updated WebXPRT version that would mitigate this issue, we are happy to report that it now looks like we’ll be able to stick with the current version!

Last Thursday, Apple released the iOS 17.2 beta for participants in the Apple Developer Program. When we tested the current version of WebXPRT 4 on iOS 17.2, the tests completed without any issues. We then successfully completed tests on iPadOS 17.2 and macOS Sonoma 14.2 with Safari 17.2. Now that we have good reasons to believe that the iOS 17.2 release will solve the problem, sticking with the current WebXPRT 4 build will maximize continuity and minimize disruption for WebXPRT users.

Apple has not yet published a public release date for iOS/iPad OS/Safari 17.2. Based on past development schedules, it seems likely that they will release it between mid-November and early December, but that’s simply our best guess. Until then, users who want to test WebXPRT 4 on devices running iOS 17/17.1, iPadOS 17/17.1, or macOS Sonoma with Safari 17/17.1 will need to update those devices to iOS/iPad OS/Safari 17.2 via the Apple Developer Program.

To help Apple users better navigate testing until the public 17.2 release, we’ve added a function to the current WebXPRT 4 start page that will notify users if they need to update their operating system to test.

We appreciate everyone’s patience as we worked to find a solution to this problem! If you have any questions or concerns about WebXPRT 4, please let us know.

Justin

Check out the other XPRTs:

Forgot your password?