BenchmarkXPRT Blog banner

Category: BenchmarkXPRT

The evolving PC market brings new opportunities for WebXPRT

Here at the XPRTs, we have to spend time examining what’s next in the tech industry, because the XPRTs have to keep up with the pace of innovation. In our recent discussions about 2024, a major recurring topic has been the potential impact of Qualcomm’s upcoming line of SOCs designed for Windows on Arm PCs.

Now, Windows on Arm PCs are certainly not new. Since Windows RT launched on the Arm-based Microsoft Surface RT in 2012, various Windows on Arm devices have come and gone, but none of them—except for some Microsoft SQ-based Surface devices—have made much of a name for themselves in the consumer market.

The reasons for these struggles are straightforward. While Arm-based PCs have the potential to offer consumers the benefits of excellent battery life and “always-on” mobile communications, the platform has historically lagged Intel- and AMD-based PCs in performance. Windows on Arm devices have also faced the challenge of a lack of large-scale buy-in from app developers. So, despite the past involvement of device makers like ASUS, HP, Lenovo, and Microsoft, the major theme of the Windows on Arm story has been one of very limited market acceptance.

Next year, though, the theme of that story may change. If it does, WebXPRT 4 is well-positioned to play an important part.

At the recent Qualcomm Technology Summit, the company unveiled the new 4nm Snapdragon X Elite SOC, which includes an all-new 12-core Oryon CPU, an integrated Adreno GPU, and an integrated Hexagon NPU (neural processing unit) designed for AI-powered applications. Company officials presented performance numbers that showed the X Elite surpassing the performance of late-gen AMD, Apple, and Intel competitor platforms, all while using less power.

Those are massive claims, and of course the proof will come—or not—only when systems are available for test. (In the past, companies have made similar claims about Windows on Arm advantages, only to see those claims evaporate by the time production devices show up on store shelves.)

Will Snapdragon X Elite systems demonstrate unprecedented performance and battery life when they hit the market? How will the performance of those devices stack up to Intel’s Meteor Lake systems and Apple’s M3 offerings? We don’t yet know how these new devices may shake up the PC market, but we do know that it looks like 2024 will present us with many golden opportunities for benchmarking. Amid all the marketing buzz, buyers everywhere will want to know about potential trade-offs between price, power, and battery life. Tech reviewers will want to dive into the details and provide useful data points, but many traditional PC benchmarks simply won’t work with Windows on ARM systems. As a go-to, cross-platform favorite of many OEMs—that runs on just about anything with a browser—WebXPRT 4 is in a perfect position to provide reviewers and consumers with relevant performance comparison data.

It’s quite possible that 2024 may be the biggest year for WebXPRT yet!

Justin

Recent XPRT mentions in the tech press

Each month, we send out a BenchmarkXPRT Development Community newsletter that contains the latest updates from the XPRT world and provides a summary of the previous month’s XPRT-related activity, including mentions of the XPRTs in the tech press. More people read the weekly XPRT blog than receive the monthly newsletter, so we realized that some blog readers may be unaware of the wide variety of tech outlets that regularly use or mention the XPRTs.

For today’s blog, we want to give readers a sampling of the XPRT press mentions we see on a weekly basis. Recent mentions include:


If you don’t currently receive the monthly BenchmarkXPRT newsletter, but would like to join the mailing list, please let us know! There is no cost to join, and we will not publish or sell any of the contact information you provide. We will send only the monthly newsletter and occasional benchmark-related announcements, such as patch notifications or news of upcoming benchmark releases.

Justin

Passing two important WebXPRT milestones

Over the past few months, we’ve been excited to see a substantial increase in the total number of completed WebXPRT runs. To put the increase in perspective, we had more total WebXPRT runs last month alone (40,453) than we had in the first two years WebXPRT was available (36,674)! This boost has helped us to reach two important milestones as we close in on the end of 2023.

The first milestone is that the number of WebXPRT 4 runs per month now exceeds the number of WebXPRT 3 runs per month. When we release a new version of an XPRT benchmark, it can take a while for users to transition from using the older version. For OEM labs and tech journalists, adding a new benchmark to their testing suite often involves a significant investment in back testing and gathering enough test data for meaningful comparisons. When the older version of the benchmark has been very successful, adoption of the new version can take longer. WebXPRT 3 has been remarkably popular around the world, so we’re excited to see WebXPRT 4 gain traction and take the lead even as the total number of WebXPRT runs increases each month. The chart below shows the number of WebXPRT runs per month for each version of WebXPRT over the past ten years. WebXPRT 4 usage first surpassed WebXPRT 3 in August of this year, and after looking at data for the last three months, we think its lead is here to stay.

The second important milestone is the cumulative number of WebXPRT runs, which recently passed 1.25 million, as the chart below shows. For us, this moment represents more than a numerical milestone. For a benchmark to succeed, developers need the trust and support of the benchmarking community. WebXPRT’s consistent year-over-year growth tells us that the benchmark continues to hold value for manufacturers, OEM labs, the tech press, and end users. We see it as a sign of trust that folks repeatedly return to the benchmark for reliable performance metrics. We’re grateful for that trust, and for everyone that has contributed to the WebXPRT development process over the years.

We look forward to seeing how far WebXPRT’s reach can extend in 2024! If you have any questions or comments about using WebXPRT, let us know!

Justin

Support for MobileXPRT 3 will likely end soon

In a past blog post, we discussed our plan to move several older versions of XPRT benchmarks to an XPRT archive page. Some of those legacy XPRTs still function correctly, and testers occasionally use them, but a few no longer work on the latest versions of the operating systems or browsers that we designed them to test. With the archive page, we can prevent potential confusion for new users who visit current XPRT pages, but still provide longtime users with continued access to old tests.

You can find more information about the XPRTs that we’ll be moving to the archive page here, but today, we want to let MobileXPRT users know that there’s a high likelihood that MobileXPRT 3 will be joining the list of archived XPRTs in the very near future. The Google Play Store has notified us that, due to evolving requirements for apps in newer versions of Android, we must update our MobileXPRT 3 app package to target an Android API level within one year of the latest Android release. If we don’t update the app to meet that requirement by November 1, users will no longer be able to access MobileXPRT 3 through the Google Play Store.

Though a small number of labs and reviewers still use MobileXPRT 3 to test phones and tablets around the world, we don’t feel current usage is high enough for us to justify committing resources to an update at this point. We had hoped that even if MobileXPRT 3 became inaccessible via the Google Play Store, it would still be possible to sideload the app for testing on newer Android devices. After experimenting with installation options in the lab, however, we think it’s likely that settings on devices running Android 11 and up will prevent both Google Play and sideload installations after November 1. The situation may change, but right now, we don’t expect any method to work after that date. If you try, you’ll likely encounter a message during the installation process that says, “This app was built for an older version of Android and may not work properly. Try checking for updates, or contact the developer.” If you attempt to continue the installation process after that message appears, the app will crash.

Both Android and Chrome developers know that the respective stores sometimes extend these types of deadlines. We hope that will be the case here, but we have no information that would lead us to anticipate an extension. If there is no extension, we will still make MobileXPRT 3 available for testing on older Android devices, but we will then have to move it to the XPRT archive page.

We’re grateful for everyone who has used MobileXPRT 3 in the past, and we apologize for any convenience this change may cause. If you have any questions or concerns about MobileXPRT 3 access, please let us know

Justin

An update on the issue with WebXPRT 4 in iOS 17

Recently, we informed XPRT blog readers that after updating Apple iPhones and iPads to iOS and iPadOS 17, respectively, we began to see WebXPRT 4 failures on those devices. In the Safari and Google Chrome browsers, WebXPRT 4 test runs were freezing while running the Encrypt Notes and OCR Scan workload. We were able to replicate the issue on every iOS/iPadOS 17 device we tested, and we also confirmed that WebXPRT 4 continues to run without issues on other non-iOS platforms.

Our team has been investigating the situation, and we’ve made some progress. It’s clear that the failed test runs are getting stuck when the WASM-based Tesseract.js Optical Character Recognition (OCR) engine attempts to scan a shopping receipt. During our research, we’ve discovered an issue when the current Tesseract.js engine runs on iOS 17. This issue is broader than WebXPRT 4, and the Tesseract team is aware of the problem. Future versions of iOS 17 or later versions of Tesseract.js may include fixes for the problem, but unfortunately, we don’t know whether or when a fix will be available.

We’re currently investigating possible workarounds for the problem, and hope to be able to start testing soon. Our goal is that any solution we implement will not significantly affect existing WebXPRT 4 scores on non-iOS 17 platforms.

We will continue to share any substantive progress updates with readers here in the blog. Once again, we apologize for any inconvenience this issue causes for WebXPRT 4 users, and we appreciate your patience while we work toward a solution. If you have any questions or comments, please feel free to contact us!

Justin

A bit of house cleaning at BenchmarkXPRT.com

When we’ve released a new version of an XPRT benchmark app, it’s been our practice for many years to maintain a link to the previous version on the benchmark’s main page. For example, visitors can start on the WebXPRT 4 homepage at WebXPRT.com and follow links to access WebXPRT 3, WebXPRT 2015, and WebXPRT 2013. Historically, we’ve maintained these links because labs and tech reviewers usually take a while to introduce a new benchmark to their testing suite. Continued access to the older benchmarks also allows users to quickly compare new devices to old devices without retesting everything.

That being said, several of the XPRT pages currently contain links to benchmarks that we no longer actively support. Some of those benchmarks still function correctly, and testers occasionally use them, but a few no longer work on the latest versions of the operating systems or browsers that we designed them to test. While we want to continue to provide a way for longtime XPRT users to access legacy XPRTs,  we also want to avoid potential confusion for new users. We believe our best way forward is to archive older tests in a separate part of the site.

In the coming weeks, we’ll be moving several legacy XPRT benchmarks to an archive section of the site. Once the new section is ready, we’ll link to it from the Extras drop-down menu at the top of BenchmarkXPRT.com. The benchmarks will still be available in the archive, but we will not actively support them or directly link to them from the homepages of active XPRTs.

During this process, we’ll move the following benchmarks to the archive section:

  • WebXPRT 2015 and 2013
  • CrXPRT 2015
  • HDXPRT 2014
  • TouchXPRT 2014
  • MobileXPRT 2015 and 2013

If you have any questions or concerns about the archive process or access to legacy XPRTs, please let us know

Justin

Check out the other XPRTs:

Forgot your password?