BenchmarkXPRT Blog banner

Category: BenchmarkXPRT development community

WebXPRT 5 source code access is on the way

Recently, a member of the tech press asked us if we were planning to offer a way for users to set up an offline version of WebXPRT 5 for locally hosted tests. The short answer is “yes.”

The long answer is that the question provides us with a good opportunity to talk about XPRT source code access and let new users know how it works.

Since the early days of the BenchmarkXPRT Development Community, we’ve provided free access to the benchmark source code. We believe that by publishing XPRT code and allowing interested parties to access and review that code, we’re doing our part to encourage transparency and honesty in the benchmarking industry.

While we offer free access to the XPRT source code, our approach to derivative work differs from some traditional open-source models that encourage developers to alter products and even take them in substantially different directions. Because benchmarking requires a product that remains static to enable valid comparisons, we prioritize maintaining the integrity and consistency of the benchmark over time. So, we allow people to download the source, but we also reserve the right to control derivative works. This approach discourages a situation where someone publishes an unauthorized version of the benchmark and calls it an “XPRT.”

For WebXPRT 5, we’ll offer the code in the form of a build package—containing all the necessary files and instructions—that will be available upon request. By offering the code upon request, as opposed to posting a permanent download link, we can prevent bots or other malicious actors from downloading it. This method also lets us engage with users interested in the source code and answer their questions.

With the WebXPRT 5 build package, you’ll be able to set up your own WebXPRT 5 instance for purposes of review, internal testing, or experimentation. We do ask that you publish only test results from the official version of WebXPRT 5 that we host at WebXPRT.com.

We expect to have the build package ready within the next few weeks. When it’s available, we’ll let readers know here in the blog, and we’ll provide more details about the access and setup process.

If you have any questions about accessing the WebXPRT 5 source code, please let us know!

Justin

WebXPRT 5 is live!

The big day has finally arrived—WebXPRT 5 is now available!

You can access the benchmark at WebXPRT.com or WebXPRT5.com. For longtime WebXPRT users, the WebXPRT 5 UI will have an all-new look but a very familiar feel. The general process for kicking off both manual and automated tests is the same as with WebXPRT 4, so the transition to WebXPRT 5 testing will be straightforward. For legacy testing purposes, we will continue to make WebXPRT 4 available on our site.

Here is a quick overview of the differences between WebXPRT 4 and WebXPRT 5:

General changes

  • We’ve updated the aesthetics of the WebXPRT UI to make WebXPRT 5 visually distinct from older versions. We did not significantly change the flow of the UI.
  • We’ve updated content in some of the workloads to reflect changes in everyday technology, such as upgrading most of the photos in the photo processing workloads to higher resolutions.
  • We’ve updated the base calibration system for score calculations and adjusted the scoring scale. WebXPRT 5 scores will be in a lower numerical range than WebXPRT 4 scores. You should not compare these results to scores from previous versions of WebXPRT.

The workloads

WebXPRT 5 includes the following seven workloads:

  • Video Background Blur with AI. Blurs the background of a video call using an AI-powered segmentation model.
  • Photo Effects. Applies a filter to six photos using the Canvas API.
  • Detect Faces with AI. Detects faces and organizes photos in an album using computer vision (OpenCV.js with Caffe Model).
  • Image Classification with AI. Labels images in an album using machine learning (OpenCV.js and ML Classify with the SqueezeNet model).
  • Document Scan with AI. Scans a document image and converts it to text using ML-based OCR (Wasm with LSTM).
  • School Science Project. Processes a DNA sequencing task using Regex and String manipulation.
  • Homework Spellcheck. Spellchecks a document using Typo.js and Web Workers.

We’re thankful for all of the feedback we received during the WebXPRT 5 development process and Preview period, and we look forward to seeing your WebXPRT 5 results. If you have any questions about WebXPRT, please feel free to contact us!

Justin

WebXPRT 5: The workload lineup

The WebXPRT 5 development process heading into the final stretch, so we’d like to share more information about the workloads you’re likely to see in the WebXPRT 5 Preview release—and when that release may be available. We’re still actively testing candidate builds, studying results from multiple system tests, and so on, so some details could change. That said, we’re now close enough to provide a clearer picture of the workload lineup.

Core workloads

WebXPRT 5 will likely include the following seven workloads:  

  • Video Background Blur with AI. Blurs the background of a video call using an AI-powered segmentation model.
  • Photo Effects. Applies a filter to six photos using the Canvas API.
  • Detect Faces with AI. Detects faces and organizes photos in an album using computer vision (OpenCV.js with Caffe Model).
  • Image Classification with AI. Labels images in an album using machine learning (OpenCV.js and ML Classify with the SqueezeNet model).
  • Document Scan with AI. Scans a document image and converts it to text using ML-based OCR (Wasm with LSTM).
  • School Science Project. Processes a DNA sequencing task using Regex and String manipulation.
  • Homework Spellcheck. Spellchecks a document using Typo.js and Web Workers.

The sub-scores for each of these tests will contribute to WebXPRT 5’s main overall score. (We’ll discuss scoring in future blogs.)

Experimental workloads

We’re currently planning to include an experimental workload section, something we’ve long discussed, in WebXPRT 5. Workloads in this section will use cutting-edge browser technologies that may not be compatible with the same broad range of platforms and devices as the technologies in WebXPRT 5’s core workloads. For that reason, we will not include the scores from the experimental section—in the Preview build and future releases—in WebXPRT 5’s main overall score.

In addition, WebXPRT 5’s experimental workloads will be completely optional.

Moving forward, WebXPRT’s experimental workload section will provide users with a straightforward way to learn how well certain browsers or systems handle new browser-based technologies (e.g., new web apps or AI capabilities). We’ll benefit from the ability to offer workloads for large-scale testing and user feedback before committing to including them as core WebXPRT workloads. Because future experimental workloads will run independently of the main test, we can add them without affecting the main WebXPRT score or requiring users to repeat testing to obtain comparable scores. We think it will be a win-win scenario in many respects.  

We’re still evaluating whether we can finish the first experimental workload in time to include it in the WebXPRT 5 Preview release, but we will definitely have at least the section and the framework for adding such a workload. When we are confident that an experimental workload is ready to go, we’ll share more information here in the blog and be all set up to incorporate it.

Timeline

If all goes well, we hope to publish the WebXPRT 5 Preview very soon, followed by a general release in early 2026. If that timeline changes significantly, we’ll provide an update here in the blog as soon as possible.

What about an “AI score”?

We’re still discussing the concept of a stand-alone WebXPRT 5 “AI score,” and we go back and forth on it. That score would combine WebXPRT’s AI-related subscores into a single score for use in AI capability comparisons. Because we’re just now beefing up WebXPRT’s AI capabilities, we’ve definitely decided not to include an AI score right now. We would love your feedback on the concept as we plan WebXPRT’s future. If that’s something that you would be interested in, please let us know!

If you have any questions about the WebXPRT 5 details we’ve shared above, please feel free to ask!

Justin

You asked, and we heard you: WebXPRT 5 is on the way!

We’re excited to announce that WebXPRT 5 is officially on the way! Since we launched WebXPRT 4 in February 2022, it’s proven to be an exceptionally successful and reliable go-to benchmark for OEM labs, the tech press, and individual users alike—to the tune of over 644,000 runs to date. In past blog posts, we’ve discussed new features and possible auxiliary workloads that we contemplated adding to WebXPRT 4. As we’ve considered user comments and suggestions, changes in web technology, and how we can best position WebXPRT as a relevant browser benchmark in the future, however, it became clear that it was time for an all-new WebXPRT.

Now that we’ve announced WebXPRT 5, the first question for many existing WebXPRT users may be, “When will WebXPRT 5 be available?” We’re not yet ready to share an anticipated WebXPRT 5 release date, but we can share that a lot of groundwork is already complete, and the remaining work is moving along rapidly. We’ll continue to issue updates here in the blog as we reach important milestones.

The second question for many existing WebXPRT users may be, “How will WebXPRT change?” We’re not yet ready to share extensive details about WebXPRT 5’s workloads—rest assured that we will as soon as we can firm up everything—but we can share a few key guidelines we tried to follow in our WebXPRT 5 design. Each of these points of emphasis is a result of feedback we’ve received from labs, as well as features that users have asked for.

  • Provide more AI-related workloads. In past blog posts, we’ve discussed the growing importance of local, browser-side AI. WebXPRT 4 already includes timed AI tasks in two of its workloads: the Organize Album using AI workload and the Encrypt Notes and OCR Scan workload. We’re working on ways to expand WebXPRT’s AI portfolio in the next version.
  • Add WebGPU workloads. As a web API, WebGPU enables web-based applications—such as image-based GenAI and inference workloads—to directly access the graphics rendering and computational capabilities of a system’s GPU. We hope to incorporate WebGPU measures in WebXPRT 5.
  • Improve WebXPRT’s utility as a tool for test labs, publications, and engineering analysis.
    • Update the workloads with longer operations. Many of WebXPRT’s existing workloads no longer challenge cutting-edge consumer hardware as much as many of us would like. Testing labs have asked for longer and more demanding workloads. We’re working on incorporating workloads that are accessible enough to be run by a broad range of devices yet challenging enough to allow performance differentiation among high-end systems.
    • Enable more precise performance measures. Labs and testers have also asked for more granular insight into the workloads to help with engineering-level performance analysis. Currently, some WebXPRT 4 workload scores include multiple timed tasks. If we separate those compound scores so that each workload reports results from only one timed task, users will be able to more precisely assess how well a device performs while handling specific operations. We’re looking into this approach.
  • Modernize the harness to make it more flexible and to speed future work. WebXPRT 4’s current harness works with server-side sessions on a LAMP (Linux, Apache, MySQL and PHP) stack. If we implement the harness via JavaScript on the client side, it will pave the way for faster development and testing cycles in the future.

We expect WebXPRT 5 to carry on the WebXPRT legacy of reliability and real-world relevance, while providing users with compelling new workloads and features. As has been our habit with new benchmark releases, however, we won’t force anyone to change versions anytime soon. Instead, we will continue to make WebXPRT 4 available for quite some time after WebXPRT 5 goes live.

If you have any questions or comments about WebXPRT, please let us know!

Justin

Recent XPRT mentions in articles, reviews, and more!

Here at the XPRTs, our primary goal is to provide free, easy-to-use benchmark tools that can help everyone—from OEM labs to tech press journalists to individual consumers—understand how well devices will perform while completing everyday computing tasks. We track progress toward that goal in several ways, but one of the most important is how much people use and discuss the XPRTs. When the name of one of our apps appears in an ad, article, or tech review, we call it a “mention.” Tracking mentions helps us gauge our reach.

We occasionally like to share a sample of recent XPRT mentions here in the blog. If you just started following the XPRTs, it may be surprising to see our program’s global reach. If you’re a longtime reader and you’re used to seeing WebXPRT or CrXPRT in major tech press articles, it may be surprising to learn more about overseas tech press publications or see how some government agencies use the XPRTs to make decisions. In any case, we hope you’ll enjoy exploring the links below!

Recent mentions include:

If you’d like to receive monthly updates on XPRT-related news and activity, we encourage you to sign up for the BenchmarkXPRT Development Community newsletter. It’s completely free, and all you need to do to join the newsletter mailing list is let us know! We won’t publish, share, or sell any of the contact information you provide, and we’ll only send you the monthly newsletter and occasional benchmark-related announcements, such as important news about patches or releases.

If you have any questions about the XPRTs, suggestions, or requests for future blog topics, please feel free to contact us.

Justin

Archiving AIXPRT and CloudXPRT

Some of our readers have been following the XPRTs since the early days, and they may remember using legacy versions of benchmarks such as HDXPRT 2014 or WebXPRT 2013. For many years, whenever we released a new version of a benchmark, we would maintain a link to the previous version on the benchmark’s main page. However, as interest in the older versions understandably waned and we stopped formally supporting them, many of those legacy XPRTs stopped working on the latest versions of the operating systems or browsers that we designed them to test. While we wanted to continue to provide a way for users to access those legacy XPRTs, we also wanted to avoid potential confusion for new users who might see links to old versions on our site. We decided that the best solution was to archive older tests in a separate section of the site—the XPRT archive.

Recently, as we discussed XPRT plans for 2025, it became clear that we needed to add AIXPRT and CloudXPRT to the archive. Both benchmarks represent landmark efforts toward our ongoing goal of providing cutting-edge performance assessment tools, but even though a few tech press publications and OEM labs experimented with them, neither benchmark gained enough widespread adoption to justify their continued support. As a result, we decided to focus our resources elsewhere and halt development on both benchmarks. Since then, ongoing updates to their respective software components and target platforms have rendered them largely unusable. By archiving both benchmarks, we hope to avoid any future confusion for visitors who may otherwise try to use them.

Over the coming weeks, we’ll be moving the AIXPRT and CloudXPRT installation packages to the XPRT archive page. We’re grateful to everyone who has used AIXPRT and CloudXPRT in the past, and we apologize for any inconvenience this change may cause.

If you have any questions or concerns about access to either of these benchmarks—or about anything else related to the XPRTs, please let us know

Justin

Check out the other XPRTs:

Forgot your password?