BenchmarkXPRT Blog banner

Tag Archives: benchmark

Looking forward to an important WebXPRT milestone

February 28, 2013 was a momentous day for the BenchmarkXPRT Development Community. On that day, we published a press release announcing the official launch of the first version of the WebXPRT benchmark, WebXPRT 2013. As difficult as it is for us to believe, the 10-year anniversary of the initial WebXPRT launch is in just a few short months!

We introduced WebXPRT as a truly unique browser performance benchmark in a field that was already crowded with a variety of measurement tools. Since those early days, the WebXPRT market presence has grown from a small foothold into a worldwide industry standard. Over the years, hundreds of tech press publications have used WebXPRT in thousands of articles and reviews, and the WebXPRT completed-runs counter rolled over the 1,000,000-run mark.

New web technologies are continually changing the way we use the web, and browser-performance benchmarks should evaluate how well new devices handle the web of today, not the web of several years ago. While some organizations have stopped development for other browser performance benchmarks, we’ve had the opportunity to continue updating and refining WebXPRT. We can look back at each of the four major iterations of the benchmark—WebXPRT 2013, WebXPRT 2015, WebXPRT 3, and WebXPRT 4—and see a consistent philosophy and shared technical lineage contributing to a product that has steadily improved.

As we get closer to the 10-year anniversary of WebXPRT next year, we’ll be sharing more insights about its reach and impact on the industry, discussing possible future plans for the benchmark, and announcing some fun anniversary-related opportunities for WebXPRT users. We think 2023 will be the best year yet for WebXPRT!

Justin

How to automate WebXPRT 4 testing

As the number of WebXPRT runs continues to grow, we realize many new WebXPRT users may be unfamiliar with all the features and capabilities of the benchmark. To help inform users about features that might facilitate their testing, we’ve decided to highlight a few WebXPRT features here in the blog. A few weeks ago, we discussed the multiple language options available in the WebXPRT 4 UI. This week, we look at WebXPRT 4 test automation.

WebXPRT 4 allows users to run scripts in an automated fashion. You can control the execution of WebXPRT 4 by appending parameters and values to the WebXPRT URL. Three parameters are available: testtype, tests, and result. Below, you’ll find a description of those parameters and instructions for utilizing automation.

Test type

The WebXPRT automation framework accounts for two test types: (1) the six core workloads and (2) any experimental workloads we might add in future builds. There are currently no experimental tests in WebXPRT 4, so always set the test type variable to 1.

  • Core tests: 1

Test scenario

This parameter lets you specify which tests to run by using the following codes:

  • Photo enhancement: 1
  • Organize album using AI: 2
  • Stock option pricing: 4
  • Encrypt notes and OCR scan using WASM: 8
  • Sales graphs: 16
  • Online homework: 32

To run a single individual test, use its code. To run multiple tests, use the sum of their codes. For example, to run Stocks (4) and Notes (8), use the sum of 12. To run all core tests, use 63, the sum of all the individual test codes (1 + 2 + 4 + 8 + 16 + 32 = 63).

Results format

This parameter lets you select the format of the results:

  • Display the result as an HTML table: 1
  • Display the result as XML: 2
  • Display the result as CSV: 3
  • Download the result as CSV: 4

To use the automation feature, start with the URL http://www.principledtechnologies.com/benchmarkxprt/webxprt/2021/wx4_build_3_7_3, append a question mark (?), and add the parameters and values separated by ampersands (&). For example, to run all the core tests and download the results, you would use the following URL: http://principledtechnologies.com/benchmarkxprt/webxprt/2021/wx4_build_3_7_3/auto.php?testtype=1&tests=63&result=4

We hope the WebXPRT automation features will make testing easier for you. If you have any questions about WebXPRT or the automation process, please feel free to ask!

Justin

The CloudXPRT v1.2 update package is now available!

We’re happy to announce that the CloudXPRT v1.2 update package is now available! The update prevents potential installation failures on Google Cloud Platform and Microsoft Azure, and ensures that the web microservices workload works on Ubuntu 22.04. The update uses updated software components such as Kubernetes v1.23.7, Kubespray v2.18.1, and Kubernetes Metrics Server v1, and incorporates some additional minor script changes.

The CloudXPRT v1.2 web microservices workload installation package is available at the CloudXPRT.com download page and the BenchmarkXPRT GitHub repository.

Before you get started with v1.2, please note the following updated system requirements:

  • Ubuntu 20.04.2 or 22.04 for on-premises testing
  • Ubuntu 18.04, 20.04.2, or 22.04 for CSP (AWS/Azure/GCP) testing

Because CloudXPRT is designed to run on high-end servers, physical nodes or VMs under test must meet the following minimum specifications:

  • 16 logical or virtual CPUs
  • 8 GB of RAM
  • 10 GB of available disk space (50 GB for the data analytics workload)

The update package includes only the updated v1.2 test harness and the updated web microservices workload. It does not include the data analytics workload. As we stated in the blog, now that we’ve published the web microservices package, we will assess the level of interest users express about a possible refresh of the v1.1 data analytics workload. For now, the v1.1 data analytics workload will continue to be available via CloudXPRT.com for some time to serve as a reference resource for users who have worked with the package in the past.

Please let us know if you have any questions about the CloudXPRT v1.2 test package. Happy testing!

Justin

We have a new XPRTs around the world infographic!

If you’ve followed the XPRT blog for a while, you know that we occasionally update the community on some of the reach metrics we track by publishing a new version of the “XPRTs around the world” infographic. The metrics we track include completed test runs, benchmark downloads, and mentions of the XPRTs in advertisements, articles, and tech reviews. Gathering this information gives us insight into how many people are using the XPRT tools, and updating the infographic helps readers and community members see the impact the XPRTs are having around the world.

This week, we published a new infographic, which includes the following highlights:

  • The XPRTs have been mentioned more than 19,500 times on over 4,000 unique sites.
  • Those mentions include more than 12,300 articles and reviews.
  • Those mentions originated in over 924 cities located in 81 countries on six continents. New cities of note include Dhaka, Bangladesh; Zagreb, Croatia; Hamilton, New Zealand; and Medina, Saudi Arabia.

In addition to the reach metrics we mention above, the XPRTs have now delivered more than 1,330,000 real-world results! We’re grateful for everyone who’s helped us get this far. Your participation is vital to our achieving our goal: to provide benchmark tools that are reliable, relevant, and easy to use.

Justin

An update on Chrome OS XPRT benchmark development

In July, we discussed the Chrome OS team’s decision to end support for Chrome apps, and how that will prevent us from publishing any future fixes or updates for CrXPRT 2. We also announced our goal of beginning development of an all-new Chrome OS XPRT benchmark by the end of this year. While we are actively discussing this benchmark and researching workload technologies and scenarios, we don’t foresee releasing a preview build this year.

The good news is that, in spite of a lack of formal support from the Chrome OS team, the CrXPRT 2 performance and battery life tests currently run without any known issues. We continue to monitor the status of CrXPRT and will inform our blog readers of any significant changes.

If you have any questions about CrXPRT, or ideas about the types of features or workloads you’d like to see in a new Chrome OS benchmark, please let us know!

Justin

The Exploring WebXPRT 4 white paper is now available

This week, we published the Exploring WebXPRT 4 white paper. It describes the design and structure of WebXPRT 4, including detailed information about the benchmark’s harness, HTML5 and WebAssembly (WASM) capability checks, and changes we’ve made to the structure of the performance test workloads. We explain the benchmark’s scoring methodology, how to automate tests, and how to submit results for publication. The white paper also includes information about the third-party functions and libraries that WebXPRT 4 uses during the HTML5 and WASM capability checks and performance workloads.

The Exploring WebXPRT 4 white paper promotes the high level of transparency and disclosure that is a core value of the BenchmarkXPRT Development Community. We’ve always believed that transparency builds trust, and trust is essential for a healthy benchmarking community. That’s why we involve community members in the benchmark development process and disclose how we build our benchmarks and how they work.

You can find the paper on WebXPRT.com and our XPRT white papers page. If you have any questions about WebXPRT 4, please let us know, and be sure to check out our other XPRT white papers.

Justin

Check out the other XPRTs:

Forgot your password?