BenchmarkXPRT Blog banner

Tag Archives: cross-platform

We want your thoughts about experimental WebXPRT 4 workloads

Justin Greene
By Justin Greene
on December 8, 2022

Two weeks ago, we discussed how users can automate WebXPRT 4 testing by appending several parameters and values to the benchmark’s URL. One of these lets you enable any available experimental workloads during the test run. While we don’t currently offer any experimental workloads for WebXPRT 4, we are seeking suggestions for possible future workload scenarios, or specific web technologies that you’d like to be able to test with an experimental workload.

The main purpose of optional, experimental workloads would be to test cutting-edge browser technologies or new use cases, even if the experimental workload doesn’t work on all browsers or devices. The individual scores for the experimental workloads would stand alone, and would not factor in the WebXPRT 4 overall score. WebXPRT 4 testers would be able to run the experimental workloads one of two ways: by adjusting a value in the WebXPRT 4 automation scripts, as mentioned above, or by manually selecting them on the benchmark’s home screen.

Testers would benefit from experimental workloads by learning how well certain browsers or systems handle new tasks (e.g., new web apps or AI capabilities). We would benefit from fielding workloads for large-scale testing and user feedback before we commit to including them as core WebXPRT workloads.

Do you have any general thoughts about experimental workloads for browser performance testing, or any specific workloads that you’d like us to consider? Please let us know.

Justin

Posted in AI, Automation, benchmark, Benchmark metrics, Benchmarking, BenchmarkXPRT, Browser-based benchmarks, Collaborative benchmark development, Cross-platform benchmarks, Future of performance evaluation, Performance benchmarking, WebXPRT, WebXPRT 4, What makes a good benchmark? | Also tagged , , , , , , ,

Looking forward to an important WebXPRT milestone

Justin Greene
By Justin Greene
on December 1, 2022

February 28, 2013 was a momentous day for the BenchmarkXPRT Development Community. On that day, we published a press release announcing the official launch of the first version of the WebXPRT benchmark, WebXPRT 2013. As difficult as it is for us to believe, the 10-year anniversary of the initial WebXPRT launch is in just a few short months!

We introduced WebXPRT as a truly unique browser performance benchmark in a field that was already crowded with a variety of measurement tools. Since those early days, the WebXPRT market presence has grown from a small foothold into a worldwide industry standard. Over the years, hundreds of tech press publications have used WebXPRT in thousands of articles and reviews, and the WebXPRT completed-runs counter rolled over the 1,000,000-run mark.

New web technologies are continually changing the way we use the web, and browser-performance benchmarks should evaluate how well new devices handle the web of today, not the web of several years ago. While some organizations have stopped development for other browser performance benchmarks, we’ve had the opportunity to continue updating and refining WebXPRT. We can look back at each of the four major iterations of the benchmark—WebXPRT 2013, WebXPRT 2015, WebXPRT 3, and WebXPRT 4—and see a consistent philosophy and shared technical lineage contributing to a product that has steadily improved.

As we get closer to the 10-year anniversary of WebXPRT next year, we’ll be sharing more insights about its reach and impact on the industry, discussing possible future plans for the benchmark, and announcing some fun anniversary-related opportunities for WebXPRT users. We think 2023 will be the best year yet for WebXPRT!

Justin

Posted in benchmark, Benchmark metrics, Benchmarking, BenchmarkXPRT, BenchmarkXPRT development community, Browser-based benchmarks, Collaborative benchmark development, Cross-platform benchmarks, Future of performance evaluation, History of benchmarking, Performance benchmarking, Web-based testing, WebXPRT, WebXPRT 2013, WebXPRT 2015, WebXPRT 3, WebXPRT 4 | Also tagged , , , , , , ,

How to automate WebXPRT 4 testing

Justin Greene
By Justin Greene
on November 23, 2022

As the number of WebXPRT runs continues to grow, we realize many new WebXPRT users may be unfamiliar with all the features and capabilities of the benchmark. To help inform users about features that might facilitate their testing, we’ve decided to highlight a few WebXPRT features here in the blog. A few weeks ago, we discussed the multiple language options available in the WebXPRT 4 UI. This week, we look at WebXPRT 4 test automation.

WebXPRT 4 allows users to run scripts in an automated fashion. You can control the execution of WebXPRT 4 by appending parameters and values to the WebXPRT URL. Three parameters are available: testtype, tests, and result. Below, you’ll find a description of those parameters and instructions for utilizing automation.

Test type

The WebXPRT automation framework accounts for two test types: (1) the six core workloads and (2) any experimental workloads we might add in future builds. There are currently no experimental tests in WebXPRT 4, so always set the test type variable to 1.

  • Core tests: 1

Test scenario

This parameter lets you specify which tests to run by using the following codes:

  • Photo enhancement: 1
  • Organize album using AI: 2
  • Stock option pricing: 4
  • Encrypt notes and OCR scan using WASM: 8
  • Sales graphs: 16
  • Online homework: 32

To run a single individual test, use its code. To run multiple tests, use the sum of their codes. For example, to run Stocks (4) and Notes (8), use the sum of 12. To run all core tests, use 63, the sum of all the individual test codes (1 + 2 + 4 + 8 + 16 + 32 = 63).

Results format

This parameter lets you select the format of the results:

  • Display the result as an HTML table: 1
  • Display the result as XML: 2
  • Display the result as CSV: 3
  • Download the result as CSV: 4

To use the automation feature, start with the URL http://www.principledtechnologies.com/benchmarkxprt/webxprt/2021/wx4_build_3_7_3, append a question mark (?), and add the parameters and values separated by ampersands (&). For example, to run all the core tests and download the results, you would use the following URL: http://principledtechnologies.com/benchmarkxprt/webxprt/2021/wx4_build_3_7_3/auto.php?testtype=1&tests=63&result=4

We hope the WebXPRT automation features will make testing easier for you. If you have any questions about WebXPRT or the automation process, please feel free to ask!

Justin

Posted in Automation, benchmark, Benchmark metrics, Benchmarking, BenchmarkXPRT, Browser-based benchmarks, Cross-platform benchmarks, Performance benchmarking, results, Web-based testing, WebAssembly, WebXPRT, WebXPRT 4 | Also tagged , , , , ,

WebXPRT passes the million-run milestone!

Justin Greene
By Justin Greene
on July 21, 2022

We’re excited to see that users have successfully completed over 1,000,000 WebXPRT runs! If you’ve run WebXPRT in any of the 924 cities and 81 countries from which we’ve received complete test data—including newcomers Bahrain, Bangladesh, Mauritius, The Philippines, and South Korea —we’re grateful for your help. We could not have reached this milestone without you!

As the chart below illustrates, WebXPRT use has grown steadily since the debut of WebXPRT 2013. On average, we now record more WebXPRT runs in one month than we recorded in the entirety of our first year. With over 104,000 runs so far in 2022, that growth is continuing.

For us, this moment represents more than a numerical milestone. Developing and maintaining a benchmark is never easy, and a cross-platform benchmark that will run on a wide variety of devices poses an additional set of challenges. For such a benchmark to succeed, developers need not only technical competency, but the trust and support of the benchmarking community. WebXPRT is now in its ninth year, and its consistent year-over-year growth tells us that the benchmark continues to hold value for manufacturers, OEM labs, the tech press, and end users like you. We see it as a sign of trust that folks repeatedly return to the benchmark for reliable performance metrics. We’re grateful for that trust, and for everyone that’s contributed to the WebXPRT development process throughout the years.

We’ll have more to share related to this exciting milestone in the weeks to come, so stay tuned to the blog. If you have any questions or comments about WebXPRT, we’d love to hear from you!

Justin

Posted in benchmark, Benchmark metrics, Benchmarking, BenchmarkXPRT, Browser-based benchmarks, Performance benchmarking, Web-based testing, WebXPRT, WebXPRT 2013, WebXPRT 2015, WebXPRT 3, WebXPRT 4 | Also tagged , , , , , , ,

CrXPRT 2 on FydeOS!

Justin Greene
By Justin Greene
on April 28, 2022

Recently, a tester contacted us with details from a CrXPRT 2 performance test run that they’d successfully completed on… an Apple MacBook Pro! Because CrXPRT 2 is a Chrome Web App that we designed for Chrome OS, it was quite a surprise to hear that it is now possible to run CrXPRT 2 on non-Chrome OS platforms by using FydeOS.

FydeOS is an operating system based on a fork of the Chromium OS project. Developers originally intended FydeOS to be a Google-independent, Chrome-like alternative for the Chinese educational market, but FydeOS is now available to the English-speaking consumer and enterprise markets as well. FydeOS users can run a Chrome-like OS on something other than a Chromebook or a Chromebox, such as a PC, Mac, virtual machine, or even a Raspberry Pi device. Additionally, FydeOS supports Android, Chrome OS, and Linux apps, and users can run those apps at the same time on the same screen.

We have not yet conducted any testing with FydeOS in our lab, but we wanted to pass along this information to any readers who may be interested. If the OS operates as described, it may provide a way for us to experiment with using CrXPRT 2 in some interesting cross-platform tests.

Justin

Posted in Android, Apple, benchmark, Benchmarking, Browser-based benchmarks, Chrome OS, Chromebooks, Cross-platform benchmarks, CrXPRT, FydeOS, Google, Google Chrome, Linux, Performance benchmarking, Raspberry Pi | Also tagged , , , , , , , ,

WebXPRT 4 is almost here!

Justin Greene
By Justin Greene
on February 24, 2022

We’re putting the last pieces in place for the WebXPRT 4 GA, and expect to take the final build live by this time next week! When we released the WebXPRT 4 Preview and encouraged testers to submit and publish results, we said we’d try to limit any changes to things that would not affect test scores. We’re happy to report that we’ve achieved that goal, and Preview testing results are comparable with GA build results.

If you missed the blog post about the differences between WebXPRT 3 and WebXPRT 4, we encourage you to check it out. Everything we mentioned about the general and workload-specific changes in the Preview build holds true for the upcoming GA.

Keep an eye on the blog and WebXPRT.com for more information in the coming week. We look forward to seeing your test results!

Justin

Posted in benchmark, BenchmarkXPRT, Browser-based benchmarks, Cross-platform benchmarks, Performance benchmarking, results, WebXPRT, WebXPRT 3, webxprt 4 | Also tagged , , , ,

Check out the other XPRTs:

Forgot your password?
© Principled Technologies, Inc. 2024   |   Privacy policy

Cookie Notice: Our website uses cookies to deliver a smooth experience by storing logins and saving user information. By continuing to use our site, you agree with our usage of cookies as our privacy policy outlines.

x