BenchmarkXPRT Blog banner

Tag Archives: benchmark

How to automate your WebXPRT 4 testing

Justin Greene
By Justin Greene
on August 22, 2024

We’re excited about the ongoing upward trend in the number of completed WebXPRT 4 runs that we’re seeing each month. OEM and tech press labs are responsible for a significant amount of that growth, and many of them use WebXPRT’s automation features to complete large blocks of hands-off testing at one time. We realize that many new WebXPRT users may be unfamiliar with the benchmark’s automation tools, so we decided to provide a quick guide to WebXPRT automation in today’s blog. Whether you’re testing one or 1,000 devices, the instructions below can help save you some time.

WebXPRT 4 allows users to run scripts in an automated fashion and control test execution by appending parameters and values to the WebXPRT URL. Three parameters are available:

  • test type
  • test scenarios
  • results

Below, you’ll find a description of those parameters and instructions for how you can use them to automate your test runs.

Test type

The WebXPRT automation framework accounts for two test types: (1) the six core workloads, and (2) any experimental workloads we might add in future builds. There are currently no experimental tests in WebXPRT 4, so always set the test type variable to 1.

  • Core tests: 1

Test scenario

The test scenario parameter lets you specify which subtest workloads to run by using the following codes:

  • Photo enhancement: 1
  • Organize album using AI: 2
  • Stock option pricing: 4
  • Encrypt notes and OCR scan using WASM: 8
  • Sales graphs: 16
  • Online homework: 32

To run a single subtest workload, use its code. To run multiple workloads, use the sum of their codes. For example, to run Stock options pricing (4) and Encrypt notes and OCR scan (8), use the sum of 12. To run all core tests, use 63, the sum of all the individual test codes (1 + 2 + 4 + 8 + 16 + 32 = 63).

Results format

The results format parameter lets you select the format of the results:

  • Display the result as an HTML table: 1
  • Display the result as XML: 2
  • Display the result as CSV: 3
  • Download the result as CSV: 4

To use the automation feature, start with the URL https://www.principledtechnologies.com/benchmarkxprt/webxprt/2021/wx4_build_3_7_3, append a question mark (?), and add the parameters and values separated by ampersands (&). For example, to run all the core tests and download the results, you would use the following URL: https://principledtechnologies.com/benchmarkxprt/webxprt/2021/wx4_build_3_7_3/auto.php?testtype=1&tests=63&result=4

We hope WebXPRT 4’s automation features will make testing easier for you. If you have any questions about WebXPRT or the automation process, please feel free to ask!

Justin

Posted in Automation, benchmark, Benchmark metrics, Benchmarking, browser performance, Browser-based benchmarks, Performance benchmarking, results, tech press, WebXPRT, WebXPRT 4 | Also tagged , , , , , , , , ,

Shopping for back-to-school tech? The XPRTs can help!

Justin Greene
By Justin Greene
on August 8, 2024

For many students, the first day of school is just around the corner, and it’s now time to shop for new tech devices that can help set them up for success in the coming year. The tech marketplace can be confusing, however, with so many brands, options, and competing claims to sort through.

Fortunately, the XPRTs are here to help!

Whether you’re shopping for a new phone, tablet, Chromebook, laptop, or desktop, the XPRTs can provide industry-trusted performance scores that can give you confidence that you’re making a smart purchasing decision.

The WebXPRT 4 results viewer is a good place to start looking for device scores. The viewer displays WebXPRT 4 scores from over 700 devices—including many of the latest releases—and we’re adding new scores all the time. To learn more about the viewer’s capabilities and how you can use it to compare devices, check out this blog post.

Another resource we offer is the XPRT results browser. The browser is the most efficient way to access the XPRT results database, which currently holds more than 3,700 test results from over 150 sources, including major tech review publications around the world, manufacturers, and independent testers. It offers a wealth of current and historical performance data across all the XPRT benchmarks and hundreds of devices. You can read more about how to use the results browser here.

Also, if you’re considering a popular device, there’s a good chance that a recent tech review includes an XPRT score for that device. There are two quick ways to find these reviews: You can either (1) search for “XPRT” on your preferred tech review site or (2) use a search engine and input the device name and XPRT name, such as “Dell XPS” and “WebXPRT.”

Here are a few recent tech reviews that use one of the XPRTs to evaluate a popular device:

Lastly, here at Principled Technologies, we frequently publish reports that evaluate the performance of hot new consumer devices, and many of those reports include WebXPRT scores. For example, check out our extensive testing of HP ZBook G10 mobile workstations or our detailed comparison of Lenovo ThinkPad, ThinkBook, and ThinkCentre devices to their Apple Mac counterparts.

The XPRTs can help anyone stuck in the back-to-school shopping blues make better-informed and more confident tech purchases. As this new school year begins, we hope you’ll find the data you need on our site or in an XPRT-related tech review. If you have any questions about the XPRTs, XPRT scores, or the results database please feel free to ask!

Justin

Posted in Acer, Apple, ASUS, benchmark, Benchmark metrics, Benchmarking, BenchmarkXPRT, browser performance, Browser-based benchmarks, Chromebooks, Cross-platform benchmarks, CrXPRT, Dell, Education, Laptops, Lenovo, Macbook, PCs, Performance benchmarking, results, tech press, WebXPRT, WebXPRT 4 | Also tagged , , , , , , , , , , , , , , , ,

Up next for WebXPRT 4: A new AI-focused workload!

Justin Greene
By Justin Greene
on May 30, 2024

We’re always thinking about ways to improve WebXPRT. In the past, we’ve discussed the potential benefits of auxiliary workloads and the role that such workloads might play in future WebXPRT updates and versions. Today, we’re very excited to announce that we’ve decided to move forward with the development of a new WebXPRT 4 workload focused on browser-side AI technology!

WebXPRT 4 already includes timed AI tasks in two of its workloads: the Organize Album using AI workload and the Encrypt Notes and OCR Scan workload. These two workloads reflect the types of light browser-side inference tasks that have been available for a while now, but most heavy-duty inference on the web has historically happened in on-prem servers or in the cloud. Now, localized AI technology is growing by leaps and bounds, and the integration of new AI capabilities with browser-based tasks is on the threshold of advancing rapidly.

Because of this growth, we believe now is the time to start work on giving WebXPRT 4 the ability to evaluate new browser-based AI capabilities—capabilities that are likely to become a part of everyday life in the next few years. We haven’t yet decided on a test scenario or software stack for the new workload, but we’ll be working to refine our plan in the coming months. There seems to be some initial promise in emerging frameworks such as ONNX Runtime Web, which allows users to run and deploy web-based machine learning models by using JavaScript APIs and libraries. In addition, new Web APIs like WebGPU (currently supported in Edge, Chrome, and tech preview in Safari) and WebNN (in development) may soon help facilitate new browser-side AI workloads.

We know that many longtime WebXPRT 4 users will have questions about how this new workload may affect their tests. We want to assure you that the workload will be an optional bonus workload and will not run by default during normal WebXPRT 4 tests. As you consider possibilities for the new workload, here are a few points to keep in mind:

  • The workload will be optional for users to run.
  • It will not affect the main WebXPRT 4 subtest or overall scores in any way.
  • It will run separately from the main test and will produce its own score(s).
  • Current and future WebXPRT 4 results will still be comparable to one another, so users who’ve already built a database of WebXPRT 4 scores will not have to retest their devices.
  • Because many of the available frameworks don’t currently run on all browsers, the workload may not run on every platform.

As we research available technologies and explore our options, we would love to hear from you. If you have ideas for an AI workload scenario that you think would be useful or thoughts on how we should implement it, please let us know! We’re excited about adding new technologies and new value to WebXPRT 4, and we look forward to sharing more information here in the blog as we make progress.

Justin

Posted in AI, benchmark, BenchmarkXPRT, BenchmarkXPRT development community, browser performance, Browser-based benchmarks, Chrome, Collaborative benchmark development, Future of performance evaluation, JavaScript, Microsoft Edge, on-device AI, ONNX Runtime Web, Performance benchmarking, Safari, WebGPU, WebNN, WebXPRT, WebXPRT 4 | Also tagged , , , , , , , , , ,

The WebXPRT 4 results viewer: A powerful tool for browsing hundreds of test results

Justin Greene
By Justin Greene
on May 16, 2024

In our recent blog post about the XPRT results database, we promised to discuss the WebXPRT 4 results viewer in more detail. We developed the results viewer to serve as a feature-rich interactive tool that visitors to WebXPRT.com can use to browse the test results that we’ve published on our site, dig into the details of each result, and compare scores from multiple devices. The viewer currently has almost 700 test results, and we add new PT-curated entries each week.

Figure 1 shows the tool’s default display. Each vertical bar in the graph represents the overall score of a single test result, with bars arranged left-to-right, from lowest to highest. To view a single result in detail, hover over a bar to highlight it, and a small popup window will display the basic details of the result. You can then click to select the highlighted bar. The bar will turn dark blue, and the dark blue banner at the bottom of the viewer will display additional details about that result.

Figure 1: The WebXPRT 4 results viewer tool’s default display

In the example in Figure 1, the banner shows the overall score (237), the score’s percentile rank (66th) among the scores in the current display, the name of the test device, and basic hardware configuration information. If the source of the result is PT, you can click the Run info button in the bottom right-hand corner of the display to see the run’s individual workload scores. If the source is an external publisher, users can click the Source link to navigate to the original site.

The viewer includes a drop-down menu that lets users quickly filter results by major device type categories, plus a tab with additional filtering options, such as browser type, processor vendor, and result source. Figure 2 shows the viewer after I used the device type drop-down filter to select only laptops.

Figure 2: Screenshot from the WebXPRT 4 results viewer showing results filtered by the device type drop-down menu.

Figure 3 shows the viewer as I use the filter tab to explore additional filter options, such as processor vendor.

Figure 3: Screenshot from the WebXPRT 4 results viewer showing the filter options available with the filter tab.

The viewer will also let you pin multiple specific runs, which is helpful for making side-by-side comparisons. Figure 4 shows the viewer after I pinned four runs and viewed them on the Pinned runs screen.

Figure 4: Screenshot from the WebXPRT 4 results viewer showing four pinned runs on the Pinned runs screen.

Figure 5 shows the viewer after I clicked the Compare runs button. The overall and individual workload scores of the pinned runs appear in a table.

Figure 5: Screenshot from the WebXPRT 4 results viewer showing four pinned runs on the Compare runs screen.

We hope that you’ll enjoy using the results viewer to browse our WebXPRT 4 results database and that it will become one of your go-to resources for device comparison data.  

Are there additional features you’d like to see in the viewer, or other ways we can improve it? Please let us know, and send us your latest test results!

Justin

Posted in benchmark, Benchmark metrics, Benchmarking, BenchmarkXPRT, browser performance, Browser-based benchmarks, Cross-platform benchmarks, Dell, HP, Intel, Performance benchmarking, results, results submission, WebXPRT, WebXPRT 4 | Also tagged , , , , , , ,

Want to know how your device performs? Explore the XPRT results database

Justin Greene
By Justin Greene
on May 2, 2024

If you only recently started using the XPRT benchmarks, you may not know about one of the free resources we offer—the XPRT results database. Our results database currently holds more than 3,650 test results from over 150 sources, including global tech press outlets, OEM labs, and independent testers. It serves as a treasure trove of current and historical performance data across all the XPRT benchmarks and hundreds of devices. You can use these results and the results of the same XPRTs on your device to get a sense of how well your device performs.

We update the results database several times a week, adding selected results from our own internal lab testing, reliable media sources, and end-of-test user submissions. (After you run one of the XPRTs, you can choose to submit the results, but don’t worry—this is opt-in. Your results do not automatically appear in the database.) Before adding a result, we also look at any available system information and evaluate whether the score makes sense and is consistent with general expectations.

There are three primary ways that you can explore the XPRT results database.

The first is by visiting the main BenchmarkXPRT results browser, which displays results entries for all of the XPRT benchmarks in chronological order (see the screenshot below). You can filter the results by selecting a benchmark from the drop-down menu. You can also type values, such as a vendor name (e.g., Dell) or the name of a tech publication (e.g., PCWorld) into the free-form filter field. For results we’ve produced in our lab, clicking “PT” in the Source column takes you to a page with additional configuration information for the test system. For sources outside our lab, clicking the source name takes you to the original article or review that contains the result.

The second way to access our published results is by visiting the results page for an individual XPRT benchmark. Start by going to the page of the benchmark that interests you (e.g., CrXPRT.com) , and looking for the blue View Results button. Clicking the button takes you to a page that displays results for only that benchmark. You can use the free-form filter on the page to filter those results, and you can use the Benchmarks drop-down menu to jump to the other individual XPRT results pages.

The third way to view our results database is with the WebXPRT 4 results viewer. The viewer provides an information-packed, interactive tool with which you can explore data from the curated set of WebXPRT 4 results we’ve published on our site. We’ll discuss the features of the WebXPRT 4 results viewer in more detail in a future post.

You can use any of these approaches to compare the results of an XPRT on your device with our many published results. We hope you’ll take some time to explore the information in our results database and that it proves to be helpful to you. If you have ideas for new features or suggestions for improvement, we’d love to hear from you!

Justin

Posted in benchmark, Benchmark metrics, Benchmarking, BenchmarkXPRT, Browser-based benchmarks, CrXPRT, History of benchmarking, Performance benchmarking, results, results submission, WebXPRT, WebXPRT 4 | Also tagged , , , , , , , ,

Another milestone for WebXPRT!

Justin Greene
By Justin Greene
on April 18, 2024

Back in November, we discussed some of the trends we were seeing in the total number of completed and reported WebXPRT runs each month. The monthly run totals were increasing at a rate we hadn’t seen before. We’re happy to report that the upward trend has continued and even accelerated through the first quarter of this year! So far in 2024, we’ve averaged 43,744 WebXPRT runs per month, and our run total for the month of March alone (48,791) was more than twice the average monthly run total for 2023 (24,280).

The rapid increase in WebXPRT testing has helped us reach the milestone of 1.5 million runs much sooner than we anticipated. As the chart below shows, it took about six years for WebXPRT to log the first half-million runs and nine years to pass the million-run milestone. It’s only taken about one-and-a-half years to add another half-million.

This milestone means more to us than just reaching some large number. For a benchmark to be successful, it should ideally have widespread confidence and support from the benchmarking community, including manufacturers, OEM labs, the tech press, and other end users. When the number of yearly WebXPRT runs consistently increases, it’s a sign to us that the benchmark is serving as a valuable and trusted performance evaluation tool for more people around the world.

As always, we’re grateful for everyone who has helped us reach this milestone. If you have any questions or comments about using WebXPRT to test your gear, please let us know! And, if you have suggestions for how we can improve the benchmark, please share them. We want to keep making it better and better for you!

Justin

Posted in benchmark, Benchmarking, BenchmarkXPRT, browser performance, Browser-based benchmarks, Cross-platform benchmarks, Performance benchmarking, WebXPRT, WebXPRT 4 | Also tagged , , , , , ,

Check out the other XPRTs:

Forgot your password?
© Principled Technologies, Inc. 2024   |   Privacy policy

Cookie Notice: Our website uses cookies to deliver a smooth experience by storing logins and saving user information. By continuing to use our site, you agree with our usage of cookies as our privacy policy outlines.

x