BenchmarkXPRT Blog banner

Tag Archives: benchmark

Recent XPRT mentions in the tech press

Justin Greene
By Justin Greene
on November 30, 2023

Each month, we send out a BenchmarkXPRT Development Community newsletter that contains the latest updates from the XPRT world and provides a summary of the previous month’s XPRT-related activity, including mentions of the XPRTs in the tech press. More people read the weekly XPRT blog than receive the monthly newsletter, so we realized that some blog readers may be unaware of the wide variety of tech outlets that regularly use or mention the XPRTs.

For today’s blog, we want to give readers a sampling of the XPRT press mentions we see on a weekly basis. Recent mentions include:

If you don’t currently receive the monthly BenchmarkXPRT newsletter, but would like to join the mailing list, please let us know! There is no cost to join, and we will not publish or sell any of the contact information you provide. We will send only the monthly newsletter and occasional benchmark-related announcements, such as patch notifications or news of upcoming benchmark releases.

Justin

Posted in Acer, Apple, ASUS, benchmark, BenchmarkXPRT, BenchmarkXPRT development community, CrXPRT, results, Samsung, WebXPRT | Also tagged , , , , , , , , , , , , , , ,

Passing two important WebXPRT milestones

Justin Greene
By Justin Greene
on November 15, 2023

Over the past few months, we’ve been excited to see a substantial increase in the total number of completed WebXPRT runs. To put the increase in perspective, we had more total WebXPRT runs last month alone (40,453) than we had in the first two years WebXPRT was available (36,674)! This boost has helped us to reach two important milestones as we close in on the end of 2023.

The first milestone is that the number of WebXPRT 4 runs per month now exceeds the number of WebXPRT 3 runs per month. When we release a new version of an XPRT benchmark, it can take a while for users to transition from using the older version. For OEM labs and tech journalists, adding a new benchmark to their testing suite often involves a significant investment in back testing and gathering enough test data for meaningful comparisons. When the older version of the benchmark has been very successful, adoption of the new version can take longer. WebXPRT 3 has been remarkably popular around the world, so we’re excited to see WebXPRT 4 gain traction and take the lead even as the total number of WebXPRT runs increases each month. The chart below shows the number of WebXPRT runs per month for each version of WebXPRT over the past ten years. WebXPRT 4 usage first surpassed WebXPRT 3 in August of this year, and after looking at data for the last three months, we think its lead is here to stay.

The second important milestone is the cumulative number of WebXPRT runs, which recently passed 1.25 million, as the chart below shows. For us, this moment represents more than a numerical milestone. For a benchmark to succeed, developers need the trust and support of the benchmarking community. WebXPRT’s consistent year-over-year growth tells us that the benchmark continues to hold value for manufacturers, OEM labs, the tech press, and end users. We see it as a sign of trust that folks repeatedly return to the benchmark for reliable performance metrics. We’re grateful for that trust, and for everyone that has contributed to the WebXPRT development process over the years.

We look forward to seeing how far WebXPRT’s reach can extend in 2024! If you have any questions or comments about using WebXPRT, let us know!

Justin

Posted in benchmark, Benchmark metrics, Benchmarking, BenchmarkXPRT, Browser-based benchmarks, Cross-platform benchmarks, History of benchmarking, Performance benchmarking, results, WebXPRT, WebXPRT 3, WebXPRT 4 | Also tagged , , , , , ,

A note about CrXPRT 2

Justin Greene
By Justin Greene
on August 31, 2023

Recent visitors to CrXPRT.com may have seen a notice that encourages visitors to use WebXPRT 4 instead of CrXPRT 2 for performance testing on high-end Chromebooks. The notice reads as follows:

NOTE: Chromebook technology has progressed rapidly since we released CrXPRT 2, and we’ve received reports that some CrXPRT 2 workloads may not stress top-bin Chromebook processors enough to give the necessary accuracy for users to compare their performance. So, for the latest test to compare the performance of high-end Chromebooks, we recommend using WebXPRT 4.

We made this recommendation because of the evident limitations of the CrXPRT 2 performance workloads when testing newer high-end hardware. CrXPRT 2 itself is not that old (2020), but when we created the CrXPRT 2 performance workloads, we started with a core framework of CrXPRT 2015 performance workloads. In a similar way, we built the CrXPRT 2015 workloads on a foundation of WebXPRT 2015 workloads. At the time, the harness and workload structures we used to ensure WebXPRT 2015’s cross-browser capabilities provided an excellent foundation that we could adapt for our new ChromeOS benchmark. Consequently, CrXPRT 2 is a close developmental descendant of WebXPRT 2015. Some of the legacy WebXPRT 2015/CrXPRT 2 workloads do not stress current high-end processors—a limitation that prevents effective performance testing differentiation—nor do they engage the latest web technologies.

In the past, the Chromebook market skewed heavily toward low-cost devices with down-bin, inexpensive processors, making this limitation less of an issue. Now, however, more Chromebooks offer top-bin processors on par with traditional laptops and workstations. Because of the limitations of the CrXPRT 2 workloads, we now recommend WebXPRT 4 for both cross-browser and ChromeOS performance testing on the latest high-end Chromebooks. WebXPRT 4 includes updated test content, newer JavaScript tools and libraries, modern WebAssembly workloads, and additional Web Workers tasks that cover a wide range of performance requirements.

While CrXPRT 2 continues to function as a capable performance and battery life comparison test for many ChromeOS devices, WebXPRT 4 is a more appropriate tool to use with new high-end devices. If you haven’t yet used WebXPRT 4 for Chromebook comparison testing, we encourage you to give it a try!

If you have any questions or concerns about CrXPRT 2 or WebXPRT 4, please don’t hesitate to ask!

Justin

Posted in battery life, benchmark, Benchmark metrics, Benchmarking, browser performance, Chrome OS, Chromebooks, CrXPRT, Google Chrome, History of benchmarking, HTML5, JavaScript, Laptops, Performance benchmarking, WebAssembly, WebXPRT, WebXPRT 2015, WebXPRT 4 | Also tagged , , , , , , , , ,

A bit of house cleaning at BenchmarkXPRT.com

Justin Greene
By Justin Greene
on August 17, 2023

When we’ve released a new version of an XPRT benchmark app, it’s been our practice for many years to maintain a link to the previous version on the benchmark’s main page. For example, visitors can start on the WebXPRT 4 homepage at WebXPRT.com and follow links to access WebXPRT 3, WebXPRT 2015, and WebXPRT 2013. Historically, we’ve maintained these links because labs and tech reviewers usually take a while to introduce a new benchmark to their testing suite. Continued access to the older benchmarks also allows users to quickly compare new devices to old devices without retesting everything.

That being said, several of the XPRT pages currently contain links to benchmarks that we no longer actively support. Some of those benchmarks still function correctly, and testers occasionally use them, but a few no longer work on the latest versions of the operating systems or browsers that we designed them to test. While we want to continue to provide a way for longtime XPRT users to access legacy XPRTs,  we also want to avoid potential confusion for new users. We believe our best way forward is to archive older tests in a separate part of the site.

In the coming weeks, we’ll be moving several legacy XPRT benchmarks to an archive section of the site. Once the new section is ready, we’ll link to it from the Extras drop-down menu at the top of BenchmarkXPRT.com. The benchmarks will still be available in the archive, but we will not actively support them or directly link to them from the homepages of active XPRTs.

During this process, we’ll move the following benchmarks to the archive section:

  • WebXPRT 2015 and 2013
  • CrXPRT 2015
  • HDXPRT 2014
  • TouchXPRT 2014
  • MobileXPRT 2015 and 2013

If you have any questions or concerns about the archive process or access to legacy XPRTs, please let us know! 

Justin

Posted in benchmark, BenchmarkXPRT, CrXPRT, HDXPRT, HDXPRT 2014, History of benchmarking, MobileXPRT, MobileXPRT 2015, TouchXPRT, TouchXPRT 2014, WebXPRT, WebXPRT 2013, WebXPRT 2015, WebXPRT 4 | Also tagged , , , , , , , , , , ,

The XPRTs are a great back-to-school shopping resource

Justin Greene
By Justin Greene
on August 10, 2023

Students of all ages will be starting a new school year over the next few weeks, and many learners will be shopping for tech devices that can help them excel in their studies. The tech marketplace can be confusing, and competing claims can be hard to navigate. The XPRTs are here to help! Whether you’re shopping for a new laptop, desktop, Chromebook, tablet, or phone, the XPRTs can provide reliable, industry-trusted performance scores that can cut through all the noise.

A good place to start looking for scores is the WebXPRT 4 results viewer. The viewer displays WebXPRT 4 scores from almost 500 devices—including many hot new releases—and we’re adding new scores all the time. To learn more about the viewer’s capabilities and how you can use it to compare devices, check out this blog post.

Another resource we offer is the XPRT results browser. The browser is the most efficient way to access the XPRT results database, which currently holds more than 3,400 test results from over 140 sources, including major tech review publications around the world, OEMs, and independent testers. It offers a wealth of current and historical performance data across all of the XPRT benchmarks and hundreds of devices. You can read more about how to use the results browser here.

Also, if you’re considering a popular device, chances are good that a recent tech review includes an XPRT score for that device. Two quick ways to find these reviews: (1) go to your favorite tech review site and search for “XPRT” and (2) go to a search engine and enter the device name and XPRT name (e.g. “Lenovo ThinkPad” and “WebXPRT”). Here are a few recent tech reviews that use one of the XPRTs to evaluate a popular device:

The XPRTs can help back-to-school shoppers make better-informed and more confident tech purchases. As this school year begins, we hope you’ll find the data you need on our site or in an XPRT-related tech review. If you have any questions about the XPRTs, XPRT scores, or the results database please feel free to ask!

Justin

Posted in ASUS, benchmark, Benchmarking, Chromebooks, Education, Laptops, PCs, Performance benchmarking, Phones, results, Tablets, tech press | Also tagged , , , , , , , , , , , , , , , , ,

Exploring the XPRT white paper library

Justin Greene
By Justin Greene
on July 20, 2023

As part of our commitment to publishing reliable, unbiased benchmarks, we strive to make the XPRT development process as transparent as possible. In the technology assessment industry, it’s not unusual for people to claim that any given benchmark contains hidden biases, so we take preemptive steps to address this issue by publishing XPRT benchmark source code, detailed system disclosures and test methodologies, and in-depth white papers. Today, we’re focusing on the XPRT white paper library.

The XPRT white paper library currently contains 21 white papers that we’ve published over the last 12 years. We started publishing white papers to provide XPRT users with more information about how we design our benchmarks, why we make certain development decisions, and how the benchmarks work. If you have questions about any aspect of one of the XPRT benchmarks, the white paper library is a great place to find some answers.

For example, the Exploring WebXPRT 4 white paper describes the design and structure of WebXPRT 4, including detailed information about the benchmark’s harness, HTML5 and WebAssembly (WASM) capability checks, and the structure of the performance test workloads. It also includes explanations of the benchmark’s scoring methodology, how to automate tests, and how to submit results for publication.

The companion WebXPRT 4 results calculation white paper explains the formulas that WebXPRT 4 uses to calculate the individual workload scenario scores and overall score, provides an overview of the statistical techniques WebXPRT uses to translate raw timings into scores, and explains the benchmark’s confidence interval and how it differs from typical benchmark variability. To supplement the white paper’s discussion of the results calculation process, we published a results calculation spreadsheet that shows the raw data from a sample test run and reproduces the exact calculations WebXPRT uses to produce test scores.

We hope that the XPRT white paper library will prove to be a useful resource for you. If you have questions about any of our white papers, or suggestions for topics that you’d like us to cover in possible future white papers, please let us know!

Justin

Posted in benchmark, Benchmarking, BenchmarkXPRT, BenchmarkXPRT development community, HTML5, WebAssembly, WebXPRT, WebXPRT 4, White papers | Also tagged , , , , , ,

Check out the other XPRTs:

Forgot your password?
© Principled Technologies, Inc. 2024   |   Privacy policy

Cookie Notice: Our website uses cookies to deliver a smooth experience by storing logins and saving user information. By continuing to use our site, you agree with our usage of cookies as our privacy policy outlines.

x