BenchmarkXPRT Blog banner

Category: White papers

Gain a deeper understanding of WebXPRT 4 with our results calculation white paper

More people around the world are using WebXPRT 4 now than ever before. It’s exciting to see that growth, which also means that many people are visiting our site and learning about the XPRTs for the first time. Because new visitors may not know how the XPRT family of benchmarks differs from other benchmarking efforts, we occasionally like to revisit the core values of our open development community here in the blog—and show how those values translate into more free resources for you.

One of our primary values is transparency in all our benchmark development and testing processes. We share information about our progress with XPRT users throughout the development process, and we invite people to contribute ideas and feedback along the way. We also publish both the source code of our benchmarks and detailed information about how they work, unlike benchmarks that use a “black box” model.

For WebXPRT 4 users who are interested in knowing more about the nuts and bolts of the benchmark, we offer several information-packed resources, including our focus for today, the WebXPRT 4 results calculation and confidence interval white paper. The white paper explains the WebXPRT 4 confidence interval, how it differs from typical benchmark variability, and the formulas the benchmark uses to calculate the individual workload scenario scores and overall score on the end-of-test results screen. The paper also provides an overview of the statistical methodology that WebXPRT uses to translate raw timings into scores.

In addition to the white paper’s discussion of the results calculation process, we’ve also provided a results calculation spreadsheet that shows the raw data from a sample test run and reproduces the calculations WebXPRT uses to generate both the workload scores and an overall score.

In potential future versions of WebXPRT, it’s likely that we’ll continue to use the same—or very similar—statistical methodologies and results calculation formulas that we’ve documented in the results calculation white paper and spreadsheet. That said, if you have suggestions for how we could improve those methods or formulas—either in part or in whole—please don’t hesitate to contact us. We’re interested in hearing your ideas!

The white paper is available on WebXPRT.com and on our XPRT white papers page. If you have any questions about the paper or spreadsheet, WebXPRT, or the XPRTs in general, please let us know.

Justin

Exploring the XPRT white paper library

As part of our commitment to publishing reliable, unbiased benchmarks, we strive to make the XPRT development process as transparent as possible. In the technology assessment industry, it’s not unusual for people to claim that any given benchmark contains hidden biases, so we take preemptive steps to address this issue by publishing XPRT benchmark source code, detailed system disclosures and test methodologies, and in-depth white papers. Today, we’re focusing on the XPRT white paper library.

The XPRT white paper library currently contains 21 white papers that we’ve published over the last 12 years. We started publishing white papers to provide XPRT users with more information about how we design our benchmarks, why we make certain development decisions, and how the benchmarks work. If you have questions about any aspect of one of the XPRT benchmarks, the white paper library is a great place to find some answers.

For example, the Exploring WebXPRT 4 white paper describes the design and structure of WebXPRT 4, including detailed information about the benchmark’s harness, HTML5 and WebAssembly (WASM) capability checks, and the structure of the performance test workloads. It also includes explanations of the benchmark’s scoring methodology, how to automate tests, and how to submit results for publication.

The companion WebXPRT 4 results calculation white paper explains the formulas that WebXPRT 4 uses to calculate the individual workload scenario scores and overall score, provides an overview of the statistical techniques WebXPRT uses to translate raw timings into scores, and explains the benchmark’s confidence interval and how it differs from typical benchmark variability. To supplement the white paper’s discussion of the results calculation process, we published a results calculation spreadsheet that shows the raw data from a sample test run and reproduces the exact calculations WebXPRT uses to produce test scores.

We hope that the XPRT white paper library will prove to be a useful resource for you. If you have questions about any of our white papers, or suggestions for topics that you’d like us to cover in possible future white papers, please let us know!

Justin

Celebrating 10 years of WebXPRT!

We’re excited to announce that it’s been 10 years since the initial launch of WebXPRT! In early 2013, we introduced WebXPRT as a unique browser performance benchmark in a market space that was already crowded with a variety of specialized measurement tools. Our goal was to offer a benchmark that could compare the performance of almost any web-enabled device, using scenarios created to mirror real-world tasks. We wanted it to be a free, easily accessible, easy-to-run, useful, and appealing testing option for OEM labs, vendors, and the tech press.

When we look back on the last 10 years of WebXPRT, we can’t help but conclude that our efforts have been successful. Since those early days, the WebXPRT market presence has grown from humble beginnings into a worldwide industry standard. Hundreds of tech press publications have used WebXPRT in thousands of articles and reviews, and testers have now run the benchmark well over 1.1 million times.

Below, I’ve listed some of the WebXPRT team’s accomplishments over the last decade. If you’ve been following WebXPRT from the beginning, this may all be familiar, but if you’re new to the  community, it may be interesting to see some of the steps that contributed to making WebXPRT what it is today.

In future blog posts, we’ll look at how the number of WebXPRT runs has grown over time, and how WebXPRT use has grown among OEMs, vendors, and the tech press worldwide. Do you have any thoughts that you’d like to share from your WebXPRT testing experience? If so, let us know!

Justin

Looking back on 2022 with the XPRTs

Around the beginning of each new year, we like to take the opportunity to look back and summarize the XPRT highlights from the previous year. Readers of our newsletter are familiar with the stats and updates we include each month, but for our blog readers who don’t receive the newsletter, we’ve compiled some highlights from 2022 below.

Benchmarks
In the past year, we released WebXPRT 4, and the CloudXPRT v1.2 update package.

XPRTs in the media
Journalists, advertisers, and analysts referenced the XPRTs thousands of times in 2022. It’s always rewarding to know that the XPRTs have proven to be useful and reliable assessment tools for technology publications around the world. Media sites that used the XPRTs in 2022 include AnandTech, Android Authority, Benchlife.info (China), BodNara (South Korea), ComputerBase (Germany), DISKIDEE (Belgium), eTeknix, Expert Reviews, Gadgets 360, Hardware.info (The Netherlands), Hardware Zone (Singapore), ITC.ua (Ukraine), ITmedia (Japan), Itndaily.ru (Russia), Notebookcheck, PCMag, PC-Welt (Germany), PCWorld, TechPowerUp, Tom’s Guide, TweakTown, and ZOL.com (China).

Downloads and confirmed runs
In 2022, we had more than 10,800 benchmark downloads and 183,300 confirmed runs. Users have run our most popular benchmark, WebXPRT, more than 1,135,500 times since its debut in 2013! WebXPRT continues to be a go-to, industry-standard performance benchmark for OEM labs, vendors, and leading tech press outlets around the globe.

XPRT media, tools, and publications
Part of our mission with the XPRTs is to produce tools and materials that help testers better understand the ins and outs of benchmarking in general and the XPRTs in particular. To help achieve this goal, we published the following in 2022:

We’re thankful for everyone who used the XPRTs, joined the community, and sent questions and suggestions throughout 2022. We’re excited to see what’s in store for the XPRTs in 2023!

Justin

The WebXPRT 4 results calculation white paper is now available

Last week, we published the Exploring WebXPRT 4 white paper. The paper describes the design and structure of WebXPRT 4, including detailed information about the benchmark’s harness, HTML5 and WebAssembly capability checks, and the structure of the performance test workloads. This week, to help WebXPRT 4 testers understand how the benchmark calculates results, we’ve published the WebXPRT 4 results calculation and confidence interval white paper.

The white paper explains the WebXPRT 4 confidence interval and how it differs from typical benchmark variability, and the formulas the benchmark uses to calculate the individual workload scenario scores and overall score. The paper also provides an overview of the statistical techniques WebXPRT uses to translate raw timings into scores.

To supplement the white paper’s discussion of the results calculation process, we’ve also published a results calculation spreadsheet that shows the raw data from a sample test run and reproduces the calculations WebXPRT uses to produce workload scores and the overall score.

The paper is available on WebXPRT.com and on our XPRT white papers page. If you have any questions about the WebXPRT results calculation process, please let us know!

Justin

The Exploring WebXPRT 4 white paper is now available

This week, we published the Exploring WebXPRT 4 white paper. It describes the design and structure of WebXPRT 4, including detailed information about the benchmark’s harness, HTML5 and WebAssembly (WASM) capability checks, and changes we’ve made to the structure of the performance test workloads. We explain the benchmark’s scoring methodology, how to automate tests, and how to submit results for publication. The white paper also includes information about the third-party functions and libraries that WebXPRT 4 uses during the HTML5 and WASM capability checks and performance workloads.

The Exploring WebXPRT 4 white paper promotes the high level of transparency and disclosure that is a core value of the BenchmarkXPRT Development Community. We’ve always believed that transparency builds trust, and trust is essential for a healthy benchmarking community. That’s why we involve community members in the benchmark development process and disclose how we build our benchmarks and how they work.

You can find the paper on WebXPRT.com and our XPRT white papers page. If you have any questions about WebXPRT 4, please let us know, and be sure to check out our other XPRT white papers.

Justin

Check out the other XPRTs:

Forgot your password?