In early May, we sent
a survey to members of the tech press who regularly use WebXPRT in articles and
reviews. We asked for their thoughts on several aspects of WebXPRT, as well as what
they’d like to see in the upcoming fourth version of the benchmark. We also
published the survey questions here in the blog, and invited
experienced WebXPRT testers to send their feedback as well. We received some
good responses to the survey, and for the benefit of our readers, we’ve
summarized some of the key comments and suggestions below.
- One respondent stated that WebXPRT is demanding enough to test
performance, but if we want to simulate modern web usage, we should find the
most up-to-date studies on common browser tasks and web technologies. This
suggestion lines up with our intention to study the feasibility of adding a WebAssembly workload.
- One respondent liked that fact that unlike many other browser
- One respondent suggested that we include a link to a WebXPRT
white paper within the UI, or at least a guide describing what happens during
- One respondent stated that they would like for WebXPRT to
automatically produce a good result file on the local test system.
- One respondent said that WebXPRT has a relatively long runtime
for a browser benchmark, and they would prefer that the runtime not increase in
- We had no direct calls for a battery life test, because many
testers already have scripts and/or methodologies in place for battery testing,
but one tester suggested adding the ability to loop the test so users can measure
performance over varying lengths of time.
- There were no requests to bring back any aspects of WebXPRT 2015
that we removed in WebXPRT 3.
- There were no reports of significant connection issues when
testing with WebXPRT.
We greatly appreciate the members of the tech press that responded to the survey. We’re still in the planning stages of WebXPRT 4, so there’s still time for anyone to send comments or ideas to email@example.com. We look forward to hearing from you!
Just this past summer, WebXPRT passed the 250,000-run milestone, and since then, the run total has already passed 330,000. September was our biggest month ever, with over 28,000 WebXPRT runs! We sometimes like to show the community how far a reach the XPRTs have around the world by reporting the latest stats on the number of articles and reviews that mention the XPRTs, and the fact is that most of those mentions involve WebXPRT. Today, I thought it would be interesting to bring the numbers to life and provide a glimpse of how the tech press uses WebXPRT. Here’s a sample of WebXPRT in action during the past couple of weeks.
While WebXPRT continues to be a useful tool for tech enthusiasts around the world, you don’t have to be a tech expert to benefit from it. If you’d like to know more about WebXPRT, check out our recent video, What is WebXPRT and why should I care?
Today, we published the Exploring WebXPRT 3 white paper. The paper describes the differences between WebXPRT 3 and WebXPRT 2015, including changes we made to the harness and the structure of the six performance test workloads. We also explain the benchmark’s scoring methodology, how to automate tests, and how to submit results for publication. Readers will also find additional detail about the third-party functions and libraries that WebXPRT uses during the HTML5 capability checks and performance workloads.
Because data collection and privacy concerns are more relevant than ever, we also discuss the WebXPRT data collection mechanisms and our commitment to respecting testers’ privacy. Finally, for readers who may be unfamiliar with the XPRTs, we describe the other benchmark tools in the XPRT family, the role of the BenchmarkXPRT Development Community, and how you can contribute to the XPRTs.
Along with the WebXPRT 3 results calculation white paper and spreadsheet, the Exploring WebXPRT 3 white paper is designed to promote the high level of transparency and disclosure that is a core value of the BenchmarkXPRT Development Community. Both WebXPRT white papers and the results calculation spreadsheet are available on WebXPRT.com and on our XPRT white papers page. If you have any questions about the WebXPRT, please let us know, and be sure to check out our other XPRT white papers.