In May, we surveyed
longtime WebXPRT users regarding the types of changes they would like to see in
a WebXPRT 4. We sent the survey to journalists at several tech press outlets,
and invited our blog readers to participate as well. We received some very helpful feedback. As we explore new possibilities for WebXPRT 4, we’ve decided to
open an updated version of the survey. We’ve adjusted the questions a bit based
on previous feedback and added some new ones, so we invite you to respond even
if you participated in the original survey.
We’re excited to have recently passed an important milestone: one million XPRT runs and downloads! Most importantly, that huge number does not just reflect past successes. As the chart below illustrates, XPRT use has grown steadily over the years. In 2021, we record, on average, more XPRT runs and downloads in one month (23,395) than we recorded in the entire first year we started tracking these stats (17,051).
We reached one million
runs and downloads in about seven and a half years. At the current rate, we’ll
reach two million in roughly three and a half more years. With WebXPRT 4 on the way, there’s a good chance we can reach that mark even sooner!
As always, we’re grateful for all the testers that have helped us reach this milestone. If you have any questions or comments about using any of the XPRTs to test your gear, let us know!
sites that publish all results they receive, we hand-select results from
internal lab testing, user submissions, and reliable tech media sources. In
each case, we evaluate whether the score is consistent with general expectations.
For sources outside of our lab, that evaluation includes confirming that there
is enough detailed system information to help us determine whether the score
makes sense. We do this for every score on the WebXPRT results page and the
general XPRT results page.
All WebXPRT results we publish automatically appear in the processor comparison
chart as well.
Submitting your score is quick and easy. At the end of the WebXPRT test run, click the Submit your results button below the overall score, complete the short submission form, and click Submit again. The screenshot below shows how the form would look if I submitted a score at the end of a WebXPRT 3 run on my personal system.
After you submit your score, we’ll contact you to confirm how we should display
the source. You can choose one of the following:
Your first and last name
“Independent tester” (for those
who wish to remain anonymous)
Your company’s name, provided
that you have permission to submit the result in their name. To use a
company name, we ask that you provide a valid company email address.
not publish any additional information about you or your company without your
We look forward to seeing your score submissions, and if you have suggestions for the processor chart or any other aspect of the XPRTs, let us know!
In early May, we sent
a survey to members of the tech press who regularly use WebXPRT in articles and
reviews. We asked for their thoughts on several aspects of WebXPRT, as well as what
they’d like to see in the upcoming fourth version of the benchmark. We also
published the survey questions here in the blog, and invited
experienced WebXPRT testers to send their feedback as well. We received some
good responses to the survey, and for the benefit of our readers, we’ve
summarized some of the key comments and suggestions below.
One respondent stated that WebXPRT is demanding enough to test
performance, but if we want to simulate modern web usage, we should find the
most up-to-date studies on common browser tasks and web technologies. This
suggestion lines up with our intention to study the feasibility of adding a WebAssembly workload.
One respondent liked that fact that unlike many other browser
One respondent suggested that we include a link to a WebXPRT
white paper within the UI, or at least a guide describing what happens during
One respondent stated that they would like for WebXPRT to
automatically produce a good result file on the local test system.
One respondent said that WebXPRT has a relatively long runtime
for a browser benchmark, and they would prefer that the runtime not increase in
We had no direct calls for a battery life test, because many
testers already have scripts and/or methodologies in place for battery testing,
but one tester suggested adding the ability to loop the test so users can measure
performance over varying lengths of time.
There were no requests to bring back any aspects of WebXPRT 2015
that we removed in WebXPRT 3.
There were no reports of significant connection issues when
testing with WebXPRT.
We greatly appreciate the members of the tech press that responded to the survey. We’re still in the planning stages of WebXPRT 4, so there’s still time for anyone to send comments or ideas to firstname.lastname@example.org. We look forward to hearing from you!
Device reviews in publications
such as AnandTech, Notebookcheck, and PCMag, among many others, often feature
WebXPRT test results, and we appreciate the many members of the tech press that
use WebXPRT. As we move forward with the WebXPRT 4 development process, we’re especially
interested in learning what longtime users would like to see in a new version
of the benchmark.
In previous posts,
we’ve asked people to weigh in on the potential addition of a WebAssembly workload or a battery life test. We’d also like to ask experienced testers some other
test-related questions. To that end, this week we’ll be sending a WebXPRT 4
survey directly to members of the tech press who frequently publish WebXPRT
Regardless of whether you are a member of the tech press, we invite you to participate by sending your answers to any or all the questions below to email@example.com. We ask you to do so by the end of May.
Do you think WebXPRT 3’s selection of workload scenarios is representative of modern web tasks?
How do you think WebXPRT compares to other common browser-based benchmarks, such as JetStream, Speedometer, and Octane?
Are there web technologies that you’d like us to include in additional workloads?
Are you happy with the WebXPRT 3 user interface? If not, what UI changes would you like to see?
Are there any aspects of WebXPRT 2015 that we changed in WebXPRT 3 that you’d like to see us change back?
Have you ever experienced significant connection issues when testing with WebXPRT?
Given the array of workloads, do you think the WebXPRT runtime is reasonable? Would you mind if the average runtime were a bit longer?
Are there any other aspects of WebXPRT 3 that you’d like to see us change?
If you’d like to discuss any topics
that we did not cover in the questions above, please feel free to include additional
comments in your response. We look forward to hearing your thoughts!
We’re excited to see
that users have successfully completed over 750,000 WebXPRT runs! If you’ve run WebXPRT in any of the more than 654 cities
and 68 countries from which we’ve received complete test data—including
newcomers Belize, Cambodia, Croatia, and Pakistan—we’re grateful for your help.
We could not have reached this milestone without you!
As the chart below illustrates, WebXPRT use has grown steadily over the years. We now record, on average, almost twice as many WebXPRT runs in one month as we recorded in the entirety of our first year. In addition, with over 82,000 runs to date in 2021, there are no signs that growth is slowing.
Developing a new
benchmark is never easy, and the obstacles multiply when you attempt to create
a cross-platform benchmark, such as WebXPRT, that will run on a wide variety of
devices. Establishing trust with the benchmarking community is another
challenge. Transparency, consistency, and technical competency on our part are critical
factors in building that trust, but the people who take time out of their busy
schedules to run the benchmark for the first time also play a role. We thank
all of the manufacturers, OEM labs, and members of the tech press who decided
to give WebXPRT a try, and we look forward to your input as we continue to improve WebXPRT in the years to come.
If you have any
questions or comments about WebXPRT, we’d love to hear from you!