recently published a set of CloudXPRT Data Analytics and Web Microservices
workload test results
submitted by Quanta Computer, Inc.
The Quanta submission is the first set of CloudXPRT results that we’ve
published using the formal results submission and approval process.
We’re grateful to the Quanta team for carefully following the submission
guidelines, enabling us to complete the review process without a hitch.
If you are unfamiliar
with the process, you can find general information about how we review
submissions in a previous blog post.
Detailed, step-by-step instructions are available on the results submission page.
As a reminder for testers who are considering submitting results for July, the
submission deadline is tomorrow, Friday July 16, and the publication date is
Friday July 30. We list the submission and publication dates for the rest of
2021 below. Please note that we do not plan to review submissions in December,
so if we receive results submissions after November 30, we may not publish them
until the end of January 2022.
Submission deadline: Tuesday 8/17/21
Publication date: Tuesday 8/31/21
Submission deadline: Thursday 9/16/21
Publication date: Thursday 9/30/21
Submission deadline: Friday 10/15/21
Publication date: Friday 10/29/21
Submission deadline: Tuesday 11/16/21
Publication date: Tuesday 11/30/21
Submission deadline: N/A
Publication date: N/A
If you have any questions about the CloudXPRT results submission, review, or publication process, please let us know!
We’re excited to have recently passed an important milestone: one million XPRT runs and downloads! Most importantly, that huge number does not just reflect past successes. As the chart below illustrates, XPRT use has grown steadily over the years. In 2021, we record, on average, more XPRT runs and downloads in one month (23,395) than we recorded in the entire first year we started tracking these stats (17,051).
We reached one million
runs and downloads in about seven and a half years. At the current rate, we’ll
reach two million in roughly three and a half more years. With WebXPRT 4 on the way, there’s a good chance we can reach that mark even sooner!
As always, we’re grateful for all the testers that have helped us reach this milestone. If you have any questions or comments about using any of the XPRTs to test your gear, let us know!
sites that publish all results they receive, we hand-select results from
internal lab testing, user submissions, and reliable tech media sources. In
each case, we evaluate whether the score is consistent with general expectations.
For sources outside of our lab, that evaluation includes confirming that there
is enough detailed system information to help us determine whether the score
makes sense. We do this for every score on the WebXPRT results page and the
general XPRT results page.
All WebXPRT results we publish automatically appear in the processor comparison
chart as well.
Submitting your score is quick and easy. At the end of the WebXPRT test run, click the Submit your results button below the overall score, complete the short submission form, and click Submit again. The screenshot below shows how the form would look if I submitted a score at the end of a WebXPRT 3 run on my personal system.
After you submit your score, we’ll contact you to confirm how we should display
the source. You can choose one of the following:
Your first and last name
“Independent tester” (for those
who wish to remain anonymous)
Your company’s name, provided
that you have permission to submit the result in their name. To use a
company name, we ask that you provide a valid company email address.
not publish any additional information about you or your company without your
We look forward to seeing your score submissions, and if you have suggestions for the processor chart or any other aspect of the XPRTs, let us know!
A few weeks ago, we discussed an error that we’d recently started encountering during the CrXPRT 2 battery life test on systems running Chrome OS v89.x and later.
The error prevents the test from completing and producing a battery life
estimate. CrXPRT stops running its normal workload cycle and produces a “Test
Error” page. The timing of the error can vary from run to run. Sometimes,
CrXPRT stops running after only a few workload iterations, while other times,
the battery life test almost reaches completion before producing the error.
We have seen the error on across multiple brands of Chromebooks running
Chrome OS v89.x and later. To our knowledge, Chromebooks running Chrome OS v88.x
and earlier versions complete the battery life test without issues. We are unaware
of any problems with the CrXPRT 2 performance test.
We’re continuing to investigate this problem. Unfortunately, we have not yet identified the root cause. Without a solution, we are recommending that for now, testers not use the CrXPRT 2 battery life test. We will post this recommendation on CrXPRT.com.
We apologize for the inconvenience that this error is causing CrXPRT 2 testers. As soon as we identify a possible solution, we will share that information here in the blog. If you have any insight into recent Chrome OS changes or flag settings that could be causing this problem, please let us know!
In early May, we sent
a survey to members of the tech press who regularly use WebXPRT in articles and
reviews. We asked for their thoughts on several aspects of WebXPRT, as well as what
they’d like to see in the upcoming fourth version of the benchmark. We also
published the survey questions here in the blog, and invited
experienced WebXPRT testers to send their feedback as well. We received some
good responses to the survey, and for the benefit of our readers, we’ve
summarized some of the key comments and suggestions below.
One respondent stated that WebXPRT is demanding enough to test
performance, but if we want to simulate modern web usage, we should find the
most up-to-date studies on common browser tasks and web technologies. This
suggestion lines up with our intention to study the feasibility of adding a WebAssembly workload.
One respondent liked that fact that unlike many other browser
One respondent suggested that we include a link to a WebXPRT
white paper within the UI, or at least a guide describing what happens during
One respondent stated that they would like for WebXPRT to
automatically produce a good result file on the local test system.
One respondent said that WebXPRT has a relatively long runtime
for a browser benchmark, and they would prefer that the runtime not increase in
We had no direct calls for a battery life test, because many
testers already have scripts and/or methodologies in place for battery testing,
but one tester suggested adding the ability to loop the test so users can measure
performance over varying lengths of time.
There were no requests to bring back any aspects of WebXPRT 2015
that we removed in WebXPRT 3.
There were no reports of significant connection issues when
testing with WebXPRT.
We greatly appreciate the members of the tech press that responded to the survey. We’re still in the planning stages of WebXPRT 4, so there’s still time for anyone to send comments or ideas to email@example.com. We look forward to hearing from you!
We recently received questions about whether we accept CloudXPRT
results submissions from testing on pre-production gear, and how we would handle
any differences between results from pre-production and production-level tests.
To answer first question, we are not opposed to pre-production
results submissions. We realize that vendors often want to include benchmark
results in launch-oriented marketing materials they release before their
hardware or software is publicly available. To help them do so, we’re happy to
consider pre-production submissions on a case-by-case basis. All such submissions
must follow the normal CloudXPRT results
submission process, and undergo
vetting by the CloudXPRT Results Review Group according to the standard review
and publication schedule. If we decide to publish pre-production results on our site, we
will clearly note their pre-production status.
In response to the second question, the CloudXPRT Results Review Group will handle any challenges to published results or perceived discrepancies between pre-production and production-level results on a case-by-case basis. We do not currently have a formal process for challenges; anyone who would like to initiate a challenge or express comments or concerns about a result should address the review group via firstname.lastname@example.org. Our primary concern is always to ensure that published results accurately reflect the performance characteristics of production-level hardware and software. If it becomes necessary to develop more policies in the future, we’ll do so, but we want to keep things as simple as possible.
If you have any questions about the CloudXPRT results submission process, please let us know!