BenchmarkXPRT Blog banner

Tag Archives: test results

We welcome your CloudXPRT results!

Justin Greene
By Justin Greene
on July 15, 2021

We recently published a set of CloudXPRT Data Analytics and Web Microservices workload test results submitted by Quanta Computer, Inc. The Quanta submission is the first set of CloudXPRT results that we’ve published using the formal results submission and approval process. We’re grateful to the Quanta team for carefully following the submission guidelines, enabling us to complete the review process without a hitch.

If you are unfamiliar with the process, you can find general information about how we review submissions in a previous blog post. Detailed, step-by-step instructions are available on the results submission page. As a reminder for testers who are considering submitting results for July, the submission deadline is tomorrow, Friday July 16, and the publication date is Friday July 30. We list the submission and publication dates for the rest of 2021 below. Please note that we do not plan to review submissions in December, so if we receive results submissions after November 30, we may not publish them until the end of January 2022.

August

Submission deadline: Tuesday 8/17/21

Publication date: Tuesday 8/31/21

September

Submission deadline: Thursday 9/16/21

Publication date: Thursday 9/30/21

October

Submission deadline: Friday 10/15/21

Publication date: Friday 10/29/21

November

Submission deadline: Tuesday 11/16/21

Publication date: Tuesday 11/30/21

December

Submission deadline: N/A

Publication date: N/A

If you have any questions about the CloudXPRT results submission, review, or publication process, please let us know!

Justin

Posted in BenchmarkXPRT, Cloud, CloudXPRT, Cross-platform benchmarks, data analytics, Datacenter, microservices, Performance benchmarking, results, results submission, Server, Servers, web microservices | Also tagged , , , , , , ,

How to submit WebXPRT results for publication

Justin Greene
By Justin Greene
on June 24, 2021

It’s been a while since we last discussed the process for submitting WebXPRT results to be considered for publication in the WebXPRT results browser and the WebXPRT Processor Comparison Chart, so we thought we’d offer a refresher.

Unlike sites that publish all results they receive, we hand-select results from internal lab testing, user submissions, and reliable tech media sources. In each case, we evaluate whether the score is consistent with general expectations. For sources outside of our lab, that evaluation includes confirming that there is enough detailed system information to help us determine whether the score makes sense. We do this for every score on the WebXPRT results page and the general XPRT results page. All WebXPRT results we publish automatically appear in the processor comparison chart as well.

Submitting your score is quick and easy. At the end of the WebXPRT test run, click the Submit your results button below the overall score, complete the short submission form, and click Submit again. The screenshot below shows how the form would look if I submitted a score at the end of a WebXPRT 3 run on my personal system.

After you submit your score, we’ll contact you to confirm how we should display the source. You can choose one of the following:

  • Your first and last name
  • “Independent tester” (for those who wish to remain anonymous)
  • Your company’s name, provided that you have permission to submit the result in their name. To use a company name, we ask that you provide a valid company email address.


We will not publish any additional information about you or your company without your permission.

We look forward to seeing your score submissions, and if you have suggestions for the processor chart or any other aspect of the XPRTs, let us know!

Justin

Posted in benchmark, Benchmark metrics, Benchmarking, Browser-based benchmarks, Cross-platform benchmarks, Performance of computing devices, results, results submission, WebXPRT, WebXPRT 3, WebXPRT Processor Comparison Chart | Also tagged , , , , ,

The Introduction to CloudXPRT white paper is now available!

Justin Greene
By Justin Greene
on July 2, 2020

Today, we published the Introduction to CloudXPRT white paper. The paper provides an overview of our latest benchmark and consolidates CloudXPRT-related information that we’ve published in the XPRT blog over the past several months. It describes the CloudXPRT workloads, choosing and downloading installation packages, submitting CloudXPRT results for publication, and possibilities for additional development in the coming months.

CloudXPRT is one of the most complex tools in the XPRT family, and there are more CloudXPRT-related topics to discuss than we could fit in this first paper. In future white papers, we will discuss in greater detail each of the benchmark workloads, the range of test configuration options, results reporting, and methods for analysis.

We hope that Introduction to CloudXPRT will provide testers who are interested in CloudXPRT with a solid foundation of understanding on which they can build. Moving forward, we will provide links to the paper in the Helpful Info box on CloudXPRT.com and the CloudXPRT section of our XPRT white papers page.

If you have any questions about CloudXPRT, please let us know!

Justin

Posted in benchmark, Benchmarking, BenchmarkXPRT, Cloud, CloudXPRT, White papers | Also tagged , , , ,

The CloudXPRT results viewer is live

Justin Greene
By Justin Greene
on June 25, 2020

We’re happy to announce that the CloudXPRT results viewer is now live with results from the first few rounds of CloudXPRT Preview testing we conducted in our lab. Here are some tips to help you to navigate the viewer more efficiently:

  • Click the tabs at the top of the table to switch from Data analytics workload results to Web microservices workload results.
  • Click the header of any column to sort the data on that variable. Single click to sort A to Z and double-click to sort Z to A.
  • Click the link in the Source/details column to visit a detailed page for that result, where you’ll find additional test configuration and system hardware information and the option to download results files.
  • By default, the viewer displays eight results per page, which you can change to 16, 48, or Show all.
  • The free-form search field above the table lets you filter for variables such as cloud service or processor.

We’ll be adding more features, including expanded filtering and sorting mechanisms, to the results viewer in the near future. We’re also investigating ways to present multiple data points in a graph format, which will allow visitors to examine performance behavior curves in conjunction with factors such as concurrency and resource utilization.

We welcome your CloudXPRT results submissions! To learn about the new submission and review process we’ll be using, take a look at last week’s blog.

If you have any questions or suggestions for ways that we can improve the results viewer, please let us know!

Justin

Posted in Benchmark metrics, Benchmarking, BenchmarkXPRT, Cloud, CloudXPRT, data analytics, Interactive tools, microservices, Performance benchmarking, results, results submission, web microservices | Also tagged , , , ,

The CloudXPRT Preview results submission schedule

Justin Greene
By Justin Greene
on June 18, 2020

A few weeks ago, we shared the general framework of the periodic results publication process we will use for CloudXPRT. Now that the CloudXPRT Preview is live, we’re ready to share more details about the results review group; the submission, review, and publication cycles; and the schedule for the first three months.

The results review group
The CloudXPRT results review group will serve as a sanity check and a forum for comments on each month’s submissions. All registered BenchmarkXPRT Development Community members who wish to participate in the review process can join the group by contacting us via email. We’ll confirm receipt of your request and add you to the review group mailing list. Any non-members who would like to join the review group can contact us and we’ll help you become community members.

The submission, review, and publication cycle
We will update the CloudXPRT results database once a month on a published schedule. While testers can submit results through the CloudXPRT results submission page at any time, two weeks prior to each publication date, we will close submissions for that review cycle. One week prior to each publication date, we will email details of that month’s submissions to the results review group, along with the deadline for sending post-publication feedback.

Schedule for the first three publication cycles
We will publish results to the database on the last business day of each month and will close the submission window at 11:59 PM on the business day that falls two weeks earlier (with occasional adjustments for holidays). The schedule will be available at least six months in advance on CloudXPRT.com.

The schedule for the first three cycles is as follows:

July
Submission deadline: Friday 7/17/20
Publication date: Friday 7/31/20
August
Submission deadline: Monday 8/17/20
Publication date: Monday 8/31/20
September
Submission deadline: Wednesday 9/16/20
Publication date: Wednesday 9/30/20

As a reminder, members of the tech press, vendors, and other testers are free to publish CloudXPRT results at any time. We may choose to add such results to our database on the monthly publication date, after first vetting them.

We look forward to reviewing the first batch of results! If you have any questions about CloudXPRT or the results submission or review process, let us know!

Justin

Posted in benchmark, Benchmark metrics, Benchmarking, BenchmarkXPRT, BenchmarkXPRT development community, Cloud, CloudXPRT, Datacenter, results, results submission | Also tagged , , , ,

More information about the CloudXPRT results submission process

Justin Greene
By Justin Greene
on May 28, 2020

Earlier this month, we discussed the possibility of using a periodic results submission process for CloudXPRT instead of the traditional rolling publication process that we’ve used for the other XPRTs. We’ve received some positive responses to the idea, and while we’re still working out some details, we’re ready to share the general framework of the process we’re planning to use.

  • We will establish a results review group, which only official BenchmarkXPRT Development Community members can join.
  • We will update the CloudXPRT database with new results once a month, on a pre-published schedule.
  • Two weeks before each publication date, we will stop accepting submissions for consideration for that review cycle.
  • One week before each publication date, we will send an email to the results review group that includes the details of that month’s submissions for review.
  • The results review group will serve as a sanity check process and a forum for comments on the month’s submissions, but we reserve the right of final approval for publication.
  • We will not restrict publishing results outside of the monthly review cadence, but we will not automatically add those results to the results database.
  • We may add externally published results to our database, but will do so only after vetting, and only on the designated day each month.

Our goal is to strike a balance between allowing the tech press, vendors, or other testers to publish CloudXPRT results on their own schedule, and simultaneously building a curated results database that OEMs or other parties can use to compete for the best results.

We’ll share more details about the review group, submission dates, and publications dates soon. Do you have questions or comments about the new process? Let us know what you think!

Justin

Posted in Benchmark metrics, BenchmarkXPRT development community, Cloud, CloudXPRT, Performance benchmarking, results, results submission, Server, Servers, Storage | Also tagged , , , ,

Check out the other XPRTs:

Forgot your password?
© Principled Technologies, Inc. 2021   |   Privacy policy

Cookie Notice: Our website uses cookies to deliver a smooth experience by storing logins and saving user information. By continuing to use our site, you agree with our usage of cookies as our privacy policy outlines.

x