BenchmarkXPRT Blog banner

Category: Storage

More information about the CloudXPRT results submission process

Earlier this month, we discussed the possibility of using a periodic results submission process for CloudXPRT instead of the traditional rolling publication process that we’ve used for the other XPRTs. We’ve received some positive responses to the idea, and while we’re still working out some details, we’re ready to share the general framework of the process we’re planning to use.

  • We will establish a results review group, which only official BenchmarkXPRT Development Community members can join.
  • We will update the CloudXPRT database with new results once a month, on a pre-published schedule.
  • Two weeks before each publication date, we will stop accepting submissions for consideration for that review cycle.
  • One week before each publication date, we will send an email to the results review group that includes the details of that month’s submissions for review.
  • The results review group will serve as a sanity check process and a forum for comments on the month’s submissions, but we reserve the right of final approval for publication.
  • We will not restrict publishing results outside of the monthly review cadence, but we will not automatically add those results to the results database.
  • We may add externally published results to our database, but will do so only after vetting, and only on the designated day each month.

Our goal is to strike a balance between allowing the tech press, vendors, or other testers to publish CloudXPRT results on their own schedule, and simultaneously building a curated results database that OEMs or other parties can use to compete for the best results.

We’ll share more details about the review group, submission dates, and publications dates soon. Do you have questions or comments about the new process? Let us know what you think!

Justin

XPRTs in the datacenter

The XPRTs have been very successful on desktops, notebooks, tablets, and phones. People have run WebXPRT over 295,000 times. It and other benchmarks such as MobileXPRT, HDXPRT, and CrXPRT are important tools globally for evaluating device performance on various consumer and business client platforms.

We’ve begun branching out with tests for edge devices with AIXPRT, our new artificial intelligence benchmark. While typical consumers won’t be able to run AIXPRT on their devices initially, we feel that it is important for the XPRTs to play an active role in a critical emerging market. (We’ll have some updates on the AIXPRT front in the next few weeks.)

Recently, both community members and others have asked about the possibility of the XPRTs moving into the datacenter. Folks face challenges in evaluating the performance and suitability to task of such datacenter mainstays as servers, storage, networking infrastructure, clusters, and converged solutions. These challenges include the lack of easy-to-run benchmarks, the complexity and cost of the equipment (multi-tier servers, large amounts of storage, and fast networks) necessary to run tests, and confusion about best testing practices.

PT has a lot of expertise in measuring datacenter performance, as you can tell from the hundreds of datacenter-focused test reports on our website. We see great potential in our working with the BenchmarkXPRT Development Community to help in this area. It is very possible that, as with AIXPRT, our approach to datacenter benchmarks would differ from the approach we’ve taken with previous benchmarks. While we have ideas for useful benchmarks we might develop down the road, more immediate steps could be drafting white papers, developing testing guidelines, or working with vendors to set up a lab.

Right now, we’re trying to gauge the level of interest in having such tools and in helping us carry out these initiatives. What are the biggest challenges you face in datacenter-focused performance and suitability to task evaluations? Would you be willing to work with us in this area? We’d love to hear from you and will be reaching out to members of the community over the coming weeks.

As always, thanks for your help!

Bill

Check out the other XPRTs:

Forgot your password?