In the coming months,
we’ll be moving forward with the first stages of the WebXPRT 4 development
process. It’s been a while since we last asked readers to send their
thoughts about web technologies and workloads that may be a good fit for
WebXPRT 4, but we’re still very much open to ideas. If you missed our previous
posts about possible changes for WebXPRT 4, we recap the most prominent ideas
below. We also request specific feedback regarding a potential battery life
Community members have asked about a WebXPRT 4 battery life test. Any such test would likely be very similar to the performance-weighted battery life test in CrXPRT 2 (as opposed to a simple rundown test). While WebXPRT runs in almost any browser, cross-browser compatibility issues could cause a WebXPRT battery life test to run in only one browser. If this turned out to be the case, would you still be interested in using the battery life test? Please let us know.
One of the most promising ideas is the potential addition of one or more WebAssembly (WASM) workloads. WASM is a low-level, binary instruction format that works across all modern browsers. It offers web developers a great deal of flexibility and provides the speed and efficiency necessary for running complex client applications in the browser. WASM enables a variety of workload scenario options, including gaming, video editing, VR, virtual machines, image recognition, and interactive educational content.
Other ideas include using a WebGL-based workload to target GPUs, and simulating common web applications.
We’ll start work on
WebXPRT 4 soon, but there’s still time to send your comments and ideas, so please
do so as quickly as possible!
As we’ve been working
on improvements and updates for CloudXPRT, we’ve been using feedback from
community members to determine which changes will help testers most in the
short term. To make some of those changes available to the community as soon as
possible, we plan to release a beta version of CloudXPRT v1.1 in the coming
During the v1.1 beta
period, the CloudXPRT v1.01 installation packages on CloudXPRT.com and our GitHub repository will continue to include the officially supported
version of CloudXPRT. However, interested testers can experiment with the v1.1
beta version in new environments while we finalize the build for official
The CloudXPRT v1.1
beta includes the following primary changes:
We’re adding support for Ubuntu 20.04.2 or later, the number one
request we’ve received.
We’re consolidating and standardizing the installation packages
for both workloads. Instead of one package for the data analytics workload and
four separate packages for the web microservices workload, each workload will
have two installation packages: one for all on-premises testing and one for
testing with all three supported CSPs.
We’re incorporating Terraform to help create and
configure VMs, which will help to prevent situations when testers do not
allocate enough storage per VM prior to testing.
We use Kubespray to manage Kubernetes
clusters, and Kubespray uses Calico as the default network plug in. Calico has not always worked
well for CloudXPRT in the CSP environment, so we’re replacing Calico with Weave.
At the start of the
beta period, we will share a link to the v1.1 beta download page here in the
blog. You’ll be free to share this link. To avoid confusion, we will not add the
beta download to the v1.01 downloads available on CloudXPRT.com.
As the beta release
date approaches, we’ll share more details about timelines, access, and any additional
changes to the benchmark. If you have any questions about the upcoming
CloudXPRT v1.1 beta, please let us know!
We’re currently formulating our 2021 development roadmap for the XPRTs. In addition to planning CloudXPRT and WebXPRT updates, we’re discussing the possibility of releasing HDXPRT 5 in 2021. It’s hard for me to believe, but it’s been about two and a half years since we started work on HDXPRT 4, and February 2021 will mark two years since the first HDXPRT 4 release. Windows PCs are more powerful than ever, so it’s a good time to talk about how we can enhance the benchmark’s ability to measure how well the latest systems handle real-world media technologies and applications.
When we plan a new
version of an XPRT benchmark, one of our first steps is updating the
benchmark’s workloads so that they will remain relevant in years to come. We
almost always update application content, such as photos and videos, to
contemporary file resolutions and sizes. For example, we added both higher-resolution
photos and a 4K video conversion task in HDXPRT 4. Are there specific types of
media files that you think would be especially relevant to high-performance
media tasks over the next few years?
Next, we will assess
the suitability of the real-world trial applications that the editing photos,
editing music, and converting videos test scenarios use. Currently, these are Adobe
Photoshop Elements, Audacity, CyberLink MediaEspresso, and HandBrake. Can you
think of other applications that belong in a high-performance media processing
In HDXPRT 4, we gave
testers the option to target a system’s discrete graphics card during the video
conversion workload. Has this proven useful in your testing? Do you have
suggestions for new graphics-oriented workloads?
We’ll also strive to
make the UI more intuitive, to simplify installation, and to reduce the size of
the installation package. What elements of the current UI do you find
especially useful or think we could improve?
We welcome your answers to these questions and any additional suggestions or comments on HDXPRT 5. Send them our way!
A few months ago, we invited readers to send in their thoughts and ideas about web
technologies and workload scenarios that may be a good fit for the next WebXPRT. We’d like to share a few of those ideas today, and we invite
you to continue to send your feedback. We’re approaching the time when we need to begin firming up
plans for a WebXPRT 4 development cycle in 2021, but there’s still plenty of
time for you to help shape the future of the benchmark.
One of the most
promising ideas for WebXPRT 4 is the potential addition of one or more WebAssembly (WASM) workloads.
WASM is a low-level, binary instruction format that works across all modern browsers.
It offers web developers a great deal of flexibility and provides the speed and
efficiency necessary for running complex client applications in the browser. WASM
enables a variety of workload scenario options, including gaming, video editing, VR, virtual
machines, image recognition, and interactive educational content.
In addition, the
Chrome team is dropping Portable Native Client (PNaCL) support in favor of
WASM, which is why we had to remove a PNaCL workload when updating CrXPRT 2015 to CrXPRT 2. We
generally model CrXPRT workloads on existing WebXPRT workloads, so
familiarizing ourselves with WASM could ultimately benefit more than one XPRT
We are also
considering adding a web-based machine learning workload with TensorFlow for
tasks including image classification, object detection, sentence encoding,
natural language processing, and more. We could also use this technology to
enhance one of WebXPRT’s existing AI-themed workloads, such as Organize Album
using AI or Encrypt Notes and OCR Scan.
Other ideas include using
a WebGL-based workload to target GPUs and investigating ways to incorporate a
battery life test. What do you think? Let us know!
BenchmarkXPRT Development Community started almost 10 years ago with the development
of the High Definition Experience & Performance Ratings Test, also known as
HDXPRT. Back then, we distributed the benchmark to interested parties by
mailing out physical DVDs. We’ve come a long way since then, as testers now
freely and easily access six XPRT benchmarks from our site and major app
hardware manufacturers, and tech journalists—the core group of XPRT testers—work
within a constantly changing tech landscape. Because of our commitment to
providing those testers with what they need, the XPRTs grew as we developed
additional benchmarks to expand the reach of our tools from PCs to servers and
all types of notebooks, Chromebooks, and mobile devices.
today’s tech landscape continues to evolve at a rapid pace, our desire to play
an active role in emerging markets continues to drive us to expand our testing
capabilities into areas like machine learning (AIXPRT)
and cloud-first applications (CloudXPRT).
While these new technologies carry the potential to increase efficiency, improve
quality, and boost the bottom line for companies around the world, it’s often
difficult to decide where and how to invest in new hardware or services. The
ever-present need for relevant and reliable data is the reason many
organizations use the XPRTs to help make confident choices about their
company’s future tech.
We just released a new video that helps to explain what the XPRTs provide and how they can play an important role in a company’s tech purchasing decisions. We hope you’ll check it out!
excited about the continued growth of the XPRTs, and we’re eager to meet the
challenges of adapting to the changing tech landscape. If you have any questions
about the XPRTs or suggestions for future benchmarks, please let us know!
much of the BenchmarkXPRT Development Community’s history, we offered community
members exclusive access to XPRT benchmark source code. Back in February,
we started to experiment with a different approach when we made the AIXPRT
source code publicly available on GitHub. By allowing anyone who is
interested in AIXPRT to download and review the source code, we reinforced our
commitment to making the XPRT development process as transparent as possible. We
also want the XPRTs to continue to contribute to fair practices in the
benchmarking world, and we believe that expanded access to the source code
encourages constructive feedback to help in this goal.
feedback we received after publishing the AIXPRT source code was very positive;
thank you to all who reached out. Because of that feedback and our desire to
increase openness, we’ve decided use standard open source licenses to make the CloudXPRT
source code available to the public when we release of the first build, or
shortly thereafter. As with AIXPRT, folks will be able to download the CloudXPRT
source code and submit potential workloads for future consideration, but we
reserve the right to control derivative works.
share more information about the first CloudXPRT release and its source code in
the coming weeks. If you have any questions about XPRT source code, feel free to ask.
We also welcome any thoughts about using
this approach to release the source code of other XPRT benchmarks. As always, feel
free to comment below or reach out by email.