We’ve designed each of the XPRT benchmarks to assess the performance of specific types of devices in scenarios that mirror the ways consumers typically use those devices. While most XPRT benchmark users are interested in producing official overall scores, some members of the tech press have been using the XPRTs in unconventional, creative ways.
One example is the use
of WebXPRT
by Tweakers, a popular tech
review site based in The Netherlands. (The site is in Dutch, so the Google
Translate extension in Chrome was helpful for me.) As Tweakers uses WebXPRT to
evaluate all kinds of consumer hardware, they also measure the sound output of each
device. Tweakers then publishes the LAeq metric for each device,
giving readers a sense of how loud a system may be, on average, while it
performs common browser tasks.
Other labs and tech
publications have also used the XPRTs in unusual ways such as automating the
benchmarks to run during screen burn-in tests or custom battery-life rundowns. If
you’ve used any of the XPRT benchmarks in creative ways, please let us know!
We are interested in learning more about your tests, and your experiences may
provide helpful information that we can share with other XPRT users.
This week, we published the Exploring WebXPRT 4 white paper. It describes the design and structure of WebXPRT 4, including detailed information about the benchmark’s harness, HTML5 and WebAssembly (WASM) capability checks, and changes we’ve made to the structure of the performance test workloads. We explain the benchmark’s scoring methodology, how to automate tests, and how to submit results for publication. The white paper also includes information about the third-party functions and libraries that WebXPRT 4 uses during the HTML5 and WASM capability checks and performance workloads.
The Exploring WebXPRT 4 white paper promotes
the high level of transparency and disclosure that is a core value of the
BenchmarkXPRT Development Community. We’ve always believed that transparency
builds trust, and trust is essential for a healthy benchmarking community.
That’s why we involve community members in the benchmark development process
and disclose how we build our benchmarks and how they work.
We’re excited to see that users have successfully completed over 1,000,000 WebXPRT runs! If you’ve run WebXPRT in any of the 924 cities and 81 countries from which we’ve received complete test data—including newcomers Bahrain, Bangladesh, Mauritius, The Philippines, and South Korea —we’re grateful for your help. We could not have reached this milestone without you!
As the chart below illustrates, WebXPRT use has grown steadily since the debut of WebXPRT 2013. On average, we now record more WebXPRT runs in one month than we recorded in the entirety of our first year. With over 104,000 runs so far in 2022, that growth is continuing.
For us, this moment represents more than a numerical milestone. Developing and maintaining a benchmark is never easy, and a cross-platform benchmark that will run on a wide variety of devices poses an additional set of challenges. For such a benchmark to succeed, developers need not only technical competency, but the trust and support of the benchmarking community. WebXPRT is now in its ninth year, and its consistent year-over-year growth tells us that the benchmark continues to hold value for manufacturers, OEM labs, the tech press, and end users like you. We see it as a sign of trust that folks repeatedly return to the benchmark for reliable performance metrics. We’re grateful for that trust, and for everyone that’s contributed to the WebXPRT development process throughout the years.
We’ll have more to share related to this exciting milestone in the weeks to come, so stay tuned to the blog. If you have any questions or comments about WebXPRT, we’d love to hear from you!
Testers
new to the XPRT benchmarks may not know about one of the free resources we
offer. The XPRT results database currently holds more than 3,000 test results
from over 120 sources, including major tech review publications around the
world, OEMs, and independent testers. It offers a wealth of current and
historical performance data across all the XPRT benchmarks and hundreds of
devices.
We update the results
database several times a week, adding selected results from our own internal
lab testing, reliable tech media sources, and end-of-test user submissions.
(After you run one of the XPRTs, you can choose to submit the results, but they
don’t automatically appear in the database.) Before adding a result, we
evaluate whether the score makes sense and is consistent with general
expectations, which we can do only when we have sufficient system information details.
For that reason, we ask testers to disclose as much hardware and software
information as possible when publishing or submitting a result.
We encourage visitors to our site to explore the XPRT results database. There are three primary ways to do so. The first is by visiting the main BenchmarkXPRT results browser, which displays results entries for all of the XPRT benchmarks in chronological order (see the screenshot below). You can narrow the results by selecting a benchmark from the drop-down menu and can type values, such as vendor or the name of a tech publication, into the free-form filter field. For results we’ve produced in our lab, clicking “PT” in the Source column takes you to a page with additional disclosure information for the test system. For sources outside our lab, clicking the source name takes you to the original article or review that contains the result.
The second way to access our published results is by visiting the results page for an individual XPRT benchmark. Go the page of the benchmark that interests you, and look for the blue View Results button. Clicking it takes you to a page that displays results for only that benchmark. You can use the free-form filter on the page to filter those results, and can use the Benchmarks drop-down menu to jump to the other individual XPRT results pages.
The third way to view
information in our results database is with the WebXPRT 4 results viewer.
The viewer provides an information-packed, interactive environment in which
users can explore data from the curated set of WebXPRT 4 results we’ve
published on our site. To learn more about the viewer’s capabilities and
features, check out this blog post
from March.
We hope you’ll take
some time to browse the information in our results database. We welcome your feedback
about what you’d like to see in the future and suggestions for improvement. Our
database contains the XPRT scores that we’ve gathered, but we publish them as a
resource for you. Let us know
what you think!
Back in March, we discussed
the WebXPRT 4 results submission process and reminded readers that everyone who
runs a WebXPRT 4
test is welcome to submit scores for us to consider for publication in the WebXPRT 4 results viewer.
Unlike sites that publish every result that users submit, we publish only
results that meet our evaluation criteria. Among other things, scores must be
consistent with general expectations and must include enough detailed system
information to help us assess whether individual scores represent valid test
runs. Today, we offer a couple of tips to increase the likelihood that we will
publish your WebXPRT 4 test results.
Tip 1: Specify your system’s processor
While testers usually include
detailed information for the device, model number, operating system, and
browser version fields, we receive many submissions with little to no information
about the test system’s processor.
In the picture below, you can see an example of the level of detail that we require to consider a submission. We need the full processor name, including the manufacturer and model number (e.g., Intel Core i9-9980HK, AMD Ryzen 3 1300X, or Apple M1 Max). Note that we do not require the processor speed reported by the system.
Tip 2: Include a valid email
address
It is also common for submissions
to not include a valid email address. While we understand the privacy concerns related
to submitting a personal or corporate email address, we need a valid address
that we can use as a point of contact to confirm test-related information when
necessary. We don’t use those addresses for any other purposes, such as selling
them, sharing them with any third parties, or adding them to a mailing list.
We hope this information explains why we might not have published your results. We look forward to receiving your future score submissions. If you have any questions about the submission process, please let us know!
Each month, we send a newsletter to members of the BenchmarkXPRT Development Community. In the newsletter, we recap the latest updates from the XPRT world and provide a summary of the previous month’s XPRT-related activity, including uses or mentions of the XPRTs in the tech press. More people read the weekly XPRT blog than receive the monthly newsletter, so we realized that some blog readers may be unaware of the wide variety of tech outlets that regularly use or mention the XPRTs.
So for today’s blog, we want to give readers a sampling of the XPRT press usage we see on a weekly basis. Recent mentions include:
Tom’s Guide used HDXPRT 4 to compare the performance of the Geekom Mini IT8 and Dell OptiPlex 7090 Ultra small-form-factor PCs.
Intel used WebXPRT 4 test data in promotional material for their line of 12th Gen) Intel Core processors(Alder Lake). Hundreds of press outlets then republished the presentation.
AnandTech used WebXPRT 4 to evaluate the Cincoze DS-1300 Industrial PC.
PCWorld used CrXPRT 2 to provide data for an article listing their top Chromebook recommendations.
TechPowerUp used WebXPRT 3 to compare the browser performance of Intel Core i9-12900KS processor-based systems and other Intel- and AMD processor-based systems.
Other outlets that have published articles, ads, or reviews mentioning the experts in the last few months include: Android Authority, ASUS, BenchLife, Gadgets 360, Good Gear Guide, Hardware.info, Hot Hardware, ITHardware (Poland), ITMedia (Japan), Itndaily (Russia), Mobile01.com (China), Notebookcheck, PCMag, ProClockers, Sohu.com (China), Tom’s Hardware, and Tweakers.
If you don’t currently receive the monthly
BenchmarkXPRT newsletter, but would like to join the mailing list, please let us know! We will not publish or sell any of the contact information you
provide, and will only send the monthly newsletter and occasional benchmark-related
announcements such as patch notifications or new benchmark releases.
Cookie Notice: Our website uses cookies to deliver a smooth experience by storing logins and saving user information. By continuing to use our site, you agree with our usage of cookies as our privacy policy outlines.
