This week, we published the Exploring WebXPRT 4 white paper. It describes the design and structure of WebXPRT 4, including detailed information about the benchmark’s harness, HTML5 and WebAssembly (WASM) capability checks, and changes we’ve made to the structure of the performance test workloads. We explain the benchmark’s scoring methodology, how to automate tests, and how to submit results for publication. The white paper also includes information about the third-party functions and libraries that WebXPRT 4 uses during the HTML5 and WASM capability checks and performance workloads.
The Exploring WebXPRT 4 white paper promotes
the high level of transparency and disclosure that is a core value of the
BenchmarkXPRT Development Community. We’ve always believed that transparency
builds trust, and trust is essential for a healthy benchmarking community.
That’s why we involve community members in the benchmark development process
and disclose how we build our benchmarks and how they work.
Recently,
Tom’s Guide published an interesting article
about how they used ChromeOS Flex to turn a
ten-year-old Apple MacBook Pro into a functioning Chromebook by replacing the
laptop’s macOS operating system with ChromeOS. ChromeOS Flex is a free Google
tool that allows users to create a bootable USB drive that they can then use to
install ChromeOS on a wide variety
of hardware platforms that traditionally run other operating systems such as
macOS or Windows. Because ChromeOS is a cloud-first, relatively low-overhead
operating system, the ChromeOS Flex option could breathe new life into an old
laptop that you have lying around.
Never having encountered a MacBook Pro with ChromeOS, we were interested to learn about Tom’s experience running XPRT benchmarks in this new environment.WebXPRT 4, WebXPRT 3, and the CrXPRT 2 performance test apparently ran without any issues, but we have not yet seen a CrXPRT 2 battery life result from a ChromeOS Flex environment. We plan to experiment with this soon.
We
were happy to publish the results on our site, and will consider any ChromeOS
Flex results we receive for publication. If you submit results from ChromeOS
Flex testing, we ask that you use the “Additional information” field in the
results submission form to clarify that you ran the tests in a ChromeOS Flex
environment. This will prevent any possible confusion when we see a submission
that lists a traditional macOS or Windows hardware platform along with a
ChromeOS version number.
Do you have experience running CrXPRT or WebXPRT with ChromeOS Flex? We’d love to hear about it!
The new school year is
upon us, and learners of all ages are looking for tech devices that have the
capabilities they will need in the coming year. The tech marketplace can be
confusing, and competing claims can be hard to navigate. The XPRTs are here to
help! Whether you’re shopping for a new phone, tablet, Chromebook, laptop, or
desktop, the XPRTs can provide reliable, industry-trusted performance scores
that can cut through all the noise.
A good place to start looking
for scores is the WebXPRT 4 results viewer. The viewer displays WebXPRT 4 scores from
over 175 devices—including many hot new releases—and we’re adding new scores
all the time. To learn more about the viewer’s capabilities and how you can use
it to compare devices, check out this blog post.
Another resource we
offer is the XPRT results browser. The browser is the most efficient way to access the XPRT
results database, which currently holds more than 3,000 test results from over 120
sources, including major tech review publications around the world, OEMs, and
independent testers. It offers a wealth of current and historical performance
data across all of the XPRT benchmarks and hundreds of devices. You can read
more about how to use the results browser here.
Also, if you’re considering a popular device, chances are good that a recent tech review includes an XPRT score for that device. Two quick ways to find these reviews: (1) go to your favorite tech review site and search for “XPRT” and (2) go to a search engine and enter the device name and XPRT name (e.g., “Apple MacBook Air” and “WebXPRT”). Here are a few recent tech reviews that use one of the XPRTs to evaluate a popular device:
The XPRTs can help consumers make better-informed and more confident tech purchases. As this school year begins, we hope you’ll find the data you need on our site or in an XPRT-related tech review. If you have any questions about the XPRTs, XPRT scores, or the results database please feel free to ask!
Testers
new to the XPRT benchmarks may not know about one of the free resources we
offer. The XPRT results database currently holds more than 3,000 test results
from over 120 sources, including major tech review publications around the
world, OEMs, and independent testers. It offers a wealth of current and
historical performance data across all the XPRT benchmarks and hundreds of
devices.
We update the results
database several times a week, adding selected results from our own internal
lab testing, reliable tech media sources, and end-of-test user submissions.
(After you run one of the XPRTs, you can choose to submit the results, but they
don’t automatically appear in the database.) Before adding a result, we
evaluate whether the score makes sense and is consistent with general
expectations, which we can do only when we have sufficient system information details.
For that reason, we ask testers to disclose as much hardware and software
information as possible when publishing or submitting a result.
We encourage visitors to our site to explore the XPRT results database. There are three primary ways to do so. The first is by visiting the main BenchmarkXPRT results browser, which displays results entries for all of the XPRT benchmarks in chronological order (see the screenshot below). You can narrow the results by selecting a benchmark from the drop-down menu and can type values, such as vendor or the name of a tech publication, into the free-form filter field. For results we’ve produced in our lab, clicking “PT” in the Source column takes you to a page with additional disclosure information for the test system. For sources outside our lab, clicking the source name takes you to the original article or review that contains the result.
The second way to access our published results is by visiting the results page for an individual XPRT benchmark. Go the page of the benchmark that interests you, and look for the blue View Results button. Clicking it takes you to a page that displays results for only that benchmark. You can use the free-form filter on the page to filter those results, and can use the Benchmarks drop-down menu to jump to the other individual XPRT results pages.
The third way to view
information in our results database is with the WebXPRT 4 results viewer.
The viewer provides an information-packed, interactive environment in which
users can explore data from the curated set of WebXPRT 4 results we’ve
published on our site. To learn more about the viewer’s capabilities and
features, check out this blog post
from March.
We hope you’ll take
some time to browse the information in our results database. We welcome your feedback
about what you’d like to see in the future and suggestions for improvement. Our
database contains the XPRT scores that we’ve gathered, but we publish them as a
resource for you. Let us know
what you think!
Back in March, we discussed
the WebXPRT 4 results submission process and reminded readers that everyone who
runs a WebXPRT 4
test is welcome to submit scores for us to consider for publication in the WebXPRT 4 results viewer.
Unlike sites that publish every result that users submit, we publish only
results that meet our evaluation criteria. Among other things, scores must be
consistent with general expectations and must include enough detailed system
information to help us assess whether individual scores represent valid test
runs. Today, we offer a couple of tips to increase the likelihood that we will
publish your WebXPRT 4 test results.
Tip 1: Specify your system’s processor
While testers usually include
detailed information for the device, model number, operating system, and
browser version fields, we receive many submissions with little to no information
about the test system’s processor.
In the picture below, you can see an example of the level of detail that we require to consider a submission. We need the full processor name, including the manufacturer and model number (e.g., Intel Core i9-9980HK, AMD Ryzen 3 1300X, or Apple M1 Max). Note that we do not require the processor speed reported by the system.
Tip 2: Include a valid email
address
It is also common for submissions
to not include a valid email address. While we understand the privacy concerns related
to submitting a personal or corporate email address, we need a valid address
that we can use as a point of contact to confirm test-related information when
necessary. We don’t use those addresses for any other purposes, such as selling
them, sharing them with any third parties, or adding them to a mailing list.
We hope this information explains why we might not have published your results. We look forward to receiving your future score submissions. If you have any questions about the submission process, please let us know!
WebXPRT 4 has been available to testers since
the end of December, and we’re excited to see that the benchmark is already
gaining significant traction in the tech press and testing communities. Several
tech publications have already published reviews that feature WebXPRT results,
and the number of WebXPRT 4 runs is growing by about fifty percent each month, more
than twice the rate of growth for WebXPRT 3 after launch.
As WebXPRT 4 use continues to grow,
and more tech publications and OEM labs add WebXPRT 4 to their benchmark
suites, we encourage you to keep an eye on the WebXPRT 4 results viewer.
The viewer currently has about 120 test results, and we’ll continue to populate
the viewer with the latest PT-curated WebXPRT 4
results each week.
You don’t have to be a tech
journalist to publish a WebXPRT 4 result, however. We publish any results—including
individual user submissions—that meet our evaluation criteria. To submit a result
for publication consideration, simply follow the straightforward submission instructions
after the test completes. Scores must be consistent with general expectations and
must include enough detailed system information that we can determine whether
the score makes sense. If you’ve tested with WebXPRT 4 on a new device, or any
device or device configuration that’s not already present in the results
viewer, we encourage you to send in the result. We want to hear from you!
Cookie Notice: Our website uses cookies to deliver a smooth experience by storing logins and saving user information. By continuing to use our site, you agree with our usage of cookies as our privacy policy outlines.
