Recently, a tester contacted us with details from a CrXPRT 2 performance test run that they’d successfully completed on… an Apple MacBook Pro! Because CrXPRT 2 is a Chrome Web App that we designed for Chrome OS, it was quite a surprise to hear that it is now possible to run CrXPRT 2 on non-Chrome OS platforms by using FydeOS.
FydeOS is an operating system based on a fork of the Chromium OS project. Developers originally intended FydeOS to be a Google-independent, Chrome-like alternative for the Chinese educational market, but FydeOS is now available to the English-speaking consumer and enterprise markets as well. FydeOS users can run a Chrome-like OS on something other than a Chromebook or a Chromebox, such as a PC, Mac, virtual machine, or even a Raspberry Pi device. Additionally, FydeOS supports Android, Chrome OS, and Linux apps, and users can run those apps at the same time on the same screen.
We have not yet conducted any testing with FydeOS in our lab, but we wanted to pass along this information to any readers who may be interested. If the OS operates as described, it may provide a way for us to experiment with using CrXPRT 2 in some interesting cross-platform tests.
the last pieces in place for the WebXPRT 4 GA, and expect to take the final
build live by this time next week! When we released the WebXPRT 4 Preview and encouraged testers to submit
and publish results, we said we’d try to limit any changes to things that would
not affect test scores. We’re happy to report that we’ve achieved that goal,
and Preview testing results are comparable with GA build results.
If you missed the blog post about the differences between WebXPRT 3 and WebXPRT 4, we
encourage you to check it out. Everything we mentioned about the general and
workload-specific changes in the Preview build holds true for the upcoming GA.
Keep an eye on the blog and WebXPRT.com for more information in the coming week. We look forward to seeing your test results!
The WebXPRT 4 development process is
far enough along that we’d like to share more about changes we are likely to
make and a rough target date for publishing a preview build. While some of the
details below will probably change, this post should give readers a good sense
of what to expect.
Some of the non-workload changes in
WebXPRT 4 relate to our typical benchmark update process, and a few result
directly from feedback we received from the WebXPRT tech press survey.
We will update the aesthetics of the WebXPRT UI to make
WebXPRT 4 visually distinct from older versions. We do not anticipate
significantly changing the flow of the UI.
We will update content in some of the workloads to
reflect changes in everyday technology. For instance, we will upgrade most
of the photos in the photo processing workloads to higher resolutions.
In response to a request from tech press survey
respondents, we are considering adding a looping function to the
We are investigating the possibility of shortening the
benchmark by reducing the default number of iterations from seven to five.
We will only make this change if we can ensure that five iterations produce
consistently low score variance.
Changes to existing workloads
Enhancement. This workload applies three effects
to two photos each (six photos total). It tests HTML5 Canvas, Canvas 2D, and
Organize Album Using AI. This workload currently uses the ConvNetJS neural network library to complete two tasks: (1) organizing five images and (2) classifying the five images in an album. We are planning to replace ConvNetJS with WebAssembly (WASM) for both tasks and are considering upgrading the images to higher resolutions.
Stock Option Pricing. This workload calculates and displays graphic views of a stock portfolio using Canvas, SVG, and dygraph.js. The only change we are considering is combining it with the Sales Graphs workload (below).
Sales Graphs. This workload provides a web-based application displaying multiple views of sales data. Sales Graphs exercises HTML5 Canvas and SVG performance. The only change we are considering is combining it with the Stock Option Pricing workload (above).
Encrypt Notes and OCR Scan. This workload uses ASM.js to sync notes, extract text from a scanned receipt using optical character recognition (OCR), and add the scanned text to a spending report. We are planning to replace ASM.js with WASM for the Notes task and with WASM-based Tesseract for the OCR task.
Online Homework. This workload uses regex, arrays, strings, and Web Workers to review DNA and spell-check an essay. We are not planning to change this workload.
Possible new workloads
Natural Language Processing (NLP). We are considering the addition of an NLP workload using ONNX Runtime and/or TensorFlowJS. The workload would use Bidirectional Encoder Representations from Transformers (BERT) to answer questions about a given text. Similar use cases are becoming more prevalent in conversational bot systems, domain-specific document search tools, and various other educational applications.
Message Scrolling. We are considering developing a new workload that would use an Angular or React.js to scroll through hundreds of messages. We’ll share more about this possible workload as we firm up the details.
The release timeline
We hope to publish a WebXPRT 4
preview build in the second half of November, with a general release before the
end of the year. If it looks as though that timeline will change significantly,
we’ll provide an update here in the blog as soon as possible.
We’re very grateful for all the
input we received during the WebXPRT 4 planning process. If you have any
questions about the details we’ve shared above, please feel free to ask!
In May, we surveyed
longtime WebXPRT users regarding the types of changes they would like to see in
a WebXPRT 4. We sent the survey to journalists at several tech press outlets,
and invited our blog readers to participate as well. We received some very helpful feedback. As we explore new possibilities for WebXPRT 4, we’ve decided to
open an updated version of the survey. We’ve adjusted the questions a bit based
on previous feedback and added some new ones, so we invite you to respond even
if you participated in the original survey.
sites that publish all results they receive, we hand-select results from
internal lab testing, user submissions, and reliable tech media sources. In
each case, we evaluate whether the score is consistent with general expectations.
For sources outside of our lab, that evaluation includes confirming that there
is enough detailed system information to help us determine whether the score
makes sense. We do this for every score on the WebXPRT results page and the
general XPRT results page.
All WebXPRT results we publish automatically appear in the processor comparison
chart as well.
Submitting your score is quick and easy. At the end of the WebXPRT test run, click the Submit your results button below the overall score, complete the short submission form, and click Submit again. The screenshot below shows how the form would look if I submitted a score at the end of a WebXPRT 3 run on my personal system.
After you submit your score, we’ll contact you to confirm how we should display
the source. You can choose one of the following:
Your first and last name
“Independent tester” (for those
who wish to remain anonymous)
Your company’s name, provided
that you have permission to submit the result in their name. To use a
company name, we ask that you provide a valid company email address.
not publish any additional information about you or your company without your
We look forward to seeing your score submissions, and if you have suggestions for the processor chart or any other aspect of the XPRTs, let us know!
Device reviews in publications
such as AnandTech, Notebookcheck, and PCMag, among many others, often feature
WebXPRT test results, and we appreciate the many members of the tech press that
use WebXPRT. As we move forward with the WebXPRT 4 development process, we’re especially
interested in learning what longtime users would like to see in a new version
of the benchmark.
In previous posts,
we’ve asked people to weigh in on the potential addition of a WebAssembly workload or a battery life test. We’d also like to ask experienced testers some other
test-related questions. To that end, this week we’ll be sending a WebXPRT 4
survey directly to members of the tech press who frequently publish WebXPRT
Regardless of whether you are a member of the tech press, we invite you to participate by sending your answers to any or all the questions below to firstname.lastname@example.org. We ask you to do so by the end of May.
Do you think WebXPRT 3’s selection of workload scenarios is representative of modern web tasks?
How do you think WebXPRT compares to other common browser-based benchmarks, such as JetStream, Speedometer, and Octane?
Are there web technologies that you’d like us to include in additional workloads?
Are you happy with the WebXPRT 3 user interface? If not, what UI changes would you like to see?
Are there any aspects of WebXPRT 2015 that we changed in WebXPRT 3 that you’d like to see us change back?
Have you ever experienced significant connection issues when testing with WebXPRT?
Given the array of workloads, do you think the WebXPRT runtime is reasonable? Would you mind if the average runtime were a bit longer?
Are there any other aspects of WebXPRT 3 that you’d like to see us change?
If you’d like to discuss any topics
that we did not cover in the questions above, please feel free to include additional
comments in your response. We look forward to hearing your thoughts!