Tag Archives: browsers

Investigating the possibility of WebXPRT user accounts

on September 2, 2021

One of our goals during the ongoing WebXPRT 4 development process is to be as responsive as possible to user feedback, and we want to emphasize that it’s not too late to send us your ideas. Until we finalize the details for each workload and complete the code work for the preview build, we still have quite a bit of flexibility around adding new features.

Just this week, a community member raised the possibility of a WebXPRT 4 feature that would enable user-specific test ID numbers or accounts. One possible implementation of the idea would allow a user to sign up for a WebXPRT test account as an individual or on behalf of their organization. The test accounts would be both free and optional; you could continue to run the benchmark without an account, but running it with an account would let you save and view your test history. Another implementation option we are considering would let users generate a permanent user ID number for themselves or their organization. They could then use that number to tag and search for their automated test runs in our database, without having to log into an account.

Our biggest question at the moment is whether our user base would be interested in WebXPRT user accounts or test IDs. If this concept piques your interest, or you have suggestions for implementation, please let us know!

Justin

Posted in Automation, benchmark, Benchmarking, Browser-based benchmarks, Collaborative benchmark development, Performance benchmarking, Web-based testing, WebXPRT, webxprt 4 | Also tagged BenchmarkXPRT Development Community, user accounts, WebXPRT, WebXPRT 4

Round 2 of the WebXPRT 4 survey is now open

By Justin Greene

on July 22, 2021

In May, we surveyed longtime WebXPRT users regarding the types of changes they would like to see in a WebXPRT 4. We sent the survey to journalists at several tech press outlets, and invited our blog readers to participate as well. We received some very helpful feedback. As we explore new possibilities for WebXPRT 4, we’ve decided to open an updated version of the survey. We’ve adjusted the questions a bit based on previous feedback and added some new ones, so we invite you to respond even if you participated in the original survey.

To do so, please send your answers to the following questions to benchmarkxprtsupport@principledtechnologies.com before July 31.

Do you think WebXPRT 3’s selection of workload scenarios is representative of modern web tasks?
How do you think WebXPRT compares to other common browser-based benchmarks, such as JetStream, Speedometer, and Octane?
Would you like to see a workload based on WebAssembly (WASM) in WebXPRT 4? Why or why not?
Would you like to see a workload based on Single Page Application (SPA) technology in WebXPRT 4? Why or why not?
Would you like to see a workload based on Motion UI in WebXPRT 4? Why or why not?
Would you like to see us include any other web technologies in additional workloads?
Are you happy with the WebXPRT 3 user interface? If not, what UI changes would you like to see?
Have you ever experienced significant connection issues when testing with WebXPRT?
Given its array of workloads, do you think the WebXPRT runtime is reasonable? Would you mind if the average runtime increased slightly?
Would you like to see us change any other aspects of WebXPRT 3?

If you would like to share your thoughts on any topics that the questions above do not cover, please include those in your response. We look forward to hearing from you!

Justin

Posted in benchmark, Benchmarking, BenchmarkXPRT, Browser-based benchmarks, Collaborative benchmark development, Future of performance evaluation, JetStream, Motion UI, Octane, Single Page Application, Speedometer, Web-based testing, WebAssembly, WebXPRT, WebXPRT 3, webxprt 4, What makes a good benchmark? | Also tagged cross-platform, Motion UI, Single Page Application, survey, WebAssembly, WebXPRT, WebXPRT 3, WebXPRT 4

How to submit WebXPRT results for publication

By Justin Greene

on June 24, 2021

It’s been a while since we last discussed the process for submitting WebXPRT results to be considered for publication in the WebXPRT results browser and the WebXPRT Processor Comparison Chart, so we thought we’d offer a refresher.

Unlike sites that publish all results they receive, we hand-select results from internal lab testing, user submissions, and reliable tech media sources. In each case, we evaluate whether the score is consistent with general expectations. For sources outside of our lab, that evaluation includes confirming that there is enough detailed system information to help us determine whether the score makes sense. We do this for every score on the WebXPRT results page and the general XPRT results page. All WebXPRT results we publish automatically appear in the processor comparison chart as well.

Submitting your score is quick and easy. At the end of the WebXPRT test run, click the Submit your results button below the overall score, complete the short submission form, and click Submit again. The screenshot below shows how the form would look if I submitted a score at the end of a WebXPRT 3 run on my personal system.

After you submit your score, we’ll contact you to confirm how we should display the source. You can choose one of the following:

Your first and last name
“Independent tester” (for those who wish to remain anonymous)
Your company’s name, provided that you have permission to submit the result in their name. To use a company name, we ask that you provide a valid company email address.

We will not publish any additional information about you or your company without your permission.

We look forward to seeing your score submissions, and if you have suggestions for the processor chart or any other aspect of the XPRTs, let us know!

Justin

Posted in benchmark, Benchmark metrics, Benchmarking, Browser-based benchmarks, Cross-platform benchmarks, Performance of computing devices, results, results submission, WebXPRT, WebXPRT 3, WebXPRT Processor Comparison Chart | Also tagged cross-platform, results, results submission, test results, WebXPRT, WebXPRT 3

Thinking ahead to WebXPRT 4

By Justin Greene

on March 5, 2020

It’s been about two years since we released WebXPRT 3, and we’re starting to think about the WebXPRT 4 development cycle. With over 529,000 runs to date, WebXPRT continues to be our most popular benchmark because it’s quick and easy to run, it runs on almost anything with a web browser, and it evaluates performance using the types of web technologies that many people use every day.

For each new version of WebXPRT, we start the development process by looking at browser trends and analyzing the feasibility of incorporating new web technologies into our workload scenarios. For example, in WebXPRT 3, we updated the Organize Album workload to include an image-classification task that uses deep learning. We also added an optical character recognition task to the Encrypt Notes and OCR scan workload, and introduced a new Online Homework workload that combined part of the DNA Sequence Analysis scenario with a writing sample/spell check scenario.

Here are the current WebXPRT 3 workloads:

Photo Enhancement: Applies three effects, each using Canvas, to two photos.
Organize Album Using AI: Detects faces and classifies images using the ConvNetJS neural network library.
Stock Option Pricing: Calculates and displays graphic views of a stock portfolio using Canvas, SVG, and dygraphs.js.
Encrypt Notes and OCR Scan: Encrypts notes in local storage and scans a receipt using optical character recognition.
Sales Graphs: Calculates and displays multiple views of sales data using InfoVis and d3.js.
Online Homework: Performs science and English assignment tasks using Web Workers and Typo.js spell check.

What new technologies or workload scenarios should we add? Are there any existing features we should remove? Would you be interested in an associated battery life test? We want to hear your thoughts and ideas about WebXPRT, so please tell us what you think!

Justin

Posted in AI, benchmark, Benchmark metrics, Benchmarking, BenchmarkXPRT, Browser-based benchmarks, Collaborative benchmark development, Cross-platform benchmarks, image classification, JavaScript, Performance benchmarking, WebXPRT, WebXPRT 3, What makes a good benchmark? | Also tagged benchmark, WebXPRT, WebXPRT 3

The Exploring WebXPRT 3 white paper is now available

By Justin Greene

on July 19, 2018

Today, we published the Exploring WebXPRT 3 white paper. The paper describes the differences between WebXPRT 3 and WebXPRT 2015, including changes we made to the harness and the structure of the six performance test workloads. We also explain the benchmark’s scoring methodology, how to automate tests, and how to submit results for publication. Readers will also find additional detail about the third-party functions and libraries that WebXPRT uses during the HTML5 capability checks and performance workloads.

Because data collection and privacy concerns are more relevant than ever, we also discuss the WebXPRT data collection mechanisms and our commitment to respecting testers’ privacy. Finally, for readers who may be unfamiliar with the XPRTs, we describe the other benchmark tools in the XPRT family, the role of the BenchmarkXPRT Development Community, and how you can contribute to the XPRTs.

Along with the WebXPRT 3 results calculation white paper and spreadsheet, the Exploring WebXPRT 3 white paper is designed to promote the high level of transparency and disclosure that is a core value of the BenchmarkXPRT Development Community. Both WebXPRT white papers and the results calculation spreadsheet are available on WebXPRT.com and on our XPRT white papers page. If you have any questions about the WebXPRT, please let us know, and be sure to check out our other XPRT white papers.

Justin

Posted in WebXPRT, WebXPRT 2015, WebXPRT 3, White papers | Also tagged benchmarks, BenchmarkXPRT, WebXPRT, WebXPRT 2015, WebXPRT 3, white paper

WebXPRT and user-agent strings

By Justin Greene

on August 31, 2017

After running WebXPRT in Microsoft Edge, a tester recently asked why the browser information field on the results page displayed “Chrome 52 – Edge 15.15063.” It’s a good question; why would the benchmark report Chrome 52 when Microsoft Edge is the browser under test? The answer lies in understanding user-agent strings and the way that WebXPRT gathers specific bits of information.

When browsers request a web page from a hosting server, they send an array of basic header information that allows the server to determine the client’s capabilities and the best way to provide the requested content. One of these headers, the user-agent, presents a string of tokens that provide information about the application making the request, the operating system and version, rendering engine compatibility, and browser platform details. In effect, the user-agent string is a way for a browser to tell the hosting server the full extent of its capabilities.

When WebXPRT attempts to identify a browser, it references the browser token in the user-agent string.

The process is generally straightforward, but in some cases, browsers spoof information from other browsers in their user-agent strings, which makes accurate browser detection difficult. The reasons for this are complex, but they involve web development practices and the fact that some web pages are not designed to recognize and work well with new or less-popular browsers. When we released WebXPRT 2015, Microsoft Edge was new. The Edge team wanted to make sure that as much advanced web content as possible would be available to Edge users, so they created a user-agent string that declared itself to be several different browsers at once.

I can see this in action if I check Edge’s user-agent string on my system. Currently, it reports “Mozilla/5.0 (Windows NT 10.0; Win64; x64; ServiceUI 9) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/52.0.2743.116 Safari/537.36 Edge/15.15063.” So, because of the way Edge’s user-agent string is constructed, and the way WebXPRT parses that information, the browser information field on WebXPRT’s results page will report “Chrome 52 – Edge 15.15063” on my system.

To try this on your system, in Edge, select the ellipses icon in the top right-hand corner, then F12 Developer Tools. Next, select the Console tab, and run “javascript:alert(navigator.userAgent).” A popup window will display the UA string.

You can find instructions for finding the user-agent string in other browsers here: http://techdows.com/2016/07/edge-ie-chrome-firefox-user-agent-strings.html.

In the next version of WebXPRT, we’ll work to refine the way that the test parses user-agent strings, and provide more accurate system information for testers. If you have any questions or suggestions regarding this topic, let us know!

Justin

Posted in Browser-based benchmarks, Google Chrome, Microsoft Edge, WebXPRT, WebXPRT 2015 | Also tagged Chrome, Microsoft Edge, WebXPRT

Tag Archives: browsers

Investigating the possibility of WebXPRT user accounts

Round 2 of the WebXPRT 4 survey is now open

How to submit WebXPRT results for publication

Thinking ahead to WebXPRT 4

The Exploring WebXPRT 3 white paper is now available

WebXPRT and user-agent strings

Check out the other XPRTs: