BenchmarkXPRT Development Community Blog

WebXPRT 3 arrives next week!

on February 22, 2018

After much development work and testing, we’re happy to report that we’ll be releasing WebXPRT 3 early next week!

Here are the final workload names and descriptions.

1) Photo Enhancement: Applies three effects to two photos each using Canvas.
2) Organize Album Using AI: Detects faces and classifies images using the ConvNetJS neural network library.
3) Stock Option Pricing: Calculates and displays graphics views of a stock portfolio using Canvas, SVG, and dygraphs.js.
4) Encrypt Notes and OCR Scan: Encrypts notes in local storage and scans a receipt using optical character recognition.
5) Sales Graphs: Calculates and displays multiple views of sales data using InfoVis and d3.js.
6) Online Homework: Performs science and English homework using Web Workers and Typo.js spell check.

As we mentioned in an earlier blog post, the updated photo workloads contain new images and a deep learning task. We also added an optical character recognition task to the Local Notes workload and combined part of the DNA Sequence Analysis scenario with a writing sample/spell check scenario to simulate online homework in the new Online Homework workload.

Longtime WebXPRT users will immediately notice a completely new UI. We worked to improve the UI’s appearance on smaller devices such as phones and we think testers will find it easier to navigate.

Testers can still choose to run individual workloads and we’re once again offering English, German, and Simplified Chinese language options.

Below my sig, I’ve included pictures of WebXPRT 3’s start test and results pages, as well as an in-test screen capture.

We’re thankful for all the interest in WebXPRT 3 so far and believe the new version of WebXPRT will be as relevant and reliable as WebXPRT 2013 and 2015—and easier to use. We look forward to seeing new results submissions next week!

Justin

Posted in BenchmarkXPRT development community, Browser-based benchmarks, Collaborative benchmark development, Cross-platform benchmarks, Performance benchmarking, WebXPRT, WebXPRT 2015, WebXPRT 3 |

A smooth transition

By Justin Greene

on February 15, 2018

We want to thank Andrei Frumusanu of AnandTech for mentioning WebXPRT 3 in the System Performance section of their Snapdragon 845 review. For testing labs and tech media, incorporating a new benchmark into a test suite can be daunting, and they don’t make the decision to do so lightly. Once a new benchmark is in play, the score database used for comparisons is suddenly empty, and a lot of testing needs to happen before anyone can compare devices on a large scale.

In the BenchmarkXPRT Development Community, we’ve designed our development and release system to minimize the stress involved in adopting new benchmark tools. A key part of that strategy is releasing community previews to members several weeks before the general release. When we release a community preview, we include no publication restrictions and we work to make sure that preview results will be comparable to results from the general release. Between a community preview and a general release, we may still tweak the UI or fix issues with non-workload-related features, but you can be sure that the results will still be good after the general release.

The community preview system allows us to solicit feedback from an expanded base of pre-release testers, but it also allows labs to backfill results for legacy devices and get a head start on incorporating the new benchmark into their testing suites.

Speaking of previews, WebXPRT 3 community preview testing is going well and we’re excited about the upcoming release. If you’d like to learn more about our development community and how you can join, send us your questions and we’ll be happy to help.

Justin

Posted in Benchmarking, BenchmarkXPRT development community, Browser-based benchmarks, Collaborative benchmark development, Community Preview, WebXPRT, WebXPRT 3 |

WebXPRT 3, Mobile World Congress, and the next HDXPRT

By Justin Greene

on February 8, 2018

We’re excited about everything that’s in store for the XPRTs, and we want to update community members on what to expect in the next few months.

The next major development is likely to be the WebXPRT 3 general release. We’re currently refining the UI and conducting extensive testing with the community preview build. We’re not ready to announce a firm release date, but hope to do so over the next few weeks. Please try the community preview and give us your feedback, if you haven’t already.

During the last week of February, Mark will be at Mobile World Congress (MWC) in Barcelona. Each year, MWC offers a great opportunity to examine the new trends and technologies that will shape mobile technology in the years to come. We look forward to sharing Mark’s thoughts on this year’s hot topics. Will you be attending MWC this year? If so, let us know!

In addition, we’re hoping to have a community preview of the next HDXPRT ready in the spring. As we mentioned a few months ago, we’re updating the workloads, applications, and UI. For the converting photos scenario, we’re considering incorporating new Adobe Photoshop tools such as the “Open Closed Eyes” feature and an automatic fix for pictures that are out of focus due to handheld camera shake. For the converting videos scenario, we’re including 4K GoPro footage that represents the quality of video captured by today’s “prosumer” demographic.

What features would you like to see in the next HDXPRT? Let us know!

Justin

Posted in BenchmarkXPRT, BenchmarkXPRT development community, Browser-based benchmarks, Collaborative benchmark development, Community Preview, Future of performance evaluation, HDXPRT, HDXPRT release cycle, Mobile devices, Mobile World Congress, Mobile World Congress, Performance benchmarking, Trade Shows, WebXPRT, WebXPRT 3 | Tagged HDXPRT, Mobile World Congress, MWC, WebXPRT |

New features for the WebXPRT Processor Comparison Chart

By Justin Greene

on February 1, 2018

We’re excited that the WebXPRT 2015 Processor Comparison Chart is becoming a valuable resource for community members and visitors interested in comparing web-browsing performance scores. When we introduced the chart, we mentioned that the first iteration was only the beginning. We planned to add additional capabilities on a regular basis, and today we’re happy to announce some new features.

The chart now offers a drop-down menu for displaying more than 20 results per page. (You can choose 20, 50, 100, or all.) When you click on a system to view the details of its score, you will see that we added a column displaying browser information. If the “Browser” column entry shows as “Unknown,” this means we can’t find browser information for a specific test result.

We’re also working on adding a drop-down menu for filtering scores by device type (laptop, tablet, etc.).

We hope these features will prove to be helpful. We want the chart to continue to be a great asset for its users, so if you have ideas for features and types of data you’d like to see, let us know!

Justin

Posted in BenchmarkXPRT, BenchmarkXPRT development community, Browser-based benchmarks, Cross-platform benchmarks, Performance benchmarking |

A look back at 2017

By Justin Greene

on January 25, 2018

At the beginning of each new year, we like to look back on the previous 12 months and review everything that we accomplished. Here’s a quick recap of an eventful 2017 for the XPRTs:

We continued our tradition of travelling to the world’s biggest tech expos. Bill went to CES in Las Vegas and MWC Shanghai, and Mark attended MWC in Barcelona. Travelling to these shows provides us with great opportunities to monitor industry trends and meet with other BenchmarkXPRT Development Community members.

We also continued work on our suite of benchmark tools and related resources. We updated BatteryXPRT in response to changes in the Android development environment, released two new HDXPRT builds to make installation and test configuration easier on new Windows 10 builds, and released the much-anticipated WebXPRT 3 Community Preview.

We released new media, including a video about our sponsorship of a team of North Carolina State University students who created an experimental VR demo workload, and new interactive tools, such as the XPRT Selector tool and the WebXPRT Processor Comparison Chart.

We continued to improve the XPRT Weekly Tech Spotlight by adding more devices, photos, and site features. We put 51 devices in the spotlight throughout the year and published updated back-to-school, Black Friday, and holiday showcases to help buyers compare devices.

At the end of the year, our most popular benchmark, WebXPRT, passed the 200,000-run milestone. WebXPRT use continues to grow around the world, and it has truly become an industry-standard performance benchmark for OEM labs, vendors, and leading tech press outlets.

We also continued work on our most challenging project to date, a benchmark for machine learning. We look forward to sharing more information on that effort with the community over the next few months.

We’re thankful for everyone who used the XPRTs, joined the community, and sent in questions and suggestions throughout 2017. Each one of you helped to make the BenchmarkXPRT Development Community a strong, vibrant, and relevant resource for people all around the world. Here’s to a great 2018!

Justin

Posted in Uncategorized |

Here’s to 100 more!

By Justin Greene

on January 18, 2018

This week’s Essential Phone entry marks the 100th device that we’ve featured in the XPRT Weekly Tech Spotlight! It’s a notable milestone for us as we work toward our goal of building a substantial library of device information that buyers can use to compare devices. In celebration, I thought it would be fun to share some Spotlight-related stats.

Our first Spotlight entry was the Google Pixel C way back on February 8, 2016, and we’ve featured a wide array of devices since then:

33 phones
16 laptops
16 tablets
16 2-in-1s
6 small-form-factor PCs
5 desktops
5 game consoles
3 all-in-ones

In addition to a wide variety of device types, we try to include a wide range of vendors. So far, we’ve featured devices from Acer, Alcatel, Alienware, Amazon, Apple, ASUS, BLU, CHUWI, Dell, Essential, Fujitsu, Google, HP, HTC, Huawei, Intel, LeEco, Lenovo, LG, Microsoft, NVIDIA, OnePlus, Razer, Samsung, Sony, Syber, Xiaomi, and ZTE. We look forward to adding many more to that list during the year ahead.

XPRT Spotlight is a great way for device vendors and manufacturers to share PT-verified specs and test results with buyers around the world. If you’re interested in sending in a device for testing, please contact XPRTSpotlight@PrincipledTechnologies.com.

There’s a lot more to come for XPRT Spotlight, and we’re constantly working on new features and improvements for the page. Are there any specific devices or features that you would like to see in the Spotlight? Let us know.

Justin

Posted in Performance benchmarking, Performance of computing devices, Performance testing on tablets, XPRT Weekly Tech Spotlight |

WebXPRT 3 arrives next week!

A smooth transition

WebXPRT 3, Mobile World Congress, and the next HDXPRT

New features for the WebXPRT Processor Comparison Chart

A look back at 2017

Here’s to 100 more!

Check out the other XPRTs: