Category: Performance benchmarking

Planning for the next TouchXPRT

on August 29, 2019

We’re in the very early planning stages for the next version of TouchXPRT, and we’d love to hear any suggestions you may have. What do you like or dislike about TouchXPRT? What features do you hope to see in a new version?

For those who are unfamiliar with TouchXPRT, it’s a benchmark for evaluating the performance of Windows 10 devices. TouchXPRT 2016, the most recent version, runs tests based on five everyday scenarios (Beautify Photos, Blend Photos, Convert Videos for Sharing, Create Music Podcast, and Create Slideshow from Photos) and produces results for each of the five scenarios plus an overall score. The benchmark is available two ways: as a Universal Windows App in the Microsoft Store and as a sideload installer package on TouchXPRT.com.

When we begin work on a new version of any benchmark, one of the first steps we take is to assess its workloads to determine whether they will provide value during the years ahead. This step involves evaluating whether to update test content such as photos and videos to more contemporary file resolutions and sizes, and can also involve removing workloads or adding completely new ones. Should we keep the TouchXPRT workloads listed above or investigate other use cases? Should we research potential AI-related workloads? What do you think?

As we did with MobileXPRT 3 and HDXPRT 4 earlier this year, we’re also planning to update the TouchXPRT UI to improve the look of the benchmark and make it easier to use. We’re just at the beginning of this process, so any feedback you send has a chance to really shape the future of the benchmark.

On a related note, TouchXPRT 2016 testers who use the installer package available on TouchXPRT.com may have noticed that the package has a new file name (TX2016.6.52.0_8.19.19.zip). Microsoft requires developers to assign a security certificate to all sideload apps, and the new TouchXPRT file contains a refreshed certificate. We did not change the benchmark in any other way, so scores from this package are comparable to previous TouchXPRT 2016 scores.

Justin

Posted in Benchmarking, BenchmarkXPRT, BenchmarkXPRT development community, Collaborative benchmark development, Future of performance evaluation, HDXPRT 4, MobileXPRT 3, Performance benchmarking, TouchXPRT, TouchXPRT 2016, What makes a good benchmark?, Windows, Windows 10 | Tagged benchmark, Microsoft, TouchXPRT, Windows, Windows 10 |

The 2019 XPRT Spotlight Back-to-School Roundup

By Justin Greene

on August 15, 2019

With the new school year approaching, we’re pleased to announce that we’ve just published our fourth annual XPRT Spotlight Back-to-School Roundup! The Roundup allows shoppers to view side-by-side comparisons of XPRT test scores and hardware specs from some of this year’s most popular Chromebooks, laptops, tablets, and convertibles. After testing the devices in our lab using XPRT benchmarks, we’ve provided performance scores as well as photo galleries, PT-verified device specs, and prices. Parents, teachers, students, and administrators who are considering buying devices to use in their education environments have many options. The Roundup helps make their decisions easier by gathering product and performance facts in one convenient place.

The Back-to-School Roundup is just one of the features we offer through the XPRT Weekly Tech Spotlight. Every week, the Spotlight highlights a new device, making it easier for consumers to select a new laptop, phone, tablet, or PC. Recent devices in the Spotlight include the Dell G7 15 Gaming laptop, the HP Stream 14, the ASUS Chromebook Flip, the OnePlus 7 Pro phone, and the 2019 Apple iPad Mini. The Spotlight device comparison page lets you view side-by-side comparisons of all of the devices we’ve tested.

If you’re interested in having your devices featured in the XPRT Weekly Tech Spotlight or in this year’s Black Friday and Holiday Showcases, which we publish in late November, visit the website for more details.

If you have any ideas for the Spotlight page or suggestions for devices you’d like to see, let us know!

Justin

Posted in Apple, benchmark, BenchmarkXPRT, BenchmarkXPRT development community, Dell, Education, Performance benchmarking, XPRT Weekly Tech Spotlight | Tagged Apple, ASUS, Back-to-School, Black Friday, Chromebooks, Dell, education, HP, OnePlus, PC, Spotlight, XPRT Weekly Tech Spotlight |

A new HDXPRT 4 build is available!

By Justin Greene

on July 18, 2019

A few weeks ago, we announced that a new HDXPRT 4 build, v1.1, was on the way. This past Monday, we published the build on HDXPRT.com.

The new build includes an updated version of HandBrake, the commercial application that HDXPRT uses for certain video conversion tasks. HandBrake 1.2.2 supports hardware acceleration with AMD Video Coding Engine (VCE), Intel Quick Sync, and the NVIDIA video encoder (NVENC). By default, HDXPRT4 v1.1 uses the encoder available through a system’s integrated graphics, but testers can target discrete graphics by changing a configuration file flag before running the benchmark. HDXPRT will then use the encoder provided by the discrete graphics hardware. This configuration setting takes effect only when more than one of the supported encoders (VCE, QSV, or NVENC) is present on the system.

As we mentioned before, in all other respects, the benchmark has not changed. That means that, apart from a scenario where a tester changes the targeted graphics hardware, scores from previous HDXPRT 4 builds will be comparable to those from the new build.

The updated HDXPRT 4 User Manual contains additional information and instructions for changing the configuration file flag. Please contact us if you have any questions about the new build. Happy testing!

Justin

Posted in AMD, benchmark, BenchmarkXPRT development community, graphics, HDXPRT, HDXPRT 4, NVIDIA, Performance benchmarking, Windows 10 | Tagged AMD, graphics, HDXPRT, HDXPRT 4, Intel, NVENC, NVIDIA, QSV, VCE, Windows |

WebXPRT: What would you like to see?

By Justin Greene

on July 11, 2019

At over 412,000 runs and counting, WebXPRT is our most popular benchmark. From the first release in 2013, it’s been popular with device manufacturers, developers, tech journalists, and consumers because it’s easy to run, it runs on almost anything with a web browser, and it evaluates device performance using the types of web-based tasks that people are likely to encounter on a daily basis.

With each new version of WebXPRT, we analyze browser development trends to make sure the test’s underlying web technologies and workload scenarios adequately reflect the ways people are using their browsers to work and play. BenchmarkXPRT Development Community members can play an important part in that process by sending us feedback on existing tests and suggestions for new workloads to include.

For example, when we released WebXPRT 3, we updated the photo workloads with new images and a deep learning task used for image classification. We also added an optical character recognition task in the Encrypt Notes and OCR scan workload, and combined part of the DNA Sequence Analysis scenario with a writing sample/spell check scenario to simulate online homework in an all-new Online Homework workload.

Consider for a moment what an ideal future version of WebXPRT would look like for you. Are there new web technologies or workload scenarios that you would like to see? Would you be interested in an associated battery life test? Should we include experimental tests? We’re interested in what you have to say, so please feel free to contact us with your thoughts or questions.

If you’re just now learning about WebXPRT, we offer several resources to help you better understand the benchmark and its range of uses. For a general overview of why WebXPRT matters, watch our video titled What is WebXPRT and why should I care? To read more about the details of the benchmark’s development and structure, check out the Exploring WebXPRT 3 white paper. To see WebXPRT 2015 and WebXPRT 3 scores from a wide range of processors, visit the WebXPRT 3 Processor Comparison Chart.

We look forward to hearing from you!

Justin

Posted in BenchmarkXPRT, BenchmarkXPRT development community, Browser-based benchmarks, Collaborative benchmark development, Cross-platform benchmarks, Future of performance evaluation, JavaScript, Performance benchmarking, Web-based testing, WebXPRT, WebXPRT 2013, WebXPRT 2015, WebXPRT 3, WebXPRT Processor Comparison Chart, What makes a good benchmark?, White papers | Tagged benchmark, browser, OCR, Performance, web, WebXPRT |

An updated HDXPRT 4 build is on the way

By Justin Greene

on July 3, 2019

HandBrake recently released a new version, v1.2.2, of their video conversion software. Among other improvements, the new version includes support for certain AMD (VCE) and NVIDIA (NVENC) hardware-accelerated video encoders. Because we include HandBrake as one of the commercial applications in the HDXPRT installer package, and because we want to keep HDXPRT 4 up-to-date for testers, we’ve put together a new HDXPRT 4 build: v1.1. It includes HandBrake 1.2.2’s new capabilities, and we’re currently testing it in the lab.

With the new build, testers will be able to choose whether HDXPRT’s HandBrake tasks target a system’s integrated or discrete graphics cards by changing a flag called “UseIntegrated” in the config file. In HDXPRT 4 v1.1, the flag is set to “true” by default, directing HandBrake to use the codec provided by the system’s integrated graphics hardware. On the other hand, if a system has both integrated and discrete graphics available, and a user sets the flag to “false,” HandBrake will use the codec provided by the discrete graphics.

This update allows users to compare the video conversion performance of different video codecs on the same system. In all other respects, the benchmark has not changed. So apart from a scenario where a tester changes the targeted graphics hardware, scores from previous HDXPRT 4 builds will be comparable to those from the new build.

We’ll let the community know as soon as the new build is available, and we’ll update the HDXPRT 4 User Manual to reflect the changes.

If you have any questions about the upcoming HDXPRT 4 build, please let us know!

Justin

Posted in AMD, Future of performance evaluation, HandBrake, HDXPRT, HDXPRT 4, HDXPRT capabilities, HDXPRT release cycle, NVIDIA, Performance benchmarking | Tagged AMD, codec, discrete, Handbrake, HDXPRT, HDXPRT 4, integrated, NVIDIA, video |

Improvements to the AIXPRT results table

By Justin Greene

on May 30, 2019

Over the last few weeks, we’ve gotten great feedback about the kinds of data points people are looking for in AIXPRT results, as well as suggestions for how to improve the AIXPRT results viewer. To make it easier for visitors to find what they’re looking for, we’ve made a number of changes:

You can now filter results in categories such as framework, target hardware, batch size, and precision, and can designate minimum throughput and maximum latency scores. When you select a value from a drop-down menu or enter text, the results change immediately to reflect the filter.
You can search for variables such as processor vendor or processor speed.
The viewer displays eight results per page by default and lets you change this to 16, 48, or Show all.

The following features of the viewer, which have been present previously, can help you to navigate more efficiently:

Click the tabs at the top of the table to switch from ResNet-50 network results to SSD-MobileNet network results.
Click the header of any column to sort the data on that variable. One click sorts A-Z and two clicks sort Z-A.
Click the link in the Source column to visit a detailed page on that result. The page contains additional test configuration and system hardware information and lets you download results files.

We hope these changes will improve the utility of the results table. We’ll continue to add features to improve the experience. If you have any suggestions, please let us know!

Justin

Posted in AI, AIXPRT, Benchmarking, Collaborative benchmark development, Community Preview, Cross-platform benchmarks, OpenVINO, Performance benchmarking, TensorFlow, TensorRT | Tagged AIXPRT, Community Preview, latency, results, throughput |

Category: Performance benchmarking

Planning for the next TouchXPRT

The 2019 XPRT Spotlight Back-to-School Roundup

A new HDXPRT 4 build is available!

WebXPRT: What would you like to see?

An updated HDXPRT 4 build is on the way

Improvements to the AIXPRT results table

Check out the other XPRTs: