BenchmarkXPRT Development Community Blog

HDXPRT 4: Getting it right

on November 8, 2018

For BenchmarkXPRT Development Community members anticipating the HDXPRT 4 Community Preview (CP), we want to thank you for your patience and explain where we are in the release process.

This past month has brought a flurry of activity in the Windows 10 development world. We’ve been testing HDXPRT 4 extensively on each of the new prerelease builds available through the Windows Insider Program. While testing on a recent Windows 10 Redstone 5 preview build, we began to see inconsistent HDXPRT 4 workload scores on some systems. The difference between those workload scores and scores on the same systems with previous Windows 10 builds was significant enough for us to decide that the best course of action is to hold off on the CP until we understand the issue. We don’t want to release a CP only to run into serious problems with an imminent Windows release. We want to take the time to figure out what’s going on and get it right.

We hope to resolve these issues and publish the HDXPRT 4 CP as soon as possible. Thanks again for your patience. We’ll update the community soon with more information on the anticipated release schedule.

Justin

Posted in Application-based benchmarks, Benchmarking, BenchmarkXPRT, BenchmarkXPRT development community, Collaborative benchmark development, Community Preview, HDXPRT, HDXPRT 4, HDXPRT development process, HDXPRT release cycle, Performance benchmarking, Windows 10 | Tagged BenchmarkXPRT Development Community, Community Preview, HDXPRT, HDXPRT 4, Windows, Windows 10 |

WebXPRT in action

By Justin Greene

on November 1, 2018

Just this past summer, WebXPRT passed the 250,000-run milestone, and since then, the run total has already passed 330,000. September was our biggest month ever, with over 28,000 WebXPRT runs! We sometimes like to show the community how far a reach the XPRTs have around the world by reporting the latest stats on the number of articles and reviews that mention the XPRTs, and the fact is that most of those mentions involve WebXPRT. Today, I thought it would be interesting to bring the numbers to life and provide a glimpse of how the tech press uses WebXPRT. Here’s a sample of WebXPRT in action during the past couple of weeks.

PCWorld published WebXPRT 2015 and WebXPRT 3 scores in a review of the Snapdragon 50-based Samsung Galaxy Book 2.
AnandTech published WebXPRT 3 and WebXPRT 2015 score in their detailed analysis of the Intel Core i9-9900K, Core i7-9700K, and Core i5-9600K.
The Tech Report published a WebXPRT 3 score in its AMD Ryzen Threadripper 2920X review.
AnandTech published WebXPRT 3 scores for the AMD Threadripper 2 2920X and 2970WX in an extensive review, and explained why they’ll be using WebXPRT in their 2019 benchmark suite.
Tom’s Hardware published a WebXPRT 2015 score in an AMD Ryzen Threadripper 2970WX review.
Notebookcheck used WebXPRT 3 in an LG Q Stylus review and WebXPRT 2015 in a Dell Chromebook 11 3181 review.
3DNews (Russia) used WebXPRT 2015 in its Apple iPhone XS Max review.

While WebXPRT continues to be a useful tool for tech enthusiasts around the world, you don’t have to be a tech expert to benefit from it. If you’d like to know more about WebXPRT, check out our recent video, What is WebXPRT and why should I care?

Justin

Posted in Browser-based benchmarks, Cross-platform benchmarks, Web-based testing, WebXPRT, WebXPRT 2015, WebXPRT 3 | Tagged 3DNews, AnandTech, BenchmarkXPRT, Notebookcheck, PCWorld, The Tech Report, Tom's Hardware, WebXPRT, WebXPRT 2015 |

A new BatteryXPRT 2014 for Android build is available

By Justin Greene

on October 26, 2018

In last week’s blog, we discussed why we now consider full BatteryXPRT rundown tests to be the most accurate and why we’re releasing a new build (v110) that increases the default BatteryXPRT test from 5.25 hours (seven iterations) to 45 hours (60 iterations). We also built v110 using Android Studio SDK 27, in order to bring BatteryXPRT up to date with current Android standards. Today, we’ve posted the new build on BatteryXPRT.com and in the Google Play Store, and we’ve also published an updated user manual. Please contact us if you have any questions about BatteryXPRT testing.

Justin

Posted in Android, Battery life, BatteryXPRT 2014 for Android, Benchmarking, BenchmarkXPRT | Tagged Android, battery life, BatteryXPRT, mobile, Performance, phones, tablets |

The new WebXPRT 3 Processor Comparison Chart

By Justin Greene

on October 25, 2018

Last fall, we published the WebXPRT 2015 Processor Comparison Chart, a tool that makes it easier to access hundreds of PT-curated, real-world performance scores from a wide range of devices including everything from TVs to phones to tablets to PCs. Today, we’re happy to announce that we’ve added a WebXPRT 3 Processor Comparison Chart.

The WebXPRT 3 chart follows the same format as the WebXPRT 2015 chart, letting you click the average score of each processor to view all the WebXPRT 3 results we currently have for that processor. You can change the number of results the chart displays on each page, and as the screenshot below shows, a new drop-down menu lets you toggle back and forth between the WebXPRT 3 and WebXPRT 2015 charts. W plan to add additional capabilities on a regular basis, so if you have ideas for features and types of data you’d like to see, let us know!

If you’d like to submit results for us to consider for publication in the chart, follow the detailed instructions here. The submission process is quick and easy. We look forward to seeing your results!

Justin

Posted in Android, Battery life, BatteryXPRT 2014 for Android, Browser-based benchmarks, Performance benchmarking, WebXPRT, WebXPRT 3, WebXPRT Processor Comparison Chart | Tagged Android, battery life, BatteryXPRT, Performance, processor, PT, WebXPRT |

A BatteryXPRT bug fix is on the way

By Justin Greene

on October 18, 2018

Some time ago, we started to see unusual BatteryXPRT battery life estimates and high variance on some devices when running tests at the default length of 5.25 hours (seven 45-minute iterations). We suspected that the problem resulted from changes in how new OS versions report battery life on certain devices (e.g., charging past a reported level of 100 percent). In addition, the progress of battery technology in general means that the average phone battery lasts much longer than it did a few years ago. Together, these factors sometimes led to BatteryXPRT runs where the OS reported little to no battery decrease during the first few iterations of a test. We concluded that 5.25 hours wasn’t long enough to produce an accurate battery life estimate.

After extensive experimentation and testing, we’ve decided to release a new build that increases the default BatteryXPRT test length from 5.25 hours (seven iterations) to 45 hours (60 iterations) to allow enough time for a full rundown on most phones. Based on our testing, we consider full rundown tests to be the most accurate and will use those exclusively in our Spotlight testing and elsewhere. Testers will still have the option of choosing shorter test durations, but BatteryXPRT will flag the results with a qualifier that recommends performing a full rundown.

We plan to release the updated build by the end of next week and update BatteryXPRT documentation to reflect the changes. We have not changed any of the workloads and both performance results and full-rundown battery life estimates will be comparable to results from earlier builds.

BatteryXPRT continues to be a useful tool for gauging the performance and expected battery life of Android devices while simulating real-world tasks. If you have any questions about BatteryXPRT, please feel free to ask!

Justin

Posted in Android, Battery life, BatteryXPRT 2014 for Android, Benchmarking, BenchmarkXPRT, Mobile devices, Performance benchmarking | Tagged Android, battery life, BatteryXPRT, mobile devices, XPRT Spotlight |

Updates on HDXPRT 4 and MobileXPRT 3

By Justin Greene

on October 11, 2018

There’s a lot going on with the XPRTs, so we want to offer a quick update.

On the HDXPRT 4 front, we’re currently testing community preview candidate builds across a variety of laptops and desktops. Testing is going well, but as is often the case prior to a release, we’re still tweaking the code as necessary when we run into bugs. We’re excited about HDXPRT 4 and look forward to the community seeing how much faster and easier to use it is than previous versions. You can read more about what’s to come in HDXPRT 4 here.

On the MobileXPRT 3 front, proof-of-concept testing for the new and updated workloads went well, and we’re now working to implement the new UI. Below, you can see a mockup of the new MobileXPRT 3 start screen for phones. The aesthetic is completely different than MobileXPRT 2015, and is in line with the clean, bright look we used for WebXPRT 3 and HDXPRT 4. We’ve made it easy to select and deselect individual workloads by tapping the workload name (deselected workloads are grayed out), and we’ve consolidated common menu items into an Android-style taskbar at the bottom of the screen. Please note that this is an early view and some aspects of the screen will change. For instance, we’re certain that the final receipt-scanning workload won’t be called “Optical character recognition.”

We’ll share more information about HDXPRT 4 and MobileXPRT 3 in the coming weeks. If you have any questions about HDXPRT or MobileXPRT, or would like to share your ideas, please get in touch!

Justin

Posted in Android, Benchmarking, BenchmarkXPRT, BenchmarkXPRT development community, Collaborative benchmark development, Community Preview, HDXPRT, HDXPRT 4, HDXPRT development process, HDXPRT release cycle, MobileXPRT, MobileXPRT 2015, MobileXPRT 3, Performance benchmarking, WebXPRT, WebXPRT 3 | Tagged Android, Community, HDXPRT, laptops, mobile devices, MobileXPRT, phones |

HDXPRT 4: Getting it right

WebXPRT in action

A new BatteryXPRT 2014 for Android build is available

The new WebXPRT 3 Processor Comparison Chart

A BatteryXPRT bug fix is on the way

Updates on HDXPRT 4 and MobileXPRT 3

Check out the other XPRTs: