BenchmarkXPRT Blog banner

Category: Performance benchmarking

The HDXPRT 4 Community Preview is now available!

Justin Greene
By Justin Greene
on January 17, 2019

Today we’re releasing the HDXPRT 4 Community Preview (CP). Just like previous versions of HDXPRT, HDXPRT 4 uses trial versions of commercial applications to complete workload tasks. For some of those programs, such as Audacity and HandBrake, HDXPRT 4 includes installers in the HDXPRT installation package. For other programs, such as Adobe Photoshop Elements 2018 and CyberLink Media Espresso 7.5, users need to download the necessary installers prior to testing by using the links and instructions in the HDXPRT 4 User Manual.

In addition to the editing photos, editing music, and converting videos workloads from prior versions of the benchmark, HDXPRT 4 includes two new Photoshop Elements scenarios. The first utilizes an AI tool that corrects closed eyes in photos, and the second creates a single panoramic photo from seven separate photos.

HDXPRT 4 is compatible with systems running Windows 10, and the installation package is slightly smaller than previous versions at just over 4.7 GB.

Because this is a community preview, it is available only to community members, who may download the preview from the HDXPRT tab in the Members’ Area. Because we expect results from CP testing to be comparable to results from the general release, members may publish their CP test results.

After you try the CP, please send us your comments. If you send information that’s relevant to the entire community, we may post an anonymous version of your comments to the forum. Thanks for your participation!

Justin

Posted in AI, Application-based benchmarks, Benchmark metrics, BenchmarkXPRT, BenchmarkXPRT development community, Collaborative benchmark development, Community Preview, Future of performance evaluation, HDXPRT, HDXPRT 4, HDXPRT release cycle, Performance benchmarking, Windows 10 |

New XPRTs for the new year

Justin Greene
By Justin Greene
on January 3, 2019

Happy 2019! January is already a busy time for the XPRTs, so we want to share a quick preview of what community members can expect in the coming months.

The MobileXPRT 3 community preview (CP) is still open, but draws to a close on January 18th. If you are not familiar with the updates and changes we implemented in the newest version of MobileXPRT, you can read more in the blog. Members can access this APK on the MobileXPRT tab in the Members’ Area. We also posted an installation guide that provides both a general overview of the app and detailed instructions for each step. The entire process takes about five minutes on most devices. If you haven’t already, give it a try!

We also recently published the first AIXPRT Request for Comments (RFC) preview build, an early version of one of the tools we’re developing to evaluate machine learning performance. You can find more details in Bill’s most recent blog post and on AIXPRT.com. Only BenchmarkXPRT Development Community members have access to our RFCs and the opportunity to provide feedback. However, because we’re seeking broad input from experts in this field, we’ll gladly make anyone interested in participating a member. To gain access to the AIXPRT repository, please send us a request.

Work on the HDXPRT 4 CP candidate build continues, and we hope to publish the preview for community members this month. We appreciate everyone’s patience as we work to get this right. We think it will be worth the wait.

On a general note, I’ll be travelling to CES 2019 in Las Vegas next week. CES is a great opportunity for us to survey emerging tech and industry trends, and I look forward to sharing my thoughts from the show. If you’ll be there and would like to discuss any aspect of the XPRTs in person, let me know.

Justin

Posted in AI, AIXPRT, Benchmarking, BenchmarkXPRT, BenchmarkXPRT development community, CES, Collaborative benchmark development, Community Preview, Future of performance evaluation, HDXPRT, HDXPRT 4, HDXPRT development process, Machine learning, MobileXPRT, MobileXPRT 3, Performance benchmarking, Trade Shows | Tagged , , , , , , |

The AIXPRT Request for Comments preview build

Bill Catchings
By Bill Catchings
on December 20, 2018

In the next few days, we’ll be publishing the first AIXPRT tool as a Request for Comments (RFC) preview build, an early version of one of the AIXPRT tools we’re developing to help evaluate machine learning performance.

We’re inviting folks to run the workload and send in their thoughts and suggestions. Only BenchmarkXPRT Development Community members have access to our RFCs and the opportunity to provide feedback. However, because we’re seeking broad input from experts in this field, we’ll gladly make anyone interested in participating a member.

This AIXPRT RFC preview build includes support for the Intel OpenVINO computer vision toolkit to run image classification workloads with ResNet-50 and SSD-MobileNet v1 networks. The test reports FP32 and FP16 levels of precision. The system requirements are:

  • Operating system = Ubuntu 16.04
  • CPU = 6th to 8th generation Intel Core or Xeon processors, or Intel Pentium processors N4200/5, N3350/5, N3450/5 with Intel HD Graphics


We welcome input on all aspects of the benchmark, including scope, workloads, metrics and scores, user experience, and reporting. We will add support for TensorFlow and TensorRT to the AIXPRT RFC preview build during the preview period. We are accepting feedback through January 25th, 2019, after which we’ll collect and evaluate responses before publishing the next build. Because this is an RFC release, we ask that testers do not publish scores or use the results for comparison purposes.

We’ll send out a community announcement when the RFC preview build is officially available, and we’ll also post an announcement and RFC preview build user guide on AIXPRT.com. We’re hosting the AIXPRT RFC preview build in a dedicated GitHub repository, so please contact us at BenchmarkXPRTsupport@principledtechnologies.com to gain access.

This is just the next step for AIXPRT. With your help, we hope to add more workloads and other frameworks in the coming months. We look forward to receiving your feedback!

Bill

Posted in AI, AIXPRT, BenchmarkXPRT development community, Collaborative benchmark development, Community Preview, Future of performance evaluation, Machine learning, Open development, OpenVINO, Performance benchmarking, TensorFlow, TensorRT, Ubuntu, What makes a good benchmark? |

HDXPRT 4: Getting it right

Justin Greene
By Justin Greene
on November 8, 2018

For BenchmarkXPRT Development Community members anticipating the HDXPRT 4 Community Preview (CP), we want to thank you for your patience and explain where we are in the release process.

This past month has brought a flurry of activity in the Windows 10 development world. We’ve been testing HDXPRT 4 extensively on each of the new prerelease builds available through the Windows Insider Program. While testing on a recent Windows 10 Redstone 5 preview build, we began to see inconsistent HDXPRT 4 workload scores on some systems. The difference between those workload scores and scores on the same systems with previous Windows 10 builds was significant enough for us to decide that the best course of action is to hold off on the CP until we understand the issue. We don’t want to release a CP only to run into serious problems with an imminent Windows release. We want to take the time to figure out what’s going on and get it right.

We hope to resolve these issues and publish the HDXPRT 4 CP as soon as possible. Thanks again for your patience. We’ll update the community soon with more information on the anticipated release schedule.

Justin

Posted in Application-based benchmarks, Benchmarking, BenchmarkXPRT, BenchmarkXPRT development community, Collaborative benchmark development, Community Preview, HDXPRT, HDXPRT 4, HDXPRT development process, HDXPRT release cycle, Performance benchmarking, Windows 10 | Tagged , , , , , |

The new WebXPRT 3 Processor Comparison Chart

Justin Greene
By Justin Greene
on October 25, 2018

Last fall, we published the WebXPRT 2015 Processor Comparison Chart, a tool that makes it easier to access hundreds of PT-curated, real-world performance scores from a wide range of devices including everything from TVs to phones to tablets to PCs. Today, we’re happy to announce that we’ve added a WebXPRT 3 Processor Comparison Chart.

The WebXPRT 3 chart follows the same format as the WebXPRT 2015 chart, letting you click the average score of each processor to view all the WebXPRT 3 results we currently have for that processor. You can change the number of results the chart displays on each page, and as the screenshot below shows, a new drop-down menu lets you toggle back and forth between the WebXPRT 3 and WebXPRT 2015 charts. W plan to add additional capabilities on a regular basis, so if you have ideas for features and types of data you’d like to see, let us know!

If you’d like to submit results for us to consider for publication in the chart, follow the detailed instructions here. The submission process is quick and easy. We look forward to seeing your results!

WebXPRT 3 Proc Chart

Justin

Posted in Android, Battery life, BatteryXPRT 2014 for Android, Browser-based benchmarks, Performance benchmarking, WebXPRT, WebXPRT 3, WebXPRT Processor Comparison Chart | Tagged , , , , , , |

A BatteryXPRT bug fix is on the way

Justin Greene
By Justin Greene
on October 18, 2018

Some time ago, we started to see unusual BatteryXPRT battery life estimates and high variance on some devices when running tests at the default length of 5.25 hours (seven 45-minute iterations). We suspected that the problem resulted from changes in how new OS versions report battery life on certain devices (e.g., charging past a reported level of 100 percent). In addition, the progress of battery technology in general means that the average phone battery lasts much longer than it did a few years ago. Together, these factors sometimes led to BatteryXPRT runs where the OS reported little to no battery decrease during the first few iterations of a test. We concluded that 5.25 hours wasn’t long enough to produce an accurate battery life estimate.

After extensive experimentation and testing, we’ve decided to release a new build that increases the default BatteryXPRT test length from 5.25 hours (seven iterations) to 45 hours (60 iterations) to allow enough time for a full rundown on most phones. Based on our testing, we consider full rundown tests to be the most accurate and will use those exclusively in our Spotlight testing and elsewhere. Testers will still have the option of choosing shorter test durations, but BatteryXPRT will flag the results with a qualifier that recommends performing a full rundown.

We plan to release the updated build by the end of next week and update BatteryXPRT documentation to reflect the changes. We have not changed any of the workloads and both performance results and full-rundown battery life estimates will be comparable to results from earlier builds.

BatteryXPRT continues to be a useful tool for gauging the performance and expected battery life of Android devices while simulating real-world tasks. If you have any questions about BatteryXPRT, please feel free to ask!

Justin

Posted in Android, Battery life, BatteryXPRT 2014 for Android, Benchmarking, BenchmarkXPRT, Mobile devices, Performance benchmarking | Tagged , , , , |

Check out the other XPRTs:

Forgot your password?
© Principled Technologies, Inc. 2025   |   Privacy policy

Cookie Notice: Our website uses cookies to deliver a smooth experience by storing logins and saving user information. By continuing to use our site, you agree with our usage of cookies as our privacy policy outlines.

x