PT-Logo
Forgot your password?
BenchmarkXPRT Blog banner

Category: BenchmarkXPRT development community

Odds and ends

Today, we want to share quick updates on a few XPRT topics.

In case you missed yesterday’s announcement, the CrXPRT 2 Community Preview (CP) is now available. BenchmarkXPRT Development Community members can access the preview using a direct link we’ve posted on the CrXPRT tab in the XPRT Members’ Area (login required). This tab also provides a link to the CrXPRT 2 CP user manual. You can find a summary of what’s new with CrXPRT 2 in last week’s blog. During the preview period, we allow testers to publish CP test scores. Note that CrXPRT 2 overall performance test scores and battery life measurements are not comparable to those from CrXPRT 2015.

We’ll soon be publishing our first AIXPRT whitepaper, Introduction to AIXPRT. It will summarize the AIXPRT toolkits and workloads; how to adjust test parameters such as batch size, levels of precision, and concurrent instances; how to use alternate test configuration files; and how to understand test results. When the paper is available, we’ll post it on the XPRT white papers page and make an announcement here in the blog.

Finally, in response to decreased downloads and usage of BatteryXPRT, we have ended support for the benchmark. We’re always monitoring usage of the XPRTs so that we can better direct our resources to the current needs of users. We’ve removed BatteryXPRT from the Google Play Store, but it is still available for download on BatteryXPRT.com.

If you have any questions about CrXPRT 2, AIXPRT, or BatteryXPRT, please let us know!

Justin

The CrXPRT 2 Community Preview is available!

We’re excited to announce that the CrXPRT 2 Community Preview (CP) is now available! BenchmarkXPRT Development Community members can access the preview using a direct link posted on the CrXPRT tab in the XPRT Members’ Area (login required), where they will also find the CrXPRT 2 CP user manual.

You can find more information about the key differences between CrXPRT 2015 and CrXPRT 2 in last week’s blog entry. During the preview period, we allow testers to publish CP test scores, but CrXPRT 2 overall performance test scores and battery life measurements are not comparable to CrXPRT 2015 scores.

If you have any questions about CrXPRT 2 or joining the community, please let us know!

Justin

The CrXPRT 2 Community Preview is almost here!

We appreciate everyone’s patience and feedback during the CrXPRT 2 development process. We’re excited to say that we’re now wrapping up some final details and expect to release the Community Preview (CP) within the next week.

Here is a summary of the key differences between CrXPRT 2015 and CrXPRT 2:

  • As we mentioned a few weeks ago, CrXPRT 2 has a completely new UI in line with the functional and aesthetic themes we used for the latest versions of WebXPRT, MobileXPRT, and HDXPRT, with a focus on intuitive navigation.
  • The CrXPRT 2 performance test includes six of the seven workloads in CrXPRT 2015. Newer versions of Chrome can’t run the Photo Collage workload without a workaround, so we removed it from CrXPRT 2.
  • We updated the images in the Photo Effects and Face Detection workloads to reflect more contemporary file resolutions and sizes.
  • The CrXPRT 2 battery life test requires a full rundown, so you’ll need charge your device to 100 percent before you can start the test, and the length of battery life tests will vary according to the battery life of the systems under test.
  • We no longer require testers to enter luminance and audio measurements in order to run a battery life test.
  • We added a second video playback segment to each battery life iteration.


We allow testers to publish CP test scores, but CrXPRT 2 overall performance test scores and battery life measurements are not comparable to CrXPRT 2015 scores.

Only BenchmarkXPRT Development Community members will be able to download the CrXPRT 2 CP. Because the Chrome team stopped providing search and browse functions for hosted and packaged Chrome apps in the Chrome Web Store, members will need a direct link to access the app. Once the app is available, we’ll post that link, along with the CrXPRT 2 CP user manual, on the CrXPRT tab in the XPRT Members’ Area (login required). We’ll also send a message to the community and post a notice here in the blog.

If you have any questions about CrXPRT 2 or joining the community, please let us know!

Justin

Principled Technologies and the BenchmarkXPRT Development Community make the AIXPRT source code available to the public

Durham, NC, February 18 — Principled Technologies and the BenchmarkXPRT Development Community release the source code for the AIXPRT benchmark to the public. AIXPRT is a free tool that allows users to evaluate a system’s machine learning inference performance by running common image-classification, object detection, and recommender system workloads.

“Publishing the AIXPRT source code is part of our commitment to making the XPRT development process as transparent as possible,” said Bill Catchings, co-founder of Principled Technologies, which administers the BenchmarkXPRT Development Community. “By allowing all interested parties to download and review our source code, we’re taking tangible steps to improve openness in the benchmarking industry.”

To access the AIXPRT source code, visit the AIXPRT GitHub repository at https://github.com/BenchmarkXPRT/AIXPRT.

AIXPRT includes support for the Intel© OpenVINO™, TensorFlow™, and NVIDIA© TensorRT™ toolkits to run image-classification and object-detection workloads with the ResNet-50 and SSD-MobileNet v1 networks, as well as the MXNet™ toolkit with a Wide and Deep recommender system workload. The test reports FP32, FP16, and INT8 levels of precision.

To access AIXPRT, visit www.AIXPRT.com.

AIXPRT is part of the BenchmarkXPRT suite of performance evaluation tools, which includes WebXPRT, CrXPRT, MobileXPRT, TouchXPRT, and HDXPRT. The XPRTs help users get the facts before they buy, use, or evaluate tech products such as computers, tablets, and phones.

To learn more about the BenchmarkXPRT Development Community, go to www.BenchmarkXPRT.com or contact a BenchmarkXPRT Development Community representative directly by sending a message to BenchmarkXPRTsupport@PrincipledTechnologies.com.

About Principled Technologies, Inc.
Principled Technologies, Inc. is a leading provider of technology marketing, as well as learning and development services. It administers the BenchmarkXPRT Development Community.

Principled Technologies, Inc. is located in Durham, North Carolina, USA. For more information, please visit www.PrincipledTechnologies.com.

Company Contact
Justin Greene
BenchmarkXPRT Development Community
Principled Technologies, Inc.
1007 Slater Road, Ste. 300
Durham, NC 27704
BenchmarkXPRTsupport@PrincipledTechnologies.com

The AIXPRT source code is now public

This week, we have good news for AIXPRT testers: the AIXPRT source code is now available to the public via GitHub. As we’ve discussed in the past, publishing XPRT source code is part of our commitment to making the XPRT development process as transparent as possible. With other XPRT benchmarks, we’ve only made the source code available to community members. With AIXPRT, we have released the source code more widely. By allowing all interested parties, not just community members, to download and review our source code, we’re taking tangible steps to improve openness and honesty in the benchmarking industry and we’re encouraging the kind of constructive feedback that helps to ensure that the XPRTs continue to contribute to a level playing field.

Traditional open-source models encourage developers to change products and even take them in new and different directions. Because benchmarking requires a product that remains static to enable valid comparisons over time, we allow people to download the source code and submit potential workloads for future consideration, but we reserve the right to control derivative works. This discourages a situation where someone publishes an unauthorized version of the benchmark and calls it an “XPRT.”

We encourage you to download and review the source and send us any feedback you may have. Your questions and suggestions may influence future versions of AIXPRT. If you have any questions about AIXPRT or accessing the source code, please feel free to ask! Please also let us know if you think we should take this approach to releasing the source code with other XPRT benchmarks.

Justin

A preview of the new CrXPRT 2 UI

As we get closer to the CrXPRT 2 Community Preview (CP), we want to provide readers with a glimpse of the new CrXPRT 2 UI. In line with the functional and aesthetic themes we used for the latest versions of WebXPRT, MobileXPRT, and HDXPRT, we’re implementing a clean, bright look with a focus on intuitive navigation. The screenshots below show how we’ve used that approach to rework the home, battery life test, performance test, and battery life test results screens. (We’re still tweaking the UI, so the screens you see in the CP may differ slightly.)

On the home screen, we kept the performance test and battery life test buttons, but made it clearer that you can choose only one. We also added a link to the user manual to the bottom ribbon for quick access.

If you choose to run a battery life test and click Next, the screen below appears. The CrXPRT 2 battery life test requires a full rundown, so you’ll need charge your device to 100 percent before you can start the test. Once you’ve done that, enter a name for the test run, unplug the system, and click Start. (Note that you no longer need to enter values for screen brightness and audio levels.)

The CrXPRT 2 performance test includes updated versions of six of the seven workloads in CrXPRT 2015. (As we discussed in a previous blog post, newer versions of Chrome can’t run the Photo Collage workload without a workaround, so we removed it from CrXPRT 2.)  To run the performance test, enter a name for the test run, customize the workloads if you wish, and click Start.

For the results screens, we wanted to highlight the most important end-of-test information while still offering clear paths for options such as getting additional details on the test, submitting results, and running the test again. Below, we show the results screen from a battery life test. Note the “Main menu” link in the upper-left corner, which we added to all screens to give users a quick way to navigate back to the home screen.

CrXPRT 2 development and testing are still underway. We don’t yet have an exact release date for the CP, but once we do, we’ll announce it here in the blog.

What do you think about the new CrXPRT 2 UI? Let us know!

Justin

Check out the other XPRTs: