Month: February 2019

All about the AIXPRT Community Preview

on February 28, 2019

Last week, Bill discussed our plans for the AIXPRT Community Preview (CP). I’m happy to report that, despite some last-minute tweaks and testing, we’re close to being on schedule. We expect to take the CP build live in the coming days, and will send a message to community members to let them know when the build is available in the AIXPRT GitHub repository.

As we mentioned last week, the AIXPRT CP build includes support for the Intel OpenVINO, TensorFlow (CPU and GPU), and TensorFlow with NVIDIA TensorRT toolkits to run image-classification workloads with ResNet-50 and SSD-MobileNet v1 networks. The test reports FP32, FP16, and INT8 levels of precision. Although the minimum CPU and GPU requirements vary by toolkit, the test systems must be running Ubuntu 16.04 LTS. You’ll be able to find more detail on those requirements in the installation instructions that we’ll post on AIXPRT.com.

We’re making the AIXPRT CP available to anyone interested in participating, but you must have a GitHub account. To gain access to the CP, please contact us and let us know your GitHub username. Once we receive it, we’ll send you an invitation to join the repository as a collaborator.

We’re allowing folks to quote test results during the CP period, and we’ll publish results from our lab and other members of the community at AIXPRT.com. Because this testing involves so many complex variables, we may contact testers if we see published results that seem to be significantly different than those from comparable systems. During the CP period, On the AIXPRT results page, we’ll provide detailed instructions on how to send in your results for publication on our site. For each set of results we receive , we’ll disclose all of the detailed test, software, and hardware information that the tester provides. In doing so, our goal is to make it possible for others to reproduce the test and confirm that they get similar numbers.

If you make changes to the code during testing, we ask that you email us and describe those changes. We’ll evaluate if those changes should become part of AIXPRT. We also require that users do not publish results from modified versions of the code during the CP period.

We expect the AIXPRT CP period to last about four to six weeks, placing the public release around the end of March or beginning of April. In the meantime, we welcome your thoughts and suggestions about all aspects of the benchmark.

Please let us know if you have any questions. Stay tuned to AIXPRT.com and the blog for more developments, and we look forward to seeing your results!

JNG

Posted in AI, AIXPRT, Benchmark metrics, Benchmarking, BenchmarkXPRT, BenchmarkXPRT development community, Collaborative benchmark development, Community Preview, Future of performance evaluation, Machine learning, OpenVINO, Performance benchmarking, TensorFlow, TensorRT, Ubuntu | Tagged AI, AIXPRT, artificial intelligence, benchmark, Community Preview, machine learning, OpenVINO, TensorFlow, TensorRT |

Principled Technologies and the BenchmarkXPRT Development Community release HDXPRT 4, a benchmark designed to show how well Windows devices handle real-world media tasks

By Justin Greene

on February 25, 2019

Durham, NC, February 25 — Principled Technologies and the BenchmarkXPRT Development Community have released HDXPRT 4, a free benchmark that gives objective information about how well Windows 10 devices handle common media-creation tasks. HDXPRT 4 uses real commercial applications, like Photoshop and MediaEspresso, to perform tasks based on three everyday scenarios: photo editing, video conversion, and music editing. After the test is finished, the tool provides an overall measure by generating a single performance score. Anyone can go to HDXPRT.com to compare existing performance results on a variety of devices, or to download the app for themselves.

“When we started working on HDXPRT 4, we knew we wanted to create a benchmark that accurately reflects the kind of work average consumers do when creating content on their PCs,” said Bill Catchings, co-founder of Principled Technologies, which administers the BenchmarkXPRT Development Community. “HDXPRT delivers clear results that make sense to the wide audience of buyers shopping for new Windows systems.”

HDXPRT is part of the BenchmarkXPRT suite of performance evaluation tools, which includes WebXPRT, MobileXPRT, TouchXPRT, CrXPRT, and BatteryXPRT. The XPRTs help users get the facts before they buy, use, or evaluate tech products such as computers, tablets, and phones.

To learn more about the BenchmarkXPRT Development Community, go to www.BenchmarkXPRT.com.

About Principled Technologies, Inc.
Principled Technologies, Inc. is a leading provider of technology marketing, as well as learning and development services. It administers the BenchmarkXPRT Development Community.

Principled Technologies, Inc. is located in Durham, North Carolina, USA. For more information, please visit www.PrincipledTechnologies.com.

Company Contact
Justin Greene
BenchmarkXPRT Development Community
Principled Technologies, Inc.
1007 Slater Road, Ste. 300
Durham, NC 27704

BenchmarkXPRTsupport@PrincipledTechnologies.com

Posted in Benchmarking, BenchmarkXPRT, BenchmarkXPRT development community, Collaborative benchmark development, Future of performance evaluation, HDXPRT, HDXPRT 4, HDXPRT development process, HDXPRT metrics, HDXPRT Press Releases, Performance benchmarking, Press Releases, WebXPRT, Windows 10 | Tagged BatteryXPRT, BenchmarkXPRT, BenchmarkXPRT Development Community, CrXPRT, HDXPRT, MobileXPRT, TouchXPRT, WebXPRT |

HDXPRT 4 is here!

By Justin Greene

on February 22, 2019

We’re excited to announce that HDXPRT 4 is now available to the public! Just like previous versions of HDXPRT, HDXPRT 4 uses trial versions of commercial applications to complete real-world media tasks. The HDXPRT 4 installation package includes installers for some of those programs, such as Audacity and HandBrake. For other programs, such as Adobe Photoshop Elements and CyberLink Media Espresso, users will need to download the necessary installers prior to testing by using the links and instructions in the HDXPRT 4 User Manual.

In addition to the editing photos, editing music, and converting videos workloads from prior versions of the benchmark, HDXPRT 4 includes two new Photoshop Elements scenarios. The first utilizes an AI tool that corrects closed eyes in photos, and the second creates a single panoramic photo from seven separate photos.

HDXPRT 4 is compatible with systems running Windows 10, and is available for download at HDXPRT.com. The installation package is about 4.8 GB, so the download may take several minutes. The setup process takes about 30 minutes on most computers, and a standard test run takes approximately an hour.

After trying out HDXPRT 4, please submit your scores here and send any comments to BenchmarkXPRTsupport@principledtechnologies.com. To see test results from a variety of systems, go to HDXPRT.com and click View Results, where you’ll find scores from a variety of devices. We look forward to seeing your results!

Posted in Benchmarking, BenchmarkXPRT, BenchmarkXPRT development community, Collaborative benchmark development, HDXPRT, HDXPRT 4, HDXPRT development process, HDXPRT metrics, HDXPRT release cycle, HDXPRT workloads, Performance benchmarking, Windows 10 | Tagged Audacity, benchmark, desktops, Handbrake, HDXPRT, HDXPRT 4, laptops, MediaEspresso, PC, Photoshop, Windows |

Preparing for the AIXPRT Community Preview

By Bill Catchings

on February 21, 2019

Thanks to everyone who downloaded the AIXPRT Request for Comments (RFC) preview build. Next week, we’re planning to publish the AIXPRT Community Preview (CP). The AIXPRT CP build includes support for the Intel OpenVINO, TensorFlow (CPU and GPU), and TensorFlow with NVIDIA TensorRT toolkits to run image-classification workloads with ResNet-50 and SSD-MobileNet v1 networks. The test reports FP32, FP16, and INT8 levels of precision. As with the RFC build, the test systems must be running Ubuntu 16.04 LTS. The minimum CPU and GPU requirements vary according to the toolkit being used, and we will publish more details about the hardware minimums next week.

As with our other community previews, we think the AIXPRT CP candidate is solid enough to allow folks to start quoting test results. During CP periods, we generally allow members to publish their own results, but wait until the build is available to the public before we post results on our site. Because community feedback is especially important for AIXPRT, we will handle things a bit differently. During the CP period, we’ll publish results that we produce as well as those from other members of the community, which you’ll be able to view at AIXPRT.com.

We’ll also provide detailed instructions for publishing results and sending them to us. Because of the high number of variables in each potential test configuration, we’ll ask testers to disclose more test, software, and hardware information than in the past. We will make this information available along with the results on AIXPRT.com. Our goal is that others can reproduce these numbers and confirm that they get similar results.

Our CP periods typically last four to six weeks before we make the benchmark available to the general public. If that schedule holds, it would place the public AIXPRT release around the end of March. During the CP period, we welcome your thoughts and suggestions about all aspects of the benchmark.

Also, we normally restrict access to our CPs to BenchmarkXPRT Development Community members. However, because we’re seeking broad input from experts in this field, we’ll gladly make the CP available to anyone interested in participating who has a GitHub account. To gain access, please contact us and let us know your GitHub username. Once we receive it, we’ll send you an invitation to join the repository as a collaborator.

Please let us know if you have any questions. We look forward to hearing your feedback.

Bill

Posted in Uncategorized | Tagged AI, AIXPRT, BenchmarkXPRT, BenchmarkXPRT Development Community, Community Preview, GitHub, Intel, machine learning, MobileNet, NVIDIA, OpenVINO, ResNet, TensorFlow, TensorRT, test |

Out with the old, and in with the new

By Justin Greene

on February 14, 2019

What we now know as the BenchmarkXPRT Development Community started many years ago as the HDXPRT Development Community forum. At the time, the community was much smaller, and HDXPRT was our only benchmark. When a member wanted to run the benchmark, they submitted a request, and then received an installation DVD in the mail.

With hundreds of members, more than a half dozen active benchmarks, and the online availability of all our tools, the current community is a much different organization. Instead of the original forum, most of our interaction with members takes place through the blog, the monthly newsletter, direct email, and our social media accounts. Because of the way the community has changed, and because the original forum is no longer very active, we believe that the time and resources that we devote to maintaining the forum could be better spent on building and maintaining other community assets. To that end, we’ve decided to end support for the original BenchmarkXPRT forum.

As always, community members’ voices are an important consideration in what we do. If you have any questions or concerns about the decision to close down the original forum, please let us know as soon as possible.

On another note, we want to thank the community members who’ve participated in the HDXPRT 4 Community Preview. Testing has gone well, and we’re planning to release HDXPRT 4 to the public towards the end of next week!

Justin

Posted in Benchmark metrics, BenchmarkXPRT, BenchmarkXPRT development community, Collaborative benchmark development, Community Preview, HDXPRT, HDXPRT 4 | Tagged benchmark, BenchmarkXPRT, Community, HDXPRT, HDXPRT 4 |

Engaging AI

By Justin Greene

on February 7, 2019

In December, we wrote about our recent collaboration with students from North Carolina State University’s Department of Computer Science. We challenged the students to create a software console that includes an intuitive user interface, computes a performance metric, and uploads results to our database. The specific objective was to make it easy for testers to configure and run an implementation of the TensorFlow framework. In general, we hoped that the end product would model some of the same basic functions we plan to implement with AIXPRT, our machine-learning performance evaluation tool, currently under development.

The students did an outstanding job, and we hope to incorporate some of their work into AIXPRT in the future. We’ve been calling the overall project “Engaging AI” because it produced a functional tool that can help users interact with TensorFlow, and it was the first time that the students had an opportunity to work with AI tools. You can read more details on the Engaging AI page. We also have a new video that describes the project, including the new skillsets our students acquired to achieve success.

Finally, interested BenchmarkXPRT Development Community members can access to the project’s source code and additional documentation on our XPRT Experiments page. We hope you’ll check it out!

Justin

Posted in AI, AIXPRT, Benchmark metrics, Benchmarking, BenchmarkXPRT, BenchmarkXPRT development community, Collaborative benchmark development, Education, Engaging AI, Future of performance evaluation, North Carolina State University, TensorFlow |

Month: February 2019

All about the AIXPRT Community Preview

Principled Technologies and the BenchmarkXPRT Development Community release HDXPRT 4, a benchmark designed to show how well Windows devices handle real-world media tasks

HDXPRT 4 is here!

Preparing for the AIXPRT Community Preview

Out with the old, and in with the new

Engaging AI

Check out the other XPRTs: