BenchmarkXPRT Blog banner

An example of the community in action

Last week, I hosted a Webinar on HDXPRT. We’ll make a recording of it available on the site fairly soon. Multiple members attended. As I was going through the slides and discussing various aspects of the benchmark, a member asked about installing the benchmark from a USB key or a server. My response was the simple truth: we hadn’t considered that approach. As I then elaborated, we clearly should have thought about it, because those capabilities would be useful in just about every production lab out there, including ours here at PT. I concluded by saying that we’d look into it.

I’m not naming the member simply because with big companies I’m never sure if doing that will be good or will cause someone trouble, and I don’t want to cause hassle for anyone. He should, though, feel free to step forward and claim the well-deserved credit for the suggestion.

Less than a week after the Webinar, I’m happy to be able to report that the team has done more than look into these capabilities; it’s implemented them! So, the next Beta release, Beta 2, which we’ll be releasing any time now (maybe even before we post this blog entry), lets you install the benchmark from a network share or a USB key.

I know this is a relatively small thing, but I think it bears reporting because it is exactly the way the community should work. A member brought the benefits of his experience to bear in a great bit of feedback, and now the benchmark is better for it—and so are all of us who use it.

Keep the good ideas coming!

Mark Van Name

Comment on this post in the forums

Our community’s goal

Computer system performance evaluation has a long and complex history. Many of the earliest tests were simple, short code snippets, such as Whetstone, that did little more than give an indication of how fast a particular computer subsystem was able to operate. Unfortunately, such simple benchmarks quickly lost their value, in part because they were very crude measures, and in part because software tools on the things they were measuring could easily optimize for them. In some cases, a compiler could even recognize a test and “optimize” the code by simply producing the final result!

Over time, though, benchmarks have become more complex and more relevant. Whole organizations exist and have existed to build benchmarks. Notable ones include the Ziff-Davis Benchmark Operation (ZDBOp), which the Ziff-Davis computer magazines funded in the 1990s and which Mark and I ran; the Standard Performance Evaluation Corporation (SPEC), which its member companies fund and of which PT is a member; and the Business Applications Performance Corporation (BAPCo), which its member companies fund. Each of these organizations has developed widely used products, such as Winstone (ZDBOp), SPEC CPU (SPEC), and SYSmark (BAPCo). Each organization has also always faced challenges. In the case of ZDBOp, for example, Ziff Davis could no longer support the costs of developing its benchmarks, so they discontinued the group. SPEC continues to develop good benchmarks, but its process can sometimes yield years between versions.

The goal with HDXPRT and the HDXPRT Development Community (HDC) is to explore a new way to develop benchmarks. By utilizing the expertise and experience of a community of interested people, we hope to be able develop benchmarks in an open and collaborative environment while keeping them timely.

HDXPRT 2011 is the first test of this approach. We believe that it and subsequent versions of it, as well as other benchmarks, will give the industry a new model for creating world-class performance measurement tools.

If you’re not a member of the HDC, please consider joining us and helping define the future of performance evaluation.

Bill

Comment on this post in the forums

HDXPRT 2011 Beta Released for Testing

The HDXPRT Development Community, created by Principled Technologies (PT), is pleased to announce the distribution of the HDXPRT 2011 Beta to its registered members.

 
HDXPRT 2011 is a benchmark for evaluating the capabilities of PCs using real-world media scenarios and common consumer media applications.

PT has invited Community members to test the HDXPRT 2011 Beta and provide assistance in evaluating the benchmark. Their feedback will help resolve any remaining issues before PT releases the benchmark to the public. The deadline for Beta testing feedback is April 29, 2011.

Participation in the HDXPRT 2011 Beta program is available only to registered members of the HDXPRT Development Community. Membership is open to anyone willing to pay the nominal annual membership fee. Community members have the opportunity to help shape future versions of the benchmark. To register for the HDXPRT 2011 Beta program and receive access to members-only content, go tohttp://www.hdxprt.com/forum/register.php and complete the registration process.

To see the latest information on the benchmark’s development, visit the official HDXPRT Development Community Web site, http://www.hdxprt.com. The Community also has a presence on Facebook and Twitter.

About HDXPRT
HDXPRT, the High Definition eXperience & Performance Ratings Test, is a software tool for assessing the capabilities of PCs at handling real-world media scenarios and common consumer applications. HDXPRT 2011 is currently planned for release in the second quarter of 2011. It includes tests for popular consumer usage models such as high-definition video transcoding, High Dynamic Range (HDR) photo manipulation, Windows 7 Drag & Drop transcoding for portable media players, and HD Flash video playback.

About Principled Technologies, Inc.
Principled Technologies, Inc. is a leading provider of technology marketing and assessment services. The founders, Mark Van Name and Bill Catchings, have worked together in technology assessment for over 25 years. As journalists, they published over a thousand articles on a wide array of technology subjects. They created and led the Ziff-Davis Benchmark Operation, which developed such industry-standard benchmarks as Ziff Davis Media’s Winstone and WebBench. They have also co-founded or led several other technology testing firms including ZD Labs, eTesting Labs, and VeriTest.

Principled Technologies, Inc. is located in Durham, North Carolina, USA. For more information, please visithttp://www.principledtechnologies.com.

Company Contact
Eric Hale
Principled Technologies, Inc.
1007 Slater Road
Suite 300
Durham, NC 27703
ehale@principledtechnologies.com
www.principledtechnologies.com

Welcome!

Welcome to the HDXPRT Development Community blog!  I’m Bill Catchings, and Mark Van Name and I will be writing this blog along with a few other key folks from the PT (Principled Technologies) team. I’ve worked on benchmarks and performance analysis since the 1980s. On the team at PT, we have people who’ve worked on such influential past benchmarks as Winstone, WinBench, 3D WinBench, and WebBench. We’re all excited to be leading the HDXPRT Development Community (HDC).

Our goal with this blog is to give a glimpse of the world behind the HDXPRT (High Definition eXperience Performance Ratings Test) benchmark and its development community. HDXPRT 2011 will be the result of the work of the HDC to define what the benchmark should do. HDXPRT 2011 is now in beta test with members of the HDC.  Over the next few weeks, we would greatly appreciate whatever help you can give us in trying out the benchmark, shaking out bugs, and producing a polished product.

Thanks in advance to the HDC!

If you are reading this and are not a member of the HDC, I’d like to encourage you to join us. Annual membership is just $20. If you join in the next week or so, you will get a copy of the HDXPRT 2011 Beta and be able to provide feedback on it. We’d love to have you on board!

Bill

Comment on this post in the forums

Principled Technologies, Inc. Announces the Formation of the HDxPRT Development Community

Principled Technologies (PT), a leading provider of independent technology marketing and assessment services, has announced the formation of the HDxPRT Development Community.

 
This group will help shape future versions of HDxPRT, a benchmark for evaluating the capabilities of PCs using real-world media scenarios and common consumer applications. Intel Corporation developed HDxPRT, the latest release of which, HDxPRT 2010, is the fifth. Members of the Community, which Intel is sponsoring, will be able to contribute to the definition of future versions of the benchmark. PT will create and administer the Community Web site, where anyone can learn more about HDxPRT or join the Community, collect input to help define future versions of HDxPRT, and coordinate and work with the members.

PT brings a long history of expertise in benchmarking and performance evaluation to the HDxPRT effort. “PT’s experience with industry-standard benchmarks will benefit both HDxPRT and the industry,” said Bill Catchings, CTO and co-founder of Principled Technologies. “The goal of HDxPRT is to address the need for relevant media and consumer-oriented usage scenarios. Our hope is that we can build on Intel’s work with HDxPRT and increase its value for consumers and for the industry. We look forward to working with a wide range of industry partners on future versions of HDxPRT.”

The HDxPRT Community will provide a forum for members to provide feedback and guidance for the ongoing evolution of the benchmark. Membership will be open to anyone willing to pay the nominal annual membership fee. “There has been a long-standing need for a consumer-oriented benchmark like HDxPRT, which is based on end-user workflows with emphasis on user experiences communicated in plain language rather than the more traditional technology-focused metrics. Intel is delighted that HDxPRT Community efforts can help these user experience metrics reach a wider audience.” said Shervin Kheradpir, General Manager of Intel’s Performance Benchmarking and Analysis Group.

The current plan is to make HDxPRT 2011 generally available in the second quarter of 2011. The official Web site for this program, http://www.hdxprt.com/, will provide more details as they are available.

About HDXPRT
HDxPRT, the High Definition eXperience & Performance Ratings Test, is a software tool for assessing the capabilities of PCs at handling real-world media scenarios and common consumer applications. In 2005, realizing a need for a benchmark to accurately assess how well PCs handle the rapidly evolving digital media usage models, Intel started developing the technology that led to HDxPRT. HDxPRT 2010, which Intel released in July, is the fifth revision. It includes tests for popular consumer usage models such as 1080p high-definition video transcoding, High Dynamic Range (HDR) photo manipulation, Windows 7 Drag & Drop transcoding for portable media players, and HD Flash video playback.

About Principled Technologies, Inc.
Principled Technologies, Inc. is a leading provider of technology marketing and assessment services. The founders, Mark Van Name and Bill Catchings, have worked together in technology assessment for over 25 years. As journalists they published over a thousand articles on a wide array of technology subjects. They created and led the Ziff-Davis Benchmark Operation, which developed such industry-standard benchmarks as Ziff Davis Media’s Winstone and WebBench. They have also co-founded or led several other technology testing firms including ZD Labs, eTesting Labs, and VeriTest.

Principled Technologies is located in Durham, North Carolina USA. For more information, please visit http://www.principledtechnologies.com.

Company Contact
Eric Hale
Principled Technologies, Inc.
1007 Slater Road
Suite 300
Durham, NC 27703
ehale@principledtechnologies.com
www.principledtechnologies.com

Check out the other XPRTs:

Forgot your password?