BenchmarkXPRT Blog banner

Month: April 2011

An example of the community in action

Last week, I hosted a Webinar on HDXPRT. We’ll make a recording of it available on the site fairly soon. Multiple members attended. As I was going through the slides and discussing various aspects of the benchmark, a member asked about installing the benchmark from a USB key or a server. My response was the simple truth: we hadn’t considered that approach. As I then elaborated, we clearly should have thought about it, because those capabilities would be useful in just about every production lab out there, including ours here at PT. I concluded by saying that we’d look into it.

I’m not naming the member simply because with big companies I’m never sure if doing that will be good or will cause someone trouble, and I don’t want to cause hassle for anyone. He should, though, feel free to step forward and claim the well-deserved credit for the suggestion.

Less than a week after the Webinar, I’m happy to be able to report that the team has done more than look into these capabilities; it’s implemented them! So, the next Beta release, Beta 2, which we’ll be releasing any time now (maybe even before we post this blog entry), lets you install the benchmark from a network share or a USB key.

I know this is a relatively small thing, but I think it bears reporting because it is exactly the way the community should work. A member brought the benefits of his experience to bear in a great bit of feedback, and now the benchmark is better for it—and so are all of us who use it.

Keep the good ideas coming!

Mark Van Name

Comment on this post in the forums

Our community’s goal

Computer system performance evaluation has a long and complex history. Many of the earliest tests were simple, short code snippets, such as Whetstone, that did little more than give an indication of how fast a particular computer subsystem was able to operate. Unfortunately, such simple benchmarks quickly lost their value, in part because they were very crude measures, and in part because software tools on the things they were measuring could easily optimize for them. In some cases, a compiler could even recognize a test and “optimize” the code by simply producing the final result!

Over time, though, benchmarks have become more complex and more relevant. Whole organizations exist and have existed to build benchmarks. Notable ones include the Ziff-Davis Benchmark Operation (ZDBOp), which the Ziff-Davis computer magazines funded in the 1990s and which Mark and I ran; the Standard Performance Evaluation Corporation (SPEC), which its member companies fund and of which PT is a member; and the Business Applications Performance Corporation (BAPCo), which its member companies fund. Each of these organizations has developed widely used products, such as Winstone (ZDBOp), SPEC CPU (SPEC), and SYSmark (BAPCo). Each organization has also always faced challenges. In the case of ZDBOp, for example, Ziff Davis could no longer support the costs of developing its benchmarks, so they discontinued the group. SPEC continues to develop good benchmarks, but its process can sometimes yield years between versions.

The goal with HDXPRT and the HDXPRT Development Community (HDC) is to explore a new way to develop benchmarks. By utilizing the expertise and experience of a community of interested people, we hope to be able develop benchmarks in an open and collaborative environment while keeping them timely.

HDXPRT 2011 is the first test of this approach. We believe that it and subsequent versions of it, as well as other benchmarks, will give the industry a new model for creating world-class performance measurement tools.

If you’re not a member of the HDC, please consider joining us and helping define the future of performance evaluation.

Bill

Comment on this post in the forums

HDXPRT 2011 Beta Released for Testing

The HDXPRT Development Community, created by Principled Technologies (PT), is pleased to announce the distribution of the HDXPRT 2011 Beta to its registered members.

 
HDXPRT 2011 is a benchmark for evaluating the capabilities of PCs using real-world media scenarios and common consumer media applications.

PT has invited Community members to test the HDXPRT 2011 Beta and provide assistance in evaluating the benchmark. Their feedback will help resolve any remaining issues before PT releases the benchmark to the public. The deadline for Beta testing feedback is April 29, 2011.

Participation in the HDXPRT 2011 Beta program is available only to registered members of the HDXPRT Development Community. Membership is open to anyone willing to pay the nominal annual membership fee. Community members have the opportunity to help shape future versions of the benchmark. To register for the HDXPRT 2011 Beta program and receive access to members-only content, go tohttp://www.hdxprt.com/forum/register.php and complete the registration process.

To see the latest information on the benchmark’s development, visit the official HDXPRT Development Community Web site, http://www.hdxprt.com. The Community also has a presence on Facebook and Twitter.

About HDXPRT
HDXPRT, the High Definition eXperience & Performance Ratings Test, is a software tool for assessing the capabilities of PCs at handling real-world media scenarios and common consumer applications. HDXPRT 2011 is currently planned for release in the second quarter of 2011. It includes tests for popular consumer usage models such as high-definition video transcoding, High Dynamic Range (HDR) photo manipulation, Windows 7 Drag & Drop transcoding for portable media players, and HD Flash video playback.

About Principled Technologies, Inc.
Principled Technologies, Inc. is a leading provider of technology marketing and assessment services. The founders, Mark Van Name and Bill Catchings, have worked together in technology assessment for over 25 years. As journalists, they published over a thousand articles on a wide array of technology subjects. They created and led the Ziff-Davis Benchmark Operation, which developed such industry-standard benchmarks as Ziff Davis Media’s Winstone and WebBench. They have also co-founded or led several other technology testing firms including ZD Labs, eTesting Labs, and VeriTest.

Principled Technologies, Inc. is located in Durham, North Carolina, USA. For more information, please visithttp://www.principledtechnologies.com.

Company Contact
Eric Hale
Principled Technologies, Inc.
1007 Slater Road
Suite 300
Durham, NC 27703
ehale@principledtechnologies.com
www.principledtechnologies.com

Welcome!

Welcome to the HDXPRT Development Community blog!  I’m Bill Catchings, and Mark Van Name and I will be writing this blog along with a few other key folks from the PT (Principled Technologies) team. I’ve worked on benchmarks and performance analysis since the 1980s. On the team at PT, we have people who’ve worked on such influential past benchmarks as Winstone, WinBench, 3D WinBench, and WebBench. We’re all excited to be leading the HDXPRT Development Community (HDC).

Our goal with this blog is to give a glimpse of the world behind the HDXPRT (High Definition eXperience Performance Ratings Test) benchmark and its development community. HDXPRT 2011 will be the result of the work of the HDC to define what the benchmark should do. HDXPRT 2011 is now in beta test with members of the HDC.  Over the next few weeks, we would greatly appreciate whatever help you can give us in trying out the benchmark, shaking out bugs, and producing a polished product.

Thanks in advance to the HDC!

If you are reading this and are not a member of the HDC, I’d like to encourage you to join us. Annual membership is just $20. If you join in the next week or so, you will get a copy of the HDXPRT 2011 Beta and be able to provide feedback on it. We’d love to have you on board!

Bill

Comment on this post in the forums

Check out the other XPRTs:

Forgot your password?