BenchmarkXPRT Development Community Blog

Looking for a winner

on October 11, 2012

This week, PT published its first two public reports using HDXPRT 2012: Performance comparison: Dell Latitude E5430 vs. HP ProBook 4440s and Performance comparison: Dell Latitude E5430 vs. Lenovo ThinkPad L430. You should check them out.

Of course, you can find the HDXPRT results from these reports in the HDXPRT 2012 results database along with results from the characterization study we did last month. The results database is a repository of HDXPRT results you can use to compare system performance. The database includes full disclosure information and lets you sort by a number of criteria, including any HDXPRT score, the processor, amount of RAM, graphic card, and so on.

Looking at the results in the database got me wondering who has the mightiest machine out there. The current winner is a custom-built system with an Intel Core i7 3770 and 8 GB of RAM. It has an HDXPRT 2012 Create HD score of 248.

Records are meant to be broken, and I know someone out there can grind that score to dust. So, we’re going to have a contest. The first person to submit a set of HDXPRT results with a score above 248 will win at least bragging rights and maybe a prize if we can find something suitable around our offices.

You’ll find instructions for submitting results at Submit your HDXPRT 2012 results.

I can’t wait to see your results!

Eric

Comment on this post in the forums

Posted in Application-based benchmarks, Benchmarking, HDXPRT 2012, HDXPRT 2012 results, Performance benchmarking |

Make sure your voice is heard

By Eric Hale

on October 4, 2012

One thing about the community model we use for developing HDXPRT is that is depends on the community. Your input is essential to making the benchmark the best it can be. As the community grows, we’re learning more about your priorities.

During the development of HDXPRT 2012, we made the decision to remove the playback tests from the benchmark. While the design document called for the playback test to include 4K H.264, Windows Media Player does not play that format by default. Because less demanding codecs were not differentiating systems, and because the stars used to report the results confused some people, it seemed like a reasonable decision. Bill announced the decision in a blog post, More HDXPRT 2012 changes.

Fast forward to September 18, when Bill hosted the HDXPRT 2012 Webinar. During the Q&A session, a new member of the community said that the playback tests from HDXPRT 2011 were what got him interested in the benchmark. For now, he has to use HDXPRT 2011 for those tests, although, as per Bill’s original blog post, we may release a more demanding playback test as a standalone inspection test later this year.

The suggestion period for HDXPRT 2013 started on October 1. Now is the time to let us know what tests are the most useful to you. If there are tests you’d like us to add, tests you’d like us to change, applications you’d like us to consider, we need to know that too. You can post your suggestions to the forum in the HDXPRT 2013 Suggestions section or mail them to hdxprtsupport@hdxprt.com.

In November, we’ll develop an RFC for HDXPRT 2013 and send it to the community for review.

While the suggestions we receive early have the best chance of being implemented, comments we receive after the formal suggestion period still get our attention. We’re always listening. Contact us anytime and make sure that HDXPRT 2013 includes the things that are important to you.

Eric

Comment on this post in the forums

Posted in Collaborative benchmark development, HDXPRT 2012, HDXPRT 2013, HDXPRT Webinar, Let us know your thoughts |

What you’re going to need

By Eric Hale

on September 27, 2012

We’re planning to release the source code for HDXPRT 2012 soon after we release the update for Windows 8, probably in mid-November. The source code will be freely available to the members of the community. In preparation for that, here’s some information about what you’ll need to build the benchmark.

As we’ve discussed before, HDXPRT is a complicated entity, with a test harness, multiple workloads, and an installer. Consequently, you’ll need several tools to edit and build
HDXPRT 2012:

• Microsoft Visual Studio 2005
• Microsoft SDK, (version 6.0.6000.0)
• Microsoft .Net Framework 2.0 SDK x64
• AutoIT v3
• InstallShield 2012 Standalone version. Note: The InstallShield Professional Version is required to edit the install script.
• Visual Build Pro 7 Note: It is possible to edit and build HDXPRT without Visual Build Pro. However, Visual Build Pro enables an automated build process.

If you install these products per their instructions, your system will be ready to build HDXPRT 2012 when the source code becomes available.

Remember, the HDXPRT 2013 suggestion period starts Monday. We’ll open a section of the forum for suggestions then. However, if you want to get an early start, feel free to go ahead and send suggestions to hdxprtsupport@hdxprt.com.

Eric

Comment on this post in the forums

Posted in HDXPRT 2012, HDXPRT development process, HDXPRT source code, Windows 8 |

Working towards Windows 8

By Eric Hale

on September 20, 2012

This past Wednesday, Bill hosted a Webinar to discuss HDXPRT 2012. He covered a lot of material. We’ll make a recording of it available on the site fairly soon.

During the Webinar, Bill mentioned that we’re working on a patch to let HDXPRT run on Windows 8. We have begun testing this patch. However, given the high level of interest in the community about testing HDXPRT on Windows 8, we are going to offer the patch on Friday to any community members that want to try it on an “as is” basis.

Using the patch is straightforward. Having installed HDXPRT on a Windows 8 system, you copy a few files to the HDXPRT\Bin folder, run a DOS script, and reboot. At that point, HDXPRT should run on the Windows 8 system. We will include detailed instructions with the download.

The patch should have no impact on the scores. This means you can compare results from Windows 8 systems with the results you already have from Windows 7 systems.

We hope that you will try HDXPRT on Windows 8 and let us know what you see. We’ll use your feedback as we finalize the update of HDXPRT 2012 that will fully support Windows 8.

When the update is available, we’ll post to the community forum, tweet, and put a notice on the Web page.

In other news, there’s a post on the forum that gives instructions for getting more detailed timing information from HDXPRT. Community members can read that post here: How to get more detailed timing information from HDXPRT 2012

Finally, the comment period for HDXPRT 2013 starts October 1. Be thinking about what you’d like to see in HDXPRT 2013!

Eric

Update: The prerelease Patch for Windows 8 is now available. You can download it here.

Comment on this post in the forums

Posted in HDXPRT 2012, HDXPRT metrics, HDXPRT Webinar, Let us know your thoughts, Uncategorized, Windows 8 |

TouchXPRT update

By Bill Catchings

on September 13, 2012

We have been spending a lot of our energy (and blog space) on HDXPRT 2012 over the last few weeks. We have not, however, forgotten about TouchXPRT.

As we discussed in the past, TouchXPRT will be a new benchmark designed for touch-based devices like tablets and touch-capable Ultrabooks. It will incorporate apps and scenarios tailored to touch-based environments and devices. The initial version will run on the Windows 8 environment formerly known as Metro.

Our current plan is to release an informal design overview later this month. We then hope to have a preview version of TouchXPRT available to community members in October. Depending upon feedback, we will work on improvements and fixes through January with a release in February. We will have more details in the forums and in blog entries over the next few weeks.

As always, we need feedback and input from the community. If you are not currently a member, now is the time to join so you can participate in specifying and developing TouchXPRT. We are excited about both the benchmark and this emerging product space. We hope you are, too!

I do have one thing to mention about HDXPRT 2012. Please remember that we are going to do a Webinar next week (Tuesday, 9/18 at 2:00pm ET). We plan to discuss HDXPRT 2012 and have plenty of time for questions from attendees. We’d love for you to join, learn more about the benchmark, and give us your thoughts and feedback. We will send out more details to community members in an email later this week.

Bill

Comment on this post in the forums

Posted in HDXPRT Webinar, TouchXPRT, TouchXPRT development process, TouchXPRT release cycle |

Real performance for real people

By Eric Hale

on September 6, 2012

As you know by now, HDXPRT 2012 uses real commercial applications and realistic workloads to evaluate PC performance. We believe this approach, where we used the same software you do, and used it to do the same types of work you do, was the best way to relate a PC’s performance to the experience you’ll have.

The point of all this has been to help you answer some of the questions you’d confront when buying a PC. How much of a performance boost does adding memory really give you? Is the faster RAM worth it? How much faster is a new Intel Core i7 processor than a second-generation processor?

This Friday, we’re publishing the HDXPRT 2012 v 1.0 Scaling White Paper, which looks at exactly these issues. To prepare the paper, we took a single system and ran dozens of tests, changing one variable at a time. This let us isolate the effect each component had on performance. We show the effect of changing the following:
• The processor: We compared half a dozen second- and third-generation Intel Core i3, i5, and i7processors.
• The amount of memory: We compared the effect of having 2, 4, 8, and 16 GB of RAM.
• Memory speed and banking: We compared 1,333MHz and 1,600MHz DDR3 RAM. We also looked at the effect of using a single DIMM vs. using two smaller DIMMs for the same total memory amount.
• Drive type: We compared traditional hard disk drives and solid state drives.

Not all workloads reacted the same, so our paper presents the effect of each change on the component scores as well as on the Create HD score. This is the type of information everyone buying a PC needs.

We’re going to have a Webinar at 2 pm ET on Tuesday September 18. We’ll talk about HDXPRT 2012 and take your questions. Details to come.

In addition, we have a new video introducing HDXPRT to system vendors and testers. I hope you enjoy it.

Eric

Comment on this post in the forums

Posted in HDXPRT 2012 results, HDXPRT 2012 results whitepaper, HDXPRT Webinar |

Looking for a winner

Make sure your voice is heard

What you’re going to need

Working towards Windows 8

TouchXPRT update

Real performance for real people

Check out the other XPRTs: