Category: Let us know your thoughts

HDXPRT 2012 and Windows 8 follow-up

on June 15, 2012

We’re hoping you will help us test the HDXPRT 2012 beta on your latest systems under Windows 8.

As Bill discussed a couple weeks ago, the beta of HDXPRT 2012 has some problems with Windows 8 and consequently will not install. However, with the latest version of Windows 8 Consumer Preview, some of the issues seem to be better. Using some command-line options, you can get around the installation restrictions. That is what we have been doing in our lab. The complete details of how to install and run the beta on the Windows 8 Consumer Preview are in this forum post (login required).

(Don’t worry, we hate hidden options as well, and there should not be any in the shipping version!)

Please let us know what you find out on your systems, either in the forum or via email. We have encountered some crashes in apps and we’d like to know what you find. We provide some instructions for running individual workloads in case you need to work around such an issue. We are also interested in hearing if you see any performance differences between Windows 7 and Windows 8 that don’t make sense to you. While the Consumer Preview license does not allow the publishing of performance results, we still need to start the process of understanding what is going on with Windows 8.

Thank you so much for your efforts to make HDXPRT 2012 as good as possible!

Eric

Comment on this post in the forums

HDXPRT 2012 and Windows 8 follow-up

By Bill Catchings

on June 8, 2012

HDXPRT 2012 and Windows 8 follow-up

As I discussed a couple weeks ago, the beta of HDXPRT 2012 has some problems with Windows 8 and consequently will not install. However, with the latest version of Windows 8 Consumer Preview, some of the issues seem to be better. Using some command-line options, you can get around the installation restrictions. That is what we have been doing in our lab. (Don’t worry, we hate hidden options as well, and there should not be any in the shipping version!) We’re hoping you can help us test the HDXPRT 2012 beta on your latest systems under Windows 8.

The complete details of how to install and run the beta on the Windows 8 Consumer Preview are in this forum post. (login required) Please let us know what you find out on your systems, either in the forum or via email. We have encountered some crashes in apps and we’d like to know what you find. We provide some instructions for running individual workloads in case you need to work around such an issue. We are also interested in hearing if you see any performance differences between Windows 7 and Windows 8 that don’t make sense to you. While the Consumer Preview license does not allow the publishing of performance results, we still need to start the process of understanding what is going on with Windows 8.

As always, thanks for your efforts to make HDXPRT 2012 as good as possible!

Bill

Comment on this post in the forums

Now it’s your turn

By Bill Catchings

on June 1, 2012

We mailed the DVDs for the HDXPRT 2012 this week. Kudos to the development and test teams. Thanks to all of the community members who confirmed their addresses. You should receive the DVDs soon. Now the fun begins.

The comment period for the beta goes through June 29. Your feedback is essential. Please install the benchmark, run it, and let us know what you think. If you have questions, we’ll answer them as quickly as possible. If you find a problem, we want to fix it. If there’s something you don’t like, we want to know. If there’s something you do like, we’d like to know that too.

If you don’t receive your DVDs next week, please contact us and we’ll make sure you get your copy ASAP. Unfortunately, the HDXPRT 2012 beta is not available for download. As we explained in an earlier blog post, the benchmark is approximately 10 GB.

Remember, the end of the comment period is June 29. Please help us make HDXPRT 2012 the best possible benchmark.

Eric

Comment on this post in the forums

Posted in HDXPRT 2012 beta, Let us know your thoughts |

Today is a milestone

By Bill Catchings

on May 18, 2012

Today is a milestone. We are freezing the beta for HDXPRT 2012. It’s taken a lot of work–developing something with this many moving parts is a bit like herding cockroaches. However, the workloads are set and the benchmark seems stable.

So, what happens next?

First, the beta goes to a team who will spend a few days intensively testing it. The test team is isolated from the development team because, let’s face it, developers can’t test their own code. (I should know. I’ve developed a lot of products in my time.)

We’ll run it on a variety of systems, checking it for stability and repeatability. We will also look at the results and make sure that they are sensible. That is, they reflect that capabilities of the systems we are running the benchmark on.

The beta is approximately 10 GB. It includes installers for nine applications and a number of large content files. Because of its size, the beta will only be available on DVD. Once the tests look good, we’ll start reproducing and verifying the DVDs. Once the DVDs are verified, we’ll mail them out. The whole process should take about a week, so community members should have their beta DVDs within a couple of weeks.

While the testing is happening, we’ll be contacting each community member to confirm that we have their correct physical mail address.

Once you have the DVDs, please run the benchmark and send feedback. We need your help to make this beta into a truly great release candidate. We will work hard to address any feedback we get. If you find a problem, let us know. If you have a complaint, let us know. It you have a compliment, by all means, let us know!

Next week, Bill will be back from his travels.

Eric

Comment on this post in the forums

Posted in HDXPRT 2012 beta, HDXPRT development process, Let us know your thoughts |

HDXPRT 2012 characterization study

By Bill Catchings

on May 4, 2012

For HDXPRT 2011, we did quite a bit of testing to characterize the benchmark. Those results appeared in an initial white paper and a follow up one. In the first, we ran tests on different processors, various amounts of RAM, internal vs. external graphics, hard disk vs. SSD, and the effects of Intel Turbo Boost Technology. In the follow up white paper, we looked in more depth at the effect of graphics cards with different processors.

I mentioned a couple weeks ago that we are starting to put together a testbed to help us characterize HDXPRT 2012. What is in that testbed will define what characteristics of the upcoming benchmark we measure. We would like to get your help defining that testbed.

Our current thinking is to do a similar set of tests this year with updated hardware. However, plenty of additional things would be interesting to look at. First, I would like to increase the range of processors we test, including AMD processors. I would also like to do some testing varying different processor characteristics such as threads, cores, and frequency. It might also be good to look at the effect of new technologies like hybrid drives (which combine a small SSD with a hard disk to try and have the best of both).

We face two challenges in doing these characterization tests. One is to try and change one only variable at a time. That is very difficult in some cases, such as comparing Intel and AMD processors—you can’t just swap them in the same motherboard. Fortunately, it is usually possible to find very similar motherboards and keep other components (like disks, graphics, and RAM) constant. The other challenge is getting all of the necessary hardware in house.

So, we have two requests for you. First, let us know what you would like to see us test. Second, help us by supplying some of that equipment. If you supply the equipment we will do our best to include results from it in the characterization study and in the new HDXPRT 2012 results database. As always, thanks for your help!

Bill

Comment on this post in the forums

Posted in HDXPRT 2012, HDXPRT development process, HDXPRT workloads, Let us know your thoughts |

HDXPRT 2012 – Testing the test

By Bill Catchings

on April 20, 2012

We’re currently starting testing on alpha versions of HDXPRT 2012. In order to do that, we’re putting together a testbed. We have two goals for the testbed that are somewhat contradictory. The first is to make the testbed as diverse as possible in terms of vendors and configurations. We want notebooks and desktops from as many vendors as possible. We want to make sure we have systems that will push the edges—both slower systems that may even be below the minimum recommended configuration and faster ones representing the current latest and greatest. These systems will help us shake out bugs and provide some raw data that we can publish when the benchmark debuts in the new results database.

The second goal for the testbed is to have systems where we can easily change one variable at a time to help us understand the characteristics of the benchmark. Typically, these are white box systems where we can swap processors, disks, RAM, and so on. We will use the results from these systems in the benchmark characterization white paper we will create for the debut of HDXPRT 2012.

We’d like your opinions on what we should be certain to test. We think we have a good handle on what to include, but we want your ideas as well.

We also are looking for additional systems to include in our testbed. If you can supply some, please let me know. That is one way to make sure HDXPRT 2012 works on your system and to get your results in the results database. Rest assured, we will not publish those results without your permission. Regardless, the more systems we can test, the better the final product will be.

There will, of course, be opportunities for you to help with the testing as we get to the beta stage in the near future.

Bill

Comment on this post in the forums

Posted in HDXPRT 2012, HDXPRT development process, Let us know your thoughts |

Category: Let us know your thoughts

HDXPRT 2012 and Windows 8 follow-up

HDXPRT 2012 and Windows 8 follow-up

Now it’s your turn

Today is a milestone

HDXPRT 2012 characterization study

HDXPRT 2012 – Testing the test

Check out the other XPRTs: