BenchmarkXPRT Blog banner

Author Archives: Mark Van Name

Mobile World Congress 2016 and the need for more

Mark Van Name
By Mark Van Name
on February 25, 2016

Nothing shows you how much more bandwidth we need than a techie trade show like Mobile World Congress 2016. No matter how much the show’s organizers made available, the attendees swamped it with data from their many devices—phones, tablets, and PCs.

This show also demonstrated that we’re going to need a lot more of something else: device performance.

Some people like to say that our current devices are fast enough, but those people either weren’t at MWC or weren’t paying attention. New, demanding workloads were on display everywhere. High-end graphics. Support for an ever-growing range of wearables. Virtual reality. Augmented reality. The ability to act as a hub for all sorts of home automation devices. These and other new capabilities place ever-increasing demands on devices—and they’re all just getting started.

As I walked all of the MWC buildings—and I did at least walk by every single exhibit—I was struck again and again by how many cool technologies are on the cusp of being ready for prime time. They’ll bring nifty features to our everyday lives, and they’ll place heavy demands on our devices to support them and enable them to run well.

Some devices will handle these demands better than others, but we won’t be able to tell the winners from the losers just by looking at them. We’ll need reliable, relevant, real-world benchmarks to sort the winners from the posers—and that means we’ll need the XPRTs.  We’ll need the XPRTs we have today, and we’ll need new XPRTs and/or new XPRT workloads for the future. We’ll need help from everyone—members of the BenchmarkXPRT Development Community and vendors yet to join it—to create these new tools, so that buyers everywhere can make smart purchase decisions.

It’s an exciting time. The future for tech, for devices, and for the XPRTs is bright.  Let’s get busy creating it.

Posted in BenchmarkXPRT, BenchmarkXPRT development community, Future of performance evaluation, Internet of things, IoT, Mobile devices, Mobile World Congress, PCs, Performance benchmarking, Trade Shows |

Meeting the old fashioned way

Mark Van Name
By Mark Van Name
on August 30, 2012

As Bill discussed last week, we work in many different ways to build the community. Facebook, Webinars, Twitter, this blog—all of these social media venues help connect those of us in the community.

Sometimes, though, it’s nice to sit down in person and chat.

Which is precisely what I’m inviting fellow community members to do at the upcoming Intel Developer Forum (IDF). IDF runs from September 11 to 13 at the Moscone West Convention Center in San Francisco. While I’ll be busy attending the show for part of each day, I’d love to spend some of my free time talking with people about both HDXPRT and TouchXPRT. We could do it informally, by simply running across each other on the show floor, or we could schedule some meetings.

We could even make a group lunch of it. If a few folks can agree on a day and time, I’d be happy to set up and host a lunch at a nearby restaurant so we can share ideas while enjoying a meal.

If you’re interested, feel free to email me, and we’ll arrange to get together. I hope to have many good conversations with community members at the show!

Mark Van Name

Comment on this post in the forums

Posted in Benchmarking, Collaborative benchmark development, Let us know your thoughts |

Waiting sucks

Mark Van Name
By Mark Van Name
on May 25, 2011

You know it does.  Time is the most precious commodity, the one thing you can never get back.  So when someone or something makes you wait, it sucks.

It particularly sucks when you have to wait on your PC.  It’s your computer, after all, and it should do the work and be quick about it.  For many tasks, it is quick, almost instantaneous.  Some, though, require so much work that the computer can spend a lot of time doing them, leaving you waiting. Tasks that involve working with different types of media often fall into that category.

Which is exactly why we have HDXPRT.

It gives you a way to compare how long different PCs require to perform some common media-manipulation tasks.  Because those times can be significant—sometimes many seconds, but also sometimes many minutes—HDXPRT can give you valuable information that you can factor into your PC buying plans.

After all, the faster a PC is at this sort of work, the less time you’ll spend waiting on it—and that’s a good thing.

Mark Van Name

Comment on this post in the forums

Posted in HDXPRT metrics, HDXPRT workloads |

Top 5 reasons for meeting us at Computex in Taipei

Mark Van Name
By Mark Van Name
on May 18, 2011

As I’ve mentioned before, Bill Catchings from PT will be at the upcoming Computex show in Taipei to debut HDXPRT 2011. At the same time, back home in North Carolina we’ll be mailing copies of the benchmark DVDs to all the members of the HDXPRT Development Community.

If you’re one of the lucky folks who gets to attend Computex, we’d love it if you would come by Bill’s room in the Hyatt (we’ll publicize the room number as soon as we know it), see the benchmark in action, and give us your thoughts about it. I know the show is huge and full of attractions, so I thought I’d give you the top five reasons you ought to make room in your schedule to visit with us.

5. Free snacks! We don’t know what they are yet, or even how we’ll persuade the hotel to let us have them, but we’re committed to providing something to quench your thirst and something to quell your hunger.

4. A break from the crowds. Not only do you get to sit, drink, eat, and see a great new benchmark, you get to do so in the quiet and luxury of a Taipei hotel suite. No more bumping shoulders with fellow show attendees or fighting to get to a place quiet enough that you can talk; in that room, you can relax.

3. You can affect the industry! The support for HDXPRT is growing. More and more organizations are using it. We don’t just want to show it to you; we want you to tell us what you think about it. Your opinions count, and they could help drive the design of the next version of the benchmark, HDXPRT 2012. Yeah, that’s right: the one in development isn’t out, and I’m already talking about the next one. Sue me: I like to live on the edge.

2. You don’t want to make Bill cry. Imagine him, sitting alone in the room, laptop humming, ready to demonstrate this cool new testing tool, and no one to keep him company. His sadness would be so unbearable that I can’t bear to think of what he might do. You can’t let that happen.

1. It’s way cooler to get your HDXPRT DVDs in person! That’s right: Bill’s not just going to show you the benchmark, he’s going to give you your very own copy! He’ll probably shake your hand, too, and thank you for coming. Admit it: that’s cooler than getting it in the mail (which is also pretty darn good—and which will happen to you if you join the HDXPRT Development Community).

Mark Van Name

Comment on this post in the forums

Posted in HDXPRT Development Community benefits |

What to do, what to do

Mark Van Name
By Mark Van Name
on May 11, 2011

When you set out to build an application-based benchmark like HDXPRT, you face many choices, but two are particularly important:  what applications do you run, and what functions do you perform in each application?

With HDXPRT the answers were straightforward, as they should be.

The applications we chose reflected a blend of market leaders, those providing emerging but important features, and the input from our community members.

The functions we perform in each application are ones that are representative of common uses of those programs—and that reflect the input of the community.

What’s so important here is the last clause of each of those paragraphs:  your input defines this benchmark.

As we finish off HDXPRT 2011 and then move to the 2012 version, we’ll begin the development cycle anew. When we do, if you want to make sure we choose the applications and functions that matter most to you, then participate, tell us what you want, let us hear your voice.  We will respond to all input, so though we can’t guarantee to accept all direction—after all, goals and desires sometimes conflict—we can guarantee that you will hear back from us and that we will explain the rationale for our decisions.

Mark Van Name

Comment on this post in the forums

Posted in Collaborative benchmark development, HDXPRT 2012, HDXPRT development process |

Putting HDXPRT in some benchmark context

Mark Van Name
By Mark Van Name
on May 6, 2011

Benchmarks come in many shapes and sizes.  Some are extremely small, simple, and focused, while others are large, complex, and cover many aspects of a system.  To help position HDXPRT in the world of benchmarks, let me share with you a little taxonomy that Bill and I have long used.  No taxonomy is perfect, of course, but we’ve found this one to be very helpful as a general categorization tool.

From the perspective of how benchmarks measure performance, you can divide most of them into three groups.

Inspection tools use highly specialized tests to target very particular parts of a system. Back in the day, lo these many decades ago—okay, it was only two decades, but in dog years two tech decades is like five generations—some groups used a simple no-op loop to measure processor performance. I know, it sounds dumb today, but for a short time many felt it was a legitimate measure of processor clock speed, which is one aspect of performance. Similarly, if you want to know how fast a graphics subsystem could draw a particular kind of line, you could write code to draw lines of that type over and over.

These tools have very limited utility, because they don’t do what real users do, but for people working close to hardware, they can be useful.

Moving closer to the real world, synthetic benchmarks are specially written programs that simulate the kinds of work their developers believe real users are doing. So, if you think your target users are spending all day in email, you could write your own mini email client and time functions in it.  These tools definitely move closer to real user work than inspection tools, but they still have the drawback of not actually running the programs real people are using.

Application-based benchmarks take that last step by using real applications, the same programs that users employ in the real world. These benchmarks cause those applications to perform the kinds of actions that real users take, and they time those actions.  You can always argue about how representative they are—more on that in a future blog entry, assuming I don’t forget to write it—but they are definitely closer to the real world because they’re using real applications.

With all of that background, HDXPRT becomes easy to classify:  it’s an application-based benchmark.

Mark Van Name

Comment on this post in the forums

Posted in Application-based benchmarks, Benchmarks in general, History of benchmarking, Performance benchmarking |

Check out the other XPRTs:

Forgot your password?
© Principled Technologies, Inc. 2024   |   Privacy policy

Cookie Notice: Our website uses cookies to deliver a smooth experience by storing logins and saving user information. By continuing to use our site, you agree with our usage of cookies as our privacy policy outlines.

x