BenchmarkXPRT Blog banner

Category: CES

CES 2017

I’ve attended many tech shows over the years, but this year’s CES has more energy than any I’ve attended in a long time. Part of the energy is the breadth of products. There are amazingly slim TVs that make my TV at home, which I thought was slim, look fat. And, there are beautiful 8K TVs that make my new 4K one feel old.

I’m seeing all manner of smartphones. I’m seeing mobile remote presence devices such as the one from Beam, and after seeing the latest Fenix 5 from Garmin, I think I’ve found my next smartwatch.

Photo Jan 05, 2 05 53 PM

There are many differing devices and approaches to VR and AR. There are drones everywhere. And, lots of massage chairs.

Photo Jan 05, 1 13 17 PM

There are also plenty of products, let’s call them longshots, that contribute to the Wild West feel of the show. Maybe you’d like the Hydreon FakeTV, a small device to make it seem like a TV is on in your house to keep away burglars? Or Dr. Fuji’s Body Shaper, a vibrating platform to “accelerate your workout”?

Another part of the dynamic feeling is the breadth of vendors. Almost all the big tech vendors are here, except for Apple, of course. The excitement for me, however, is the small vendors displaying things that may well never see the light of day, but give glimpses of the future. For example, while I doubt most of the drone vendors at CES will be around in a few years, I think the trend toward small, inexpensive selfie drones will be.

The main reason for the energy at this year’s CES could be the convergence of multiple big industries. The most obvious example of this phenomenon is the large auto presence. Cars have been at CES before—the first time I drove my current car, a BMW i3, was at CES 2014—but this time around they seem to really want to make a statement. Faraday Future is making a splash by trying to be the next Tesla with its FF91.

Photo Jan 05, 3 08 30 PM

Multiple vendors, including VW, Nissan, Toyota, and Mercedes, have concept cars on display—most of them are electric and all of them are heavy on technology. The biggest tech they’re touting is autonomous driving. The auto companies are showing their products while companies like NVIDIA, Intel, and Magic Eye are displaying the tech they have as well.

Regardless of the source of the energy at this CES, I see many opportunities for the existing XPRTs to continue to be important resources. I also see how important emerging technologies like machine learning and VR/AR are going to be and how the XPRTs can be of help there as well.

Exciting times!


Creating a machine-learning benchmark

Recently, we wrote about one of the most exciting emerging technology areas, machine learning, and the question of what role the XPRTs could play in the field.

Experts expect machine learning to be the analytics backbone of the IoT data explosion. It is a disruptive technology with potential to influence a broad range of industries. Consumer and industrial applications that take advantage of machine-learning advancements in computer vision, natural language processing, and data analytics are already available and many more are on the way.

Currently, there is no comprehensive machine-learning or deep-learning benchmark that includes home, automotive, industrial, and retail use cases. The challenge with developing a benchmark for machine learning is that these are still the early days of the technology. A fragmented software and hardware landscape and lack of standardized implementations makes benchmarking machine learning complex and challenging.

Based on the conversations we’ve had over the last few weeks, we’ve decided to take on that challenge. With the community’s help, of course!

As we outlined in a blog entry last month, we will work with interested folks in the community, key vendors, and academia to pull together what we are internally calling MLXPRT.

While the result may differ substantially from the existing XPRTs, we think the need for something is great. Whether that will turn out to be a packaged tool or just sample code and workloads remains to be seen.

What we need most your help. We need both general input about what you would like to see as well as any expertise you may have. Let us know any questions you may have or ways you can help.

On a related note, I’ll be at CES 2017 in Las Vegas during the first week of January. I’d love to meet and talk more about machine learning, benchmarking, or the XPRTs. If you’re planning to be there and would like to connect, let us know.

We will not have a blog entry next week over the holidays, so we wish all of you a wonderful time with your families and a great start to the new year.


Quarterly review

It’s been one of our busiest quarters ever! Here’s a quick review of what’s been happening:

The XPRTs were on the road a lot!

While I was at CES, I was lucky enough to be able to sit down and talk on-the-record with a couple of community members:

Many thanks to them for being so generous with their time and their insights.

We also gave folks a lot to look at:

That is a great start to the year, but we’re going to top it – Next week, we’ll kick off Q2 with one of our biggest announcements ever!


Is it hot in here?

One of the great meetings I had at CES was with another community member, Patrick Chang, Senior System Engineer in the Tablet Performance group at Dell.  I was glad to hear that, when he tests his devices, Patrick makes frequent use of TouchXPRT.

While TouchXPRT stresses the system appropriately, Patrick’s job requires him to understand not only how well the device performs, but why it performs that way. He was wondering what we could do to help him correlate the temperature, power consumption, and performance of a device.

That information is not typically available to software and apps like the XPRTs. However, it may be possible to add some hooks that would let the XPRTs coordinate with third-party utilities and hardware that do.

As always, the input from the community guides the design of the XPRTs. So, we’d love to know how much interest the community has in having this type of information. If you have thoughts about this, or other kinds of information you’d like the XPRTs to gather, please let us know!


Last week in the XPRTs
We published the XPRT Weekly Tech Spotlight on the Apple iPad Pro.
We added one new BatteryXPRT ’14 result.
We added one new CrXPRT ’15 result.
We added one new MobileXPRT ’13 result.
We added four new WebXPRT ’15 results.

Comparing apples and oranges?

My first day at CES, I had breakfast with Brett Howse from AnandTech. It was a great opportunity to get the perspective of a savvy tech journalist and frequent user of the XPRTs.

During our conversation, Brett raised concerns about comparing mobile devices to PCs. As mobile devices get more powerful, the performance and capability gaps between them and PCs are narrowing. That makes it more common to compare upper-end mobile devices to PCs.

People have long used different versions of benchmarks when comparing these two classes of devices. For example, the images for benchmarking a phone might be smaller than those for benchmarking a PC. Also, because of processor differences, the benchmarks might be built differently, say a 16- or 32-bit executable for a mobile device, and a 64-bit version for a PC. That was fine when no one was comparing the devices directly, but can be a problem now.

This issue is more complicated than it sounds. For those cases where a benchmark uses a dumbed-down version of the workload for mobile devices, comparing the results is clearly not valid. However, let’s assume that the workload stays the same, and that you run a 32-bit benchmark on a tablet, and a 64-bit version on a PC. Is the comparison valid? It may be, if you are talking about the day-to-day performance a user is likely to encounter. However, it may not be valid if you are making statement about the potential performance of the device itself.

Brett would like the benchmarking community to take charge of this issue and provide guidance about how to compare mobile devices and PCs. What are your thoughts?


In the spotlight

I’m happy to be back in North Carolina, but I had a really great time at CES. I talked to over a dozen companies about the XPRTs and the XPRT Weekly Tech Spotlight, and had some good conversations. Hopefully, some of these companies’ devices will be among the first ones we showcase when the XPRT Weekly Tech Spotlight goes live next month.

Of course, I saw some really great tech at CES! Amazing TVs and cars, magic mirrors, all kinds of drones, and the list goes on. Before the show, the Internet of Things was predicted to be big this year, and boy, was it! Smart refrigerators, door locks, and thermostats were just the beginning. Some of my favorite examples were the chopsticks and footbath—both Bluetooth enabled—and “the world’s first remote controlled game shoe.”

Clearly IoT is the Wild West of technology right now. We’re had some conversations about how the XPRTs might be able to help consumers navigate the chaos. However, with a class of products this diverse, there are a lot of issues to consider. If you have any thoughts about this, let us know!


Check out the other XPRTs:

Forgot your password?