BenchmarkXPRT Blog banner

Thinking ahead at CES 2018

Justin Greene
By Justin Greene
on January 11, 2018

It may sound trite to say that a show like CES is all about the future, but this year’s show is prompting me to think about how our lives will evolve in the coming years. Some technological breakthroughs change the way we do everyday things like play music or hail a cab—while some transform the way we do everything. For technological innovation to truly shift society on a wide scale, it has to coincide with markets of scale in a way that makes life-changing tech accessible to almost everyone (think: smartphones in 2005 versus smartphones in 2018).

These technical and economic forces are coinciding once again in the areas of AI, automation, the Internet of Things (IoT), and consumer robotics. While many of our daily activities will stay the same, the ways we organize and engage with those activities are changing dramatically.

I’ll leave you with a few general observations from the show:

  • Huawei has a huge presence here. A new tagline for them that I haven’t seen before is a play on their name, “Wow Way.” I suspect we may have a Mate 10 Pro XPRT Spotlight entry in the near future.
  • The Kino-mo Hypervsn 3D Holograph display blew me away. People were crowding in to see it and couldn’t stop staring. It’s straight out of sci-fi, and its appearance is similar to Princess Leia’s hologram message in Star Wars. (By the way, it looks way better in real life than in the video.)
  • Sony is making a big push into smart homes by building systems that work cross-platform with a range of smart speakers and assistants. Between their smart home push and some of the cool home theater tech they had on display, I can see them gaining some brand power.
  • To me, the most exciting concepts at the show involved smart infrastructure, which promises enormous potential to boost the efficient distribution of water, energy, and transportation resources.
  • Surprisingly, I saw automation, smart city, and smart infrastructure displays from companies that I don’t always associate with IoT or AI, like Bosch and Panasonic. Panasonic was marketing an array of semi-autonomous vehicle cockpit prototypes, and had a section highlighting their partnership with Tesla.
  • By far, the strangest thing I’ve seen at CES has been the Psychasec booth, staffed by eerie attendants in pure white outfits who talked confidently about “downloading your cortical stack into customized bodies made from organic materials.” The Psychasec staff absolutely refused to break character, which made the whole scene even stranger. Check out the link above for the story behind Psychasec.



And, while I’m probably not supposed to admit this, my favorite part of the show so far has been the line of expensive massage chair vendors doling out free sessions…

More to follow soon,

Justin

Posted in AI, Automated cars, Automation, CES, Holograph, Internet of things, IoT, Trade Shows |

Gearing up for a busy year ahead

Justin Greene
By Justin Greene
on January 4, 2018

We hope everyone’s 2018 kicked off on a happy note, and you’re starting the year rested, refreshed, and inspired. Here at the XPRTs, we already have a busy slate of activity lined up, and we want to share a quick preview of what community members can expect in the coming months.

Next week, I’ll be travelling to CES 2018 in Las Vegas. CES provides us with a great opportunity to survey emerging tech and industry trends, and I look forward to sharing my thoughts and impressions from the show. If you’re attending this year, and would like to meet and discuss any aspect of the XPRTs, let me know.

There’s also more WebXPRT news to come. We’re working on several new features for the WebXPRT Processor Comparison Chart that we think will prove to be useful, and we hope to take the updated chart live very soon. We’re also getting closer to the much-anticipated WebXPRT 3 general release! If you’ve been testing the WebXPRT 3 Community Preview, be sure to send in your feedback soon.

Work on the next version of HDXPRT is progressing as well, and we’ll share more details about UI and workload updates as we get closer to a community preview build.

Last but not least, we’re considering the prospect of updating TouchXPRT and MobileXPRT later in the year. We look forward to working with the community on improved versions of each of those benchmarks.

Justin

Posted in Uncategorized | Tagged , , , , |

Machine learning in 2018

Bill Catchings
By Bill Catchings
on December 21, 2017

We are almost to the end of 2017 and, as you have probably guessed, we will not have a more detailed proposal of our machine learning benchmark ready by the end of the year.

The key aspects of the benchmark proposal we wrote about a few months ago haven’t changed, but we are running behind schedule. We are still hoping to have the proposal ready in Q1 2018 and the tool based on that proposal later in the year. We will keep you posted.

In the meantime, we hope you enjoy as much as we did the recent CGP Grey tech video explanation of machine learning. There are actually two videos—the first one gives a general overview and then the second one does a better job of looking at the current state of machine learning. It talks mainly about the training aspects of machine learning rather than the inference aspects that we are looking into with AIXPRT/MLXPRT.

From all of us in the BenchmarkXPRT Development Community, we hope you and yours have a wonderful holiday and a great start to 2018!

Bill

Posted in BenchmarkXPRT development community, Collaborative benchmark development, Future of performance evaluation, Machine learning, Performance benchmarking | Tagged , |

The WebXPRT 3 Community Preview is here!

Justin Greene
By Justin Greene
on December 14, 2017

Today we’re releasing the WebXPRT 3 Community Preview (CP). As we discussed in the blog last month, in the new version of WebXPRT, we updated the photo-related workloads with new images and a new deep learning task for the Organize Album workload. We also added an optical character recognition task to the Local Notes workload and combined a portion of the DNA Sequence Analysis scenario with a writing sample/spell check scenario to simulate an online homework hub in the new “Online Homework” workload.

Also, longtime WebXPRT users will immediately notice a completely new, but clean and straightforward, UI. We’re still tweaking aspects of the UI and implementing full functionality for certain features such as social media sharing and German language translation, but we don’t anticipate making any significant changes to the overall test or individual workloads before the general release.

As with all community previews, the WebXPRT 3 CP is available only to BenchmarkXPRT Development Community members, who can access the link from the WebXPRT tab in the Members’ Area.

After you try the WebXPRT 3 CP, please send us your comments. Thanks and happy testing!

Justin

Posted in BenchmarkXPRT development community, Browser-based benchmarks, Collaborative benchmark development, Community Preview, Cross-platform benchmarks, Future of performance evaluation, German, Performance benchmarking, WebXPRT, WebXPRT 3 |

News about WebXPRT and BatteryXPRT

Justin Greene
By Justin Greene
on December 7, 2017

Last month, we gave readers a glimpse of the updates in store for the next WebXPRT, and now we have more news to report on that front.

The new version of WebXPRT will be called WebXPRT 3. WebXPRT 3 will retain the convenient features that made WebXPRT 2013 and WebXPRT 2015 our most popular tools, with more than 200,000 combined runs to date. We’ve added new elements, including AI, to a few of the workloads, but the test will still run in 15 minutes or less in most browsers and produce the same easy-to-understand results that help compare browsing performance across a wide variety of devices.

We’re also very close to publishing the WebXPRT 3 Community Preview. For those unfamiliar with our open development community model, BenchmarkXPRT Development Community members have the ability to preview and test new benchmark tools before we release them to the general public. Community previews are a great way for members to evaluate new XPRTs and send us feedback. If you’re interested in joining, you can register here.

In BatteryXPRT news, we recently started to see unusual battery life estimates and high variance when running battery life tests at the default length of 5.25 hours. We think this may be due to changes in how new OS versions are reporting battery life on certain devices, but we’re in the process of extensive testing to learn more. In the meantime, we recommend that BatteryXPRT users adjust the test run time to allow for a full rundown.

Do you have questions or comments about WebXPRT or BatteryXPRT? Let us know!

Justin

Posted in AI, Battery life, BatteryXPRT 2014 for Android, BenchmarkXPRT, BenchmarkXPRT development community, Browser-based benchmarks, Community Preview, Cross-platform benchmarks, Future of performance evaluation, Performance benchmarking, WebXPRT, WebXPRT 3 |

How to submit results for the WebXPRT Processor Comparison Chart

Justin Greene
By Justin Greene
on November 30, 2017

The WebXPRT 2015 Processor Comparison Chart is in its second month, and we’re excited to see that people are browsing the scores. We’re also starting to receive more WebXPRT score submissions for publication, so we thought it would be a good time to describe how that process works.

Unlike sites that publish any results they receive, we hand-select results from internal lab testing, end-of-test user submissions, and reliable tech media sources. In each case, we evaluate whether the score is consistent with general expectations. For sources outside of our lab, that evaluation includes checking to see whether there is enough detailed system information to get a sense of whether the score makes sense. We do this for every score on the WebXPRT results page and the general XPRT results page.

If we decide to publish a WebXPRT result, that score automatically appears in the processor comparison chart as well. If you would like to submit your score, the submission process is quick and easy. At the end of the WebXPRT test run, click the Submit button below the individual workload scores, complete the short submission form, and click Submit again. The screenshot below shows how the form would look if I submitted a score at the end of a WebXPRT run on my personal system.

WebXPRT results submission

After you submit your score, we’ll contact you to confirm the name we should display as the source for the data. You can use one of the following:

  • Your first and last name
  • “Independent tester,” if you wish to remain anonymous
  • Your company’s name, provided that you have permission to submit the result in their name. If you want to use a company name, we ask that you provide your work email address.


We will not publish any additional information about you or your company without your permission.

We look forward to seeing your score submissions, and if you have suggestions for the processor chart or any other aspect of the XPRTs, let us know!

Justin

Posted in Browser-based benchmarks, Performance benchmarking, Web-based testing, WebXPRT, WebXPRT 2015, WebXPRT Processor Comparison Chart |

Check out the other XPRTs:

Forgot your password?
© Principled Technologies, Inc. 2025   |   Privacy policy

Cookie Notice: Our website uses cookies to deliver a smooth experience by storing logins and saving user information. By continuing to use our site, you agree with our usage of cookies as our privacy policy outlines.

x