BenchmarkXPRT Blog banner

Category: Trade Shows

Browser-based AI tests in WebXPRT 4: face detection and image classification

I recently revisited an XPRT blog entry that we posted from CES Las Vegas back in 2020. In that post, I reflected on the show’s expanded AI emphasis, and I wondered if we were reaching a tipping point where AI-enhanced and AI-driven tools and applications would become a significant presence in people’s daily lives. It felt like we were approaching that point back then with the prevalence of AI-powered features such as image enhancement and text recommendation, among many others. Now, seamless AI integration with common online tasks has become so widespread that many people unknowingly benefit from AI interactions several times a day.

As AI’s role in areas like everyday browser activity continues to grow—along with our expectations for what our consumer devices should be able to handle—reliable AI-oriented benchmarking is more vital than ever. We need objective performance data that can help us understand how well a new desktop, laptop, tablet, or phone will handle AI tasks.

WebXPRT 4 already includes timed AI tasks in two of its workloads: the “Organize Album using AI” workload and the “Encrypt Notes and OCR Scan” workload. These two workloads reflect the types of light browser-side inference tasks that are now fairly common in consumer-oriented web apps and extensions. In today’s post, we’ll provide some technical information about the Organize Album workload. In a future post, we’ll do the same for the Encrypt Notes workload.

The Organize Album workload includes two different timed tasks that reflect a common scenario of organizing online photo albums. The workload utilizes the AI inference and JavaScript capabilities of the WebAssembly (Wasm) version of OpenCV.js—an open-source computer vision and machine learning library. In WebXPRT 4, we used OpenCV.js version 4.5.2.

Here are the details for each task:

  • The first task measures the time it takes to complete a face detection job with a set of five 720 x 480 photos that we sourced from commercial photo sites. The workload loads a Caffe deep learning framework model (res10_300x300_ssd_iter_140000_fp16.caffemodel) using the commands found here
  • The second task measures the time it takes to complete an image classification job (labeling based on object detection) with a different set of five 718 x 480 photos that we sourced from the ImageNet computer vision dataset. The workload loads an ONNX-based SqueezeNet machine learning model (squeezenet.onnx v 1.0) using the commands found here.

To produce a score for each iteration of the workload, WebXPRT calculates the total time that it takes for a system to organize both albums. In a standard test, WebXPRT runs seven iterations of the entire six-workload performance suite before calculating an overall test score. You can find out more about the WebXPRT results calculation process here.

We hope this post will give you a better sense of how WebXPRT 4 measures one kind of AI performance. As a reminder, if you want to dig into the details at a more granular level, you can access the WebXPRT 4 source code for free. In previous blog posts, you can find information about how to access and use the code. You can also read more about WebXPRT’s overall structure and other workloads in the Exploring WebXPRT 4 white paper.

If you have any questions about this workload or any other aspect of WebXPRT 4, please let us know!

Justin

Looking back on 2023 with the XPRTs

Around the beginning of each new year, we like to take the opportunity to look back and summarize the XPRT highlights from the previous year. Readers of our newsletter are familiar with the stats and updates we include each month, but for our blog readers who don’t receive the newsletter, we’ve compiled highlights from 2023 below.

Benchmarks
In March, we celebrated the 10-year anniversary of WebXPRT! WebXPRT 4 has now taken the lead as the most commonly-used version of WebXPRT, even as the overall number of runs has continued to grow.

XPRTs in the media
Journalists, advertisers, and analysts referenced the XPRTs thousands of times in 2023. It’s always rewarding to know that the XPRTs have proven to be useful and reliable assessment tools for technology publications around the world. Media sites that used the XPRTs in 2023 include 3DNews (Russia), AnandTech, Benchlife.info (China), CHIP.pl (Poland), ComputerBase (Germany), eTeknix, Expert Reviews, Gadgetrip (Japan), Gadgets 360, Gizmodo, Hardware.info, IT168.com (China), ITC.ua (Ukraine), ITWorld (Korea), iXBT.com (Russia), Lyd & Bilde (Norway), Notebookcheck, Onchrome (Germany), PCMag, PCWorld, QQ.com (China), Tech Advisor, TechPowerUp, TechRadar, Tom’s Guide, TweakTown, Yesky.com (China), and ZDNet.

Downloads and confirmed runs
In 2023, we had more than 16,800 benchmark downloads and 296,800 confirmed runs. Users have run our most popular benchmark, WebXPRT, more than 1,376,500 times since its debut in 2013! WebXPRT continues to be a go-to, industry-standard performance benchmark for OEM labs, vendors, and leading tech press outlets around the globe.

Trade shows
In January, Justin attended the 2023 Consumer Electronics Show (CES) Las Vegas. In March, Mark attended Mobile World Congress (MWC) 2023 in Barcelona. You can view Justin’s recap of CES here and Mark’s thoughts from MWC here.

We’re thankful for everyone who used the XPRTs and sent questions and suggestions throughout 2023. We’re excited to see what’s in store for the XPRTs in 2024!

Justin

The Mobile World Congress 2023 recap video is live!

Once again, the talented studio team here at Principled Technologies has worked with the XPRT team to put together some great content. This week, we published a recap video of Mark’s recent trip to Mobile World Congress (MWC) 2023 in Barcelona. In the video, Mark discusses how “velocity” was the headline theme of this year’s MWC, but infrastructure was the true backbone of the show. The challenges of developing the infrastructure necessary for tomorrow’s tech aren’t limited to obvious topics like 5G deployment, bandwidth, and accessibility; they also include “soft” infrastructure topics like security, management, and reliability. As everything gets faster, we’ll need tools such as the XPRTs to provide reliable information about which devices can handle the increased demands of tomorrow’s tech.

We encourage you to check out the video, along with Mark’s blog post from the show and MWC-related photos on social media. To view the video, you can follow this link or click the screenshot below. If you attended or followed MWC this year and have any thoughts about how the XPRTs can help to evaluate cutting-edge technologies, we’d love to hear from you!

Justin

Mobile World Congress 2023: Infrastructure led the way

When the tech industry is at its best, a virtuous cycle of capabilities and use cases chases its own tail to produce ever-better tech for us all. Faster CPUs drive new usage models, which in turn emerge and swamp the CPUs, which then must get faster. Better screens make us want higher-quality video, which requires more bandwidth to deliver and causes us to desire even better displays. Apps connect us in more ways, but those connections require more bandwidth, which leads to new apps that can take advantage of those faster connections. And on and on.

Put a finger on the cycle at any given moment, and you’ll see that while all the elements are in motion, some are the stars of the moment. To keep the cycle going, it’s crucial for these areas to improve the most. At Mobile World Congress 2023 (#MWC23), that distinction belonged to infrastructure. Yes, some new mobile phones were on display, Lenovo showed off new ThinkPads, and other mobile devices were in abundance, but as I walked the eight huge halls, I couldn’t help but notice the heavy emphasis on infrastructure.

5G, for example, is real now—but it’s far from everywhere. Telecom providers have to figure out how to profitably build out the networks necessary to support it. The whole industry must solve the problems of delivering 5G at huge scale, handle the traffic increases it will bring, switch and route the data, and ultimately make sure the end devices can take full advantage of that data. Management and security remain vital whenever data is flying around, so those softer pieces of infrastructure also matter greatly.

Inevitably and always, to know if we as an industry are meeting these challenges, we must measure performance—both in the raw speed sense and in the broader sense of the word. Are we seeing the full bandwidth we expect? Are devices handling the data properly and at speed? Where’s the bottleneck now? Are we delivering on the schedules we promised? Questions such as these are key concerns in every tech cycle—and some of them are exactly what the XPRTs focus on.

As we improve our infrastructure, we hope to see the benefits at a personal level. When you’re using a device—whether it’s a smart watch, a mobile phone, or a laptop—you need it to do its job well, respond to you quickly, and show you what you want when you want it. When your device makes you wait, it can be helpful to know if the bandwidth feeding data to the device is the bottleneck or if the device simply can’t keep up with the flow of data it’s receiving. The XPRTs can help you figure out what’s going on, and they will continue to be useful and important as the tech cycle spins on. If history is our guide, the infrastructure focus of MWC23 will lead to greater capabilities that require even better devices down the line. We look forward to testing them.

The XPRTs will be at Mobile World Congress later this month!

Mobile World Congress (MWC) 2023 kicks off on February 27th, and we’re excited that Mark Van Name will be attending the event for the first time since the last pre-pandemic show in 2019. Each year, MWC offers a great opportunity to examine the new trends and technologies that will shape mobile technology in the years to come. The major themes of this year’s show include the latest advances in 5G and IoT technologies, along with what GSMA is calling “Reality+.” Reality+ refers to the intersection of AI, AR, VR, and 5G, and the potential impacts of these immersive technologies on our future.

Mark will be sharing his thoughts from this year’s show here in the XPRT blog, so be sure to stayed tuned. Will you be attending MWC this year? If so, let us know!

Justin

Check out our CES 2023 recap video!

We have a very talented studio team here at Principled Technologies, and this week, the team worked with the XPRTs to put together a CES 2023 recap video. In it, I discuss why we traveled to CES, my overall impressions from the show, and how the ideas and technologies presented at the show may influence the development of future XPRT benchmarks. If you missed last week’s post about my initial thoughts on the advances in AR and VR technology at the show, or you didn’t get a chance to see some of our photos from the show on social media, this week’s video provides a good summary of our CES-related activity.

To view the video, you can follow this link or click the screenshot below. If you followed CES this year and have any thoughts about how the XPRTs can help to evaluate emerging technologies, we’d love to hear from you!

Justin

Check out the other XPRTs:

Forgot your password?