BenchmarkXPRT Blog banner

Category: Data privacy

Local AI and new frontiers for performance evaluation

Recently, we discussed some ways the PC market may evolve in 2024, and how new Windows on Arm PCs could present the XPRTs with many opportunities for benchmarking. In addition to a potential market shakeup from Arm-based PCs in the coming years, there’s a much broader emerging trend that could eventually revolutionize almost everything about the way we interact with our personal devices—the development of local, dedicated AI processing units for consumer-oriented tech.

AI already impacts daily life for many consumers through technologies such as such as predictive text, computer vision, adaptive workflow apps, voice recognition, smart assistants, and much more. Generative AI-based technologies are rapidly establishing a permanent, society-altering presence across a wide range of industries. Aside from some localized inference tasks that the CPU and/or GPU typically handle, the bulk of the heavy compute power that fuels those technologies has been in the cloud or in on-prem servers. Now, several major chipmakers are working to roll out their own versions of AI-optimized neural processing units (NPUs) that will enable local devices to take on a larger share of the AI load.

Examples of dedicated AI hardware in recently-released or upcoming consumer devices include Intel’s new Meteor Lake NPU, Apple’s Neural Engine for M-series SoCs, Qualcomm’s Hexagon NPU, and AMD’s XDNA 2 architecture. The potential benefits of localized, NPU-facilitated AI are straightforward. On-device AI could reduce power consumption and extend battery life by offloading those tasks from the CPUs. It could alleviate certain cloud-related privacy and security concerns. Without the delays inherent in cloud queries, localized AI could execute inference tasks that operate much closer to real time. NPU-powered devices could fine-tune applications around your habits and preferences, even while offline. You could pull and utilize relevant data from cloud-based datasets without pushing private data in return. Theoretically, your device could know a great deal about you and enhance many areas of your daily life without passing all that data to another party.

Will localized AI play out that way? Some tech companies envision a role for on-device AI that enhances the abilities of existing cloud-based subscription services without decoupling personal data. We’ll likely see a wide variety of capabilities and services on offer, with application-specific and SaaS-determined privacy options.

Regardless of the way on-device AI technology evolves in the coming years, it presents an exciting new frontier for benchmarking. All NPUs will not be created equal, and that’s something buyers will need to understand. Some vendors will optimize their hardware more for computer vision, or large language models, or AI-based graphics rendering, and so on. It won’t be enough for business and consumers to simply know that a new system has dedicated AI processing abilities. They’ll need to know if that system performs well while handling the types of AI-related tasks that they do every day.

Here at the XPRTs, we specialize in creating benchmarks that feature real-world scenarios that mirror the types of tasks that people do in their daily lives. That approach means that when people use XPRT scores to compare device performance, they’re using a metric that can help them make a buying decision that will benefit them every day. We look forward to exploring ways that we can bring XPRT benchmarking expertise to the world of on-device AI.

Do you have ideas for future localized AI workloads? Let us know!

Justin

The XPRTs and GDPR

If you’re like me, your inbox has been receiving a constant stream of “updates to our privacy policy” messages from seemingly every company you’ve ever interacted with. The timing is not coincidental: most of the notifications are the result of companies taking steps to comply with the European Union’s General Data Protection Regulation (GDPR), which goes into effect next Friday, May 25th.

The GDPR was designed to introduce clarity and consistency to privacy laws in Europe, protect the data privacy rights of citizens, and change the way corporations approach data privacy issues. The gist is that if any party collects any personal data from European citizens or organizations, they must follow specific guidelines:

  • They may only collect data for clearly-stated, legitimate reasons.
  • They must identify what data they’re collecting.
  • They must allow data subjects to see the data they’ve collected.
  • They must provide a quick, easy way for data subjects to have their data deleted.


Europeans use the XPRTs quite a bit, and some of our community members live in Europe, so we’re taking the necessary steps to comply with the new regulations. Thankfully, we don’t have to change very much. We built respect for testers’ and community members’ privacy rights into our processes from the very beginning and we are committed to clarity and transparency regarding our data collection practices.

Our blog post on data privacy from last month contains more details about how the XPRTs handle your data, and we’ll update our privacy policies and data collection notices where necessary to make things as clear as possible.

We also want to make sure community members know that if they would like to leave the community and have us delete their name, username, and email address from our records, simple send us a message at benchmarkxprtsupport@principledtechnologies.com and we’ll fulfill your request within 72 hours.

If you have any questions or comments about this topic, please let us know.

Justin

Check out the other XPRTs:

Forgot your password?