ONNX Runtime Web | BenchmarkXPRT Development Community Blog

Tag Archives: ONNX Runtime Web

WebXPRT 5: AI tests now, lots of room for growth

on July 9, 2026

In past blog posts, we’ve discussed our goal of developing one or more experimental WebXPRT workloads focused on local, browser-side AI technologies. While many of us regularly interact with cloud-based AI apps and services through a browser, on-device AI capabilities are growing rapidly, and we want WebXPRT to continue to evolve with them.

There are several driving factors behind that growth. Web API technologies keep maturing, giving browsers direct access to the hardware they need for real inference work. Advanced GPU and NPU technology is now widely available in consumer devices, so the local computing power necessary to run AI applications on-device is in reach for many users. And for many organizations, there are compelling reasons to execute increasingly vital work like LLM inferencing and agentic coding tasks on local machines—such as data privacy, regulatory compliance, and cost control.

The reasons for the experimental workload approach

The expansion of on-device AI is exactly the type of shift we built the experimental workload concept to capture. As we shared when we first announced the WebXPRT 5 workload lineup, an experimental workload section gives us the flexibility to put cutting-edge measurement tools in users’ hands—even if those tools won’t yet run on every platform WebXPRT has traditionally supported. Experimental scores stay separate from the main overall score and are completely optional, so we can add tests without affecting comparability or asking anyone to retest. That approach maintains WebXPRT’s strengths while preparing the benchmark for the future—and giving all of us valuable information today.

The AI functions that WebXPRT 5 measures today

WebXPRT 5 already includes four workloads that utilize AI capabilities: Video Background Blur with AI, Detect Faces with AI, Image Classification with AI, and Document Scan with AI. These workloads use machine learning—computer vision and OCR models such as a Caffe-based face detector, SqueezeNet for image labeling, and an LSTM-based OCR engine. WebXPRT’s ability to measure how well devices handle those types of workloads has real value, and it reflects the kinds of light browser-side inference tasks that have been in widespread use for a while.

We recognize, though, that there’s a clear need for more demanding local, browser-based AI workloads—especially LLM inference. We’re targeting that need with our experimental work. Like pretty much everyone else, we’re also developing in the midst of an incredibly dynamic technical environment. We want to purposefully move forward without sacrificing WebXPRT’s stability and reliability for the sake of expedience.

The main decisions we face

Choosing a Web AI framework. We’re still researching our open-source framework options, including candidates like ONNX Runtime Web, Transformers.js, MediaPipe, and TensorFlow.js. The ground here continues to shift. For example, Transformers.js v4 now supports a WebGPU backend and spans a very broad range of model architectures. So, one of our ongoing challenges is picking a durable foundation.

Choosing a web API. Of the primary options we’re investigating, WebGPU now has the broadest browser support (Chrome, Edge, and partial support in Firefox and Safari). WebNN remains the most promising option in the long term because it can directly target NPUs, but it’s still not ready for production—its W3C spec only reached Candidate Recommendation status in early 2026, and browser support outside of flagged, experimental builds isn’t there yet. Our web API outlook hasn’t changed much from before: WebGPU is the most practical path today, and WebNN may be an exciting possibility for tomorrow.

Choosing and sizing workloads. We’ll ideally find workloads demanding enough to genuinely stress new hardware, but light enough to run on slightly older gear without forcing huge model downloads or overextending the test’s runtime. The sweet spot for browser inference today tends to be small, quantized models, and memory ceilings and cold-start downloads are real constraints. Striking the right balance is another part of the challenge we’re working through.

We appreciate your patience

We’ve been talking about experimental WebXPRT AI workloads for a while. While we wish we already had everything worked out, we think the end product will be worth the wait. We appreciate your patience as we work through the details, and we’ll keep updating you here in the blog as we make progress.

As always, we’re open to suggestions. If you have ideas for a browser-based AI workload scenario, a framework or API you think we should weigh, a browser-based AI application you want us to consider, or any other related thoughts, please let us know!

Justin

Posted in AI, benchmark, Benchmarking, BenchmarkXPRT, browser performance, Browser-based benchmarks, Caffe, Chrome, Chromium, Collaborative benchmark development, computer vision, Cross-platform benchmarks, Future of performance evaluation, GPU, HTML5, image classification, inference, JavaScript, large language models, MediaPipe, Microsoft Edge, NPU, OCR, ONNX Runtime Web, Performance benchmarking, Performance of computing devices, Safari, SqueezeNet, TensorFlow, Wasm, Web AI, web API, Web-based testing, WebAssembly, WebGPU, WebNN, WebXPRT, WebXPRT 5 | Also tagged AI, AI workloads, benchmark, browser benchmark, browser performance, Chrome, Edge, Firefox, image classification, image processing, large language models, LLM, LSTM, MediaPipe, NPU, OCR, Safari, TensorFlow, Transformers.js, Web AI, web API, WebGPU, WebNN, WebXPRT, WebXPRT 5

Web AI frameworks: Possible paths for the AI-focused WebXPRT 4 auxiliary workload

By Justin Greene

on September 5, 2024

A few months ago, we announced that we’re moving forward with the development of a new auxiliary WebXPRT 4 workload focused on local, browser-side AI technology. Local AI has many potential benefits, and it now seems safe to say that it will be a common fixture of everyday life for many people in the future. As the growth of browser-based inference technology picks up steam, our goal is to equip WebXPRT 4 users with the ability to quickly and reliably evaluate how well devices can handle substantial local inference tasks in the browser.

To reach our goal, we’ll need to make many well-researched and carefully considered decisions along the development path. Throughout the decision-making process, we’ll be balancing our commitment to core XPRT values, such as ease of use and widespread compatibility, with the practical realities of working with rapidly changing emergent technologies. In today’s blog, we’re discussing one of the first decision points that we face—choosing a Web AI framework.

AI frameworks are suites of tools and libraries that serve as building blocks for developers to create new AI-based models and apps or integrate existing AI functions in custom ways. AI frameworks can be commercial, such as OpenAI, or open source, such as Hugging Face, PyTorch, and TensorFlow. Because the XPRTs are available at no cost for users and we publish our source code, open-source frameworks are the right choice for WebXPRT.

Because the new workload will focus on locally powered, browser-based inference tasks, we also need to choose an AI framework that has browser integration capabilities and does not rely on server-side computing. These types of frameworks—called Web AI—use JavaScript (JS) APIs and other web technologies, such as WebAssembly and WebGPU, to run machine learning (ML) tasks on a device’s CPU, GPU, or NPU.

Several emerging Web AI frameworks may provide the compatibility and functionality we need for the future WebXPRT workload. Here are a few that we’re currently researching:

ONNX Runtime Web: Microsoft and other partners developed the Open Neural Network Exchange (ONNX) as an open standard for ML models. With available tools, users can convert models from several AI frameworks to ONNX, which can then be used by ONNX Runtime Web. ONNX Runtime Web allows developers to leverage the broad compatibility of ONNX-formatted ML models—including pre-trained vision, language, and GenAI models—in their web applications.
Transformers.js: Transformers.js, which uses ONNX Runtime Web, is a JS library that allows users to run AI models from the browser and offline. Transformers.js supports language, computer vision, and audio ML models, among others.
MediaPipe: Google developed MediaPipe as a way for developers to adapt TensorFlow-based models for use across many platforms in real-time on-device inference applications such as face detection and gesture recognition. MediaPipe is particularly useful for inference work in images, videos, and live streaming.
TensorFlow.js: TensorFlow has been around for a long time, and the TensorFlow ecosystem provides users with a broad variety of models and datasets. TensorFlow is an end-to-end ML solution—training to inference—but with available pre-trained models, developers can focus on inference. TensorFlow.js is an open-source JS library that helps developers integrate TensorFlow with web apps.

We have not made final decisions about a Web AI framework or any aspect of the future workload. We’re still in the research, discussion, and experimentation stages of development, but we want to be transparent with our readers about where we are in the process. In future blog posts, we’ll discuss some of the other major decision points in play.

Most of all, we invite you to join us in these discussions, make recommendations, and give us any other feedback or suggestions you may have, so please feel free to share your thoughts!