Category: Web-based testing

Glimpses of the next WebXPRT

on November 2, 2017

Development work on the next version of WebXPRT is well underway, and we think it’s a good time to offer a glimpse of what’s to come.

We’ve updated the photo-related workloads with new images and are experimenting with adding a new task to the Organize Album workload. The task utilizes ConvNetJS, a JavaScript library designed for training neural networks within the browser itself, to assign classifications to a set of images. It’s an example of the type of integrated deep learning tasks that will be showing up in all sorts of devices in the years to come.

We’re also testing an additional task in the Local Notes workload using Tesseract.js, a popular OCR (optical character recognition) engine. Our scenario uses OCR technology to scan images of purchase receipts and gather relevant information.

We’re testing these new tasks now, and will include them only once we’re confident that they produce consistent and reliable results without extending the benchmark’s runtime unnecessarily. Consequently, the next WebXPRT might contain variations of these tasks, or other new technologies altogether. We mention them now to offer some insight into the types of workload enhancements that we’re considering.

We’ve been working hard on the new WebXPRT UI as well. The image below shows the new start page from an early development build. We’re still making adjustments, so the final product will probably differ, but you do get a sense of the new UI’s clean look.

As we’ve said before, we’re committed to making sure that WebXPRT runs in most browsers and produces results that are useful for comparing web browsing performance across a wide variety of devices. We appreciate the feedback we’ve gotten so far, and are happy to receive more. Do you have ideas for the next WebXPRT? Let us know!

Justin

Posted in Browser-based benchmarks, Cross-platform benchmarks, Future of performance evaluation, Machine learning, Web-based testing, WebXPRT, WebXPRT 2017 |

Introducing the WebXPRT 2015 Processor Comparison Chart

By Justin Greene

on October 5, 2017

Today, we’re excited to announce the WebXPRT 2015 Processor Comparison Chart, a new tool that makes it easier to access hundreds of PT-curated, real-world performance scores from a wide range of devices covering everything from TVs to phones to tablets to PCs.

The chart offers a quick way to browse and compare WebXPRT 2015 results grouped by processor. Unlike benchmark-score charts that may contain results from unknown sources, PT hand-selected each of the results from internal lab testing and reliable tech media sources. If we published multiple scores for an individual processor, the score presented in the chart will be an average of those scores. Users can hover over and click individual score bars for additional information about the test results and test sources for each processor.

We think the WebXPRT Processor Comparison Chart will be a valuable resource for folks interested in performance testing and product evaluation, but the current iteration is only the beginning. We plan to add additional capabilities on a regular basis, such a detailed filtering and enhanced viewing and navigational options. It’s also possible that we may integrate other XPRT benchmarks down the road.

Most importantly, we want the chart to be a great asset for its users, and that’s where you come in. We’d love to hear your feedback on the features and types of data you’d like to see. If you have suggestions, please let us know!

Justin

Posted in Benchmark metrics, Browser-based benchmarks, Cross-platform benchmarks, Performance benchmarking, Web-based testing, WebXPRT, WebXPRT 2015 |

WebXPRT in 2017

By Bill Catchings

on December 8, 2016

Over the last few weeks, we’ve discussed the future of HDXPRT and BatteryXPRT. This week, we’re discussing what’s in store for WebXPRT in 2017.

WebXPRT is our most popular tool. Manufacturers, developers, consumers, and media outlets in more than 350 cities and 57 countries have run WebXPRT over 113,000 times to date. The benchmark runs quickly and simply in most browsers and produces easy-to-understand results that are useful for comparing web browsing performance across a wide variety of devices and browsers. People love the fact that WebXPRT runs on almost any platform that has a web browser, from PCs to phones to game consoles.

More people are using WebXPRT in more places and in more ways than ever before. It’s an unquestioned success, but we think this is a good time to make it even better by beginning work on WebXPRT 2017. Any time change comes to a popular product, there’s a risk that faithful fans will lose the features and functionality they’ve grown to love. Relevant workloads, ease of use, and extensive compatibility have always been the core components of WebXPRT’s success, so we want to reassure users that we’re committed to maintaining all of those in future versions.

Some steps in the WebXPRT 2017 process are straightforward, such as the need to reassess the existing workload lineup and update content to reflect advances in commonly used technologies. Other steps, such as introducing new workloads to test emerging browser technologies, may be tricky to implement, but could offer tremendous value in the months and years ahead.

Are there test scenarios or browser technologies you would like to see in WebXPRT 2017, or tests you think we should get rid of? Please let us know. We want to hear from you and make sure that we’re crafting a performance tool that continues to meet your needs.

Bill

Posted in BatteryXPRT 2014 for Android, Future of performance evaluation, HDXPRT, HTML5, Performance benchmarking, Web-based testing, WebXPRT, WebXPRT 2017 |

Ending the year with a bang!

By Eric Hale

on January 2, 2013

As we promised in the blog post The newest member of the family, we made the WebXPRT 2013 community preview available this week. It has already been used in a review! The AnandTech review of the Acer Iconia W510 includes results from the WebXPRT 2013 community preview for that device and for the Microsoft Surface RT and the Apple iPad 4. The review has results from the TouchXPRT 2013 community preview for the Acer Iconia W510, Microsoft Surface RT, and the ASUS VivoTab RT as well.

Obviously, we’ve been doing some testing ourselves. Here’s a sampling of the devices on which we’ve successfully run WebXPRT:

Device	Processor	Operating system	Browser	Score	Confidence interval
HP Envy 2	1.8 GHz Intel Atom Z2760	Windows 8	Internet Explorer 10.0.92	201	+/- 6
Asus VivoTab RT	1.2 GHz Tegra 3 T30L	Windows RT	Internet Explorer 10.0.92	160	+/- 5
Kindle Fire	1.2 GHz ARM Cortex-A9	Android OS 2.3 (customized: 6.3.1_user_4107720)	Safari 5	92	+/- 2
ASUS-made Google Nexus 7	1.2 GHz Tegra 3 T30L	Android 4.2	Chrome 18	201	+/- 4
Motorola DroidX phone	1 GHz TI OMAP3630-1000	Android 4.5.621	Browser version 2.3.4	26	+/- 1
iPhone 5	1.3 GHz Apple A6	iOS 6.0.2	Safari 6	168	+/- 2
iPad mini	1GHz Apple A5	iOS 6.0.2	Safari 6	110	+/- 1
iPad 4	1.4 GHz Apple A6X	iOS 6.0.1	Safari 6	180	+/- 2

As the results above show, WebXPRT can run on a wide range of devices. We are working to get results on lots of different devices and would like your help. We’ll set up a forum thread for results that starts with these. We’ll then add additional ones we produce. Please respond in the thread with results you get.

In addition to performance results, the WebXPRT 2013 community preview also provides a report on the HTML 5 capabilities of your device. For those who want to know more about the capabilities of HTML 5, there’s more good news. The W3C community released the feature-complete spec for HTML 5 and Canvas 2D this week.

You can find an explanation of scenarios in the WebXPRT 2013 community preview, and an explanation of how it calculates its results in the WebXPRT 2013 CP1 Overview. Let us know what you think. There’s still time to help us shape the final version of both WebXPRT 2013 and TouchXPRT 2013.

Eric

Comment on this post in the forums

Posted in Benchmarking, HTML5, Let us know your thoughts, Performance testing on tablets, TouchXPRT, Web-based testing, WebXPRT |

The newest member of the family

By Eric Hale

on December 14, 2012

In his blog post TouchXPRT Web test update, Bill mentioned that we would be releasing Web-based workloads for the community to try out. Although we developed them as part of TouchXPRT, the cross-platform nature of these tests suggested to us that they should stand on their own.

After seeking community input, we have decided to make them a separate benchmark. So, we are proud to announce WebXPRT. The first community preview will be available mid-next week. WebXPRT CP1 contains four workloads: Photo Effects, Face Detect, Stocks Dashboard, and Offline Notes. Because the workloads are all HMTL5 based, they run on a wide variety of devices and operating systems—from iPad tablets to Android phones to Windows computers.

As with all community previews, we are very interested in your opinion. Tell us what you like or don’t like about the workloads. Are there other use cases you’d like to see?

Now that we have three benchmarks, the old HDXPRT-centric model of the community needs updating. Earlier this week, a message went out to the community announcing that we will be reorganizing the benchmarks under the umbrella of BenchmarkXPRT. This reorganization will touch all aspects of the community, from the Web site to Facebook, Twitter, and even the memberships themselves. We’ll be rolling out these changes over the next few weeks, and we’ll keep you informed every step of the way.

Eric

Comment on this post in the forums

Posted in BenchmarkXPRT, HTML5, Let us know your thoughts, Web-based testing, WebXPRT |

TouchXPRT Web test update

By Bill Catchings

on November 29, 2012

On October 22, we released TouchXPRT CP1 to the community. We took the unprecedented step of releasing CP1 without any restrictions on publishing results, and since then reviews of the Microsoft Slate and the Sony Duo 11 Convertible Laptop have used TouchXPRT.

The five scenarios in CP1 focus on media manipulation. While this is an important activity on touch devices, we know this is not all people do.

Next week, we plan to release Web-based scenarios. They use HTML 5 for a variety of activities. Unlike the original scenarios in TouchXPRT CP1, there will be nothing to download. You simply browse to a URL and run the tests online. There’s nothing to set up, just browse and run.

That means that there is nothing preventing you from running these tests on pretty much any system-browser combination that supports HTML5, not just on touch-based, Windows 8 devices like the rest of TouchXPRT. That started us wondering whether these Web-based activities should be thought of as a different benchmark entirely. When these tests are available, please try them out and let us know what you think. Do you think they are worthwhile for a broader range of devices? Do you think their scenario-based emphasis is a good alternative to existing lower-level Web-based tests?

Please keep in mind that it’s not too late to give feedback on TouchXPRT CP1. Let us know how you like the scenarios on CP1 as well as what other activities you would like to see.

-Bill Catchings

Comment on this post in the forums

Posted in Benchmarking, HTML5, Performance testing on tablets, Touch-based benchmarking, Touch-based devices, TouchXPRT, Web-based testing |

Category: Web-based testing

Glimpses of the next WebXPRT

Introducing the WebXPRT 2015 Processor Comparison Chart

WebXPRT in 2017

Ending the year with a bang!

The newest member of the family

TouchXPRT Web test update

Check out the other XPRTs: