Month: September 2013

Interesting questions

on September 26, 2013

We’ve had a couple of interesting questions about WebXPRT this week.

The first question was about the Face detect test in WebXPRT. One person, having noticed that changing the version of Firefox affected the WebXPRT score on a particular device, asked whether the test used the JavaScript Canvas element. The answer is yes, the Face detection test does use the Canvas element. It is based on the JavaScript library by Dr. Liu Liu.

As we have discussed in the past, the software stack on a device affects the benchmark scores. WebXPRT is a HTML5 benchmark and uses elements in the HTML5 specification, such as Canvas. Browsers implement HTML in their JavaScript engines, whose performance depends on the OS and the underlying platform. So, WebXPRT scores are influenced by the browser and OS, as well as the platform.

The second question was whether it is possible to run WebXPRT without an Internet connection. Generally speaking, the answer to that is no. WebXPRT is a hosted application, and to run the official version, you must be able to connect to the WebXPRT servers.

However, community members can download the WebXPRT source and configure local servers that will run WebXPRT, if they desire. Note: As we discussed in Sources, any published results must be from the version hosted at webxprt.com.

Thanks for the questions and keep experimenting!

Eric

Comment on this post in the forums

Posted in HTML5, WebXPRT |

Sources

By Eric Hale

on September 19, 2013

If you’ve checked out the MobileXPRT and WebXPRT pages recently, you’ve probably noticed that the number of results has started to grow. The results are coming from three sources:

Internal testing at PT.
Results submitted by the public.
Results published on the Web. We link back to the source from these results. Results published on-line include results in reports PT publishes for clients and reviews by other parties.

While we are excited about the growing number of results, we do sanity check them. We compare the results with other runs for the same device when available, or with similar devices if not.

The source code for the benchmarks is available, and we encourage experimentation. However, it should go without saying that valid runs must come from the builds of the benchmarks the development community has published. We can’t compare two results generated by different builds.

That being said, if you change the code and get an interesting result, by all means do contact us. You may have discovered something that we’ll want to include in a future version

Keep the results coming and keep experimenting!

Eric

Comment on this post in the forums

Posted in BenchmarkXPRT, BenchmarkXPRT development community, MobileXPRT, TouchXPRT, WebXPRT |

Improvements

By Eric Hale

on September 12, 2013

We recently made some changes to how MobileXPRT installs. Previously, when you installed MobileXPRT from Google Play, it downloaded the user experience (UX) tests from the Principled Technologies servers. This required you to have the Unknown sources option set, which allows installation of non-market apps, even if you installed MobileXPRT from Google Play.

We have removed this restriction. MobileXPRT now installs the UX tests from Google play, so installation is cleaner. However, the UX tests are not intended to be installed outside of MobileXPRT.

An unrelated fix now makes sure that MobileXPRT cleans out all of its content when you uninstall it. This fix required the location of the results file to move. See Submit your MobileXPRT 2013 results! for details.

If you prefer, you can still install MobileXPRT from the PT site. However, when installing from the PT site, you will still need the Unknown sources option set.

We continue looking for ways to improve the benchmarks. If you have any things you’d like to see us improve, please post to the forums or send an e-mail to benchmarkxprtsupport@principledtechnologies.com

Eric

Comment on this post in the forums

Posted in MobileXPRT |

Endurance

By Bill Catchings

on September 5, 2013

In There’s always something new to measure, we proposed several possible tests for the next version of WebXPRT. Of those, battery life testing generated the most interest.

Battery life testing poses a number of challenges. It’s not as simple as making WebXPRT loop. The biggest challenge is that different devices take different measures when the battery runs low. These measures range from dimming the screen, to stopping the hard disk, to totally shutting down the device. While these are perfectly reasonable, they are out of the benchmark’s control. Worse, most current browsers offer no way of knowing that these measures even happen nor do they offer good ways of querying the device to find out the state of its battery. We want to make sure that our approach does not unfairly advantage one device over another and gives a fair and accurate measure.

Because WebXPRT is a hosted application, we are looking at one of the other XPRT benchmarks for our first attempt at adding battery life to an existing benchmark. MobileXPRT seems to be the best fit. It runs on Android, which has a functional API for monitoring and managing power events, and the diversity of the Android ecosystem forces the benchmark to deal with a greater range of devices and OS configurations than TouchXPRT.

We are trying a number of approaches, and we have made some progress. We will discuss what we have learned in the next few weeks.

Our hope is that what we learn from MobileXPRT will better equip us to add battery life testing to WebXPRT.

Have any thoughts or comments? Post to the forums or e-mail benchmarkxprtsupport@principledtechnologies.com to let us know.

-Bill

Comment on this post in the forums

Posted in TouchXPRT, WebXPRT |

Month: September 2013

Interesting questions

Sources

Improvements

Endurance

Check out the other XPRTs: