Month: July 2013

There’s always something new to measure

on July 25, 2013

Scientific Reports published an article this month showing that electrically charged insects, such as honey bees, can cause some spider webs to deform. The effect, as pictures in the report show, is that the webs can literally reach out a short distance to snag the passing insect. It’s a cool optimization for the web, and one no one had measured before.

Like the scientists in the report, we are always looking for interesting things to measure. Even as we head toward the public release of MobileXPRT, we’re already thinking ahead to the next versions of all the XPRT benchmarks. This week, I’d like to share a few of the things we have been thinking about, based on conversations with folks (both within and outside of the Development Community) for WebXPRT.

Enhancing the benchmark UI to provide better feedback about the test progress
Extending coverage to other Web technologies such as Web Workers and CSS 3
Extending the workload categories to include other areas, such as productivity
Enabling WebXPRT to be used as a platform-independent battery life test

There are just some of the ideas we’ve been talking about. Which of these excite you? Even better, do you have any cool ideas of your own? Either way, post to the forums or e-mail BenchmarkXPRT@principledtechnologies.com

Eric

Comment on this post in the forums

Posted in Benchmarking, BenchmarkXPRT, Collaborative benchmark development, WebXPRT |

Oh no, not another learning experience!

By Eric Hale

on July 18, 2013

We are in the process of submitting TouchXPRT 2013 to the Microsoft App Store, and we are learning a lot in the process. Of course, we knew to create a Live ID for the process. However, we were surprised by the verification for the corporate account. We had to produce a lot of documentation, including our articles of incorporation, respond to phone calls and e-mail verifications, and agree to a pretty stringent legal agreement. Given the legal weight attached to the Live ID, we keep its information restricted to a minimum number of people.

Because the packages you upload to the store must be digitally signed by the Live account associated with the app store registration, a designated representative has to rebuild the package from the development team before we can submit it.

When we were preparing TouchXPRT 2013 for the app store, we restructured it to download its content after it’s installed. This allows the package in the app store to be smaller. Hosting the content was not a problem, but we did run into a couple of issues.

The first time we submitted TouchXPRT 2013 to the app store, they rejected it because they thought it was incomplete. We added a note to the testers about the download, and that resolved that issue. Next time, of course, we have learned to value of very specific notes to the testers.

The second hurdle was that, now that TouchXPRT downloaded its data, it qualified as a “network aware” application. All network aware applications must have a privacy policy and the privacy policy must be available from the application setting on the charms menu. Simply having the privacy policy was not enough. We have fixed this and submitted the application again.

The privacy policy is very simple: “TouchXPRT 2013 uses the network only to download content for its performance tests. TouchXPRT does not report any information to Principled Technologies or any other entity.” You can find it here.

In the next few weeks, we will be releasing MobileXPRT 2013 to the general public. We are already preparing it to submit to Google Play. Hopefully, things will go smoother this time!

Eric

Comment on this post in the forums

Posted in MobileXPRT, TouchXPRT |

The show is in previews

By Eric Hale

on July 11, 2013

I love theater, both as an actor and as an audience member. Seeing a show in preview means you have the chance to see what might be the next big thing before the critics do. It also means you may be watching all the things they need to fix before opening night.

Microsoft released the Windows 8.1 preview at the end of June. Of course we had to give it a look. After installing it, we ran the compatible XPRT benchmarks: HDXPRT, TouchXPRT, and WebXPRT. (Because MobileXPRT is Android based, it does not run on Windows.)

WebXPRT ran without any problems. To date, WebXPRT has run on everything we’ve thrown at it.

We took a Windows 8 system with TouchXPRT already installed and upgraded it to Windows 8.1. The previously installed version of TouchXPRT had some problems. However, when we did a clean install of Windows 8.1 and then installed TouchXPRT, it ran just fine.

HDXPRT failed while configuring the applications. We are looking into this and hope to have a solution soon.

I’m not going to talk about the results, because this is a preview of Windows. By the time of the official release, the results from the benchmarks certainly may change.

The Windows 8.1 RTM is currently set for August. I’m looking forward to seeing how it performs!

Eric

Comment on this post in the forums

Posted in HDXPRT 2012, TouchXPRT, WebXPRT, Windows 8.1 |

Back to the source

By Eric Hale

on July 4, 2013

Last week, we released MobileXPRT CP1.1.This week, we released the source code for MobileXPRT CP1.1. You can download it here (login required). The procedure for building it is the same as for the previous CP. As we discussed in Kick the tires. Look under the hood, it’s easy to set up the environment and all the necessary software is free.

We believe that one of the most important things we can do is make the source code available. We believe that increasing the transparency of the benchmarking process and stimulating the participation of industry players and the public in the definition, development, understanding, and auditing of the benchmarks will lead to better benchmarks.

You may be thinking “Then why not open source the benchmark?” The short answer is that we need to make sure that the results from any version of our benchmarks are ones you can trust. You can watch Bill discuss this in BenchmarkXPRT: It’s not a benchmark.

We believe that the community model—which gives you total visibility into the benchmark, allows you to run your own experiments and contribute to future versions of the benchmark, and still protects the integrity of the results—strikes the right balance.

If you’re not a member, please consider joining. It’s easy.

If you are a member, check out the code and tell us how it can be better!

Eric

Comment on this post in the forums

Posted in BenchmarkXPRT development community, Collaborative benchmark development, MobileXPRT 2013 source, Video |

Month: July 2013

There’s always something new to measure

Oh no, not another learning experience!

The show is in previews

Back to the source

Check out the other XPRTs: