BenchmarkXPRT Blog banner

Month: October 2016

Tracking device evolution with WebXPRT ’15, part 2

Last week, we used the Apple iPhone as a test case to show how hardware advances are often reflected in benchmark scores over time. When we compared WebXPRT 2015 scores for various iPhone models, we saw a clear trend of progressively higher scores as we moved from phones with an A7 chip to phones with A8, A9, and A10 Fusion chips. Performance increases over time are not surprising, but WebXPRT ’15 scores also showed us that upgrading from an iPhone 6 to an iPhone 6s is likely to have a much greater impact on web-browsing performance than upgrading from an iPhone 6s to an iPhone 7.

This week, we’re revisiting our iPhone test case to see how software updates can boost device performance without any changes in hardware. The original WebXPRT ’15 tests for the iPhone 5s ran on iOS 8.3, and the original tests for the iPhone 6s, 6s Plus, and SE ran on variants of iOS 9. We updated each phone to iOS 10.0.2 and ran several iterations of WebXPRT ’15.

Upgrading from iOS 8.3 to iOS 10 on the iPhone 5s caused a 17% increase in web-browsing performance, as measured by WebXPRT. Upgrading from iOS 9 to iOS 10 on the iPhone 6s, 6s Plus, and SE produced web-browsing performance gains of 2.6%, 3.6%, and 3.1%, respectively.

The chart below shows the WebXPRT ’15 scores for a range of iPhones, with each iPhone’s iOS version upgrade noted in parentheses. The dark blue columns on the left represent the original scores, and the light blue columns on the right represent the upgrade scores.

Oct 27 iPhone chart

As with our hardware comparison last week, these scores are the median of a range of scores for each device in our database. These scores come both from our own testing and from device reviews from popular tech media outlets.

These results reinforce a message that we repeat often, that many factors other than hardware influence performance. Designing benchmarks that deliver relevant and reliable scores requires taking all factors into account.

What insights have you gained recently from WebXPRT ’15 testing? Let us know!

Justin

Tracking device evolution with WebXPRT ‘15

The XPRT Spotlight on the Apple iPhone 7 Plus gives us a great opportunity to look at the progression of WebXPRT 2015 scores for the iPhone line and see how hardware and software advances are often reflected in benchmark scores over time. This week, we’ll see how the evolution of Apple’s mobile CPU architecture has boosted web-browsing performance. In a future post, we’ll see the impact of iOS development.

As we’ve discussed in the past, multiple factors can influence benchmark results. While we’re currently using the iPhone as a test case, the same principles apply to all types of devices. We should also note that WebXPRT is an excellent gauge of expected web-browsing performance during real-world tasks, which is different than pure CPU performance in isolation.

When looking at WebXPRT ’15 scores in our database, we see that iPhone web-browsing performance has more than doubled in the last three years. In 2013, an iPhone 5s with an Apple A7 chip earned an overall WebXPRT ’15 score of 100. Today, a new iPhone 7 Plus with an A10 Fusion chip reports a score somewhere close to 210. The chart below shows the WebXPRT ’15 scores for a range of iPhones, with each iPhone’s CPU noted in parentheses.

Oct 20 iPhone chart

Moving forward from the A7 chip in the iPhone 5s to the A8 chip in the iPhone 6 and the A9 chip in the iPhone 6s and SE, we see consistent score increases. The biggest jump, at over 48%, appears in the transition from the A8 to the A9 chip, implying that folks upgrading from an iPhone 6 or 6 Plus to anything newer would notice a huge difference in web performance.

In general, folks upgrading from an A9-based phone (6S, 6S Plus, or SE) to an A10-based phone (7 and 7 Plus) could expect an increase in web performance of over 6.5%.

The scores we list represent the median of a range of scores for each device in our database. These scores come from our own testing, as well as from device reviews from media outlets such as AnandTech, Notebookcheck, and Tom’s Hardware. It’s worth noting that the highest A9 score in our database (AnandTech’s iPhone SE score of 205) overlaps with the lowest A10 Fusion score (Tom’s Hardware of Germany’s iPhone 7 score of 203), so while the improvement in median scores is clear, performance will vary according to individual phones and other factors.

Soon, we’ll revisit our iPhone test case to see how software updates can boost device performance without any changes in hardware. For more details on the newest iPhones, visit the Spotlight comparison page to see how iPhone 7 and 7 Plus specs and WebXPRT scores stack up.

Justin

Taking a detour

Back in April, Bill announced that we would be starting development of a cross-platform benchmark. This announcement generated a lot of interest and we got lots of good feedback and ideas.

We knew from the start that getting a cross-platform benchmark right would be hard. However, it proved to be even trickier than we thought. As I explained before, benchmarks not only have to run well, but the results must be fair to all platforms involved. Achieving both of these requirements has been a challenge.

At the same time we’ve been devoting a great deal of effort and resources to the cross-platform benchmark, some increasingly popular new use cases have been receiving less attention than they deserve. We’ve decided that the cross-platform benchmark is not the best use of the Community’s resources, and are going to put it on the shelf for a while. This will free up the resources to let us really dig into some newer technologies.

Thanks again to everyone who responded.

Eric

An exciting milestone for WebXPRT!

If you’re familiar with the run counter on WebXPRT.com, you may have noticed that WebXPRT recently passed a pretty significant milestone. Since we released WebXPRT 2013, users running WebXPRT 2013 and 2015 have successfully completed over 100,000 runs!

We’re thrilled about WebXPRT’s ongoing popularity, and we think that it’s due to the benchmark’s unique combination of characteristics: it’s easy to run, it runs quickly and on a wide variety of platforms, and it evaluates device performance using real-world tasks. Manufacturers, developers, consumers, and media outlets in more than 358 cities, from Aberdeen to Zevenaar, and 57 countries, from Argentina to Vietnam, have used WebXPRT’s easy-to-understand results to compare how well devices handle everyday tasks. WebXPRT has definitely earned its reputation as a “go-to” benchmark.

If you haven’t run WebXPRT yet, give it a try. The test is free and runs in almost any browser.

We’re grateful for everyone who’s helped us reach this milestone. Here’s to another 100,000 runs!

Justin

WebXPRT delivers over 100,000 real-world results

DURHAM, NC–(Marketwired – October 05, 2016) – Today, Principled Technologies (PT) announced that the popular WebXPRT tool recently crossed a milestone of 100,000 total runs.

WebXPRT lets users everywhere compare the performance of any web-enabled device—from iPad tablets to Android phones to Windows computers. Manufacturers, developers, consumers, and media outlets in more than 358 cities and 57 countries have used WebXPRT’s easy-to-understand results to compare how well different devices handle everyday tasks.

WebXPRT is available to the public for free and runs in almost any browser. To read more about WebXPRT and run the test, visit WebXPRT.com.

WebXPRT is part of the BenchmarkXPRT family of benchmarks (the XPRTs), a set of apps designed to help test how well devices handle everyday tasks such as editing photos, playing movies, and browsing the web. Some XPRT tools even measure battery life. All of the BenchmarkXPRT family (the XPRTs) of evaluation tools are available to the public for free.

The BenchmarkXPRT Development Community is a forum where registered members can contribute to the process of creating and improving the XPRTs. To learn more about the BenchmarkXPRT Development Community and the XPRT family of benchmarks, visit BenchmarkXPRT.com.

About Principled Technologies, Inc.

Principled Technologies, Inc. is a leading provider of technology marketing and learning & development services. It administers the BenchmarkXPRT Development Community. The company is located in Durham, North Carolina, in NC’s Research Triangle Park region. For more information, please visit PrincipledTechnologies.com.

Company Contact
Eric Hale
Principled Technologies, Inc.
1007 Slater Road, Suite 300
Durham, NC 27703

Check out the other XPRTs:

Forgot your password?