Over the past few months, we’ve been excited to see a substantial increase in the total number of completed WebXPRT runs. To put the increase in perspective, we had more total WebXPRT runs last month alone (40,453) than we had in the first two years WebXPRT was available (36,674)! This boost has helped us to reach two important milestones as we close in on the end of 2023.
The first milestone is that the number of WebXPRT 4 runs per month now exceeds the number of WebXPRT 3 runs per month. When we release a new version of an XPRT benchmark, it can take a while for users to transition from using the older version. For OEM labs and tech journalists, adding a new benchmark to their testing suite often involves a significant investment in back testing and gathering enough test data for meaningful comparisons. When the older version of the benchmark has been very successful, adoption of the new version can take longer. WebXPRT 3 has been remarkably popular around the world, so we’re excited to see WebXPRT 4 gain traction and take the lead even as the total number of WebXPRT runs increases each month. The chart below shows the number of WebXPRT runs per month for each version of WebXPRT over the past ten years. WebXPRT 4 usage first surpassed WebXPRT 3 in August of this year, and after looking at data for the last three months, we think its lead is here to stay.
The second important milestone is the cumulative number of WebXPRT runs, which recently passed 1.25 million, as the chart below shows. For us, this moment represents more than a numerical milestone. For a benchmark to succeed, developers need the trust and support of the benchmarking community. WebXPRT’s consistent year-over-year growth tells us that the benchmark continues to hold value for manufacturers, OEM labs, the tech press, and end users. We see it as a sign of trust that folks repeatedly return to the benchmark for reliable performance metrics. We’re grateful for that trust, and for everyone that has contributed to the WebXPRT development process over the years.
We look forward to seeing how far
WebXPRT’s reach can extend in 2024! If you have any
questions or comments about using WebXPRT, let us know!
Yesterday, Apple revealed
the iPhone 15 and iPhone 15 Pro at its annual fall event, along with a new version of the iOS mobile operating system (iOS 17). The official iOS
17 launch will take place on September 18th, but before then, users
of newer iPhones can install the OS via the Apple Beta Software Program.
Today, a tech journalist informed us that during their testing of iPhone 15 and iPhone 15 Pro with iOS 17 Beta models, WebXPRT 4 has been freezing while running the Encrypt Notes and OCR Scan workload in the Safari 17 browser. Here in the lab, we were able to immediately replicate the issue on an iPhone 12 Pro with iOS 17 Beta model.
troubleshooting confirmed that WebXPRT 3 successfully runs to completion on iOS
17 Beta, so it appears that the problem is specific to WebXPRT 4. We also
confirmed that WebXPRT 4 freezes at the same place when running in the Google
Chrome browser on iOS 17 Beta, so we know that the problem does not occur only in
investigating the issue, and will publish our findings here in the blog as soon
as we feel confident that we’ve identified both the root cause and a workable
solution, if a solution is necessary. One reason a solution would not be
necessary is that the issue is a bug on the iOS 17 Beta side that Apple will resolve
before the official launch.
We apologize for any inconvenience this issue might cause for tech reviewers and iPhone users, and we appreciate your patience while we figure out what’s going on. If you have any questions about WebXPRT 4, please don’t hesitate to ask!
When we’ve released a
new version of an XPRT benchmark app, it’s been our practice for many years to
maintain a link to the previous version on the benchmark’s main page. For
example, visitors can start on the WebXPRT 4 homepage at WebXPRT.com and follow links to access
WebXPRT 3, WebXPRT 2015, and WebXPRT 2013. Historically, we’ve maintained these
links because labs and tech reviewers usually take a while to introduce a new
benchmark to their testing suite. Continued access to the older benchmarks also
allows users to quickly compare new devices to old devices without retesting
That being said, several of the XPRT pages currently contain links to benchmarks that we no longer actively support. Some of those benchmarks still function correctly, and testers occasionally use them, but a few no longer work on the latest versions of the operating systems or browsers that we designed them to test. While we want to continue to provide a way for longtime XPRT users to access legacy XPRTs, we also want to avoid potential confusion for new users. We believe our best way forward is to archive older tests in a separate part of the site.
In the coming weeks, we’ll
be moving several legacy XPRT benchmarks to an archive section of the site. Once
the new section is ready, we’ll link to it from the Extras drop-down menu at
the top of BenchmarkXPRT.com. The benchmarks will still be available in the
archive, but we will not actively support them or directly link to them from
the homepages of active XPRTs.
During this process,
we’ll move the following benchmarks to the archive section:
WebXPRT 2015 and 2013
MobileXPRT 2015 and 2013
If you have any questions or concerns about the archive process or access to legacy XPRTs, please let us know!
If you’ve ever spent time exploring WebXPRT.com, you may have noticed a line that says, “If you are in East Asia, you can run WebXPRT from our Singapore host,” followed by a hyperlink with Simplified Chinese characters. We realize that some people may not know why we have a WebXPRT mirror host site in Singapore—or how to use it—so today’s post will cover the basics.
When we first released WebXPRT 2013, some users in mainland China reported slow download times when running the benchmark. These slowdowns affected initial page and workload content load times, but not workload execution, which happens locally. As a result, subtest and overall scores were still consistent with expectations for the devices under test, but it took longer than normal for test runs to complete. In response, we set up a mirror host site in Singapore to facilitate WebXPRT testing in China and other East Asian countries. We continued this practice with subsequent WebXPRT versions, and currently offer Singapore-based instances of WebXPRT 4, WebXPRT 3, and WebXPRT 2015.
The default UI language on the Singapore site is Simplified Chinese, but users can opt to change the language to English or German. Apart from a different default language, the WebXPRT mirror instances hosted in Singapore are identical to the instances on the main WebXPRT site. If you test a device on WebXPRT Singapore and WebXPRT.com, you should see similar performance scores from both sites.
We hope that the WebXPRT mirror host site in Singapore will make it easier for people in East Asia to use the benchmark. Do you find the site useful? If so, we’d love to hear from you! Also, if you encounter any unexpected issues or interruptions while testing, please let us know!
In our last
blog post, we reflected on the 10-year
anniversary of the WebXPRT launch by looking at the consistent
growth in the number of WebXPRT runs over the last decade. Today, we wrap
up our focus on WebXPRT’s anniversary by sharing some data about the
benchmark’s truly global reach.
update the community on some of the reach metrics we track by publishing a new
version of the “XPRTs
around the world” infographic. The metrics include completed test runs,
benchmark downloads, and mentions of the XPRTs in advertisements, articles, and
tech reviews. This information gives us insight into how many people are using
the XPRT tools, and publishing the infographic helps readers and community
members see the impact the XPRTs are having around the world.
WebXPRT is our
most widely used benchmark by far, and is responsible for much of the XPRT’s
global reach. Since February 2013, users have run WebXPRT more than 1,176,000
times. Those test runs took place in over 924 cities located in 81 countries on
six continents. Some interesting new locations for completed WebXPRT runs
include Rajarampur, Bangladesh; Al Muharraq, Bahrain; Manila, The Philippines;
Skopje, Macedonia; and Ljubljana, Slovenia.
that WebXPRT has proven to be a useful and reliable performance evaluation tool
for so many people in so many geographically distant locations. If you’ve ever
run WebXPRT in a country that is not highlighted in the “XPRTs around the
world” infographic, we’d love to hear
In our last blog post, we celebrated the 10-year anniversary of the WebXPRT launch by looking back on the WebXPRT team’s accomplishments over the last decade. The incremental steps and milestone improvements we discussed all contributed to carving out a lasting place for WebXPRT in the benchmarking world and helped to grow its reputation for being a reliable, effective, and easy to use measurement tool.
WebXPRT’s growth is most evident when we look at the rising number of completed test runs over the last 10 years. Since the first WebXPRT launch in 2013, we’ve seen a steady increase in the number of tests people are running. To put the increase in perspective, we had more runs last month alone (17,300) than we recorded in the first 10 months that WebXPRT was available (11,984).
That growth has helped
us to reach and surpass the million-run mark, but the most exciting aspect of
seeing a consistent increase in WebXPRT testing is the knowledge that the
benchmark is proving to be useful to more people in more places around the
world. In our next blog post, we’ll discuss WebXPRT’s truly global reach and
some of the surprising cities and countries where people have been using it to
test their gear.
We’re grateful for all the testers that have helped WebXPRT grow during the last decade. If you have any questions or comments about using WebXPRT, let us know!