BenchmarkXPRT Blog banner

Author Archives: Justin Greene

Let the XPRTs be your holiday shopping companion!

Justin Greene
By Justin Greene
on November 9, 2023

The holiday shopping season is right around the corner, and choosing the right tech gift can be a daunting task. If you’re considering new phones, tablets, Chromebooks, laptops, or desktops as gifts this year, and are unsure where to get reliable device information, the XPRTs can help!

The XPRTs provide objective, reliable measures of a device’s performance that can help cut through competing marketing claims. For example, instead of guessing whether the performance of a new phone justifies its price, you can use its WebXPRT performance score to see how it stacks up against both older models and competitors while tackling everyday tasks.

A good place to start looking for device scores is our XPRT results browser, which lets you access our database of more than 3,500 test results from over 165 sources, including major tech review publications around the world, OEMs, and independent testers. You can find a wealth of current and historical performance data across all the XPRT benchmarks and hundreds of devices. Learn how to use the results browser here.

If you’re considering a popular device, chances are good that a recent tech review includes an XPRT score for it. Go to your favorite tech review site and search for “XPRT,” or enter the name of the device and the appropriate XPRT (e.g., “Pixel” and “WebXPRT”) in a search engine. Here are a few recent tech reviews that used the XPRTs to evaluate popular devices:


The XPRTs can help consumers make better-informed and more confident tech purchases this holiday season, and we hope you’ll find the data you need on our site or in an XPRT-related tech review. If you have any questions about the XPRTs, XPRT scores, or the results database please feel free to ask!

Justin

Posted in Acer, Apple, ASUS, benchmark, Chromebooks, CrXPRT, Google, HDXPRT, Laptops, Phones, results, Samsung, Tablets, WebXPRT, WebXPRT 4 | Tagged , , , , , , , , , , , , , , , , , |

Good news for WebXPRT 4 testing!

Justin Greene
By Justin Greene
on November 2, 2023

Over the past several weeks, we’ve been working to find a solution to a problem with WebXPRT 4 test failures on Apple devices running iOS 17/17.1, iPadOS 17/17.1, and macOS Sonoma with Safari 17/17.1. While we put significant effort into an updated WebXPRT version that would mitigate this issue, we are happy to report that it now looks like we’ll be able to stick with the current version!

Last Thursday, Apple released the iOS 17.2 beta for participants in the Apple Developer Program. When we tested the current version of WebXPRT 4 on iOS 17.2, the tests completed without any issues. We then successfully completed tests on iPadOS 17.2 and macOS Sonoma 14.2 with Safari 17.2. Now that we have good reasons to believe that the iOS 17.2 release will solve the problem, sticking with the current WebXPRT 4 build will maximize continuity and minimize disruption for WebXPRT users.

Apple has not yet published a public release date for iOS/iPad OS/Safari 17.2. Based on past development schedules, it seems likely that they will release it between mid-November and early December, but that’s simply our best guess. Until then, users who want to test WebXPRT 4 on devices running iOS 17/17.1, iPadOS 17/17.1, or macOS Sonoma with Safari 17/17.1 will need to update those devices to iOS/iPad OS/Safari 17.2 via the Apple Developer Program.

To help Apple users better navigate testing until the public 17.2 release, we’ve added a function to the current WebXPRT 4 start page that will notify users if they need to update their operating system to test.

We appreciate everyone’s patience as we worked to find a solution to this problem! If you have any questions or concerns about WebXPRT 4, please let us know.

Justin

Posted in Apple, benchmark, Benchmarking, browser performance, Browser-based benchmarks, Cross-platform benchmarks, iOS, iPadOS, Performance benchmarking, Safari, WebXPRT, WebXPRT 4 | Tagged , , , , , , , , , , |

Support for MobileXPRT 3 will likely end soon

Justin Greene
By Justin Greene
on October 20, 2023

In a past blog post, we discussed our plan to move several older versions of XPRT benchmarks to an XPRT archive page. Some of those legacy XPRTs still function correctly, and testers occasionally use them, but a few no longer work on the latest versions of the operating systems or browsers that we designed them to test. With the archive page, we can prevent potential confusion for new users who visit current XPRT pages, but still provide longtime users with continued access to old tests.

You can find more information about the XPRTs that we’ll be moving to the archive page here, but today, we want to let MobileXPRT users know that there’s a high likelihood that MobileXPRT 3 will be joining the list of archived XPRTs in the very near future. The Google Play Store has notified us that, due to evolving requirements for apps in newer versions of Android, we must update our MobileXPRT 3 app package to target an Android API level within one year of the latest Android release. If we don’t update the app to meet that requirement by November 1, users will no longer be able to access MobileXPRT 3 through the Google Play Store.

Though a small number of labs and reviewers still use MobileXPRT 3 to test phones and tablets around the world, we don’t feel current usage is high enough for us to justify committing resources to an update at this point. We had hoped that even if MobileXPRT 3 became inaccessible via the Google Play Store, it would still be possible to sideload the app for testing on newer Android devices. After experimenting with installation options in the lab, however, we think it’s likely that settings on devices running Android 11 and up will prevent both Google Play and sideload installations after November 1. The situation may change, but right now, we don’t expect any method to work after that date. If you try, you’ll likely encounter a message during the installation process that says, “This app was built for an older version of Android and may not work properly. Try checking for updates, or contact the developer.” If you attempt to continue the installation process after that message appears, the app will crash.

Both Android and Chrome developers know that the respective stores sometimes extend these types of deadlines. We hope that will be the case here, but we have no information that would lead us to anticipate an extension. If there is no extension, we will still make MobileXPRT 3 available for testing on older Android devices, but we will then have to move it to the XPRT archive page.

We’re grateful for everyone who has used MobileXPRT 3 in the past, and we apologize for any convenience this change may cause. If you have any questions or concerns about MobileXPRT 3 access, please let us know! 

Justin

Posted in Android, Android 11, Benchmarking, BenchmarkXPRT, Google, Google Chrome, Google Play Store, Mobile devices, MobileXPRT, MobileXPRT 3, Performance testing on tablets, Phones | Tagged , , , , , , , , |

Making progress with WebXPRT 4 in iOS 17

Justin Greene
By Justin Greene
on October 12, 2023

In recent blog posts, we discussed an issue that we encountered when attempting to run WebXPRT 4 on iOS 17 devices. If you missed those posts, you can find more details about the nature of the problem here. In short, the issue is that the Encrypt Notes and OCR scan subtest in WebXPRT 4 gets stuck when the Tesseract.js Optical Character Recognition (OCR) engine attempts to scan a shopping receipt. We’ve verified that the issue occurs on devices running iOS 17, iPadOS 17, and macOS Sonoma with Safari 17.

After a good bit of troubleshooting and research to try and identify the cause of the problem, we decided to build an updated version of WebXPRT 4 that uses a newer version of Tesseract for the OCR task. Aside from updating Tesseract in the new build, we aimed to change as little as possible. To try and maximize continuity, we’re still using the original input image for the receipt scanning task, and we decided to stick with using the WASM library instead of a WASM-SIMD library. Aside from a new version of tesseract.js, WebXPRT 4 version number updates, and updated documentation where necessary, all other aspects of WebXPRT 4 will remain the same.

We’re currently testing a candidate build of this new version on a wide array of devices. The results so far seem promising, but we want to complete our due diligence and make sure this is the best approach to solving the problem. We know that OEM labs and tech reviewers put a lot of time and effort into compiling databases of results, so we hope to provide a solution that minimizes results disruption and inconvenience for WebXPRT 4 users. Ideally, folks would be able to integrate scores from the new build without any questions or confusion about comparability.

We don’t yet have an exact release date for a new WebXPRT 4 build, but we can say that we’re shooting for the end of October. We appreciate everyone’s patience as we work towards the best possible solution. If you have any questions or concerns about an updated version of WebXPRT 4, please let us know.

Justin

Posted in benchmark, Benchmark metrics, browser performance, Browser-based benchmarks, Performance benchmarking, results, WebAssembly, WebXPRT, WebXPRT 4 | Tagged , , , , , , , , , , |

An update on the issue with WebXPRT 4 in iOS 17

Justin Greene
By Justin Greene
on September 28, 2023

Recently, we informed XPRT blog readers that after updating Apple iPhones and iPads to iOS and iPadOS 17, respectively, we began to see WebXPRT 4 failures on those devices. In the Safari and Google Chrome browsers, WebXPRT 4 test runs were freezing while running the Encrypt Notes and OCR Scan workload. We were able to replicate the issue on every iOS/iPadOS 17 device we tested, and we also confirmed that WebXPRT 4 continues to run without issues on other non-iOS platforms.

Our team has been investigating the situation, and we’ve made some progress. It’s clear that the failed test runs are getting stuck when the WASM-based Tesseract.js Optical Character Recognition (OCR) engine attempts to scan a shopping receipt. During our research, we’ve discovered an issue when the current Tesseract.js engine runs on iOS 17. This issue is broader than WebXPRT 4, and the Tesseract team is aware of the problem. Future versions of iOS 17 or later versions of Tesseract.js may include fixes for the problem, but unfortunately, we don’t know whether or when a fix will be available.

We’re currently investigating possible workarounds for the problem, and hope to be able to start testing soon. Our goal is that any solution we implement will not significantly affect existing WebXPRT 4 scores on non-iOS 17 platforms.

We will continue to share any substantive progress updates with readers here in the blog. Once again, we apologize for any inconvenience this issue causes for WebXPRT 4 users, and we appreciate your patience while we work toward a solution. If you have any questions or comments, please feel free to contact us!

Justin

Posted in Apple, benchmark, BenchmarkXPRT, browser performance, Browser-based benchmarks, Cross-platform benchmarks, Google Chrome, iOS, Performance benchmarking, Phones, Safari, Tablets, WebXPRT, WebXPRT 4 | Tagged , , , , , , , , , , , , |

Investigating a possible issue with WebXPRT 4 in iOS 17

Justin Greene
By Justin Greene
on September 14, 2023

Yesterday, Apple revealed the iPhone 15 and iPhone 15 Pro at its annual fall event, along with a new version of the iOS mobile operating system (iOS 17). The official iOS 17 launch will take place on September 18th, but before then, users of newer iPhones can install the OS via the Apple Beta Software Program.

Today, a tech journalist informed us that during their testing of iPhone 15 and iPhone 15 Pro with iOS 17 Beta models, WebXPRT 4 has been freezing while running the Encrypt Notes and OCR Scan workload in the Safari 17 browser. Here in the lab, we were able to immediately replicate the issue on an iPhone 12 Pro with iOS 17 Beta model.

Our initial troubleshooting confirmed that WebXPRT 3 successfully runs to completion on iOS 17 Beta, so it appears that the problem is specific to WebXPRT 4. We also confirmed that WebXPRT 4 freezes at the same place when running in the Google Chrome browser on iOS 17 Beta, so we know that the problem does not occur only in Safari.

We’re currently investigating the issue, and will publish our findings here in the blog as soon as we feel confident that we’ve identified both the root cause and a workable solution, if a solution is necessary. One reason a solution would not be necessary is that the issue is a bug on the iOS 17 Beta side that Apple will resolve before the official launch.

We apologize for any inconvenience this issue might cause for tech reviewers and iPhone users, and we appreciate your patience while we figure out what’s going on. If you have any questions about WebXPRT 4, please don’t hesitate to ask!

Justin

Posted in Apple, benchmark, browser performance, Browser-based benchmarks, Cross-platform benchmarks, Google Chrome, iOS, iPhone, Performance benchmarking, Phones, Safari, WebXPRT, WebXPRT 3, WebXPRT 4 | Tagged , , , , , , , , , , , , |

Check out the other XPRTs:

Forgot your password?
© Principled Technologies, Inc. 2024   |   Privacy policy

Cookie Notice: Our website uses cookies to deliver a smooth experience by storing logins and saving user information. By continuing to use our site, you agree with our usage of cookies as our privacy policy outlines.

x