Data reduction testing

Determine data compression and deduplication ratios

Data center sprawl can be very costly, so storage solutions that minimize the amount of space your data occupies are increasingly valuable. In our data reduction testing, we measure before-and-after storage usage to evaluate the compression and deduplication capabilities of different storage solutions.

Power efficiency testing

Determine performance-per-watt metrics

The server selection process might initially focus on reliability and performance, but energy consumption is an increasingly important consideration. A server that delivers more work for each watt of electricity it consumes can translate to lower data center power and cooling expenses. In our power efficiency testing, we measure a solution’s energy usage and combine it with performance data to determine performance-per-watt metrics that help buyers make informed decisions.

VM and container density testing

Determine how many virtual machines or containers a solution can support

By selecting solutions that achieve greater VM or container density, companies maximize the value of virtualization and can save by being able to perform a given amount of work with a smaller investment. In our density testing, we typically select a workload, such as an OLTP database, and measure the performance of a single VM or container running that workload; we then have a baseline. We next add virtual machines or containers until the average per-VM/per-container performance falls well below the baseline. For example, if performance dropped at eight VMs, we would say the solution supported seven virtual machines at this level of performance.

PCMark

Demonstrate device performance in the workplace

PCMark is a benchmark set that, per UL, “features a comprehensive set of tests that cover the wide variety of tasks performed in the modern workplace.”¹ The main tool (PCMark 10 benchmark) evaluates PCs based on real-world tasks, such as web browsing, video calls, working with spreadsheets, photo editing, and more. Based on a system’s responsiveness while completing the tasks, PCMark delivers scores for each task and category of tasks, as well as an overall score—the higher the score, the smoother the experience end users can expect.

Along with benchmarks for applications and storage, PCMark also features a battery life test that measures how long a worker could expect their device to run on battery

1. “PCMark 10,” accessed February 9, 2023, https://benchmarks.ul.com/pcmark10.

Cinebench

Evaluate PC hardware performance

According to Maxon, “Cinebench offers a real-world benchmark that incorporates a user’s common tasks within Cinema 4D to measure a system’s performance.”¹ Outputting scores for both single-core and multi-core CPU performance, Cinebench measures how a device runs under a high CPU load, allows you to gauge how well the cooling system works during longer tasks, and tests how the device works with intensive 3D tasks.² Higher scores could indicate faster PC response times on graphics-intensive games, product development and design software, and scientific simulations.

1. “Cinebench,” accessed February 9, 2023, https://www.maxon.net/en/cinebench.
2. “Cinebench,” accessed February 9, 2023, https://www.maxon.net/en/cinebench.

MobileMark

Put battery life and device performance to the test

Designed to run on a PC device while it’s unplugged, MobileMark simulates a worker performing productivity and creativity tasks on applications such as Microsoft 365 apps, Google Chrome, and Adobe Creative Cloud. The benchmark reports battery life in hours and minutes while performing these tasks, a battery performance score, and an index score that shows the balance between battery life and performance.¹

1. “An Overview of MobileMark 25,” accessed February 9, 2023, https://bapco.com/wp-content/uploads/2022/11/bapco_mobilemark_25_white_paper_v1.2.pdf.

CrossMark

Evaluate the user experience

How quickly a device completes workloads (performance) and responds to a user’s input (responsiveness) both affect user experience. The CrossMark benchmark uses models of real-world applications to measure performance and responsiveness and assigns an overall score it bases on both. According to BAPCo, developers of CrossMark, it is a “cross-platform benchmark that enables direct comparison of desktops and touch enabled devices across Windows, iOS or macOS, and Android.”¹

1. “CrossMark,” accessed February 10, 2023, https://bapco.com/products/crossmark/.

PugetBench

Measure Adobe Creative Cloud capabilities

The Adobe Creative Cloud suite comprises applications that place different kinds of frequently heavy demands on compute resources. It can be challenging for creative professionals and those who buy gear for them to understand how well a given workstation can support these applications. Puget Systems has developed the PugetBench suite of benchmarks “to thoroughly test many of Adobe’s most popular applications using real-world projects and workflows.”¹

The suite includes PugetBench for Photoshop, PugetBench for Lightroom Classic, PugetBench for Premiere Pro, and PugetBench for After Effects.

1. “PugetBench for Adobe Creative Cloud,” accessed February 10, 2023, https://www.pugetsystems.com/labs/articles/PugetBench- for-Adobe-Creative-Cloud-1642/.

WebXPRT

Quantify browser performance

WebXPRT 4 is an industry-standard browser benchmark that compares the performance of web-enabled devices when executing real-world tasks. It contains HTML5, JavaScript, and WebAssembly-based scenarios that mirror activities users perform: Photo Enhancement, Organize Album Using AI, Stock Option Pricing, Encrypt Notes and OCR Scan using WASM, Sales Graphs, and Online Homework.

Principled Technologies is the publisher of the XPRT family of benchmarks and the administrator of the BenchmarkXPRT Development Community.

Procyon benchmark suite

Gauge device performance for professional users

The Procyon benchmark suite from UL comprises five benchmarks that target “professional users in industry, enterprise, government, retail and press.”¹ The Office Productivity Benchmark can measure Windows or macOS device performance for office productivity work using Microsoft Office apps. Also available for Windows or macOS, the Photo Editing Benchmark uses Adobe Lightroom to import, process, and modify a selection of images, then applies multiple edits and layer effects to a photograph in Adobe Photoshop. The AI Inference Benchmark uses neural network models such as MobileNet V3, Inception V4, SSDLite V3, and DeepLab V4. The Battery Life Benchmark measures the battery life of Windows laptops, notebooks, and tablets across different scenarios. And lastly, the Video Editing Benchmark measures the time it takes a PC to apply edits, adjustments, and effects and then export video project files to common formats in Adobe Premiere Pro. These latter three benchmarks are currently Windows-specific, but UL says macOS support is on the way.

1. UL, “UL Procyon benchmark suite,” accessed May 19, 2023, https://benchmarks.ul.com/procyon.

SYSmark

Measure system performance

According to BAPCo, developers of SYSmark, the benchmark measures system performance using “real-world applications and workloads.”¹ It generates an overall rating based on a combination of results that measure system performance while running office and media tasks and scenarios that model common pain points, such as launching files and multitasking. We can run the latest version, SYSmark 30, or the second-to-latest version, SYSmark 25, depending on your preferences.

1. “SYSmark 30,” accessed March 10, 2023, https://bapco.com/products/sysmark-30/.

Geekbench

Measure the single- and multi-core power of a system’s processor

Geekbench is a cross-platform utility that aims to measure the single-core and multi-core power of the central processing unit of a computer. Primate Labs, developer of Geekbench, says the latest version, Geekbench 6, “measures performance in new application areas including Augmented Reality and Machine Learning.”¹

1. Primate Labs, “Introducing Geekbench 6,” accessed March 8, 2023, https://www.geekbench.com/.

Maxon Redshift

Measure GPU-accelerated rendering performance

According to Maxon, Redshift is “a powerful GPU-accelerated renderer, built to meet the specific demands of contemporary high-end production rendering. Tailored to support creative individuals and studios of every size, Redshift offers a suite of powerful features and integrates with industry standard CG applications.”¹ We test a system’s performance by measuring how long the system takes to complete a RedShift render.

1. Maxon, “Redshift,” accessed February 10, 2023, https://www.maxon.net/en/redshift.

Blender

Assess CPU and GPU performance

The Blender benchmark measures CPU and GPU performance by determining the number of samples per minute a system can handle. Blender Open Data is a platform the Blender community provides to collect, display, and query the results of hardware and software performance tests.¹

1. Blender, “About,” accessed February 10, 2023, https://opendata.blender.org/about/.

Onshape

Measure CAD capabilities

Onshape is a computer-aided design software system, available using a software-as-a-service model. The Onshape online performance check provides the browser and GPU with an increasing workload and reports the device limits when the lines and triangles per second framerates start to drop.¹ The higher the framerate a device can support, the better the CAD performance users can expect.

1. Onshape forum, “Performance test results,” accessed February 10, 2023, https://forum.onshape.com/discussion/5225/performance-test-results.

Speedtest

Speedtest by Ookla tests the speed of devices’ connections to the internet. It outputs results in megabits per second (Mbps) for both downloads and uploads. To ensure that any differences in connection speed are due to the devices rather than the internet service provider or other external networking elements, when using this test for multiple devices, we run tests on all devices at the same time and in the same networking environment.

PassMark

Assess multiple elements of PC performance

PassMark offers a number of benchmarks for performance testing of Windows and Linux systems. Among others, these include BurnInTest, which quickly stress-tests PCs; PerformanceTest, which combines 28 speed tests for easy benchmarking of CPU, 2D and 3D graphics, disk, and memory; and MemTest86, which stress-tests a system’s memory.¹ Output depends on which PassMark benchmark you choose.

1. PassMark Software, “Software Products,” accessed February 13, 2023, https://www.passmark.com/products/index.php.

HDXPRT

Test how Windows 10 PCs handle media and more

HDXPRT 4 is a benchmark that assesses the performance of Windows 10 devices by measuring how well they handle real-world media tasks. HDXPRT 4 uses real commercial applications, such as Photoshop and MediaEspresso, to carry out tasks based on photo editing, video conversion, and music editing.

Principled Technologies is the publisher of the XPRT family of benchmarks and the administrator of the BenchmarkXPRT Development Community.

MobileXPRT

Assess Android device performance

MobileXPRT 3 is a benchmark for evaluating the capabilities of Android devices. It runs six performance scenarios: Apply Photo Effects, Create Photo Collages, Create Slideshow, Encrypt Personal Content, Detect Faces to Organize Photos, and Scan Receipts for Spreadsheet. In addition to scoring each scenario, it generates a single overall performance score.

Principled Technologies is the publisher of the XPRT family of benchmarks and the administrator of the BenchmarkXPRT Development Community.

CrXPRT

Measure Chromebook performance and battery life

The CrXPRT 2 benchmark evaluates the performance and battery life of Chromebooks. The CrXPRT performance test measures how quickly a Chromebook handles everyday tasks such as playing video games, watching movies, editing pictures, and doing homework. It generates an overall score and individual scores for each workload. The CrXPRT battery life test produces an estimated battery life, a separate performance score, and a frames-per-second (FPS) rate for a built-in HTML5 gaming component.

Principled Technologies is the publisher of the XPRT family of benchmarks and the administrator of the BenchmarkXPRT Development Community.

TouchXPRT

See how Windows 10 devices stack up

TouchXPRT is a benchmark for evaluating the performance of Windows 10 devices. It runs tests based on five common scenarios: Beautify Photos, Blend Photos, Convert Videos for Sharing, Create Music Podcast, and Create Slideshow from Photos. TouchXPRT produces results for each of the five test scenarios plus an overall score.

Principled Technologies is the publisher of the XPRT family of benchmarks and the administrator of the BenchmarkXPRT Development Community.

AIXPRT

Evaluate your system’s machine learning inference performance

AIXPRT is an AI benchmark tool that makes it easier to evaluate a system's machine learning inference performance by running common image-classification, object-detection, and recommender system workloads. AIXPRT includes support for the Intel OpenVINO, TensorFlow, and NVIDIA TensorRT toolkits to run image-classification and object-detection workloads with the ResNet-50 and SSD-MobileNet v1 networks, as well as a Wide and Deep recommender system workload with the Apache MXNet toolkit. The test reports FP32, FP16, and INT8 levels of precision. Test systems must be running Ubuntu 18.04 LTS or Windows 10, and the minimum CPU and GPU requirements vary by toolkit.

Principled Technologies is the publisher of the XPRT family of benchmarks and the administrator of the BenchmarkXPRT Development Community.

Task-based time testing

See how systems perform real-world tasks

A user’s real-world experience of their system depends a great deal on how long it takes to accomplish the tasks they do every day. We select tasks representative of real-world workflows—such as booting a laptop, opening a document, or exporting a photo—and use hand timers to measure how many seconds a system requires to complete them. We can perform this testing with any task or series of tasks, making it a flexible, realistic approach to highlighting how a system could boost (or harm) productivity.

Thermal testing (end-user)

Discover surface temperatures while idle or under stress

Especially for hybrid or remote workforces who may use devices on their laps, a notebook’s surface temperature can be an important consideration. Excessive heat can be a matter of discomfort or, in extreme cases, could spell hardware failure. By measuring devices’ surface temperatures, we can demonstrate which might offer a more comfortable user experience or be less susceptible to overheating. We can perform this testing while a machine is at rest or under load and supplement the data with thermal camera images.

Thermal testing (enterprise)

See which systems run cooler under load

For organizations with large data centers, power and cooling can be a significant expense. In addition to assessing the quality of power management software tools and a system’s energy efficiency (see “Manageability testing studies” and “Power efficiency testing”), we can measure how well a device’s cooling mechanisms are functioning by assessing how much heat the device is outputting and whether that heat is having an effect on the device’s performance. Depending on the exact tests we run, we may be able to use that data to determine how much money an organization might save on power and cooling by choosing that device.

3DMark

Evaluate graphics and compute performance with demanding gaming workloads

According to UL Solutions, maker of 3DMark, “3DMark is used by millions of gamers, hundreds of hardware review sites, and many of the world's leading manufacturers.”¹ 3DMark offers a suite of benchmarks, each of which runs intensive gaming workloads that put a system’s graphics and compute capabilities to the test. While it outputs useful results for gamers, 3DMark can also help indicate system performance while running compute-intensive workloads that deal with 3D objects.

1. UL Solutions, “3DMark user guide,” accessed March 9, 2023 https://support.benchmarks.ul.com/support/solutions/articles/44002146295-3dmark-user-guide

Acoustic testing

Assess how much noise a device emits

In office environments, background noise can disrupt users’ focus and even their memory.¹ While noise-canceling headphones or closed doors can help, these measures could also leave users feeling isolated. Instead, to cut down on distracting noises, businesses might consider quieter devices for their employees. We test devices’ noise outputs while those devices are idle or running an intensive workload. As another useful point of comparison, we can present these results in conjunction with real-world sound levels.

1. Simon Danbury and Dianne C. Berry, “Disruption of office-related tasks by speech and office noise,” accessed March 9, 2023 https://bpspsychub.onlinelibrary.wiley.com/doi/pdf/10.1111/j.2044-8295.1998.tb02699.x

Feature comparison

Compare features on multiple devices

Performance and battery life are critical concerns for buyers of end-user devices, but they’re not the only elements customers consider. We compare devices’ features in a host of areas, including size, weight, screen resolution, number and type of ports, camera and microphone specs, touchscreen and pen-and-ink options, convertibility, and much more. For any feature your device boasts, we can highlight its utility and functionality against the equivalent feature(s) from the competition.

Application verification

Verify apps are working as they should

If a user’s everyday work and/or their hobbies rely on specialty applications—such as CAD apps, engineering applications, or compute-intensive games—they need to know that those applications will run smoothly on any device they might purchase. We can install, open, and complete tasks in any application on any system to verify that the system can handle the application(s) and will function as a user would expect.

Speedometer

See how quickly web-based applications respond to user input

Speedometer is a browser-based responsiveness test for web applications. The benchmark simulates user actions in a demo app, then measures the time it takes to complete those tasks. A higher Speedometer score indicates faster responsiveness, which could translate to a smoother user experience.

Battery life testing

Test how long a device’s battery lasts under varying conditions

Battery life is a top concern for many buyers of end-user devices, whether the device is for business or personal use. We can assess a device’s battery life using a benchmark, such as PCMark or MobileMark, or by continuously running a real-world workload, such as local or streaming video playback, and measuring how long the device’s battery lasts before dying. We can also use this data to assess the efficiency of the battery in minutes per WHr.

Unigine benchmarks

Measure compute-intensive performance for gaming

The Unigine performance benchmarks “generate true in-game rendering workloads across multiple platforms” and are useful for assessing “the stability of PC hardware (CPU, GPU, power supply, cooling system) under extremely stressful conditions, as well as for overclocking.”¹ The three benchmarks—Superposition, Valley, and Heaven—incorporate interactivity and large maps of detailed 3D terrain, making them suitable for assessing a device’s potential gaming and/or 3D rendering performance.

1. Unigine, accessed March 15, 2023 https://benchmark.unigine.com/.

In-game benchmarks

Assess devices’ real-world gaming performance

If one of a PC user’s primary concerns is gaming performance, what better way to assess the device than using a benchmark built into a game? Several PC games, including Sid Meier’s Civilization IV and Deus X: Mankind Divided, include built-in benchmarks that measure metrics such as AI turn speed and frame rate. These benchmarks can also be valuable for an audience that does not prioritize gaming, allowing us to assess how a device performs under a very graphics-intensive workload.

Basemark GPU

Evaluate a device’s graphics performance

Basemark GPU, which Basemark calls “the ultimate graphics performance benchmark,” evaluates graphics performance on Windows, macOS, Ubuntu, Linux, Android, and iOS devices.¹ It does this by running through a graphically intensive scene and generating a score that users can compare to other Basemark scores. It offers two modes: High Quality, for desktop systems, and Medium Quality, for mobile systems.

1. Basemark, “Basemark GPU,” accessed March 6, 2023 https://www.basemark.com/benchmarks/basemark-gpu/.

Durability testing (end-user)

Discover how a device holds up under wear and tear

Many users primarily rely on their devices at home and in the office, packing them carefully into backpacks for the transition. But others need a device that can stand up to wear and tear—environments as varied as hospitals, construction sites, and classrooms all require tech that delivers both high performance and strong durability. We can test a laptop’s or tablet’s toughness from multiple perspectives, including how it handles drops, spills, scratches, heavy rain, bright sunlight, extreme temperatures, and more.

Thermal testing (enterprise)

See which systems run cooler under load

For organizations with large data centers, power and cooling can be a significant expense. In addition to assessing the quality of power management software tools and a system’s energy efficiency (see “Manageability studies” and “Power efficiency testing”), we can measure how well a device’s cooling mechanisms are functioning by assessing how much heat the device is outputting and whether that heat is having an effect on the device’s performance. Depending on the exact tests we run, we may be able to use that data to determine how much money an organization might save on power and cooling by choosing that device.

Deployment studies (on end-user devices)

Quantify and qualify the benefits of automated end-user device deployment

Deploying desktops and laptops to hundreds or thousands of employees has never been a straightforward task for IT, and today’s increasingly remote or hybrid environments have only added complexity. We can test your deployment tools and services the way real IT teams would use them, assessing how much time and effort they save compared to a manual deployment process or competing solution. Our reports incorporate in-depth discussions of our experience using your solution, focusing on benefits to both IT and end-users.

SPECviewperf

Quantify graphics performance with workloads based on professional applications

For systems running under OpenGL and DirectX application programming interfaces, SPECviewperf is “the worldwide standard for measuring graphics performance based on professional applications,” according to the SPEC organization.¹ Running rendering workloads based on applications such as 3ds Max, Maya, and Solidworks, SPECviewperf outputs a score where higher numbers indicate better performance.

1. Standard Performance Evaluation Corporation, “SPECviewperf® 2020 v3,” accessed March 9, 2023 https://gwpg.spec.org/benchmarks/benchmark/specviewperf-2020-v3-1/.

SPECworkstation

Assess workstation performance with a variety of workloads

The SPECworkstation benchmark suite tests a device’s processor, graphics card, I/O, and memory bandwidth performance across a series of 30 workloads containing almost 140 total tests. In addition to general tasks, workload categories include tests specific to industries such as product development, financial services, energy (oil and gas), and more. The SPEC organization modeled tests on real-world applications relevant to key sectors, such as the Blender and Maya apps for the media and entertainment industries.¹ The higher each workload score, the faster you might expect a device to perform for employees doing that type of work.

1. Standard Performance Evaluation Corporation, “SPECworkstation® 3.1,” accessed March 9, 2023 https://gwpg.spec.org/benchmarks/benchmark/specworkstation-3_1/.

Backup and restore studies (end-user)

Measure the efficacy of any end-user backup and recovery solution

Losing laptop data can be a logistical and personal nightmare—not to mention a major impact to productivity at work. We can test the efficacy, ease of use, and features of your backup and recovery services and your competitors’, proving time savings and exploring your solution’s benefits to users and organizations alike.