CloudXPRT is a cloud benchmark that can accurately measure the performance of modern, cloud-first applications deployed on modern infrastructure as a service (IaaS) platforms, whether those platforms are paired with on-premises (datacenter), private cloud, or public cloud deployments. The benchmark currently supports two workloads:
- In the web microservices workload, a simulated user logs in to a web application that does three things: provides a selection of stock options, performs Monte-Carlo simulations with those stocks, and presents the user with options that may be of interest. The workload reports performance in transactions per second, which testers can use to directly compare IaaS stacks and to evaluate whether any given stack is capable of meeting service-level agreement (SLA) thresholds.
- The data analytics workload calculates XGBoost model training time. XGBoost is a gradient-boosting framework that data scientists often use for ML-based regression and classification problems. The purpose of the workload in the context of CloudXPRT is to evaluate how well an IaaS stack enables XGBoost to speed and optimize model training. The workload reports latency and throughput rates. Testers can use this workload’s metrics to compare IaaS stack performance and to evaluate whether any given stack is capable of meeting SLA thresholds.
Before getting started, please note the following system requirements:
- Web microservices on-premises testing - Ubuntu 20.04.2 or 22.04.
- Web microservices CSP (AWS/Azure/GCP) testing - Ubuntu 18.04, 20.04.2, or 22.04.
- Data analytics on-premises testing - Ubuntu 20.04.2.
- Data analytics CSP (AWS/Azure/GCP) testing - Ubuntu 18.04 or 20.04.2.
- CloudXPRT is designed to run on high-end servers. For functional testing, physical nodes or VMs under test must meet the following minimum specifications:
- 16 logical or virtual CPUs
- 8 GB RAM
- 10 GB of available disk space (50 GB for the data analytics workload)
The table below displays currently available CloudXPRT test packages. To download the correct package, click the Download link in the row corresponding to the workload you wish to test. The Readme links in the table’s far right column will provide the readme for each respective installation package.