A Principled Technologies report: Hands-on testing. Real-world results.
Accelerate your containerized workloads with VMware vSphere Kubernetes Service
On Kafka and OLTP workloads, VMware vSphere Kubernetes Service outperformed an OpenShift 4.19 solution in our testing
Enterprises have many options for tailoring Kubernetes deployments to their performance needs, operational models, and existing infrastructure. Two main approaches, virtualized and bare metal, offer considerable differences. Virtualized Kubernetes environments can offer resource isolation, simple management, and shared infrastructure across diverse workloads. Bare metal Kubernetes deployments can minimize abstraction layers and provide direct access to hardware resources without overhead. Understanding the performance impact of these approaches for workloads, such as modern streaming data pipelines or traditional online transaction processing, can help you choose the Kubernetes platform that more closely aligns with your operational goals.
We set out to compare the performance of two popular Kubernetes platforms: VMware® vSphere® Kubernetes Service (VKS) with VMware Cloud Foundation (VCF) and Red Hat® OpenShift® Container Platform (OCP) on bare metal. VKS represents a VM-based Kubernetes approach and Red Hat OCP on bare metal runs Kubernetes directly on hardware. To capture performance data of containerized modern and traditional workloads, we used both a Kafka workload and a HammerDB TPROC-C workload with PostgreSQL. On both workloads, the VKS solution provided significantly better performance than the Red Hat OCP bare metal solution. Our report explains our findings and how your choice of Kubernetes platform could impact your business.
Our test environment
We tested the two solutions remotely on two four-node clusters that Broadcom supplied and that we validated. Each Kubernetes solution ran on a four-node cluster of Dell™ PowerEdge™ R640 servers. The two clusters had the same hardware resources and differed only in how they handle Kubernetes containers. VKS runs Kubernetes on VMs on VMware ESXi™ hosts of VMware Cloud Foundation™ (VCF) 9.0.1, while Red Hat OCP on bare metal runs Kubernetes directly on hardware using OpenShift 4.19.
We ran each test three times and report the median results. To learn more about our testing and see our methodologies, read the science behind the report.
What we found: Kafka
Scale real-time data streaming with less lag
Many modern organizations rely on the open-source, distributed event-streaming platform Apache Kafka for ingesting and processing streaming data tied to real-time decision-making, digital services, and more.1 As the volume of real-time streaming data increases, sustaining high throughput and low latency becomes essential for keeping Kafka applications responsive and reliable to meet critical service-level agreements (SLAs) and keep data moving for users. Even small delays can impact application behavior, user experience, and downstream systems that depend on immediate data availability.
How we tested
To understand how both Kubernetes platforms can handle real-world Kafka demands, we deployed twelve Kafka pods, three per Kubernetes worker node, to both Kubernetes platforms. On the VCF testbed, we deployed a single Kubernetes worker node per physical host. Each pod contained a Kafka application and necessary resources. We also ran client pods, which generated and sent records as the producers, on the servers.
We started with a single producer and a single platform, or “topic,” and ran the producer workload. Then we scaled up the number of topics, with each topic having a single producer, and ran the workload again at each level (two, four, six, and eight topic/producer pairs).
Throughput
For enterprises that depend on rapid ingestion, such as those in retail, logistics, and fintech, higher throughput can mean systems process more data in less time. This can help reduce backlogs, keep dashboards up to date, and ensure that downstream applications receive fresh data as quickly as possible.
With one topic, the Kubernetes solutions supported equivalent throughput, showing that VKS can match bare-metal performance even at low levels of contention and with little to no overhead from vSphere virtualization. As we increased the number of topics, VKS delivered higher throughput than OCP. Figure 1 shows all the throughput data from our Kafka producer testing.
Kafka producer workload throughput results from the VKS and OCP solutions. Source: PT.
Latency
Low and consistent latency is central to the success of streaming applications. Faster response times help ensure that fraud detection engines remain accurate, supply chain systems stay synchronized, and user-facing applications provide timely updates. In our testing, the VKS solution supported lower latency at every level (see Figure 2).
Not only did the VKS solution support lower latency, but it also maintained that advantage with a larger throughput pipeline with two additional topics (22 ms for the VKS solution with six producers vs. 39 ms for the OCP solution with four topics). Despite latency increasing with load, the VKS solution continued to push more data than the OCP solution.
Kafka producer workload latency results from the VKS and OCP solutions. Source: PT.
Meeting throughput and latency targets
Every team sets its own latency thresholds, which define the limits of acceptable performance, based on use case and preference. For our testing, we set a latency threshold of 50 milliseconds, a publisher latency that could introduce noticeable delays in the end-to-end latency of real-time streaming.2 While still low, latency above this level could indicate a bottleneck with the system, which could affect transaction processing, event sequencing, and other time-sensitive streaming operations.
Based on this threshold, the VKS solution delivered acceptable performance all the way up through 6 topics, while the OCP solution could support only 4 topics with latency under 50ms, meaning that the VKS solution could support 50% more topics with acceptable latency.
At 8 topics, both the VKS solution and the OCP solution surpass the 50ms latency threshold we set. For organizations with a higher latency threshold, however, it may be valuable to see that the VKS solution delivers significantly higher throughput and lower latency than its competitor.
What we found: PostgreSQL
Support faster transaction processing
SQL databases form the operational backbone of organizations of all sizes, from mom-and-pop retail stores to large, multinational enterprises. Because many of these databases are mission-critical, maintaining high and reliable performance is essential.
For our testing, we used PostgreSQL, an open-source relational database used for “many web, mobile, geospatial, and analytics applications.”3 For real-world results, we used the HammerDB TPROC-C workload, which simulates five types of common online transaction processing.
The VKS solution provided dramatically stronger performance than the OCP solution (see Figure 3). With VKS, the system supported 80 percent more new orders per minute (NOPM) and significantly lower latency. Note that in order to achieve the best possible results for each solution, we tuned the number of virtual users with which we tested and present the results for both solutions with the highest NOPM.
When your database can process more transactions in the same amount of time, your customers and staff get a more responsive experience, critical for engagement and productivity.
HammerDB TPROC-C results from the VKS and OCP solutions using a PostgreSQL database. Source: PT.
HammerDB TPROC-C latency results from the VKS and OCP solutions using a PostgreSQL database. We took the weighted average of the six latency measurements that the benchmark provided. Source: PT.
Comparing to Red Hat OCP in a virtualized environment
We also tested virtualized OCP in the same VCF environment as the VKS solution, to compare how virtualized OCP handles virtual Kubernetes environments to both VKS and OCP on bare metal. For the Kafka producer workload testing, virtualized OCP delivered comparable throughput to VKS, edging out VKS by a small margin running one and two Kafka topics. At four topics, the throughput advantage shifted to VKS, and when running six topics, VKS supported 5 percent more throughput than the virtualized OCP solution. Latency during the Kafka workload showed a different story: while virtualized OCP offered slightly lower latency with a single topic, latency increased significantly at scale, exceeding 100 ms with six topics. In contrast, VKS maintained lower latency across all multi-producer scenarios, meeting our 50s threshold at all levels except eight topics. Against the bare metal OCP environment, virtualized OCP handled greater throughput and supported lower latency at every level, which reaffirms the notion that virtualized Kubernetes deployments can match performance of bare metal Kubernetes environments for single Kafka applications and then surpass performance as applications scale. Table 1 presents the Kafka results for all three solutions.
Table 1: Kafka producer workload results from the VKS solution and two OCP configurations. Higher throughput and lower latency are better Source: PT.
VMware vSphere Kubernetes Service
Red Hat OpenShift Container Platform (virtualized on VCF)
VKS win % vs. OCP (virtualized)
Red Hat OpenShift Container Platform (bare metal)
VKS win % vs. OCP (bare metal)
One topic
Throughput (MB/sec)
221.42
222.84
-0.64%
221.43
-0.01%
Latency (ms)
1.84
1.51
-21.85%
1.99
7.54%
Two topics
Throughput (MB/sec)
436.56
445.62
-2.03%
420.10
3.92%
Latency (ms)
2.47
3.71
33.33%
6.64
62.77%
Four topics
Throughput (MB/sec)
845.04
843.04
0.24%
723.01
16.88%
Latency (ms)
8.24
14.20
41.96%
39.00
78.87%
Six topics
Throughput (MB/sec)
1,202.66
1,146.54
4.89%
870.81
38.11%
Latency (ms)
22.51
117.91
80.91%
79.62
71.73%
Eight topics
Throughput (MB/sec)
1,450.24
1,232.84
17.63%
833.72
73.95%
Latency (ms)
106.06
176.88
40.04%
156.99
32.44%
In our PostgreSQL OLTP testing with HammerDB, VKS outperformed the virtualized OCP deployment by a small margin (see Table 2). The virtualized OCP deployment outperformed the bare metal OCP environment.
Table 2: HammerDB results from the VKS solution and two OCP configurations. Note that we did not gather latency data on the virtualized OHP solution, so we cannot report it here. Source: PT.
VMware vSphere Kubernetes Service
Red Hat OpenShift Container Platform (virtualized on VCF)
VKS win % vs. OCP (virtualized)
Red Hat OpenShift Container Platform (bare metal)
VKS win % vs. OCP (bare metal)
New orders per minute (NOPM)
112,947
105,706
6.85%
62,459
80.83%
Latency (ms)
18.33
N/A
N/A
66.57
72.46%
Conclusion
Kubernetes powers many real-time services, streaming analytics, and mission-critical databases. These workloads demand high throughput, low latency, and the ability to scale efficiently as data volumes grow. In our testing, a VMware VKS virtualized Kubernetes deployment delivered stronger performance than a Red Hat OCP bare metal environment across both Kafka producer and OLTP workloads.
VKS sustained higher throughput for Kafka producer workloads, including up to 73 percent more throughput and up to 78 percent lower latency, and supported more topics before exceeding the potentially problematic 50ms latency threshold.4 In OLTP testing with PostgreSQL, VKS delivered 80 percent more NOPM than OCP on bare metal. Together, these results indicate that deploying a virtualized Kubernetes environment with VKS can deliver faster modern application responses, improved real-time data processing, and more efficient use of infrastructure.
Principled Technologies is a registered trademark of Principled Technologies, Inc.
All other product names are the trademarks of their respective owners.
Principled Technologies disclaimer
Principled Technologies is a registered trademark of Principled Technologies, Inc. All other product names are the trademarks of their respective owners.
DISCLAIMER OF WARRANTIES; LIMITATION OF LIABILITY: Principled Technologies, Inc. has made reasonable efforts to ensure the accuracy and validity of its testing, however, Principled Technologies, Inc. specifically disclaims any warranty, expressed or implied, relating to the test results and analysis, their accuracy, completeness or quality, including any implied warranty of fitness for any particular purpose. All persons or entities relying on the results of any testing do so at their own risk, and agree that Principled Technologies, Inc., its employees and its subcontractors shall have no liability whatsoever from any claim of loss or damage on account of any alleged error or defect in any testing procedure or result.
In no event shall Principled Technologies, Inc. be liable for indirect, special, incidental, or consequential damages in connection with its testing, even if advised of the possibility of such damages. In no event shall Principled Technologies, Inc.’s liability, including for direct damages, exceed the amounts paid in connection with Principled Technologies, Inc.’s testing. Customer’s sole and exclusive remedies are as set forth herein.