Session: A Tug-of-War: Performance vs Security Tradeoff in Containers
Cloud providers prefer containers over virtual machines (VMs) for their lightweight and scalability. Kubernetes provides scalability of containers across a cluster of nodes using various container runtimes. Though container vulnerabilities can compromise the security of co-located containers or the entire host as they share the operating system, VM-hardened container runtimes can help prevent the propagation of vulnerabilities. However, these runtimes are not entirely bug-free, so to protect bare metal hosts from runtime bugs and vulnerabilities, many sites may choose to deploy these Kubernetes clusters in VMs. We refer to them as nested clusters. The vulnerable surface of the cluster is mostly contained within the VM and not propagated to the bare-metal hosts, avoiding a full-blown attack, but this brings virtualization overhead. Though existing academic and industry contributions focus on the security and performance of container runtimes on bare-metal Kubernetes clusters, they lack rigorous empirical data on performance analysis and the feasibility of running different runtimes in a nested Kubernetes cluster environment. We will discuss the challenges of running Kubernetes clusters with different runtimes, such as containerd, kata containers, urunc, and kubevirt, on bare metal and VM. We have also analyzed performance metrics such as network, memory, CPU, and disk using benchmarks such as iPerf, Redis, stress-ng, pgbench, and fio on both hosts. Finally, we address performance bottlenecks with our mitigation strategies. Our evaluation and mitigation strategies help runtimes in the nested Kubernetes cluster perform almost the same as those in the bare-metal cluster and improve runtimes that perform poorly on the bare-metal cluster due to hostile configurations. Finally, our contributions help select the optimal runtime and host for the Kubernetes cluster, ensuring container security with minimum performance trade-off.
