Kubernetes Security: CSPM’s Blind Spot

This article focuses on a critical missing component of Cloud Security Posture Management (CSPM): visibility into Kubernetes. Scroll down to read all you need to know about it. 

Today, almost every business uses some form of cloud service. The fact that trusted technology companies (Amazon, Google, Microsoft) dominate the cloud market, has led to the false perception that the cloud is secure by default. Yet according to the 2022 Thales Cloud Security Study, 45% of organizations have experienced a data breach or failed an audit for the data or for applications running in their cloud. No industry or company division is immune to the many risks cloud pose, including malware, ransomware, and denial of service. Big players too, like Facebook, LinkedIn, Accenture, and Marriott International, have all fallen victim to such incidents.

There are many automated tools available to identify risks and solve potential problems in order to create a more secure cloud. Gartner came up with the term Cloud Security Posture Management (CSPM) for this category of applications. The primary goal of these tools is to explore and monitor IaaS and PaaS environments in order to find potential risks and automatically fix them. This mechanism saves valuable engineering time.

This article focuses on a critical missing component of CSPM: visibility into Kubernetes. The Kubernetes container orchestration system is unlike other cloud services in its architecture and layered workload management. For this reason, current CSPM tools fall short when it comes to securing Kubernetes clusters.

Kubernetes: Operating System of the Cloud

Kubernetes is the leading container orchestration platform. It offers critical features such as self-healing application management and service discovery out of the box. These features, and its extensibility, make Kubernetes much more than just a container orchestrator. In fact, it works in a similar way to an operating system, handling resource allocation, scheduling, networking, and data persistence.

Let’s review in brief how Kubernetes manages an application in the cloud. As an API-centric platform, Kubernetes and all its resources are declarative definitions. This requires specifying which container image and how many instances are required. For each container group, referred to as a pod in Kubernetes, it finds a suitable worker node to run.

If there are storage requirements, Kubernetes connects to the cloud storage provider to ensure that volumes are reserved, attached to nodes, and available to pods. When the pods want to connect to other containers or outside the cluster, Kubernetes provides in-cluster networking programming as well. If there are configurations to be distributed to pods, Kubernetes offers a way to access that data. Sensitive data such as certificates, passwords, or usernames are often distributed to pods in a similar fashion.

In order to access your application from the internet, Kubernetes offers gateway resources to make it accessible behind a load balancer service. As the application becomes more popular, with many incoming requests, Kubernetes can automatically scale up the number of pods by observing metrics. Thus, ensuring that all instances are healthy. It is even capable of increasing the cluster size if there is no room left for new pods.

All of these characteristics and processes are similar to how a cloud provider works. In other words, Kubernetes creates and operates a cloud within the cloud.

Kubernetes and Security

Though Kubernetes works like a cloud operating system, there is a critical missing piece: security by default. In its default form, Kubernetes is a platform open to all kinds of threats, in addition to the risks of the cloud platform. According to a Red Hat report from May 2022, 93% of users reported they’d had at least one security event in their Kubernetes environments in the preceding 12 months. Furthermore, 55% of participants reported having to delay or slow down application deployment due to a security concern. In short, the results show that Kubernetes security is critical, with nearly every user having experienced incidents. In fact, half of the participants stopped working on their deployments to Kubernetes due to security concerns.

Let’s take a look at the lifecycle of the applications running on Kubernetes to better understand the security challenges.

Build

In this first stage, applications are built and packaged into containers.

• Kubernetes nodes are regular virtual machines shared by multiple applications running in the cluster. Therefore, ensuring strong isolation between applications is critical for hardening and avoiding escalation attacks.
• Kubernetes distributes and runs container images without checking their content. When there are vulnerabilities in the application, library, or the base image of the containers, they create exposure risks. It is vital to scan the container images in the CI stage before deploying to the cluster.

Deploy

The deployment stage focuses on putting applications into clusters:

• Kubernetes clusters expose a management API to deploy and manage applications. Thus, access to Kubernetes API should be limited and controlled with RBAC, often integrating with corporate identity access management (IAM) tools.
• Kubernetes resources are declarative definitions that should be checked for security configurations, default values, and custom policies before deploying to a cluster. Since Kubernetes is highly scalable, a wrong setting could easily lead to serious threats at a large scale.
• Barring unique circumstances, the best practice is to have only one way to deploy applications into production. Aside from the security advantages, the GitOps model of a repository of resource files provides an audit trail and helps cut down on configuration drift in your clusters as well as human error.

Runtime

Runtime covers all the operations after the deployment of applications:

• By default, Kubernetes has an all-permissive network communication between pods. In order to manage communication, it offers a highly configurable set of network policy resources. Pod communication can be limited for data-sensitive or critical applications.
• Kubernetes clusters, and the applications running on them, should be tracked for audits and compliance requirements. This is possible with Kubernetes-native audit logs and third-party exporters to collect and analyze data.
• It is critical to have intrusion detection and prevention capabilities to block malicious activities and respond in case of an attack in the clusters.

When the security lifecycle is considered, it is clear that Kubernetes security brings its own challenges to the cloud ecosystem.

KSPM: CSPM Mindset for Kubernetes

From the cloud provider point of view, Kubernetes is a service similar to a managed database or a storage bucket. It has a dashboard in the cloud console, and with just a couple of clicks, you can have a running Kubernetes cluster. CSPM uses a similar approach to evaluate Kubernetes. It focuses on permissions for VM nodes, network access in cloud networking, configuration, and policy checks at cloud service level.

However, Kubernetes creates its own layer on top of all the cloud services, complete with its own access management, network, and configuration. The characteristics and internal operations of Kubernetes are no more within the scope of CSPM than any other application you might deploy on a VM, and therefore are not covered by it. Relying solely on the CSPM toolset leads to insecure Kubernetes clusters.

Kubernetes Security Posture Management (KSPM) is a set of tools and processes, similar to CSPM, only to secure Kubernetes clusters and workloads. The mindset is much like CSPM, but the approach is Kubernetes-native, with the following characteristics:

• Focuses on multiple layers, including the Kubernetes API, the workload inside the cluster, cloud infrastructure, and the network. CSPM tools focus on workloads running in the cloud such as databases, functions, or message queues. It is possible to deploy workloads such as databases, functions, and message queues into Kubernetes clusters; however, those workloads will not be checked by the CSPM tools.
• Works coherently with the extensibility of Kubernetes, such as a dynamically extended Kubernetes API with custom resources. When you open a cloud provider dashboard—or check its API documentation—there is a list of available services. CSPM tools protect and cover these services. However, Kubernetes provides the ability to extend its API dynamically, thus security posture management tools need to be adaptable. 
• Works with Kubernetes-native access controls such as RBAC and cloud-provider identity management in tandem. CSPM tools focus on identity and access management at the cloud level. When the applications run inside a Kubernetes cluster, access to the applications must be managed within the cluster as well. In addition, there are secrets and sensitive data living in the Kubernetes clusters—which need protection and access control. 
• Protects not only Kubernetes clusters, but also the application lifecycle, including CI/CD stages. Currently, there are no CSPM tools available in the market that cover the process of building container images with vulnerability scanning, deploying them in a secure and controlled manner, and providing runtime protection in the clusters.
• Aware of cross-cloud deployment. Kubernetes is a portable orchestration platform that could run on a single, hybrid, or multi-cloud environment. While CSPM’s target deployments in the public cloud, some of them require a separate deployment for each cloud environment. This requires additional consideration around cost, maintenance and the holistic view of the security of the Kubernetes infrastructure.
• Overlooks Kubernetes on-premises installations. Currently, CSPM tools in the market focus only on public cloud services to provide protection. However, Since Kubernetes can run on-premises, it also requires security posture management for on-premises infrastructure and the applications running on it. 

Without these specific characteristics, CSPM barely scratches the surface of a Kubernetes cluster in terms of security posture management. However, a KSPM solution provides insights from inside the cluster and creates complete Kubernetes cluster security. 

KSPM in Action: Kubescape

Kubescape is an open-source Kubernetes-native security platform covering the entire Kubernetes security lifecycle and CICD pipeline. The tool implements the core principles of KSPM:

• Scans all Kubernetes configuration and resource manifest files; integrates into CI/CD stages to check before deployment to the clusters.
• Checks the worker node settings with a Kubernetes mindset, checking not only VM settings but also kubelet configurations.
• Ensures the security of the Kubernetes API server, since it is the entry point to the cluster.
• Consolidates all available data to discover unforeseen risks, such as showing if a container image with a security vulnerability is open to the outside with an ingress.
• Supports flexible deployment options. It will run wherever you run Kubernetes, be it public cloud, private cloud or in you data center.

Going Beyond CSPM

Securing a cloud-native environment with a one-size-fits-all approach is impossible due to its scalable and flexible nature. Even well-established CSPM solutions fail to allow for the distinctive characteristics of Kubernetes architecture and security lifecycle. This is a result of CSPM tools using an agentless approach as well as their reliance on the APIs of public cloud providers. However, there are Kubernetes-native resources in a Kubernetes cluster that are only accessible via the Kubernetes API which are not covered by CSPM solutions. Relying on a CSPM alone for securing Kubernetes in your cloud environment while neglecting KSPM leaves you open to threats. 

Start managing your Kubernetes securely with Kubescape, the open-source Kubernetes security platform. Watch a demo or sign up for free access to ARMO Platform, and discover additional enterprise features.

Editor’s note: This post was originally published in December 2022 and has been updated for accuracy and comprehensiveness.

Unifying AppSec, CloudSec and DevSec

The only runtime-driven, open-source first, cloud security platform:

Continuously minimizes cloud attack surface

Secures your registries, clusters and images

Protects your on-prem and cloud workloads

Powered by Kubescape & eBPF

See ARMO in Action