Project post originally published on DZone by Jim Bugwadia
Learn how to audit your clusters for compliance with the latest Kubernetes Pod Security Standards without installing anything in the cluster.
In this post, I will show you how to audit your clusters for compliance with the latest Kubernetes Pod Security Standards without installing anything in the cluster.
Pods are the basic unit of execution in Kubernetes, and pod security is necessary for all clusters. Without pod security checks enabled, any user with permissions to run a pod can elevate privileges. Attackers can exploit the lack of pod security to execute a container escape. All clusters, including Dev/Test and staging clusters, which are common entry points for attackers, should implement pod security.
The Kubernetes project publishes the Pod Security Standards which contains security controls organized into three profile levels that should be enforced.
Kubernetes v1.25 offers an in-tree admission controller for the pod security standards, which offers namespace-level validation and enforcement and needs to be configured at the API server. In most cases, more granular controls will be needed. The blog post “Examining Pod Security Admission” provides a good analysis.
To check for compliance with the Kubernetes Pod Security Standards, we will run the Kyverno CLI from outside the cluster and execute policies for each of the controls defined in the pod security standards. To perform the audit you will need access to the cluster via kubectl, but do not need to install anything in the cluster.
Step 1: Install Krew and Kustomize if Needed
Krew is a package manager for kubectl the Kubernetes CLI (installation instructions are available).
Kustomize is a kubectl subcommand that simplifies configuration management. Since the version distributed with kubectl tends to be old, check this link to install the latest version.
Step 2: Install the Kyverno kubectl Plugin
Next, install the kyverno kubectl plugin:
kubectl krew install kyvernoThe output should look like this:
Updated the local copy of plugin index.
Installing plugin: kyverno
Installed plugin: kyverno
\
| Use this plugin:
| kubectl kyverno
| Documentation:
| https://github.com/kyverno/kyverno
| Caveats:
| \
| | The plugin requires access to create Policy and CustomResources
| /
/
WARNING: You installed plugin "kyverno" from the krew-index plugin repository.
These plugins are not audited for security by the Krew maintainers.
Run them at your own risk.Step 3: Scan Your Cluster
Run the kyverno command line as follows:
kustomize build https://github.com/kyverno/policies/pod-security | kubectl kyverno apply --cluster -The above command runs against the entire cluster. You can optionally use the --namespace option to scan a single namespace.
Here is the output from my cluster’s default namespace where I ran a busybox image:
❯ kubectl run busybox --image busybox
pod/busybox created❯ kustomize build https://github.com/kyverno/policies/pod-security | kubectl kyverno apply --cluster --namespace default -Applying 17 policies to 1 resource...
policy disallow-capabilities-strict -> resource default/Pod/busybox failed:
1. require-drop-all: validation failure: Containers must drop `ALL` capabilities.
policy disallow-privilege-escalation -> resource default/Pod/busybox failed:
1. privilege-escalation: validation error: Privilege escalation is disallowed. The fields spec.containers[*].securityContext.allowPrivilegeEscalation, spec.initContainers[*].securityContext.allowPrivilegeEscalation, and spec.ephemeralContainers[*].securityContext.allowPrivilegeEscalation must be set to `false`. Rule privilege-escalation failed at path /spec/containers/0/securityContext/
policy require-run-as-nonroot -> resource default/Pod/busybox failed:
1. run-as-non-root: validation error: Running as root is not allowed. Either the field spec.securityContext.runAsNonRoot must be set to `true`, or the fields spec.containers[*].securityContext.runAsNonRoot, spec.initContainers[*].securityContext.runAsNonRoot, and spec.ephemeralContainers[*].securityContext.runAsNonRoot must be set to `true`. Rule run-as-non-root[0] failed at path /spec/securityContext/runAsNonRoot/. Rule run-as-non-root[1] failed at path /spec/containers/0/securityContext/.
policy restrict-seccomp-strict -> resource default/Pod/busybox failed:
1. check-seccomp-strict: validation error: Use of custom Seccomp profiles is disallowed. The fields spec.securityContext.seccompProfile.type, spec.containers[*].securityContext.seccompProfile.type, spec.initContainers[*].securityContext.seccompProfile.type, and spec.ephemeralContainers[*].securityContext.seccompProfile.type must be set to `RuntimeDefault` or `Localhost`. Rule check-seccomp-strict[0] failed at path /spec/securityContext/seccompProfile/. Rule check-seccomp-strict[1] failed at path /spec/containers/0/securityContext/.
pass: 15, fail: 4, warn: 0, error: 0, skip: 38The output above shows that the busybox pod violates four controls in the pod security standards.
Conclusion
Kyverno is a powerful and simple tool for Kubernetes security and automation. It typically runs as an admission controller, in the Kubernetes control plane.
The Kyverno CLI can execute Kyverno policies against a set of files containing Kubernetes resource YAML declarations or can execute policies against a cluster. Here, we used the Kyverno CLI to execute policies that implement the Pod Security Standards against a cluster.
As a next step, you can install Kyverno in your cluster or try the free trial of the Nirmata Kubernetes Policy Manager.