Pod Security Policies

A Pod Security Policy is a cluster-level resource that controls security sensitive aspects of the pod specification. The PodSecurityPolicy objects define a set of conditions that a pod must run with in order to be accepted into the system, as well as defaults for the related fields.

Why enable Pod Security Policy?

Enabling Pod Security Policies allow us to pass a number of requirements (not all) as defined in CIS Benchmarks. The following benchmark requirements are now enabled/enforced by enabling the Pod Security Policy that comes with Component Pack:
  • 4.1 Create a user for the container
    • All containers pass this requirement
  • 5.4 Do not use privileged containers
    • All containers pass this requirement except for ElasticSearch
  • 5.21 Ensure the default seccomp profile is not Disabled
    • All containers pass this requirement except for cnx-ingress
  • 5.25 Restrict container from acquiring additional privileges
    • All containers pass this requirement except for cnx-ingress

Component Pack Pod Security Policies

The following pod security policies (PSP) are applied with the k8s-psp Component Pack helm chart:

Table 1. k8s-psp Component Pack helm chart
NAME PRIV CAPS SELINUX RUNASUSER FSGROUP SUPGROUP READONLYROOTFS VOLUMES
privileged true * RunAsAny RunAsAny RunAsAny RunAsAny false *
restricted false <blank> RunAsAny MustRunAsNonRoot MustRunAs

- from 1 to 65535

 MustRunAs

- from 1 to 65535

 false configMap

emptyDir

projected

secret

downwardAPI

persistentVolumeClaim
filebeat false <blank> RunAsAny RunAsAny MustRunAs

- from 1 to 65535

 MustRunAs

- from 1 to 65535

 false configMap

emptyDir

projected secret

downwardAPI

persistentVolumeClaim

hostPath
infra-mongo-solr-zookeeper

NB: ServiceAccount & RBAC required

 false <blank> RunAsAny RunAsAny MustRunAs

- from 1 to 65535

 MustRunAs

- from 1 to 65535

 false

configMap

emptyDir

projected

secret

downwardAPI

persistentVolumeClaim
infra-elasticSearch

NB: ServiceAccount & RBAC required

 true IPC_LOCK

SYS_RESOURCE

 RunAsAny RunAsAny MustRunAs

- from 1 to 65535

 MustRunAs

- from 1 to 65535

 false configMap

emptyDir

projected

secret

downwardAPI

persistentVolumeClaim
cnx-ingress false NET_BIND_SERVICE RunAsAny MustRunAsNonRoot MustRunAs: from 1 to 65535 MustRunAs: from 1 to 65535 false configMap,secret

Notes

  • Most pods will fall under the "restricted" PSP.
  • The "privileged" policy applies to pods in the kube-system namespace only.

Common items that apply to all the above polices (except for privileged and cnx-ingress)

  • Seccomp: docker/default - The Docker default seccomp profile is used.
  • HostNetwork: Controls whether the pod may use the node network namespace. Doing so gives the pod access to the loopback device, services listening on localhost, and could be used to snoop on network activity of other pods on the same node. Is set to false.
  • HostPID: Controls whether the pod containers can share the host process ID namespace. Is set to false.
  • HostIPC: Controls whether the pod containers can share the host IPC namespace. Is set to false.

Glossary

  • PRIV: Privileged - determines if any container in a pod can enable privileged mode.
  • CAPS: Linux capabilities provide a finer grained breakdown of the privileges traditionally associated with the superuser. If blank, no additional capabilities can be added. e.g. IPC_LOCK capability is enabled to allow Elasticsearch to lock the heap in memory so it will not be swapped.
  • SELINUX: RunAsAny - No default provided. Allows any SELinuxOptions to be specified.
  • RUNASUSER: Controls which user ID the containers are run with.
  • FSGROUP: Controls the supplemental group applied to some volumes.
  • SUPGROUP: Controls which group IDs containers add.
  • READONLYROOTFS: Requires that containers must run with a read-only root filesystem (i.e. no writable layer)
  • VOLUMES: Provides a whitelist of allowed volume types.

Why do we need specific Pod Security Polices?

Filebeat
  • Runs as root
  • Requires access to Docker logs on the host. For example, hostPath required
ElasticSearch
  • Runs an init container as a privileged user to set vm.max_map_count=262144 on the host OS
  • Runs as root
  • Adds Linux capabilities:

    - IPC_LOCK capability is enabled to allow Elasticsearch to lock the heap in memoryso it will not be swapped.

    - SYS_RESOURCE capability is set to control and override various resource limits.

Mongo/SOLR/Zookeeper
  • Runs an init container as root to change ownership and permissions of Persistent Storage. Core containers run as non-root.
cnx-ingress
  • Adds Linux capabilities: NET_BIND_SERVICE
  • Allows Privilege Escalation: required to run the default back end service.

Enabling Pod Security Policies

Pod security policy control is implemented as an optional (but recommended) admission controller. PodSecurityPolicies are enforced by enabling the admission controller, but doing so without authorizing any policies will prevent any pods from being created in the cluster.

To enable the Component Pack Pod Security Policies:
  1. Enable the PodSecurityPolicies admission plugin in your cluster and apply the privileged-psp RBAC:
  2. Deploy the k8s-psp Component Pack helm chart

    • Deploy the k8s-psp Component Pack helm chart using the steps in the topic Applying the Pod Security Policies for Component Pack

      Important: The applied Pod Security Policies are only set up to allow Component Pack pods to run. If you wish to deploy other services to the same kubernetes cluster, you may have to create custom policies for those to run. See the official kubernetes documentation for additional information: Pod Security Information

Troubleshooting Pod Security Policies

If a service does not comply with the Pod Security Policies on an enabled server, the pods will not be able to start. In fact, the pods will not even display. Tip: You can use the following command to check if the Pod Security Policy admission controller is blocking pods from starting: kubectl get events -n <namespace>.