Last modified October 6, 2017
Securing your Cluster with RBAC and PSP
Two of the most central mechanisms to secure your cluster in Kubernetes are RBAC (Role Based Access Control) and PSP (Pod Security Policies). Together, these two allow you to create fine-grained roles and policies to manage access control for users and software running on your cluster. Both are enabled by default on Giant Swarm clusters.
Role Based Access Control
The RBAC API defines both roles and bindings on either namespace or cluster level. Like any other Kubernetes API object, these can be defined by writing YAML (or JSON) manifests.
Note that to apply these manifests, you need a user with higher level access than the access you want to set up. When in doubt, use a Cluster Admin account to apply RBAC manifests.
Roles & Cluster Roles
Role is used to grant permissions within a single namespace, a
ClusterRole can grant the same permissions, but can be bound on a cluster-scope.
Note: If you want to grant access to resources that are not namespaced you always need a
Role grants read access to pods in the
kind: Role apiVersion: rbac.authorization.k8s.io/v1beta1 metadata: namespace: default name: pod-reader rules: - apiGroups: [""] # "" indicates the core API group resources: ["pods"] verbs: ["get", "watch", "list"]
ClusterRole like the following you can grant read access to secrets. You can do this in any particular namespace or across all namespaces, depending on how the
ClusterRole is bound.
kind: ClusterRole apiVersion: rbac.authorization.k8s.io/v1beta1 metadata: # "namespace" omitted since ClusterRoles are not namespaced name: secret-reader rules: - apiGroups: [""] resources: ["secrets"] verbs: ["get", "watch", "list"]
Role & Cluster Role Bindings
With bindings you can bind Roles or Cluster Roles to subjects (users, groups, or service accounts).
RoleBinding grants access within a certain namespace, while a
ClusterRoleBinding grants cluster-wide access.
RoleBinding can either reference a
Role or a
ClusterRole, but will only grant permissions on the namespace it is applied to.
The following RoleBinding grants the “pod-reader” role to the user “jane” within the “default” namespace. This allows “jane” to read pods in the “default” namespace.
kind: RoleBinding apiVersion: rbac.authorization.k8s.io/v1beta1 metadata: name: read-pods namespace: default subjects: - kind: User name: jane apiGroup: rbac.authorization.k8s.io roleRef: kind: Role name: pod-reader apiGroup: rbac.authorization.k8s.io
Binding Cluster Roles with Role Bindings allows cluster adminstrators to define a set of common roles for the entire cluster, then reuse them within multiple namespaces.
For instance, even though the following RoleBinding refers to a ClusterRole, “dave” (the subject) will only be able read secrets in the “development” namespace (the namespace of the RoleBinding).
kind: RoleBinding apiVersion: rbac.authorization.k8s.io/v1beta1 metadata: name: read-secrets namespace: development # This only grants permissions within the "development" namespace. subjects: - kind: User name: dave apiGroup: rbac.authorization.k8s.io roleRef: kind: ClusterRole name: secret-reader apiGroup: rbac.authorization.k8s.io
Finally, a ClusterRoleBinding may be used to grant permission at the cluster level and in all namespaces. The following ClusterRoleBinding allows any user in the group “manager” to read secrets in any namespace.
kind: ClusterRoleBinding apiVersion: rbac.authorization.k8s.io/v1beta1 metadata: name: read-secrets-global subjects: - kind: Group name: manager apiGroup: rbac.authorization.k8s.io roleRef: kind: ClusterRole name: secret-reader apiGroup: rbac.authorization.k8s.io
Referring to Subjects
Bindings can refer to subjects that are either single users, groups, or service accounts. The latter are needed to grant API access (and with PSPs also Pod privileges) to certain Pods, e.g. for monitoring and logging agents.
For a detailed explanation of how to refer to subjects in Bindings we defer to the official RBAC documentation.
Default Roles and Role Bindings
Your Kubernetes cluster comes by default with a set of Roles and Cluster Roles as well as some default Bindings. These are automatically reconcilliated and thus cannot be changed or deleted.
You can use the Roles and Cluster Roles to create bindings for your users. Following example would give all users in the group
mynamespace-admin admin access to the
kind: ClusterRoleBinding apiVersion: rbac.authorization.k8s.io/v1beta1 metadata: name: mynamespace-admin namespace: mynamespace subjects: - kind: Group name: mynamespace-admin apiGroup: rbac.authorization.k8s.io roleRef: kind: ClusterRole name: admin apiGroup: rbac.authorization.k8s.io
One of the most important default Role Bindings is the Cluster Admin role, which depicts a super-user in the cluster. By default it is bound to the
system:masters group. Thus, if you need cluster admin access to your Kubernetes cluster, just use a user (e.g. by creating a key pair with gsctl or happa) that carries that group.
For a complete overview of default roles and bindings we defer to the official RBAC documentation.
Verifiyng if you have Access
If you are not sure if your (or another) user is allowed to do a certain action, you can verify that with the
kubectl auth can-i command.
$ kubectl auth can-i create deployments --namespace production yes
You can also impersonate users to check their access:
$ kubectl auth can-i create deployments --namespace production --as jane no
or impersonate a Service Account to check if it is set up right:
$ kubectl auth can-i use podsecuritypolicies/privileged -n kube-system --as=system:serviceaccount:kube-system:calico-node yes
You can also verfiy access for a whole group:
$ kubectl auth can-i get pods --as-group=system:masters yes
Pod Security Policies
PodSecurityPolicy object defines a set of conditions that a pod must run with in order to be accepted into the system. It governs the ability to make requests on a Pod that affect the
SecurityContext that will be applied to a Pod and container.
By default, Giant Swarm clusters come with two PSPs defined - a
privileged policy that allows almost any Security Context and a
restricted policy that mainly restricts users from running privileged containers and mount host paths as volumes.
Additionally, there are two respective cluster roles defined -
By default all authenticated users are assigned the
If you need to run privileged containers, you need to either use a cluster admin user or bind the
privileged-psp-user role to your desired user or group.
Keep in mind that Pods that need extensive Security Contexts need to run with a Service Account that has a similarly extensive PSP bound to it. You can either bind the already existing
privileged-psp-user role to it, or create a PSP that caters specifically to your desired Security Context.
For details on PSPs we defer to the official PSP documentation.
User Management with Giant Swarm
If you are managing users through the Giant Swarm API or its clients,
gsctl or Happa, you can specify a Common Name Prefix and Organizations when you create key-pairs of kubeconfigs.
Using Common Name as Username in Kubernetes
Setting a Common Name Prefix results in a username like the following:
<cn-prefix> is a username of your choice and
<cluster-domain> is your cluster’s domain, e.g.
When binding roles to a user you need to use the full username mentioned above.
Using Organizations as Groups in Kubernetes
Organizations you set when creating key-pairs or kubeconfigs get mapped to groups inside Kubernetes.
You can then assign roles to a whole group of users.
A user can be part of multiple groups and thus be assigned multiple roles, too.
There is only a single predefined user group inside Kubernetes. Members of the
system:masters group will directly be assigned the default
cluster-admin role inside your cluster, which is allowed to do anything. We recommend only using this kind of user, when bootstrapping security settings for the cluster.