02 Rostami
02 Rostami
Authorization in Kubernetes
Garsha Rostami
Abstract
In computer systems security, role-based access control (RBAC) or role-
based security is an approach to restricting system access to authorized
users [1]. This paper will describe how the Kubernetes RBAC authorization
sub-system works, how to leverage it to secure access to resources in the
cluster, and how to validate the set policies through impersonation to ensure
users and service accounts are granted the intended rights.
1 Introduction
Kubernetes is a very complex system. Managing and securing it requires a
lot of planning and expertise. Role-based Access Control (RBAC) is just one
aspect of overall Kubernetes security system and as such a brief discussion
of overall Kubernetes security is provided in this section.
Kubernetes has a layered architecture. Each layer has its own security
requirements and settings. This layered architecture provides defence in depth
which is highly effective and desirable, provided that these layers are properly
protected. [Figure 1] shows an overview of this architecture. Below provides
a brief discussion of these layers from a security standpoint:
In the above picture, when our hypothetical user “Alice” issues “kubectl
get pods” command, the kubectl utility sends Alice’s credential along with
the command to the API server. Which credential is sent, depends which
authentication module the administrator has configured the API server to
use. For instance, if “X509 Client Certificate” authentication is enabled then
Alice’s client cert is sent. If “OpenID connect” is enabled then kubectl
sends her “id token” (which she got when she was authenticated by an
OpenID Connect provider), in a header called “Authorization” to the API
server [3].
Likewise, depending on which authorization module a Kubernetes admin-
istrator has enabled (Role-Based Access Control (RBAC) or Attribute-Based
Access Control (ABAC)), that particular module will decide if Alice is
authorized to list pods. Since this is a read-only request, the request will not
be evaluated by the Admission Controllers.
Notes:
1. “Node authorization” authorizes Kubelets API requests. It is not used
for authorizing people or service accounts.
2. As alluded earlier, ABAC is another and older method of Kubernetes
authorization. ABAC provides finer grain authorization than RBAC but
it is more complex and generally it is recommended to use RBAC if
possible.
3. Webhook [4] is a custom authorization enabler where authorization
requests are intercepted and posted to a web service that performs
the actual authorization and notifies Kubernetes to allow/disallow the
request.
--client-key=john.doe.key \
--client-certificate=john.doe.crt \
--embed-certs=true \
--kubeconfig=john.doe.conf
#Create the context section
kubectl config set-context john.doe@cluster1 \
--cluster=cluster1\
--user=john.doe \
--kubeconfig=john.doe.conf
#View the completed config for our user "john.do"
kubectl config view --kubeconfig=john.doe.conf
Finally, the admin will send the generated config file which has the client
certificated embedded in it to user “John Doe”. The user will copy the file to
his home directory under the “./kube” folder. The embedded client certificate
serves as the user’s credentials and his user id and groups he belongs to will
be available for authorization purposes.
4 RBAC Overview
RBAC is an essential part of Kubernetes security to safeguard its resources.
As the [Figure 3] illustrates, it allows Kubernetes administrators to grant
access to Kubernetes resource(s) to subjects(users), to perform select
opeartions on those resource(s):
• Subjects: These include people, service accounts and other Kuberenetes
objects such as kubelets idendified by their X509 client certificates.
• Kubernetes resources: Some resources such as “Persistentvolumes,
Nodes, Storageclasses, Certificatesigningrequests” are cluster scoped,
where as other resources such as “Pods, Services, Deployments, Dae-
monsets” are name space scoped.
• Opeartions/Verbs: These represent permisions such as “get, list, watch,
create, patch, update, delete, deletecollection”.
In order to create RBAC rules, we need to:
1. Determine where the resource is located within Kubernetes “API
Groups” (we will cover API groups this in the next section).
2. Decide the scope of the rule (namespace/cluster). Kubernetes defines
a set of objects to facilitate that, namely “Role” and “RoleBinding”
244 G. Rostami
Figure 3 Overview of elements involved in RBAC decision making: Users, Resources, and
Permissions.
"/apis/apps",
"/apis/apps/v1",
"/apis/storage.k8s.io",
"/apis/storage.k8s.io/v1",
"/apis/storage.k8s.io/v1beta1",
------------------
]
}
(Listing 5.1.1)
As you can see from the above result (Listing 5.1.1), the API service is
organized into different groups based on functionality and version. This is
just a sample to illustrate the point, there are many other API groups that are
not shown here due to space constraint.
The “core” API group “/api/v1” is the oldest group and contains many
resources such as pods, namespaces, services, etc. If we execute the command
below, we’ll see what resources are available under the core API group and
for each resource what verbs and other attributes are supported. For brevity,
only the Pod resource listed below (Listing 5.1.2) but there are many other
resources available in the core API group:
$ curl http://localhost:8001/api/v1
----
{
"name": "pods",
"singularName": "",
"namespaced": true,
"kind": "Pod",
"verbs": [
"create",
"delete",
"deletecollection",
"get",
"list",
"patch",
"update",
"watch"
],
"shortNames": [
"po"
246 G. Rostami
],
"categories": [
"all"
],
"storageVersionHash": "xPOwRZ+Yhw8="
}
(Listing 5.1.2)
Similarly, you can examine other API groups such as "/apis/apps/v1", etc.
Table 1
Name Short Names API Version Name Spaced Kind Verbs
pods po v1 true Pod [create delete
deletecollection get list
patch update watch]
secrets v1 true Secrets [create delete
deletecollection get list
patch update watch]
serviceaccounts sa v1 true Service [create delete
Account deletecollection get list
patch update watch]
services svc v1 true Service [create delete
deletecollection get list
patch update watch]
As we can see, it provides great info such as under which API group the
resource lives, its scope namespace/cluster, and what verbs it supports.
Running the “kubectl api-resources” returns all resources but if say we are
for looking for a specific resource, say “daemonsets”, we could do something
like:
kubectl api-resources -o wide |grep daemonsets
Role-based Access Control (RBAC) Authorization in Kubernetes 247
Table 2
Name Short Names API Version Name Spaced Kind Verbs
daemonsets ds apps/v1 true DaemonSet [create delete
deletecollection get list
patch update watch]
#Result:
As it can be seen result shown in [Table 2], “daemonsets” resides under
the “apps/v1” API group.
Figure 4 Role.
rules:
- apiGroups: [""] # "" represents the "core" or "v1" API group.
resources: ["pods"]
verbs: ["get", "watch", "list"]
(Listing 6.1)
Notes:
1. This Role (Listing 6.1) is bound to the “Marketing” namespace. In other
words, RBAC enforces this rule only in the “Marketing” namespace.
2. We can specify multiple commas separated resources, for example to
define a role that includes both “pods” and “daemonsets”, we could
express “resources” as [“pods”,” daemonsets”]. Since “daemonsets” are
in the “apps/v1” API group (see Section 5.2), we also need to add ”apps”
to the “apiGroup”:
rules:
- apiGroups: ["","apps"]
resources: ["pods","daemonsets"]
verbs: ["get", "watch", "list"]
3. We can also set RBAC policy to a specific resource, for instance the
“marketing-pod-reader” Role (Listing 6.1) gives read-only access to all
the Pods in the marketing namespace but we could limit the scope to
specific Pod("inventory-pod" in this case):
rules:
- apiGroups: [""]
resources: ["pods"]
resourceNames: ["inventory-pod"]
verbs: ["get", "watch", "list"]
To assign(bind) user “jane.doe” to the newly created “marketing-pod-
reader”, we need to create a “RoleBinding” object:
Here is the yaml representation:
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
name: marketing-pod-reader-binding
namespace: marketing
subjects:
Role-based Access Control (RBAC) Authorization in Kubernetes 249
- kind: User
name: jane.doe
apiGroup: rbac.authorization.k8s.io
roleRef: # Signifies the binding to a Role / ClusterRole
kind: Role # In this case we are binding to a “Role”.
name: marketing-pod-reader #Must match the name of the Role or Cluster-
Role to bind to
apiGroup: rbac.authorization.k8s.io
(Listing 6.2)
Figure 5 RoleBinding.
Notes:
1. “RoleBinding” is namespace scoped. In this example (Listing 6.2), it is
bound to the “marketing” namespace.
2. Under “subjects”, we can either specify specific user(s) by setting “-
kind: User” or group(s) by specifying “-kind: Group”. Groups could be
the group specified in the “Subject” of a client certificate (if client certs
are used) or the groups the user belongs to in LDAP/Open ID Connect
systems.
3. Subjects can also be Kubernetes service accounts/group of service
accounts as well, for example:
name: marketing-svc1
namespace: marketing
apiGroup: rbac.authorization.k8s.io
Figure 6 ClusterRoleBinding.
As you can see, unlike “Role”, a “ClusterRole” does not have a “names-
pace” element.
Here is the yaml representation of our read only POD ClusterRole:
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
Role-based Access Control (RBAC) Authorization in Kubernetes 251
metadata:
name: cluster-pod-reader
rules:
- apiGroups: [""] # "" represents the "core" or "v1" API group.
resources: ["pods"]
verbs: ["get", "watch", "list"]
(Listing 7.1)
Once a ClusterRole has been created, we can bind users/groups to it. Here
is the yaml representation of our ClusterRoleBinding object:
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
name: cluster-pod-reader-binding
subjects:
- kind: User
name: terry.jones
apiGroup: rbac.authorization.k8s.io
roleRef:
kind: ClusterRole
name: cluster-pod-reader #must match the name of the Role or ClusterRole
to bind to
apiGroup: rbac.authorization.k8s.io
(Listing 7.2)
Notes:
1. In this example (Listing 7.2), “Terry.Jones” will have read access to all
Pods regardless of the namespaces they are bound to.
name: terry.jones
apiGroup: rbac.authorization.k8s.io
roleRef:
kind: ClusterRole
name: pod-reader-role
apiGroup: rbac.authorization.k8s.io
(Listing 8.2)
#Marketing RoleBinding
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
name: marketing-pod-reader-binding
namespace: marketing
subjects:
- kind: User
name: john.doe
apiGroup: rbac.authorization.k8s.io
roleRef:
kind: ClusterRole
name: pod-reader-role
apiGroup: rbac.authorization.k8s.io
(Listing 8.3)
9 Aggregating ClusterRoles
Kubernetes RBAC allows aggregating two or more ClusterRoles into a single
ClusterRole. Let’s look at the example below:
The ClusterRole “support” (Listing 9.1) inherits all its rules from “manage-
pods”, manage-endpoints-services, manage-deployments, and “manage-
daemonsets” ClusterRoles. As you can see its “rules:” section is empty.
The “aggregationRule” will cause the control plane to automatically populate
its “rules” with the rules from other ClusterRoles that have a matching label.
As shown below (Listing 9.1), the other ClusterRoles that follow the “sup-
port” ClusterRole all have the matching “‘acme.com/aggregate-to-support:
"true"’ label.
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
254 G. Rostami
metadata:
name: support
aggregationRule:
clusterRoleSelectors:
- matchLabels:
acme.com/aggregate-to-support: "true"
rules: [] # The API Server fills in the rules form other ClusterRoles with
matching label.
(Listing 9.1)
#If we run the following command, after creating the other ClusterRoles
below, we’ll see all the inherited rules (Listing 9.2):
kubectl describe ClusterRole support
PolicyRule:
Resources Non-Resource Resource Verbs
URLs Names
--------- ----------------- -------- -----
deployments.apps [] [] [get list watch create update patch delete collection]
endpoints [] [] [get list watch create update patch delete]
pods/logs [] [] [get list watch create update patch delete]
pods [] [] [get list watch create update patch delete]
services [] [] [get list watch create update patch delete]
daemonsets.apps [] [] [get list watch create update patch delete]
(Listing 9.2)
#Below (Listing 9.3 through 9.6) are the yaml declaration of the ClusterRoles
that the “support” ClusterRole (Listing 8.1) inherited rules from:
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
name: manage-pods
labels:
acme.com/aggregate-to-support: "true"
rules:
- apiGroups: [""]
resources: ["pods","pods/logs"]
verbs: ["get", "list", "watch", "create", "update", "patch", "delete"]
(Listing 9.3)
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
Role-based Access Control (RBAC) Authorization in Kubernetes 255
name: manage-endpoints-services
labels:
acme.com/aggregate-to-support: "true"
rules:
- apiGroups: [""]
resources: ["endpoints","services"]
verbs: ["get", "list", "watch", "create", "update", "patch", "delete"]
(Listing 9.4)
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
name: manage-deployments
labels:
acme.com/aggregate-to-support: "true"
rules:
- apiGroups: ["apps"]
resources: ["deployments"]
verbs: ["get", "list", "watch", "create", "update", "patch", "delete"]
(Listing 9.5)
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
name: manage-daemonsets
labels:
acme.com/aggregate-to-support: "true"
rules:
- apiGroups: ["apps"]
resources: ["daemonsets"]
verbs: ["get", "list", "watch", "create", "update", "patch", "delete"]
(Listing 9.6)
12 Conclusion
Kubernetes has become the facto standard for hosting modern cloud-native
applications. It provides many benefits such as application load balancing,
high availability, ease of deployment, ease of scale up/down based on busi-
ness demands, etc. It has a layered architecture. Each layer has its own
security requirements and settings. This layered architecture provides defence
in depth which is highly effective and desirable, provided that these layers
are properly protected. The focus of this paper of was protecting Kubernetes
258 G. Rostami
the control plane layer (more specifically the objects in the etcd database)
through Role Based Access Control (RBAC). Through RBAC we can exer-
cise fine grain access control on all Kubernetes objects such as PODs, name
spaces, secrets, deployments, end points, etc. RBAC enables us to create roles
(permission sets on Kubernetes objects) and bind them to Kubernetes users,
allowing users to perform specific operations such as list, create, delete, etc.
Roles can be defined at the name space (Role) or cluster level (ClusterRole).
Users can be bound to Roles at the name space (RobeBinding) level, or to
ClusterRoles at the cluster level (ClusterRoleBinding). ClusterRoles can also
be bound to users at the name space level through RoleBinding. Using this
technique, Kubernetes admins can create generic roles such as “Developers,
Team Leads”, “Deployment”, “Readers”, etc. scoped at the cluster and bind
them to users at individual name spaces. This will provide great reusability
and reduces administrative burden of managing Roles at individual name
space. RBAC also provides some built-in ClusterRoles such as “cluster-
admin”, “admin”, “edit”, and “view” that Kubernetes admins can leverage as
needed (see Table 3). Kubernetes RBAC also provides role aggregation fea-
ture where super ClusterRoles can be created to automatically inherit rights
from one or more ClusterRoles. Finally, impersonation allows Kubernetes
admins to impersonate users to ensure they have just the right level of access
to Kubernetes object to do their jobs (no more, no less).
References
[1] Role-based access control (Wiki):
https://en.wikipedia.org/wiki/Role-based_access_control.
[2] Using Admission Controllers (Kubernetes documentation):
https://kubernetes.io/docs/reference/access-authn-authz/admission-con
trollers.
[3] OpenID Connect Tokens (Kubernetes documentation):
https://kubernetes.io/docs/reference/access-authn-authz/authentication/
#openid-connect-tokens.
[4] Webhook Mode (Kubernetes documentation):
https://kubernetes.io/docs/reference/access-authn-authz/webhook/.
[5] Kubernetes Service Accounts
https://kubernetes.io/docs/concepts/security/service-accounts/.
[6] Configure Kubernetes to use OpenID Connect Authentication
https://youtu.be/M9KABid_sCY.
[7] User-facing roles (Kubernetes documentation):
Role-based Access Control (RBAC) Authorization in Kubernetes 259
https://kubernetes.io/docs/reference/access-authn-authz/rbac/#user-fac
ing-roles.
[8] Best Practice Guide to Implementing the Least privilege (Netwrix):
https://www.netwrix.com/guide_to_implementing_the_least_privilege_
principle.
[9] User impersonation (Kubernetes documentation):
https://kubernetes.io/docs/reference/access-authn-authz/authentication/
#user-impersonation.
Biography