Online Talk: RBAC in Kubernetes

https://github.com/javsalgar/rbac-online-talk
I - Creating users
Question
● When starting with K8s, we tend to use full administrator
credentials. Examples: minikube, k8s sandbox...

● In a real cluster we may want to have different users, groups

and privileges

user: jsalmeron user: jjo

group: dev, tech-lead group: sre, tech-lead
Developer Administrator

user: juan user: dbarranco

group: dev group: sre, devops
Developer Administrator

● If in Kubernetes everything is modelled as an API Object,

maybe there’s something like

kubectl create user ...

3
User management in Kubernetes

● Kubernetes provides no API objects for users*

● User management must be configured by the cluster administrator. Examples:

● Certificate-based authentication

● Token-based authentication

● Basic authentication

● OAuth2

*At least something like we have for Deployments, Pods… etc.

4
Certificate-based authentication
● Kubernetes is configured with a Certificate Authority (CA)

/etc/kubernetes/pki/ca.crt /etc/kubernetes/pki/ca.key
Public certificate Private key

● Every SSL certificate signed with this CA will be accepted by the Kubernetes API

● Possible options for creating certificates: OpenSSL or CloudFlare's PKI toolkit

● Two important fields in the SSL certificate:

● Common Name (CN): Kubernetes will interpret this value as the user

● Organization (O): Kubernetes will interpret this value as the group

5
 Creating user certificate: steps
● Create private key (if it does not exist)

openssl genrsa -out juan.key 2048

● Create certificate signing request (CSR)

Developer openssl req -new -key juan.key -out juan.csr -subj "/CN=juan/O=devs"
user group
● Send the CSR to the administrator

● Create certificate from CSR using the cluster authority

Administrator
openssl x509 -req -in juan.csr -CA CA_LOCATION/ca.crt -CAkey
CA_LOCATION/ca.key -CAcreateserial -out juan.crt -days 500

6
Next step: Create kubectl configuration
● To add in your local machine the new configuration:

● Download the cluster authority and generated certificate

● Add the new cluster to kubectl

kubectl config set-cluster sandbox --certificate-authority=ca.pem
--embed-certs=true --server=https://<PUBLIC_ADDRESS_OF_YOUR_CLUSTER>:6443

● Add the new credentials to kubectl

kubectl config set-credentials juan --client-certificate=juan.crt
--client-key=juan.key --embed-certs=true

● Add the new context to kubectl

kubectl config set-context sandbox-juan --cluster=sandbox --user=juan

7
Finally: Test your new configuration
● Change to the newly created context

kubectl config use-context sandbox-juan

You can have multiple clusters and configurations

● Let’s execute a basic command

kubectl get pods

● What happened?

Error from server (Forbidden): pods is forbidden: User "juan" cannot list
pods in the namespace "default"

8
II - Role Based Access Control (RBAC)
RBAC in Kubernetes
● Three important groups
ConfigMaps
Pod
e AutoS
Servic caler
list get
Developer Developer Deploym Secrets
ent
create watch
PV
ReplicaSe
ts Ingress
Administrator Administrator Namespace
delete patch
DaemonSet
Job
Nodes
OS Process Process in CronJob
Pod PVC

Subjects API Resources Operations

(Verbs)

● RBAC connects the three of them

10
RBAC in Kubernetes: Roles
● Establish a set of allowed operations (rules) over a set of resources in a namespace

role pod-access
ConfigMaps
Pod list get
AutoS
caler
e
Servic
Secrets create delete
Deploym
ent
watch
ts Ingress role ns-admin patch
ReplicaSe

Job
DaemonSet
CronJob

Namespaced API Resources Operations

namespace “test”

11
RBAC in Kubernetes: Roles
● Need to specify:
kind: Role ○ Api group
○ Name
apiVersion: rbac.authorization.k8s.io/v1beta1
metadata: Find it in the API reference, examples
namespace: test
name: pod-access
rules:
- apiGroups: [""] WHICH RESOURCES
resources: ["pods"]
verbs: ["get", "list"] WHICH OPERATIONS

When it is core, we use an empty string

12
RBAC in Kubernetes: Roles

kind: Role
apiVersion: rbac.authorization.k8s.io/v1beta1
metadata:
namespace: test
name: ns-admin
rules:
- apiGroups: ["*"] ● Wildcards are allowed
resources: ["*"]
verbs: ["*"]

13
RBAC in Kubernetes: RoleBindings
● Connects a role to a subject or set of subjects

user: jsalmeron
group: tech-lead, dev
Developer
role pod-access

user: dgalvez
group: dev
Developer

role ns-access
user: juan
group: dev
Developer

namespace “test”

14
RBAC in Kubernetes: RoleBinding
● Examples
kind: RoleBinding
○ User
apiVersion: rbac.authorization.k8s.io/v1 ○ Group
metadata: ○ ...
name: devs-read-pods
namespace: test Later we will see another one
subjects:
- kind: Group
name: devs WHICH SUBJECTS
apiGroup: rbac.authorization.k8s.io
roleRef: Used to specify which api group the kind
belongs to
kind: Role
name: pod-access WHICH ROLE (ONLY ONE PER BINDING)

apiGroup: rbac.authorization.k8s.io

15
RBAC in Kubernetes: RoleBinding
kind: RoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
name: salme-ns-admin
namespace: test
subjects:
- kind: User Mini-exercise: Another way of doing this?
name: jsalmeron # Name is case sensitive
apiGroup: rbac.authorization.k8s.io
roleRef:
kind: Role
name: ns-admin
apiGroup: rbac.authorization.k8s.io

16
RBAC in Kubernetes: ClusterRoles
● Establish a set of allowed operations over a set of resources in the whole cluster

role all-pods-access
ConfigMaps
Pod list get
AutoS
caler
e
Servic
create
PV delete
Deploym Secrets
ent role pv-admin watch
ts Ingress patch
ReplicaSe
Nodes
Job
DaemonSet
CronJob PVC

All API Resources Operations

17
 RBAC in Kubernetes: ClusterRoles
● Roles and ClusterRoles have very similar yaml

kind: Role kind: ClusterRole

apiVersion: rbac.authorization.k8s.io/v1beta1 apiVersion: rbac.authorization.k8s.io/v1beta1
metadata: metadata:
name: pod-access name: all-pod-access
namespace: test
Only difference
rules: rules:
- apiGroups: [""] - apiGroups: [""]
resources: ["pods"] resources: ["pods"]
verbs: ["get", "list"] verbs: ["get", "list"]

18
RBAC in Kubernetes: ClusterRoleBinding
● Connects a role to a subject or set of subjects

user: jsalmeron
group: tech-lead, dev
Developer
role all-pod-access

user: dbarranco
group: sre, dev, devops
Administrator

role pv-admin
user: jbianquetti
group: sre, dev
Administrator

Whole cluster

19
 RBAC in Kubernetes: ClusterRoleBinding
● Just like the previous case, very similar YAML

kind: RoleBinding kind: ClusterRoleBinding

apiVersion: rbac.authorization.k8s.io/v1 apiVersion: rbac.authorization.k8s.io/v1
metadata: metadata:
name: devs-read-pods name: salme-reads-all-pods
namespace: test
Only differences
subjects: subjects:
- kind: User - kind: User
name: jsalmeron # Name is case sensitive name: jsalmeron # Name is case sensitive
apiGroup: rbac.authorization.k8s.io apiGroup: rbac.authorization.k8s.io
roleRef: roleRef:
kind: Role kind: ClusterRole
Only differences
name: ns-admin name: all-pod-access
apiGroup: rbac.authorization.k8s.io apiGroup: rbac.authorization.k8s.io

20
Default ClusterRoleBindings
● Kubernetes includes some ClusterRoleBindings. For example:

● system:basic-user: For unauthenticated users (group system:unauthenticated). No

operations are allowed.

● cluster-admin: For members of the system:masters group. Can do any operation on

the cluster (using cluster-admin ClusterRole).

Admin accounts can be created

Administrator belonging to this group

openssl req ... -subj "/CN=dbarranco/O=system:masters"

● ClusterRoleBindings for the different components of the cluster (kube-controller-manager,

kube-scheduler, kube-proxy …)

21
 list get
More about the possible actions (verbs) create watch
● TRIVIA: Example operations and their requirements delete
patch
kubectl run --image=bitnami/mongodb my-mongodb deployments: create

kubectl get deployments -w deployments: get, list, watch

kubectl delete deployment my-mongodb deployments: get, delete

kubectl edit deployment my-mongodb mypod deployments: get, patch

kubectl expose deployment my-mongodb --port=27017 deployments: get

services: create
--type=NodePort

pods: get
kubectl exec -ti mypod bash pods/exec: create

22
Questions
● Find the necessary RBAC rules so the user can contact Helm’s Tiller pod

● We know that this command should work with the previously created RBAC rules (salme-ns-admin)

helm install stable/wordpress --namespace test

● And what about this command?

helm install stable/wordpress --namespace default

● Regenerate the Tiller pod and try the command again

helm reset --force && helm init

Error: rpc error: code = Unknown desc = configmaps is forbidden: User

"system:serviceaccount:kube-system:default" cannot list configmaps in the namespace
"kube-system"

23
 Helm under the hood
A server called tiller is in charge of rendering and deploying charts
helm install my-wordpress/ Your cluster
kind: Deployment kind: Deployment
metadata: metadata:
Process in Pod
name: {{ template "fullname" . }} name: pilfering-anaconda
spec: spec:
replicas: {{ .Values.replicaCount }} Tiller replicas: 1
template: template:
spec: spec:
containers: containers:
- name: wp - name: wp
image: {{ .Values.image }} image: bitnami/wordpress:4.8.3
Kubernetes API
... ...

How do we configure this? Do we need to provide a certificate to the pod?

2018 Bitnami. Proprietary and confidential. 24
RBAC in Kubernetes (again): ServiceAccount Tiller

● While regular users are not handled by Kubernetes, processes inside pods do have an API object

ServiceAccount
Developer Administrator OS Process Process in
Pod

● Necessary for pods that need to contact Kubernetes API

● Also used for other operations like storing image pull secrets

25
RBAC in Kubernetes (again): ServiceAccount
apiVersion: v1
kind: ServiceAccount ● An API token will be automatically
created and stored in the cluster
metadata:
name: my-service-account

● Can be used in RoleBinding and ClusterRoleBinding as subjects ● Examples

○ User
○ Group
● ServiceAccounts are used in Pod/RS/Deployment declarations
○ ...
apiVersion: v1
Later we will see another one
kind: Pod
metadata:
name: my-pod
● The API token will be mounted
spec: inside the containers
serviceAccountName: my-service-account
If not specified it will use the “default” ServiceAccount

26
Deploying Tiller
● Create a Tiller ServiceAccount kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1
apiVersion: v1 metadata:
kind: ServiceAccount name: tiller-rolebinding
metadata:
name: tiller-sa subjects:
namespace: kube-system - kind: ServiceAccount
name: tiller-sa # Name is case sensitive
● Set up RBAC for Tiller
apiGroup: rbac.authorization.k8s.io
namespace: kube-system
● Which operations requires Tiller?
roleRef:
● In principle, it can deploy kind: ClusterRole
ANYTHING in ANY NAMESPACE name: cluster-admin
apiGroup: rbac.authorization.k8s.io

27
Deploying Tiller

● Update the tiller pod

helm init --service-account tiller-sa --upgrade

● Let’s check if Tiller works now

helm ls

28
Next steps in Kubernetes Cluster Administration

● Different type of authentications like OAuth

● Limits and Quotas: ResourceQuota and LimitRanges

● NetworkPolicies

● PodSecurityPolicies

Check Bitnami Documentation for several Kubernetes How-To’s:

https://docs.bitnami.com/kubernetes/how-to/

29
Thank
You
For more
information,
visit bitnami.com