Fundamental Kubernetes Training Labs

(v0.0.7)
last update: 2018/10/01
Sources from and thanks to Michael Hausenblas, Marko Lukša, Bob Killen
Presenter: Arash Kaffamanesh
 About this project

● This k8s trainings slides are created for everyone who’d like to
use it for k8s fundamental trainings
● The video recording of one of our tainings is available here →
● GET STARTED, ingress point :-) is here →
● Additional study guides and training material will be developed
for CKA and CKAD Candidates →
● For more information please join us on slack →
○ If you’d like to become a CKA(D), please ask to join the
CKA(D)s channel, everyone in the group can add you to the
CKA(D)s channel
 How to prepare for online trainings (1)

● Install minikube →
● Install minishift (OKD) →
● Install a k8s-vagrant-multi-node cluster →
● Or Kubeadm DinD Cluster →
● Optional: you need an account on GCP with billing enabled
○ Get started with $300 free credits →
○ Create a project and enable GKE service
○ Install gcloud SDK / CLI: →
Source: https://kubernauts.gitbooks.io/kubernauts-kubernetes-training-courses/content/courses/novice.html
 How to prepare for online trainings (2)

● Optional: create a Kubernetes Cluster on AWS or OpenStack

○ Use Kubernauts’ TK8 installer for AWS, OpenStack and
Bare-Metal →
○ Use Kubernauts’ TK8EKS installer for AWS, OpenStack
and Bare-Metal →
○ Use our Kops Faststart guide for AWS →
○ Use Heptio’s Quickstart for AWS →
○ New: use Dave Karre’s OKD 3.10 implementation →
Source: https://kubernauts.gitbooks.io/kubernauts-kubernetes-training-courses/content/courses/novice.html
 How to prepare for online trainings (3)

● Checkout the code of Kubernetes By Example

$ git clone https://github.com/openshift-evangelists/kbe
$ cd kbe/specs/

● Checkout the code of Kubernetes By Action

$ git clone https://github.com/luksa/kubernetes-in-action.git
$ cd kubernetes-in-action/

Source: https://kubernauts.gitbooks.io/kubernauts-kubernetes-training-courses/content/courses/novice.html
 Kubernetes Learning Resources List

● Everything you need to know about Kubernetes & more:

○ https://goo.gl/Rywkpd
● Recommended Books and references:
 TOPICS (1)

● What is Kubernetes (“k8s” or “kube”)

● Kubernetes Architecture
● Core Concepts of Kubernetes
● Kubernetes resources explained
● Application dependency on Kubernetes primitives
● Kubernetes effect on the software development life cycle
● Local and Distributed Abstractions and Primitives
● Container Design Patterns and best practices
● Deployment and release strategy with Kubernetes
 TOPICS (2)

● Kubernetes v1.8: A Comprehensive Overview →

● Getting started with Kubernetes
○ Get started with k8s w/o installation with Katacoda →
○ Install k8s everywhere
○ Play with Simple Apps on k8s
○ Kubernetes by Example →
○ Deploying and Updating App with Kubernetes
○ Deploy complex apps on k8s
 TOPICS (3)

● Kubectl: manage k8s resources

● Kubectl in action →
● Kubernetes Admin Helper →
● Kubernetes & the Container Ecosystem →
● Kubernetes & Helm, Kedge & Co.
● Extending Kubernetes
● Exercises
● Horizontal Pod Autoscaling (HPA)
● Troubleshooting & Maintenance (wip)
 TOPICS (4)

● Resources for CKA(D) Candidates

○ Curriculum for CNCF Certification Courses →
○ Please refer to the section “For CKA Candidates” →
 What Is Kubernetes?

● Kubernetes is the linux kernel of distributed systems

● Kubernetes is the linux of the cloud!
● Kubernetes is a platform and container orchestration tool for
automating deployment, scaling, and operations of application
containers.
● Kubernetes supports Containerd (docker), Rkt, Cri-o
containers. Kata (formerly clear and hyper) and Virtlet support
are coming soon.
 Kubernetes Architecture (physical layout)

Source: http://k8s.info/cs.html#cs-menu
 Kubernetes Architecture (abstraction overview)

Source: http://k8s.info/cs.html#cs-menu
 Kubernetes Architecture (abstraction details)

Source: http://k8s.info/cs.html#cs-menu
 Kubernetes’ High-Level Architecture Overview

Source: https://www.weave.works/blog/what-does-production-ready-really-mean-for-a-kubernetes-cluster
 Kubernetes Architecture Overview

Source: Kubernetes - a comprehensive Overview

 Kubernetes HA

Don’t miss: https://medium.com/@dominik.tornow/kubernetes-high-availability-d2c9cbbdd864

 Core Concepts of Kubernetes (1)

● Pod →
● Label and selectors →
● Controllers
○ Deployments →
○ ReplicaSet →
○ ReplicationController →
○ DaemonSet →
● Service →
 Core Concepts of Kubernetes (2)

● StatefulSets →
● ConfigMaps →
● Secrets →
● Persistent Volumes (attaching storage to containers) →
● Life Cycle of Applications in Kubernetes →
○ Updating Pods
○ Rolling updates
○ Rollback
 Kubernetes resources explained (1)

Resource (abbr.) [API version] Description

Namespace* (ns) [v1] Enables organizing resources into non-overlapping groups

(for example, per tenant)

Deploying Pod (po) [v1] The basic deployable unit containing one or more processes
Workloads in co-located containers

ReplicaSet Keeps one or more pod replicas running

ReplicationController The older, less-powerful equivalent of a ReplicaSet

Job Runs pods that perform a completable task

CronJob Runs a scheduled job once or periodically

DaemonSet Runs one pod replica per node (on all nodes or only on
those matching a node selector)

StatefulSet Runs stateful pods with a stable identity

Deployment Declarative deployment and updates of pods

 Kubernetes resources explained (2)

Resource (abbr.) [API version] Description

Services Service (svc) [v1] Exposes one or more pods at a single and stable IP
address and port pair
Endpoints (ep) [v1]
Defines which pods (or other servers) are exposed
through a service
Ingress (ing) [extensions/v1beta1] Exposes one or more services to external clients through
a single externally reachable IP address

Config ConfigMap (cm) [v1] A key-value map for storing non-sensitive config options
for apps and exposing it to them

Secret [v1] Like a ConfigMap, but for sensitive data

Storage PersistentVolume* (pv) [v1] Points to persistent storage that can be mounted into a
pod through a PersistentVolumeClaim

PersistentVolumeClaim (pvc) [v1] A request for and claim to a PersistentVolume

StorageClass* (sc) [storage.k8s.io/v1] Defines the type of storage in a PersistentVolumeClaim

 Kubernetes resources explained (4)

Resource (abbr.) [API version] Description

Scaling HorizontalPodAutoscaler (hpa) Automatically scales number of pod replicas based on

[autoscaling/v2beta1**] CPU usage or another metric

PodDisruptionBudget (pdb) Defines the minimum number of pods that must remain
[policy/v1beta1] running when evacuating nodes

Resources LimitRange (limits) [v1] Defines the min, max, default limits, and default requests
for pods in a namespace

ResourceQuota (quota) [v1] Defines the amount of computational resources available

to pods in the namespace

Cluster Node* (no) [v1] Represents a Kubernetes worker node

state
Cluster* [federation/v1beta1] A Kubernetes cluster (used in cluster federation)

ComponentStatus* (cs) [v1] Status of a Control Plane component

Event (ev) [v1] A report of something that occurred in the cluster

 Kubernetes resources explained (4)

Resource (abbr.) [API version] Description

Security ServiceAccount (sa) [v1] An account used by apps running in pods

Role [rbac.authorization.k8s.io/v1] Defines which actions a subject may perform on which

resources (per namespace)

ClusterRole* Like Role, but for cluster-level resources or to grant access

[rbac.authorization.k8s.io/v1] to resources across all namespaces

RoleBinding Defines who can perform the actions defined in a Role or

[rbac.authorization.k8s.io/v1] ClusterRole (within a namespace)

ClusterRoleBinding* Like RoleBinding, but across all namespaces

[rbac.authorization.k8s.io/v1]

PodSecurityPolicy* (psp) A cluster-level resource that defines which security-

[extensions/v1beta1] sensitive features pods can use

NetworkPolicy (netpol) Isolates the network between pods by specifying which

[networking.k8s.io/v1] pods can connect to each other
 Application dependency on Kubernetes primitives

Source: Kubernetes effect by Bilgin Ibryam

 Kubernetes effect on software development life cycle

CN- Container Cloud

Distributed Best
Design Design Native
Primitives Practices
Principles Patterns Benefits

Source: Kubernetes effect by Bilgin Ibryam

 Local and distributed abstractions and primitives

Source: Kubernetes effect by Bilgin Ibryam

 Local and distributed abstractions and primitives

Source: Kubernetes effect by Bilgin Ibryam

 Deployment and Release Strategy with Kubernetes

Source: Kubernetes effect by Bilgin Ibryam

 Getting started with Kubernetes

● Kubernetes.IO documentation → && Kubernetes Bootcamp →

● Install Kubernetes CLI kubectl
● Create a local cluster with Minikube, Tectonic, MiniShift
● Create a Kubernetes cluster on GKE
● Create a Kubernetes cluster on AWS
○ Kubeadm →
○ Kops FastStart →
○ Kubicorn →
○ TK8 →
 Install Kubernetes CLI kubectl

● On macOS: brew install kubectl

● On linux and windows follow the official documentation:
https://kubernetes.io/docs/tasks/tools/install-kubectl/

● “kubectl version” gives the client and server version

● “which kubectl”
● alias k=’kubectl’
● Enable shell autocompletion (e.g. on linux):
○ echo "source <(kubectl completion bash)" >> ~/.bashrc
 kubectx and kubens tools

● Great kubectl helpers by Ahmet Alp Balkan

○ kubectx and kubens →
● Kubernetes prompt for bash and zsh
○ kube-ps1 →
● Kubed-sh (kube-dash) →
 Useful aliases

● alias k="kubectl"
● alias g="gcloud"
● alias kx="kubectx"
● alias kn="kubens"
● alias kon="kubeon"
● alias koff="kubeoff"
● alias kcvm="kubectl config view --minify"
● alias kgn="kubectl get nodes"
● alias kgp="kubectl get pods"
 Local Development Environment using Minikube

● Follow the best Minikube tutorial ever by the awesome

Abhishek Tiwari
○ https://abhishek-tiwari.com/local-development-environment-
for-kubernetes-using-minikube/
 Create a Kubernetes cluster on GKE (1)

● You need an account on GCP with billing enabled

● Create a project and enable GKE service
● Install gcloud SDK / CLI:
○ https://cloud.google.com/sdk/

Source:
 Create a Kubernetes cluster on GKE (2)

● gcloud projects create kubernauts-trainings

● gcloud config set project kubernauts-trainings
● gcloud container clusters create my-training-cluster
--zone=us-central1-a
○ Note: message=The Kubernetes Engine API is not enabled
for project training-220218. Please ensure …
● Kubectl get nodes
 How you’re interacting with your three-node Kubernetes cluster

Source: Kubernetes in Action book by Marko Lukša

 Running the container image in Kubernetes

Source: Kubernetes in Action book by Marko Lukša

 Create a Kubernetes cluster on GKE (3)

● List your clusters

○ gcloud container clusters list
● Set a default Compute Engine zone
○ gcloud config set compute/zone us-central1-a
● Define a standard project with your ProjectID
○ gcloud config set project kubernauts-trainings
● Access the Kubernetes dashboard
○ kubectl proxy →
Source:
 Create a Kubernetes cluster on GKE (4)

● Login to one of the nodes

○ gcloud compute ssh <node-name>

● Get more information about a node

○ kubectl describe node <node name>

● Delete / clean up your training cluster

○ gcloud container clusters delete my-training-cluster --zone=europe-west3-a

Note: deleting a cluster doesn’t delete your storage / disks on GKE, you’ve to delete them manually

Source:
 Create a Kubernetes cluster on AWS / OpenStack

● Create a Kubernetes cluster on AWS

○ Kubdeadm →
○ Kops FastStart →
○ Kubicorn →
○ TK8 → (supports OpenStack as well, Bare-Metal, Azure,
GCP coming soon)
 Create a Kubernetes cluster on ACS

● Create a Kubernetes cluster on ACS

○ ToDo → see Kubernetes CookBook

Source:
 Kubernetes API Groups, OpenAPI and Swagger UI (1)

● Install Swagger UI on Minikube / Minishift / Tectonic

○ k run swagger-ui --image=swaggerapi/swagger-ui:latest
○ On Tectonic →
■ k expose deployment swagger-ui --port=8080
--external-ip=172.17.4.101 --type=NodePort
○ On Minikube →
■ k expose deployment swagger-ui --port=8080
--external-ip=$(minikube ip) --type=NodePort
○ Use swagger.json to explore the API
Kubernetes API Groups, OpenAPI and Swagger UI (2)
 Kubernetes API Groups, OpenAPI and Swagger UI (3)

Enjoy the Kubernetes API deep dive →

 Play with Simple Apps on Kubernetes

● Start the Ghost micro-blogging platform

○ kubectl run ghost --image=ghost:0.9
○ kubectl expose deployments ghost --port=2368
--type=LoadBalancer
○ k expose deployment ghost --port=2368
--external-ip=$(minikube ip) --type=NodePort
○ kubectl get svc
○ kubectl get deploy
○ kubectl edit deploy ghost (change the nr. of replicas to 3)
 Play with Simple Apps on Kubernetes

● Log into the pod

○ kubectl exec -it ghost-xxx bash
● Get the logs from the pod
○ kubectl logs ghost-xxx
● Delete the Ghost micro-bloging platform
○ kubectl delete deploy ghost
● Get the cluster state
○ kubectl cluster-info dump --all-namespaces
--output-directory=$PWD/cluster-state
 Deploying and Updating Apps with Kubernetes

● Please read and understand this great free chapter from

Kubernetes in Action book by Marko Lukša.
 Understanding Kubernetes NodePort vs LoadBalancer vs Ingress

Source: https://medium.com/google-cloud/kubernetes-nodeport-vs-loadbalancer-vs-ingress-when-should-i-use-what-922f010849e0
 Kubernetes ingress with Ambassador

● Ambassador is an open source, Kubernetes-native

microservices API gateway built on the Envoy Proxy.
● Ambassador is awesome and powerful, eliminates the
shortcomings of Kubernetes ingress capabilities
● Ambassador is easily configured via Kubernetes annotations
● Ambassador is in active development by datawire.io
● Needles to say Ambassador is open source →

Source: https://blog.getambassador.io/kubernetes-ingress-nodeport-load-balancers-and-ingress-controllers-6e29f1c44f2d
 Kubernetes by Example

● By the awesome Kubernaut Michael Hausenblas

● Hands-On introduction to Kubernetes →
Note: you can run the examples on minikube,
OpenShift, GKE or any other Kubernetes
Installations.
More Exercises
 Exercise 1: Create a deployment for nginx ...

● Create a deployment running nginx version 1.12.2 that will run

in 2 pods
○ Scale this to 4 pods
○ Scale it back to 2 pods
○ Upgrade the nginx image version to 1.13.8
○ Check the status of the upgrade
○ Check the history
○ Undo the upgrade
○ Delete the deployment

Source:
 Exercise 1: Create a deployment for nginx ...

● Create nginx version 1.12.2 with 2 pods

○ kubectl run nginx --image=nginx:1.12.2 --replicas=2 --record
● Scale to 5 pods
○ kubectl scale --replicas=5 deployment nginx
● Scale back to 2 pods
○ kubectl scale --replicas=2 deployment nginx
● Upgrade the nginx image to 1.13.8 version
○ kubectl set image deployment nginx nginx=nginx:1.13.8

Source:
 Exercise 1: Create a deployment for nginx ...

● Check the status of the upgrade

○ kubectl rollout status deployment nginx
● Get the history of the actions
○ kubectl rollout history deployment nginx
● Undo / rollback the upgrade
○ kubectl rollout undo deployment nginx
● Delete the deployment
○ k delete deploy/nginx
Source:
 Exercise 1: Create a deployment for nginx from a manifest file

$ cat nginx.yaml
● Create the deployment with a manifest: apiVersion: extensions/v1beta1
○ kubectl create -f nginx.yaml kind: Deployment
metadata:
name: nginx
Note: Pods, services, configmaps, secrets in our examples are labels:
app: nginx
all part of the /api/v1 API group, while deployments are part of spec:
the /apis/extensions/v1beta1 API group. replicas: 2
selector:
The group an object is part of is what is referred to as apiVersion matchLabels:
in the object specification, available via the API reference. app: nginx
template:
metadata:
labels:
app: nginx
spec:
containers:
- name: nginx
image: nginx:1.12.2
ports:
- containerPort: 80
 Exercise 1: Create a deployment for nginx ...

● Edit the deployment: change the replicas to 5 and image

version to 1.13.8
○ kubectl edit deployment nginx
● Get some info about the deployment and ReplicaSet
○ kubectl get deploy
○ kubectl get rs
○ k get pods -o wide (set alias k=’kubectl’)
○ K describe pod nginx-xyz
 Exercise 1: Create a service to the pods with expose

● kubectl expose deployments nginx --port=80 --type=LoadBalancer

● k get svc
 Exercise 2: write an ingress rule ...

● Write an ingress rule that redirects calls to /foo to one service

and to /bar to another $ cat ingress.yaml
apiVersion: extensions/v1beta1
○ k create -f ingress.yaml kind: Ingress
metadata:
name: test
annotations:
ingress.kubernetes.io/rewrite-target: /
spec:
rules:
- host: kubernauts.io
http:
paths:
- path: /foo
backend:
serviceName: s1
servicePort: 80
- path: /bar
backend:
serviceName: s2
servicePort: 80
 Exercise 3: deployment, RC & RS

kubectl run kubia --image=luksa/kubia --port=8080 --generator=run/v1

kubectl run kubia --image=luksa/kubia --port=8080
k get svc
k get pods
k get rc
k get rs
kubectl describe rs kubia-57478bf476
k get svc
k expose rc kubia --type=LoadBalancer --name kubia-http
k expose rs kubia --type=LoadBalancer --name kubia-http2
k expose rs kubia-57478bf476 --type=LoadBalancer --name kubia-http2
k get pods
k scale rc kubia --replicas=3
k get pods
k scale rs kubia-57478bf476 --replicas=3 —> can’t work, you should scale the deployment
k scale deployment kubia --replicas=3
K port-forward kubia-xxxxx 8888:8080
http://127.0.0.1:8888/

Note: the kubia image is from the Kubernetes in Action book by Marko Lukša
 Exercise 4: horizontal pod autoscaling (hpa)

● On GKE:

kubectl run ghost --image=ghost:0.9 --requests="cpu=100m"

k expose deployment ghost --port=2368 --type=LoadBalancer
k autoscale deployment ghost --min=1 --max=4 --cpu-percent=10
export loadbalancer_ip=$(k get svc -o wide | grep ghost | awk '{print $4}')
while true; do curl http://$loadbalancer_ip:2368/ ; done
k get hpa -w
k describe hpa

● On Minikube (hpa doesn’t work for now on minikube → bug??)

minikube addons enable heapster

kubectl run ghost --image=ghost:0.9 --requests="cpu=100m"
k expose deployment ghost --port=2368 --type=NodePort --external-ip=$(minikube ip)
k autoscale deployment ghost --min=1 --max=4 --cpu-percent=10
while true; do curl http://$(minikube ip):2368/ ; done
k get hpa -w
k describe hpa
→ unable to get metrics for resource cpu
 Exercise 5: deploying replicated stateful applications

gcloud compute disks create --size=1GiB --zone=us-central1-a pv-a

gcloud compute disks create --size=1GiB --zone=us-central1-a pv-b
gcloud compute disks create --size=1GiB --zone=us-central1-a pv-c
k create -f persistent-volumes-gcepd.yaml
k create -f kubia-service-headless.yaml
k create -f kubia-statefulset.yaml
k get po
k get po kubia-0 -o yaml
k get pvc
k proxy
k create -f kubia-service-public.yaml
k proxy

Note: This example is from the Chapter 10 of the Kubernetes in Action book by Marko Lukša
 Exercise 6: Play with RBAC

minikube stop
minikube start --extra-config=apiserver.Authorization.Mode=RBAC
k create ns foo
k create ns bar
k run test --image=luksa/kubectl-proxy -n foo
k run test --image=luksa/kubectl-proxy -n bar
k get po -n foo
k get po -n bar
k exec -it test-xxxxxxxxx-yyyyy -n foo sh
k exec -it test-yyyyyyyyy-xxxxx -n bar sh
curl localhost:8001/api/v1/namespaces/foo/services
curl localhost:8001/api/v1/namespaces/bar/services
cd Chapter12/
cat service-reader.yaml
k create -f service-reader.yaml -n foo
k create role service-reader --verb=get --verb=list --resource=services -n bar
k create rolebinding test --role=service-reader --serviceaccount=foo:default -n foo
k create rolebinding test --role=service-reader --serviceaccount=bar:default -n bar
k edit rolebinding test -n foo
k edit rolebinding test -n bar
Note: This example is from the Chapter 12 of the Kubernetes in Action book by Marko Lukša
 Exercise 7: Load Testing with Apache Jmeter on Kubernetes and OpenShift

● A more complete example: https://goo.gl/k5rFpb

 Exercise 8: Running Rancher on Kubernetes

● TK8 on Github:
https://github.com/kubernauts/tk8
 Exercise 9: Kafka Confluent on Kubernetes or OpenShift

● Github link:
○ https://github.com/kubernauts/kafka-confluent-platform
 Exercise 10: Cassandra on Kubernetes

● Github link: coming soon

 Appendix (1)

● Run Minikube with RKT or CRI-O

○ minikube start --container-runtime=rkt --network-plugin=cni

○ minikube start --container-runtime=crio --network-plugin=cni

 Get in Touch

1. Slack - https://kubernauts-slack-join.herokuapp.com/
2. #kubernetes-teachers on https://kubernetes.slack.com
3. GitHub - https://github.com/kubernauts
4. Twitter - @kubernauts
5. Meetup group - https://www.meetup.com/kubernauts/
6. And finally, kubernauts.io