Kubernetes Training

Introduction

Software deployment is all of the activities that make a software system available for the end
user, those activities are, preparing an infrastructure that your software will run on top of, install
all the dependencies needed to run the software on the infrastructure, move your application to
the newly created environment and finally configure it to be accessible by the end user, this
process takes a long time even if everything goes well (it’s never the case), and if we have a lot of
software to deploy this process becomes a nightmare, preparing an infrastructure like VMs or
bare metal servers for each software and monitoring them all is nearly impossible.
This used to be the only way to deploy software, but with the advancement of today’s time in
computer sciences, it is not the case anymore, let us breakdown the changes step by step
starting with the infrastructure.
Infrastructure.
infrastructure is the physical and or virtual resources that the software runs on top of.
Preparing a physical server and spinning Virtual Machines (VM) on top if it seems the best
approach, but creating VMs is time consuming and it is not a very efficient way to you use the
whole resource capability of your physical server. Containers have come to solve this issue.
A container is an isolated environment packages up code and all its dependencies so the
application runs quickly and reliably from one computing environment to another, it is similar to
a VM but is way lighter and faster, Docker is an open source project that creates images for
containers and run them on a Linux based or Windows OS, the developer writes a descriptive
document written in YAML called a Dockerfile, in it, the developer describe what his software
needs to be able to function (base OS, dependencies, configuration etc...), create a docker image
from that File, and finally, spin up a container from the newly created image.
The container only takes a few seconds to run, and from one single image we can launch
multiple containers, and that is one of the many reasons for how a container is superior to a VM.
Now that we are running a lot of containers, we need a way to manage and orchestrate them.
Docker Swarm can be a good example of container management system, it can link multiple
servers into one cluster, and run the containers across those servers from one machine, but
there is an even superior solution over Docker Swarm, and that is Kubernetes.
Kubernetes is an open-source container orchestration system that was developed by google and
now maintained by the Cloud Native Computing Foundation, it does all the things Docker Swarm
does and more, and it is going rapidly and being adapted as the de-facto container orchestration
solution by most of the tech companies.
The Diagram below shows the transition from the traditional deployment to containerized
deployment in the point of view of the infrastructure layer.

Working with this approach creates a new form of problem and that is managing and storing all
the docker images created in each release. Docker has a solution that is Docker Hub, it is a
docker image repository that stores all your images in the cloud, the problem with this solution is
that all your images are accessible by all the docker hub users, and if a company wishes to make
their images private, they need pay for the service, there is an alternative to this solution,
creating your own private docker repository with the use of Nexus Repository.
Nexus Repository is an open-source project that lets you store and manage all your artifacts
(jars, docker images etc...), it can be deployed on the Kubernetes cluster for fast image pull or in
a separate machine to separate your deployment for your docker repositor.
This modern way of deployment needs a modern way of coding, Git is a version control software
that manages code versions and lets multiple developers to work on the same project without
interfering with each other's work. Then merge their work in one release branch.
The figure below explains how it works.

Git also lets you push the code into a cloud-based code repository like GitHub or Gitlab and this
gives the development team from anywhere the opportunity to collaborate and contribute to
any project.
This approach like any other comes with its issues, letting multiple developers work on a single
project, can lead to a badly written code, as each developer has his logic and the level of
expertise may vary, the code may contain some security vulnerability or code imperfections,
This issue is solvable by implementing a code checking system, SonarQube is an open-source
platform developed by SonarSource for continuous inspection of code quality to perform
automatic reviews with static analysis of code to detect bugs, code smells, and security
vulnerabilities on 20+ programming languages. This makes SonarQube the perfect solution that
meets our needs.
Until now, we improved on every aspect of the lifecycle of software development project, but
the proposed system adds a lot of layers of complications compared to the old implementation,
and it still lakes a key feature that is Automation, we need a tool that pulls code from our version
control system Git, check the quality of the code, build a docker image of the project, push the
code to our private docker repository and finally deploy the project to Kubernetes cluster. All the
major Version Control repository like GitHub and Gitlab provide an integrated automation
system like the one we need, GitHub has GitHub actions a cloud-base Continuous Integration
Continuous Delivery (CI/CD), it detects new code commits, automatically build and deploy the
software, we could use these systems to build and deploy to our Kubernetes cluster, but these
systems are made for a seamless integration with clusters provided by cloud providers like
Azure, AWS and GCP, although they can deploy to private Kubernetes clusters and private cloud
solutions, but they need addons and some workaround. So, it is not ideal in our case, luckily
there is some open-source alternative to these CI/CD systems, and the best know example is
Jenkins.
Jenkins is an open-source automation tool written in Java with plugins built for Continuous
Integration purposes. It is used to build and test a software projects continuously making it
easier for developers to integrate changes to the project, it works by deploying the Jenkins
application in a dedicated server or inside the Kubernetes Cluster, configure it to pull code from
your project repository, then follow a list of instruction to build and then deploy the project, it
only needs to be configured one time and then run the job automatically.

Kubernetes
Kubernetes is an open-source container orchestration platform designed to automate the
deployment, scaling, and management of containerized applications. In fact, Kubernetes has
established itself as the DeFacto standard for container orchestration and is the flagship project
of the Cloud Native Computing Foundation (CNCF), backed by key players like Google, AWS,
Microsoft, IBM, Intel, Cisco, and Red Hat.
The name Kubernetes originates from Greek, meaning helmsman or pilot.
The central component of Kubernetes is the cluster. A cluster is made up of many virtual or
physical machines that each serve a specialized function either as a master or as a node. Each
node hosts groups of one or more containers (which contain your applications), and the master
communicates with nodes about when to create or destroy containers. At the same time, it tells
nodes how to re-route traffic based on new container alignments.
The following diagram depicts a general outline of a Kubernetes cluster:
The Kubernetes master
The Kubernetes master is the access point (or the control plane) from which administrators and
other users interact with the cluster to manage the scheduling and deployment of containers. A
cluster will always have at least one master, but may have more depending on the cluster’s
replication pattern.
The master stores the state and configuration data for the entire cluster in etcd, a persistent and
distributed key-value data store. Each node has access to ectd and through it, nodes learn how
to maintain the configurations of the containers they are running.
The master communicates with the rest of the cluster through the kube-apiserver the main
access point to the control plane. For example, the kube-apiserver makes sure that
configurations in etcd match with configurations of containers deployed in the cluster.
The kube-controller-manager handles control loops that manage the state of the cluster via the
Kubernetes API server. Deployments, replicas, and nodes have controls handled by this service.
Node workloads in the cluster are tracked and managed by the kube-scheduler. This service
keeps track of the capacity and resources of nodes and assigns work to nodes based on their
availability.
Nodes
All nodes in a Kubernetes cluster must be configured with a container runtime, which is typically
Docker.
Each Kubernetes node runs an agent process called a kubelet that is responsible for managing
the state of the node: starting, stopping, and maintaining application containers based on
instructions from the control plane. The kubelet collects performance and health information
from the node, pods and containers it runs and shares that information with the control plane to
help it make scheduling decisions.
The kube-proxy is a network proxy that runs on each node in your cluster, implementing part of
the Kubernetes Service concept. Kube-proxy maintains network rules on nodes. These network
rules allow network communication to your Pods from network sessions inside or outside of your
cluster.
The basic scheduling unit is a pod, which consists of one or more containers. Each pod is
assigned a unique IP address within the cluster, allowing the application to use ports without
conflict.

You describe the desired state of the containers in a pod through a YAML object called a Pod
Spec. These objects are passed to the kubelet through the API server.
A pod can define one or more volumes, such as a local disk or network disk, and expose them to
the containers in the pod, which allows different containers to share storage space.
Pods may be exposed to internal or external clients via services. Services provide network
connectivity to Pods that work uniformly across clusters. Service discovery is the actual process
of figuring out how to connect to a service.
By default, services can be accessed only from within the cluster, but you can enable public
access to them as well if you want them to receive requests from outside the cluster.

Deployments and replicas

A deployment is a YAML object that defines the pods and the number of container instances,
called replicas, for each pod. You define the number of replicas you want to have running in the
cluster via a ReplicaSet, which is part of the deployment object. So, for example, if a node
running a pod dies, the replica set will ensure that another pod is scheduled on another available
node.

Create Kubernetes cluster

Local Kubernetes cluster
In this section we are going to implements a local Kubernetes cluster on Ubuntu using a tool
called Minikube.
Minikube is a tool for local Kubernetes application development and it supports all Kubernetes
features.

Minikube installation
Minikube creates a single-node Kubernetes cluster in a virtual machine on your personal
computer.
The first step is to install the kubectl binary in your Linux machine, The Kubernetes command-
line tool, kubectl, allows us to run commands on Kubernetes clusters from our local machine. We
can use kubectl to deploy applications, inspect and manage cluster resources, and view logs.

• Download the latest release with the command:

o curl -LO https://storage.googleapis.com/kubernetes-release/release/`curl -s
https://storage.googleapis.com/kubernetes-
release/release/stable.txt`/bin/linux/amd64/kubectl

• Make the kubectl binary executable.

o chmod +x ./kubectl

• Move the binary into your PATH.

o sudo mv ./kubectl /usr/local/bin/kubectl

• Test to ensure the version you installed is up-to-date.

o kubectl version –client
For minikube to work, docker is necessary.

installing Docker in Ubuntu is simple and straightforward, in fact it is a one simple command.

• sudo apt install docker.io

We need to start docker and enable it so it is automatically starting when we reboot our system

• Sudo systemctl enable docker

• Sudo systemctl start docker
After installing and enabling docker service, let us check its status

• Sudo systemctl status docker

We should expect status: running.

Next, we need to install a type 2 hypervisor(emulator) for minikube to use

Install a virtual box

• sudo apt install virtualbox

Finally, we install minikube

• Download the binary

o curl -Lo minikube
https://storage.googleapis.com/minikube/releases/latest/minikube-linux-amd64

• Then we make it executable

o chmod +x minikube

• Finally, we install the binary in the bin path

o sudo install minikube /usr/local/bin/

Everything is ready for us now; all we must do is to lunch our Kubernetes cluster in minikube by
typing the following command

• Lunch Minikube
o minikube start --vm-driver=virtualbox

If we want to use another hypervisor, we just change <driver_name> to the name of the
hypervisor you installed
o minikube start --vm-driver=<driver_name>
Now we have a single node Kubernetes cluster running in minikube, and that is what we need as
a testing environment,
To test it run the following command.
o Kubectl get pods --all-namespaces

The result of this command should be a list of Pods, these pods are the necessary components of
k8s.
Kubernetes cluster on azure

To create an AKS cluster, complete the following steps:

❖ On the Azure portal menu or from the Home page, select Create a resource.
❖ Select Containers > Kubernetes Service.
 ❖ On the Basics page, configure the following options:
• Project details: Select an Azure Subscription, then select or create an Azure
Resource group, such as myResourceGroup.
• Cluster details: Enter a Kubernetes cluster name, such as myAKSCluster. Select a
Region and Kubernetes version for the AKS cluster.
• Primary node pool: Select a VM Node size for the AKS nodes. The VM size can't be
changed once an AKS cluster has been deployed. - Select the number of nodes to
deploy into the cluster. For this QuickStart, set Node count to 1. Node count can
be adjusted after the cluster has been deployed.

By default, Basic networking is used, and Azure Monitor for containers is enabled. Click Review +
create and then Create when validation completes.
It takes a few minutes to create the AKS cluster. When your deployment is complete, click Go to
resource, or browse to the AKS cluster resource group, such as myResourceGroup, and select the
AKS resource, such as myAKSCluster. The AKS cluster dashboard is shown, as in this example:

Connect to the cluster

To manage a Kubernetes cluster, you use kubectl, the Kubernetes command-line client. The
kubectl client is pre-installed in the Azure Cloud Shell.

Open Cloud Shell using the >_ button on the top of the Azure portal.
To connect to the cluster from your terminal, first you need to install azure cli and install kubectl
on your machine.
To configure kubectl to connect to your Kubernetes cluster, use the az aks get-credentials
command. This command downloads credentials and configures the Kubernetes CLI to use them.
The following example gets credentials for the cluster name myAKSCluster in the resource group
named myResourceGroup:
 az aks get-credentials --resource-group myResourceGroup --name myAKSCluster

To verify the connection to your cluster, use the kubectl get command to return a list of the
cluster nodes.

kubectl get nodes

Kubernetes cluster on aws

Unlike azure, aws require a VPC before you can create a Kubernetes cluster, so make sure you have one.

To create your cluster with the console

❖ Open the Amazon EKS console at https://console.aws.amazon.com/eks/home#/clusters.
❖ Choose Create cluster.

On the Configure cluster page, fill in the following fields:

• Name – A unique name for your cluster.

• Kubernetes version – The version of Kubernetes to use for your cluster.
• Cluster service role – Choose the Amazon EKS cluster role to allow the Kubernetes
control plane to manage AWS resources on your behalf. For more information, see
Amazon EKS cluster IAM role.
• Secrets encryption – (Optional) Choose to enable envelope encryption of Kubernetes
secrets using the AWS Key Management Service (AWS KMS). If you enable envelope
encryption, the Kubernetes secrets are encrypted using the customer master key (CMK)
that you select. The CMK must be symmetric, created in the same region as the cluster,
and if the CMK was created in a different account, the user must have access to the CMK.
For more information, see Allowing users in other accounts to use a CMK in the AWS Key
Management Service Developer Guide.

• Kubernetes secrets encryption with an AWS KMS CMK requires Kubernetes version 1.13
or later. If no keys are listed, you must create one first. For more information, see
Creating keys.

❖ Select Next.
❖ On the Specify networking page, select values for the following fields:
• VPC – Select an existing VPC to use for your cluster. If none are listed, then you
need to create one first. For more information, see Creating a VPC for your
Amazon EKS cluster.
• Subnets – By default, the available subnets in the VPC specified in the previous
field are preselected. Select any subnet that you don't want to host cluster
resources, such as worker nodes or load balancers. The subnets must meet the
 requirements for an Amazon EKS cluster. For more information, see Cluster VPC
considerations.
• Security groups – The SecurityGroups value from the AWS CloudFormation output
that you generated with Create your Amazon EKS cluster VPC. This security group
has ControlPlaneSecurityGroup in the drop-down name.
• For Cluster endpoint access – Choose one of the following options:
▪ Public – Enables only public access to your cluster's Kubernetes API server
endpoint. Kubernetes API requests that originate from outside of your
cluster's VPC use the public endpoint. By default, access is allowed from
any source IP address. You can optionally restrict access to one or more
CIDR ranges such as 192.168.0.0/16, for example, by selecting Advanced
settings and then selecting Add source.
▪ Private – Enables only private access to your cluster's Kubernetes API
server endpoint. Kubernetes API requests that originate from within your
cluster's VPC use the private VPC endpoint.
❖ Select Next.
❖ On the Configure logging page, you can optionally choose which log types that you want
to enable. By default, each log type is Disabled. For more information, see Amazon EKS
control plane logging.
❖ Select Next.
❖ On the Review and create page, review the information that you entered or selected on
the previous pages. Select Edit if you need to make changes to any of your selections.
Once you're satisfied with your settings, select Create. The Status field shows CREATING
until the cluster provisioning process completes.
❖ Now that you have created your cluster, follow the procedures in Installing aws-iam-
authenticator and Create a kubeconfig for Amazon EKS to enable communication with
your new cluster.

Scenario

In this section we are going to build and deploy a microservice application to our Kubernetes
cluster, this microservice application was developed for training purposes, before starting the
deployment, we are going to explain this application and it’s microservices.
The system that we are deploying is a system for a transport company that tracks the movement
of its tucks when they are making delivery, each of the vehicles is equipped with a GPS that
sends its coordinates to the server, then all the coordinates of all the trucks will be displayed in
map that will be displayed in a web app, now the problem is that we don’t have real vehicles to
track on our disposal, and for that we have a microservice that mimics this scenario,

the position simulator, and as the name suggests, this microservice is going to simulate the
vehicles moving across a map, it is a simple java spring boot application that reads coordinates
from a file in the src folder of the application, it runs on an infit loop so we never run out of
generated positions. Ounce this microservice reads a position of a truck, it sends it to another
microservice, one of the goals of a microservice architecture, is that a single microservice should
only do one thing, in this case our position simulator microservice’s job is to read vehicles
coordinates, and after that it should hand the data to another microservice,
So as our microservice generates simulated positions, it sends those positions to a queue.
A Queue is a very common aspect of a microservice architecture, it allows us to send data across
the system without coupling the microservices together, for this system we are going to work
with ActiveMQ,
The next microservice of the system is one called Position tracker, the job of this microservice is
to read the position from the Queue, and to do various calculations on those positions, for
example, calculating the speed of the vehicle and store the history of where the vehicle has
been.
Those three components represent the entire back end of the microservice application, in real word
microservice application we would have many more than just three components, but what we have here
is enough for a demo.

So of course, we need to have a frond-end application for our system, in our case it is going to be java-
script application, it happens to be implemented using angular.

Our front-end application needs to communicate with our back-end microservices, but to which one? And
should it communicate to all of them?

It is considered an Anti-pattern if we allow our front-end to communicate directly to the microservices,

because a the microservices and constantly changing, and are adding more and more microservices to
our system if we are going to upgrade it, and with each upgrade, we need to go back to our front-end
application and make some changes, over time, this becomes time consuming, and really frustrating.

For that reason, we are going to use an API Gateway.

The idea of an API gateway is very simple, this becomes the single point of entry for the entire
application, from now on the java script code is only going be allowed to reference the API Gateway, and
the API Gateway will transfer the request to the proper microservice,

This is an over simplification of the API Gateway, for further readings, please refer to the this Link or refer
to the last section of this document for more links.

Kubernetes Lab
In this section we are going to deploy our microservice application into our local Kubernetes
cluster.
The first microservice that we are going to deploy, is the web app microservice.

Before we start let’s check if our cluster is running

minikube status

This command should return a running cluster status.

Now that our cluster is ready, we can get to work.

To deploy an application on Kubernetes, we have to write a declarative deployment in form of a YAML
file, YAML — or, “Yet Another Markup Language” — is a text format used to specify data related to
configuration.

Create a file named pod.yaml and use a text editor to edit this file like the following.

apiVersion: v1

kind: Pod

metadata:

name: webapp

spec:

containers:

- name: webapp

image: richardchesterwood/k8s-fleetman-webapp-angular:release0

Save and exit the file, make sure you’re on the same directory where the pod.yaml file is.

Now that we declared in the yaml file how we want to deploy the webapp, it is time to deploy it using
kubectl

Kubectl apply –f pod.yaml

It should take some time because Kubernetes will have to download the image of our web app,

To see all the pods deployed and its status run:

Kubectl get pods

To see more info about the state of our pod use the describe command:

Kubectl describe pod webapp

We should see that Kubernetes is still pulling the image from docker hub.

Even though each pod has its own ip address, the IP isn’t accessible outside of the cluster, it is a private
IP, to be able to access our web app from outside the cluster we need to deploy a service and link this
service to our pod.

The service is a networking layer that load-balances traffic between the pods.
Same as deploying a pod, or any other Kubernetes interaction, we are going to write a YAML file and
describe what this service needs to do.

Create a file named service.yaml and write what's bellow.

apiVersion: v1

kind: Service

metadata:

name: fleetman-webapp

spec:

selector:

app: webapp

release: "0"

ports:

- name: http

port: 80

nodePort: 30080

type: NodePort

Now we need to modify our pod.yaml file and add the labels object so the service can match it.

In the metadata object under name, add the following lines:

labels:

app: webapp

release: "0"
Once our modifications are done, save and exit both files and run:

Kubectl apply –f pod.yaml

Kubectl apply –f service.yaml

Now if we want to access the interface of our deployed application, we need to know the ip of our
cluster, in our case, running the following command will show is the ip:

minikube ip

After that we write the IP of the cluster following the node port that we provided in service.yaml, and the
interface of our application should appear.

Exercise 1:

Your job now is to deploy a pod of the Queue microservice and create a service for it:

The docker image for the Queue: richardchesterwood/k8s-fleetman-queue:release1

The port 8161 is admin console of active MQ, you should expose it to a browser with port 30010.

So far, we learned how to create pods and service on Kubernetes, but in a production environment, we
rarely deploy pods, a pod is the most basic and disposable objects in Kubernetes, if a pod died for some
reason, the only thing we can do is to re-deploy it again, but if our cluster has a lot of pods (which in
production it is the case almost anywhere) it nearly impossible to keep checking every pods’ health and
re-deploy it manually, that why we use replicasets and deployment in Kubernetes to that automatically
for us.

Let's start with the Replicaset, a Replicaset is just an additional piece of configuration for Kubernetes, we
specify how many instances of this pod, do we want Kubernetes to make sure are running.

That means, if for some reason our pod died, then Kubernetes will spring up another one

So how to define a ReplicaSet ??

Same as before, we create a YAML file, we name it replicaset.yaml, and inside it we declare our replicaset
in the object kind, and how many replicas do we, bellow is the YAML file for creating a replicaset for our
webapp microservice

apiVersion: apps/v1

kind: ReplicaSet

metadata:

name: webapp

spec:

selector:

matchLabels:

app: webapp
 replicas: 2

template:

metadata:

labels:

app: webapp

spec:

containers:

- name: webapp

image: richardchesterwood/k8s-fleetman-webapp-angular:release0-5

What we did here is set the replication factor to two, that translates two, at any given time, we expect
two webapp pods on our cluster, the other thing we did is we update the version of our webapp
application to release0-5.

We can check our replicaset by running the following command:

Kubectl get all

We should see our replicaset spawn with our two pods or replicas.

Now we are going to work with “deployment” instead of ReplicaSet.

In Kubernetes, a deployment is basically a more sophisticated form of replicaset, it is a replicaset with one
additional feature.

A Deployment in Kubernetes will create a corresponding replicaset, the replicaset will create the pods,
you can think of it this way. A deployment is an entity that manages replicasets for you.

Let's define a deployment, it is nearly identical to defining a ReplicaSet, we only need to change the two
line in our replicaset.yaml file.

Before doing that, we need to terminate our previous pods and replicasets:

Kubectl delete –f pod.yaml

Kubectl delete –f replicaset.yaml

Now we open the replicaset.yaml file and change the kind object from ReplicaSet to Deployment

And change the image from release0-5 to just release so we go back to the first version of the application.

We save and exit like before, we can also change the name of our file to deployment.yaml instead of
replicaset.yaml, and finally deploy the definition.
 Kubectl apply –f deployment.yaml

We can check our deployment by running the following command:

Kubectl get all

We expect to see a deployment, a replicaset and two pods spawn.

You might be asking yourself, didn’t we delete the replicaset ?, yes, we did, a deployment in Kubernetes
will create a corresponding replicaset, the replicaset will create the pods, you can think of it this way,

A deployment is an entity that manages replicasets for you.

The rolling update functionality in deployment works this way,

When we update our application, and create a new version, all we need to do is to change the image
name in the deployment file and then apply it with the kubectl command, and what Kubernetes does, is it
creates a new replicaset with the new version, and keep the old one but with the replication factor down
to 0 like the image below illustrates.
Now let's test this,

Open the deployment.yaml file, change the version of the image to release0-5, save and exit and run:

Kubectl apply –f deployment.yaml

Now you can keep refreshing the interface of the app in your browser and you’ll see it change without
going down.

And if you run

Kubectl get all

You should see two replicasets like bellow

Imagine that the deployment that we just did is not working properly and should immediately be changed
to the previous version, with deployment we can do rollbacks to previous versions easily

Kubectl rollout history deployment webapp

This command will return the history of our deployment (all the versions that we deployed)

We can rollback to a version using the following command

Kubectl rollout undo deployment webapp –to-revision=3

And choose any revision you like to rollback to.

And just like that we changed the version of our application without changing the deployment.yaml file.

The rollout feature should only be used in emergency rollbacks only, it is considered good practice to
always have your yaml files up to date, and with the rollout, it is not the case, in our case we are in the
release 0 while our yaml file is pointing that we are in the release 0-5.

Exercise 2:

Now that we know how our microservice application work, and we have learned how to deploy a
microservice on Kubernetes, your job now is to update the release of the webapp and gueue to release2
and deploy the rest of the microservice application.

deployments

• Position-tracker
▪ Image: richardchesterwood/k8s-fleetman-position-tracker:release2
• position-simulator
▪ Image: richardchesterwood/k8s-fleetman-position-simulator:release2
• api-gateway
▪ image: richardchesterwood/k8s-fleetman-api-gateway:release2
Services

• fleetman-position-tracker
▪ port: 8080
▪ type: ClusterIP
• fleetman-api-gateway
▪ port: 8080
▪ nodePort: 30020
▪ type: NodePort

Persistence:

Our Applications now should be fully deployed into the Kubernetes cluster, but as we discussed earlier in
this document, pods can be replaced, alongside the data that it contains, and one of our microservices
has a data that we need to persist, that microservice is the position-tracker, as we explained it keeps the
data of the vehicles in directory inside the container, which is not ideal. What we should do is to persist
the data in an in our local Kubernetes’ cluster and in the case of working with cloud provider like Azure or
AWS we can persist our data on AWS’s EBS or Azure Disk, this is still not ideal, if we persist the data
directly from microservice container, it could corrupt our data at some point if we have more than one
instance of our microservice running in the cluster, which is why we need a database, in this training we
are going to use mongo dB to store the position-tracker data.

To deploy mongo DB to the cluster, all we need to do is to create a new YAML file, named it mongo.yaml
and write the following

apiVersion: apps/v1
kind: Deployment

metadata:

name: mongodb

spec:

selector:

matchLabels:

app: mongodb

replicas: 1

template: # template for the pods

metadata:

labels:

app: mongodb

spec:

containers:

- name: mongodb

image: mongo:3.6.5-jessie

To deploy it into our cluster, we just run:

Kubectl apply –f mongo.yaml

Now we have a database deployed in our cluster, but it is not yet linked to our microservice, because we
only deployed a pod, that can have any ip, but the source code of our position-tracker is looking for a dns
of a database not an IP address, so we need a service for our database.

The YAML bellow is the definition of the service that we need.

kind: Service

apiVersion: v1

metadata:

name: fleetman-mongodb

spec:

selector:

app: mongodb

ports:

- name: mongoport

port: 27017

type: ClusterIP

Create a file named mongo-service.yaml containing the yaml above and apply it with kubectl.

The next step is to persist the data of that database on our Cluster storage, so that we don’t lose any
valuable data.

Kubernetes uses the docker volumes to store data that lives inside a container in external or internal
storage, it does that with the help of Persistent Volume (PV), a PV is another resource type within
Kubernetes it can be mapped to EBS, AzureDisk, ascsi etc …, to over simply this, a PV allows us to store
the Data from the container inside the pod, locally or to our cloud storage solution.

A PV can be created manually by the Kubernetes admin, or directly using a provisioner and a Persistent
Volume Claim PVC, a PVC is pointer towards a PV, and if the PV does not exist, it gets created
automatically by the provisioner, this provisioner is referred to by a StorageClass in Kubernetes, we
defined it once, and it takes care of creating PV for us. The image bellow will help you understand this
concept better.
Now let’s translate all these concepts to YAML

First, we need to modify our mongo.yaml file so can Mount our volume to a PVC

apiVersion: apps/v1

kind: Deployment

metadata:

name: mongodb

spec:

selector:
 matchLabels:

app: mongodb

replicas: 1

template: # template for the pods

metadata:

labels:

app: mongodb

spec:

containers:

- name: mongodb

image: mongo:3.6.5-jessie

volumeMounts:

- name: mongo-persistent-storage

mountPath: /data/db

volumes:

- name: mongo-persistent-storage

# pointer to the configuration of HOW we want the mount to be implemented

persistentVolumeClaim:

claimName: mongo-pvc

next step is to create a PVC and a PV (in case we want to migrate to AWS we change the PV definition into
a Storage Class), to do that, create a new file storage.yaml and write the following.

# What do want?
apiVersion: v1

kind: PersistentVolumeClaim

metadata:

name: mongo-pvc

spec:

storageClassName: mylocalstorage

accessModes:

- ReadWriteOnce

resources:

requests:

storage: 20Gi

---

# How do we want it implemented

apiVersion: v1

kind: PersistentVolume

metadata:

name: local-storage

spec:

storageClassName: mylocalstorage

capacity:

storage: 20Gi

accessModes:

- ReadWriteOnce

hostPath:

path: "/mnt/some new/directory/structure/"

type: DirectoryOrCreate
Apply the changes with kubectl, and our data should be persistent Locally now.

Monitoring:

The number of pods and replicasets is continuously growing in our cluster, and monitoring the status of
the pods and replicasets with command line is becoming difficult, luckily, Kubernetes provide a
monitoring dashboard accessible via a browser that show us the status of each pod and show is
information about them,

Kubernetes dashboard is built in Minikube

To access it just run

minikube dashboard

If you are running a cluster on the cloud or locally but without minikube, you can deploy the dashboard
with the following command

kubectl apply -f
https://raw.githubusercontent.com/kubernetes/dashboard/v2.0.0/aio/deploy/recommended.yaml

You can use Dashboard to deploy containerized applications to a Kubernetes cluster, troubleshoot your
containerized application, and manage the cluster resources. You can use Dashboard to get an overview
of applications running on your cluster, as well as for creating or modifying individual Kubernetes
resources (such as Deployments, Jobs, DaemonSets, etc.). For example, you can scale a Deployment,
initiate a rolling update, restart a pod or deploy new applications using a deploy wizard.

Dashboard also provides information on the state of Kubernetes resources in your cluster and on any
errors that may have occurred.
With the help of the dashboard, we can determine that a pod is using a lot of resource, and with that
information, we can scale the pod and terminate it.

There are other monitoring solutions in Kubernetes, the most known one is Prometheus and Grafana, it is
more flexible than Kubernetes dashboard, you create you costume dashboard in Grafana using the
metrics that you like from Prometheus, the downside that it needs a lot of work and some getting used to
the interface, because it is not as straight forward as Kubernetes's dashboard.

The other monitoring solution that is gaining attention these past few months is lens, lens is a desktop
application that connects to your cluster via the kubeconfig file, it deploys automatically, Prometheus to
you cluster to collect information about hardware and pods, then send that info to the application and
shows it through a simple to manage dashboard

CI/CD Component installation.

In this section we are going to install all the necessary tools to implement a Continuous
integration Pipeline, and the continuous deployment into a Kubernetes cluster.

Below is the list of tools that we are going to install.

• Git
• Jenkins
• SonarQube

The first tool we are going to use is “git” to be able to push our code into GitHub or any other
version control,
Git is easy to install, we only need one command to do so.
o Sudo apt update
o Sudo apt install git

The next tool we are going to install is Jenkins.

Jenkins
Jenkins is an open-source Continuous Integration server capable of orchestrating a chain of
actions that help to achieve the Continuous Integration process (and not only) in an automated
fashion.
By using Jenkins, software companies can accelerate their software development process, as
Jenkins can automate build and test at a rapid rate. Jenkins supports the complete development
lifecycle of software from building, testing, documenting the software, deploying and other
stages of a software development lifecycle.
What is Continuous Integration?
In Continuous Integration after a code commit, the software is built and tested immediately. In a
large project with many developers, commits are made many times during a day. With each
commit code is built and tested. If the test is passed, build is tested for deployment. If
deployment is a success, the code is pushed to production. This commit, build, test, and deploy
is a continuous process and hence the name continuous integration/deployment.
A Continuous Integration Pipeline is a powerful instrument that consists of a set of tools
designed to host, monitor, compile and test code, or code changes, like:

• Continuous Integration Server (Jenkins, Bamboo, Cruise Control, TeamCity, and others)
• Source Control Tool (e.g., CVS, SVN, GIT, Mercurial, Perforce, ClearCase and others)
• Build tool (Make, ANT, Maven, Ivy, Gradle, and others)
• Automation testing framework (Selenium, Appium, TestComplete, UFT, and others

Below Is a representative schema of how Jenkins work.

Jenkins installation

First step is to import Jenkins signing key and enable Jenkins repository.

o wget -q -O - https://pkg.jenkins.io/debian/jenkins.io.key | sudo apt-key add -

o sudo sh -c 'echo deb http://pkg.jenkins.io/debian-stable binary/ >
/etc/apt/sources.list.d/jenkins.list'
o sudo apt update

Next step is to install an appropriate Java JDK version.

o sudo apt install openjdk-11-jdk-headless

Now we are ready to install Jenkins. To do so run:

o sudo apt install jenkins

Enable Jenkins to start after reboot and start the service:

 o sudo systemctl start jenkins
o sudo systemctl enable jenkins

Now Jenkins Web UI should be visible on localhost:8080

For security reasons, Jenkins web ui require the administration password generated by default
during the installation, to ensure that only the admin should configure Jenkins for the first time.
As mentioned in the alert shown by Jenkins, the password will be found in
/var/lib/jenkins/secrets/initialAdminPassword

To display the admin password run this command

• cat /var/lib/jenkins/secrets/initialAdminPassword

Now copy the password, and paste it in the Jenkins's web UI as follow and then click continue
Now Jenkins will ask us to install the necessary plugins, here we are going to use the
recommended plugins, and later we are going to add other plugins as we need them.
This step should take a couple of minutes depending on the internet connection, because
Jenkins will download some plugins and install them automatically.
Once the installation is completed, we need to create an admin credentials like this example
Leave the Jenkins URL as the default http://localhost:8080
Now your Jenkins installation is done, and it ready to be used.

SonarQube:

SonarQube is a web-based open-source platform used to measure and analyze the source code
quality. Code quality analysis makes your code more reliable and more readable.
Sonar covers the 7 sections of code quality

• Architecture and Design

• Unit tests
• Duplicated code
• Potential bugs
• Complex code
• Coding standards
• Comments
SonarQube receives files as an input and analyzes them along with barriers. Then calculates a set
of metrics, stores them in a database and shows them on a dashboard. This recursive
implementation helps in analysis of code quality and how code improves over time.
SonarQube installation

We are going to install SonarQube using Docker, it is simpler than a traditional installation.
SonarQube provides an official docker image in the official docker repository.
o docker run -d --name sonarqube -e
SONAR_ES_BOOTSTRAP_CHECKS_DISABLE=true -p 9000:9000 sonarqube:latest

Log in to http://localhost:9000 with System Administrator credentials (login=admin,

password=admin).

SonarQube is now running with its default database, this practice is not recommended in a
production environment, it is better to run your own database and link SonarQube to it.

Automation Deployment

In the last section we learned the basics of Kubernetes and how to deploy applications on it, in this
section we are going to automate that process and link it to the development of the project. Creating an
automated pipeline from development to deployment.

We are going to use git, GitHub and Jenkins to create this pipeline.
Before we start, make sure you have a GitHub account, we also need a repository in GitHub, so make sure
to create a new empty repository there.

Now we need to configure the GitHub and Jenkins integration. Let’s begin with the GitHub side.

Configuring GitHub

go to your GitHub repository (the new one that you just created) and click on ‘Settings’.

Click on Webhooks and then click on ‘Add webhook’.

in the ‘Payload URL’ field, paste your Jenkins environment URL. At the end of this URL add /github-
webhook/. In the ‘Content type’ select ‘application/json’ and leave the ‘Secret’ field empty.
in the ‘Which events would you like to trigger this webhook?’ choose ‘Let me select individual events.’
Then, check ‘Pull Requests’ and ‘Pushes’. At the end of this option, make sure that ‘Active’ is checked and
click on ‘Add webhook’.
We're done with the configuration on GitHub’s side! Now let's move on to Jenkins.

Configuring Jenkins

In Jenkins, click on ‘New Item’ to create a new project.

Give your project a name, them choose ‘Freestyle project’ and finally click on ‘OK’.
Click on the ‘Source Code Management’ tab.

Click on Git and paste your GitHub repository URL in the ‘Repository URL’ field, and the script paht write
Jenkinsfile.

Click on the ‘Build Triggers’ tab and then on the ‘GitHub hook trigger for GITScm polling’. Or, choose the
trigger of your choice.
That's it, our GitHub repository is now integrated with Jenkins.

Now let’s test it.

First, we need to create an empty directory in our machine and then we are going to pull the microservice
code from GitHub, add our changes and then push it into our new GitHub repository.

mkdir project

cd project

git clone https://github.com/DickChesterwood/k8s-fleetman.git

cd k8s-fleetman-webapp-angular

mv src dist

Now copy the deployment.yaml that we worked with in Kubernetes lab alongside service.yaml and past
them in the present working directory.

We have the code we need, the Dockerfile and yaml files for the deployment in one place, all what is left
to do is to create a Jenkinsfile, A Jenkinsfile is a text file that contains the definition of a Jenkins Pipeline
and is checked into source control.

Use any code editor you are comfortable using, I'm using vs code below

code Jenkinsfile

In the jenkinsfile we are going to declare the stage that we would like for Jenkins to do for us
automatically,

In our case, we want for Jenkins to build a docker image from or source code and then deploy this image
into our Kubernetes cluster.

Below is the Jenkinsfile that does that for us.

Save and quit.

Next, commit the changes git, but before that, we need to back off to the previous directory so we push
the whole project to GitHub

cd ..

git add .

git commit –m “added jenkinsfile”

Now that our changes are commited locally all we need to do is to push those changes to GitHub

Add your GitHub repository link to git remote using the following command

Git remote add origin “link of your github repo”

Finally lets push our changes

Git push origin master

If you go check Jenkins interface, we should see that our Pipeline has been activated by this code push,

If we click on our pipline and check the console we should be able to see the stages that we declared
going through.
Now let’s try the same with another microservice, like the position simulator microservice,

As we discussed earlier it is a java program that simulate vehicle positions, so let’s discuss the stages
needed for the project to be deployed.

If we change to k8s-fleetman-position-simulator director and do an ls, we can determine that the code
isn’t compiled, and we need a Dockerfile to containerize our application so we can later deploy it into our
cluster, and finally we need to a Jenkinsfile that declare these stages that we just listed and more.

First, we create our Dockerfile

code Dockerfile

And write the following

FROM openjdk:8u131-jre,

ADD target/positionsimulator-0.0.1-SNAPSHOT.jar webapp.jar

CMD ["java","-jar","webapp.jar"]

This dockerfile will create an image that contains the jar of our project

The next step is to create the Jenkinsfile that describes those stages that we need to automate.

Code Jenkinsfile

And write the following.

All what is left to do is to create a new freestyle project in jenkins, link the project to this Jenkinsfile and
finally commit and push the change, and watch you work get automated.

This work is going to be your next exercise.

The only thing missing in our pipeline is Code Quality test, that is what we are going to do just that now.

And you can guess that it is just a matter of adding a new step to our jenkinsfile.

Let's take a look at what we just added.

This stage will call the SonarQube plugin in Jenkins, and run the command mvn sonar:sonar which will
scan the code of our application and then send the report to our SonarQube container.

But we haven’t yet added SonarQube plugin in Jenkins.

First go to the main page of our Jenkins server click on manage plugins, search for SonarQube Scanner,
click on install.

Next, install sonar scanner on the ec2 instance

wget https://binaries.sonarsource.com/Distribution/sonar-scanner-cli/sonar-scanner-cli-
4.2.0.1873-linux.zip

unzip sonar-scanner-cli-4.2.0.1873-linux.zip

mv sonar-scanner-4.2.0.1873-linux /opt/sonar-scanner

Edit the sonar-scanner.properties file

Edit the sonar-scanner.properties file

Configure the Sonarqube scanner to connect to your Sonarqube server.

sonar.host.url=http://localhost:9000

sonar.sourceEncoding=UTF-8

We need to add the sonar-scanner command to the PATH variable.

Let’s create a file to automate the required environment variables configuration

vi /etc/profile.d/sonar-scanner.sh

Here is the sonar-scanner.sh file content.

#/bin/bash

export PATH="$PATH:/opt/sonar-scanner/bin"

use the source command to add the sonar scanner command to the PATH variable.

source /etc/profile.d/sonar-scanner.sh

Everything is ready now, all we have to do is commit our changes and push the code to our GitHub repo.

Check the SonarQube server interface to see the quality of our code.

Further Reading.
In this section I'm going to share some links for further reading about what we learned in this training
session.

Microservice:

Microservice: https://microservices.io/

Api-gateway: https://microservices.io/patterns/apigateway.html

Docker:

Docker engine: https://docs.docker.com/engine/api/

Docker registry: https://docs.docker.com/registry/spec/api/

Dockerfile: https://docs.docker.com/engine/reference/builder/

Compose file:

Version 1: https://docs.docker.com/compose/compose-file/compose-file-v1/

Version 2: https://docs.docker.com/compose/compose-file/compose-file-v2/

Version 3: https://docs.docker.com/compose/compose-file/compose-file-v3/
Kubernetes:

kube-proxy: https://kubernetes.io/docs/reference/command-line-tools-reference/kube-proxy/

kube-apiserver: https://kubernetes.io/docs/reference/command-line-tools-reference/kube-
apiserver/

Etcd: https://kubernetes.io/docs/tasks/administer-cluster/configure-upgrade-etcd/

kube-controller-manager: https://kubernetes.io/docs/reference/command-line-tools-
reference/kube-controller-manager/

kube-scheduler: https://kubernetes.io/docs/reference/command-line-tools-reference/kube-
scheduler/

Kubelet: https://kubernetes.io/docs/reference/command-line-tools-reference/kubelet/

Pod: https://kubernetes.io/docs/concepts/workloads/pods/

Service: https://kubernetes.io/docs/concepts/services-networking/service/

Deployment: https://kubernetes.io/docs/concepts/workloads/controllers/deployment/

Volumes: https://kubernetes.io/docs/concepts/storage/volumes/

Jenkins:

Jenkins: https://www.jenkins.io/doc/

Syntax-reference: https://www.jenkins.io/doc/book/pipeline/syntax/

SonarQube:

SonarQube: https://docs.sonarqube.org/latest/

Scanners: https://docs.sonarqube.org/latest/analysis/overview/

CI integration: https://docs.sonarqube.org/latest/analysis/branch-pr-analysis-overview/