Kuber Net Es
Kuber Net Es
Kubernetes Architecture
Master controls the cluster, and the nodes in it. It ensures the execution only happens in
nodes and coordinates the act.
Nodes host the containers; in-fact these Containers are grouped logically to form Pods. Each
node can run multiple such Pods, which are a group of containers, that interact with each
other, for a deployment.
Replication Controller is Master’s resource to ensure that the requested no. of pods are
always running on nodes.
- Replica Set: replica sets are created by deployment, these deployments contains declaration
of containers which you want to run in cluster. like image/tag, env variable, data volumes,
Kubernetes has several components in its architecture.
Service is an object on Master that provides load balancing across a replicated group of Pods.
- minion/node/worker node: is the node on which all the services run. You can have many
minions running at one point in time. Each minion will host one or more POD.
- POD: are Mortal & is the smallest unit of deployment in K8S object mode or is like hosting
a service. Each POD then contains the Docker containers(xyz). Each POD can host a
different set of Docker containers. The proxy is then used to control the exposing of these
services to the outside world. You cannot create your own PODs, they are created by
replicaset.
etcd − This component is a highly available key-value store that is used for storing shared
configuration and service discovery. Here the various applications will be able to connect to
the services via the discovery service.
Flannel − This is a back-end network which is required for the containers.
kube-apiserver − This is an API which can be used to orchestrate the Docker containers.
kube-controller-manager − This is used to control the Kubernetes services.
Kubernetes Terminology
1. Nodes:
Hosts that run Kubernetes applications
2. Containers:
Units of packaging
3. Pods:
Units of deployment which is collection of containers
4. Replication Controller:
Ensures availability and scalability
5. Labels:
Key-value pairs for identification
6. Services:
Collection of pods exposed as an endpoint
$ vi hello-pod.yml
apiVersion: v1
kind: Pod
metadata:
name: hello-pod
labels:
zones: prod
version: v1
spec:
containers:
- name: hello-ctr
image: nigelpoulton/pluralsight-docker-ci:latest
ports:
- containerPort: 8080
$ curl http://localhost:30779
http://10.244.0.83:30779/
$ vi soaktestrc.yml
apiVersion: v1
kind: ReplicationController
metadata:
name: soaktestrc
spec:
replicas: 3
selector:
app: soaktestrc
template:
metadata:
name: soaktestrc
labels:
app: soaktestrc
spec:
containers:
- name: soaktestrc
image: nickchase/soaktest
ports:
- containerPort: 80
# kubectl create -f soaktestrc.yml
replicationcontroller "soaktestrc" created
$ curl http://localhost:30779
http://10.244.0.83:30779/
Deployment lifecycle
There are three stages a deployment can be in its lifecycle.
• Progressing
• Complete
• Failed
K8S Deployments:
• The smallest unit of deployment, a Pod, runs containers. Each Pod has its own IP address and
shares a PID namespace, network, and host name.
• You can define a deployment to create a Replica Set or to remove deployments and adopt all
their resources with new deployments.
• When you revise a deployment, a Replica Set is created that describes the state that you want.
During a rollout, the deployment controller changes the actual state to the state that you want
at a controlled rate.
• Each deployment revision can also be rolled back. Deployments can also be scaled.
kubectl create deployment new-nginx --image=nginx:latest
$ curl http://localhost:30779
http://10.244.0.83:30779/
services are REST objects in K8s, service stands in front of Pod so that outside world can
interact to Pods via it. service never change mean its IP, DNS, Ports are reliable, unlike Pods
which are unreliable in nature
now since pods are mortal and they come and go, so Its Endpoint which maintains the list of
available pods dynamically.
Service use Labels to identify the Pods and do the things on them.
Our service definition looks like this:
apiVersion: v1
kind: Service
metadata:
name: myapp-service
spec:
type: NodePort
ports:
- targetPort: 80
port: 80
nodePort: 30050
selector:
app: myapp
type: front-end
##djangoservice.yaml
apiVersion: v1
kind: Service
metadata:
name: djangoapp-server
labels:
app: djangoapp
spec:
clusterIP: None
ports:
- port: 8080
name: server
selector:
app: djangoapp
Name: djangoapp.default.svc.cluster.local
Address: 10.0.0.210
Headless service - you will get the IP of each Pod:
Name: djangoapp.default.svc.cluster.local
Address: 172.17.0.1
Name: djangoapp.default.svc.cluster.local
Address: 172.17.0.2
Name: djangoapp.default.svc.cluster.local
Address: 172.17.0.3
There are some scenarios where this is not feasible as the pods start one by one and only the
latest pod could discover other pods.
Taints and tolerations, pod and node affinities demystified
Enterprises often use multi-tenant and heterogenous clusters to deploy their applications to
Kubernetes. These applications usually have needs which require special scheduling
constraints. Pods may require nodes with special hardware, isolation, or colocation with other
pods running in the system.
Taints and tolerations
• This Kubernetes feature allows users to mark a node (taint the node) so that no pods
can be scheduled to it, unless a pod explicitly tolerates the taint.
• Using this Kubernetes feature we can create nodes that are reserved (dedicated) for
specific pods. E.g. pods which require that most of the resources of the node be
available to them in order to operate flawlessly should be scheduled to nodes that are
reserved for them.
• In practice tainted nodes will be more like pseudo-reserved nodes, since taints and
tolerations won’t exclude undesired pods in certain circumstances:
• system pods are created with toleration settings that tolerate all taints thus can be
scheduled onto any node. This is by design, as system pods are required by the
Kubernetes infrastructure (e.g. kube-proxy) or by the Cloud Provider in case of
managed Kubernetes (e.g. on EKS the aws-node system pod).
• users can’t be stopped from deploying pods that tolerate “wrong” taint thus, beside
system pods, pods other than desired ones may still run on the reserved nodes
• NoSchedule - instructs Kubernetes scheduler not to schedule any new pods to the
node unless the pod tolerates the taint.
• NoExecute - instructs Kubernetes scheduler to evict pods already running on the node
that don’t tolerate the taint.
Taints in Kubernetes
• NoSchedule - this means that no pod will be able to schedule onto node unless it
has a matching toleration.
• PreferNoSchedule - this is a “preference” or “soft” version of NoSchedule – the
system will try to avoid placing a pod that does not tolerate the taint on the node,
but it is not required.
• NoExecute - the pod will be evicted from the node (if it is already running on the
node), and will not be scheduled onto the node (if it is not yet running on the node).
Tolerations
In order to schedule to the “tainted” node pod should have some special tolerations, let’s
take a look on system pods in kubeadm, for example, etcd pod:
Similar to this you can restrict pods to run on some node with a special hardware.
Node affinity
With node affinity we can tell Kubernetes which nodes to schedule to a pod using the labels
on each node.
Since node affinity identifies the nodes on which to place pods via labels, we first need to add
a label to our node.
labels:
...
test-node-affinity: test
...
• Pod affinity and anti-affinity allows placing pods to nodes as a function of the labels
of other pods.
• These Kubernetes features are useful in scenarios like: an application that consists of
multiple services, some of which may require that they be co-located on the same
node for performance reasons; replicas of critical services shouldn’t be placed onto
the same node to avoid loss in the event of node failure.
kubectl create -f nginx-deployment.yaml
vi productlib-pod.yaml
apiVersion: v1
kind: Pod
metadata:
name: productlib-pod
spec:
restartPolicy: Never
containers:
- name: app-container
image: nimmis/apache-php7
- name: cypress-container
image: rabbitmq
Load Balancing is one of the most common and the standard ways of exposing the services.
There are two types of load balancing in Kubernetes and they are:
1. Internal load balancer – This type of balancer automatically balances loads and
allocates the pods with the required configuration.
2. External Load Balancer – This type of balancer directs the traffic from the external
loads to backend pods.
The followings are the different types of services being provided by the Kubernetes:
• Cluster IP
• Node Port
• Load Balancer
• External name
ClusterIP: This is the default service type which exposes the service on a cluster-internal IP
by making the service only reachable within the cluster.
• A ClusterIP service is the default Kubernetes service. It gives you a service inside
your cluster that other apps inside your cluster can access.
• There is no external access.
1. Debugging your services, or connecting to them directly from your laptop for some
reason
Because this method requires you to run kubectl as an authenticated user, you should NOT use
this to expose your service to the internet or use it for production services.
NodePort: This exposes the service on each Node’s IP at a static port. Since,
a ClusterIP service, to which the NodePort service will route, is automatically created. We
can contact the NodePort service outside the cluster.
• A NodePort service is the most primitive way to get external traffic directly to your
service.
NodePort, as the name implies, opens a specific port on all the Nodes (the VMs), and any
traffic that is sent to this port is forwarded to the service
For these reasons, I don’t recommend using this method in production to directly expose your
service. If you are running a service that doesn’t have to be always available, or you are very
cost sensitive, this method will work for you. A good example of such an application is a
demo app or something temporary.
ExternalName: This service type maps the service to the contents of the externalName field
by returning a CNAME record with its value.
So, guys that was all about services. Now, you might be wondering how do external services
connect to these networks right?
Ingress Network
Unlike all the above examples, Ingress is actually NOT a type of service. Instead, it sits in
front of multiple services and act as a “smart router” or entry point into your cluster.
You can do a lot of different things with an Ingress, and there are many types of Ingress
controllers that have different capabilities.
The default GKE ingress controller will spin up a HTTP(S) Load Balancer for you. This will
let you do both path based and subdomain based routing to backend services. For example,
you can send everything on foo.yourdomain.com to the foo service, and everything under the
yourdomain.com/bar/ path to the bar service.
Ingress Network
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: my-ingress
spec:
backend:
serviceName: other
servicePort: 8080
rules:
- host: foo.mydomain.com
http:
paths:
- backend:
serviceName: foo
servicePort: 8080
- host: mydomain.com
http:
paths:
- path: /bar/*
backend:
serviceName: bar
servicePort: 8080
Unlike all the above examples, Ingress is actually NOT a type of service. Instead, it sits in
front of multiple services and act as a “smart router” or entry point into your cluster.
You can do a lot of different things with an Ingress, and there are many types of Ingress
controllers that have different capabilities.
The default GKE ingress controller will spin up a HTTP(S) Load Balancer for you. This will
let you do both path based and subdomain based routing to backend services. For example,
you can send everything on foo.yourdomain.com to the foo service, and everything under the
yourdomain.com/bar/ path to the bar service.
1. Ingress Resources
Ingress Resources defines how you want the requests to the services to be routed. It
contains the main routing rules.
2. Ingress controller
what ingress controller does is, it reads the ingress resource’s information and process
the data accordingly. So basically, ingress resources contain the rules to route the
traffic and ingress controller routes the traffic.Traffic routing is done by an ingress
controller.
There are many types of Ingress controllers, from the Google Cloud Load
Balancer, Nginx, Contour, Istio, and more.
Kubelet is a type of primary node agents that especially runs on each node. Kubelet only
works on the descriptions that the containers provide to the Podspec. Kubelet also makes sure
that the container described in Podspec is healthy and running.
Firstly GKE stands for Google Kubernetes Engine. GKE is a management and an
orchestration system that is used for Docker container and all the container clusters that
basically run within the Google’s public cloud services. Google Kubernetes engine is based
on Kubernetes.
• DNS
• Nodeports
• Configure maps and secrets
• Dashboards
• Enabling CNI
• Ingress
• Container runtime: Docker, rkt, CRI – O and containerd
8. What is minikube?
Minikube is a type of tool that makes the Kubernetes easy to run locally. Minikube basically
runs on the single nodes Kubernetes cluster that is inside the virtual machine on your laptop.
This is also used by the developers who are trying to develop by using Kubernetes day to
day.
Heapster is a type of cluster-wide aggregator that helps in the process of monitoring and
event data. Heapster helps to enable the container cluster monitoring and performance
analysis for Kubernetes.
10. What are the initial namespaces from which the Kubernetes starts?
The followings are the three initial namespaces from which the Kubernetes starts:
• Default
• Kube – system
• Kube – public
Kubernetes is especially intended for the use of the environments with many other users that
are being spread across multiple teams or projects. Namespaces are the way to divide the
cluster resources between the multiple users.
12. What are pods in Kubernetes?
A Kubernetes pod is a group of containers that are being deployed in the same host. Pods
have the capacity to operate one level higher than the individual containers. This is because
pods have the group of containers that work together to produce an artefact or to process a set
of work.
The followings are the main components that the node status:
• Address
• Condition
• Capacity
• Info
A node is a type of work machine in Kubernetes that was previously known as a minion. A
node can be a type of virtual machine or the physical machine. It always depends upon the
clusters. Each of the nodes provides the services that are necessary to run pods, and it is also
managed by the master components.
Kubernetes is basically a type of an open – source container. Kubernetes has the potential to
hold the container deployment, scaling and descaling of the container and load balancing.
Kubernetes was being developed in the year of 2014. It is also used to manage the Linux
containers across the privates, hybrid and cloud environments.
16. What are the difference between Kubernetes and Docker Swarm?
The followings are the main difference between Kubernetes and docker and they are:
KUBERNETES - SECRETS
Secrets
Secrets are secure objects which store sensitive data, such as passwords, tokens, and ssh
keys.
Storing sensitive data in Secrets is more secure than plaintext ConfigMaps or in Pod
specifications.
Using Secrets gives us control over how sensitive data is used, and reduces the risk of
exposing the data to unauthorized users.
Kubernetes keeps those info within a single central place (etcd), which may cause security
issues as described Securing Kubernetes secrets : How to efficiently secure access to etcd and
protect your secrets.
The kubectl create secret command packages these files into a Secret and creates the object
on the Apiserver:
$ kubectl create secret generic db-user-pass \
--from-file=./username.txt --from-file=./password.txt
secret/db-user-pass created
The data field is used to store arbitrary data, encoded using base64. The stringData field is
provided for convenience, and allows us to provide secret data as unencoded strings.
For example, to store two strings in a Secret using the data field, convert them to base64 as
follows:
apiVersion: v1
kind: Secret
metadata:
name: mysecret
type: Opaque
data:
username: YWRtaW4=
password: MWYyZDFlMmU2N2Rm
apiVersion: v1
kind: Pod
metadata:
name: mypod
spec:
containers:
- name: mypod
image: redis
volumeMounts:
- name: foo
mountPath: "/etc/foo"
readOnly: true
volumes:
- name: foo
secret:
secretName: mysecret
Create the pod and check if our volume for the secret is properly mounted:
$ kubectl create -f mysecret-pod.yaml
pod/mypod created
$ kubectl get po
NAME READY STATUS RESTARTS AGE
mypod 1/1 Running 0 26s
We can see inside the container that mounts a secret volume, the secret keys appear as files
and the secret values are base-64 decoded and stored inside these files. The program in a
container is responsible for reading the secrets from the files.
apiVersion: v1
kind: Pod
metadata:
name: secret-env-pod
spec:
containers:
- name: mycontainer
image: redis
env:
- name: SECRET_USERNAME
valueFrom:
secretKeyRef:
name: mysecret
key: username
- name: SECRET_PASSWORD
valueFrom:
secretKeyRef:
name: mysecret
key: password
restartPolicy: Never
$ kubectl get po
NAME READY STATUS RESTARTS AGE
secret-env-pod 1/1 Running 0 27s
Labels
Labels are key-value pairs which are attached to pods, replication controller and services.
They are used as identifying attributes for objects such as pods and replication controller.
They can be added to an object at creation time and can be added or modified at the run
time.
With Label Selectors, we can select a subset of objects. Kubernetes supports two types of
Selectors:
• Equality-Based Selectors
Equality-Based Selectors allow filtering of objects based on label keys and values. With this
type of Selectors, we can use the =, ==, or !=operators. For example, with env==dev we are
selecting the objects where the env label is set to dev.
• Set-Based Selectors
Set-Based Selectors allow filtering of objects based on a set of values. With this type of
Selectors, we can use the in, notin, and exist operators. For example, with env in (dev,qa),
we are selecting objects where the env label is set to dev or qa.
Let us walk through kubectl commands for filtering resources using set based requirements.
Similarly, you can play by using various combinations in set based requirements too for
selecting a set of pods.
Helm is a package manager which allows users to package, configure, and deploy
applications and services to the Kubernetes cluster.
helm init # when you execute this command client is going to create a deployment in the
cluster and that deployment will install the tiller, the server side of Helm
The packages we install through client are called charts. They are bundles of templatized
manifests. All the templating work is done by the Tiller
What is the difference between config map and secret? (Differentiate the answers as
with examples)
Config maps ideally stores application configuration in a plain text format whereas Secrets
store sensitive data like password in an encrypted format. Both config maps and secrets can
be used as volume and mounted inside a pod through a pod definition file.
Config map:
Secret:
If a node is tainted, is there a way to still schedule the pods to that node?
When a node is tainted, the pods don't get scheduled by default, however, if we have to still
schedule a pod to a tainted node we can start applying tolerations to the pod spec.
spec:
tolerations:
- key: "key"
operator: "Equal"
value: "value"
effect: "NoSchedule"
We can introduce probes. A liveness probe with a Pod is ideal in this scenario.
A liveness probe always checks if an application in a pod is running, if this check fails the
container gets restarted. This is ideal in many scenarios where the container is running but
somehow the application inside a container crashes.
spec:
containers:
- name: liveness
image: k8s.gcr.io/liveness
args:
- /server
livenessProbe:
httpGet:
path: /healthz
Having a Pod with two containers, can I ping each other? like using the container
name?
Containers on same pod act as if they are on the same machine. You can ping them using
localhost:port itself. Every container in a pod shares the same IP. You can `ping localhost`
inside a pod. Two containers in the same pod share an IP and a network namespace and They
are both localhost to each other. Discovery works like this: Component A's pods -> Service
Of Component B -> Component B's pods and Services have domain names
servicename.namespace.svc.cluster.local, the dns search path of pods by default includes that
stuff, so a pod in namespace Foo can find a Service bar in same namespace Foo by
connecting to `bar`
In Kubernetes we can create a cluster of servers that are connected to work as a single unit.
We can deploy a containerized application to all the servers in a cluster without specifying
the machine name.
We have to package applications in such a way that they do not depend on a specific host.
Ingress
1. When running the cluster and deployments on a cloud platform like AWS or GKE,
the load balancing feature is available out of the box and there’s no need to define
ingress rules. But again, using external load balancers means spending more money
and especially when your deployment is a small-scale deployment and you have a
tight budget, you might as well use Kubernetes Ingress which is absolutely free and
economical.
2. Ingress is an abstraction of layer 7 load balancing and not layer 4. When we talk about
layer 7, we refer the layer 7 of the OSI model which is the application layer. It routes
the traffic using the actual content of the message. When the load balancer has access
to the content of the message, it can obviously make smarter routing decisions
compared to counter load balancing techniques which use Layer 4(transport layer).
Let’s get further into the details. Ingress is split into two main parts – Ingress resources and
ingress controller
1. Ingress Resources
Ingress Resources defines how you want the requests to the services to be routed. It
contains the main routing rules.
2. Ingress controller
What ingress controller does is, it reads the ingress resource’s information and
process the data accordingly. So basically, ingress resources contain the rules to route
the traffic and ingress controller routes the traffic.
Routing using ingress is not standardized i.e. different ingress controller have different
semantics (different ways of routing).
At the end of the day, you need to build your own ingress controller based on your
requirements and implementations. Ingress is the most flexible and configurable routing
feature available.
Step 1: Create an instance, name it kubectl. We’re going to deploy the cluster using this
instance. This instance only has the kubectl tool installed that will interact with the master, it
does not have Kubernetes installed on it.
Note: We have three services in AWS that we’ll be using here – s3 bucket which stores all
the files, EC2 which is used to create an instance and deploy the service and IAM which is
used to configure permissions. These three pictures have no clue about each other’s existence
and hence Roles come into the picture.
Attach the appropriate policy to your Role (for this example admin access is given)
Kubernetes Certification Training
• Instructor-led Sessions
• Real-life Case Studies
• Assignments
• Lifetime Access
Explore Curriculum
Next, it’ll ask you to add tags which are optional. In my case, I haven’t attached any tags.
Give your Role a name and review the policies assigned to it and then press Create role.
Step 3: Attach the role to the instance. Go to instance settings -> Attach/Replace IAM
role -> attach the role you’ve created and then click on Apply.
Step 4: Once you’ve created the instance and attached the role, open the command emulator
i.e. cmder or putty and connect to the AWS instance. I’ll be using cmder for this demo. Once
you’ve connected to the instance, update the repository and install aws-cli using the
following commands:
$ wget https://github.com/kubernetes/kops/releases/download/1.10.0/kops-linux-amd64
$ chmod +x kops-linux-amd64
$ mv kops-linux-amd64 /usr/local/bin/kops
Step 7: With Kops installed, you must configure a domain for your cluster to access it from
outside. Create a hosted zone for it
Add a domain name for your cluster, change the type from Public Hosted Zone to Private
Hosted Zone for Amazon VPC and copy your instance VPC ID from the instance page to
the VPC ID column and add the region you want to create your hosted zone in.
Copy the VPC ID
The above screenshot shows where to add Domain name and VPC ID
You can now see your Hosted Zone is created.
Step 8: Create a bucket as the same name as domain name using the following command:
$ aws s3 mb s3://kube-demo.com
$ export KOPS_STATE_STORE=s3://kube-demo.com
Step 9: Before you create the cluster, you’ll have to create SSH public key.
$ ssh-keygen
Enter file where you want your key pair to be saved and create a password to access the ssh
public key. In this case, I’ve chosen the default location and used no password.
Step 10: Now that you’ve created the SSH key, create the cluster using the following
command:
You can find more information about kubeconfigs in the Kubernetes documentation:
https://kubernetes.io/docs/concepts/configuration/organize-cluster-access-kubeconfig/
You can even define multiple contexts in a kubeconfig file, allowing you to easily switch
between multiple clusters.
We use kubeconfigs to store the configuration data that will allow the many components of
Kubernetes to connect to and interact with the Kubernetes cluster.
How will the kubelet service on one of our worker nodes know how to locate the Kubernetes
API and authenticate with it? It will use a kubeconfig!
In the next lesson, we will generate the kubeconfigs that our cluster needs.