Section 2 : Scheduling

Manual scheduling

● Manual scheduling is manual defining the node onto which the pod will
be placed on. This can useful when kube-scheduler is not active.
● Property nodeName is defined in the spec section of the pod
definition file.
○ Example :-
◆ nodeName : node01
◆ nodeName : controlplane

Labels and selectors

● Labels and selectors are used to filter k8s objects such as pods with
their labels based on a criteria.
● To filter them the --selector option is used.
○ Example :-
◆ k get pods --selector label=value
◆ k get pods --selector

label=value,label=value,label=value,…n

Taints and tolerations

● Taints allow nodes to repel pods.

○ To taint a node use command :-
◆ k taint node node-name key=value:taint-effect
○ Taint effect can be :-
◆ NoSchedule : no pod will be schedule on the node unless it

has a matching toleration.

◆ PreferNoSchedule : the system will try to avoid placing a pod

that does not tolerate the taint on the node, but it is not
required.
◆ NoExecute : new pods will not be scheduled on the node and

existing pods if any, will be evicted if they do not have a

matching toleration.

● Tolerations are applied to pods for them to be able to be placed on

 ●

matching tainted nodes.

● Taints and tolerations work together to ensure that pods are not
scheduled onto inappropriate nodes. One or more taints are applied to
a node; this marks that the node should not accept any pods that do
not tolerate the taints.
● Taints and tolerations set nodes to only accept certain pods, but do no
guarantee that the pods will be schedule on their rightful nodes.
That’s where node selectors/affinity comes into play since taints and
tolerations does not tell a pod to be scheduled on a particular node
instead it tells the node to only accept pods with certain tolerations.

Node selector

● Node selectors are placed on pods in order to make sure that pods are
schedule only on node with the specified labels.
○ To label a node use command :-
◆ k label node node-name label-key=label-value
○ Now that we have labeled the node we can create a pod with

nodeSelector property.
◆ nodeSelector:

label-key: label-value
○ This places the pod on node with the defined labels.

● Node selectors have limitations in situations where there are many

criteria to consider. That’s when we use node affinity.

Node affinity and Anti-affinity

● Node affinity allows you to constrain which nodes your Pod can be
scheduled on based on node labels.
● This provides advanced capability to limit pod placement on specific
nodes, in cases where we consider many conditions.
○ As on node selector, we first make sure that nodes are labeled.
○ Then set node affinity rules to match the labels.
◆ affinity:

nodeAffinity:
requiredDuringSchedulingIgnoredDuringExecution:
nodeSelectorTerms:
- matchExpressions:
- key: <key-1>
operator: In | Exists
 values:
- <value-1>

○ If Exists operator is used there is no need to provide a list of

values since the system only checks if the key exists in node
labels.

Taints and tolerations vs Node affinity

● Using either taints and tolerations or node affinity rules alone cannot
100% guarantee that nodes are only dedicated to specific pods or that
pods will only be scheduled on specific nodes.
● A combination of taints & tolerations and node affinity rules can be
used together to completely dedicate nodes to specific pods.
○ Steps :-

. Use taints and tolerations to prevent undesired pods from

being placed on the nodes.
. Use node affinity rules to prevent our pods from being placed
on the wrong node.

Resource limits

● Resource limits can be used to define the amount of resources a pod

will use in a pod definition file.

● PODS
○ These are defined as resources in the spec.containters. section

in the pod definition file.

◆ resources:

requests:
memory: “1Gi”
cpu: 1
limits:
memory: “2Gi”
cpu: 2
○ Default values for cpu is .5 and memory is 256 Mi.

DaemonSets
 ● DaemonSets are used to deploy one copy of a pod on each node. They
work as replicaSets.
● This is useful when deploying monitoring or logging agents, kube-
proxy and networking solutions such as weave-net.
○ The definition file is exactly similar to the one of replicaSets,

except for kind property.

Static pods

● Static pods are managed directly by the kubelet daemon on a specific

node, without the API Server observing them.
● Unlike Pods that are managed by the control plane (for example,
a Deployment); instead, the kubelet watches each static Pod (and
restarts it if it fails).
○ Static Pods are always bound to one kubelet on a specific node.
○ Static Pods definitions file reside in a directory which is

periodically checked by kubelet.

◆ To know which directory in the details of kubelet.service :-

– Inspect the file passed as an option in --pod-manifest-

path=/etc/kubernetes/manifests \\
– For kubeadm clusters, use the option --
config=kubeconfig.yaml, and define the path as
staticPodPath=/path/ option in the kubeconfig.yaml file.

○ When kubelet finds a change in any of the files in the directory it

recreates the pod.
● You can only create static pods. You cannot create Static replicaSets,
deployments etc.

Use cases of static pods

– The are used to deploy control-plane components as pods in a

kubeadm cluster.

Multiple schedulers

● In some cases, the default Kubernetes scheduler may not fit your need
and you may want to deploy your own custom scheduler.
● You can deploy your own custom scheduler by configuring the
 ●

following :-
. Kube Scheduler Configuration
. ConfigMap
. Service Account
. ClusterRoleBinding as Kube scheduler
. ClusterRoleBinding as volume scheduler
. RoleBinding to authenticate to Kube API Server
. Deployment for the custom scheduler

○ To schedule a Pod to use your custom scheduler, add a property

under spec section :-
◆ schedulerName : <scheduler-name>

Configuring Scheduler Profiles

● When pods are created they go through :-

. Scheduling queue
◆ This is where they are sorted according to their priority.

– Priorities can be defined in a config file of kind:

PriorityClass.
. Filtering phase
◆ Node which do not meet the requirements to host the pod are

filtered out.
. Scoring phase
◆ Nodes are scored with different weights such as storage space

or CPUs that would remain after scheduling the pod.

. Binding phase
◆ The pod is bound to the node with the highest score.

● All of this is achieved with scheduling plugins in each of the 4 phases.

Some of these plugins are :-

. Scheduling queue
– PrioritySort : checks if a pod has priority property set on it.
. Filtering phase
– NodeResourcesFit : identifies the nodes with sufficient
resources required by the pod and filters our the ones that
doesn’t.
– NodeName : checks if a pod has nodeName property set on it
and filters out all other nodes with different names.
– NodeUnschedulable : filters out nodes which has
unschedulable option set true. This are drained / cordoned
 –

nodes.
– TaintToleration : checks for taints on nodes and tolerations on
pods.
– NodeAffinity : checks for node affinity rules on pods.
. Scoring phase
– NodeResourcesFit : identifies the nodes with sufficient
resources required by the pod and associates a high score to
nodes with much resources.
– ImageLocality : associates a high score to each node that
already has the container image to be used by the pod.
◆ If there is no node with the image this plugin will be

ignored.
. Binding phase
– DefaultBinder : provides the binding mechanism.

○ All the above plugins are used in the 4 phases with help of
extension points on each scheduling phase. This are points to
which a plugin can be plugged to in order to be used in that
particular phase. Some plugins span across multiple extension
points.
○ Extensions include :-
◆ Scheduling queue

. QueueSort
◆ Filtering phase

. preFilter
. filter
. postFilter
◆ Scoring phase

. preScore
. score
. reserve
◆ Scheduling queue

. permit
. preBind
. bind
. postBind

● How can we change the default behavior of how the plugins are
called(invoked) or how to create our own plugins.