Bilkent University

Department of Computer Engineering

CS342 Operating Systems

CPU Scheduling

Last Update: Feb 18, 2022

1
 Objectives and Outline

Objective
Outline
• ToBasic
introduce
ConceptsCPU scheduling, which is the basis for multi-programmed (multi-
• tasking) operating
Scheduling Criteriasystems
•• To describe Algorithms
Scheduling various CPU-scheduling algorithms
•• To discuss
Thread evaluation criteria for selecting a CPU-scheduling algorithm for a
Scheduling
• particular system Scheduling
Multiple-Processor
• Operating Systems Examples
• Algorithm Evaluation

2
 Basic Concepts

• Maximum CPU utilization obtained

with multiprogramming

• CPU–I/O Burst Cycle –

• Process execution consists of a cycle
of CPU execution and I/O wait

3
Histogram of CPU-burst Times

CPU burst distribution

4
 CPU Scheduler

• Selects from among the processes in memory that are ready to

execute, and allocates the CPU to one of them

• CPU scheduling decisions may take place when a process:

1. Switches from running to waiting state
2. Switches from running to ready state (upon interrupts)
3. Switches from waiting to ready
4. Terminates

• Scheduling under 1 and 4 is non-preemptive

• All other scheduling is preemptive
5
 Dispatcher

• Dispatcher module gives control of the CPU

to the process selected by the CPU P1
scheduler; this involves: executing

Dispatch latency
– switching context save state
into PCB1
– switching to user mode
– jumping to the proper location in the user reload state
from PCB2
program to restart that program
P2
executing
• Dispatch latency – time it takes for the
dispatcher to stop one process and start
another running

6
 Linux tools

• Assume you have a process started with pid x

• Type at command shell:
cat /proc/x/status

• at the bottom of the output you will see the number of context
switches that a process experiences. Like the following:

voluntary_ctxt_switches: 9
nonvoluntary_ctxt_switches: 18

7
 Scheduling Criteria

• CPU
Maximize
utilization
CPU utilization
– keep the CPU as busy as possible
• Maximize
Throughputthroughput
– # of processes that complete their execution per
time unit
• Turnaround time – amount
Minimize turnaround time of time to execute a particular
• process
Minimize waiting time

• Waiting time – total amount of time a process has been waiting

in the ready queue
Turn. Time = Finish time (completion time) – Start time (arrival time)

8
 Scheduling Criteria

• Response time – amount of

Important for Interactive Processes
time it takes from when a (interacting with a user)
request was submitted until
the first response is produced
(for time-sharing
environment)
Response arrived

ready running

waiting
Request submitted

We want to minimize response time

9
Scheduling Algorithms

10
First-Come, First-Served (FCFS) Scheduling

Process Burst Time (ms)

P1 24
P2 3
P3 3
• Suppose that the processes arrive in the order: P1 , P2 , P3
The Gantt Chart for the schedule is:

P1 P2 P3

0 24 27 30

• Waiting time for P1 = 0; P2 = 24; P3 = 27

• Average waiting time: (0 + 24 + 27)/3 = 17 ms

11
 FCFS Scheduling (Cont)

Suppose that the processes arrive in the order

P2 , P3 , P1
• The Gantt chart for the schedule is:

P2 P3 P1

0 3 6 30
• Waiting time for P1 = 6; P2 = 0; P3 = 3
• Average waiting time: (6 + 0 + 3)/3 = 3 ms
• Much better than previous case
• Convoy effect: short process behind long process

12
 Shortest-Job-First (SJF) Scheduling

• Associate with each process the length of its next CPU burst. Use these
lengths to schedule the process with the shortest time

• SJF is optimal – gives minimum average waiting time for a

given set of processes
– The difficulty is knowing the length of the next CPU
request

13
 Example of SJF

Process Arrival Time Burst Time

P1 0.0 6
P2 0.0 8
P3 0.0 7
P4 0.0 3
• SJF scheduling chart

P4 P1 P3 P2

0 3 9 16 24
• Average waiting time = (3 + 16 + 9 + 0) / 4 = 7 ms

14
 Determining Length of Next CPU Burst

• Can only estimate (predict) the length

• Can be done by using the length of previous CPU bursts
• using exponential averaging of previous burst lengths

15
 Predicting the Length of next CPU Burst:
Exponential Averaging
• Let tn denotes the length of the nth CPU burst.
– This is the actual length (known after burst executes)
• Let n+1 denote the predicted value for the next CPU burst
• Assume the first CPU burst is Burst0 and its length is t0.
• n denotes the previous estimate (before tn )

• Define  to be:
0 <=  <= 1

n+1 =  tn + (1 -  ) n

16
 Exponential Averaging

• We have CPU bursts as: Burst(0), Burst(1), Burst(2)….Burst(n),

Burst(n+1). The actual lengths of these bursts are denoted by: t0,
t1, t2, t3, …., tn, tn+1. Let 0 be initial estimate (i.e., estimate for
Burst(0)) and let it be a constant value like 10 ms. Then
1 =  t0 + (1 -  ) 0

• If we expand the formula, we get:

n+1 =  tn + (1 - ) tn-1 + …. + (1 -  )j  tn-j + …..
+ (1 -  )n  t0 + (1 -  )n +1 0

• Since both  and (1 - ) are less than or equal to 1, each

successive term has less weight than its predecessor
17
 Exponential Averaging

• If  =0
– n+1 = n
– Recent history does not count

• If  =1
– n+1 =  tn
– Only the actual last CPU burst counts

• Usually we have  between 0 and 1, for example 0.5

18
 Example

• 0 = 10 ms
• Measured CPU bursts: t0 = 8ms, t1=16ms, t2=20ms, t3=10ms
• Assume  = ½
– 1 = ½ x 8 + ½ x 10 = 9
– 2 = ½ x 16 + ½ x 9 = 12.5
– 3 = ½ x 20 + ½ x 12.5 = 16.25
– 4 = ½ x 10 + ½ x 16.25 = 13.125

– The next CPU burst is estimated to be 13.125 ms. After burst is executed,
it is measured as t4.

19
 Shortest Remaining Time First (SRTF)

• Preemptive version of SJF

• While a job A is running, if a new job B comes whose length is

shorter than the remaining time of job A, then B preempts A and
B is started to run.

20
 Shortest Remaining Job First (SRTF)

Process Arrival Time Burst Time

P1 0.0 8
P2 1.0 4
P3 2.0 9
P4 3.0 5
• SRJF scheduling chart

P1 P2 P4 P1 P3

0 1 5 10 17 26

• Average waiting time = (9 + 0 + 2 + 15) / 4 = 6.5 ms

21
 Example

• Assume we have the following processes. Find out the finish time, waiting time and
turnaround time of each process for the following scheduling algorithms: FCFS, SJF,
SRJF.

Process Arv time CPU Burst

A 0 30
B 5 20
C 10 12
D 15 10

22
 Example

FCFS: Processes will run in the order they arrive. The following is the finish,
turnaround, waiting time of each process.

Arv Burst Finish Turnarou Waiting

nd
A 0 30 30 30 0

B 5 20 50 45 25

C 10 12 62 52 40

D 15 10 72 57 47

23
 Example

SJF: running order will be: A(30) D(10) C(12) B(20)

Arv Burst Finish Turnarou Waiting

nd
A 0 30 30 30 0

B 5 20 72 67 47

C 10 12 52 42 30

D 15 10 40 25 15

24
 Example

SRTF: running order will be: A(5) B(5) C(12) D(10) B(15) A(25)

Arv Burst Finish Turnarou Waiting

nd
A 0 30 72 72 42

B 5 20 47 42 22

C 10 12 22 12 0

D 15 10 32 17 7

25
 Round Robin (RR)

• Each process gets a small unit of CPU time (time quantum – time
slice), usually 10-100 milliseconds. After this time has elapsed,
the process is preempted and added to the end of the ready
queue.
• If there are n processes in the ready queue and the time quantum
is q, then each process gets 1/n of the CPU time in chunks of at
most q time units at once.
• No process waits more than (n-1)q time units.

26
 Round Robin (RR)

• Performance
– q large  FIFO
– q small  good CPU sharing
q must be large with respect to context switch time, otherwise
overhead is too high

27
 Example of RR with Time Quantum = 4

Process Burst Time

P1 24
P2 3
P3 3

• The Gantt chart is:

P1 P2 P3 P1 P1 P1 P1 P1

0 4 7 10 14 18 22 26 30

• Typically, higher average turnaround than SJF,

but better response time

28
 Example

Finish time of each process?

a) Round Robin q=30
b) Round Robin q=10

29
 Example

Solution

30
 RR vs FCFS

• Round Robin is good for fast response, not for low turnaround
time.
Assume 3 jobs all arrived at time 0. Each has a CPU burst = 10

C C C
B B B
A A A

RR q=5 FCFS
A: 20 A: 10
B: 25 B: 20
C: 30 C: 30
Turnaround times Turnaround times
31
Time Quantum and Context Switch Time

32
Turnaround Time Varies
With The Time Quantum

33
 Priority Scheduling

• A priority number (integer) is associated with each process

• The CPU is allocated to the process with the highest priority
(smallest integer  highest priority)
– Preemptive (higher priority process preempts the running
one)
– Non-preemptive

• SJF is a priority scheduling (non-preemptive) where priority is the predicted next CPU
burst time.
• SRTF is preemptive

34
 Example

Arv CPU burst Priority

A 0 20 3
B 5 15 2
C 10 20 0
D 25 15 1
E 30 20 1

Nonpreemptive priority scheduling:

AAAACCCCDDDEEEEBBB
assuming each letter is 5 time units
Finish times: A: 20, B: 90, C: 40, D: 55, E: 75

Preemptive priority scheduling:

ABCCCCDDDEEEEBBAAA
Finish times: A: 90, B: 75, C:30, D: 45, E: 65
35
 Priority Scheduling

Problem  Starvation – low priority

processes may never execute

Solution  Aging – as time progresses

increase the priority of the process

36
 Priority Scheduling

• Priority scheduling can be combined with RR.

– Highest priority process is run first.
– If there are multiple processes with the same priority, they
can be served with RR.

P1 4 (3)
P2 5 (2)
P3 8 (2) P4 P2 P3 P2 P3 P 2 P 3 P 1 P 5 P 1 P5
P4 7 (1)
P5 3 (3) 0 7 9 11 13 15 16 20 22 24 26 27

(priority)
Time quantum=2
burst time

37
 Multilevel Queue Scheduling

• Ready queue is partitioned into separate (multiple) queues:.

• For example:
– Two queues
• foreground (interactive) and background (batch)
– Each queue has its own scheduling algorithm
• foreground – RR
• background – FCFS

38
 Multilevel Queue Scheduling

• Scheduling must be done between the queues as well

– Fixed (strict) priority scheduling; (i.e., serve all from
foreground then from background).
Possibility of starvation.
– Time slicing – each queue gets a certain amount of CPU time
which it can schedule amongst its processes; i.e., 80% to
foreground in RR; 20% to background in FCFS.

39
 Multilevel Queue Scheduling

Can be served with RR

Priority = 0 T1 T2 T3 T4 T5 T6 (if burst bigger than q,
preempted and put to the tail)

Priority = 1 T7 T8 T9

Priority = 2 T10 As long there are

tasks in the highest priority
Priority = 3 Empty queue queue, they are served.
…….
(Or time
slicing can be applied).
Priority = n Tx Ty Tz

40
 Multilevel Queue Scheduling

Highest priority

Real time processes

System processes

Interactive processes

Batch processes

Lowest priority

41
 Multilevel Feedback Queue (MLFQ) scheduling

• A process can move between the various queues; aging can be

implemented this way
• A specific Multilevel-Feedback-Queue scheduler is defined by
the following parameters:
– number of queues
– scheduling algorithms for each queue
– method used to determine when to upgrade a process
– method used to determine when to demote a process
– method used to determine which queue a process will enter
when that process needs service

42
 Example of Multilevel Feedback Queue
• Three queues:
– Q0 – RR with time quantum 8 milliseconds
– Q1 – RR time quantum 16 milliseconds
– Q2 – FCFS
• Scheduling
– A new job enters queue Q0 which is served RR (q=8). When
it gains CPU, job receives 8 milliseconds. If it does not
finish in 8 milliseconds, job is moved to queue Q1.
– At Q1 job is again served RR and receives 16 additional
milliseconds. If it still does not complete, it is preempted and
moved to queue Q2.
– At Q2, jobs are served with FCFS 43
Multilevel Feedback Queues

44
Thread Scheduling

45
 Thread Scheduling
• Distinction between user-level and kernel-level threads

• In many-to-one and many-to-many models, user-level thread

library schedules user-level threads to run on an LWP (virtual
processor)
– This is known as process-contention scope (PCS), since
scheduling competition is happening within the process
• In one-to-one model, kernel schedules a thread onto available
CPU.
– This is known as system-contention scope (SCS) – since
scheduling competition is happening with the system.

46
 Pthread Scheduling

• API allows specifying either PCS or SCS during thread creation

– PTHREAD SCOPE PROCESS schedules threads using PCS
scheduling
– PTHREAD SCOPE SYSTEM schedules threads using SCS
scheduling.

• Since Linux uses 1:1 model, it always schedules with SCS.

47
 Pthread Scheduling API
#include <pthread.h>
#include <stdio.h>
#define NUM THREADS 5
int main(int argc, char *argv[])
{
int i;
pthread t tid[NUM THREADS];
pthread attr t attr;
/* get the default attributes */
pthread attr init(&attr);
/* set the scheduling algorithm to PROCESS or SYSTEM */
pthread attr setscope(&attr, PTHREAD_SCOPE_SYSTEM);
/* set the scheduling policy - FIFO, RT, or OTHER */
pthread attr setschedpolicy(&attr, SCHED_OTHER);
/* create the threads */
for (i = 0; i < NUM THREADS; i++)
pthread create(&tid[i],&attr,runner,NULL);

48
 Pthread Scheduling API

/* now join on each thread */

for (i = 0; i < NUM THREADS; i++)
pthread join(tid[i], NULL);
}
/* Each thread will begin control in this function */
void *runner(void *param)
{
printf("I am a thread\n");
pthread exit(0);
}

49
Multiprocessor Scheduling

50
 Multiple-Processor Scheduling

• CPU scheduling more complex when multiple CPUs are

available
• Different multiprocessor architectures:
– Multicore CPUs
– Multithreaded cores
– NUMA systems
– Heterogenous multiprocessing

51
 Multiprocessor scheduling

• Asymmetric multiprocessing – only one processor accesses

the system data structures, alleviating the need for data
sharing

• Symmetric multiprocessing (SMP) –

– each processor is self-scheduling,

52
 Multiple-Processor Scheduling
SMP

CPU CPU CPU

Cache Cache Cache

sch …
P2 P10
Kernel P1 P3

Main Memory

53
 SMP

– all processes in common ready queue (a)

– or each has its own private queue of ready processes (b)
– SMP is very common.
load balancing may be needed
Too much contention
on the same queue

54
 Multicore Processors

• Recent trend to place multiple processor cores (CPUs) on same

physical chip (die)
• Faster and consumes less power
• Multiple threads per core also growing (hardware threading)
– Takes advantage of memory stall to make progress on another
thread while memory retrieve is happening.

55
 Multithreaded Multicore System

• Each core has > 1 hardware threads.

• If one thread has a memory stall, switch to another thread!

Appears to OS as two CPUs

56
 Multithreaded Multicore System

• Chip-multithreading (CMT)
assigns each core multiple hardware
threads. (Intel refers to this as
hyperthreading.)

• On a quad-core system with 2

hardware threads per core, the
operating system sees 8 logical
processors.

57
NUMA and CPU Scheduling

58
 Processor Affinity

• A thread (process) has affinity to the CPU it is running (due to

per CPU cache).
• If run in the same CPU, no need to re-populate the cache.
• Soft affinity: system tries to schedule to the same CPU
• Hard affinity: system always schedules to the same CPU

• Linux has sched_setaffinity() system call for hard affinity.

59
 Load Balancing

• If load is unbalanced:
– Load balancing among the cores in the same chip can be
performed.
– Load balancing between different NUMA processors is not
desirable.
• Cache needs to be populated
• Access to memory in other chip is slower.

60
 Heterogenous Multiprocessing (HMP)

• In mobile devices, there are

– Little cores (slow CPUs) (consuming less power)
– Big cores (fast CPUs) (consuming more power)

• Tasks are assigned to them depending on their processing needs.

• Tasks that need high performance are scheduling to big cores.
But they should not run too long.
• Tasks in little cores can run too long.

61
Real-Time CPU Scheduling

62
 Real time systems

• Real-time processes (tasks) running in a system

– Hard/Soft real time system
• Commonly, RT systems are event-driven systems
– An event occurs: need to respond (run a task) as quickly as possible
– Minimizing latency is important
• Latency
– Interrupt latency: time from interrupt till start of ISR
– Dispatch latency: time between task becomes runnable till task gets
running (includes the stopping of a current running process)

63
Latency

64
 Scheduling

• Priority based scheduling

– Priority based algorithm used
– Preemption may be provided as well
– Soft real time systems apply this.
– No hard guarantees

• Hard real time guarantees

– There are deadlines for processes (tasks) to start executing
– Process may be periodic or not
– Admission control required

65
 Hard guarantees

– Periodic tasks:
• period (p) and rate (1/p),
• cpu time (t),
• deadline (d);
We must have: 0 <= t <= d <= p
• Example scheduling: Rate monotonic scheduling

– Non periodic tasks:

• Example scheduling: Earliest deadline first (EDF)

66
 Rate monotonic scheduling

• Static priority used

– Priority inversely proportional with period.
• Preemptive: high priority task preempts low priority task
• CPU burst time (processing time) equal in all periods.
• CPU burst must be executed before the next period.
– Implicit Deadline = beginning of next period.

• Can underutilize CPU.

• Is optimal for static priority case (i.e., if we can not assign
priorities dynamically)
67
 Rate monotonic scheduling
Example: P1: cputime:20; period 50 20/50 + 35/100 = 75 / 100 less than 1
P2: cputime:35; period100

Example: P1: cputime:25; period 50

25/50 + 35/80 = 375/400 less than 1
P2: cputime:35; period 80

deadline missed
68
 Earliest Deadline First (EDF)

• Dynamically assigns priorities depending on the Deadlines.

– When a process becomes runnable (ready), it announces its
deadline.
– The process that has the Earliest Deadline is scheduled.
• Tasks do not need to be periodic
– CPU bursts may be different
– Theoretically optimal: can meet deadlines and can utilize
100%

69
 Earliest Deadline First (EDF)

• Example: Two periodic processes (tasks):

P1: cputime: 25; period 50
P2: cputime 35; period

70
 Proportional Share Scheduling

• Allocate T shares among all applications (processes)

• If an application has N shares, it can get N/T of CPU time.
• System ensures that each process gets CPU time proportional to
its shares.
• Admission control policy required.

71
Examples from Operating Systems

72
 Operating System Examples

• Solaris scheduling
• Windows scheduling
• Linux scheduling

73
Solaris Dispatch Table

74
Solaris Scheduling

75
Windows Priorities

76
 Linux Scheduling

• For ordinary processes that we run: earlier O(1) scheduling; now,

CFS scheduling is used
– CFS: Completely Fair Scheduling
• Linux uses scheduling classes. Two classes at the moment.
• Each class uses a fixed priority range.
– 1) time-sharing (default scheduling) class (100-140)
(CFS algorithm is used)
2) real-time class (0-99).

77
 Linux Scheduling – CFS Algorithm

• In time sharing:
– user can specify a nice value (priority value)
• between -20 and +19. (-20 is highest priority)
– CFS assigns a proportion of CPU time based on nice value
indirectly.
• Lower nice value -> higher CPU time given at the end
– Targeted latency: a time interval each ready process must run
once
• Interval allocated proportionally among processes
• Example: if there are 5 processes with equal priority in
ready state, and targeted latency (i.e., sched_latency) is
100ms, then each process will get a time slice of 20 ms.
78
 Linux Scheduling – CFS Algorithm

• For time sharing class, CFS does not assign priorities directly.
– There are no separate runqueues.
• Keep track how long each process has run as virtual runtime.
• Virtual runtime is advanced more for a lower priority (higher
nice value) process, less for a higher priority process.
– Hence decay for low priority is higher.
• Example: 200 ms actual run time will be more for low
priority process, less for high priority process, same for
nice value 0.
• Select the process that has the lowest virtual runtime.

79
 Linux Scheduling – CFS Algorithm

timeslice of a process with nice value k:

time_slice(k) = [weight(k) / sum-of-weights-of-ready-processes] x sched_latency

updating virtualruntime of a process with nice value k:

virtualruntime(k) = virtualruntime(k) + [weight(0) / weight(k)] * runtime(k)

80
 Linux Scheduling – CFS Algorithm

– Keep processes in runnable state in a red-black tree (a

balanced tree) with respect to their virtual runtimes.
– The process on the left lower side is the one to run next.
– Selection: O(1);
– Insertion of a process into tree: O(logN)

81
Algorithm Evaluation

82
 Algorithm Evaluation

• Deterministic modeling – takes a particular predetermined

workload and defines the performance of each algorithm for
that workload
– One form of analytic evaluation
– Valid for a particular scenario and input.
• Queuing models
• Simulation
• Implementation

83
Evaluation of CPU schedulers by Simulation

84
 Queuing Models
• Average throughput, utilization, and waiting time can be
computing by using queuing models for a system
– In this case, one or more CPUs and one or more ready queues
make the system.
• Arrivals of tasks modeled by a distribution (inter-arrival times)
– Exponential distribution is a good approximation for
stochastic arrivals (as opposed to deterministic arrivals)
• Service times (cpu times) modeled by a distribution
– Exponential distribution is a good approximation for CPU
times required (i.e., services times).
• A branch of mathematics called queuing networks analysis uses
such modes to analyze systems for waiting time, throughput, etc.
85
 Queuing Models
• Little’s formula relates the arrival rate, waiting time,
and queue length in a system that may have random l
arrivals and random service (CPU) times.
• Little’s formula: N = W * l
– N: number of tasks in the queue (excluding the
one in the cpu)
– W: the average waiting in the queue for a task
(excluding the time spent in CPU)
cpu
– l: arrival rate (number of tasks arriving per
second)
• Assumes a stable system (service rate > arrival rate)
• Valid for any arrival and service time distribution
86
 Queuing Models
• Example: Assume arrival rate is 30 tasks/sec. What is the
average queue length (excluding the task executing), if on the
average a task waits 200 ms in the system (excluding the cpu
execution)?
• Answer:
– N=W*l
– Therefore, N = 200 ms * 30 tasks/sec = 6. That means there
are 6 tasks in the system on the average, that are waiting in
the queue.
• Note that: if arrivals and service times are deterministic, no queuing and
therefore no waiting in the queue will happen. Waiting in the queue happens
due to random (stochastic) arrivals, i.e., bursty arrivals.

87
 References

• The slides here are adapted/modified from the textbook and its slides:
Operating System Concepts, Silberschatz et al., 9th edition, Wiley.
• Operating System Concepts, 9th edition, Silberschatz et al. Wiley.
• Modern Operating Systems, Andrew S. Tanenbaum, 3rd edition, 2009

88