Recall: Scheduling Policy Goals/Criteria

•  Minimize Response Time

CS162! –  Minimize elapsed time to do an operation (or job)
Operating Systems and! –  Response time is what the user sees:
»  Time to echo a keystroke in editor
Systems Programming! »  Time to compile a program
Lecture 10! »  Real-time Tasks: Must meet deadlines imposed by World
! •  Maximize Throughput
Scheduling –  Maximize operations (or jobs) per second
–  Throughput related to response time, but not identical:
»  Minimizing response time will lead to more context switching than if you
only maximized throughput
February 24th, 2016 –  Two parts to maximizing throughput
»  Minimize overhead (for example, context-switching)
Prof. Anthony D. Joseph
»  Efficient use of resources (CPU, disk, memory, etc)
http://cs162.eecs.Berkeley.edu •  Fairness
–  Share CPU among users in some equitable way
–  Fairness is not minimizing average response time:
»  Better average response time by making system less fair
2/24/16 Joseph CS162 ©UCB Spring 2016 Lec 10.2

Recall: First-Come, First-Served (FCFS) Scheduling Recall: FCFS Scheduling (Cont.)

•  First-Come, First-Served (FCFS) •  Example continued:
–  Also “First In, First Out” (FIFO) or “Run until done” –  Suppose that processes arrive in order: P2 , P3 , P1 !
»  In early systems, FCFS meant one program ! Now, the Gantt chart for the schedule is:!
scheduled until done (including I/O)
»  Now, means keep CPU until thread blocks P2 P3 P1
•  Example: Process Burst Time!
P1 24! 0 3 6 30
P2 3! –  Waiting time for P1 = 6; P2 = 0; P3 = 3
P3 3 –  Average waiting time: (6 + 0 + 3)/3 = 3
–  Suppose processes arrive in the order: P1 , P2 , P3 ! –  Average Completion time: (3 + 6 + 30)/3 = 13
The Gantt Chart for the schedule is:!
! •  In second case:
! P1 P2 P3 –  average waiting time is much better (before it was 17)
!
! –  Average completion time is better (before it was 27)
0 24 27 30 •  FIFO Pros and Cons:
–  Waiting time for P1 = 0; P2 = 24; P3 = 27 –  Simple (+)
–  Average waiting time: (0 + 24 + 27)/3 = 17
–  Short jobs get stuck behind long ones (-)
–  Average Completion time: (24 + 27 + 30)/3 = 27 »  Safeway: Getting milk, always stuck behind cart full of small items. Upside:
•  Convoy effect: short process behind long process get to read about space aliens!
2/24/16 Joseph CS162 ©UCB Spring 2016 Lec 10.3 2/24/16 Joseph CS162 ©UCB Spring 2016 Lec 10.4
 Round Robin (RR) Example of RR with Time Quantum = 20
•  FCFS Scheme: Potentially bad for short jobs! •  Example: Process Burst Time!
–  Depends on submit order P1 53!
P2 8!
–  If you are first in line at supermarket with milk, you don’t care who is P3 68!
behind you, on the other hand… P4 24
•  Round Robin Scheme –  The Gantt chart is:
–  Each process gets a small unit of CPU time ! P1 P2 P3 P4 P1 P3 P4 P1 P3 P3
(time quantum), usually 10-100 milliseconds
–  After quantum expires, the process is preempted ! 0 20 28 48 68 88 108 112 125 145 153
and added to the end of the ready queue. –  Waiting time forP1=(68-20)+(112-88)=72
–  n processes in ready queue and time quantum is q ⇒ P2=(20-0)=20!
»  Each process gets 1/n of the CPU time P3=(28-0)+(88-48)+(125-108)=85!
»  In chunks of at most q time units P4=(48-0)+(108-68)=88
»  No process waits more than (n-1)q time units
–  Average waiting time = (72+20+85+88)/4=66¼
•  Performance
–  q large ⇒ FCFS –  Average completion time = (125+28+153+112)/4 = 104½
–  q small ⇒ Interleaved (really small ⇒ hyperthreading?) •  Thus, Round-Robin Pros and Cons:
–  q must be large with respect to context switch, otherwise overhead is –  Better for short jobs, Fair (+)
too high (all overhead) –  Context-switching time adds up for long jobs (-)
2/24/16 Joseph CS162 ©UCB Spring 2016 Lec 10.5 2/24/16 Joseph CS162 ©UCB Spring 2016 Lec 10.6

Round-Robin Discussion Comparisons between FCFS and Round Robin

•  How do you choose time slice? •  Assuming zero-cost context-switching time, is RR always better than
–  What if too big? FCFS?
»  Response time suffers •  Simple example: 10 jobs, each take 100s of CPU time!
RR scheduler quantum of 1s!
–  What if infinite (∞)? All jobs start at the same time
»  Get back FIFO
•  Completion Times: Job # FIFO RR
–  What if time slice too small?
1 100 991
»  Throughput suffers!
2 200 992
•  Actual choices of timeslice: … … …
–  Initially, UNIX timeslice one second: 9 900 999
»  Worked ok when UNIX was used by one or two people.
10 1000 1000
»  What if three compilations going on? 3 seconds to echo each
keystroke! –  Both RR and FCFS finish at the same time
–  Average response time is much worse under RR!
–  In practice, need to balance short-job performance and long-job »  Bad when all jobs same length
throughput:
»  Typical time slice today is between 10ms – 100ms •  Also: Cache state must be shared between all jobs with RR but can
»  Typical context-switching overhead is 0.1ms – 1ms be devoted to each job with FIFO
»  Roughly 1% overhead due to context-switching –  Total time for RR longer even for zero-cost switch!
2/24/16 Joseph CS162 ©UCB Spring 2016 Lec 10.7 2/24/16 Joseph CS162 ©UCB Spring 2016 Lec 10.8
 Handling Differences in Importance:!
Earlier Example with Different Time Quantum Strict Priority Scheduling
P2 P4 P1 P3 Priority 3 Job 1 Job 2 Job 3
Best FCFS: [8] [24] [53] [68] Priority 2 Job 4
0 8 32 85 153 Priority 1
Quantum P1 P2 P3 P4 Average Priority 0 Job 5 Job 6 Job 7
Best FCFS 32 0 85 8 31¼
Q=1 84 22 85 57 62 •  Execution Plan
Q=5 82 20 85 58 61¼ –  Always execute highest-priority runable jobs to completion
Wait –  Each queue can be processed in Round-Robin fashion with some time-quantum
Q=8 80 8 85 56 57¼
Time •  Problems:
Q = 10 82 10 85 68 61¼
Q = 20 72 20 85 88 66¼ –  Starvation:
Worst FCFS 68 145 0 121 83½ »  Lower priority jobs don’t get to run because higher priority tasks always running
Best FCFS 85 8 153 32 69½ –  Deadlock: Priority Inversion
Q=1 137 30 153 81 100½ »  Not strictly a problem with priority scheduling, but happens when low priority task
has lock needed by high-priority task
Q=5 135 28 153 82 99½ »  Usually involves third, intermediate priority task that keeps running even though high-
Completion
Q=8 133 16 153 80 95½ priority task should be running
Time
Q = 10 135 18 153 92 99½ •  How to fix problems?
Q = 20 125 28 153 112 104½ –  Dynamic priorities – adjust base-level priority up or down based on heuristics
Worst FCFS 121 153 68 145 121¾ about interactivity, locking, burst behavior, etc…
2/24/16 Joseph CS162 ©UCB Spring 2016 Lec 10.9 2/24/16 Joseph CS162 ©UCB Spring 2016 Lec 10.10

Scheduling Fairness Administrivia

•  What about fairness?
–  Strict fixed-priority scheduling between queues is unfair (run highest, •  Midterm coming up soon
then next, etc): –  Wednesday 3/9 6-7:30PM in 10 EVANS and 155 DWINELLE
»  long running jobs may never get CPU
–  Rooms assignment: aa-eh 10 Evans, ej-oa 155 Dwinelle
»  In Multics, shut down machine, found 10-year-old job
–  Must give long-running jobs a fraction of the CPU even when there are –  Closed book, no calculators, one double-side page of
shorter jobs to run handwritten notes
–  Tradeoff: fairness gained by hurting avg response time! –  No class that day, extra office hours
•  How to implement fairness? –  Review session TBA on Sat or Sun afternoon
–  Could give each queue some fraction of the CPU
»  What if one long-running job and 100 short-running ones?
»  Like express lanes in a supermarket—sometimes express lanes get so •  Topics will include the material through lecture 12 (Wed 3/2)
long, get better service by going into one of the other lines –  Includes lectures, project 1, homeworks, readings, textbook
–  Could increase priority of jobs that don’t get service
»  What is done in some variants of UNIX
»  This is ad hoc—what rate should you increase priorities? •  Apple Core OS Tech Talk Infosession next week
»  And, as system gets overloaded, no job gets CPU time, so everyone –  Tuesday, March 1 6:15 – 7:30PM in Woz Lounge
increases in priority⇒Interactive jobs suffer

2/24/16 Joseph CS162 ©UCB Spring 2016 Lec 10.11 2/24/16 Joseph CS162 ©UCB Spring 2016 Lec 10.12
 Lottery Scheduling
•  Yet another alternative: Lottery Scheduling
–  Give each job some number of lottery tickets
–  On each time slice, randomly pick a winning ticket
–  On average, CPU time is proportional to number of tickets given
to each job
•  How to assign tickets?
–  To approximate SRTF, short running jobs get more, long running
jobs get fewer
–  To avoid starvation, every job gets at least one ticket (everyone
makes progress)
•  Advantage over strict priority scheduling: behaves gracefully as
BREAK load changes
–  Adding or deleting a job affects all jobs proportionally,
independent of how many tickets each job possesses

2/24/16 Joseph CS162 ©UCB Spring 2016 Lec 10.13 2/24/16 Joseph CS162 ©UCB Spring 2016 Lec 10.14

Lottery Scheduling Example How to Evaluate a Scheduling algorithm?

•  Lottery Scheduling Example •  Deterministic modeling
–  Assume short jobs get 10 tickets, long jobs get 1 ticket –  takes a predetermined workload and compute the performance of
each algorithm for that workload
# short jobs/ •  Queueing models
% of CPU each % of CPU each –  Mathematical approach for handling stochastic workloads
# long jobs short jobs gets long jobs gets
•  Implementation/Simulation:
1/1 91% 9% –  Build system which allows actual algorithms to be run against actual
0/2 N/A 50% data. Most flexible/general.
2/0 50% N/A
10/1 9.9% 0.99%
1/10 50% 5%
–  What if too many short jobs to give reasonable !
response time?
»  If load average is 100, hard to make progress
»  One approach: log some user out

2/24/16 Joseph CS162 ©UCB Spring 2016 Lec 10.15 2/24/16 Joseph CS162 ©UCB Spring 2016 Lec 10.16
 Recall: Assumption – CPU Bursts How to Handle Simultaneous Mix of Diff Types of Apps?
•  Can we use Burst Time (observed) to decide which application gets CPU
time?
Weighted toward small bursts
•  Consider mix of interactive and high throughput apps:
–  How to best schedule them?
–  How to recognize one from the other?
»  Do you trust app to say that it is “interactive”?
–  Should you schedule the set of apps identically on servers, workstations, pads,
and cellphones?
•  Assumptions encoded into many schedulers:
•  Execution model: programs alternate between bursts of CPU and I/O –  Apps that sleep a lot and have short bursts must be interactive apps – they
should get high priority
–  Program typically uses the CPU for some period of time, then does I/O,
then uses CPU again –  Apps that compute a lot should get low(er?) priority, since they won’t notice
–  Each scheduling decision is about which job to give to the CPU for use intermittent bursts from interactive apps
by its next CPU burst •  Hard to characterize apps:
–  With timeslicing, thread may be forced to give up CPU before finishing –  What about apps that sleep for a long time, but then compute for a long time?
current CPU burst
–  Or, what about apps that must run under all circumstances (say periodically)
2/24/16 Joseph CS162 ©UCB Spring 2016 Lec 10.17 2/24/16 Joseph CS162 ©UCB Spring 2016 Lec 10.18

What if we Knew the Future? Discussion

•  Could we always mirror best FCFS? •  SJF/SRTF are the best you can do at minimizing average response
•  Shortest Job First (SJF): time
–  Run whatever job has the least amount of ! –  Provably optimal (SJF among non-preemptive, SRTF among
computation to do preemptive)
–  Sometimes called “Shortest Time to !
Completion First” (STCF) –  Since SRTF is always at least as good as SJF, focus on SRTF
•  Shortest Remaining Time First (SRTF):
–  Preemptive version of SJF: if job arrives and has a shorter time to •  Comparison of SRTF with FCFS and RR
completion than the remaining time on the current job, immediately –  What if all jobs the same length?
preempt CPU
»  SRTF becomes the same as FCFS (i.e. FCFS is best can do if all jobs the
–  Sometimes called “Shortest Remaining Time to Completion same length)
First” (SRTCF)
–  What if jobs have varying length?
•  These can be applied either to a whole program or the current
CPU burst of each program »  SRTF (and RR): short jobs not stuck behind long ones
–  Idea is to get short jobs out of the system
–  Big effect on short jobs, only small effect on long ones
–  Result is better average response time
2/24/16 Joseph CS162 ©UCB Spring 2016 Lec 10.19 2/24/16 Joseph CS162 ©UCB Spring 2016 Lec 10.20
 Example to illustrate benefits of SRTF SRTF Example continued:
Disk Utilization:
A or B C C A B 9/201
C ~ 4.5%

RR 100ms time slice Disk Utilization:

C’s C’s C’s C’s C’s
I/O I/O I/O I/O ~90%
I/O but lots of
•  Three jobs: CABAB… C wakeups!
–  A,B: both CPU bound, run for week!
C: I/O bound, loop 1ms CPU, 9ms disk I/O
RR 1ms time slice
–  If only one at a time, C uses 90% of the disk, A or B could use C’s C’s
100% of the CPU I/O I/O
Disk Utilization:
•  With FIFO: C A A A 90%
–  Once A or B get in, keep CPU for two weeks
•  What about RR or SRTF? SRTF
C’s C’s
–  Easier to see with a timeline I/O I/O
2/24/16 Joseph CS162 ©UCB Spring 2016 Lec 10.21 2/24/16 Joseph CS162 ©UCB Spring 2016 Lec 10.22

SRTF Further discussion Predicting the Length of the Next CPU Burst
•  Starvation •  Adaptive: Changing policy based on past behavior
–  SRTF can lead to starvation if many small jobs! –  CPU scheduling, in virtual memory, in file systems, etc
–  Large jobs never get to run –  Works because programs have predictable behavior
•  Somehow need to predict future »  If program was I/O bound in past, likely in future
–  How can we do this? »  If computer behavior were random, wouldn’t help
–  Some systems ask the user •  Example: SRTF with estimated burst length
»  When you submit a job, have to say how long it will take
–  Use an estimator function on previous bursts: !
»  To stop cheating, system kills job if takes too long Let tn-1, tn-2, tn-3, etc. be previous CPU burst lengths. Estimate next
–  But: Even non-malicious users have trouble predicting runtime of their burst τn = f(tn-1, tn-2, tn-3, …)
jobs –  Function f could be one of many different time series estimation
•  Bottom line, can’t really know how long job will take schemes (Kalman filters, etc)
–  However, can use SRTF as a yardstick ! –  For instance, !
for measuring other policies exponential averaging!
–  Optimal, so can’t do any better τn = αtn-1+(1-α)τn-1!
•  SRTF Pros & Cons with (0<α≤1)
–  Optimal (average response time) (+) !
–  Hard to predict future (-)
–  Unfair (-)
2/24/16 Joseph CS162 ©UCB Spring 2016 Lec 10.23 2/24/16 Joseph CS162 ©UCB Spring 2016 Lec 10.24
 Multi-Level Feedback Scheduling Scheduling Details
•  Result approximates SRTF:
Long-Running Compute! –  CPU bound jobs drop like a rock
Tasks Demoted to !
Low Priority –  Short-running I/O bound jobs stay near top
•  Scheduling must be done between the queues
–  Fixed priority scheduling:
»  serve all from highest priority, then next priority, etc.
•  Another method for exploiting past behavior –  Time slice:
–  First used in CTSS »  each queue gets a certain amount of CPU time
–  Multiple queues, each with different priority »  e.g., 70% to highest, 20% next, 10% lowest
»  Higher priority queues often considered “foreground” tasks
•  Countermeasure: user action that can foil intent of the OS
–  Each queue has its own scheduling algorithm designer
»  e.g. foreground – RR, background – FCFS –  For multilevel feedback, put in a bunch of meaningless I/O to keep
»  Sometimes multiple RR priorities with quantum increasing exponentially job’s priority high
(highest:1ms, next:2ms, next: 4ms, etc)
–  Of course, if everyone did this, wouldn’t work!
•  Adjust each job’s priority as follows (details vary)
•  Example of Othello program:
–  Job starts in highest priority queue
–  Playing against competitor, so key was to do computing at higher
–  If timeout expires, drop one level priority the competitors.
–  If timeout doesn’t expire, push up one level (or to top) »  Put in printf’s, ran much faster!
2/24/16 Joseph CS162 ©UCB Spring 2016 Lec 10.25 2/24/16 Joseph CS162 ©UCB Spring 2016 Lec 10.26

Case Study: Linux O(1) Scheduler

Kernel/Realtime Tasks User Tasks
0 100 139
•  Priority-based scheduler: 140 priorities
–  40 for “user tasks” (set by “nice”), 100 for “Realtime/Kernel”
–  Lower priority value ⇒ higher priority (for nice values)
–  Highest priority value ⇒ Lower priority (for realtime values)
–  All algorithms O(1) – schedule n processes in constant time
»  Compute timeslices/priorities/interactivity credits when job finishes time slice
»  140-bit bit mask indicates presence or absence of job(s) at given priority level
•  Two separate priority queues (arrays): “active” and “expired”
BREAK –  All tasks in the active queue use up their timeslices and get placed on the
expired queue, after which queues swapped
•  Timeslice depends on priority – linearly mapped onto timeslice range
–  Like multi-level queue (1 queue per priority) with diff timeslice at each level
–  Execution split into “Timeslice Granularity” chunks – RR through priority
2/24/16 Joseph CS162 ©UCB Spring 2016 Lec 10.27 2/24/16 Joseph CS162 ©UCB Spring 2016 Lec 10.28
 O(1) Scheduler Continued Linux Completely Fair Scheduler (CFS)
•  Heuristics •  First appeared in 2.6.23, modified in 2.6.24
–  User-task priority adjusted ±5 based on heuristics
–  “CFS doesn’t track sleeping time and doesn’t use heuristics to
»  p->sleep_avg = sleep_time – run_time identify interactive tasks—it just makes sure every process gets a fair
»  Higher sleep_avg ⇒ more I/O bound the task, more reward (and vice versa) share of CPU within a set amount of time given the number of
–  Interactive Credit runnable processes on the CPU.”
»  Earned when task sleeps for “long” time, Spend when task runs for “long” time
»  IC is used to provide hysteresis to avoid changing interactivity for temporary •  Inspired by Networking “Fair Queuing”
changes in behavior
–  However, “interactive tasks” get special dispensation –  Each process given their fair share of resources
»  To try to maintain interactivity –  Models an “ideal multitasking processor” in which N processes
»  Placed back into active queue, unless another task has starved for too long… execute simultaneously as if they truly got 1/N of the processor
•  Real-Time Tasks »  Tries to give each process an equal fraction of the processor
–  Always preempt non-RT tasks and no dynamic adjustment of priorities –  Priorities reflected by weights such that increasing a task’s priority by
1 always gives the same fractional increase in CPU time – regardless
–  Scheduling schemes:
of current priority
»  SCHED_FIFO: preempts other tasks, no timeslice limit
»  SCHED_RR: preempts normal tasks, RR scheduling amongst tasks of same priority

2/24/16 Joseph CS162 ©UCB Spring 2016 Lec 10.29 2/24/16 Joseph CS162 ©UCB Spring 2016 Lec 10.30

CFS (Continued) CFS Examples

•  Suppose Targeted latency = 20ms and Minimum Granularity = 1ms
•  Idea: track “virtual time” received by each process when it is running
–  Take real execution time, scale by weighting factor
»  Lower priority ⇒ real time divided by greater weight •  Two CPU bound tasks with same priorities
»  Actually – multiply by sum of all weights/current weight –  Both switch with 10ms
–  Keep virtual time advancing at same rate •  Two CPU bound tasks separated by nice value of 5
–  One task gets 5ms, another gets 15ms
•  Targeted latency (TL): period of time after which all processes get to
•  40 tasks: each gets 1ms (no longer totally fair – miss target latency)
run at least a little
–  Each process runs with quantum (wp&&/ Σwi&) •  One CPU bound task, one interactive task same priority
–  Never smaller than “minimum granularity” –  While interact task sleeps, CPU bound task runs, increments vruntime
–  When interact task wakes up, runs immediately (it’s behind on vruntime)
•  Red-Black tree holds all runnable processes sorted on vruntime •  Group scheduling facilities (2.6.24)
–  O(log n) time to perform insertions/deletions –  Can give fair fractions to groups (user or other process group)
»  Cache the item at far left (item with earliest vruntime) –  So, two users, one starts 1 process, other starts 40, each gets 50% CPU
–  Scheduler always takes process with smallest vruntime (far left item)
2/24/16 Joseph CS162 ©UCB Spring 2016 Lec 10.31 2/24/16 Joseph CS162 ©UCB Spring 2016 Lec 10.32
 Real-Time Scheduling (RTS) Summary
•  Efficiency is important but predictability is essential: •  Round-Robin Scheduling:
–  Give each thread a small amount of CPU time when it executes; cycle between
–  We need to predict with confidence worst case response times for systems all ready threads
–  Pros: Better for short jobs
–  In RTS, performance guarantees are:
•  Shortest Job First (SJF)/Shortest Remaining Time First (SRTF):
»  Task- and/or class centric and often ensured a priori –  Run whatever job has the least amount of computation to do/least remaining
–  In conventional systems, performance is: amount of computation to do
–  Pros: Optimal (average response time)
»  System/throughput oriented with post-processing (… wait and see …) –  Cons: Hard to predict future, Unfair
–  Real-time is about enforcing predictability, and does not equal fast computing!!! •  Multi-Level Feedback Scheduling:
–  Multiple queues of different priorities and scheduling algorithms
•  Hard Real-Time –  Automatic promotion/demotion of process priority in order to approximate SJF/
SRTF
–  Attempt to meet all deadlines •  Lottery Scheduling:
–  EDF (Earliest Deadline First), LLF (Least Laxity First), ! –  Give each thread a priority-dependent number of tokens (short tasks⇒more
RMS (Rate-Monotonic Scheduling), DM (Deadline Monotonic Scheduling) tokens)
•  Linux CFS Scheduler: Fair fraction of CPU
•  Soft Real-Time –  Approximates a “ideal” multitasking processor
–  Attempt to meet deadlines with high probability •  Realtime Schedulers such as EDF
–  Minimize miss ratio / maximize completion ratio (firm real-time) –  Guaranteed behavior by meeting deadlines
–  Realtime tasks defined by tuple of compute time and period
–  Important for multimedia applications
–  Schedulability test: is it possible to meet deadlines with proposed set of
–  CBS (Constant Bandwidth Server) processes?
2/24/16 Joseph CS162 ©UCB Spring 2016 Lec 10.33 2/24/16 Joseph CS162 ©UCB Spring 2016 Lec 10.34

EDF: Schedulability Test

Theorem (Utilization-based Schedulability Test):
A task set T1 , T2 ,…, Tnwith Di = Pi is schedulable by the
earliest deadline first (EDF) scheduling algorithm if
n
( Ci %
∑ && D ## ≤ 1
i =1 ' i $

Exact schedulability test (necessary + sufficient)

Proof: [Liu and Layland, 1973]

2/24/16 Joseph CS162 ©UCB Spring 2016 Lec 10.35