JNTUA Operating Systems Notes - R20
JNTUA Operating Systems Notes - R20
me/jntua
LECTURE NOTES
ON
Operating Systems
20A05402T
(JNTUA-R20)
2020-21
PHI, 2006.
2. Dhamdhere D M, Operating Systems A Concept Based Approach, 3rd edition, Tata
McGraw-
Hill, 2012.
3. Stallings W, Operating Systems -Internals and Design Principles, 6th edition, Pearson
Education, 2009
4. Nutt G, Operating Systems, 3rd edition, Pearson Education, 2004
Online Learning Resources:
https://nptel.ac.in/courses/106/106/106106144/
http://peterindia.net/OperatingSystems.html
UNIT-1
Operating System Overview
OVER VIEW OF OPERATING SYSTEM
What is an Operating System?
A program that acts as an intermediary between a user of a computer and the computer hardware
Operating system goals:
• Execute user programs and make solving user problems easier
• Make the computer system convenient to use
• Use the computer hardware in an efficient manner
Computer System Structure
• Computer system can be divided into four components
• Hardware – provides basic computing resources
• CPU, memory, I/O devices
Operating system
Controls and coordinates use of hardware among various applications and users
Application programs – define the ways in which the system resources are used to solve the computing
problems of the usersWord processors, compilers, web browsers, database systems, video games
Users
People, machines, other computers
Four Components of a Computer System
Page 5
• One or more CPUs, device controllers connect through common bus providing access to shared
memory
• Concurrent execution of CPUs and devices competing for memory cycles
Operating-System Operations
• Interrupt driven by hardware
• Software error or request creates exception ortrap
• Division by zero, request for operating system service
• Other process problems include infinite loop, processes modifying each other or the
operating system
• Dual-mode operation allows OS to protect itself and other systemcomponents
o User mode and kernelmode
o Mode bit provided byhardware
o Providesabilitytodistinguishwhensystemisrunningusercodeorkernel code
o Someinstructionsdesignatedasprivileged,onlyexecutableinkernel mode
o System call changes mode to kernel, return from call resets it to user
Transition from User to Kernel Mode
✓ Timer to prevent infinite loop / process hogging resources
• Set interrupt after specific period
• Operating system decrements counter
• When counter zero generate an interrupt
Set up before scheduling process to regain control or terminate program that exceeds allotted
Page 6
Computing Environments:
Client-Server Computing
o Dumb terminals supplanted by smart PCs
o Many systems now servers, responding to requests generated byclients
• Compute-
serverprovidesaninterfacetoclienttorequestservices(i.e.,
database)
• File-server provides interface for clients to store and retrieve files
Page 7
Peer to Peer:
✓ P2P does not distinguish clients and servers
o Instead all nodes are considered peers
o May each act as client, server orboth
o Node must join P2P network
o Registers its service with central lookup service on network, or
o Broadcast request for service and respond to requests for service via
discovery protocol
o Examples include Napster andGnutella
Web-Based Computing
✓ Web has become ubiquitous
✓ PCs most prevalent devices
✓ More devices becoming networked to allow webaccess
✓ New category of devices to manage web traffic among similar servers: load balancers
✓ Use of operating systems like Windows 95, client-side, have evolved into Linux and
Windows XP, which can be clients andservers
Open-Source Operating Systems:
✓ Operating systems made available in source-code format rather than just binary closed-
source
✓ Counter to the copy protectionand Digital Rights Management(DRM)movement
✓ Started by Free Software Foundation (FSF), which has “copyleft” GNU Public
License (GPL)
Page 8
✓ Examples include GNU/Linux and BSD UNIX(including core of Mac OS X), and many
more
Operating System Services:
✓ Operating systems provide an environment for execution of programs and services to
programs and users
✓ One set of operating-system services provides functions that are helpful to theuser:
o User interface - Almost all operating systems have a user interface(UI).
4 Varies between Command-Line (CLI), Graphics User Interface (GUI),
Batch
o Program execution - The system must be able to load a program into memory
and to run that program, end execution, either normally or abnormally (indicating
error)
o I/O operations - A running program may require I/O, which may involve a file
or an I/O device
o File-system manipulation - The file system is of particular interest. Programs
need to read and write files and directories, create and delete them, search them,
list file Information, permission management.
o Communications – Processes may exchange information, on the same computer
or between computers over a network
4Communicationsmaybeviasharedmemoryorthroughmessagepassing
(packets moved by theOS)
o Error detection – OS needs to be constantly aware of possibleerrors
4MayoccurintheCPUandmemoryhardware,inI/Odevices,inuser
program
4Foreachtypeoferror,OSshouldtaketheappropriateactiontoensure
correct and consistentcomputing
4Debuggingfacilitiescangreatlyenhancetheuser’sandprogrammer’s
abilities to efficiently use thesystem
✓ Another set of OS functions exists for ensuring the efficient operation of the system itself
via resource sharing
o Resource allocation - When multiple users or multiple jobs running
concurrently, resources must be allocated to each of them
4Manytypesofresources-Some(suchasCPUcycles,mainmemory,and
filestorage)mayhavespecialallocationcode,others(suchasI/Odevices)
may have general request and releasecode
Page 9
o Accounting - To keep track of which users use how much and what kinds of
computer resources
o Protection and security - The owners of information stored in a multiuser or
networked computer system may want to control use of that information,
concurrent processes should not interfere with eachother
4Protectioninvolvesensuringthatallaccesstosystemresourcesis
controlled
4 Security of the system from outsiders requires user authentication,
extendstodefendingexternalI/Odevicesfrominvalidaccessattempts
4Ifasystemistobeprotectedandsecure,precautionsmustbeinstituted
throughoutit.Achainisonlyasstrongasitsweakestlink.
System Calls:
✓ Programming interface to the services provided by theOS
✓ Typically written in a high-level language (C or C++)
✓ Mostly accessed by programs via a high-level Application Program Interface
(API)rather than direct system calluse
✓ Three most common APIs are Win32 API for Windows, POSIX API for POSIX-based
systems (including virtually all versions of UNIX, Linux, and Mac OS X), and Java API
for the Java virtual machine(JVM)
✓ Why use APIs rather than systemcalls?
Page 10
(Note that the system-call names used throughout this text are generic)
Page 11
o Typically, these programs format and print the output to the terminal or other
output devices
o Some systems implement a registry - used to store and retrieve configuration
information
✓ File modification
o Text editors to create and modify files
o Special commands to search contents of files or perform transformations of the
text
o Programming-language support - Compilers, assemblers, debuggers and
interpreters sometimes provided
✓ Program loading and execution- Absolute loaders, relocatable loaders, linkage editors,
and overlay-loaders, debugging systems for higher-level and machinelanguage
✓ Communications - Provide the mechanism for creating virtual connections among
processes, users, and computersystems
o Allow users to send messages to one another’s screens, browse web pages, send
electronic-mail messages, log in remotely, transfer files from one machine to
another
Operating-System Debugging:
✓ Debuggingis finding and fixing errors, orbugs
✓ OSes generate log filescontaining errorinformation
✓ Failure of an application can generate core dumpfile capturing memory of theprocess
✓ Operating system failure can generate crash dumpfile containing kernel memory
✓ Beyond crashes, performance tuning can optimize systemperformance
✓ Kernighan’s Law: “Debugging is twice as hard as writing the code in the firstplace.
Therefore, if you write the code as cleverly as possible, you are, by definition, not smart
enough to debug it.”
✓ DTrace tool in Solaris, FreeBSD, Mac OS X allows live instrumentation on production
systems
o Probes fire when code is executed, capturing state data and sending it to
consumers of those probes
Operating System Generation:
✓ Operating systems are designed to run on any of a class of machines; the system must be
configured for each specific computer site
✓ SYSGEN program obtains information concerning the specific configuration of the
hardware system
Page 13
Page 14
UNIT-2
PROCESS THREADS, PROCESS SYNCHRONISATON,
CPUSCHEDULING
Process Concept:
✓ An operating system executes a variety ofprograms:
o Batch system –jobs
o Time-shared systems – user programs ortasks
o Textbook uses the terms job and process almostinterchangeably
✓ Process – a program in execution; process execution must progress in sequentialfashion
✓ A process includes:
o program counter
o stack
o data section
The Process:
✓ Multiple parts
o The program code, also called textsection
o Current activity including program counter, processorregisters
o Stack containing temporary data
o Function parameters, return addresses, local variables
o Data section containing global variables
o Heap containing memory dynamically allocated during runtime
✓ Program is passive entity, process isactive
o Program becomes process when executable file loaded intomemory
✓ Execution of program started via GUI mouse clicks, command line entry of its name,etc
✓ One program can be several processes
o Consider multiple users executing the same program
Page 15
Process State:
✓ As a process executes, it changesstate
o new: The process is beingcreated
o running: Instructions are being executed
o waiting: The process is waiting for some event tooccur
o ready: The process is waiting to be assigned to aprocessor
o terminated: The process has finishedexecution
✓ Process state
✓ Program counter
✓ CPU registers
✓ CPU scheduling information
✓ Memory-management information
✓ Accounting information
✓ I/O status information
Process Scheduling:
✓ Maximize CPU use, quickly switch processes onto CPU for timesharing
✓ Process scheduler selects among available processes for next execution onCPU
✓ Maintains scheduling queues of processes
Page 17
Schedulers:
✓ Long-term scheduler(or job scheduler) – selects which processes should be brought into
the ready queue
✓ Short-term scheduler(or CPU scheduler) – selects which process should be executed
next and allocates CPU
o Sometimes the only scheduler in a system
✓ Short-term scheduler is invoked very frequently (milliseconds) (must be fast)
✓ Long-term scheduler is invoked very infrequently (seconds, minutes) (may beslow)
✓ The long-term scheduler controls the degree ofmultiprogramming
✓ Processes can be described aseither:
o I/O-bound process– spends more time doing I/O than computations, many short
CPU bursts
o CPU-bound process– spends more time doing computations; few very long CPU
bursts
Page 18
Threads
Page 19
User Threads:
Thread management done by user-level threads library
Three primary thread libraries:
o POSIX Pthreads
o Win32 threads
o Java threads
Kernel Threads:
Supported by the Kernel
Examples
o Windows XP/2000
o Solaris
o Linux
o Tru64 UNIX
o Mac OS X
Multithreading Models:
Page 20
Many-to-One
One-to-One
Many-to-Many
Many-to-One
Many user-level threads mapped to single kernel thread
Examples:
o Solaris Green Threads
o GNU Portable Threads
One-to-One:
Each user-level thread maps to kernel thread
Examples
o Windows NT/XP/2000
o Linux
o Solaris 9 and later
Page 21
Many-to-Many Model:
Allows many user level threads to be mapped to many kernelthreads
Allows the operating system to create a sufficient number of kernelthreads
Solaris prior to version 9
Thread Libraries:
Thread library provides programmer with API for creating and managingthreads
Two primary ways of implementing
o Library entirely in user space
o Kernel-level library supported by the OS
Pthreads
May be provided either as user-level or kernel-level
A POSIX standard (IEEE 1003.1c) API for thread creation andsynchronization
API specifies behavior of the thread library, implementation is up to development of the
library
Common in UNIX operating systems (Solaris, Linux, Mac OSX)
Page 22
Java Threads:
Java threads are managed by the JVM
Typically implemented using the threads model provided by underlyingOS
Java threads may be created by:
o Extending Thread class
o Implementing the Runnable interface
Threading Issues:
Semantics of fork() and exec() systemcalls
Page 23
Scheduler activations
Thread Cancellation:
Terminating a thread before it has finished
Two general approaches:
o Asynchronous cancellation terminates the target thread immediately.
o Deferred cancellation allows the target thread to periodically check if it should be
cancelled.
Thread Pools:
Create a number of threads in a pool where they await work
Advantages:
o Usually slightly faster to service a request with an existing thread than create a
new thread
o Allows the number of threads in the application(s) to be bound to the size of the
pool
Scheduler Activations:
Both M:M and Two-level models require communication to maintain the appropriate
number of kernel threads allocated to the application
Page 24
Each process must ask permission to enter critical section in entry section, may follow
critical section with exit section, then remaindersection
Especially challenging with preemptive kernels
General structure of process piis
Page 25
2. Progress - If no process is executing in its critical section and there exist some processes that
wish to enter their critical section, then the selection of the processes that will enter the critical
section next cannot be postponed indefinitely
3. Bounded Waiting - A bound must exist on the number of times that other processes are
allowed to enter their critical sections after a process has made a request to enter its critical
section and before that request is granted
ó Assume that each process executes at a nonzerospeed
ó No assumption concerning relative speed of the nprocesses
Peterson’s Solution:
Two process solution
Assume that the LOAD and STORE instructions are atomic; that is, cannot beinterrupted
The two processes share two variables:
o intturn;
o Booleanflag[2]
o The variable turn indicates whose turn it is to enter the criticalsection
The flag array is used to indicate if a process is ready to enter the critical section.flag[i]
= true implies that process Piis ready!
Algorithm for Process Pi
do {
flag[i] = TRUE;
turn = j;
while (flag[j] && turn == j);
critical section
flag[i] = FALSE;
remainder section
} while (TRUE);
Provable that
1. Mutual exclusion is preserved
2. Progress requirement is satisfied
3. Bounded-waiting requirement is met
Synchronization Hardware:
Many systems provide hardware support for critical sectioncode
Page 26
Page 27
. .
signal(S); signal (Q);
signal(Q); signal (S);
Starvation – indefinite blocking
o A process may never be removed from the semaphore queue in which it is
suspended
Page 29
// writing is performed
signal (wrt) ;
} while (TRUE);
The structure of a reader process
Page 26
wait (mutex)
;readcount ++
;
if (readcount == 1) wait (wrt) ;
signal (mutex)
// reading is performed
wait (mutex) ;
readcount - -;
if (readcount == 0)
signal (wrt) ;
signal (mutex) ;
} while (TRUE);
Dining-Philosophers Problem
Page 27
Page 28
wait ( chopstick[i] );
wait ( chopStick[ (i + 1) % 5] );
// eat
signal ( chopstick[i] );
signal (chopstick[ (i + 1) % 5] );
// think
} while (TRUE);
Monitors
A high-level abstraction that provides a convenient and effective mechanism for process
synchronization
Abstract data type, internal variables only accessible by code within theprocedure
Only one process may be active within the monitor at atime
But not powerful enough to model some synchronization schemes
monitor monitor-name
{
// shared variable declarations
procedure P1 (…) { …. }
procedurePn (…) {……}
Initialization code (…) { … }
}
}
Schematic view of a Monitor
Page 28
Scheduling Criteria:
CPU utilization – keep the CPU as busy aspossible
Throughput – # of processes that complete their execution per time unit
Turnaround time – amount of time to execute a particularprocess
Waiting time – amount of time a process has been waiting in the readyqueue
Response time – amount of time it takes from when a request was submitted until the
first response is produced, not output (for time-sharingenvironment)
Scheduling Algorithm Optimization Criteria
Max CPU utilization
Max throughput
Min turnaround time
Min waiting time
Min response time
First-Come, First-Served (FCFS) Scheduling
Process BurstTime
P1 24
P2 3
P3 3
Suppose that the processes arrive in the order: P1,P2, P3
The Gantt Chart for the schedule is:
P1 P2 P
0 24 27 30
Page 29
P2 P3 P1
0 3 6 30
Page 30 CSE
TheCPUisallocatedtotheprocesswiththehighestpriority(smallestintegerhighest
priority)
o Preemptive
o Nonpreemptive
o SJF is priority scheduling where priority is the inverse of predicted next CPU
burst time
Problem Starvation– low priority processes may never execute
SolutionAging– as time progresses increase the priority of the
processProcessBurst TimePriority
P1 10 3
P2 1 1
P3 2 4
P4 1 5
P5 5 2
Priority scheduling Gantt Chart
P2 P4 P5P1 P1 P3P3 P4 P2
Page 31
UNIT-3
Virtual Memory, Main Memory, Deadlocks
Program must be brought (from disk) into memory and placed within a process for it to
be run
Main memory and registers are only storage CPU can accessdirectly
Memory unit only sees a stream of addresses + read requests, or address + data and write
requests
Register access in one CPU clock (orless)
Main memory can take many cycles
Cache sits between main memory and CPUregisters
Protection of memory required to ensure correct operation
Base and Limit Registers
A pair of base and limit registers define the logical addressspace
Page 32
To start, consider simple scheme where the value in the relocation register is added to
every address generated by a user process at the time it is sent tomemory
o Base register now called relocationregister
o MS-DOS on Intel 80x86 used 4 relocation registers
The user program deals with logical addresses; it never sees the real physicaladdresses
o Execution-time binding occurs when reference is made to location inmemory
o Logical address bound to physicaladdresses
Dynamic relocation using relocation register
Page 33
Dynamic Loading
Routine is not loaded until it is called
Better memory-space utilization; unused routine is never loaded
All routines kept on disk in relocatable load format
Useful when large amounts of code are needed to handle infrequently occurringcases
No special support from the operating system isrequired
o Implemented through programdesign
o OS can help by providing libraries to implement dynamicloading
Dynamic Linking
Static linking – system libraries and program code combined by the loader into the binary
program image
Dynamic linking –linking postponed until execution time
Small piece of code, stub, used to locate the appropriate memory-resident libraryroutine
Stub replaces itself with the address of the routine, and executes theroutine
Operating system checks if routine is in processes’ memoryaddress
o If not in address space, add to addressspace
Dynamic linking is particularly useful for libraries
System also known as sharedlibraries
Consider applicability to patching system libraries
o Versioning may be needed
Swapping
A process can be swapped temporarily out of memory to a backing store, and then
brought back into memory for continued execution
o Total physical memory space of processes can exceed physicalmemory
Backing store – fast disk large enough to accommodate copies of all memory images for
all users; must provide direct access to these memoryimages
Roll out, roll in – swapping variant used for priority-based scheduling algorithms; lower-
priority process is swapped out so higher-priority process can be loaded andexecuted
Major part of swap time is transfer time; total transfer time is directly proportional to the
amount of memory swapped
Page 34
System maintains a ready queue of ready-to-run processes which have memory images
on disk
Does the swapped out process need to swap back in to same physicaladdresses?
Depends on address binding method
o Plus consider pending I/O to / from process memoryspace
Modified versions of swapping are found on many systems (i.e., UNIX, Linux, and
Windows)
o Swapping normally disabled
o Started if more than threshold amount of memory allocated
o Disabled again once memory demand reduced below threshold
Contiguous Allocation
Main memory usually into two partitions:
o Resident operating system, usually held in low memory with interruptvector
o User processes then held in highmemory
o Each process contained in single contiguous section ofmemory
Relocation registers used to protect user processes from each other, and fromchanging
operating-system code and data
o Base register contains value of smallest physical address
o Limit register contains range of logical addresses – each logical address must be
less than the limit register
o MMU maps logical addressdynamically
Page 35
o Can then allow actions such as kernel code being transient and kernel changing
size
Hardware Support for Relocation and Limit Registers
Multiple-partition allocation
o Degree of multiprogramming limited by number of partitions
o Hole – block of available memory; holes of various size are scattered throughout
memory
o When a process arrives, it is allocated memory from a hole large enough to
accommodate it
o Process exiting frees its partition, adjacent free partitionscombined
o Operating system maintains information about:
a) allocatedpartitionsb) free partitions (hole)
Dynamic Storage-Allocation Problem
First-fit: Allocate the first hole that is bigenough
Best-fit: Allocate the smallest hole that is big enough; must search entire list, unless
ordered by size
o Produces the smallest leftover hole
Worst-fit: Allocate the largest hole; must also search entirelist
o Produces the largest leftover hole
Fragmentation
External Fragmentation – total memory space exists to satisfy a request, but it is not
contiguous
Internal Fragmentation – allocated memory may be slightly larger than requested
memory; this size difference is memory internal to a partition, but not beingused
Page 36
First fit analysis reveals that given N blocks allocated, 0.5 N blocks lost tofragmentation
o 1/3 may be unusable ->50-percentrule
Paging
Physical address space of a process can be noncontiguous; process is allocated physical
memory whenever the latter is available
Divide physical memory into fixed-sized blocks calledframes
o Size is power of 2, between 512 bytes and 16Mbytes
Divide logical memory into blocks of same size called pages
Keep track of all free frames
To run a program of size N pages, need to find N free frames and loadprogram
Set up a page table to translate logical to physicaladdresses
Backing store likewise split into pages
Still have Internal fragmentation
Address generated by CPU is dividedinto:
o Page number (p) – used as an index into a page table which contains base
address of each page in physical memory
o Page offset (d) – combined with base address to define the physical memory
address that is sent to the memoryunit
Paging Hardware
Page 37
Free Frames
Page 38
Page 39
Page 40
Page 41
Entry consists of the virtual address of the page stored in that real memory location, with
information about the process that owns thatpage
Decreases memory needed to store each page table, but increases time needed to search
the table when a page reference occurs
Use hash table to limit the search to one — or at most a few — page-tableentries
o TLB can accelerate access
But how to implement sharedmemory?
o One mapping of a virtual address to the shared physicaladdress
Page 42
Segmentation
Memory-management scheme that supports user view of memory
A program is a collection of segments
A segment is a logical unit such as:
main program, procedure ,function, method, object, local variables, global
variables, common block, stack, symbol table, arrays
Segmentation Architecture
Logical address consists of a twotuple:
<segment-number, offset>,
Segment table – maps two-dimensional physical addresses; each table entryhas:
o base – contains the starting physical address where the segments reside in
memory
o limit – specifies the length of thesegment
o Segment-table base register (STBR) points to the segment table’s location in
memory
Page 43
Page 44
Demand Paging
Could bring entire process into memory at loadtime
Or bring a page into memory only when it isneeded
o Less I/O needed, no unnecessary I/O
o Less memory needed
o Faster response
o More users
Page is needed reference toit
o invalid reference abort
o not-in-memory bring tomemory
Lazy swapper – never swaps a page into memory unless page will beneeded
o Swapper that deals with pages is apager
Page 45
Valid-Invalid Bit
With each page table entry a valid–invalid bit is associated
(v in-memory – memory resident, inot-in-memory)
Initially valid–invalid bit is set toion all entries
Page Fault
If there is a reference to a page, first reference to that page will trap to operatingsystem:
Page 46
page fault
1. Operating system looks at another table to decide:
o Invalid reference abort
o Just not in memory
2. Get empty frame
3. Swap page into frame via scheduled disk operation
4. Reset tables to indicate page now in memory
Set validation bit = v
5. Restart the instruction that caused the page fault
6. Extreme case – start process with no pages in memory
o OS sets instruction pointer to first instruction of process, non-memory-resident -
>page fault
o And for every other process pages on firstaccess
o Pure demand paging
7. Actually, a given instruction could access multiple pages -> multiple pagefaults
o Pain decreased because of locality ofreference
8. Hardware support needed for demand paging
o Page table with valid / invalidbit
o Secondary memory (swap device with swapspace)
o Instruction restart
Page 47
Page Replacement
Prevent over-allocation of memory by modifying page-fault service routine to include
page replacement
Use modify (dirty) bit to reduce overhead of page transfers – only modified pages are
written to disk
Page replacement completes separation between logical memory and physical memory –
large virtual memory can be provided on a smaller physical memory
Need For Page Replacement
Page 48
Page 49
15 page faults
Optimal Algorithm:
Replace page that will not be used for longest period of time
No of page faults: 9
Least Recently Used (LRU) Algorithm:
Use past knowledge rather than future
Replace page that has not been used in the most amount of time
Associate time of last use with eachpage
Page faults:12
Page 50
Second-chance algorithm
o Generally FIFO, plus hardware-provided reference bit
o Clock replacement
o If page to be replaced has
4 Reference bit = 0 -> replace it
4 reference bit = 1 then:
– set reference bit 0, leave page in memory
– replace next page, subject to same rules
Counting Algorithms
Page 51
Keep a counter of the number of references that have been made to each page
l Not common
LFU Algorithm: replaces page with smallestcount
MFU Algorithm: based on the argument that the page with the smallest count was
probably just brought in and has yet to beused
Applications and Page Replacement
All of these algorithms have OS guessing about future pageaccess
Some applications have better knowledge – i.e.databases
Memory intensive applications can cause double buffering
l OS keeps copy of page in memory as I/Obuffer
l Application keeps page in memory for its own work
Operating system can given direct access to the disk, getting out of the way of the
applications
l Raw diskmode
Bypasses buffering, locking, etc
Allocation of Frames
Each process needs minimum number offrames
Example: IBM 370 – 6 pages to handle SS MOVEinstruction:
o instruction is 6 bytes, might span 2 pages
o 2 pages to handlefrom
o 2 pages to handleto
Maximum of course is total frames in thesystem
Two major allocation schemes
o fixed allocation
o priority allocation
Many variations
Fixed Allocation
Equal allocation – For example, if there are 100 frames (after allocating frames for the
OS) and 5 processes, give each process 20frames
o Keep some as free frame buffer pool
Page 52
Page 53
Page 54
Disadvantage -fragmentation
Slab Allocator
Alternate strategy
Slab is one or more physically contiguouspages
Cache consists of one or moreslabs
Single cache for each unique kernel data structure
o Each cache filled with objects – instantiations of the datastructure
o When cache created, filled with objects marked asfree
When structures stored, objects marked asused
If slab is full of used objects, next object allocated from emptyslab
o If no empty slabs, new slab allocated
o Benefits include no fragmentation, fast memory request satisfaction
Page 55
Page 56
o request
Page 57
o use
o release
DeadlockCharacterization
Deadlock can arise if four conditions hold simultaneously.
Mutual exclusion: only one process at a time can use aresource
Hold and wait: a process holding at least one resource is waiting to acquire additional
resources held by other processes
No preemption: a resource can be released only voluntarily by the process holding it,
after that process has completed itstask
Circular wait: there exists a set {P0, P1, …, Pn} of waiting processes such that P0is
waiting for a resource that is held by P1, P1 is waiting for a resource that is heldby
P2, …,Pn–1 is waiting for a resource that is held by Pn, and Pn is waiting for a resource
that is held by P0.
Resource-Allocation Graph
A set of vertices V and a set of edges E.
V is partitioned into two types:
o P = {P1, P2, …, Pn}, the set consisting of all the processes in thesystem
o R = {R1, R2, …, Rm}, the set consisting of all resource types in thesystem
o request edge – directed edge PiRj
assignment edge – directed edgeRjPi
Page 58
Page 59
Mutual Exclusion – not required for sharable resources; must hold fornonsharable
resources
Hold and Wait – must guarantee that whenever a process requests a resource, it does not
hold any other resources
o Require process to request and be allocated all its resources before it begins
execution, or allow process to request resources only when the process hasnone
o Low resource utilization; starvation possible
No Preemption–
o If a process that is holding some resources requests another resource that cannot
be immediately allocated to it, then all resources currently being held arereleased
o Preempted resources are added to the list of resources for which the process is
waiting
o Process will be restarted only when it can regain its old resources, as well as the
new ones that it is requesting
Circular Wait – impose a total ordering of all resource types, and require that each
process requests resources in an increasing order ofenumeration
Deadlock Avoidance
Requires that the system has some additional a priori informationavailable
Simplest and most useful model requires that each process declare the maximumnumber
of resources of each type that it may need
Page 60
o When Piterminates, Pi+1 can obtain its needed resources, and soon
If a system is in safe state nodeadlocks
If a system is in unsafe state possibility ofdeadlock
Avoidance ensure that a system will never enter an unsafe state
Avoidance algorithms
Single instance of a resource type
o Use a resource-allocation graph
Multiple instances of a resource type
o Use the banker’s algorithm
Resource-Allocation Graph Scheme
Claim edgePiRjindicated that process Pjmay request resource Rj; represented by a
dashed line
Claim edge converts to request edge when a process requests aresource
Request edge converted to an assignment edge when the resource is allocated to the
process
When a resource is released by a process, assignment edge reconverts to a claimedge
Resources must be claimed a priori in thesystem
Page 61
Banker’s Algorithm
Multiple instances
Each process must a priori claim maximum use
When a process requests a resource it may have towait
When a process gets all its resources it must return them in a finite amount of time
Let n = number of processes, and m = number of resourcestypes.
Available: Vector of length m. If available [j] = k, there are k instances of resourcetype
Rjavailable
Max: n x m matrix. If Max [i,j] = k, then process Pimay request at most k instances of
resource typeRj
Allocation: n x m matrix. If Allocation[i,j] = k then Piis currently allocated k instances
ofRj
Need: n x m matrix. If Need[i,j] = k, then Pimay need k more instances of Rjto complete
its task
Page 62
SafetyAlgorithm
1. Let Work and Finish be vectors of length m and n, respectively. Initialize:
Work = Available
Finish [i] = false fori = 0, 1, …,n- 1
2. Find an isuch that both:
(a) Finish [i] =false
(b) NeediWork
If no such iexists, go to step 4
3. Work = Work +
AllocationiFinish[i] =true
go to step 2
4.IfFinish [i] == true for all i, then the system is in a safestate
Resource-Request Algorithm for Process Pi
Request = request vector for process Pi. If Requesti[j] = k then process Piwants k instances of
resource typeRj
1. If RequestiNeedigo to step 2. Otherwise, raise error condition, since process has
exceeded its maximum claim
Page 63
The system is in a safe state since the sequence <P1, P3, P4, P2, P0> satisfies safety
criteria
P1 Request (1,0,2)
Check that Request Available (that is, (1,0,2) (3,3,2) true
Executing safety algorithm shows that sequence <P1, P3, P4, P0, P2> satisfies safety
requirement
Page 64
Periodically invoke an algorithm that searches for a cycle in the graph. If there is a cycle,
there exists a deadlock
Page 65
4. If Finish[i] == false, for some i, 1 in, then the system is in deadlock state. Moreover,if
Finish[i] == false, then Piis deadlocked
Page 66
UNIT-4
Secondary-Storage Systems, File-System Interface and Implementation
Overview of Secondary Storage Structure
Magnetic disks provide bulk of secondary storage of moderncomputers
o Drives rotate at 60 to 250 times per second
o Transfer rate is rate at which data flow between drive and computer
o Positioning time (random-access time) is time to move disk arm to desired
cylinder (seek time) and time for desired sector to rotate under the disk head
(rotational latency)
o Head crash results from disk head making contact with the disksurface
4 That’s bad
Disks can be removable
Drive attached to computer via I/O bus
o Busses vary, including EIDE, ATA, SATA, USB, Fibre Channel, SCSI,SAS,
Firewire
o Host controller in computer uses bus to talk to disk controller built into drive or
storage array
Magnetic Disks
Platters range from .85” to 14” (historically)
o Commonly 3.5”, 2.5”, and 1.8”
Range from 30GB to 3TB per drive
Performance
o Transfer Rate – theoretical – 6Gb/sec
o Effective Transfer Rate – real –1Gb/sec
o Seek time from 3ms to 12ms – 9ms common for desktop drives
o Average seek time measured or calculated based on 1/3 of tracks
o Latency based on spindle speed
4 1/(RPM * 60)
o Average latency = ½ latency
Page 67
Magnetic Tape
Was early secondary-storage medium
o Evolved from open spools to cartridges
Relatively permanent and holds large quantities of data
Access time slow
Random access ~1000 times slower thandisk
Mainly used for backup, storage of infrequently-used data, transfer medium
between systems
Kept in spool and wound or rewound past read-write head
Once data under head, transfer rates comparable to disk
o 140MB/sec and greater
200GB to 1.5TB typical storage
Common technologies are LTO-{3,4,5} and T10000
Disk Structure
Disk drives are addressed as large 1-dimensional arrays of logical blocks, where the
logical block is the smallest unit of transfer
The 1-dimensional array of logical blocks is mapped into the sectors of the disk
sequentially
o Sector 0 is the first sector of the first track on the outermostcylinder
o Mapping proceeds in order through that track, then the rest of the tracks in that
cylinder, and then through the rest of the cylinders from outermost toinnermost
o Logical to physical address should be easy
Page 68
Page 69
Network-Attached Storage
Network-attached storage (NAS) is storage made available over a network rather than
over a local connection (such as a bus)
o Remotely attaching to file systems
NFS and CIFS are common protocols
Implemented via remote procedure calls (RPCs) between host and storage over typically
TCP or UDP on IP network
iSCSI protocol uses IP network to carry the SCSIprotocol
o Remotely attaching to devices (blocks)
Disk Scheduling
The operating system is responsible for using hardware efficiently — for the disk drives,
this means having a fast access time and diskbandwidth
Minimize seek time
Seek time seekdistance
Disk bandwidth is the total number of bytes transferred, divided by the total time
between the first request for service and the completion of the lasttransfer
There are many sources of disk I/O request
OS
System processes
Users processes
I/O request includes input or output mode, disk address, memory address, number of
sectors to transfer
OS maintains queue of requests, per disk or device
Idle disk can immediately work on I/O request, busy disk means work mustqueue
Page 70
Note that drive controllers have small buffers and can manage a queue of I/O requests (of
varying “depth”)
Several algorithms exist to schedule the servicing of disk I/Orequests
The analysis is true for one or many platters
We illustrate scheduling algorithms with a request queue (0-199)
98, 183, 37, 122, 14, 124, 65, 67
Head pointer 53
SSTF
Shortest Seek Time First selects the request with the minimum seek time from the current
head position
SSTF scheduling is a form of SJF scheduling; may cause starvation of somerequests
Illustration shows total head movement of 236cylinders
Page 71
SCAN
The disk arm starts at one end of the disk, and moves toward the other end, servicing
requests until it gets to the other end of the disk, where the head movement is reversed
and servicing continues.
SCAN algorithm Sometimes called the elevatoralgorithm
Illustration shows total head movement of 208cylinders
But note that if requests are uniformly dense, largest density at other end of disk and
those wait the longest
C-SCAN
Page 72
C-LOOK
LOOK a version of SCAN, C-LOOK a version ofC-SCAN
Arm only goes as far as the last request in each direction, then reverses direction
immediately, without first going all the way to the end of thedisk
Page 73
Disk Management
Low-level formatting, or physical formatting — Dividing a disk into sectors that the
disk controller can read and write
o Each sector can hold header information, plus data, plus error correction code
(ECC)
o Usually 512 bytes of data but can be selectable
o To use a disk to hold files, the operating system still needs to record its own data
structures on the disk
o Partition the disk into one or more groups of cylinders, each treated as a logical
disk
o Logical formatting or “making a filesystem”
o To increase efficiency most file systems group blocks intoclusters
4DiskI/Odoneinblocks
4FileI/Odoneinclusters
4 Boot block initializes system
o The bootstrap is stored in ROM
o Bootstrap loader program stored in boot blocks of bootpartition
o Methods such as sector sparing used to handle bad blocks
Swap-Space Management
Swap-space — Virtual memory uses disk space as an extension of mainmemory
o Less common now due to memory capacity increases
Swap-space can be carved out of the normal file system, or, more commonly, it can be in
a separate disk partition(raw)
Swap-space management
o 4.3BSD allocates swap space when process starts; holds text segment (the
program) and data segment
o Kernel uses swap maps to track swap-spaceuse
o Solaris 2 allocates swap space only when a dirty page is forced out of physical
memory, not when the virtual memory page is firstcreated
4 File data written to swap space until write to file system requested
4Other dirty pages go to swap space due to no other home
4 Text segment pages thrown out and reread from the file system as needed
Page 74
RAID Structure
RAID – multiple disk drives provides reliability viaredundancy
Increases the mean time tofailure
Frequently combined with NVRAM to improve writeperformance
RAID is arranged into six different levels
Several improvements in disk-use techniques involve the use of multiple disks working
cooperatively
Disk striping uses a group of disks as one storageunit
RAID schemes improve performance and improve the reliability of the storage system by
storing redundant data
n Mirroring or shadowing (RAID 1) keeps duplicate of eachdisk
n Striped mirrors (RAID 1+0) or mirrored stripes (RAID 0+1) provides high
performance and high reliability
n Block interleaved parity (RAID 4, 5, 6) uses much lessredundancy
RAID within a storage array can still fail if the array fails, so automatic replication of
the data between arrays is common
Page 75
File-System Interface
File Concept
Contiguous logical address space
Types:
o Data
o numeric
o character
o binary
o Program
File Structure
None - sequence of words,bytes
Simple record structure
o Lines
o Fixed length
o Variable length
Complex Structures
o Formatted document
o Relocatable load file
Page 76
Can simulate last two with first method by inserting appropriate controlcharacters
Who decides:
o Operating system
o Program
File Attributes
Name – only information kept in human-readableform
Identifier – unique tag (number) identifies file within filesystem
Type – needed for systems that support differenttypes
Location – pointer to file location on device
Size – current filesize
Protection – controls who can do reading, writing,executing
Time, date, and user identification – data for protection, security, and usagemonitoring
Information about files are kept in the directory structure, which is maintained on thedisk
File Operations
File is an abstract datatype
Create
Write
Read
Reposition within file
Delete
Truncate
Open(Fi) – search the directory structure on disk for entry Fi, and move the content of
entry to memory
Close (Fi) – move the content of entry Fiin memory to directory structure ondisk
File Types – Name, Extension
Page 77
Access Methods
Sequential Access
read next
write next
reset
no read after last write
(rewrite)
DirectAccess
readnwr
iten
position to n
read next
write next
rewriten
n = relative block number
Sequential-access File
Page 78
Page 79
Directory Structure
A collection of nodes containing information about all files
Disk can be subdivided intopartitions
Disks or partitions can be RAID protected againstfailure
Disk or partition can be used raw – without a file system, or formatted with a filesystem
Partitions also known as minidisks, slices
Entity containing file system known as avolume
Each volume containing file system also tracks that file system’s info in device directory
or volume table of contents
As well as general-purpose file systems there are many special-purpose file systems,
frequently all within the same operating system orcomputer
Page 80
Two-Level Directory
Separate directory for each user
Page 81
Path name
Can have the same file name for different user
Efficient searching
No grouping capability
Tree-Structured Directories
Efficient searching
Grouping Capability
Current directory (working directory)
o cd /spell/mail/prog
o type list
Page 82
Acyclic-Graph Directories
Page 83
File Sharing
Sharing of files on multi-user systems isdesirable
Sharing may be done through a protectionscheme
On distributed systems, files may be shared across anetwork
Network File System (NFS) is a common distributed file-sharingmethod
File Sharing – Multiple Users
User IDs identify users, allowing permissions and protections to beper-user
Group IDs allow users to be in groups, permitting group accessrights
Page 84
Page 85
File-System Structure
File structure
o Logical storage unit
o Collection of related information
File system resides on secondary storage(disks)
o Provided user interface to storage, mapping logical to physical
o Provides efficient and convenient access to disk by allowing data to be stored,
located retrieved easily
Disk provides in-place rewrite and randomaccess
o I/O transfers performed in blocks of sectors (usually 512bytes)
File control block – storage structure consisting of information about afile
Device driver controls the physicaldevice
File system organized into layers
Layered File System
File-System Implementation
We have system calls at the API level, but how do we implement theirfunctions?
o On-disk and in-memory structures
Boot control block contains info needed by system to boot OS from thatvolume
o Needed if volume contains OS, usually first block ofvolume
Volume control block (superblock, master file table) contains volumedetails
o Total # of blocks, # of free blocks, block size, free block pointers or array
Directory structure organizes the files
Page 86
The following figure illustrates the necessary file system structures provided by the
operating systems
Figure 12-3(a) refers to opening a file
Figure 12-3(b) refers to reading a file
Plus buffers hold data blocks from secondary storage
Open returns a file handle for subsequent use
Data from read eventually copied to specified user process memoryaddress
Page 87
VFS allows the same system call interface (the API) to be used for different types of file
systems
o Separates file-system generic operations from implementation details
o Implementation can be one of many file systems types, or network filesystem
4 Implements vnodes which hold inodes or network file details
o Then dispatches operation to appropriate file system implementationroutines
The API is to the VFS interface, rather than any specific type of filesystem
Page 88
Directory Implementation
Linear list of file names with pointer to the data blocks
o Simple to program
o Time-consuming to execute
4 Linear search time
4 Could keep ordered alphabetically via linked list or use B+ tree
Hash Table – linear list with hash datastructure
o Decreases directory search time
o Collisions – situations where two file names hash to the samelocation
o Only good if entries are fixed size, or use chained-overflow method
Allocation Methods – Contiguous
An allocation method refers to how disk blocks are allocated for files:
Contiguous allocation – each file occupies set of contiguousblocks
o Best performance in most cases
o Simple – only starting location (block #) and length (number of blocks) are
required
o Problems include finding space for file, knowing file size, external fragmentation,
need for compaction off-line (downtime) oron-line
Page 89
Linked
Linked allocation – each file a linked list of blocks
o File ends at nil pointer
o No external fragmentation
o Each block contains pointer to next block
o No compaction, external fragmentation
o Free space management system called when new blockneeded
o Improve efficiency by clustering blocks into groups but increases internal
fragmentation
o Reliability can be a problem
o Locating a block can take many I/Os and diskseeks
FAT (File Allocation Table) variation
o Beginning of volume has table, indexed by block number
o Much like a linked list, but faster on disk and cacheable
o New block allocation simple
Page 90
File-Allocation Table
Indexed
Indexed allocation
o Each file has its own index block(s) of pointers to its datablocks
Free-Space Management
File system maintains free-space list to track availableblocks/clusters
Linked list (free list)
o Cannot get contiguous space easily
o No waste of space
o No need to traverse the entire list (if # free blocksrecorded)
Page 91
Grouping
Modify linked list to store address of next n-1 free blocks in first free block, plus a
pointer to next block that contains free-block-pointers (like thisone).
Counting
Because space is frequently contiguously used and freed, with contiguous-allocation
allocation, extents, or clustering.
Keep address of first free block and count of following freeblocks.
Free space list then has entries containing addresses andcounts.
Page 92
UNIT-5
I/O Systems, Protection, Security
I/O Hardware
Incredible variety of I/O devices
o Storage
o Transmission
o Human-interface
Common concepts – signals from I/O devices interface withcomputer
o Port – connection point fordevice
o Bus - daisy chain or shared directaccess
o Controller (host adapter) – electronics that operate port, bus,device
4 Sometimes integrated
4Sometimes separate circuit board (host adapter)
4 Contains processor, microcode, private memory, bus controller, etc
– Some talk to per-device controller with bus controller, microcode,
memory, etc
A Typical PC Bus Structure
Page 93
Devices usually have registers where device driver places commands, addresses, and data
to write, or read data from registers after command execution
l Data-in register, data-out register, status register, control register
l Typically 1-4 bytes, or FIFObuffer
Page 94
Page 95
Page 96
Page 97
Network Devices
Varying enough from block and character to have owninterface
Unix and Windows NT/9x/2000 include socketinterface
o Separates network protocol from networkoperation
o Includes select()functionality
Approaches vary widely (pipes, FIFOs, streams, queues,mailboxes)
Clocks and Timers
Provide current time, elapsed time,timer
Normal resolution about 1/60second
Page 98
Page 99
Page 100
Error Handling
OS can recover from disk read, device unavailable, transient writefailures
o Retry a read or write, forexample
o Some systems more advanced – Solaris FMA,AIX
4Trackerrorfrequencies,stopusingdevicewithincreasingfrequencyof
retry-ableerrors
Most return an error number or code when I/O requestfails
System error logs hold problemreports
I/O Protection
User process may accidentally or purposefully attempt to disrupt normal operation via
illegal I/O instructions
o All I/O instructions defined to beprivileged
o I/O must be performed via systemcalls
Memory-mapped and I/O port memory locations must be protected
Use of a System Call to Perform I/O
Page 101
Page 102
Page 103
Protection
Goals of Protection:
In one protection model, computer consists of a collection of objects, hardware or
software
Each object has a unique name and can be accessed through a well-defined set of
operations
Protection problem - ensure that each object is accessed correctly and only by those
processes that are allowed to doso
Principles of Protection
Guiding principle – principle of least privilege
o Programs, users and systems should be given just enough privileges to perform
their tasks
o Limits damage if entity has a bug, gets abused
o Can be static (during life of system, during life ofprocess)
o Or dynamic (changed by process as needed) – domain switching, privilege
escalation
o “Need to know” a similar concept regarding access todata
Must consider “grain”aspect
o Rough-grained privilege management easier, simpler, but least privilege now
done in large chunks
o For example, traditional Unix processes either have abilities of the
associated user, or of root
o Fine-grained management more complex, more overhead, but moreprotective
o File ACL lists,RBAC
o Domain can be user, process,procedure
Domain Structure
Access-right = <object-name,rights-set>
where rights-set is a subset of all valid operations that can be performed on the object
Domain = set ofaccess-rights
Page 104
Page 105
Access Matrix
View protection as a matrix (accessmatrix)
Rows represent domains
Columns represent objects
Access(i, j) is the set of operations that a process executing in Domaini can invoke on
Objectj
Page 106
4Ifensuresthatthematrixisonlymanipulatedbyauthorizedagentsand that
rules are strictlyenforced
oPolicy
4 User dictates policy
4 Who can access what object and in what mode
4 But doesn’t solve the general confinement problem
Access Matrix of Figure A with Domains as Objects
Page 107
Page 108
o Resulting per-object list consists of ordered pairs <domain, rights-set > defining
all domains with non-empty set of access rights for theobject
o Easily extended to contain default set -> If M ∈ default set, also allowaccess
Each column = Access-control list for one object
Defines who can perform whatoperation
Option 4 – Lock-key
o Compromise between access lists and capability lists
o Each object has list of unique bit patterns, calledlocks
Page 109
Access Control
Protection can be applied to non-fileresources
Solaris 10 provides role-based access control (RBAC) to implement leastprivilege
o Privilege is right to execute system call or use an option within a systemcall
o Can be assigned to processes
o Users assigned roles granting access to privileges andprograms
4 Enable role via password to gain its privileges
o Similar to accessmatrix
Page 110
o Partial vs.total
o Temporary vs. permanent
Access List – Delete access rights from accesslist
o Simple – search access list and removeentry
o Immediate, general or selective, total or partial, permanent ortemporary
Capability List – Scheme required to locate capability in the system before capability
can be revoked
o Reacquisition – periodic delete, with require and denial ifrevoked
o Back-pointers – set of pointers from each object to all capabilities of that object
(Multics)
o Indirection – capability points to global table entry which points to object – delete
entry from global table, not selective(CAL)
o Keys – unique bits associated with capability, generated when capabilitycreated
4 Master key associated with object, key matches master key for access
4 Revocation – create new master key
4Policydecisionofwhocancreateandmodifykeys–objectowneror
others?
Capability-Based Systems
Hydra
o Fixed set of access rights known to and interpreted by thesystem
4 i.e. read, write, or execute each memory segment
4Usercandeclareotherauxiliaryrightsandregisterthosewithprotection
system
4 Accessing process must hold capability and know name of operation
4Rightsamplificationallowedbytrustworthyproceduresforaspecific
type
o Interpretation of user-defined rights performed solely by user's program;system
provides access protection for use of these rights
o Operations on objects defined procedurally – procedures are objectsaccessed
indirectly by capabilities
o Solves the problem of mutually suspicioussubsystems
o 1111cIncludes library of prewritten securityroutines
Page 110
Cambridge CAPSystem
o Simpler butpowerful
o Data capability - provides standard read, write, execute of individualstorage
segments associated with object – implemented inmicrocode
o Software capability -interpretation left to the subsystem, through itsprotected
procedures
4 Only has access to its own subsystem
4 Programmers must learn principles and techniques of protection
Language-Based Protection
Specification of protection in a programming language allows the high-leveldescription
of policies for the allocation and use of resources
Security
The Security Problem:
System secure if resources used and accessed as intended under all circumstances
l Unachievable
Intruders (crackers) attempt to breachsecurity
Threat is potential securityviolation
Attack is attempt to breachsecurity
Attack can be accidental ormalicious
Easier to protect against accidental than malicious misuse
Security Violation Categories
Breach of confidentiality
o Unauthorized reading ofdata
Breach of integrity
o Unauthorized modification ofdata
Breach of availability
o Unauthorized destruction ofdata
Page 111
Theft of service
o Unauthorized use ofresources
Denial of service (DOS)
o Prevention of legitimateuse
Security Violation Methods
Masquerading (breachauthentication)
o Pretending to be an authorized user to escalateprivileges
Replay attack
o As is or with messagemodification
Man-in-the-middle attack
o Intruder sits in data flow, masquerading as sender to receiver and viceversa
Session hijacking
o Intercept an already-established session to bypassauthentication
Page 112
Trojan Horse
o Code segment that misuses itsenvironment
o Exploits mechanisms for allowing programs written by users to be executed by
other users
o Spyware, pop-up browser windows, covert channels
o Up to 80% of spam delivered by spyware-infectedsystems
Trap Door
o Specific user identifier or password that circumvents normal securityprocedures
o Could be included in a compiler
Logic Bomb
o Program that initiates a security incident under certaincircumstances
Stack and BufferOverflow
o Exploits a bug in a program (overflow either the stack or memorybuffers)
o Failure to check bounds on inputs,arguments
o Write past arguments on the stack into the return address onstack
o When routine returns from call, returns to hacked address
n Pointed to code loaded onto stack that executes maliciouscode
Page 113
Page 114
Page 115
Port scanning
o Automated attempt to connect to a range of ports on one or a range of IP
addresses
o Detection of answering serviceprotocol
o Detection of OS and version running onsystem
o nmap scans all ports in a given IP range for aresponse
o nessus has a database of protocols and bugs (and exploits) to apply against a
system
o Frequently launched from zombiesystems
4 To decrease trace-ability
Page 116
Denial of Service
o Overload the targeted computer preventing it from doing any usefulwork
o Distributed denial-of-service (DDOS) come from multiple sites atonce
o Consider the start of the IP-connection handshake(SYN)
4 How many started-connections can the OS handle?
o Consider traffic to a website
o Accidental – CS students writing bad fork()code
o Purposeful – extortion,punishment
Cryptography as a Security Tool
Broadest security toolavailable
o Internal to a given computer, source and destination of messages can be known
and protected
4 OS creates, manages, protects process IDs, communication ports
o Source and destination of messages on network cannot be trusted without
cryptography
4 Local network – IP address?
– Consider unauthorized host added
4 WAN / Internet – how to establish authenticity
– Not via IPaddress
Cryptography
Means to constrain potential senders (sources) and / or receivers (destinations)of
messages
o Based on secrets(keys)
o Enables
4 Confirmation of source
4 Receipt only by certain destination
4Trust relationship between sender and receiver
Page 117
Encryption
Encryption algorithm consistsof
o Set K ofkeys
o Set M ofMessages
o Set C of ciphertexts (encryptedmessages)
o A function E : K → (M→C). That is, for each k K, E(k) is a function for
generating ciphertexts from messages
4 Both E and E(k) for any k should be efficiently computable functions
o A function D : K → (C → M). That is, for each k K, D(k) is a function for
generating messages from ciphertexts
4 Both D and D(k) for any k should be efficiently computable functions
An encryption algorithm must provide this essential property: Given a ciphertext c C,
acomputer can compute m such that E(k)(m) = c only if it possessesD(k)
Page 118
o Thus, a computer holding D(k) can decrypt ciphertexts to the plaintexts used to
produce them, but a computer not holding D(k) cannot decryptciphertexts
o Since ciphertexts are generally exposed (for example, sent on the network), it is
important that it be infeasible to derive D(k) from theciphertexts
Symmetric Encryption
Same key used to encrypt anddecrypt
o E(k) can be derived from D(k), and viceversa
o DES is most commonly used symmetric block-encryption algorithm (createdby
USGovt)
o Encrypts a block of data at a time
o Triple-DES considered moresecure
Advanced Encryption Standard (AES), twofishup andcoming
RC4 is most common symmetric stream cipher, but known to havevulnerabilities
o Encrypts/decrypts a stream of bytes (i.e., wirelesstransmission)
o Key is a input to psuedo-random-bitgenerator
4Generatesaninfinitekeystream
AsymmetricEncryption
Public-key encryption based on each user having twokeys:
o public key – published key used to encryptdata
o private key – key known only to individual user used to decryptdata
Must be an encryption scheme that can be made public without making it easy tofigure
out the decryptionscheme
o Most common is RSA blockcipher
o Efficient algorithm for testing whether or not a number isprime
o No efficient algorithm is know for finding the prime factors of anumber
Formally, it is computationally infeasible to derive D(kd , N) from E(ke , N), and so E(ke,
N) need not be kept secret and can be widelydisseminated
o E(ke , N) (or just ke) is the publickey
o D(kd , N) (or just kd) is the privatekey
o N is the product of two large, randomly chosen prime numbers p and q (for
example, p and q are 512 bits each)
Page 119
o EncryptionalgorithmisE(ke,N)(m)=mkemodN,wherekesatisfieskekdmod (p−1)(q
−1) =1
o The decryption algorithm is then D(kd , N)(c) = ckd mod N
Asymmetric Encryption Example
For example. make p = 7and q =13
We then calculate N = 7∗13 = 91 and (p−1)(q−1) =72
We next select kerelatively prime to 72 and<72, yielding 5
Finally,we calculate kdsuch that kekdmod 72 = 1, yielding29
We how have our keys
o Public key, ke, N = 5,91
o Private key, kd , N = 29,91
o Encrypting the message 69 with the public key results in the cyphertext62
Cyphertext can be decoded with the privatekey
o Public key can be distributed in cleartext to anyone who wants to communicate
with holder of publickey
Authentication
Constraining set of potential senders of amessage
o Complementary and sometimes redundant toencryption
o Also can prove messageunmodified
Algorithmcomponents
o A set K ofkeys
o A set M ofmessages
o A set A ofauthenticators
o A function S :K → (M→A)
4Thatis,foreachkK,S(k)isafunctionforgeneratingauthenticators
frommessages
4 Both S and S(k) for any k should be efficiently computable functions
o AfunctionV:K→(M×A→{true,false}).Thatis,foreachkK,V(k)isafunction
for verifying authenticators onmessages
4 Both V and V(k) for any k should be efficiently computable functions
Page 120
Computer not holding S(k) cannot generate authenticators on messages that can be
verified usingV(k)
Since authenticators are generally exposed (for example, they are sent on the network
with the messages themselves), it must not be feasible to derive S(k) from the
authenticators
Authentication – Hash Functions
Basis of authentication
Creates small, fixed-size block of data (message digest, hash value) fromm
Hash Function H must be collision resistant onm
o Must be infeasible to find an m’ ≠ m such that H(m) =H(m’)
o If H(m) = H(m’), then m =m’
o The message has not been modified
o Common message-digest functions include MD5, which produces a 128-bithash,
and SHA-1, which outputs a 160-bithash
Authentication – MAC
Symmetric encryption used in message-authentication code (MAC) authentication
algorithm
Simpleexample:
o MAC defines S(k)(m) = f (k,H(m))
4Where f is a function that is one-way on its first argument
– k cannot be derived from f (k,H(m))
4Becauseofthecollisionresistanceinthehashfunction,reasonably
assurednoothermessagecouldcreatethesameMAC
4 A suitable verification algorithm is V(k)(m, a) ≡ ( f (k,m) = a)
4NotethatkisneededtocomputebothS(k)andV(k),soanyoneableto
compute one can compute theother
Authentication – Digital Signature
Based on asymmetric keys and digital signaturealgorithm
Page 121
User identity most often established through passwords, can be considered a special case
of either keys or capabilities
Passwords must be keptsecret
o Frequent change ofpasswords
o History to avoidrepeats
Page 122
o Use of “non-guessable”passwords
o Log all invalid access attempts (but not the passwords themselves)
o Unauthorizedtransfer
Passwords may also either be encrypted or allowed to be used onlyonce
o Does encrypting passwords solve the exposureproblem?
4 Might solve sniffing
4 Consider shoulder surfing
4 Consider Trojan horse keystroke logger
Passwords
Encrypt to avoid having to keepsecret
o But keep secret anyway (i.e. Unix uses superuser-only readably file/etc/shadow)
o Use algorithm easy to compute but difficult toinvert
o Only encrypted password stored, neverdecrypted
o Add “salt” to avoid the same password being encrypted to the samevalue
One-timepasswords
o Use a function based on a seed to compute a password, both user andcomputer
o Hardware device / calculator / key fob to generate thepassword
4 Changes very frequently
Biometrics
o Some physical attribute (fingerprint, handscan)
o Multi-factorauthentication
o Need two or more factors forauthentication
4i.e.USB“dongle”,biometricmeasure,andpassword
Implementing Security Defenses
Page 123
Page 124
Page 125