System Programming in Linux
System Programming in Linux
PROGRAMMING
IN
LINUX
UTTAM K. ROY
Topics to be discussed
Advanced Java Programming
SYSTEM
Linux Internals
General Introduction to Operating Systems
Process management
Scheduling
Memory Management
PROGRAMMING
File Management
IN LINUX
Input/Output
System Files Systems management
Managing File Systems
Process Control
Network Management
Signals
Interprocess Communication
Network programming
| U. K. Roy | [ ]
System Programming in Linux 3
SYSTEM
Wesley publication
PROGRAMMING
• Richard Stones & Neil Matthew, “Beginning Linux
Programming”, A Wrox publication
IN LINUX
• Olaf Kirch, Terry Dawson, “Linux Network Administrator's
Guide”, A O’relly publication
Computer-system operation
SYSTEM
One or more CPUs, device controllers connect through
common bus providing access to shared memory
Concurrent execution of CPUs and devices competing for
memory cycles
PROGRAMMING
IN LINUX
| U. K. Roy | [ ]
System Programming in Linux 6
SYSTEM
PROGRAMMING
IN LINUX
| U. K. Roy | [ ]
System Programming in Linux 7
Components (Contd.)
Advanced Java Programming
SYSTEM
PROGRAMMING
A computer system consists of
IN LINUX
hardware
system programs
application programs
| U. K. Roy | [ ]
System Programming in Linux 8
Components (Contd.)
Advanced Java Programming
SYSTEM
Computer system components
Hardware – provides basic computing resources
• CPU, memory, I/O devices
Operating system
• Controls and coordinates use of hardware among various
PROGRAMMING
applications and users
IN LINUX
systems, video games
Users
• People, machines, other computers
| U. K. Roy | [ ]
System Programming in Linux 9
SYSTEM
A program that acts as an intermediary
between a user of a computer and the
computer hardware.
PROGRAMMING
Operating system goals:
Execute user programs and make solving user problems
easier.
Make the computer system convenient to use.
Puspose: IN LINUX
Use the computer hardware in an efficient manner.
| U. K. Roy | [ ]
System Programming in Linux 10
Process Management
The creation and deletion of both user and system process
The
The
The
SYSTEM
suspension and resumption of process
provision of mechanisms for process synchronization
provision of mechanism for deadlock handling
PROGRAMMING
Keep track of which parts of memory are being used and by
whom
Decide which processes are to be loaded into the memory when
memory space becomes available
Allocates and deallocates memory as and when needed
IN LINUX
File Management
Creation and deletion of files
Creation and deletion of directories
The support of primitives for manipulating files and directories
| U. K. Roy | [ ]
System Programming in Linux 11
SYSTEM
and spooling
A general device driver interface
Drivers for specific hardware interface
PROGRAMMING
Free space management
Storage allocation
Disk Scheduling
Networking
IN LINUX
Communication task
Process Management
Advanced Java Programming
PROGRAMMING
Providing mechanisms for process synchronization
Providing mechanisms for process communication
Providing mechanisms for deadlock handling
IN LINUX
| U. K. Roy | [ ]
System Programming in Linux 14
Process—Basic Concept
Advanced Java Programming
SYSTEM
A process is a program in execution. It is a unit of
work within the system. Program is a passive entity,
process is an active entity.
PROGRAMMING
CPU, memory, I/O, files
Initialization data
IN LINUX
| U. K. Roy | [ ]
System Programming in Linux 15
Process—Basic Concept
Advanced Java Programming
SYSTEM
specifying location of next instruction to execute
Process executes instructions sequentially, one at a
time, until completion
PROGRAMMING
thread
IN LINUX
Concurrency by multiplexing the CPUs among the
processes / threads
| U. K. Roy | [ ]
System Programming in Linux 16
SYSTEM
PROGRAMMING
Possible process states
IN LINUX
running—Instructions are being executed
Blocked—Process is waiting for some event to occur
Ready—The process in waiting to be assigned to a processor
Transitions between states shown
| U. K. Roy | [ ]
System Programming in Linux 17
SYSTEM
PROGRAMMING
IN LINUX
Fields of a Process Control Block entry
| U. K. Roy | [ ]
System Programming in Linux 18
Scheduling—Basic Concept
Advanced Java Programming
SYSTEM
PROGRAMMING
IN LINUX
Three level scheduling
| U. K. Roy | [ ]
System Programming in Linux 19
Process Scheduling
Advanced Java Programming
SYSTEM
Single user cannot keep CPU and I/O devices busy at all
times
Multiprogramming organizes jobs (code and data) so CPU
always has one to execute
PROGRAMMING
A subset of total jobs in system is kept in memory
One job selected and run via job scheduling
When it has to wait (for I/O for example), OS switches to
another job
IN LINUX
Context Switch
| U. K. Roy | [ ]
System Programming in Linux 20
Process Scheduling
Advanced Java Programming
SYSTEM
Timesharing (multitasking) is logical extension in
which CPU switches jobs so frequently that users can
interact with each job while it is running, creating
interactive computing
Response time should be small enough
PROGRAMMING
Each user has at least one program executing in memory
process
If several jobs ready to run at the same time CPU
scheduling
If processes don’t fit in memory, swapping moves them
IN LINUX
in and out to run
Virtual memory allows execution of processes not
completely in memory
| U. K. Roy | [ ]
System Programming in Linux 21
CPU Scheduling
Advanced Java Programming
Basic Concept
SYSTEM
Maximum CPU utilization obtained with
multiprogramming
PROGRAMMING
CPU burst distribution
IN LINUX
| U. K. Roy | [ ]
System Programming in Linux 22
SYSTEM
PROGRAMMING
IN LINUX
Bursts of CPU usage alternate with periods of I/O wait
a CPU-bound process
an I/O bound process
| U. K. Roy | [ ]
System Programming in Linux 23
SYSTEM
PROGRAMMING
IN LINUX
| U. K. Roy | [ ]
System Programming in Linux 24
SYSTEM
Selects from among the processes in memory that are ready to
execute, and allocates the CPU to one of them
PROGRAMMING
2. Switches from running to ready state
3. Switches from waiting to ready
4. Terminates
IN LINUX
All other scheduling is preemptive
| U. K. Roy | [ ]
System Programming in Linux 25
SYSTEM
Dispatcher module gives control of the CPU to
the process selected by the short-term
scheduler; this involves:
switching context
switching to user mode
PROGRAMMING
jumping to the proper location in the user
program to restart that program
IN LINUX
dispatcher to stop one process and start
another running
| U. K. Roy | [ ]
System Programming in Linux 26
Scheduling Criteria
Advanced Java Programming
CPU utilization
keep the CPU as busy as possible
Throughput SYSTEM
# of processes that complete their execution per time unit
Turnaround time
PROGRAMMING
amount of time to execute a particular process
Waiting time
amount of time a process has been waiting in the ready queue
SYSTEM
PROGRAMMING
IN LINUX
| U. K. Roy | [ ]
System Programming in Linux 28
SYSTEM
Process Burst Time
P1 24
P2 3
P3 3
Suppose that the processes arrive in the order: P1 , P2 , P3
The Gantt Chart for the schedule is:
PROGRAMMING P1 P2 P3
Waiting
0 time for P1 = 0; P2 = 24;24P3 = 2727 30
IN LINUX
Average waiting time: (0 + 24 + 27)/3 = 17
| U. K. Roy | [ ]
System Programming in Linux 29
SYSTEM
Suppose that the processes arrive in the order
P2 , P3 , P1
The Gantt chart for the schedule is:
PROGRAMMING
P2 P3 P1
Waiting0time for3P1 = 6; P
6 =0 P =3
2 ; 3
30
Average waiting time: (6 + 0 + 3)/3 = 3
Much better than previous case
IN LINUX
Convoy effect short process behind long process
| U. K. Roy | [ ]
System Programming in Linux 30
Two schemes:
PROGRAMMING
nonpreemptive – once CPU given to the process it cannot be
preempted until completes its CPU burst
preemptive – if a new process arrives with CPU burst length
less than remaining time of current executing process,
preempt. This scheme is know as the
IN LINUX
Shortest-Remaining-Time-First (SRTF)
Process
P1
SYSTEM
Arrival Time
0.0
Burst Time
7
P2 2.0 4
P3 4.0 1
PROGRAMMING
P4 5.0 4
SJF (non-preemptive)
P1 P3 P2 P4
IN LINUX
0 3 7 8 12 16
Average waiting time = (0 + 6 + 3 + 7)/4 = 4
| U. K. Roy | [ ]
System Programming in Linux 32
Process
P1
SYSTEMArrival Time
0.0
Burst Time
7
P2 2.0 4
P3 4.0 1
PROGRAMMING
P4 5.0 4
SJF (preemptive)
P1 P2 P3 P2 P4 P1
IN LINUX
0 2 4 5 7 11 16
Average waiting time = (9 + 1 + 0 +2)/4 = 3
| U. K. Roy | [ ]
Determining Length of Next CPU
System Programming in Linux 33
SYSTEM
Can be done by using the length of previous CPU
bursts, using exponential averaging
PROGRAMMING
2. τ n +1 = predicted value for the next CPU burst
3. α , 0 ≤ α ≤ 1
4. Define : τ n +1 = α t n + (1 − α )τ n .
IN LINUX
| U. K. Roy | [ ]
Prediction of System the Programming
Length of the Next CPU
in Linux 34
SYSTEM
PROGRAMMING
IN LINUX
| U. K. Roy | [ ]
System Programming in Linux 35
α =0
τn+1 = τn
α =1
SYSTEM
Recent history does not count
τn+1 = α tn
PROGRAMMING
Only the actual last CPU burst counts
IN LINUX
+(1 - α )n +1 τ0
Priority Scheduling
Advanced Java Programming
SYSTEM
The CPU is allocated to the process with the highest priority
(smallest integer ≡ highest priority)
Preemptive
Nonpreemptive
PROGRAMMING
SJF is a priority scheduling where priority is the predicted next
CPU burst time
IN LINUX
process
| U. K. Roy | [ ]
System Programming in Linux 37
SYSTEM
Each process gets a small unit of CPU time (time
quantum), usually 10-100 milliseconds. After this time
has elapsed, the process is preempted and added to the
end of the ready queue.
PROGRAMMING
time quantum is q, then each process gets 1/n of the
CPU time in chunks of at most q time units at once. No
process waits more than (n-1)q time units.
Performance
IN LINUX
q large ⇒ FIFO
q small ⇒ q must be large with respect to context switch,
otherwise overhead is too high
| U. K. Roy | [ ]
Example of RR with Time Quantum =
System Programming in Linux 38
SYSTEM
Process Burst Time
P1 53
P2 17
P3 68
P4 24
PROGRAMMING
The Gantt chart is:
P1 P2 P3 P4 P1 P3 P4 P1 P3 P3
better response
IN LINUX
Typically, higher average turnaround than SJF, but
| U. K. Roy | [ ]
System Programming in Linux 39
Time Quantum
Advanced Java Programming and Context Switch Time
SYSTEM
PROGRAMMING
IN LINUX
| U. K. Roy | [ ]
System Programming in Linux 40
SYSTEM
PROGRAMMING
IN LINUX
| U. K. Roy | [ ]
System Programming in Linux 41
Multilevel Queue
Advanced Java Programming
SYSTEM
foreground (interactive)
background
PROGRAMMING
foreground – RR
background – FCFS
IN LINUX
Fixed priority scheduling
• serve all from foreground then from background—Possibility of
starvation.
Time slice
• each queue gets a certain amount of CPU time which it can schedule
amongst its processes; e.g., 80% to foreground in RR and 20% to
background in FCFS
| U. K. Roy | [ ]
System Programming in Linux 42
SYSTEM
PROGRAMMING
IN LINUX
| U. K. Roy | [ ]
System Programming in Linux 43
SYSTEM
A process can move between the various
queues
aging can be implemented this way
Multilevel-feedback-queue scheduler
PROGRAMMING
defined by the following parameters:
number of queues
scheduling algorithms for each queue
method used to determine when to upgrade a process
IN LINUX
method used to determine when to demote a process
method used to determine which queue a process will
enter when that process needs service
| U. K. Roy | [ ]
Example of Multilevel Feedback
System Programming in Linux 44
SYSTEM
Q0 – RR with time quantum 8 milliseconds
Q1 – RR time quantum 16 milliseconds
Q2 – FCFS
PROGRAMMING
Scheduling
A new job enters queue Q0 which is served RR. When it gains
CPU, job receives 8 milliseconds. If it does not finish in 8
IN LINUX
milliseconds, job is moved to queue Q1.
Multiple-Processor Scheduling
Advanced Java Programming
PROGRAMMING
Load sharing
Use separate queue for each processor
• One could be idle while another was very busy
Use Common ready queue
IN LINUX
• Each processor picks a process—may result inconsistency
• Use one processor as scheduler—creates master-slave relationship
Asymmetric multiprocessing
only one processor accesses the system data structures
Others executes user processes
| U. K. Roy | [ ]
System Programming in Linux 46
Real-Time Scheduling
Advanced Java Programming
SYSTEM
time
PROGRAMMING
Schedulable real-time system
Given
m events (processes)
process i occurs within period Pi and requires Ci seconds
IN LINUX
Then the load can only be handled if
m
C
∑ ≤1 i =1 Pi
i
| U. K. Roy | [ ]
System Programming in Linux 47
Algorithm Evaluation
Advanced Java Programming
SYSTEM
response time is 1 second
Maximum throughput such that average turnaround time
linearly proportional to execution time
PROGRAMMING
Deterministic Modeling
Evaluates using a predetermined work load
• can not cope with the dynamic behavior of the system
IN LINUX
Queueing Models
Uses mathematical analysis
• Complicated—hence used unrealistic assumtions
• Uses some assumptions—that may not occur
• An approximation of real system and accuracy is questionable
| U. K. Roy | [ ]
PROCESS
SYNCHRONIZATION
System Programming in Linux 49
Process Synchronization
Advanced Java Programming
Background
PROGRAMMING
Solutions of Classic Problems of Synchronization
IN LINUX
| U. K. Roy | [ ]
System Programming in Linux 50
SYSTEM
Implementation uses shared logical address space or shared
data using files
PROGRAMMING
Maintaining data consistency requires mechanisms to
ensure the orderly execution of cooperating processes
IN LINUX
Suppose that we wanted to provide a solution to the
consumer-producer problem
| U. K. Roy | [ ]
System Programming in Linux 51
Producer-Consumer Problem
Advanced Java Programming
while (true) {
…
SYSTEM while (true) {
…
/* produce an item to nextp */ while (count == 0)
while (count == BUFFER_SIZE) ; // buffer is empty, do nothing
PROGRAMMING
; // buffer is full, do nothing
buffer [in] = nextp;
in = (in + 1) % BUFFER_SIZE;
nextc = buffer[out];
out = (out + 1) % BUFFER_SIZE;
count--;
count++; /* consume the item from nextc
… …
IN LINUX
} }
| U. K. Roy | [ ]
System Programming in Linux 52
SYSTEM
register1 = register1 + 1
count = register1
PROGRAMMING
register2 = register2 - 1
count = register2
IN LINUX
S2: consumer execute register2 = count {register2 = 5}
S3: consumer execute register2 = register2 - 1 {register2 = 4}
S4: producer execute count = register1 {count = 6 }
S5: consumer execute count = register2 {count = 4}
Race condition
Outcome depends upon the order of execution
| U. K. Roy | [ ]
System Programming in Linux 53
Critical Section
segment of code where shared variables are used
Mutual Exclusion
SYSTEM
Solution must satisfy the following criteria
PROGRAMMING
Progress
If no process is executing in its critical section and there exist some
processes that wish to enter their critical section, then the selection of the
processes that will enter the critical section next cannot be postponed
indefinitely
IN LINUX
Bounded Waiting
A bound must exist on the number of times that other processes are
allowed to enter their critical sections after a process has made a request to
enter its critical section and before that request is granted
Solution 1
Assume that the LOAD and STORE instructions are atomic; that is,
SYSTEM
cannot be interrupted.
The two processes share a variable:
int turn;
The variable turn indicates whose turn it is to enter the critical
section.
while (true) {
PROGRAMMING …
while (turn != i) ;
<CS>
IN LINUX
turn = j;
REMAINDER SECTION
}
Structure of process i
| U. K. Roy | [ ]
System Programming in Linux 55
Solution 2
The two processes share a variable:
SYSTEM
Boolean flag[2]
The flag array is used to indicate if a process is ready to enter the
critical section. flag[i] = true implies that process Pi is ready!
while (true) {
…
PROGRAMMING flag[i]=true;
while (flag[j]) ;
<CS>
IN LINUX
flag[i]=false;
REMAINDER SECTION
}
Structure of process i
| U. K. Roy | [ ]
System Programming in Linux 56
Solution 3
SYSTEM
while (true) {
…
flag[i]=true;
turn=j
while (flag[j] && turn==j) ;
PROGRAMMING <CS>
flag[i]=false;
}
IN LINUX
REMAINDER SECTION
Structure of process i
| U. K. Roy | [ ]
System Programming in Linux 57
Synchronization Hardware
Advanced Java Programming
SYSTEM
section code
PROGRAMMING
• Operating systems using this not broadly scalable
IN LINUX
Either test memory word and set value
Or swap contents of two memory words
| U. K. Roy | [ ]
System Programming in Linux 58
TestAndSet Instruction
Advanced Java Programming
Definition:
SYSTEM
bool TestAndSet (bool &target) {
bool temp = target;
target = true;
return temp:
}
PROGRAMMING
Solution using TestAndSet
Shared boolean variable lock., initialized to false.
while (true) {
while ( TestAndSet (lock ))
IN LINUX
; /* do nothing
<CS>
lock = FALSE;
// remainder section
}
| U. K. Roy | [ ]
System Programming in Linux 59
Swap Instruction
Advanced Java Programming
Definition:
SYSTEM
void Swap (bool &a, bool &b) {
bool temp = a;
a = b;
b = temp:
}
Solution
PROGRAMMING
Shared Boolean variable lock initialized to false; Each
process has a local Boolean variable key.
Solution:
while (true) {
key = true;
IN LINUX
while ( key == true)
Swap (lock, key );
// critical section
lock = false;
// remainder section
}
| U. K. Roy | [ ]
System Programming in Linux 60
SYSTEM
Two standard operations modify S: wait() and signal()
Originally called P() and V()
Less complicated
Can only be accessed via two indivisible (atomic) operations
PROGRAMMING
wait (S) {
while S <= 0
; // no-op
S--;
IN LINUX
}
signal (S) {
S++;
}
| U. K. Roy | [ ]
System Programming in Linux 61
SYSTEM
Binary semaphore – integer value can range only between 0 and
1; can be simpler to implement
Also known as mutex, locks
PROGRAMMING
Provides mutual exclusion
Semaphore S; // initialized to 1
wait (S);
Critical Section
IN LINUX
signal (S);
| U. K. Roy | [ ]
System Programming in Linux 62
Requirement:
SYSTEM
S2 be executed only after S1
Solution:
PROGRAMMING
Use a a semaphore variable sync initialized to 0
… …
S1 wait(sync)
IN LINUX
Signal(sync) S2
… …
Process P1 Process P2
| U. K. Roy | [ ]
System Programming in Linux 63
Semaphore Implementation
Advanced Java Programming
SYSTEM
Thus, implementation becomes the critical section problem
where the wait and signal code are placed in the crtical
section.
Could now have busy waiting in critical section implementation
• But implementation code is short
PROGRAMMING
• Little busy waiting if critical section rarely occupied
IN LINUX
| U. K. Roy | [ ]
System Programming in Linux 64
PROGRAMMING
Two operations:
IN LINUX
and place it in the ready queue.
| U. K. Roy | [ ]
System Programming in Linux 65
Implementation of wait:
wait (S){
value--; SYSTEM
if (value < 0) {
add this process to waiting queue
block();
PROGRAMMING
}
}
Implementation of signal:
IN LINUX
Signal (S){
value++;
if (value <= 0) {
remove a process P from the waiting queue
wakeup(P);
}
| U. K. Roy}| [ ]
System Programming in Linux 66
SYSTEM
Let S and Q be two semaphores initialized to 1
P0 P1
IN LINUX
signal (S); signal (Q);
signal (Q); signal (S);
SYSTEM
Bounded-Buffer Problem
Readers and Writers Problem
Dining-Philosophers Problem
PROGRAMMING
IN LINUX
| U. K. Roy | [ ]
System Programming in Linux 68
Bounded-Buffer Problem
Advanced Java Programming
SYSTEM
N buffers, each can hold one item
Semaphore mutex initialized to the value 1
Semaphore full initialized to the value 0
Semaphore empty initialized to the value N.
PROGRAMMING
IN LINUX
| U. K. Roy | [ ]
System Programming in Linux 69
The structure of the producer process The structure of the consumer process
while (true) {
//
SYSTEM
produce an item
while (true) {
wait (full);
wait (mutex);
PROGRAMMING
wait (empty); // remove an item from buffer
wait (mutex);
signal (mutex);
signal (empty);
// add the item to the buffer
IN LINUX
signal (mutex);
signal (full); }
}
| U. K. Roy | [ ]
System Programming in Linux 70
Readers-Writers Problem
Advanced Java Programming
SYSTEM
Readers – only read the data set; they do not perform any updates
Writers – can both read and write.
PROGRAMMING
the same time.
Shared Data
Data set
IN LINUX
Semaphore mutex initialized to 1.
Semaphore wrt initialized to 1.
Integer readcount initialized to 0.
| U. K. Roy | [ ]
System Programming in Linux 71
SYSTEM
The structure of a writer process The structure of a reader process
while (true) {
while (true) {
wait (mutex) ;
wait (wrt) ; readcount ++ ;
if (readercount == 1) wait (wrt) ;
PROGRAMMING
// writing is performed signal (mutex)
// reading is performed
signal (wrt) ;
} wait (mutex) ;
readcount - - ;
IN LINUX
if (redacount == 0) signal (wrt) ;
signal (mutex) ;
}
| U. K. Roy | [ ]
System Programming in Linux 72
Dining-Philosophers Problem
Advanced Java Programming
SYSTEM
PROGRAMMING
• Shared data
IN LINUX
Bowl of rice (data set)
Semaphore chopstick [5] initialized to 1
| U. K. Roy | [ ]
System Programming in Linux 73
SYSTEM
While (true) {
wait ( chopstick[i] );
wait ( chopStick[ (i + 1) % 5] );
PROGRAMMING // eat
signal (chopstick[ (i + 1) % 5] );
signal ( chopstick[i] );
} IN LINUX
// think
| U. K. Roy | [ ]
System Programming in Linux 74
IN LINUX
signal(mutex)
// think
}
Asymmetric solution
• Odd philosopher will pick first left then right while even
philosopher picks up first left and then right chopstick
| U. K. Roy | [ ]
System Programming in Linux 75
•
SYSTEM
signal (mutex) …. wait (mutex)
• Mutual exclusion requirement is violated
PROGRAMMING
• Violation of progress condition
• Possibility of deadlock
IN LINUX
| U. K. Roy | [ ]
DEADLOCKS
System Programming in Linux 77
System Model
SYSTEM
The Deadlock Problem
Deadlock Characterization
Methods for Handling Deadlocks
PROGRAMMING
Deadlock Prevention
Deadlock Avoidance
Deadlock Detection
Recovery from Deadlock
IN LINUX
| U. K. Roy | [ ]
System Programming in Linux 78
PROGRAMMING
Example 1
System has 2 disk drives.
P1 and P2 each hold one disk drive and each needs another one.
Example 2
IN LINUX
semaphores A and B, initialized to 1
P0 P1
wait (A); wait(B)
wait (B); wait(A)
Example 3
Each philosopher picks left chopstick in dining philosopher problem
| U. K. Roy | [ ]
System Programming in Linux 79
SYSTEM
Traffic only in one direction.
PROGRAMMING
Each section of a bridge can be viewed as a resource.
If two cars approach each other at the crossing, neither can
start until the other has gone(Dead locks)
If a deadlock occurs, it can be resolved if one car backs up
IN LINUX
| U. K. Roy | [ ]
System Programming in Linux 80
SYSTEM
CPU cycles, memory space, I/O devices
PROGRAMMING
Process may requests as many resources as it requires to
perform its designated task
Must be less the total number of resources available
Use
Release
IN LINUX
requesting process must wait
| U. K. Roy | [ ]
System Programming in Linux 81
Deadlock Characterization
Advanced Java Programming
Deadlock can arise if four conditions hold simultaneously.
SYSTEM
Mutual exclusion: at least one resource must in non-sharable
mode i.e. only one process at a time can use that resource.
PROGRAMMING
No preemption: a resource can be released only voluntarily by the
process holding it, after that process has completed its task.
IN LINUX
is waiting for a resource that is held by P2, …, Pn–1 is waiting for a
resource that is held by Pn, and Pn is waiting for a resource that is
held by P0.
| U. K. Roy | [ ]
System Programming in Linux 82
Resource-Allocation Graph
Advanced Java Programming
SYSTEM
PROGRAMMING
P = {P1, P2, …, Pn}, the set consisting of all the processes
in the system.
R = {R1, R2, …, Rm}, the set consisting of all resource
types in the system.
IN LINUX
E is partitioned into two types:
request edge – directed edge Pi → Rj
assignment edge – directed edge Rj → Pi
| U. K. Roy | [ ]
System Programming in Linux 83
SYSTEM
Process
PROGRAMMING
Pi requests an instance of Rj Pi
Rj
Pi
IN LINUX
Pi is holding an instance of Rj
Rj
| U. K. Roy | [ ]
System Programming in Linux 84
SYSTEM
P = {P1, P2, P3}
R = {R1, R2, R3, R4}
E = {P1 R1, P2 R3, R1 P2 ,
R2 P2, R2 P1, R3 P3}
PROGRAMMING
Resource Instances
One instance of resource type R1
Two instances of resource type R2
IN LINUX
One instance of resource type R3
Three instances of resource type R3
| U. K. Roy | [ ]
System Programming in Linux 85
SYSTEM
If graph contains no cycles ⇒ no deadlock.
PROGRAMMING
if several instances per resource type, possibility of
deadlock.
IN LINUX
| U. K. Roy | [ ]
System Programming in Linux 86
SYSTEM
PROGRAMMING
IN LINUX
| U. K. Roy | [ ]
System Programming in Linux 87
SYSTEM
PROGRAMMING
IN LINUX
| U. K. Roy | [ ]
System Programming in Linux 88
SYSTEM
Ensure that the system will never enter a deadlock state.
• Deadlock prevention—ensure that one of the necessary
conditions can not hold
• Deadlock avoidance—systems additional information about
PROGRAMMING
resource usage pattern during the execution
IN LINUX
occur in the system; used by most operating systems,
including UNIX.
| U. K. Roy | [ ]
System Programming in Linux 89
Deadlock Prevention
Advanced Java Programming
SYSTEM
• Restrain the ways request can be made.
PROGRAMMING
requests a resource, it does not hold any other resources.
Two possibilities
Require process to request and be allocated all its resources before it
begins execution
allow process to request resources only when the process has none.
IN LINUX
• Low resource utilization; possibility of starvation
• Data inconsistency
| U. K. Roy | [ ]
System Programming in Linux 90
No Preemption –
SYSTEM
Two possibilities
If a process that is holding some resources requests another
resource that cannot be immediately allocated to it, then all
resources currently being held are released.
PROGRAMMING
If a process requests some resources that are not available as
they are allocated to other waiting processes, then preempt
desired resources from waiting processes
• Preempted resources are added to the list of resources for which the
process is waiting.
IN LINUX
• Process will be restarted only when it can regain its old resources, as
well as the new ones that it is requesting.
| U. K. Roy | [ ]
System Programming in Linux 91
SYSTEM
enumeration.
F(tape drive)=1
F(disk drive)=5
F(printer)=12
PROGRAMMING
Facts:I
If a process currently holds resource type Ri , the process can request
another resource type Rj if F(Rj) > F(Ri)
OR
If a process requests a resource type Rj , if it has released any resource
IN LINUX
type Ri such the F(Ri) ≥ F(Rj)
Proof(by contradiction):
Set of deadlocked process {P0, P1, P2, …, Pn}
F(R0) < F(R1) < F(R2) <…< F(Rn) < F(R0) which is impossible
So there can be no circular wait—no deadlock
| U. K. Roy | [ ]
System Programming in Linux 92
Deadlock Avoidance
Advanced Java Programming
• Requires that the system has some additional information available.
SYSTEM
• With this additional information, system can decide whether a newly arrived
request can be satisfied (served) or not
PROGRAMMING
each type that it may need.
IN LINUX
| U. K. Roy | [ ]
System Programming in Linux 93
SYSTEM
System is in safe state if there exists a sequence <P1, P2, …, Pn>
of ALL the processes in the systems such that for each Pi, the
resources that Pi can still request can be satisfied by currently
available resources + resources held by all the Pj, with j < i.
PROGRAMMING
That is:
If Pi needs some resources that are not immediately available, then
Pi can wait until all Pj have finished.
When Pj is finished, Pi can obtain needed resources, execute, return
IN LINUX
allocated resources, and terminate.
When Pi terminates, Pi +1 can obtain its needed resources, and so on.
SYSTEM
PROGRAMMING
12 disk drives
Snapshot at time T0:
SYSTEM
PROGRAMMING
IN LINUX
| U. K. Roy | [ ]
System Programming in Linux 96
Avoidance algorithms
Advanced Java Programming
SYSTEM
Single instance of a resource type. Use a
resource-allocation graph
PROGRAMMING
IN LINUX
| U. K. Roy | [ ]
System Programming in Linux 97
SYSTEM
resource Rj; represented by a dashed line.
PROGRAMMING
Request edge converted to an assignment edge when the
resource is allocated to the process.
IN LINUX
Resources must be claimed a priori in the system.
| U. K. Roy | [ ]
System Programming in Linux 98
Resource-Allocation Graph
Advanced Java Programming
SYSTEM
PROGRAMMING
IN LINUX
| U. K. Roy | [ ]
System Programming in Linux 99
SYSTEM
Suppose that process Pi requests a resource Rj
PROGRAMMING
(including the claim edge )in the resource allocation graph
IN LINUX
| U. K. Roy | [ ]
System Programming in Linux 100
SYSTEM
PROGRAMMING
IN LINUX
| U. K. Roy | [ ]
System Programming in Linux 101
Banker’s Algorithm
Advanced Java Programming
SYSTEM
Multiple instances.
PROGRAMMING
When a process requests a resource it
may have to wait.
Let n = number of processes, and m = number of resources types.
SYSTEM
Available: Vector of length m. If Available [j] = k, there are k
instances of resource type Rj available.
PROGRAMMING
Allocation: n x m matrix. If Allocation[i,j] = k then Pi is
currently allocated k instances of Rj.
IN LINUX
instances of Rj to complete its task.
| U. K. Roy | [ ]
System Programming in Linux 103
SYSTEM
Work = Available
Finish [i] = false for i = 0, 1, …, n- 1.
PROGRAMMING
If no such i exists, go to step 4.
IN LINUX
4. If Finish [i] == true for all i, then the system is in a safe
state.
Algorithm requires an order of O(m x n2) operations to detect
whether the system is in safe state.
| U. K. Roy | [ ]
System Programming in Linux 104
SYSTEM
If Requesti ≤ Needi go to step 2. Otherwise, raise error condition,
since process has exceeded its maximum claim.
PROGRAMMING
If Requesti ≤ Available, go to step 3. Otherwise Pi must wait, since
resources are not available.
IN LINUX
Allocationi = Allocationi + Requesti;
Needi = Needi – Requesti;
| U. K. Roy | [ ]
System Programming in Linux 105
SYSTEM
3 resource types:
Available
PROGRAMMING
Snapshot at time T0:
Allocation Max Available
ABC ABC ABC
P0 0 1 0 753 332
IN LINUX
P1 2 0 0 322
P2 3 0 2 902
P3 2 1 1 222
P4 0 0 2 4 3 3
| U. K. Roy | [ ]
System Programming in Linux 106
Example (Cont.)
Advanced Java Programming
SYSTEM
Max – Allocation.
Need
ABC
PROGRAMMING
P0 743
P1 122
P2 600
P3 011
IN LINUX
P4 431
| U. K. Roy | [ ]
System Programming in Linux 107
SYSTEM
(3,3,2) ⇒ true.
Allocation Need Available
ABC ABC ABC
P0 0 1 0 743 230
PROGRAMMING
P1 3 0 2 020
P2 3 0 1 600
P3 2 1 1 011
P4 0 0 2 431
IN LINUX
Executing safety algorithm shows that sequence <
P1, P3, P4, P0, P2> satisfies safety requirement.
Can request for (3,3,0) by P4 be granted?
Can request for (0,2,0) by P0 be granted?
| U. K. Roy | [ ]
System Programming in Linux 108
Deadlock Detection
Advanced Java Programming
SYSTEM
System must provide
Detection algorithm
• Checks the state of the system to determine whether a
Recovery scheme
IN LINUX
| U. K. Roy | [ ]
System Programming in Linux 109
SYSTEM
Maintain wait-for graph
Nodes are processes.
Pi → Pj if Pi is waiting for Pj.
PROGRAMMING
Periodically invoke an algorithm that searches for a
cycle in the graph. If there is a cycle, there exists a
deadlock.
IN LINUX
order of n2 operations, where n is the number of
vertices in the graph.
| U. K. Roy | [ ]
System Programming in Linux 110
SYSTEM
PROGRAMMING
IN LINUX
ResourceAllocation Graph Corresponding waitfor graph
| U. K. Roy | [ ]
System Programming in Linux 111
SYSTEM
Available: A vector of length m indicates the number of available
resources of each type.
PROGRAMMING
Request: An n x m matrix indicates the current request of each
process. If Request [ij] = k, then process Pi is requesting k more
instances of resource type. Rj.
IN LINUX
| U. K. Roy | [ ]
System Programming in Linux 112
Detection Algorithm
Advanced Java Programming
SYSTEM
(a) Work = Available
For i = 1,2, …, n, if Allocationi ≠ 0, then
Finish[i] = false;otherwise, Finish[i] = true.
PROGRAMMING
(b) Requesti ≤ Work
If no such i exists, go to step 4.
IN LINUX
4. If Finish[i] == false, for some i, 1 ≤ i ≤ n, then the system is in
deadlock state. Moreover, if Finish[i] == false, then Pi is
deadlocked.
Algorithm requires an order of O(m x n2) operations to detect
whether the system is in deadlocked state.
| U. K. Roy | [ ]
System Programming in Linux 113
SYSTEM
three resource types
A (7 instances), B (2 instances), and C (6 instances).
PROGRAMMING
P0 010 000 000
P1 200 202
P2 303 000
P3 211 100
P4 002 002
i.
IN LINUX
Sequence <P0, P2, P3, P1, P4> will result in Finish[i] = true for all
| U. K. Roy | [ ]
System Programming in Linux 114
Example (Cont.)
Advanced Java Programming
SYSTEM
Request
ABC
P0 0 0 0
P1 2 0 1
PROGRAMMING P2 0 0 1
P3 1 0 0
P4 0 0 2
IN LINUX
State of system?
Can reclaim resources held by process P0, but insufficient
resources to fulfill other processes; requests.
Deadlock exists, consisting of processes P1, P2, P3, and P4.
| U. K. Roy | [ ]
System Programming in Linux 115
Detection-Algorithm Usage
Advanced Java Programming
SYSTEM
How often a deadlock is likely to occur?
How many processes will need to be rolled back?
• one for each disjoint cycle
PROGRAMMING
Considerably overhead
Run once per hour
Run when CPU/resource utilization drops below 40%
If detection algorithm is invoked arbitrarily, there may be
IN LINUX
many cycles in the resource graph and so we would not be
able to tell which of the many deadlocked processes
“caused” the deadlock.
| U. K. Roy | [ ]
System Programming in Linux 116
cycle is eliminated.
SYSTEM
Abort one process at a time until the deadlock
PROGRAMMING
Priority of the process.
How long process has computed, and how much longer to
completion.
Resources the process has used.
Resources process needs to complete.
IN LINUX
How many processes will need to be terminated.
Is process interactive or batch?
| U. K. Roy | [ ]
System Programming in Linux 117
SYSTEM
Selecting a victim – minimize cost.
PROGRAMMING
Starvation – same process may always be picked as
victim, include number of rollback in cost factor.
IN LINUX
| U. K. Roy | [ ]
Memory
Management
System Programming in Linux 119
Memory Management
Advanced Java Programming
SYSTEM
Background
Swapping
Contiguous Memory Allocation
Paging
PROGRAMMING
Structure of the Page Table
Segmentation
Example: The Intel Pentium
IN LINUX
| U. K. Roy | [ ]
System Programming in Linux 120
SYSTEM
To provide a detailed description of various
ways of organizing memory hardware
To discuss various memory-management
techniques, including paging and
segmentation
PROGRAMMING
To provide a detailed description of the
Intel Pentium, which supports both pure
segmentation and segmentation with
paging
IN LINUX
| U. K. Roy | [ ]
System Programming in Linux 121
Background
Advanced Java Programming
SYSTEM
Program must be brought (from disk) into
memory and placed within a process for it to be
run
Main memory and registers are only storage
CPU can access directly
PROGRAMMING
Register access in one CPU clock (or less)
Main memory can take many cycles
Cache sits between main memory and CPU
registers
IN LINUX
Protection of memory required to ensure correct
operation
| U. K. Roy | [ ]
System Programming in Linux 122
SYSTEM
A pair of base and limit registers define the
logical address space
PROGRAMMING
IN LINUX
| U. K. Roy | [ ]
System Programming in Linux 123
SYSTEM
Address binding of instructions and data to
memory addresses can happen at three different
stages
Compile time: If memory location known a priori,
absolute code can be generated; must recompile
code if starting location changes
IN LINUX
support for address maps (e.g., base and limit
registers)
| U. K. Roy | [ ]
System Programming in Linux 124
SYSTEM
PROGRAMMING
IN LINUX
| U. K. Roy | [ ]
System Programming in Linux 125
SYSTEM
The concept of a logical address space that is
bound to a separate physical address
space is central to proper memory
management
Logical address – generated by the CPU;
also referred to as virtual address
PROGRAMMING
Physical address – address seen by the
memory unit
Logical and physical addresses are the same
in compile-time and load-time address-
binding schemes; logical (virtual) and
IN LINUX
physical addresses differ in execution-time
address-binding scheme
| U. K. Roy | [ ]
System Programming in Linux 126
SYSTEM
Hardware device that maps virtual to physical
address
PROGRAMMING
a user process at the time it is sent to memory
IN LINUX
| U. K. Roy | [ ]
System Programming in Linux 127
SYSTEM
PROGRAMMING
IN LINUX
| U. K. Roy | [ ]
System Programming in Linux 128
Dynamic Loading
Advanced Java Programming
SYSTEM
Routine is not loaded until it is called
Better memory-space utilization; unused
routine is never loaded
Useful when large amounts of code are needed
to handle infrequently occurring cases
PROGRAMMING
No special support from the operating system is
required implemented through program design
IN LINUX
| U. K. Roy | [ ]
System Programming in Linux 129
Dynamic Linking
Advanced Java Programming
SYSTEM
Linking postponed until execution time
Small piece of code, stub, used to locate
the appropriate memory-resident library
routine
Stub replaces itself with the address of the
PROGRAMMING
routine, and executes the routine
Operating system needed to check if
routine is in processes’ memory address
Dynamic linking is particularly useful for
IN LINUX
libraries
System also known as shared libraries
| U. K. Roy | [ ]
System Programming in Linux 130
SYSTEM
A process can be swapped temporarily out of memory to a
backing store, and then brought back into memory for
continued execution
Backing store – fast disk large enough to accommodate
copies of all memory images for all users; must provide
direct access to these memory images
PROGRAMMING
Roll out, roll in – swapping variant used for priority-
based scheduling algorithms; lower-priority process is
swapped out so higher-priority process can be loaded and
executed
Major part of swap time is transfer time; total transfer
time is directly proportional to the amount of memory
IN LINUX
swapped
Modified versions of swapping are found on many systems
(i.e., UNIX, Linux, and Windows)
System maintains a ready queue of ready-to-run
processes which have memory images on disk
| U. K. Roy | [ ]
System Programming in Linux 131
SYSTEM
PROGRAMMING
IN LINUX
| U. K. Roy | [ ]
System Programming in Linux 132
Contiguous Allocation
Advanced Java Programming
SYSTEM
Main memory usually into two partitions:
Resident operating system, usually held in low
memory with interrupt vector
User processes then held in high memory
PROGRAMMING
processes from each other, and from changing
operating-system code and data
Base register contains value of smallest physical
address
Limit register contains range of logical addresses –
IN LINUX
each logical address must be less than the limit
register
MMU maps logical address dynamically
| U. K. Roy | [ ]
System Programming in Linux 133
SYSTEM
PROGRAMMING
IN LINUX
| U. K. Roy | [ ]
System Programming in Linux 134
SYSTEM
Multiple-partition allocation
Hole – block of available memory; holes of
various size are scattered throughout memory
When a process arrives, it is allocated memory
from a hole large enough to accommodate it
PROGRAMMING
Operating system maintains information about:
a) allocated partitions b) free partitions (hole)
OS OS OS OS
IN LINUX
process 9 process 9
process 8 process 10
| U. K. Roy | [ ]
System Programming in Linux 135
SYSTEM
How to satisfy a request of size n from a list of free holes
PROGRAMMING
Produces the smallest leftover hole
Worst-fit: Allocate the largest hole; must also
search entire list
Produces the largest leftover hole
Firstfit and bestfit better than worstfit in terms of
speed and storage utilization
IN LINUX
| U. K. Roy | [ ]
System Programming in Linux 136
Fragmentation
Advanced Java Programming
SYSTEM
External Fragmentation – total memory space
exists to satisfy a request, but it is not contiguous
Internal Fragmentation – allocated memory may
be slightly larger than requested memory; this size
difference is memory internal to a partition, but not
being used
PROGRAMMING
Reduce external fragmentation by compaction
Shuffle memory contents to place all free memory
together in one large block
Compaction is possible only if relocation is dynamic,
and is done at execution time
IN LINUX
I/O problem
• Latch job in memory while it is involved in I/O
• Do I/O only into OS buffers
| U. K. Roy | [ ]
System Programming in Linux 137
SYSTEM
Logical address space of a process can be
noncontiguous; process is allocated physical
memory whenever the latter is available
Divide physical memory into fixed-sized blocks
called frames (size is power of 2, between 512
bytes and 8,192 bytes)
PROGRAMMING
Divide logical memory into blocks of same size
called pages
Keep track of all free frames
To run a program of size n pages, need to find n
free frames and load program
IN LINUX
Set up a page table to translate logical to
physical addresses
Internal fragmentation
| U. K. Roy | [ ]
System Programming in Linux 138
SYSTEM
Address generated by CPU is divided into:
PROGRAMMING
Page offset (d) – combined with base address
to definepage number
the physical memory address that is
page offset
sent to the memory unit
p d
m n n
IN LINUX
For given logical address space 2m and page size
2n
| U. K. Roy | [ ]
System Programming in Linux 139
Paging Hardware
Advanced Java Programming
SYSTEM
PROGRAMMING
IN LINUX
| U. K. Roy | [ ]
System Programming in Linux 140
SYSTEM
PROGRAMMING
IN LINUX
| U. K. Roy | [ ]
System Programming in Linux 141
Paging Example
Advanced Java Programming
SYSTEM
PROGRAMMING
IN LINUX
32byte memory and 4byte pages
| U. K. Roy | [ ]
System Programming in Linux 142
SYSTEM
PROGRAMMING
IN LINUX
Before allocation After allocation
| U. K. Roy | [ ]
System Programming in Linux 143
SYSTEM
Page table is kept in main memory
Page-table base register (PTBR) points to
the page table
Page-table length register (PRLR) indicates
size of the page table
In this scheme every data/instruction access
PROGRAMMING
requires two memory accesses. One for the
page table and one for the data/instruction.
The two memory access problem can be solved
by the use of a special fast-lookup hardware
cache called associative memory or
translation look-aside buffers (TLBs)
IN LINUX
Some TLBs store address-space identifiers
(ASIDs) in each TLB entry – uniquely identifies
each process to provide address-space
protection for that process
| U. K. Roy | [ ]
System Programming in Linux 144
Associative Memory
Advanced Java Programming
SYSTEM
Associative memory – parallel search
Page # Frame #
PROGRAMMING
Address translation (p, d)
If p is in associative register, get frame # out
Otherwise get frame # from page table in memory
IN LINUX
| U. K. Roy | [ ]
System Programming in Linux 145
SYSTEM
PROGRAMMING
IN LINUX
| U. K. Roy | [ ]
System Programming in Linux 146
SYSTEM
Associative Lookup = ε time unit
Assume memory cycle time is 1 microsecond
Hit ratio – percentage of times that a page number
is found in the associative registers; ratio related to
PROGRAMMING
number of associative registers
Hit ratio = α
Effective Access Time (EAT)
EAT = (1 + ε) α + (2 + ε)(1 – α)
=2+ε–α
IN LINUX
| U. K. Roy | [ ]
System Programming in Linux 147
Memory Protection
Advanced Java Programming
SYSTEM
Memory protection implemented by
associating protection bit with each frame
PROGRAMMING
“valid” indicates that the associated page is in
the process’ logical address space, and is thus
a legal page
“invalid” indicates that the page is not in the
process’ logical address space
IN LINUX
| U. K. Roy | [ ]
Valid (v) or Invalid (i) Bit In A Page
System Programming in Linux 148
SYSTEM
PROGRAMMING
IN LINUX
| U. K. Roy | [ ]
System Programming in Linux 149
SYSTEM
Shared code
One copy of read-only (reentrant) code
shared among processes (i.e., text editors,
compilers, window systems).
Shared code must appear in same location in
PROGRAMMING
the logical address space of all processes
IN LINUX
code and data
The pages for the private code and data can
appear anywhere in the logical address space
| U. K. Roy | [ ]
System Programming in Linux 150
SYSTEM
PROGRAMMING
IN LINUX
| U. K. Roy | [ ]
System Programming in Linux 151
SYSTEM
Hierarchical Paging
PROGRAMMING
Inverted Page Tables
IN LINUX
| U. K. Roy | [ ]
System Programming in Linux 152
SYSTEM
Break up the logical address space into multiple
page tables
PROGRAMMING
IN LINUX
| U. K. Roy | [ ]
System Programming in Linux 153
SYSTEM
PROGRAMMING
IN LINUX
| U. K. Roy | [ ]
System Programming in Linux 154
SYSTEM
A logical address (on 32-bit machine with 1K page size)
is divided into:
a page number consisting of 22 bits
a page offset consisting of 10 bits
PROGRAMMING
Since the page table is paged, the page number is
further divided into:
a 12-bit page number
page number page offset
a 10-bit page offset
pi p2 d
Thus, a logical address is as follows:
IN LINUX 12 10 10
| U. K. Roy | [ ]
System Programming in Linux 155
Address-Translation Scheme
Advanced Java Programming
SYSTEM
PROGRAMMING
IN LINUX
| U. K. Roy | [ ]
System Programming in Linux 156
SYSTEM
PROGRAMMING
IN LINUX
| U. K. Roy | [ ]
System Programming in Linux 157
SYSTEM
Common in address spaces > 32 bits
PROGRAMMING
Virtual page numbers are compared in this
chain searching for a match. If a match is
found, the corresponding physical frame is
IN LINUX
extracted.
| U. K. Roy | [ ]
System Programming in Linux 158
SYSTEM
PROGRAMMING
IN LINUX
| U. K. Roy | [ ]
System Programming in Linux 159
SYSTEM
One entry for each real page of memory
Entry consists of the virtual address of the
page stored in that real memory location,
with information about the process that
owns that page
PROGRAMMING
Decreases memory needed to store each
page table, but increases time needed to
search the table when a page reference
occurs
IN LINUX
Use hash table to limit the search to one
— or at most a few — page-table entries
| U. K. Roy | [ ]
System Programming in Linux 160
SYSTEM
PROGRAMMING
IN LINUX
| U. K. Roy | [ ]
System Programming in Linux 161
SYSTEM
Memory-management scheme that supports user
view of memory
A program is a collection of segments. A segment is
a logical unit such as:
main program,
procedure,
PROGRAMMING function,
method,
object,
local variables, global variables,
common block,
IN LINUX
stack,
symbol table, arrays
| U. K. Roy | [ ]
System Programming in Linux 162
SYSTEM
PROGRAMMING
IN LINUX
| U. K. Roy | [ ]
System Programming in Linux 163
1 SYSTEM 4
PROGRAMMING 3
4
2
IN LINUX
user space physical memory space
| U. K. Roy | [ ]
System Programming in Linux 164
Segmentation Architecture
Advanced Java Programming
SYSTEM
Logical address consists of a two tuple:
<segment-number, offset>,
Segment table – maps two-dimensional
physical addresses; each table entry has:
base – contains the starting physical address
PROGRAMMING
where the segments reside in memory
limit – specifies the length of the segment
Segment-table base register (STBR) points
to the segment table’s location in memory
Segment-table length register (STLR)
indicates number of segments used by a
IN LINUX
program;
segment number s is legal if s <
STLR
| U. K. Roy | [ ]
System Programming in Linux 165
SYSTEM
Protection
With each entry in segment table associate:
• validation bit = 0 ⇒ illegal segment
• read/write/execute privileges
Protection bits associated with segments;
PROGRAMMING
code sharing occurs at segment level
Since segments vary in length, memory
allocation is a dynamic storage-allocation
problem
IN LINUX
A segmentation example is shown in the
following diagram
| U. K. Roy | [ ]
System Programming in Linux 166
Segmentation Hardware
Advanced Java Programming
SYSTEM
PROGRAMMING
IN LINUX
| U. K. Roy | [ ]
System Programming in Linux 167
Example of Segmentation
Advanced Java Programming
SYSTEM
PROGRAMMING
IN LINUX
| U. K. Roy | [ ]
System Programming in Linux 168
paging
SYSTEM
Supports both segmentation and segmentation with
PROGRAMMING
• Which produces linear addresses
Linear address given to paging unit
• Which generates physical address in main memory
• Paging units form equivalent of MMU
IN LINUX
| U. K. Roy | [ ]
Logical to Physical Address Translation in
System Programming in Linux 169
SYSTEM
PROGRAMMING
IN LINUX
| U. K. Roy | [ ]
System Programming in Linux 170
SYSTEM
PROGRAMMING
IN LINUX
| U. K. Roy | [ ]
System Programming in Linux 171
SYSTEM
PROGRAMMING
IN LINUX
| U. K. Roy | [ ]
System Programming in Linux 172
SYSTEM
Broken into four parts:
PROGRAMMING
IN LINUX
| U. K. Roy | [ ]
System Programming in Linux 173
SYSTEM
PROGRAMMING
IN LINUX
| U. K. Roy | [ ]
LINUX COMMANDS
System Programming in Linux 175
SYSTEM
In this session, you will learn to:
Use the basic Unix commands like
pwd
date
PROGRAMMING
who
ls
man
Use “man” pages
IN LINUX
| U. K. Roy | [ ]
System Programming in Linux 176
Simple commands
Advanced Java Programming
pwd SYSTEM
Displays the current working directory.
date
PROGRAMMING
Displays the current date and time
IN LINUX
| U. K. Roy | [ ]
System Programming in Linux 177
Simple commands
Advanced Java Programming
who SYSTEM
Displays the names of all the users who have currently
logged in
PROGRAMMING
who am i
Displays the name of the current user.
IN LINUX
| U. K. Roy | [ ]
System Programming in Linux 178
ls
SYSTEM
•
• Syntax :ls [options] [file….]
• options: -l list in long format
• -a list all file including those
beginning with a dot
• -i list inode no of file in first
PROGRAMMING
column
• -s reports disk blocks occupied by
file
• -R recursively list all sub
directories
IN LINUX
• -F mark type of each file
• -C display files in columns
| U. K. Roy | [ ]
System Programming in Linux 179
Meta characters
Advanced Java Programming
SYSTEM
Meta Purpose Example
Characters
* Match with one or more characters or none $ ls –l *.c file*
? Match with any single character $ ls –l file?
[…] Match with any single character within the $ ls –l file[abc]
brackets
; Command separator $ cat file1; cat file2
PROGRAMMING
| Pipe two commands $ cat abc | wc
() Group commands $ (echo “==== x.c ====”; cat x.c) > out
Useful when the output of thecommand group
has to be redirected
`command` Execute the command enclosed within back count=`expr $count + 1`
quotes. Useful when the output of a command
into a variable in a shell script assuming count has value3, this
IN LINUX
increments the value of count
‘string’ Quote all characters with no substitution echo ‘expr $count + 1’
(ex. no special meaning for $) displays expr $count + 1
“string” Quote all characters with substitution. echo “expr $count + 1”
The characters $, \ (back slash) and back quote displays expr 3 + 1
have special meaning. assuming the variable count has value 3
| U. K. Roy | [ ]
System Programming in Linux 180
$ ls –l
SYSTEM
total 6
-rwxr-xr-x 1 user1 projA 12373 Dec 15 14:45 a.out
drwxr-xr-x 2 user2 projD 4096 Dec 22 14:00 awkpro
-rw-r--r-- 1 user1 projA 12831 Dec 12 13:59 c
-rw------- 1 user1 projA 61440 Dec 15 11:16 core
-rw-r--r-- 1 user3 projC 255 Dec 20 14:29 cs
PROGRAMMING
File access File size Date & time of File name
permissions User id
in bytes modification
File type Group id
Link count
IN LINUX
| U. K. Roy | [ ]
System Programming in Linux 181
SYSTEM
on-line to explain the usage of the Unix system and
commands.
• Syntax:
• man [options] command_name
PROGRAMMING
Common Options
-k keyword
matches
list command synopsis line for all keyword
•
•
IN LINUX
info command_name - help for the internal commands
help -–command_name– gives command synatx
| U. K. Roy | [ ]
System Programming in Linux 182
•
SYSTEM
In this session, you have learned to …
use the basic Unix commands like
• pwd
• date
PROGRAMMING
• who
• ls
• man
• use “man” pages
IN LINUX
| U. K. Roy | [ ]
FILES AND
DIRECTORIES
System Programming in Linux 184
SYSTEM
In this session, you will learn to:
set file permissions using the chmod command
use directory-related commands namely mkdir, rmdir,
cd commands
PROGRAMMING
use file-related commands namely cp, mv, rm
commands
access advanced file permissions using commands
umask, suid, sgid, linking files, stickybit
create and edit files using the vi editor
IN LINUX
| U. K. Roy | [ ]
System Programming in Linux 185
SYSTEM
respect to the following
Permission Levels
User (owner) (u)
PROGRAMMING
Group (wheel, staff, daemon, etc.) (g)
World (guest, anonymous and all other users) (o)
Permission Settings
IN LINUX
Read (r)
Write (w)
Execute (x)
| U. K. Roy | [ ]
System Programming in Linux 186
SYSTEM
List the contents of directory
Remove the directory
No Write permission does not allow the user to :
copy files to the directory
PROGRAMMING
remove files from the directory
rename files in the directory
make a subdirectory
remove a subdirectory from the directory
IN LINUX
move files to, and from the directory
| U. K. Roy | [ ]
System Programming in Linux 187
SYSTEM
No execute permission does not allow the user to:
PROGRAMMING
change to the directory
IN LINUX
copy a file to, or from the directory
| U. K. Roy | [ ]
System Programming in Linux 188
SYSTEM
Syntax:
chmod <category> <operation> <permission>
<filename(s)>
or
chmod <octal number> filename
PROGRAMMING
Octal
4
Number
- for read
2 - for write
3 - for execution
IN LINUX
$ chmod 744 xyz
this will set rwx for user, r– for group, r—for others
| U. K. Roy | [ ]
System Programming in Linux 189
Directory creation
Advanced Java Programming
Command Syntax
SYSTEM
mkdir [OPTION] DIRECTORY
$mkdir <path>/<directory>
$mkdir –m <directory>
$mkdir –p
<directory1>/<directory2>/<directory3>
PROGRAMMING
Example:
mkdir project1
IN LINUX
This creates a directory project1 under current
directory
Directory removal
Advanced Java Programming
SYSTEM
Syntax
rmdir <directory name>
Example
Removes project1 directory in the current directory
rmdir project1
PROGRAMMING
Remove multiple directories
rmdir pos1 pos2
Remove the directory recursively
rmdir –p dir1/dir2/dir2
IN LINUX
Rule: rmdir can be executed to remove a directory
if it is empty and not the current directory
| U. K. Roy | [ ]
System Programming in Linux 191
Command - cd
Advanced Java Programming
directory
SYSTEM
cd command is used to change the
PROGRAMMING
cd
cd ..
cd /
- take to the home directory
- takes to the parent directory
- takes to the root directory
IN LINUX
| U. K. Roy | [ ]
System Programming in Linux 192
SYSTEM
File Command
Operation
Copying cp
a file
mv
PROGRAMMING
Moving a file
Removing rm
a file
cat
IN LINUX
Displayinga file
and concatenating files
| U. K. Roy | [ ]
System Programming in Linux 193
Command - cp
Advanced Java Programming
Syntax
SYSTEM
Used to copy files across directories
PROGRAMMING
Example
cp file1 file2
IN LINUX
| U. K. Roy | [ ]
System Programming in Linux 194
Command - cp
Advanced Java Programming
-p
•
owner id
group id
SYSTEM
Copies the file and preserves the following attributes
•
• permissions
• last modification time
PROGRAMMING
-r
recursive copy; copy subdirectories under the directory if
any
-i
IN LINUX
interactive; prompts for confirmation before overwriting the
target file, if it already exists
| U. K. Roy | [ ]
System Programming in Linux 195
Command - mv
Advanced Java Programming
SYSTEM
Used to move a file, or rename a file
PROGRAMMING
• group id
• permissions
• Last modification time
IN LINUX
-i prompts before overwriting destination file
| U. K. Roy | [ ]
System Programming in Linux 196
Command – rm
Advanced Java Programming
•
-f
SYSTEM
Syntax : rm file(s)
suppresses all prompting
PROGRAMMING
IN LINUX
Caution: Use “i” option along with “r” to get
notified on deletion
| U. K. Roy | [ ]
System Programming in Linux 197
$ ls –l
-rwxr-xr-x 1user1 training 12373 Dec 15 14:45 a.out
SYSTEM
-rwxr-xr-x 3 user1 faculty 4096 Dec 24 11:56 awkpro
$chown user2 a.out
$ls –l
-rwxr-xr-x 1user2 training 12373 Dec 15 14:45 a.out
-rwxr-xr-x 3 user1 faculty 4096 Dec 24 11:56 awkpro
PROGRAMMING
IN LINUX
| U. K. Roy | [ ]
System Programming in Linux 198
Command - umask
Advanced Java Programming
SYSTEM
a file and directory while creating
umask command is used to see the default mask for
the file permission
Default umask value will be set in the system
PROGRAMMING
environment file like /etc/profile
umask 022 will set a mask of 022 for the current
session
The file permission after setting this umask
IN LINUX
value will be 644
And the directory permission will be 755
| U. K. Roy | [ ]
System Programming in Linux 199
Command - ln
Advanced Java Programming
Linking files
SYSTEM
Hard Link (in the same filesystem)
$ ln /usr/bin/clear /usr/bin/cls
PROGRAMMING
Hard link uses the same inode number
IN LINUX
$ ln –s /usr/bin/clear /home/user1/cls
| U. K. Roy | [ ]
System Programming in Linux 200
SYSTEM
This means that if the SUID bit is set for any application
then your user ID would be set as that of the owner of
application/file rather than the current user, while running
that application
PROGRAMMING
“set user ID” bit can be set in one of the two ways:
• chmod u+s <filename>
IN LINUX
• The leftmost octal number 4 indicates “set user ID” bit to be set,
other octal digits indicate regular file permissions. This is
meaningful for executable files only.
| U. K. Roy | [ ]
System Programming in Linux 201
SYSTEM
Just like SUID, setting the SGID bit for a file sets your
group ID to the file's group while the file is executing
“set group ID” bit can be set in one of the two ways:
PROGRAMMING
• chmod g+s <filename>
IN LINUX
meaningful for executable files only.
SYSTEM
Typically set to a directory that is shareable
Any user can create a file in such sharable directory
Only owner of the file or super user (root) can remove a
file from the directory
PROGRAMMING
“sticky” bit can be set in one of the two ways:
• chmod +t <directoryname>
IN LINUX
• The leftmost octal number 1 indicates “sticky” bit to be set, other
octal digits indicate regular file permissions.
| U. K. Roy | [ ]
System Programming in Linux 203
SYSTEM
A screen-oriented text editor
Included with most UNIX system distributions
Command driven
PROGRAMMING
Categories of commands include
Cursor movement
Editing commands
Search and replace commands
<filename> IN LINUX
The vi editor is invoked by the following command: vi
| U. K. Roy | [ ]
System Programming in Linux 204
the
the
2
w
q
q
Backspa
uick
uick
w
h
ce
SYSTEM
brown
brown
fox
fox
j k
the
the
q
q
Spa
uick
uick
ce
brown
$
brown
^
fox
fox
b b b
PROGRAMMING
IN LINUX
| U. K. Roy | [ ]
System Programming in Linux 205
Editing commands
Advanced Java Programming
i
a
SYSTEM
- inserts text to the left of the cursor
- inserts text to the right of the cursor
I - inserts text at the beginning of the line
PROGRAMMING
A - appends text at end of the line
o - opens line below
O - opens line above
R - replaces text from cursor to right
s - replaces a single character with any number of
S IN LINUX
characters
- replaces entire line
| U. K. Roy | [ ]
System Programming in Linux 206
Editing commands
Advanced Java Programming
Deletion
x
3x
SYSTEM
- to delete character at cursor position
- to delete 3 characters at cursor position
dw - to delete word
PROGRAMMING
2dw - to delete 2 word
dd - to delete a line
2dd - to delete 2 lines
IN LINUX
| U. K. Roy | [ ]
System Programming in Linux 207
Editing commands
Advanced Java Programming
Yanking
SYSTEM
Y- copy line into buffer
3Y - copy 3 lines into buffer
p - copy buffer below cursor
P - copy buffer above cursor
PROGRAMMING
Save and quit
:w - to save
:w! - to name a file (:w! filename -> save as)
IN LINUX
:x - save and quit
:q - cancel changes
:q! - cancel and quit
| U. K. Roy | [ ]
System Programming in Linux 208
/pat
cursor SYSTEM
searches for the pattern pat and places
PROGRAMMING
/ repeat last search
:#,#s/old/new/g
numbers of IN LINUX where #,# are replaced with the
the two lines.
| U. K. Roy | [ ]
System Programming in Linux 209
to …
SYSTEM
In this session, you have learned how
PROGRAMMING
use directory-related commands namely mkdir, rmdir, cd
commands
use file-related commands namely cp, mv, rm commands
access advanced file permissions using commands umask,
suid, sgid, linking the files, stickybit
IN LINUX
create and edit files using the vi editor
| U. K. Roy | [ ]
Chapter 4
UNIX Utilities
System Programming in Linux 211
SYSTEM
PROGRAMMING
• sort, grep, cut, head, tail, tr, and paste
use communication commands
• telnet, ftp
use backup commands
IN LINUX
• zip/gzip and tar
| U. K. Roy | [ ]
System Programming in Linux 212
SYSTEM
sends the output to the monitor
PROGRAMMING
$ cat>file1
Type the content here
press <ctrl d>
$ cat file1
IN LINUX
Displays the content of the file
$cat>>file1 (will append to the content of the
file)
| U. K. Roy | [ ]
System Programming in Linux 213
SYSTEM
touch is used to change the time stamp of the file
PROGRAMMING
a to change the access time
m to change the modification time
c no create if not exists
touch <file> will change the time of change of the file if the file
exists
IN LINUX
If the file does not exist, it will create a file of zero byte size.
| U. K. Roy | [ ]
System Programming in Linux 214
SYSTEM
screen
echo “This is an example”
This is an example
PROGRAMMING
x=10
echo $x
10
IN LINUX
read command allows to read input from user and
assign it to the variable specified.
read x
| U. K. Roy | [ ]
System Programming in Linux 215
more
file
time. SYSTEM
Allows user to view one page-full of information at a
PROGRAMMING
tty
IN LINUX
| U. K. Roy | [ ]
System Programming in Linux 216
wc
SYSTEM
A filter used to count the number of lines, words, and
characters in a disk file or from the standard input.
-l - displays the number of lines
-w - displays the number of words
PROGRAMMING
-c - displays the number of characters
IN LINUX
| U. K. Roy | [ ]
System Programming in Linux 217
cmp
SYSTEM
Returns the offset and the line number of the first
position where the two files differ.
comm
PROGRAMMING
col1 - unique lines of first file
col2 - unique lines of second file
col3 - common lines
IN LINUX
| U. K. Roy | [ ]
System Programming in Linux 218
diff
SYSTEM
Indicate the differences between the files
a Lines added
d Lines deleted
c Lines changed
PROGRAMMING
IN LINUX
| U. K. Roy | [ ]
System Programming in Linux 219
SYSTEM
directories based on various criteria
Syntax: find [path...] [expression]
[path]
PROGRAMMING
where to search
[expression]
what type of file to search (specified with –type option)
What action to be applied (–exec, –print, etc.)
Name of the files (specified as part of –name option, enclosed in
“ “)
Example IN LINUX
find . –name “*.c” -print
| U. K. Roy | [ ]
System Programming in Linux 220
SYSTEM
– size [+ –]n[bc]
PROGRAMMING
• find . –size 1000c
exactly 1000 bytes in size
• find . –size +1000c
lists all files that are
IN LINUX
than 1000 bytes in size
| U. K. Roy | [ ]
System Programming in Linux 221
SYSTEM
modified time (mtime)
– atime [+-]n
– mtime [+-]n
PROGRAMMING
• n represents number of days ( actually 24 * n hours)
IN LINUX
2 days ago
• find / –mtime –2 lists files modified less than
2 days ago
| U. K. Roy | [ ]
System Programming in Linux 222
SYSTEM
exec and –ok options
– exec command {} \;
PROGRAMMING
(does not prompt user)
• find . -name “*.dat” –exec ls –l {} \;
• Long listing of all files with .dat extension in the current and
its subdirectories
-ok command {} \;
IN LINUX
• Functionality is similar to –exec, but prompts user before
applying the command on the file matching the criteria.
| U. K. Roy | [ ]
System Programming in Linux 223
• SYSTEM
Used to display a file in a format to be printed.
Breaks up a file into pages with a header, text and
footer area
PROGRAMMING
• Options
-l to alter the length of the file
-h to set the header
-t to suppress the header and the footer
IN LINUX
-n to set the line number
| U. K. Roy | [ ]
System Programming in Linux 224
Standard files
Advanced Java Programming
SYSTEM
Keyboard, file descriptor is 0
PROGRAMMING
Standard Error file
Monitor, file descriptor is 2
IN LINUX
| U. K. Roy | [ ]
System Programming in Linux 225
I/O redirection
Advanced Java Programming
SYSTEM
> file redirect standard output to file
2> file redirect standard error to file
2>&1 merge standard error with standard
output
PROGRAMMING
$ cat > abc
$ ls –l > outfile
IN LINUX
$ cat xyz abc > outfile 2>&1
| U. K. Roy | [ ]
System Programming in Linux 226
SYSTEM
standard input file, process it, and sends it to the
standard output file.
PROGRAMMING
sort
grep
cut
head
IN LINUX
tail
paste
| U. K. Roy | [ ]
System Programming in Linux 227
SYSTEM
Sorts the contents of the given file based on the first
char of each line.
-n --numeric sort
-r --Reverse sort
-t -Specify delimiter for fields
PROGRAMMING
IN LINUX
can specify the characters in the field
| U. K. Roy | [ ]
System Programming in Linux 228
SYSTEM
grep -Global Regular Expression Printer is used for
searching regular expressions
Syntax
grep <options> <pattern> <filename(s)>
PROGRAMMING
IN LINUX
| U. K. Roy | [ ]
System Programming in Linux 229
grep options
Advanced Java Programming
-n
SYSTEM
displays line numbers along with the lines
PROGRAMMING
-v displays all lines except lines matching
pattern
IN LINUX
| U. K. Roy | [ ]
System Programming in Linux 230
r
SYSTEM
[^pqr] - Matches a single character which is not p ,q or
PROGRAMMING
pqr$ -Matches pqr at the end of the line
filename IN LINUX
\ - ignores the special meaning. grep “New\[abc\]”
| U. K. Roy | [ ]
System Programming in Linux 231
SYSTEM
Displays the top n lines of the file
$ head -3 file1
PROGRAMMING
IN LINUX
| U. K. Roy | [ ]
System Programming in Linux 232
$ tail -3 file1
PROGRAMMING
Can also specify the line number from which the data
has to be displayed
$ tail +5 file1
IN LINUX
| U. K. Roy | [ ]
System Programming in Linux 233
Filter command - tr
Advanced Java Programming
SYSTEM
tr - translate filter used to translate a given set of
characters
usage :
PROGRAMMING
tr [a-z] [A-Z] <filename
IN LINUX
| U. K. Roy | [ ]
System Programming in Linux 234
Filter command - tr
Advanced Java Programming
SYSTEM
Useful options for tr
-s char
Squeeze multiple contiguous occurrences of the character
PROGRAMMING
into single char
-d char
Remove the character
IN LINUX
| U. K. Roy | [ ]
System Programming in Linux 235
Command piping
Advanced Java Programming
SYSTEM
• Allows the output (only the standard output) of a
command to be sent as input to another command.
PROGRAMMING
Example:
$ cat * | wc
IN LINUX
$ cat fil1 | head | wc -l
| U. K. Roy | [ ]
System Programming in Linux 236
•
SYSTEM
standard output, as well as to a file
• PROGRAMMING
Example
• who | tee userlist
• cat - | tee file1 | wc -l
IN LINUX
| U. K. Roy | [ ]
System Programming in Linux 237
SYSTEM
of certain commands
-c used to extract characters
-d Delimiter for fields
-f Field no.
PROGRAMMING
Examples
cut -c2-5 file1
cut -d “|” -f2,3 file1
IN LINUX
| U. K. Roy | [ ]
System Programming in Linux 238
SYSTEM
vertically
-s Pastes the contents of file2 below file1
PROGRAMMING
-d Specify delimiter
$ paste -d”|” file1 file2
IN LINUX
| U. K. Roy | [ ]
System Programming in Linux 239
SYSTEM
telnet hostname
or
telnet <ip address>
PROGRAMMING
IN LINUX
| U. K. Roy | [ ]
System Programming in Linux 240
ftp hostname
SYSTEM
put/send: Transfer to Other hosts
put [local_file [remote_file]]
PROGRAMMING
get/recv: Transfer From Other hosts
IN LINUX
mput: Multiple Transfer to Other hosts
| U. K. Roy | [ ]
System Programming in Linux 241
Compression utilities
Advanced Java Programming
gzip,
SYSTEM
Usage is very similar to compress and pack utilities in Unix:
gzip [-vc] filename
PROGRAMMING
where -v displays the compression ratio.
-c sends the compressed output to standard output and leaves
the original file intact.
gunzip
IN LINUX
gunzip can uncompress files originally compressed with compress.
| U. K. Roy | [ ]
System Programming in Linux 242
SYSTEM
• Tar is an archiving utility to store and retrieve files
from an archive, known as tarfile.
PROGRAMMING
– tar c|t|x [vf destination] source...
Examples:
IN LINUX
Create a new tar file containing all .dat files (assuming
a.dat, b.dat and c.dat exist)
$ tar –cf mytar *.dat
| U. K. Roy | [ ]
System Programming in Linux 243
SYSTEM
• use the Unix Utilities like
• cat, echo, touch, more, file, wc, cmp,
comm, find
• employ redirection operators
PROGRAMMING
• use filters like
• sort, grep, cut, head, tail, tr, and paste
• communication commands
• telnet, ftp
IN LINUX
• backup commands
• zip/gzip and tar
| U. K. Roy | [ ]
Chapter 5
Process
System Programming in Linux 245
• SYSTEM
Define a process
Use process-related commands like
• ps, kill, sleep
• Start a Background process
PROGRAMMING
• Use background and foreground-related commands
like
• bg, fg, jobs , nice , nohup
IN LINUX
| U. K. Roy | [ ]
System Programming in Linux 246
•
SYSTEM
Process - a program in execution
•
PROGRAMMING
The process is alive till the execution of the program
is complete.
IN LINUX
•
| U. K. Roy | [ ]
System Programming in Linux 247
SYSTEM
executes the login shell.
PROGRAMMING
IN LINUX
| U. K. Roy | [ ]
System Programming in Linux 248
SYSTEM
•
process
• It fetches the pid, tty, time, and the command which has
started the process.
PROGRAMMING
-f lists the pid of the parent process also.
-u lists the processes of a given user
-a lists the processes of all the users
-e lists all the processes including the system
processes
IN LINUX
| U. K. Roy | [ ]
System Programming in Linux 249
Background process
Advanced Java Programming
SYSTEM
Enables the user to do more than one task at a time.
If the command terminates with an ampersand (&),
UNIX executes the command in the background
Shell returns by displaying the process ID (PID) and job
PROGRAMMING
id of the process
IN LINUX
| U. K. Roy | [ ]
System Programming in Linux 250
Background process
Advanced Java Programming
nohup
SYSTEM
Lets processes to continue to run even after logout
The output of the command is sent to nohup.out if not
redirected
PROGRAMMING
$ nohup command args
| U. K. Roy | [ ]
System Programming in Linux 251
Background process
Advanced Java Programming
wait command
SYSTEM
can be used when a process has to wait for the output
of a background process
The wait command, can be used to let the shell wait
PROGRAMMING
for all background processes terminate.
$ wait
It is possible to wait for completion of one specific
process as well.
IN LINUX
| U. K. Roy | [ ]
System Programming in Linux 252
• jobs
•
SYSTEM
List the background process
• fg % <job id>
PROGRAMMING
• Runs a process in the foreground
• bg %<job id>
• Runs a process in the background
IN LINUX
| U. K. Roy | [ ]
System Programming in Linux 253
Process priority
Advanced Java Programming
SYSTEM
• nice
• Used to reduce the priority of jobs
PROGRAMMING
IN LINUX
| U. K. Roy | [ ]
System Programming in Linux 254
•
SYSTEM
kill: Kills or terminates a process
PROGRAMMING
• kill -9 (SIGKILL)
• Terminates the process abruptly
IN LINUX
| U. K. Roy | [ ]
System Programming in Linux 255
• SYSTEM
Define a process
Use process-related commands like
• ps, kill, sleep
Start a background process
PROGRAMMING
•
IN LINUX
| U. K. Roy | [ ]
Chapter 6
•
SYSTEM
Use Shell variables
Write scripts to process positional parameters
Use “test” command
• Use “if” construct
PROGRAMMING
• Use “for” loop
• Use “while” loop
• Use “case” construct
• Define and use functions
IN LINUX
• Debug shell scripts
| U. K. Roy | [ ]
System Programming in Linux 258
SYSTEM
• Bourne shell sh
• C shell csh
PROGRAMMING
• Korn shell ksh
IN LINUX
| U. K. Roy | [ ]
System Programming in Linux 259
Command processing
Advanced Java Programming
•
by the user. SYSTEM
Displays the shell prompt and reads the command typed
PROGRAMMING
(built-in), or an external command.
IN LINUX
| U. K. Roy | [ ]
System Programming in Linux 260
Shell features
Advanced Java Programming
SYSTEM
Parent shell process
(bash)
$ vi test.c
command typed by
fork user
PROGRAMMING
Child shell process
Exec of “vi test.c”
(bash) User
Mode
Kernel
IN LINUX
Mode
| U. K. Roy | [ ]
System Programming in Linux 261
•
SYSTEM
Each shell, apart from the basic features, provides additional
features such as:
PROGRAMMING
Maintaining command history (C, korn and bash)
IN LINUX
Programming language (all shells)
| U. K. Roy | [ ]
System Programming in Linux 262
SYSTEM
• some UNIX shells support command history
• facility to keeps track of commands that were executed
• facility to rerun previously executed commands
• bash shell supports the following
PROGRAMMING
!! recall the last command and execute it.
IN LINUX
the num specified after !
| U. K. Roy | [ ]
System Programming in Linux 263
SYSTEM
• alias can be used to give new name to an existing command
• A better name that represents a single command or a sequence of
commands to be executed, often with appropriate options
• alias is an internal command
PROGRAMMING
alias newname=command
IN LINUX
The unalias command cancels previously defined alias.
| U. K. Roy | [ ]
System Programming in Linux 264
SYSTEM
•
the string into following components:
• Command (the first part of the string, till the first
space char)
Command arguments (the subsequent parts of
PROGRAMMING
•
the string)
SYSTEM
• In arguments of a command, the shell recognizes
certain characters – such as *, ?, [ ], and - as
special characters and expands these characters
into a filename list before executing the command.
PROGRAMMING
• To see how this works, enter following commands
while in /bin directory
$ ls a*
$ ls ??
IN LINUX
| U. K. Roy | [ ]
System Programming in Linux 266
Shell programming
Advanced Java Programming
• Allows
•
• SYSTEM
Defining and referencing variables
Logic control structures such as if, for,
while, case
• Input and output
PROGRAMMING
IN LINUX
| U. K. Roy | [ ]
System Programming in Linux 267
Shell variables
Advanced Java Programming
SYSTEM
offers a symbolic way to represent and manipulate data
variables in the shell. They are classified as follows
• user-defined variables
• environment variables
•
PROGRAMMING
predefined variables
IN LINUX
| U. K. Roy | [ ]
System Programming in Linux 268
Shell variables
Advanced Java Programming
•
variables.
SYSTEM
The shell provides the facility to define normal, and environment
•
PROGRAMMING
An environment variable can be used in the shell where it is
defined, plus any child shells invoked from that shell.
IN LINUX
| U. K. Roy | [ ]
System Programming in Linux 269
SYSTEM
syntax:
variable_name=value
• Examples:
PROGRAMMING
$ x=10
$ textline_1=‘This line was entered by
$USER’
$ textline_2=“This line was entered by
$USER”
IN LINUX
$ allusers=`who`
$ usercount=`who | wc –l`
| U. K. Roy | [ ]
System Programming in Linux 270
SYSTEM
• Once variables are defined, one can use the echo
command to display the value of each variable:
$ echo $x
$ echo $textline_1
PROGRAMMING
$ echo $textline_2
$ echo $allusers
$ echo $usercount
IN LINUX
| U. K. Roy | [ ]
System Programming in Linux 271
•
SYSTEM
To define an environment variable, use following syntax:
variable_name=value
• export variable_name
•
PROGRAMMING
Examples:
• $ x=10; export x
• $ allusers=`who` ; export allusers
IN LINUX
| U. K. Roy | [ ]
System Programming in Linux 272
• PATH
• BASH_ENV SYSTEM • MAIL
• USER
• HOME • LOGNAME
PROGRAMMING
• PWD • PS1
• SHELL • PS2
• TERM
IN LINUX
| U. K. Roy | [ ]
System Programming in Linux 273
#! /bin/bash
#
SYSTEM
# The above line has a special meaning. It must be the
# first line of the script. It says that the commands in
PROGRAMMING
# this shell script should be executed by the bash
# shell (/bin/bash).
# ---------------------------------------------------------------
echo “Hello $USER….”
echo “Welcome to programming shell scripts..”
# ---------------------------------------------------------------
IN LINUX
| U. K. Roy | [ ]
System Programming in Linux 274
SYSTEM
By passing the shell script name as an argument
to the shell. For example:
PROGRAMMING
sh script1.sh
./script1.sh IN LINUX
| U. K. Roy | [ ]
System Programming in Linux 275
•
SYSTEM
parameter can be passed to a shell script
• parameters are specified after the name of the shell script when
invoking the script.
•
PROGRAMMING
Within the shell script, parameters are referenced using the
predefined variables $1 through $9.
IN LINUX
| U. K. Roy | [ ]
System Programming in Linux 276
Built-in variables
Advanced Java Programming
• SYSTEM
Following are built-in variables
supported
• $0, $1…$9 - positional arguments
PROGRAMMING
• $* - all arguments
• $@ - all arguments
IN LINUX
| U. K. Roy | [ ]
System Programming in Linux 277
SYSTEM
• Consider following shell script:
----------------------script2.sh--------------------------
echo “Total parameters entered: $#”
echo “First parameter is : $1”
PROGRAMMING
echo “The parameters are: $*”
shift
echo “First parameter is : $1”
------------------------------------------------------------
IN LINUX
• Execute the above script using the “script2.sh these
are the parameters” command.
| U. K. Roy | [ ]
System Programming in Linux 278
SYSTEM
The shell parameters are passed as strings.
•
PROGRAMMING
For example,
IN LINUX
| U. K. Roy | [ ]
System Programming in Linux 279
Example,
SYSTEM
performed using expr command.
• x=10
y=5
PROGRAMMING
•
IN LINUX
number_4 = `expr $x \* $y`
number_5 = `expr $x % $y`
| U. K. Roy | [ ]
System Programming in Linux 280
SYSTEM
test <expression>
PROGRAMMING
• The expression can be formed using a
combination of shell variables and the operators
supported by the test command. These operators
provide facility to compare numbers, string and
logical values, file types and file access modes.
IN LINUX
| U. K. Roy | [ ]
System Programming in Linux 281
available:
SYSTEM
To compare two integers using test following operators are
PROGRAMMING
-ne (not equal to)
-lt (less than)
-le (less than or equal to)
-gt (greater than)
-ge (greater than or equal to)
IN LINUX
| U. K. Roy | [ ]
System Programming in Linux 282
SYSTEM
Please note that you can only compare two integers using test.
General syntax
PROGRAMMING
test [expression]
IN LINUX
| U. K. Roy | [ ]
System Programming in Linux 283
SYSTEM
To compare two strings using the test command, following
operators are available:
PROGRAMMING
string1 != string2 (not equal to)
string1 (string is not NULL)
-n string1 (string is not NULL and exists)
-z string1 (string is NULL and exists)
IN LINUX
| U. K. Roy | [ ]
System Programming in Linux 284
SYSTEM
The syntax for this string comparison is:
PROGRAMMING
[ string1 operator string2 ]
OR
test operator string
OR
[ operator string ]
IN LINUX
| U. K. Roy | [ ]
System Programming in Linux 285
SYSTEM
To check a file type/access permissions using the
test command, following operators are available:
PROGRAMMING
-f file (Ordinary file and exists)
-d file (file is a directory and exists)
-r file (file is readable and exists)
-w file (file is write-able and exists)
IN LINUX
-x file (file is executable and exists)
| U. K. Roy | [ ]
System Programming in Linux 286
SYSTEM
test command, following operators are available:
PROGRAMMING
-p file (file is a named pipe and exists)
-g file (file has sticky bit set)
-u file (file has setuid bit set)
IN LINUX
-t file_des (file descriptor is standard
output)
| U. K. Roy | [ ]
System Programming in Linux 287
Combining conditions
Advanced Java Programming
SYSTEM
It is possible to combine conditions by using
following operators:
PROGRAMMING
-o (logical OR operator)
! (logical NOT operator)
IN LINUX
| U. K. Roy | [ ]
System Programming in Linux 288
Combining conditions
Advanced Java Programming
OR SYSTEM
test expression_1 –a expression _2,
[ expression _1 –a expression _2 ]
PROGRAMMING
test expression_1 –o expression _2,
OR
[ expression_1 –o expression_2 ]
test ! expression _1
OR
IN LINUX
[ ! expression _1 ]
| U. K. Roy | [ ]
System Programming in Linux 289
SYSTEM
Bash shell provides the if command to test if a condition is true.
The general format of this command is:
if condition
PROGRAMMING
then
command
fi
IN LINUX
The condition is typically formed using the test command.
| U. K. Roy | [ ]
System Programming in Linux 290
directory
SYSTEM
# to check if the current directory is the same as your home
curdir=`pwd`
if test “$curdir” != “$HOME”
PROGRAMMING
then
echo your home dir is not the same as your pesent
working directory
else
echo $HOME is your current directory
fi
IN LINUX
| U. K. Roy | [ ]
System Programming in Linux 291
SYSTEM
The complex form of if statement is as follows:
if condition_1
then
PROGRAMMING
command
elif condition_2
then
command
else
fi
command
IN LINUX
| U. K. Roy | [ ]
System Programming in Linux 292
for loop. SYSTEM
The Bash shell provides a
Example:
PROGRAMMING do
IN LINUX
$i `
… done
command
done
| U. K. Roy | [ ]
System Programming in Linux 293
SYSTEM
----------------------script.sh--------------------------
#! /bin/sh
usernames=`who | cut –d “ “ –f1`
echo “Total users logged in = $#usernames”
PROGRAMMING
#
for user in ${usernames}
do
echo $user
done
IN LINUX
------------------------------------------------------------
| U. K. Roy | [ ]
System Programming in Linux 294
SYSTEM
The Bash shell provides a while loop. The syntax of this loop is:
while condition
do
PROGRAMMING command
…
command
done
IN LINUX
| U. K. Roy | [ ]
System Programming in Linux 295
eg: read nam
SYSTEM
Shell script checks for a blank/non blank string
while [ -z “$nam” ]
do
PROGRAMMING
read nam
done
echo the string is $nam
IN LINUX
| U. K. Roy | [ ]
System Programming in Linux 296
#!/bin/bash
SYSTEM
Shell script to compute factorial of a given number
n=$1
PROGRAMMING
if [ $n -eq 0 ]; then
fact=0
else
fact=1
while [ $n –ne 0 ]
do
IN LINUX
fact=`expr $fact \* $n`
n=`expr $n – 1`
done
fi
echo $fact
| U. K. Roy | [ ]
System Programming in Linux 297
case statement
Advanced Java Programming
SYSTEM
- THE CASE STATEMENT
case value in
pattern1)
command
PROGRAMMING
command;;
pattern2)
command
command;;
patternn)
IN LINUX
command;;
esac
| U. K. Roy | [ ]
System Programming in Linux 298
SYSTEM
echo enter a choice
read choice
case $choice in
1) echo enter 2 nos
PROGRAMMING
read num1
read num2
res=`expr $num1 + $num2`
echo result is $res;;
2) who;;
IN LINUX
*) echo invalid choice;;
esac
| U. K. Roy | [ ]
System Programming in Linux 299
PROGRAMMING
2) echo 2nd
break;;
3) echo 3rd
break;;
*) echo ${number}th
break;;
IN LINUX
esac
| U. K. Roy | [ ]
System Programming in Linux 300
SYSTEM
Shell functions are a way to group commands for later execution
using a single name for the group. They are executed just like a
"regular" command.
PROGRAMMING
Shell functions are executed in the current shell context; no new
process is created to interpret them. Functions are declared
using this syntax:
IN LINUX
| U. K. Roy | [ ]
System Programming in Linux 301
SYSTEM
Shell functions can accept arguments
Arguments are passed in the same way as given to commands
Functions refer to arguments using $1, $2 etc., similar to the
way shell scripts refer to command line arguments
PROGRAMMING
IN LINUX
| U. K. Roy | [ ]
System Programming in Linux 302
SYSTEM
Function to convert standard input into upper case
toupper()
{
PROGRAMMING
tr A-Z a-z
}
IN LINUX
$ cat abc | toupper
| U. K. Roy | [ ]
System Programming in Linux 303
SYSTEM
Two options help in debugging shell scripts
“-v” (verbose) option:
causes shell to print the lines of the script as
they are read.
PROGRAMMING
$ bash –v script-file
IN LINUX
prints commands and their arguments as they
are executed.
$ bash –x script-file
| U. K. Roy | [ ]
System Programming in Linux 304
Programming in C vs shell
Advanced Java Programming
Comparison between
SYSTEM
• A solution in C
• A shell solution written like a C program
• A proper “shell/unix” solution
PROGRAMMING
e.g:
The problem is to count the no of lines in a file ( the
file is called the_file)
IN LINUX
| U. K. Roy | [ ]
Programming in C vs shell
SYSTEM
Use Shell variables
•
PROGRAMMING
•
•
Use “for” loop
Use “while” loop
• Use “case” construct
IN LINUX
Define and use functions
•
| U. K. Roy | [ ]
System Programming in Linux 307
Bibliography
Advanced Java Programming
SYSTEM
Thank You
PROGRAMMING
IN LINUX
| U. K. Roy | [ ]