Inter-Process Communications (IPC)

ICS332 — Operating Systems

Henri Casanova ([email protected])

Spring 2018

Henri Casanova ([email protected]) Inter-Process Communications (IPC)

Communicating Processes?

So far we have seen independent processes

Each process runs code independently
Parents and aware of their children, and children are aware of
their parents, but they do not interact
Besides the ability to wait for a process termination

Henri Casanova ([email protected]) Inter-Process Communications (IPC)

Communicating Processes?

So far we have seen independent processes

Each process runs code independently
Parents and aware of their children, and children are aware of
their parents, but they do not interact
Besides the ability to wait for a process termination
But often we need processes to cooperate
To share information (e.g., access to common data)
To speed up computation (e.g., to use multiple cores)
Because it’s convenient (e.g., some applications are naturally
implemented as sets of interacting processes)

Henri Casanova ([email protected]) Inter-Process Communications (IPC)

Communicating Processes?

So far we have seen independent processes

Each process runs code independently
Parents and aware of their children, and children are aware of
their parents, but they do not interact
Besides the ability to wait for a process termination
But often we need processes to cooperate
To share information (e.g., access to common data)
To speed up computation (e.g., to use multiple cores)
Because it’s convenient (e.g., some applications are naturally
implemented as sets of interacting processes)
In general, the means of communication between cooperating
processes is called Inter-Process Communication (IPC)

Henri Casanova ([email protected]) Inter-Process Communications (IPC)

Communication Models

Example: Process A needs to communicate with Process B

Kernel Kernel

Process A Process A

Process B Process B

Available Available
Memory Memory

Henri Casanova ([email protected]) Inter-Process Communications (IPC)

Communication Models

Example: Process A needs to communicate with Process B

Message Passing
Kernel Kernel

Process A Process A
M

Process B Process B

Available Available
Memory Memory

Henri Casanova ([email protected]) Inter-Process Communications (IPC)

Communication Models

Example: Process A needs to communicate with Process B

Message Passing
Kernel M Kernel

Process A Process A
M

Process B Process B

Available Available
Memory Memory

Henri Casanova ([email protected]) Inter-Process Communications (IPC)

Communication Models

Example: Process A needs to communicate with Process B

Message Passing
Kernel M Kernel

Process A Process A
M

Process B Process B
M

Available Available
Memory Memory

Henri Casanova ([email protected]) Inter-Process Communications (IPC)

Communication Models

Example: Process A needs to communicate with Process B

Message Passing
Kernel M Kernel

Process A Process A
M

Process B Process B
M

Available Available
Memory Memory
Shared Memory

Henri Casanova ([email protected]) Inter-Process Communications (IPC)

Communication Models

Example: Process A needs to communicate with Process B

Message Passing Shared Memory
Kernel M Kernel

Process A Process A
M

Process B Process B
M

Available Available
Memory Memory
Shared Memory

Henri Casanova ([email protected]) Inter-Process Communications (IPC)

Pros and Cons
Message Passing
Performed through the kernel
memory space

Henri Casanova ([email protected]) Inter-Process Communications (IPC)

Pros and Cons
Message Passing
Performed through the kernel
memory space
Simple to implement
(pre-defined region in
memory)

Henri Casanova ([email protected]) Inter-Process Communications (IPC)

Pros and Cons
Message Passing
Performed through the kernel
memory space
Simple to implement
(pre-defined region in
memory)
Limited by kernel size ⇒
Small messages

Henri Casanova ([email protected]) Inter-Process Communications (IPC)

Pros and Cons
Message Passing
Performed through the kernel
memory space
Simple to implement
(pre-defined region in
memory)
Limited by kernel size ⇒
Small messages
One system call per
communication operation,
i.e., one send, one receive:
high overhead

Henri Casanova ([email protected]) Inter-Process Communications (IPC)

Pros and Cons
Message Passing
Performed through the kernel
memory space
Simple to implement
(pre-defined region in
memory)
Limited by kernel size ⇒
Small messages
One system call per
communication operation,
i.e., one send, one receive:
high overhead
Cumbersome for developers:
code will be sprinkled with
send and receive everywhere

Henri Casanova ([email protected]) Inter-Process Communications (IPC)

Pros and Cons
Message Passing
Shared Memory
Performed through the kernel
memory space Performed using available
memory
Simple to implement
(pre-defined region in
memory)
Limited by kernel size ⇒
Small messages
One system call per
communication operation,
i.e., one send, one receive:
high overhead
Cumbersome for developers:
code will be sprinkled with
send and receive everywhere

Henri Casanova ([email protected]) Inter-Process Communications (IPC)

Pros and Cons
Message Passing
Shared Memory
Performed through the kernel
memory space Performed using available
memory
Simple to implement
(pre-defined region in Large messages allowed (only
memory) limited by physical memory)

Limited by kernel size ⇒

Small messages
One system call per
communication operation,
i.e., one send, one receive:
high overhead
Cumbersome for developers:
code will be sprinkled with
send and receive everywhere

Henri Casanova ([email protected]) Inter-Process Communications (IPC)

Pros and Cons
Message Passing
Shared Memory
Performed through the kernel
memory space Performed using available
memory
Simple to implement
(pre-defined region in Large messages allowed (only
memory) limited by physical memory)

Limited by kernel size ⇒ Violates the principle of

Small messages memory protection between
processes
One system call per
communication operation,
i.e., one send, one receive:
high overhead
Cumbersome for developers:
code will be sprinkled with
send and receive everywhere

Henri Casanova ([email protected]) Inter-Process Communications (IPC)

Pros and Cons
Message Passing
Shared Memory
Performed through the kernel
memory space Performed using available
memory
Simple to implement
(pre-defined region in Large messages allowed (only
memory) limited by physical memory)

Limited by kernel size ⇒ Violates the principle of

Small messages memory protection between
processes
One system call per
communication operation, More difficult to implement:
i.e., one send, one receive: processes need to be aware of
high overhead the shared memory region’s
location
Cumbersome for developers:
code will be sprinkled with
send and receive everywhere

Henri Casanova ([email protected]) Inter-Process Communications (IPC)

Pros and Cons
Message Passing
Shared Memory
Performed through the kernel
memory space Performed using available
memory
Simple to implement
(pre-defined region in Large messages allowed (only
memory) limited by physical memory)

Limited by kernel size ⇒ Violates the principle of

Small messages memory protection between
processes
One system call per
communication operation, More difficult to implement:
i.e., one send, one receive: processes need to be aware of
high overhead the shared memory region’s
location
Cumbersome for developers:
code will be sprinkled with Easy for developers: A few
send and receive everywhere system calls to allocate the
shared region, and then just
read/write to it

Henri Casanova ([email protected]) Inter-Process Communications (IPC)

Message Passing

There are two fundamental system calls: send and receive

Although we’re talking about communication within a
machine here, many of the design options are similar to some
questions in the field of networking:
Fixed or variable length messages, ...
Uni-directional or bi-directional link?
Automatic or explicit buffering?
Direct or indirect communication?
Synchronous or asynchronous communication?x
...

Henri Casanova ([email protected]) Inter-Process Communications (IPC)

Message Passing

There are two fundamental system calls: send and receive

Although we’re talking about communication within a
machine here, many of the design options are similar to some
questions in the field of networking:
Fixed or variable length messages, ...
Uni-directional or bi-directional link?
Automatic or explicit buffering?
Direct or indirect communication?
Synchronous or asynchronous communication?x
...
Picking options above is about making trade-offs between:
Ease of implementation in the kernel (will it be bug-free and
maintainable?)
Convenience to users (will they like using it)
Expressiveness (can users do everything they want with it?)
Performance (is it fast? is it memory-efficient?)

Henri Casanova ([email protected]) Inter-Process Communications (IPC)

A Word on API Design
In your professional lives you’ll use and define APIs
You probably already have encountered APIs that you liked,
and APIs that you disliked?
It’s often not easy to pinpoint the flaws of an API

Henri Casanova ([email protected]) Inter-Process Communications (IPC)

A Word on API Design
In your professional lives you’ll use and define APIs
You probably already have encountered APIs that you liked,
and APIs that you disliked?
It’s often not easy to pinpoint the flaws of an API
API design has deep implications (difficult to foresee problems;
dramatic snowball effect; can lead to costly full rewrites; ...)
It is thus worth spending a lot of time defining good APIs
Being good at designing APIs (and thus abstractions) is an
invaluable skill, which comes with experience

Henri Casanova ([email protected]) Inter-Process Communications (IPC)

A Word on API Design
In your professional lives you’ll use and define APIs
You probably already have encountered APIs that you liked,
and APIs that you disliked?
It’s often not easy to pinpoint the flaws of an API
API design has deep implications (difficult to foresee problems;
dramatic snowball effect; can lead to costly full rewrites; ...)
It is thus worth spending a lot of time defining good APIs
Being good at designing APIs (and thus abstractions) is an
invaluable skill, which comes with experience
Pedagogic challenge: Conveying to college students how
important/crucial this is, when initially it all seems like a
bunch of pointless nitpicking
You wouldn’t believe the number of hours spent daily on
minuscule API details in the software industry; because you
haven’t yet experienced the above “snowball effect” of your
poorly designed API

Henri Casanova ([email protected]) Inter-Process Communications (IPC)

A Word on API Design
In your professional lives you’ll use and define APIs
You probably already have encountered APIs that you liked,
and APIs that you disliked?
It’s often not easy to pinpoint the flaws of an API
API design has deep implications (difficult to foresee problems;
dramatic snowball effect; can lead to costly full rewrites; ...)
It is thus worth spending a lot of time defining good APIs
Being good at designing APIs (and thus abstractions) is an
invaluable skill, which comes with experience
Pedagogic challenge: Conveying to college students how
important/crucial this is, when initially it all seems like a
bunch of pointless nitpicking
You wouldn’t believe the number of hours spent daily on
minuscule API details in the software industry; because you
haven’t yet experienced the above “snowball effect” of your
poorly designed API
But let’s try anyway in the context of IPCs...
Henri Casanova ([email protected]) Inter-Process Communications (IPC)
Direct vs. Indirect Communication
Direct Communication: sends and receives
When sending, the message target is explicit:
e.g., send(Message message, Process targetProcess);
When receiving, the message source is explicit:
e.g., Message ← recv(Process sourceProcess);

Henri Casanova ([email protected]) Inter-Process Communications (IPC)

Direct vs. Indirect Communication
Direct Communication: sends and receives
When sending, the message target is explicit:
e.g., send(Message message, Process targetProcess);
When receiving, the message source is explicit:
e.g., Message ← recv(Process sourceProcess);
Indirect Communication: mailboxes
A mailbox is an opaque ID
One sends and receive to/from a mailbox: e.g., send(Message
message, String mailbox);
e.g., Message ← recv(String mailbox);

Henri Casanova ([email protected]) Inter-Process Communications (IPC)

Direct vs. Indirect Communication
Direct Communication: sends and receives
When sending, the message target is explicit:
e.g., send(Message message, Process targetProcess);
When receiving, the message source is explicit:
e.g., Message ← recv(Process sourceProcess);
Indirect Communication: mailboxes
A mailbox is an opaque ID
One sends and receive to/from a mailbox: e.g., send(Message
message, String mailbox);
e.g., Message ← recv(String mailbox);
Believe it or not, the above really matters, e.g.:
With direct communication, you must know the process that
will receive the message (which must be running already)
With indirect, who receives the message can be decided well
after sending
But if two processes want to receive from a mailbox “at the
same time”, which one gets the message (and what about
message ordering?)
...
Henri Casanova ([email protected]) Inter-Process Communications (IPC)
Message Passing: Synchronous or asynchronous?

Synchronous = Blocking
Synchronous send: Block until the message is received
Synchronous recv: Block until a message is available
When both send and receive are blocking, the operation is a
rendez-vous

Henri Casanova ([email protected]) Inter-Process Communications (IPC)

Message Passing: Synchronous or asynchronous?

Synchronous = Blocking
Synchronous send: Block until the message is received
Synchronous recv: Block until a message is available
When both send and receive are blocking, the operation is a
rendez-vous
Asynchronous = Non-Blocking
Non-blocking send: Send and continue
Usually comes with the option to check the status later (Was
the message received?)
Non-blocking receive: e.g. Read any number of bytes (possibly
0) or any message (possibly the empty one, or null)

Most OSes propose both options in various ways

Henri Casanova ([email protected]) Inter-Process Communications (IPC)

A “real-life” Story
I realize that the previous couple of slides are probably a bit
abstract and underwhelming, so let me attempts a story from
my own software endeavors
For more than a decade I’ve co-led/co-developed a simulation
project called SimGrid
It’s a simulator of distributed systems running on distributed
compute platforms, used by parallel and distributed computing
researchers
Therefore it offers a process abstraction, and IPC abstractions
Here is a brief history of our IPC API development:
Circa 2005: direct only; synchronous only

Henri Casanova ([email protected]) Inter-Process Communications (IPC)

A “real-life” Story
I realize that the previous couple of slides are probably a bit
abstract and underwhelming, so let me attempts a story from
my own software endeavors
For more than a decade I’ve co-led/co-developed a simulation
project called SimGrid
It’s a simulator of distributed systems running on distributed
compute platforms, used by parallel and distributed computing
researchers
Therefore it offers a process abstraction, and IPC abstractions
Here is a brief history of our IPC API development:
Circa 2005: direct only; synchronous only
Circa 2010: indirect only; synchronous only

Henri Casanova ([email protected]) Inter-Process Communications (IPC)

A “real-life” Story
I realize that the previous couple of slides are probably a bit
abstract and underwhelming, so let me attempts a story from
my own software endeavors
For more than a decade I’ve co-led/co-developed a simulation
project called SimGrid
It’s a simulator of distributed systems running on distributed
compute platforms, used by parallel and distributed computing
researchers
Therefore it offers a process abstraction, and IPC abstractions
Here is a brief history of our IPC API development:
Circa 2005: direct only; synchronous only
Circa 2010: indirect only; synchronous only
Circa 2015: indirect only; synchronous and asynchronous

Henri Casanova ([email protected]) Inter-Process Communications (IPC)

A “real-life” Story
I realize that the previous couple of slides are probably a bit
abstract and underwhelming, so let me attempts a story from
my own software endeavors
For more than a decade I’ve co-led/co-developed a simulation
project called SimGrid
It’s a simulator of distributed systems running on distributed
compute platforms, used by parallel and distributed computing
researchers
Therefore it offers a process abstraction, and IPC abstractions
Here is a brief history of our IPC API development:
Circa 2005: direct only; synchronous only
Circa 2010: indirect only; synchronous only
Circa 2015: indirect only; synchronous and asynchronous
Each of these decisions took hours of design meetings, has
had drastic implications on our design/implementation of
SimGrid, and has had massive implications for our users
Henri Casanova ([email protected]) Inter-Process Communications (IPC)
Communication Models (again)

Example: Process A needs to communicate with Process B

Message Passing Shared Memory
Kernel M Kernel

Process A Process A
M

Process B Process B
M

Available Available
Memory Memory
Shared Memory

Henri Casanova ([email protected]) Inter-Process Communications (IPC)

Shared Memory

IPC is performed outside the kernel space (but still on the

same host)
One process creates a shared memory segment
Other processes can then attach it to their address spaces
(Bye bye memory protection for processes)
It is the processes’ (and therefore the developer’s)
responsibility to make sure that processes are not stepping on
each other’s toes
The OS is not involved: “What happens in Shared Memory
Segments stays in Shared Memory Segments”
Memory is freed by the requester

Henri Casanova ([email protected]) Inter-Process Communications (IPC)

Shared Memory: POSIX API

Implementation in C
SystemV Implementation:
Creation/Request for a new shared memory segment:
id = shmget(IPC PRIVATE, size, IPC R | IPC W)
Attaching a process to the id shared memory segment:
shmsAddress = shmat(id, NULL, 0);
Detach the memory segment:
shmdt(shmsAddress);
Release control of the shared memory segment:
shmctl(idm, IPC RMID, NULL);
When the process is attached:
“sending”: sprintf(shmsAddress, "Hello, World!");
“receiving”: sscanf(shmsAddress, "%s", &message);
Let’s look at the posix shm example.c example

Henri Casanova ([email protected]) Inter-Process Communications (IPC)

Shared Memory needs Message Passing?

How do the non-requesters know the id of the shared memory

segment?
fork(): parent requests and gives it to children (since the
child will have a copy of the parent’s address space)
For non-related processes, their many ways...
Through a file?
Argument on the command-line?
Other Message Passing mechanism?
Shared Memory Segments are flagged by the OS as “Shared
Memory”
Let’s run the ipcs -m command on a Linux box and see if we
find any...

Henri Casanova ([email protected]) Inter-Process Communications (IPC)

Remote Procedure Calls
So far, we’ve viewed messages as unstructured sequences of
bytes: the receiver has to interpret the message to know its
meaning
RPC provides a procedure invocation abstraction across
processes (and actually across machines)
A client invokes a procedure in another process just as it would
invoke it directly
It has a lot of usages, of course for client-server applications
(RPC is a building blocks or microkernels)

Henri Casanova ([email protected]) Inter-Process Communications (IPC)

Remote Procedure Calls
So far, we’ve viewed messages as unstructured sequences of
bytes: the receiver has to interpret the message to know its
meaning
RPC provides a procedure invocation abstraction across
processes (and actually across machines)
A client invokes a procedure in another process just as it would
invoke it directly
It has a lot of usages, of course for client-server applications
(RPC is a building blocks or microkernels)
The “magic” is performed through a client stub (one stub for
each RPC):
Marshal the parameters (structured data to bytes stream)
Send the data over to the server
Wait for the server’s answer
Unmarshal the returned values (bytes stream to structured
data)
A lot of different implementations exist... including in Java
Henri Casanova ([email protected]) Inter-Process Communications (IPC)
RPC à la Java: Remote Method Invocation

RPC in Java: Remote Method Invocation (RMI)

A process in a JVM can invoke a method of an object living in
another JVM
Marshalling/Unmarshalling performed by the JVM
(The class need to implement the java.io.Serializable
interface)
RMI hides all the gory details of RPC/IPC
See this Java RMI Tutorial for more info

Henri Casanova ([email protected]) Inter-Process Communications (IPC)

Remote Procedure Calls: Main Issue
Local procedure calls never fail (i.e., if they reach an error
condition, that error can be locally managed)

Henri Casanova ([email protected]) Inter-Process Communications (IPC)

Remote Procedure Calls: Main Issue
Local procedure calls never fail (i.e., if they reach an error
condition, that error can be locally managed)
Not so easy when execution is remote: there are many
“failure” cases
RPC could be in execution but taking a long time and perhaps
appear stuck
RPC could have partially executed and then failed halfway
through causing the server process to crash
RPC could have successfully executed, but then failed when
replying with some “it worked” message perhaps due to a
network problem (when running across hosts)

Henri Casanova ([email protected]) Inter-Process Communications (IPC)

Remote Procedure Calls: Main Issue
Local procedure calls never fail (i.e., if they reach an error
condition, that error can be locally managed)
Not so easy when execution is remote: there are many
“failure” cases
RPC could be in execution but taking a long time and perhaps
appear stuck
RPC could have partially executed and then failed halfway
through causing the server process to crash
RPC could have successfully executed, but then failed when
replying with some “it worked” message perhaps due to a
network problem (when running across hosts)
What we want is a strong execute exactly once semantic:
When the RPC completes (with perhaps hidden retries), then
you know it’s been executed exactly once successfully, or not
executed at all and failed
This gets us to difficult distributed systems issues, which are
often part of graduate courses...
Henri Casanova ([email protected]) Inter-Process Communications (IPC)
Pipes

One of the most ancient, yet simple, useful, and powerful IPC
mechanism provided by OSes is typically called pipes

Henri Casanova ([email protected]) Inter-Process Communications (IPC)

Pipes

One of the most ancient, yet simple, useful, and powerful IPC
mechanism provided by OSes is typically called pipes

We explore this in a programming assignment, so it’s a good

idea to pay close attention
But first, let’s take a little detour about UNIX file descriptors
and output redirection...

Henri Casanova ([email protected]) Inter-Process Communications (IPC)

stdin, stdout, stderr

In UNIX, every process comes with 3 already opened “files”

Not real files, but in UNIX “everything looks like a file”
These files are:
stdin: the standard input stream
stdout: the standard output stream
stderr: the standard error stream
You’ve encountered these when developing code (C/C++,
Java, Python, etc.)
e.g., printf writes to stdout

Henri Casanova ([email protected]) Inter-Process Communications (IPC)

stdin, stdout, stderr

In UNIX, every process comes with 3 already opened “files”

Not real files, but in UNIX “everything looks like a file”
These files are:
stdin: the standard input stream
stdout: the standard output stream
stderr: the standard error stream
You’ve encountered these when developing code (C/C++,
Java, Python, etc.)
e.g., printf writes to stdout
Each file in UNIX is associated an integer file descriptor
An index into some “this process’ open files” table

Henri Casanova ([email protected]) Inter-Process Communications (IPC)

stdin, stdout, stderr

In UNIX, every process comes with 3 already opened “files”

Not real files, but in UNIX “everything looks like a file”
These files are:
stdin: the standard input stream
stdout: the standard output stream
stderr: the standard error stream
You’ve encountered these when developing code (C/C++,
Java, Python, etc.)
e.g., printf writes to stdout
Each file in UNIX is associated an integer file descriptor
An index into some “this process’ open files” table
By convention, the file descriptors for each standard stream
are (see /usr/include/unistd.h):
stdin: STDIN FILENO = 0
stdout: STDOUT FILENO = 1
stderr: STDERR FILENO = 2

Henri Casanova ([email protected]) Inter-Process Communications (IPC)

Re-directing output
Perhaps some of you have wondered how come something like
ls > file.txt can work?
After all, ls has code that looks like
fprintf(stdout, "%s", filename);
So how can this code magically knows to write to a file
instead of to stdout???

Henri Casanova ([email protected]) Inter-Process Communications (IPC)

Re-directing output
Perhaps some of you have wondered how come something like
ls > file.txt can work?
After all, ls has code that looks like
fprintf(stdout, "%s", filename);
So how can this code magically knows to write to a file
instead of to stdout???
This is one of the famous UNIX “tricks”

Henri Casanova ([email protected]) Inter-Process Communications (IPC)

Re-directing output
Perhaps some of you have wondered how come something like
ls > file.txt can work?
After all, ls has code that looks like
fprintf(stdout, "%s", filename);
So how can this code magically knows to write to a file
instead of to stdout???
This is one of the famous UNIX “tricks”
In UNIX, when I open a new file, this file gets the first
available file descriptor number

Henri Casanova ([email protected]) Inter-Process Communications (IPC)

Re-directing output
Perhaps some of you have wondered how come something like
ls > file.txt can work?
After all, ls has code that looks like
fprintf(stdout, "%s", filename);
So how can this code magically knows to write to a file
instead of to stdout???
This is one of the famous UNIX “tricks”
In UNIX, when I open a new file, this file gets the first
available file descriptor number
SO, if I close stdout, and open a file right after, this file will
have file descriptor 1

Henri Casanova ([email protected]) Inter-Process Communications (IPC)

Re-directing output
Perhaps some of you have wondered how come something like
ls > file.txt can work?
After all, ls has code that looks like
fprintf(stdout, "%s", filename);
So how can this code magically knows to write to a file
instead of to stdout???
This is one of the famous UNIX “tricks”
In UNIX, when I open a new file, this file gets the first
available file descriptor number
SO, if I close stdout, and open a file right after, this file will
have file descriptor 1
THEREFORE, printf will write to it as if it were stdout
Because fprintf(stdout, ...) really means “write to file
descriptor 1”

Henri Casanova ([email protected]) Inter-Process Communications (IPC)

Re-directing output
Perhaps some of you have wondered how come something like
ls > file.txt can work?
After all, ls has code that looks like
fprintf(stdout, "%s", filename);
So how can this code magically knows to write to a file
instead of to stdout???
This is one of the famous UNIX “tricks”
In UNIX, when I open a new file, this file gets the first
available file descriptor number
SO, if I close stdout, and open a file right after, this file will
have file descriptor 1
THEREFORE, printf will write to it as if it were stdout
Because fprintf(stdout, ...) really means “write to file
descriptor 1”
And I don’t need to change the code of ls at all!!!
Let’s see an example program
Henri Casanova ([email protected]) Inter-Process Communications (IPC)
Output Redirect Example

Example program fragment (should check for errors)

...
pid_t pid = fork();
if (!pid) { // child
close(1); // close stdout
FILE ∗file = fopen(”/tmp/stuff”, ”w”); // open a new file, which gets file descriptor 1
// exec the ”ls −la” command
char* const arguments[] = {"ls", "-la", NULL};
execv("ls", arguments);
}
...
}

This program will run ls -la and write its output to file
/tmp/stuff!
Let’s look at output redirect example1.c

Henri Casanova ([email protected]) Inter-Process Communications (IPC)

What if I opened the file before calling fork()?
In the previous example, the sequence of operation is:
Close stdout
Open a new file, which then gets file descriptor 1

Henri Casanova ([email protected]) Inter-Process Communications (IPC)

What if I opened the file before calling fork()?
In the previous example, the sequence of operation is:
Close stdout
Open a new file, which then gets file descriptor 1
What if I have already opened the file and it has some other
file descriptor?

Henri Casanova ([email protected]) Inter-Process Communications (IPC)

What if I opened the file before calling fork()?
In the previous example, the sequence of operation is:
Close stdout
Open a new file, which then gets file descriptor 1
What if I have already opened the file and it has some other
file descriptor?
This is what the dup() system call is there: file descriptor
duplication!
Essentially, dup() allows you to say “Create another file
descriptor for an existing opened file”, and it will always pick
to lowest unused descriptor number

Henri Casanova ([email protected]) Inter-Process Communications (IPC)

What if I opened the file before calling fork()?
In the previous example, the sequence of operation is:
Close stdout
Open a new file, which then gets file descriptor 1
What if I have already opened the file and it has some other
file descriptor?
This is what the dup() system call is there: file descriptor
duplication!
Essentially, dup() allows you to say “Create another file
descriptor for an existing opened file”, and it will always pick
to lowest unused descriptor number
The fileno() library call returns the descriptor of an open file

Henri Casanova ([email protected]) Inter-Process Communications (IPC)

What if I opened the file before calling fork()?
In the previous example, the sequence of operation is:
Close stdout
Open a new file, which then gets file descriptor 1
What if I have already opened the file and it has some other
file descriptor?
This is what the dup() system call is there: file descriptor
duplication!
Essentially, dup() allows you to say “Create another file
descriptor for an existing opened file”, and it will always pick
to lowest unused descriptor number
The fileno() library call returns the descriptor of an open file
So the sequence is:
FILE *some file = fopen(....);
close(1);
dup(fileno(some file));
After this sequence, writing to file descriptor 1 writes to the
file instead!
Let’s see a simple example again...
Henri Casanova ([email protected]) Inter-Process Communications (IPC)
Another Output Redirect Example

Example program fragment (should check for errors)

...
FILE ∗file = fopen(”/tmp/stuff”, ”w”); // open a new file
pid_t pid = fork();
if (!pid) { // child
close(1); // close stdout
dup(fileno(file)) // duplicate the file’s file descriptor
// exec the ”ls −la” command
char* const arguments[] = {"ls", "-la", NULL};
execv("ls", arguments);
}
...
}

This program will run ls -la and write its output to file
/tmp/stuff!
Let’s look at output redirect example2.c

Henri Casanova ([email protected]) Inter-Process Communications (IPC)

UNIX Pipes

A pipe is a simple IPC mechanisms between two processes

One can create a pipe so that process A can write to it and
process B reads from it and B can read from the pipe

Henri Casanova ([email protected]) Inter-Process Communications (IPC)

UNIX Pipes

A pipe is a simple IPC mechanisms between two processes

One can create a pipe so that process A can write to it and
process B reads from it and B can read from the pipe
Available in the shell with the | symbol: the output of a
process becomes the input of other(s)
e.g.: Count the files whose names contain foo but not bar in
the /tmp directory
List all files in /tmp: find /tmp -type f
Keep those with foo: grep foo
But remove those with bar: grep -v bar
And count the lines that remain: wc -l
Putting everything together:
find /tmp -type f | grep foo | grep -v bar | wc -l

Henri Casanova ([email protected]) Inter-Process Communications (IPC)

popen(): fork() with a pipe!

Very convenient library functions are popen and pclose

Sounds like “pipe open” and “pipe close”, but it’s MUCH
more than that

Henri Casanova ([email protected]) Inter-Process Communications (IPC)

popen(): fork() with a pipe!

Very convenient library functions are popen and pclose

Sounds like “pipe open” and “pipe close”, but it’s MUCH
more than that
popen()() does:
Creates a (bi-directional) pipe
Forks and execs a child process (e.g., ”ls -a”)
Returns the pipe, which is in fact a file (FILE *)
Both the parent and the child can “talk” through the pipe!
pclose() does:
Waits for the child process to complete
Closes the pipe

Henri Casanova ([email protected]) Inter-Process Communications (IPC)

popen(): fork() with a pipe!

Very convenient library functions are popen and pclose

Sounds like “pipe open” and “pipe close”, but it’s MUCH
more than that
popen()() does:
Creates a (bi-directional) pipe
Forks and execs a child process (e.g., ”ls -a”)
Returns the pipe, which is in fact a file (FILE *)
Both the parent and the child can “talk” through the pipe!
pclose() does:
Waits for the child process to complete
Closes the pipe
These are implemented with several system calls: fork,
waitpid, pipe (which creates a pipe), close, open, dup
Re-implementing popen/pclose would be a bit too much
here, but let’s just see an example program that uses it...

Henri Casanova ([email protected]) Inter-Process Communications (IPC)

popen() / pclose() Example

Example program fragment (should check for errors)

...
// fork/exec a child process and get a pipe to READ from
FILE ∗pipe = popen(”/usr/bin/ls −la”, ”r”);

// Get lines of output from the pipe, which is just a FILE ∗, until EOF is reached
char buffer[2048];
while (fgets(buffer, 2048, pipe)) {
fprintf(stderr,"LINE: %s", buffer);
}

// Wait for the child process to terminate

pclose(pipe);
}

This program prints all the output produced by ls -la

Of course, if that’s the only thing you want to do in this
program, just run ls -la directly :)
But perhaps you want to tweak the output and call this
program my ls?
Let’s look at and run popen example.c

Henri Casanova ([email protected]) Inter-Process Communications (IPC)

Conclusions

We’ve seen two main mechanisms for processes to

communicate:
Message Passing: Within the kernel Space
Shared Memory: Outside the kernel Space

Henri Casanova ([email protected]) Inter-Process Communications (IPC)

Conclusions

We’ve seen two main mechanisms for processes to

communicate:
Message Passing: Within the kernel Space
Shared Memory: Outside the kernel Space
Both mechanisms implemented in all mainstream OS
Many variants and extensions exist:
RPCs, RMI, Pipes, and many others we didn’t mention

Henri Casanova ([email protected]) Inter-Process Communications (IPC)

Conclusions

We’ve seen two main mechanisms for processes to

communicate:
Message Passing: Within the kernel Space
Shared Memory: Outside the kernel Space
Both mechanisms implemented in all mainstream OS
Many variants and extensions exist:
RPCs, RMI, Pipes, and many others we didn’t mention

Textbook readings:
About Pipes (Section 5.4)
About Client-Servers and RPCs (Section 47.5)

Quiz next week on this Module (Processes and IPCs)

Let’s look at Homework Assignments #4

Henri Casanova ([email protected]) Inter-Process Communications (IPC)