Processes: Threads Introduction to threads

Introduction to threads

Distributed Systems
Basic idea
(3rd Edition)
We build virtual processors in software, on top of physical processors:
Processor: Provides a set of instructions along with the capability of
automatically executing a series of those instructions.
Maarten van Steen Andrew S. Tanenbaum
Thread: A minimal software processor in whose context a series of
instructions can be executed. Saving a thread context implies
stopping the current execution and saving all the data needed to
Chapter 03: Processes continue the execution at a later stage.
Process: A software processor in whose context one or more threads may
Edited by: Hicham G. Elmongui be executed. Executing a thread, means executing a series of
instructions in the context of that thread.

2 / 36

Processes: Threads Introduction to threads Processes: Threads Introduction to threads

Context switching Context switching

Contexts
Processor context: The minimal collection of values stored in the registers Observations
of a processor used for the execution of a series of instructions (e.g., 1 Threads share the same address space. Thread context switching can be
stack pointer, addressing registers, program counter). done entirely independent of the operating system.
Thread context: The minimal collection of values stored in registers and 2 Process switching is generally (somewhat) more expensive as it involves
memory, used for the execution of a series of instructions (i.e., processor getting the OS in the loop, i.e., trapping to the kernel.
context, state).
3 Creating and destroying threads is much cheaper than doing so for
Process context: The minimal collection of values stored in registers and processes.
memory, used for the execution of a thread (i.e., thread context, but now
also at least MMU register values).

3 / 36 4 / 36

Processes: Threads Introduction to threads Processes: Threads Introduction to threads

Why use threads Avoid process switching

Avoid expensive context switching

Process A Process B

Some simple reasons

Avoid needless blocking: a single-threaded process will block when doing S1: Switch from user space
to kernel space
I/O; in a multi-threaded process, the operating system can switch the CPU S3: Switch from kernel
space to user space
to another thread in that process.
Operating system
Exploit parallelism: the threads in a multi-threaded process can be
scheduled to run in parallel on a multiprocessor or multicore processor. S2: Switch context from
process A to process B
Avoid process switching: structure large applications not as a collection of
processes, but through multiple threads. Trade-offs
Threads use the same address space: more prone to errors
No support from OS/HW to protect threads using each other’s memory
Thread context switching may be faster than process context switching
Thread usage in nondistributed systems 5 / 36 Thread usage in nondistributed systems 6 / 36
Processes: Threads Introduction to threads Processes: Threads Introduction to threads

Threads and operating systems Threads and operating systems

Main issue Kernel solution

Should an OS kernel provide threads, or should they be implemented as The whole idea is to have the kernel contain the implementation of a thread
user-level packages? package. This means that all operations return as system calls:
Operations that block a thread are no longer a problem: the kernel
User-space solution schedules another available thread within the same process.
All operations can be completely handled within a single process ⇒ handling external events is simple: the kernel (which catches all events)
implementations can be extremely efficient. schedules the thread associated with the event.
All services provided by the kernel are done on behalf of the process in The problem is (or used to be) the loss of efficiency due to the fact that
which a thread resides ⇒ if the kernel decides to block a thread, the each thread operation requires a trap to the kernel.
entire process will be blocked.
Threads are used when there are lots of external events: threads block on Conclusion – but
a per-event basis ⇒ if the kernel can’t distinguish threads, how can it Try to mix user-level and kernel-level threads into a single concept, however,
support signaling events to them? performance gain has not turned out to outweigh the increased complexity.

Thread implementation 7 / 36 Thread implementation 8 / 36

Processes: Threads Threads in distributed systems Processes: Threads Threads in distributed systems

Using threads at the client side Using threads at the server side

Multithreaded web client

Improve performance
Hiding network latencies:
Starting a thread is cheaper than starting a new process.
Web browser scans an incoming HTML page, and finds that more files Having a single-threaded server prohibits simple scale-up to a
need to be fetched. multiprocessor system.
Each file is fetched by a separate thread, each doing a (blocking) HTTP As with clients: hide network latency by reacting to next request while
request. previous one is being replied.
As files come in, the browser displays them.
Better structure
Multiple request-response calls to other machines (RPC)
Most servers have high I/O demands. Using simple, well-understood
A client does several calls at the same time, each one by a different blocking calls simplifies the overall structure.
thread. Multithreaded programs tend to be smaller and easier to understand due
It then waits until all results have been returned. to simplified flow of control.
Note: if calls are to different servers, we may have a linear speed-up.

Multithreaded clients 9 / 36 Multithreaded servers 10 / 36

Processes: Threads Threads in distributed systems Processes: Virtualization Principle of virtualization

Why multithreading is popular: organization Virtualization

Dispatcher/worker model Observation
Dispatcher thread
Request dispatched Virtualization is important:
to a worker thread Server
Hardware changes faster than software
Ease of portability and code migration
Worker thread Isolation of failing or attacked components
Request coming in
from the network Principle: mimicking interfaces
Operating system
Program

Interface A
Overview
Program Implementation of
Model Characteristics mimicking A on B

Multithreading Parallelism, blocking system calls Interface A Interface B

Single-threaded process No parallelism, blocking system calls Hardware/software system A Hardware/software system B
Finite-state machine Parallelism, nonblocking system calls
Multithreaded servers 11 / 36 12 / 36
Processes: Virtualization Principle of virtualization Processes: Virtualization Principle of virtualization

Mimicking interfaces Ways of virtualization

(a) Process VM, (b) Native VMM, (c) Hosted VMM
Application/Libraries

Four types of interfaces at three different levels Application/Libraries Application/Libraries Operating system

1 Instruction set architecture: the set of machine instructions, with two Runtime system Operating system Virtual machine monitor
subsets:
Privileged instructions: allowed to be executed only by the operating Operating system Virtual machine monitor Operating system
system.
General instructions: can be executed by any program. Hardware Hardware Hardware

2 System calls as offered by an operating system. (a) (b) (c)

3 Library calls, known as an application programming interface (API)

Differences
(a) Separate set of instructions, an interpreter/emulator, running atop an OS.
(b) Low-level instructions, along with bare-bones minimal operating system
(c) Low-level instructions, but delegating most work to a full-fledged OS.

Types of virtualization 13 / 36 Types of virtualization 14 / 36

Processes: Virtualization Principle of virtualization Processes: Virtualization Principle of virtualization

Zooming into VMs: performance Condition for virtualization

Refining the organization Necessary condition

Application/Libraries
For any conventional computer, a virtual machine monitor may be constructed
if the set of sensitive instructions for that computer is a subset of the set of
Guest operating system Privileged instruction: if and only if privileged instructions.
executed in user mode, it causes
Virtual machine monitor
a trap to the operating system Problem: condition is not always satisfied
There may be sensitive instructions that are executed in user mode without
Privileged Host operating system Nonpriviliged
General instruction: the rest
instructions instructions causing a trap to the operating system.
Hardware

Solutions
Special instructions Emulate all instructions
Control-sensitive instruction: may affect configuration of a machine (e.g., Wrap nonprivileged sensitive instructions to divert control to VMM
one affecting relocation register or interrupt table).
Paravirtualization: modify guest OS, either by preventing nonprivileged
Behavior-sensitive instruction: effect is partially determined by context sensitive instructions, or making them nonsensitive (i.e., changing the
(e.g., POPF sets an interrupt-enabled flag, but only in system mode). context).
Types of virtualization 15 / 36 Types of virtualization 16 / 36

Processes: Virtualization Application of virtual machines to distributed systems Processes: Clients Networked user interfaces

VMs and cloud computing Client-server interaction

Three types of cloud services

Distinguish application-level and middleware-level solutions
Infrastructure-as-a-Service covering the basic infrastructure
Client machine Server machine
Platform-as-a-Service covering system-level services Client machine Server machine

Software-as-a-Service containing actual applications Application Application Application Application

Application- Application-
specific independent
IaaS Middleware protocol Middleware Middleware protocol Middleware
Local OS Local OS Local OS Local OS
Instead of renting out a physical machine, a cloud provider will rent out a VM
(or VMM) that may possibly be sharing a physical machine with other
Network Network
customers ⇒ almost complete isolation between customers (although
performance isolation may not be reached).

17 / 36 18 / 36
Processes: Clients Networked user interfaces Processes: Clients Networked user interfaces

Example: The X Window system Improving X

Basic organization
Application server Application server User's terminal Practical observations
Window Application Xlib interface There is often no clear separation between application logic and
manager user-interface commands
Xlib Xlib Applications tend to operate in a tightly synchronous manner with an X
Local OS Local OS
kernel
X protocol

X kernel Alternative approaches

Device drivers
Let applications control the display completely, up to the pixel level (e.g.,
Terminal (includes display
keyboard, mouse, etc.) VNC)
Provide only a few high-level display operations (dependent on local video
X client and server drivers), allowing more efficient display operations.
The application acts as a client to the X-kernel, the latter running as a server
on the client’s machine.

Example: The X window system 19 / 36 Thin-client network computing 20 / 36

Processes: Clients Client-side software for distribution transparency Processes: Servers General design issues

Client-side software Servers: General organization

Generally tailored for distribution transparency

Access transparency: client-side stubs for RPCs
Location/migration transparency: let client-side software keep track of
actual location
Replication transparency: multiple invocations handled by client stub:
Basic model
Client machine Server 1 Server 2 Server 3
A process implementing a specific service on behalf of a collection of clients. It
Client Server Server Server waits for an incoming request from a client and subsequently ensures that the
appl appl appl appl
request is taken care of, after which it waits for the next incoming request.

Client side handles

request replication
Replicated request

Failure transparency: can often be placed only at client (we’re trying to

mask server and communication failures).

21 / 36 22 / 36

Processes: Servers General design issues Processes: Servers General design issues

Concurrent servers Contacting a server

Observation: most services are tied to a specific port

ftp-data 20 File Transfer [Default Data]
Two basic types
ftp 21 File Transfer [Control]
Iterative server: Server handles the request before attending a next telnet 23 Telnet
request. smtp 25 Simple Mail Transfer
Concurrent server: Uses a dispatcher, which picks up an incoming www 80 Web (HTTP)
request that is then passed on to a separate thread/process.
Dynamically assigning an end point
Observation Server machine Server machine
Concurrent servers are the norm: they can easily handle multiple requests, Client machine
2. Request
service
Register
Client machine
2. Continue
service Specific
Server end point
notably in the presence of blocking operations (to disks or other servers). server
Client Client
Super- Create server
1. Ask for Daemon 1. Request server and hand off
end point End-point service request
table

Concurrent versus iterative servers 23 / 36 Contacting a server: end points 24 / 36

Processes: Servers General design issues Processes: Servers General design issues

Out-of-band communication Servers and state

Stateless servers
Issue Never keep accurate information about the status of a client after having
handled a request:
Is it possible to interrupt a server once it has accepted (or is in the process of
accepting) a service request? Don’t record whether a file has been opened (simply close it again after
access)
Solution 1: Use a separate port for urgent data Don’t promise to invalidate a client’s cache
Don’t keep track of your clients
Server has a separate thread/process for urgent messages
Urgent message comes in ⇒ associated request is put on hold Consequences
Note: we require OS supports priority-based scheduling
Clients and servers are completely independent
State inconsistencies due to client or server crashes are reduced
Solution 2: Use facilities of the transport layer Possible loss of performance because, e.g., a server cannot anticipate
Example: TCP allows for urgent messages in same connection client behavior (think of prefetching file blocks)
Urgent messages can be caught using OS signaling techniques
Question
Does connection-oriented communication fit into a stateless design?
Interrupting a server 25 / 36 Stateless versus stateful servers 26 / 36

Processes: Servers General design issues Processes: Servers Server clusters

Servers and state Three different tiers

Common organization
Logical switch Application/compute servers Distributed
Stateful servers (possibly multiple) file/database
Keeps track of the status of its clients: system

Record that a file has been opened, so that prefetching can be done Dispatched
Knows which data a client has cached, and allows clients to keep local Client requests
request

copies of shared data

Observation
The performance of stateful servers can be extremely high, provided clients
First tier Second tier Third tier
are allowed to keep local copies. As it turns out, reliability is often not a major
problem.
Crucial element
The first tier is generally responsible for passing requests to an appropriate
server: request dispatching

Stateless versus stateful servers 27 / 36 Local-area clusters 28 / 36

Processes: Servers Server clusters Processes: Servers Server clusters

Request Handling Server clusters

The front end may easily get overloaded: special measures may be needed
Observation Transport-layer switching: Front end simply passes the TCP request to
Having the first tier handle all communication from/to the cluster may lead to a one of the servers, taking some performance metric into account.
bottleneck. Content-aware distribution: Front end reads the content of the request
and then selects the best server.
A solution: TCP handoff
Combining two solutions
Logically a
single TCP Response
Server 6. Server responses
connection Application
5. Forward server 3. Hand off
other TCP connection
Request messages Distributor
Request (handed off)
Client Switch
Other messages
Dis-
Client Switch 4. Inform patcher
Setup request switch
Server 1. Pass setup request Distributor 2. Dispatcher selects
to a distributor server
Application
server

Local-area clusters 29 / 36 Local-area clusters 30 / 36

Processes: Servers Server clusters Processes: Code migration Reasons for migrating code

When servers are spread across the Internet Reasons to migrate code
Observation Load distribution
Spreading servers across the Internet may introduce administrative problems. Ensuring that servers in a data center are sufficiently loaded (e.g., to
These can be largely circumvented by using data centers from a single cloud prevent waste of energy)
provider.
Minimizing communication by ensuring that computations are close to
where the data is (think of mobile computing).
Request dispatching: if locality is important
Common approach: use DNS: Flexibility: moving code to a client when needed
1 Client looks up specific service through DNS - client’s IP address is part 2. Client and server
communicate
of request Client Server

2 DNS server keeps track of replica servers for the requested service, and
returns address of most local server.

Client transparency
1. Client fetches code
To keep client unaware of distribution, let DNS resolver act on behalf of client. Service-specific
client-side code
Problem is that the resolver may actually be far from local to the actual client. Code repository

Wide-area clusters 31 / 36 32 / 36
Avoids pre-installing software and increases dynamic configuration.

Processes: Code migration Reasons for migrating code Processes: Code migration Reasons for migrating code

Models for code migration Models for code migration

Before execution After execution Before execution After execution

Client Server Client Server Client Server Client Server

code code code code

CS exec exec* CoD exec ←− exec* ←−
resource resource resource resource

code code code code

REV −→ exec −→ exec* MA exec −→ −→ exec*
resource resource resource resource resource resource

CS: Client-Server REV: Remote evaluation CoD: Code-on-demand MA: Mobile agents

33 / 36 34 / 36

Processes: Code migration Migration in heterogeneous systems Processes: Code migration Migration in heterogeneous systems

Migration in heterogeneous systems Migrating a virtual machine

Main problem Migrating images: three alternatives

The target machine may not be suitable to execute the migrated code 1 Pushing memory pages to the new machine and resending the ones that
The definition of process/thread/processor context is highly dependent on are later modified during the migration process.
local hardware, operating system and runtime system 2 Stopping the current virtual machine; migrate memory, and start the new
virtual machine.
Only solution: abstract machine implemented on different platforms
3 Letting the new virtual machine pull in new pages as needed: processes
Interpreted languages, effectively having their own VM start on the new virtual machine immediately and copy memory pages on
demand.
Virtual machine monitors

35 / 36 36 / 36