CA464: D ISTRIBUTED P ROGRAMMING

Architectures

Architectures
Architectural styles
Software architectures
Architectures versus middleware
Self-management in distributed systems

2.1

Architectural styles

Architectural styles
Basic idea
Organize into logically different components, and distribute those components over the various machines.
Layer N

Object

Object

Layer N-1
Method call

Object

Request
flow

Response
flow
Object

Layer 2
Object
Layer 1
(a)

(b)

(a) Layered style is used for client-server system

(b) Object-based style for distributed object systems.

Architectural Styles
Observation
Decoupling processes in space (anonymous) and also time (asynchronous) has led to
alternative styles.

Component

Component
Event delivery

Component

Component
Publish

Data delivery
Event bus
Publish

Shared (persistent) data space

Component
(a)

(b)

(a) Publish/subscribe [decoupled in space]

(b) Shared dataspace [decoupled in space and time]

Geoff Hamilton/Martin Crane (from originals by Tanenbaum & Van Steen)

CA464: D ISTRIBUTED P ROGRAMMING

2.2

System Architectures

Centralized Architectures
Basic ClientServer Model
Characteristics:
There are processes offering services (servers)
There are processes that use services (clients)
Clients and servers can be on different machines
Clients follow request/reply model wrt to using services
Wait for result

Client
Request

Reply

Server
Provide service

Time

Application Layering
Traditional three-layered view
User-interface layer contains units for an applications user interface
Processing layer contains the functions of an application, i.e. without specific
data
Data layer contains the data that a client wants to manipulate through the application components
Observation
This layering is found in many distributed information systems, using traditional
database technology and accompanying applications.
Application Layering
User-interface
level

User interface

Keyword expression

HTML page
containing list
HTML
generator

Query
generator
Database queries

Database
with Web pages

Ranked list
of page titles

Processing
level

Ranking
algorithm
Web page titles
with meta-information

Data level

Geoff Hamilton/Martin Crane (from originals by Tanenbaum & Van Steen)

CA464: D ISTRIBUTED P ROGRAMMING

Multi-Tiered Architectures
Single-tiered: dumb terminal/mainframe configuration
Two-tiered: client/single server configuration
Three-tiered: each layer on separate machine
Traditional two-tiered configurations:
Client machine
User interface

User interface

User interface

User interface

User interface

Application

Application

Application
Database

User interface
Application

Application

Database

Database

Application
Database

Database

Database

(d)

(e)

Server machine
(a)

(b)

(c)

Decentralized Architectures
Observation
In the last couple of years we have been seeing a tremendous growth in peer-to-peer
systems.
Structured P2P: nodes are organized following a specific distributed data structure
Unstructured P2P: nodes have randomly selected neighbors
Hybrid P2P: some nodes are appointed special functions in a well-organized
fashion
Note
In virtually all cases, we are dealing with overlay networks (i.e. a network where
the nodes are the procs & the links are the communication channels) : data is routed
over connections setup between the nodes (cf. application-level multicasting)
Structured P2P Systems
Basic idea
Organize the nodes in a structured overlay network such as a logical ring, and make
specific nodes responsible for services based only on their ID.

Geoff Hamilton/Martin Crane (from originals by Tanenbaum & Van Steen)

CA464: D ISTRIBUTED P ROGRAMMING

Actual node
15
14

{13,14,15}

{0,1}

2
3

13
12 {8,9,10,11,12}

{2,3,4}

Associated
data keys

11
10

{5,6,7}
9

Note
The system provides an operation LOOKUP(key) that will efficiently route the
lookup request to the associated node.
Structured P2P Systems
Details of Chord Algorithm
Assign a random key to data item & a random number to node in system from
identifier space,
Implement an efficient & deterministic system to map a data item to a node based
on some distance metric,
LOOKUP(key) returning network address of node responsible for that data
item,
Do this by routing a request for the data item to responsible node.
Note
The Chord Algorithm will be described more fully later in the course.
Structured P2P Systems
Other example
Organize nodes in a d-dimensional space and let every node take the responsibility
for data in a specific region. When a node joins split a region.

Geoff Hamilton/Martin Crane (from originals by Tanenbaum & Van Steen)

CA464: D ISTRIBUTED P ROGRAMMING

Keys associated with

node at (0.6,0.7)
(0,1)

(1,1)
(0.9,0.9)

(0.9,0.9)

(0.2,0.8)

(0.2,0.8)
(0.6,0.7)

Actual node

(0.6,0.7)
(0.9,0.6)

(0.9,0.6)
(0.2,0.45)

(0.2,0.3)
(0.7,0.2)

(0.7,0.2)
(0.2,0.15)

(0,0)

(1,0)
(a)

(b)

Unstructured P2P Systems

Observation
Many unstructured P2P systems attempt to maintain a random graph.
Basic principle
Each node is required to contact a randomly selected other node:
Let each peer maintain a partial view of the network, consisting of c other nodes
Each node P periodically selects a node Q from its partial view
P and Q exchange information and exchange members from their respective partial views
Note
It turns out that, depending on the exchange, randomness, but also robustness of the network
can be maintained.

Topology Management of Overlay Networks

Basic idea
Distinguish two layers: (1) maintain random partial views in lowest layer; (2) be selective
on who you keep in higher-layer partial view.

Structured
overlay

Protocol for
specific
overlay

Links to topologyspecific other nodes

Random peer

Random
overlay

Protocol for
randomized
view

Links to randomly
chosen other nodes

Note
Lower layer feeds upper layer with random nodes; upper layer is selective when it comes to
keeping references.

Geoff Hamilton/Martin Crane (from originals by Tanenbaum & Van Steen)

CA464: D ISTRIBUTED P ROGRAMMING

Topology Management of Overlay Networks

Constructing a torus
Consider a N N grid. Keep only references to nearest neighbors:
k (a1 , a2 ) (b1 , b2 ) k= d1 + d2
di = min{N |ai bi |, |ai bi |}

Time

Topology Management of Overlay Networks

Example
Consider a 4 4 grid. Keep only references to nearest neighbors:
k (a1 , a2 ) (b1 , b2 ) k= d1 + d2
di = min{N |ai bi |, |ai bi |}

Here, there are two points

(a1 , a2 ) = (1, 3) and (b1 , b2 ) = (3, 0)
hence
d1 = min{4 2, 2} = 2 (both paths same length)
and
d2 = min{4 3, 3} = 1 (green path is shorter)

Geoff Hamilton/Martin Crane (from originals by Tanenbaum & Van Steen)

CA464: D ISTRIBUTED P ROGRAMMING

Topology Management of Overlay Networks

Explanation
With the minimum distance condition, a toroidal shape emerges.

Superpeers
Observation
Sometimes it helps to select a few nodes to do specific work: superpeer.

Regular peer
Superpeer

Superpeer
network

Examples
Peers maintaining an index (for search)
Peers monitoring the state of the network
Peers being able to setup connections

Hybrid Architectures: Client-server combined with P2P

Example
Edge-server architectures, which are often used for Content Delivery Networks

Geoff Hamilton/Martin Crane (from originals by Tanenbaum & Van Steen)

CA464: D ISTRIBUTED P ROGRAMMING

Client

Content provider
ISP

ISP

Core Internet
Edge server
Enterprise network

Hybrid Architectures: C/S with P2P BitTorrent

Client node
K out of N nodes
Node 1

Lookup(F)
A BitTorrent
Web page
Web server

Ref. to
file
server

.torrent file
for F

Ref. to
tracker

List of nodes
storing F

File server

Tracker

Node 2

Node N

Basic idea
Once a node has identified where to download a file from, it joins a swarm of
downloaders who in parallel get file chunks from the source, but also distribute these
chunks amongst each other.

2.3

Architectures versus Middleware

Architectures versus Middleware

Problem
In many cases, distributed systems/applications are developed according to a specific architectural style. The chosen style may not be optimal in all cases need to
(dynamically) adapt the behavior of the middleware.
Interceptors
Intercept the usual flow of control when invoking a remote object.
Interceptors

Geoff Hamilton/Martin Crane (from originals by Tanenbaum & Van Steen)

CA464: D ISTRIBUTED P ROGRAMMING

Client application

Intercepted call

B.do_something(value)

Application stub
Request-level interceptor

Nonintercepted call
invoke(B, &do_something, value)

Object middleware
Message-level interceptor

send([B, "do_something", value])

Local OS
To object B

Adaptive Middleware
Separation of concerns: Try to separate extra functionalities and later weave them
together into a single implementation only toy examples so far.
Computational reflection: Let a program inspect itself at runtime and adapt/change
its settings dynamically if necessary mostly at language level and applicability
unclear.
Component-based design: Organize a distributed application through components
that can be dynamically replaced when needed highly complex, also many
intercomponent dependencies.
Fundamental question
Do we need adaptive software at all, or is the issue adaptive systems?

2.4

Self-management in Distributed Systems

Self-managing Distributed Systems

Observation
Distinction between system and software architectures blurs when automatic adaptivity needs to be taken into account:
Self-configuration
Self-managing
Self-healing
Self-optimizing

Geoff Hamilton/Martin Crane (from originals by Tanenbaum & Van Steen)

CA464: D ISTRIBUTED P ROGRAMMING

10

Self-*
Warning
There is a lot of hype going on in this field of autonomic computing.
Feedback Control Model
Observation
In many cases, self-* systems are organized as a feedback control system.
Uncontrollable parameters (disturbance / noise)

Initial configuration

Corrections

+/-

+/-

Core of distributed system

Observed output

+/Reference input

Adjustment
measures

Metric
estimation

Analysis
Measured output

Adjustment triggers

Example: Globule
Globule
Collaborative CDN that analyzes traces to decide where replicas of Web content
should be placed. Decisions are driven by a general cost model:
cost = (w1 m1 ) + (w2 m2 ) + + (wn mn )
Example: Globule
Client
Origin server

ISP

ISP

Core Internet
Replica server
Enterprise network
Client

Client

Globule origin server collects traces and does what-if analysis by checking what
would have happened if page P would have been placed at edge server S.
Many strategies are evaluated, and the best one is chosen.

Geoff Hamilton/Martin Crane (from originals by Tanenbaum & Van Steen)