Distributed Systems

COMP3100/COMP6105
Lecture 2: System Architectures and Models

S1, 2025 @ Macquarie University 1

Assignment 1 - Case Study

S1, 2025 @ Macquarie University 2

 2008 – David Patterson, U. C. Berkeley
“The data center is the computer”
1993 – Eric Schmidt, Google
“When the network becomes as fast as
the processor, the computer hollows out and
spreads across the network.”

1984 – John Gage, Sun Microsystems

“The network is the computer”
S1, 2025 @ Macquarie University 3
 Lecture 2:

System Architectures and Models

S1, 2025 @ Macquarie University 4

System models
• Fundamental models take an abstract perspective in order
to describe solutions to individual issues faced by most
distributed systems.
• Physical models consider the types of computers and devices that
constitute a system and their interconnectivity, without details of
specific technologies.
• Architectural models describe a system in terms of the
computational and communication tasks performed by its
computational elements; the computational elements being individual
computers or aggregates of them supported by appropriate network
interconnections. Client-server and peer-to-peer are two of the most
commonly used forms of architectural model for distributed systems.
S1, 2025 @ Macquarie University 5
How the models evolved over time?

S1, 2025 @ Macquarie University 6

Architectural models
• Centralized system architectures
• Client-server models
• A service provided by multiple servers
• Web proxy server
• Web applets

• Decentralized system architectures

• Peer-to-peer models: structured or unstructured

• Hybrid system architectures

• Super-peer networks
• Edge-Server Architecture S1, 2025 @ Macquarie University 7
Centralized system architectures
• Basic Client–Server Model
• Characteristics:
• There are processes offering services (servers)
• There are processes that use services (clients)
• Clients and servers can be on different machines
• Clients follow request/reply model with respect to using services

S1, 2025 @ Macquarie University 8

Architectural models
• Client-server models

Client invoc ation Serv er

invoc ation

result result
Serv er

Client
Key:
Proc ess: Computer:

S1, 2025 @ Macquarie University 9

Two-party communication
• Server
1 from socket import *
2 s = socket(AF_INET, SOCK_STREAM)
3 (conn, addr) = s.accept() #returns new socket and addr.client
4 while True: #forever
5 data = conn.recv(1024) #receive data from client
6 ifnot data: break #stop if clients topped
7 conn.send(str(data)+"*") #return sent data plus an" *"
8 conn.close() #close the connection

• Client
1 from socket import *
2 s = socket(AF_INET, SOCK_STREAM)
3 s.connect((HOST, PORT)) #connect to server(block until accepted)
4 s.send(’Hello,world’) #send some data
5 data = s.recv(1024) #receive the response
6 print data #print the result
7 s.close() #close the connection

S1, 2025 @ Macquarie University 10

Architectural models
• A service provided by multiple servers
Serv ice

Serv er
Client

Serv er

Client
Serv er

S1, 2025 @ Macquarie University 11

Architectural models
• Web proxy server

Client Web
server
Prox y
server

Client Web
server

S1, 2025 @ Macquarie University 12

Architectural models
• Web applets
a) c lient reques t results in the downloading of appl et c ode

Client Web
Applet code server

b) c lient interacts with the applet

Web
Client Applet server

S1, 2025 @ Macquarie University 13

 Decentralized system architectures
• Basic Peer-to-peer models:
• There are no fixed roles; each node can act as both a client
and a server.
• All peers share resources directly with each other.
• Peers communicate without a central authority managing the
network.
• Peers follow a collaborative request/reply model, where any
node can request or provide a service.
• Peers can be on different machines and may dynamically
join or leave the network.
• The network structure can be structured (distributed hash
tables, DHT-based) or unstructured (random connection).
• Fault tolerance is higher, as there is no single point of failure.

S1, 2025 @ Macquarie University 14

Structured P2P
• Essence
• Make use of a semantic-free index: each data item is uniquely associated with
a key, in turn used as an index. Common practice: use a hash function
key(data item) = hash(data item’s value).
P2P system now responsible for storing (key,value) pairs.

• Simple example: hypercube

Looking up d with key k ∈{0, 1, 2, . . . , 24 − 1} means routing request to

node with identifier k .
S1, 2025 @ Macquarie University 15
Example: Chord
• Principle
• Nodes are logically organized in a ring. Each node has an m-bit identifier. Each data item is
hashed to an m-bit key.
• Data item with key k is stored at node with smallest identifier id ≥ k , called the successor
of key k .
• The ring is extended with various shortcut links to other nodes.
• It supports O(log N) lookup time for key-value storage across N nodes.

S1, 2025 @ Macquarie University 16

Example: Chord

S1, 2025 @ Macquarie University 17

Unstructured P2P
• Essence
• Each node maintains an ad hoc list of neighbors. The resulting overlay resembles a
random graph: an edge (u, v) exists only with a certain probability P[(u, v )].

• Searching
• Flooding: issuing node u passes request for destination node d to all neighbors. Request
is ignored when receiving node had seen it before. Otherwise, v searches locally for d
(recursively). May be limited by a Time-To-Live: a maximum number of hops.

• Random walk: issuing node u passes request for d to randomly chosen neighbor, v . If v
does not have d , it forwards request to one of its randomly chosen neighbors, and so
on.

S1, 2025 @ Macquarie University 18

Flooding versus random walk
• Model
• Assume N nodes and that each data item is replicated across randomly chosen r nodes.

• Random walk
• P[k ] probability that item is found after k attempts:

• S (“search size”) is expected number of nodes that need to be probed:

S1, 2025 @ Macquarie University 19

Flooding versus random walk
• Flooding
• Flood to d randomly chosen neighbors
• After k steps, some R(k ) = d · (d − 1)k −1 will have been reached (assuming k is small).
• With fraction r /N nodes having data, if , we will have found the data item.

• Comparison
• If r /N = 0.001, then S ≈ 1000
• With flooding and d = 10, k = 4, we contact 7290 nodes.
• Random walks are more communication efficient, but might take longer before they find
the result.

S1, 2025 @ Macquarie University 20

 Hybrid system architectures
• Edge-server architecture (e.g., live streaming example in Week 1)
• Systems deployed on the Internet where servers are placed at the edge of the network: the boundary
between enterprise networks and the actual Internet.
• There are edge servers located closer to end-users to process data locally.
• Edge servers act as intermediaries between clients and central cloud servers.
• Clients send requests to nearby edge servers, reducing latency and bandwidth usage.
• Edge servers can operate autonomously, handling requests without always relying on the cloud.
• The architecture follows a hybrid model, combining centralized cloud with decentralized edges.
• Improves real-time processing by reducing the need to send all data to the cloud.
• Common in IoT, content delivery networks (CDNs), and smart applications.

S1, 2025 @ Macquarie University 21

Super-peer networks
Essence
• It is sometimes sensible to break the symmetry in pure peer-to-peer networks:
• When searching in unstructured P2P systems, having index servers improves
performance
• Deciding where to store data can often be done more efficiently through brokers.

S1, 2025 @ Macquarie University 22

Skype’s principle operation: A wants
to contact B
• Both A and B are on the public Internet
• A TCP connection is set up between A and B for control packets.
• The actual call takes place using UDP packets between negotiated ports.
• A operates behind a firewall, while B is on the public Internet
• A sets up a TCP connection (for control packets) to a super peer S, S sets up a
TCP connection (for relaying control packets) to B
• The actual call takes place through UDP and directly between A and B
• Both A and B operate behind a firewall
• A connects to an online super peer S through TCP
• S sets up TCP connection to B.
• For the actual call, another super peer is contacted to act as a relay R: A
• sets up a connection to R, and so will B.
• All voice traffic is forwarded over the two TCP connections, and through R.
S1, 2025 @ Macquarie University 23
Collaboration: The BitTorrent case
• Principle: search for a file F
• Lookup file at a global directory ⇒ returns a torrent file
• Torrent file contains reference to tracker: a server keeping an accurate account of active
nodes that have (chunks of) F .
• P can join swarm, get a chunk for free, and then trade a copy of that
• chunk for another one with a peer Q also in the swarm.

S1, 2025 @ Macquarie University 24

BitTorrent under the hood
• Some essential details
• A tracker for file F returns the set of its downloading processes: the current
swarm.
• A communicates only with a subset of the swarm: the neighbor set NA. if B ∈NA
then also A ∈NB .
• Neighbor sets are regularly updated by the tracker

• Exchange blocks
• A file is divided into equally sized pieces(typically each being 256 KB) Peers
exchange blocks of pieces, typically some 16 KB.
• A can upload a block d of piece D, only if it has piece D.
• Neighbor B belongs to the potential set PA of A, if B has a block that A
• needs.
• If B ∈PA and A ∈PB : A and B are in a position that they can trade a block.
S1, 2025 @ Macquarie University 25
BitTorrent phases
• Bootstrap phase
• A has just received its first piece (through optimistic unchoking: a node from NA
unselfishly provides the blocks of a piece to get a newly arrived node started).

• Trading phase
• |PA| > 0: there is (in principle) always a peer with whom A can trade.

• Last download phase

• |PA| = 0: A is dependent on newly arriving peers in NA in order to get the last
missing pieces. NA can change only through the tracker.

S1, 2025 @ Macquarie University 26

BitTorrent phases
• Development of |P| relative to |N|.

S1, 2025 @ Macquarie University 27

Architectural styles
• Basic idea
A style is formulated in terms of
• (replaceable) components with well-defined interfaces the way
that components are connected to each other the data
exchanged between components
• how these components and connectors are jointly configured into
a system.
• Connector
• A mechanism that mediates communication, coordination, or
cooperation among components.
• Example: facilities for (remote) procedure call, messaging, or
streaming.
S1, 2025 @ Macquarie University 28
 Architectural styles
• To address the challenges of contemporary distributed systems (ultra-large
scale, heterogeneity, QoS issues), various architectural styles are used

S1, 2025 @ Macquarie University 29

 Layered architecture
• An Architectural Style where the system is divided into hierarchical layers
• Different layered organizations

S1, 2025 @ Macquarie University 30

Multi-tiered centralized system
• Some traditional organizations
• Single-tiered: dumb terminal/mainframe configuration
• Two-tiered: client/single server configuration
• Three-tiered: each layer on separate machine

• Traditional two-tiered configurations

S1, 2025 @ Macquarie University 31

Two-tier and three-tier architectures

S1, 2025 @ Macquarie University 32

Being client and server at the same
time
• Three-tiered architecture

S1, 2025 @ Macquarie University 33

Example: communication protocols
• Protocol, service, interface

S1, 2025 @ Macquarie University 34

Software and hardware service layers
in distributed systems

Applications, serv ices

Mi ddleware

Operating sys tem

Platform

Computer and network hardware

S1, 2025 @ Macquarie University 35

Application Layering
• Traditional three-layered view
• Application-interface layer contains units for interfacing to users or
external applications
• Processing layer contains the functions of an application, i.e.,
without specific data
• Data layer contains the data that a client wants to manipulate
through the application components
• Observation
• This layering is found in many distributed information systems, using
traditional database technology and accompanying
applications.

S1, 2025 @ Macquarie University 36

Application Layering
Example: a simple search engine

S1, 2025 @ Macquarie University 37

Using legacy to build middleware
• Problem
• The interfaces offered by a legacy component are most likely not
suitable for all applications.

• Solution
• A wrapper or adapter offers an interface acceptable to a client
application. Its functions are transformed into those available at
the component.

S1, 2025 @ Macquarie University 38

 Organizing wrappers
• Two solutions: 1-on-1 or through a broker

• Complexity with N applications

1-on-1: requires N × (N − 1) = O(N 2) wrappers
broker: requires 2N = O(N) wrappers

S1, 2025 @ Macquarie University 39

Developing adaptable middleware
• Problem
• Middleware contains solutions that are good for most applications ⇒ you may
want to adapt its behaviour for specific applications.

S1, 2025 @ Macquarie University 40

Intercept the usual flow of control

S1, 2025 @ Macquarie University 41 / 36

Alternative organizations
• Vertical distribution
• Comes from dividing distributed applications into three logical layers, and running the
components from each layer on a different server (machine).

• Horizontal distribution
• A client or server may be physically split up into logically equivalent parts, but each part
is operating on its own share of the complete data set.

• Peer-to-peer architectures
• Processes are all equal: the functions that need to be carried out are represented by
every process ⇒ each process will act as a client and a server at the same time (i.e.,
acting as a servant).

S1, 2025 @ Macquarie University 42

Python fundamentals
• Is Python easier to use?

• Enhanced readability and rapid development

• More popular in AI/Machine Learning

• But, slower execution

S1, 2025 @ Macquarie University 43

For loop

S1, 2025 @ Macquarie University 44

While loop

S1, 2025 @ Macquarie University 45

Range

S1, 2025 @ Macquarie University 46

If/Else

S1, 2025 @ Macquarie University 47

Logical operators

S1, 2025 @ Macquarie University 48

List functions

S1, 2025 @ Macquarie University 49

Functions

S1, 2025 @ Macquarie University 50

String

S1, 2025 @ Macquarie University 51

File loading

S1, 2025 @ Macquarie University 52