Basics
Basics
1 2
Distributed Systems, Spring 2003 Distributed Systems, Spring 2003
3 4
Distributed Systems, Spring 2003 Distributed Systems, Spring 2003
Characteristics
(1) Heterogeneity hidden
(2) Interact with a consistent and uniform way
(3) Availability
(4) Scale
1.1
Issues
(1) Concurrency
(2) No global clock Note that the middleware layer extends over multiple machines.
(3) Independent failures
5 6
Distributed Systems, Spring 2003 Distributed Systems, Spring 2003
1
A typical portion of the Internet
Examples of Distributed Systems
The Internet
Intranets intranet
Mobile and Ubiquitous Computing ISP
The Web
Local area
Web server network
• A portion of the Internet separately administrated • Devices: laptop computers, handheld devices (e.g., PDAs, video cameras),
wearable devices, devices embedded in appliances
• Several LANs linked by backbone connections
• Mobile computing, ubiquitous computing, location-aware computing
• Connected to the Internet via a router
• In the figure above: 3 different forms of wireless connections: wireless
• Firewalls protects an intranet by preventing unauthorized messages LAN, mobile phone through WAP, infra-red link
leaving or entering; implementing by filtering messages
9 10
Distributed Systems, Spring 2003 Distributed Systems, Spring 2003
http://www.google.comlsearch?q=kindberg
Computers in the Internet
www.google.com
www.cdk3.net Internet
http://www.cdk3.net/
www.w3c.org
Date Computers Web servers
File system of http://www.w3c.org/Protocols/Activity.html
Protocols
www.w3c.org
1979, Dec. 188 0
Activity.html 1989, July 130,000 0
1999, July 56,218,000 5,560,866
• WWW a system for publishing and accessing resources and services
across the Internet
• Web browsers act as client
• Request resources (e.g., web pages) from web servers
• CERN , 1989
• Hypertext structure among documents
11 12
Distributed Systems, Spring 2003 Distributed Systems, Spring 2003
2
Computers vs. Web servers in the Internet
Goals
13 14
Distributed Systems, Spring 2003 Distributed Systems, Spring 2003
Typical resources
access transparency
Printers, computers, storage facilities,
data, files Hide differences in data representation and how
Why sharing? a resource is accessed
Security
Unwanted Communication
15 16
Distributed Systems, Spring 2003 Distributed Systems, Spring 2003
Hide that a resource may move to another location Hide that a resource may be shared by several
competitive users
leave the resource in a consistent state
relocation transparency more refined mechanism: transactions
Hide that a resource may move to another location
while in use
example, mobile users
17 18
Distributed Systems, Spring 2003 Distributed Systems, Spring 2003
3
Transparency in a Distributed System Different Forms of Transparency in a
Distributed System (summary)
failure transparency
19 20
Distributed Systems, Spring 2003 Distributed Systems, Spring 2003
Openness
Goals Open distributed system
3. Openness
• Rules formalized in protocols
4. Scalability • Services specified through interfaces (described in an Interface
Definition Language (IDL) (but only the syntax part)
• Neutral and complete specifications (with regards to a potential
implementation)
23 24
Distributed Systems, Spring 2003 Distributed Systems, Spring 2003
4
Openness
Openness • A system organized as a collection of relatively small and easily
replaceable or adaptable components
•Provide definitions of the internal parts of the system as well
• Interoperability: to what extend can work together
• Portability: to what extend an application developed for A
can be executed on B that implements the same interface
• Separate Policy from Mechanism
with A
A distributed system provides only mechanisms
Policies specified by applications and users
Example policies:
• What level of consistency do we require for client-cached data?
• Which operations do we allow downloaded code to perform?
• Which QoS requirements do we adjust in the face of varying
bandwidth?
• What level of secrecy do we require for communication?
25 26
Distributed Systems, Spring 2003 Distributed Systems, Spring 2003
Scalability Problems
Scalability
Concept Example
Along three different dimensions: Centralized services A single server for all users
• size (number of users and/or processes) Centralized data A single on-line telephone book
• geographical (maximum distance between nodes) Centralized algorithms Doing routing based on complete information
27 28
Distributed Systems, Spring 2003 Distributed Systems, Spring 2003
Scalability
Scaling Techniques
Geographical scalability:
Three techniques:
Synchronous communication
In WAN, Unreliable and point-to-point
• hiding communication latencies
How to scale a distributed system across multiple, independent
• distribution
administrative domains: conflicting policies with respect to
resource usage (and payment), management and security
• replication
Expand to a new domain
• Protect itself against malicious attacks from the new
domain
• The new domain has to protect itself against malicious
attacks from the distributed system
29 30
Distributed Systems, Spring 2003 Distributed Systems, Spring 2003
5
Scaling Techniques
Scaling Techniques
31 32
Distributed Systems, Spring 2003 Distributed Systems, Spring 2003
Example:
(1) The World Wide Web
33 34
Distributed Systems, Spring 2003 Distributed Systems, Spring 2003
Scaling Techniques
Replication
Caching (client-driven)
Models
• increase availability
35 36
Distributed Systems, Spring 2003 Distributed Systems, Spring 2003
6
System Models System Models
1. Architectural models
Fundamental models are concerned with a more formal description
2. Fundamental models of the properties that are common in all of the architectural models
Models:
An architectural model of a distributed system is
concerned with the placement of its parts and the • Interaction model deals with performance and with the difficulty
relationships between them of setting time limits in distributed systems, for example for
message delivery
Examples include the client-server model and the p2p model
• Failure model gives a precise specification of the faults that can
• determine the distribution of data and computational tasks be exhibited by processes and communication channels. Defines
amongst the physical nodes of the system reliable communication and correct processes.
• useful when evaluating the performance, reliability, scalability
and other properties of distributed systems • Security model discusses the possible threats to processes and
communication channels.
37 38
Distributed Systems, Spring 2003 Distributed Systems, Spring 2003
39 40
Distributed Systems, Spring 2003 Distributed Systems, Spring 2003
Based on whether they set time limits (lower and upper bounds) Classification of failures:
on:
•Omission failures:
• Process execution speeds when a process or communication channel fails to perform
actions that is supposed to do
• Message transmission delays
Process omission failure: crash
• Clock drift rates
Fail-stop crash is other processes can detect certainly
that the process has crashed
Synchronous distributed systems (can set timeouts, can be built)
Asynchronous distributed systems (e.g., Internet, web)
7
Failure Model Failure Model
Classification of failures:
Communication channel
Outgoing message buffer Incoming message buffe
43 44
Distributed Systems, Spring 2003 Distributed Systems, Spring 2003
In synchronous systems:
Class of failure Affects Description
Fail-stop Process Process halts and remains halted. Other processes may
detect this state. Class of Failure Affects Description
Crash Process Process halts and remains halted. Other processes may Clock Process Process’s local clock exceeds the bounds on its
not be able to detect this state. rate of drift from real time.
Omission Channel A message inserted in an outgoing message buffer never Performance Process Process exceeds the bounds on the interval
arrives at the other end’s incoming message buffer. between two steps.
Send-omission Process A process completes a send,but the message is not put Performance Channel A message’s transmission takes longer than the
in its outgoing message buffer. stated bound.
Receive-omission Process A message is put in a process’s incoming message
buffer, but that process does not receive it.
Arbitrary Process or Process/channel exhibits arbitrary behaviour: it may
(Byzantine) channel send/transmit arbitrary messages at arbitrary times,
commit omissions; a process may stop or take an
incorrect step.
45 46
Distributed Systems, Spring 2003 Distributed Systems, Spring 2003
47 48
Distributed Systems, Spring 2003 Distributed Systems, Spring 2003
8
Classification of Multiple CPU Computer Systems
Hardware Concepts Multicomputers: each machine has its own private memory.
Either Homogeneous or Heterogeneous
Switch
49 50
Distributed Systems, Spring 2003 Distributed Systems, Spring 2003
Hardware Concepts
Multiprocessors
Scalability
Different method to connect the memory with the CPU ⇒
divide the memory in modules
51 52
Distributed Systems, Spring 2003 Distributed Systems, Spring 2003
CPU-to-CPU communication
aka System Area Networks (SANs)
53 54
Distributed Systems, Spring 2003 Distributed Systems, Spring 2003
9
Homogeneous Multicomputer Systems Heterogeneous Multicomputer Systems
transparency is harder
Software Concepts
• Much like an OS (resource managers, hides underlying hardware)
•Tightly-coupled (maintain a global view) – loosely coupled
57 58
Distributed Systems, Spring 2003 Distributed Systems, Spring 2003
59 60
Distributed Systems, Spring 2003 Distributed Systems, Spring 2003
10
Multicomputer Operating Systems Multicomputer Operating Systems
Processor communication by message passing
• Often no simple global communication
General structure
• No simple system-wide synchronization mechanisms
• Virtual (distributed) shared memory requires OS to
maintain global memory map in software (Distributed
Shared Memory (DSM) vs Only message passing
• Inherent distributed resource management: no central
Each node has each own kernel: modules for managing local resources point where allocation decisions can be made
(memory, local CPU, local disk, etc) + handling interprocess communication
(sending and receiving messages to and from other nodes)
Practice: only very few truly multicomputer OS exist
Common layer of software: implements the OS as a virtual machine
supporting parallel and concurrent execution of tasks.
Facilities: assigning a task to a processor, providing transparent storage,
general interprocess communication
61 62
Distributed Systems, Spring 2003 Distributed Systems, Spring 2003
63 64
Distributed Systems, Spring 2003 Distributed Systems, Spring 2003
The address space is divided up into pages with the General structure
pages being spread over all the processors in the
system
When a processor references an address that is not
present locally, a trap occurs, and the OS fetches
the pafe
65 66
Distributed Systems, Spring 2003 Distributed Systems, Spring 2003
11
Network Operating System Network Operating System
Middleware
Network Operating Systems
Some characteristics:
• Each computer has its own OS with networking facilities
• Computers work independently (i.e, they may even have
different OS)
• Services are to individual nodes (ftp, telnet, www)
• Highly file oriented (basically, processors share only
files)
• Compared to distributed OSs
• Middleware itself does not manage an individual mode
• Lack of transparency (harder to use; need to be
managed independently) • OS on each computer need not know about the other computers
• Easier to add/remove a machine (scalability, • OS on different computers need not be the same
openness) • Services are generally (transparently) distributed across
computers
69 70
Distributed Systems, Spring 2003 Distributed Systems, Spring 2003
Based on some model or paradigm, such as: Communication services (offer high-level communication
facilities to hide low-level message passing)
• all resources are treated as files (UNIX and Plan 9)
• Procedure calls across networks
• Distributed file systems
• Remote-object method invocation
• Remote Procedure Calls (RPCs): allow a process to call a
procedure whose implementation is located on a remote • Message-queuing systems
machine
• Advanced communication streams
• Distributed objects: transparently invoke objects
• Event notification service
residing on remote machines
• Distributed documents
71 72
Distributed Systems, Spring 2003 Distributed Systems, Spring 2003
12
Middleware Services Middleware Services
Information system services (help manage data) Control services (giving applications control over when,
where and how they access data)
• Large scale system-wide naming services
• Code migration
• Advanced directory services (search engines)
• Distributed transaction processing
• Location services for tracking mobile objects
• Persistent storage facilities
Security services
• Data caching and replication
• Authentication and authorization services
• Simple encryption services
• Auditing service
73 74
Distributed Systems, Spring 2003 Distributed Systems, Spring 2003
Number of copies of OS 1 N N N
Shared
Basis for communication Messages Files Model specific
memory
Global, Global,
Resource management Per node Per node
central distributed
Scalability No Moderately Yes Varies
In an open middleware-based distributed system, the protocols
used by each middleware layer should be the same, as well as the Openness Closed Closed Open Open
interfaces they offer to applications.
75 76
Distributed Systems, Spring 2003 Distributed Systems, Spring 2003
77 78
Distributed Systems, Spring 2003 Distributed Systems, Spring 2003
13
Application Layering Application Layering
result result
Server
Client
Key:
Process: Computer:
81 82
Distributed Systems, Spring 2003 Distributed Systems, Spring 2003
83 84
Distributed Systems, Spring 2003 Distributed Systems, Spring 2003
14
Alternative Architectures
Collaborating servers
Cooperating servers: service is physically distributed across a
Service
collection of services:
• Traditional multi-tiered architectures
• Replicated files systems Server
85 86
Distributed Systems, Spring 2003 Distributed Systems, Spring 2003
Modern Architectures
Extra Slides
87 88
Distributed Systems, Spring 2003 Distributed Systems, Spring 2003
monitor Counter {
private:
int count = 0;
public:
1.11
int value() { return count;}
void incr () { count = count + 1;}
void decr() { count = count – 1;}
}
15
Multiprocessor Operating Systems Distributed Shared Memory Systems
1.18
An Example Client and Server (2) An Example Client and Server (3)
1-27 b
A sample server.
95
A client using the server to copy a file. 96
Distributed Systems, Spring 2003 Distributed Systems, Spring 2003
16
Figure 2.4 Figure 2.5
Web proxy server A distributed application based on peer
processes
Application Application
Client Web
server Coordination Coordination
code code
Proxy
server
Client Web
server
Application
Coordination
code
97 98
Distributed Systems, Spring 2003 Distributed Systems, Spring 2003
Web
Client Applet server
99 100
Distributed Systems, Spring 2003 Distributed Systems, Spring 2003
Hotel wireless
send
network Z
Discovery
service receive receive
Camera
m3 m1 m2
A
receive receive receive
TV/PC Guests t1 t2 t3
Laptop PDA
devices
101 102
Distributed Systems, Spring 2003 Distributed Systems, Spring 2003
17
Figure 2.13 Figure 2.14
Objects and principals The enemy
Access rights Object
invocation
Copy of m
103 104
Distributed Systems, Spring 2003 Distributed Systems, Spring 2003
Figure 2.15
Secure channels
Principal A Principal B
105
Distributed Systems, Spring 2003
18