1

Term Test 1

1. What is a distributed system? What are examples of distributed systems?

Answer: A distributed system is a collection of independent computers that are used

jointly to perform a single task or to provide a single service.

Examples:
★ Collection of Web Servers: Distributed database of hypertext and multimedia
documents.
★ Distributed file system on a LAN
★ Domain Name System(DNS)
★ Cray XK7 & CLE(massive multiprocessor)
★ Telephone networks and cellular network
★ Computer networks such as the Internet or intranet
★ ATM(bank) machines
★ Distributed database and distributed database management system
★ Network of workstation
★ Mobile computing etc.

2. Define a middleware and show in a small diagram where it is positioned.

Answer: Middleware: Middleware is software that is used to bridge the gap between
applications and operating systems. Middleware sits between an operating system and
the applications that run on it to provide a method of communication and data
management.
 2

3. What is Masquerading? Explain briefly the main security techniques.

Answer: Masquerading: Sending/receiving messages, using the identity of another

principal without their authority.

The main security techniques are explained briefly below:

★ Access Control: Access control ensures that users/processes access computer

resources in a controlled and authorized manner. It implements resource
protection, e.g. file protection, and is essential for distributed systems(remote
control).
★ Firewalls: A firewall is a set of processes, at different protocol levels like IP
packet filtering, TCP gateway, application-level gateway, and Bastion. It monitors
and controls all communication into and out of an intranet.
★ Cryptographic Algorithms: 1. Ciphers 2. Authentication 3. Digital signatures

4. People/Programmers often made some false assumptions about distributed

systems. Write four of those false assumptions and also write the facts for each of
them.

Answer:
1. The network is reliable: The network is reliable and will lead to trouble when the
network starts dropping packets.
2. Latency is zero: Assuming latency is zero will undoubtedly lead to scalability
problems as the application grows geographically, or is moved to a different kind
of network.
3. Bandwidth is infinite: Assuming that you continue to increase data size on a
channel without limit, can be quite a mistake. This problem only turns its head
when scale difficulties enter the conversation and specific communication
channels hit their limits.
4. The network is secure: Assuming you can trust the network you are on or the
people you are building your system for can be a crucial mistake.

5. Write the Persistent Asynchronous and Transient Synchronous(Delivery-based)

communication properties with the figure.

Answer:

1. Persistent Asynchronous:
 3

★ A sends a message and continues

★ B is not running
★ A stopped running
★ B starts and receives a message

2. Transient Synchronous(Delivery based)

● A sends a request and waits until accepted.

● Request is received(B)
● B is running but doing something else
● Process request

6. Give one example for each.

Answer:
Distributed Hash Table: Chord, Pastry, Kademlia.

Super-peer Networks: Skype

Collaborative Distributed Systems: BitTorrent

Edge Server Network: Akamai, CloudFront, coralCDN.

Term Test 2

1. What is synchronization and coordination? Briefly explain the Berkeley algorithm

in distributed system?

Answer:
Synchronization: Synchronization refers to the process of aligning the clocks or
timestamps of different nodes in a distributed system./ the right time/ordering of all
actions.
 4

Coordination: Coordination focuses on the management of shared resources and the

sequencing of operations to prevent and ensure data integrity./the right thing./coordinate
actions and agree on values.

Berkeley Algorithm:

The Berkeley algorithm assumes no machine has an accurate time source, and obtains
the average participating computers all clocks to average. Machine run time daemon
process that implements the protocol. One machine is elected(or designated) as the
server(master) then others are slave. Master polls each machine periodically where ask
each machine for time and can use Christian's algorithm to compensate for network
latency. When results are in compute average including master's time, average cancels
out individuals clock tendencies to run fast or slow. It sends offset by which each clock
needs adjustment to each slave,avoids problems with network delays if we want a time
stamp. The algorithm has pŕovisions for ignoring readings from clocks whose skew is too
great to compute a fault-tolerant average if the master fails any slave can take over.

Example:

2. In a distributed system, distinguish between forward and backward recovery.

Answer:

Forward Recovery Backward recovery

In forward recovery states if a transaction is a In backward recovery, if there is any error

failure then it removes the error and then it restarts the process.
 5

continues the process.

More complex as it requires anticipating Simpler, as it relies on existing checkpointing

potential errors and implementation and rollback mechanisms.
mechanisms to handle them.

Can be more efficient as it avoids Can be less efficient as rollbacks can be

unnecessary rollbacks. resource-intensive.

Suitable for systems where performance is Suitable for systems where consistency is
critical and rollbacks are costly. paramount and rollbacks are acceptable.

Example: Erasure correction missing packet Example: Packet retransmission when a

reconstructed from successfully delivered packet is lost.
packets.

3. What is partial failure? Explain with an example the justification of the CAP
theorem.

Answer: Partial Failure: One or more(but not all) components in a distributed system fail.

CAP theorem: C: Consistency: Linearisability

A: Availability: Timely response
P: Partition-Tolerance: Functions in the face of a partition.

According to CAP theory, any distributed system can only guarantee two of the three properties
at any point in time. You can’t guarantee all three properties at once.
 6

Justification of CAP theorem with an Example:

Banking System(C & A): Imagine a banking system that prioritizes consistency and availability.
In this scenario, every time a client requests their account balance or processes a transaction,
they expect to receive the most up-to-date information. However, if a network partition occurs,
the system will be unable to continue processing requests, potentially causing service disruption
and loss of data.

Social Media Platform(A & P): A social media platform like Twitter, on the other hand, may
prioritize availability and partition tolerance. This ensures that the platform remains accessible
even during network partitions. In this case, users may see slightly outdated content, but they
can still interact with the platform without significant disruptions.

4. What is Distributed Shared Memory(DSM)? Describe the application for DSM.

Answer: DSM: DSM is the collection of shared memory(SM) and multicomputer(MC)./

DSM is a mechanism for allowing user processes to access shared data without using
interprocess communication.

DSM = MC + SM
 7

Application:

1. Scientific parallel computing

★ Bioinformatics (gene sequence analysis)
★ Simulations (climate modeling, economic modeling)
★ Data processing (physics, astronomy)
2. Graphics (image processing, rendering)
3. Data server (distributed FS, Web server)
4. Data Storage

Chapter-01

Introduction to Distributed Computing

Distributed System: A distributed system is a collection of independent computers that are

used jointly to perform a single task or to provide a single service./
A distributed system is a collection of independent computers, interconnected via a network,
capable of collaborating on a task.

Features: A distributed system can be characterized as a collection of multiple autonomous

computers that communicate over a communication network and having the following features:

★ No common Physical clock Enhanced Reliability

★ Increased performance/cost ratio
★ Access to geographically remote data and resources
★ Scalability

Examples of Distributed System:

★ Telephone Networks and Cellular Networks

★ Computer Networks Such as internet or intranet
 8

★ ATM(bank) Machines
★ Distributed database and distributed database management system
★ Network of Workstations
★ Mobile Computing
★ Collection of Web servers: distributed database of hypertext and
multimedia documents
★ Distributed file system on a LAN
★ Domain Name Service (DNS)
★ Cray XK7 & CLE (massive multiprocessor)

Automatic Banking (Teller Machine) System in Distributed System:

★ Primary requirements: Security and Reliability

★ Consistency of replicated data.
★ Concurrent transactions(operations that involve accounts in different banks,
simultaneous access from several users etc)
★ Fault tolerance

Advantages of Distributed System:

★ Information Sharing among Distributed Users

★ Resource Sharing
★ Extensibility and Incremental growth
★ Shorter Response Time and Higher Output
★ Higher Reliability
★ Better Flexibility’s in meeting User’s needs
★ Better price/performance ratio
★ Scalability
★ Transparency

Or, What are economic and technical reasons for having distributed systems?/
Advantages of Distributed System

★ Cost: Better price/performance as long as commodity hardware is used for the

component computers.
★ Performance: By using the combined processing and storage capacity of many
nodes, performance levels can be reached that are out of the scope of
centralized machines.
★ Scalability: Resources such as processing and storage capacity can be
increased incrementally.
★ Reliability: By having redundant components, the impact of hardware and
 9

software faults on users can be reduced.

★ Inherent distribution: Some applications like the Web are naturally distributed.

Disadvantages of Distributed Systems:

★ Difficulties of developing distributed software

★ Networking Problems
★ Security Problems
★ Performance
★ Openness
★ Reliability and Fault Tolerance

Or, What problems are there in the use and development of distributed systems? /
Disadvantages of Distributed System

★ New component: Networks are needed to connect independent nodes, are

subject to performance limits.
★ Software complexity: Distributed software is more complex and harder to
develop than conventional software; hence, it is more expensive and harder to
get right.
★ Failure: More elements that can fail, and the failure must be dealt with.
★ Security: Easier to compromise distributed systems.

Hardware Architecture:
● Uniprocessor:

Properties: Single processor, direct memory access.

● Multicomputer:
 10

Properties: Multiple computers, No direct memory access, Network,

Homogeneous vs. Heterogeneous
● Multiprocessor:

Properties: Multiple processors, Direct memory access, Uniform memory access

(e.g., SMP, multicore), Non Uniform memory access (e.g., NUMA)
Software Architecture:
★ Uniprocessor OS:

★ Multiprocessor OS: Similar to a uniprocessor OS but:

→ Kernel designed to handle multiple CPUs
→ Number of CPUs is transparent
→ Communication uses same primitives as uniprocessor OS
→ Single system image

★ Network OS:
Properties:
➜ No single system image
 11

➜ Individual nodes are highly autonomous

➜ All distribution of tasks is explicit to the user
➜ Examples: Linux, Windows

★ Distributed OS:
Properties:
➜ High degree of transparency
➜ Single system image (FS, process, devices, etc.)
➜ Homogeneous hardware
➜ Examples: Amoeba, Plan 9, Chorus, Mungi

★ Middleware:
Properties:
➜ System independent interface for distributed programming
➜ Improves transparency (e.g., hides heterogeneity)
➜ Provides services (e.g., naming service, transactions, etc.)
➜ Provides programming model (e.g., distributed objects)
Kernel: It is the core that provides basic services for all other parts of
the OS.
 12

Why is Middleware ’Winning’?:

★ Builds on commonly available abstractions of network OSs(processes and
message passing)
★ Examples: RPC, NFS, CORBA, DCOM, J2EE, .NET
★ Also languages (or language modifications) specially designed for distributed
computing
★ Examples: Erlang, Ada, Limbo, etc.
➢ Usually runs in user space
➢ Raises the level of abstraction for programming ➼ less error-prone
➢ Independence from OS, network protocol, programming language, etc. ➼
Flexibility
➢ Feature dump and bloated interfaces

PARALLEL COMPUTING
● Parallel computing: improve performance by using multiple processors per
application
● There are two flavors:
1. Shared-memory systems:
➢ Multiprocessor (multiple processors share a single bus and
memory unit)
➢ SMP support in OS
➢ Much simpler than distributed systems
➢ Limited scalability
2. Distributed memory systems:
➢ Multicomputer (multiple nodes connected via a network)
➢ These are a form of distributed systems
➢ share many of the challenges discussed here
➢ Better scalability & cheaper

Parallel Computing and Parallel System: A system is said to be a Parallel System in

 13

which multiple processors have direct access to shared memory which forms a common
address space.
Usually, tightly coupled systems are referred to as Parallel Systems. In these systems,
there is a single system-wide primary memory (address space) that all the processors
share. On the other hand, Distributed Systems are loosely coupled systems.
Parallel computing is the use of two or more processors (cores, computers) in
combination to solve a single problem.

Applications of Parallel System:

★ An example of Parallel computing would be two servers that share the workload
of routing mail, managing connections to an accounting system or database,
solving a mathematical problem, etc.
★ Supercomputers are usually placed in parallel system architecture
★ Terminals connected to single server

Advantages of Parallel System:

★ Provide Concurrency(do multiple things at the same time)

★ Taking advantage of non-local resources
★ Cost Savings
★ Overcoming memory constraints
★ Save time and money
★ Global address space provides a user-friendly programming perspective to
memory

Disadvantages of Parallel System

★ Primary disadvantage is the lack of scalability between memory and CPUs.

★ Programmer responsibility for synchronization constructs that ensure "correct"
global memory access.
★ It becomes increasingly difficult and expensive to design and produce shared
memory machines with ever-increasing numbers of processors.

Parallel VS Distributed System

 14

Features Parallel System Distributed System

Memory Tightly coupled system Weakly coupled system

shared memory Distributed memory

Control Global clock control No global clock control

Processor interconnection Order of Tbps Order of Gbps

Main focus Performance Scientific Performance(cost and

computing scalability)
Reliability/availability
Information/resource
sharing.

Why parallel and distributed computing?***

Programming multi-core systems is fundamentally different from programming

traditional computers. Parallelism needs to be explicitly expressed in the program. This
is traditionally referred to as parallel computing. PDC is not really a choice anymore.

BASIC GOALS OF DISTRIBUTED SYSTEMS

We want distributed systems to have the following properties:

★ Transparency
★ Scalability
★ Dependability
★ Performance
★ Flexibility

1. TRANSPARENCY: Concealment of the separation of the components of a distributed

system (single image view). There are several forms of transparency
★ Access: Local and remote resources are accessed in the same way.
★ Location: Users are unaware of the location of resources.
★ Migration: Resources can migrate without a name change.
★ Replication: Users are unaware of the existence of multiple copies.
★ Failure: Users are unaware of the failure of individual components.
★ Concurrency: Users are unaware of sharing resources with others.
 15

2. SCALABILITY: A system is said to be scalable if it can handle the addition of

users and resources without suffering a noticeable loss of performance or
increase in administrative complexity.[B. Clifford Neuman]. The scale has three
dimensions:
★ Size: number of users and resources (problem: overloading)
★ Geography: distance between users and resources (problem:
communication)
★ Administration: number of organizations that exert administrative control
over parts of the system (problem: administrative mess

Note:
★ Scalability often conflicts with (small system) performance
★ Claim of scalability is often abused
Techniques for scaling:
★ Decentralization
★ Hiding communication latencies (asynchronous communication, reduce
communication)
★ Distribution (spreading data and control around)
★ Replication (making copies of data and processes)
Decentralization:
Avoid centralizing:
★ Services (e.g., single server)
★ Data (e.g., central directories)
★ Algorithms (e.g., based on complete information).
With regards to algorithms:
★ Do not require the machine to hold a complete system state.
★ Allow nodes to make decisions based on local info.
★ Algorithms must survive failure of nodes
★ No assumption of a global clock
Decentralization is hard.

3. DEPENDABILITY: Dependability of distributed systems is a double-edged

sword:
• Distributed systems promise higher availability:
– Replication
• But availability may degrade:
– More components ➼ more potential points of failure
Dependability requires consistency, security, and fault tolerance
 16

4. PERFORMANCE;
➜ Any system should strive for maximum performance
➜ In distributed systems, performance directly conflicts with some other
desirable properties
★ Transparency
★ Security
★ Dependability
★ Scalability

5. FLEXIBILITY:
★ Build a system out of (only) required components
★ Extensibility: Components/services can be changed or added
★ Openness of interfaces and specification
➢ allows reimplementation and extension
★ Interoperability
★ Separation of policy and mechanism
➢ standardized internal interfaces

COMMON MISTAKES: False assumptions commonly made:

★ Reliable network
★ Zero latency
★ Infinite bandwidth
★ Secure network
★ Topology does not change
★ One administrator
★ Zero transport cost
★ Everything is homogeneous

Moore’s Law: the number of transistors double every 18 months.

Previous Year Question Solving:

1. What is a Distributed/Parallel System? Write down the advantages/disadvantages

of it.

Answer: Distributed System: A distributed system is a collection of independent computers

that are used jointly to perform a single task or to provide a single service.

Parallel System: A system is said to be a Parallel System in which multiple processors

have direct access to shared memory which forms a common address space.
 17

Advantages and disadvantages of Distributed System:

Advantages of Distributed System:

★ Information Sharing among Distributed Users

★ Resource Sharing
★ Extensibility and Incremental growth
★ Shorter Response Time and Higher Output
★ Higher Reliability
★ Better Flexibility in meeting User’s needs
★ Better price/performance ratio
★ Scalability
★ Transparency

Disadvantages of Distributed System:

★ Difficulties of developing distributed software

★ Networking Problem
★ Security Problems
★ Performance
★ Openness
★ Reliability and Fault Tolerance

Advantages and Disadvantages of Distributed System:

Advantages of Parallel System:

★ Provide Concurrency(do multiple things at the same time)

★ Taking advantage of non-local resources
★ Cost Savings
★ Overcoming memory constraints
★ Save time and money
★ Global address space provides a user-friendly programming perspective to
memory

Disadvantages of Parallel System

★ Primary disadvantage is the lack of scalability between memory and CPUs.

★ Programmer responsibility for synchronization constructs that ensure "correct"
global memory access.
★ It becomes increasingly difficult and expensive to design and produce shared
memory machines with ever-increasing numbers of processors.
 18

2. Write down the difference between parallel and distributed systems.

Answer:

Features Parallel System Distributed System

Memory Tightly coupled system Weakly coupled system

shared memory Distributed memory

Control Global clock control No global clock control

Processor interconnection Order of Tbps Order of Gbps

Main focus Performance Scientific Performance(cost and

computing scalability)
Reliability/availability
Information/resource
sharing.

Chapter-02

System Architecture and Communication

PRINCIPLES: Several key principles underlying the functioning of all distributed

systems
★ System Architecture
★ Communication
★ Replication and Consistency
★ Synchronisation
★ Naming
★ Fault Tolerance
★ Security

Software Architecture: Logical organization and roles of software components

 19

★ Layered
★ Object-oriented
★ Data-centered
★ Service-oriented
★ Event-based
There is no single best architecture
★ depends on application requirements
★ and the environment!

CLIENT-SERVER MODEL

STRUCTURED OVERLAY

● Distributed Hash Table:

★ Nodes have identifier and range, Data has identifier
★ Node is responsible for data that falls in its range
★ Search is routed to appropriate node
★ Examples: Chord, Pastry, Kademlia

HYBRID ARCHITECTURES: Combination of architectures.

Examples:
★ Superpeer networks
★ Collaborative distributed systems
★ Edge-server systems
 20

1. Super Peer Networks:

★ Regular peers are clients of super peers

★ Super Peers are servers for regular peers
★ Super Peers are peers among themselves
★ Superpeers may maintain large indexes, or act as brokers
★ Example: Skype

2. Collaborative Distributed Systems:

★ Example: BitTorrent
★ Node downloads chunks of file from many other nodes
★ Node provides downloaded chunks to other nodes
★ Tracker keeps track of active nodes that have chunks of file
★ Enforce collaboration by penalizing selfish nodes

3. Edge-Server Networks:

★ Servers placed at the edge of the network

★ Servers replicate content
★ Mostly used for content and application distribution
★ Content Distribution Networks: Akamai, CloudFront, CoralCDN

STATEFUL VS STATELESS SERVERS

Stateful:
★ Keeps persistent information about clients
★ Improved performance
 21

★ Expensive crash recovery

★ Must track clients
Stateless:
★ Does not keep the state of clients
★ soft state design: limited client state
★ Can change own state without informing clients
★ No cleanup after crash
★ Easy to replicate
★ Increased communication

COMMUNICATION MODES
★ Data oriented vs control oriented communication
★ Synchronous vs asynchronous communication
★ Transient vs persistent communication
★ Provider-initiated vs consumer-initiated communication
★ Direct -addressing vs indirect-addressing communication

1. Data-Oriented vs Control-Oriented Communication:

Data-oriented communication
★ Facilitates data exchange between threads
★ Shared address space, shared memory & message passing
Control-oriented communication
★ Associates a transfer of control with communication
★ Active messages, remote procedure call (RPC) & remote method
invocation (RMI)

Feature Data-Oriented Communication Control-Oriented

Communication

Focus Data exchange Control transfer and

execution

Mechanis Sending/receiving data objects Sending

m commands/receiving
responses
 22

Examples Publish-subscribe, message RPCs, RMI

queues, data streams

Coupling Decoupled Tightly coupled

Synchroniz Asynchronous Synchronous

ation

Scalability Highly scalable Less scalable

Flexibility Highly flexible Less flexible

Application Large-scale data sharing, Remote services, device

s microservices control

2. Synchronous vs Asynchronous Communication:

Synchronous
★ Sender blocks until message received
■ Often sender blocked until the message is processed and a reply
received
★ Sender and receiver must be active at the same time
★ Receiver waits for requests, processes them (ASAP), and returns reply
★ Client-Server generally uses synchronous communication

Asynchronous
★ Sender continues execution after sending the message (does not block
waiting for reply)
★ Message may be queued if receiver not active
★ Message may be processed later at receiver’s convenience

3. Transient vs Persistent Communication

Transient
★ Message discarded if cannot be delivered to receiver immediately
★ Example: HTTP request
Persistent
★ Message stored (somewhere) until receiver can accept it
 23

★ Example: email
Coupling: Time

4. Provider-Initiated vs Consumer-Initiated Communication

Provider-Initiated
★ Message sent when data is available
★ Example: notifications
Consumer-Initiated
★ Request sent for data
★ Example: HTTP request

5. Direct-Addressing vs Indirect-Addressing Communication

Direct-Addressing
★ Message sent directly to receiver
★ Example: HTTP request
Indirect-Addressing
★ Message not sent to a particular receiver
★ Example: broadcast, publish/subscribe
Coupling: Space

COMBINATIONS:
There are a number of alternative ways, or modes, in which communication can take
place. It is important to know and understand these different modes because they are
used to describe the different services that a communication subsystem offers to higher
layers. The first distinction is between the two modes of data-oriented communication
and control-oriented communication. In the first mode, communication serves solely to
exchange data between processes. Although the data might trigger an action at the
receiver, there is no explicit transfer of control implied in this mode. The second mode,
control-oriented communication, explicitly associates a transfer of control with every
data transfer. Data-oriented communication is clearly the type of communication used in
communication via shared address space and shared memory, as well as message
passing. Control-oriented communication is the mode used by abstractions such as
remote procedure call, remote method invocation, active messages, etc.
(communication abstractions are described in the next section). Next, communication
operations can be synchronous or asynchronous. In synchronous communication the
sender of a message blocks until the message has been received by the intended
recipient. Synchronous communication is usually even stronger than this in that the
sender often blocks until the receiver has processed the message and the sender has
received a reply. In asynchronous communication, on the other hand, the sender
 24

continues execution immediately after sending a message (possibly without having

received an answer). Another possible alternative involves the buffering of
communication. In the buffered case, a message will be stored if the receiver is not able
to pick it up right away. In the unbuffered case, the message will be lost. Communication
can also be transient and persistent. In transient communication, a message will only be
delivered if a receiver is active. If there is no active receiver process (i.e., no one
interested in or able to receive messages) then an undeliverable message will simply be
dropped. In persistent communication, however, a message will be stored in the system
until it can be delivered to the intended recipient. As Figure 6 shows, all combinations of
synchronous/asynchronous and transient/persistent are possible.

There are also varying degrees of reliability of the communication. With reliable
communication, errors are discovered and fixed transparently. This means that the
processes can assume that a message that is sent will arrive at the destination (as long
as the destination process is there to receive it). With unreliable communication,
messages may get lost and processes have to deal with it. Finally, it is possible to
provide guarantees about the ordering of messages. Thus, for example, a
communication system may guarantee that all messages are received in the same
order that they are sent, while another system may make no guarantees about the order
of arrival of messages.

REQUEST-REPLY COMMUNICATION

Request:
★ a service
★ data
Reply:
★ result of executing service
★ data
Requirement:
★ Message formatting
★ Protocol
 25

REMOTE PROCEDURE CALL (RPC)

Idea: Replace I/O oriented message passing model by execution of a procedure call on
a remote node [BN84]
★ Synchronous - based on blocking messages
★ Message-passing details hidden from application
★ Procedure call parameters used to transmit data
★ Client calls local “stub” which does messaging and marshaling Confusion of local
and remote operations can be dangerous.

RPC IMPLEMENTATIONS

1. client calls client stub (normal procedure call)

2. client stub packs parameters into message data structure
3. client stub performs send() syscall and blocks
4. kernel transfers the message to remote kernel
5. remote kernel delivers to server stub, blocked in receive()
6. server stub unpacks message, calls server (normal proc call)
7. server returns to the stub, which packs result into message
8. server stub performs send() syscall
9. kernel delivers to client stub, which unpacks and returns

ASYNCHRONOUS RPC

★ When no reply is required

★ When the reply isn’t needed immediately (2 asynchronous RPCs - deferred
synchronous RPC)
 26

REMOTE METHOD INVOCATION (RMI)

The transition from Remote Procedure Call (RPC) to Remote Method Invocation (RMI)
is a transition from the server metaphor to the object metaphor.
Why is this important?
★ RPC: explicit handling of host identification to determine the destination
★ RMI: addressed to a particular object
★ Objects are first-class citizens
★ Can pass object references as parameters
★ More natural resource management and error handling
★ But still, only a small evolutionary step

TRANSPARENCY CAN BE DANGEROUS

Why is the transparency provided by RPC and RMI dangerous?
★ Remote operations can fail in different ways
★ Remote operations can have arbitrary latency
★ Remote operations have a different memory access model
★ Remote operations can involve concurrency in subtle ways
What happens if this is ignored?
★ Unreliable services and applications
★ Limited scalability
★ Bad performance

Chapter-03

Security & Threats

★ Security: is a set of measures to guarantee the privacy, integrity, and availability

of resources(objects, databases, servers, processes, channels, etc) and involves
protection of objects and securing processes and communication channels.
★ Security policies: Specify who is authorized to access resources (e.g. file
ownership)
★ Security mechanisms: Enforce security policy (e.g. file access control)
 27

Threat & Attack

★ Threat: A potential for violation of security, which exists when there is a
circumstance, capability, action, or event that could breach security and cause
harm. That is, a threat is a possible danger that might exploit a vulnerability.

★ Attack: An assault on system security that derives from an intelligent threat; that
is, an intelligent act that is a deliberate attempt (especially in the sense of a
method or technique) to evade security services and violate the security policy of
a system.

Security Model
★ Object: intended for use by different clients, via remote invocation
★ Principal: authority on whose behalf invocation is issued

Security Threat

★ Online shopping/banking
➢ intercept credit card information
➢ Purchase goods using stolen credit card details
➢ Replay bank transaction, e.g. credit an account
★ Online stock market information service
➢ observe frequency or timing of requests to deduce useful information, e.g.
the level of stock
★ Website
➢ flooding with requests (denial of service)
★ My computer
➢ receive/download malicious code (virus)

Types of Security Threats

★ Eavesdropping: Obtaining copies of messages without authority

★ Masquerading: Sending/receiving messages using the identity of another
principal without their authority
★ Message tampering: Intercepting and altering messages
★ Replaying: Intercepting, storing and replaying messages
★ Denial of service: Flooding a channel with requests to deny access to others

Defeating the enemy: How?

★ Encryption (scrambling a message to hide its contents)
 28

➢ does not prove the identity of sender

★ Shared secrets (keys)
➢ messages encrypted with the shared key
➢ can only be decrypted if the key is known
★ Identification (are you who you are?)
➢ password protection, etc
★ Authentication (are you who you say you are?)
➢ include in message identity of principal/data, timestamp
➢ encrypt with shared key

Secure channels

Processes: reliably know the identity of principal Messages: protected against

tampering, timestamped to prevent replaying/reordering.

Designing Secure Systems

★ Basic message
➢ Networks are insecure
➢ Interfaces are exposed
★ Threat analysis
➢ assume worst-case scenario
➢ list all threats - complex scenarios!!!
★ Design guidelines
➢ log at points of entry so that violations are detected
➢ limit the lifetime and scope of each secret
➢ publish algorithms, restrict access to shared keys
➢ minimize trusted base

Main Security Techniques

★ Access control
➢ implement resource protection, e.g. file protection
➢ essential in distributed systems (remote login)
★ Firewalls
➢ monitor traffic into and out of intranet
★ Cryptographic algorithms
 29

➢ ciphers
➢ authentication
➢ digital signatures

Main Security Techniques: Access Control

★ Definition
➢ ensure that users/processes access computer resources in a controlled
and authorized manner
★ Protection domain
➢ is a set of rights for each resource, e.g. Unix files
➢ associated with each principal
★ Two implementations of protection domains
➢ Capabilities
★ request accompanied by key, simple access check
★ open to key theft, or key retained when person left company
➢ Access control lists
★ list of rights stored with each resource
★ request requires authentication of principal

How does it(Access Control) work? Reference Monitor

★ intercepts all access attempts
★ authenticates request and principal’s credentials
★ applies access control
➢ If Yes, access proceeds
➢ If No, access is denied, error message returned to the subject

Firewalls

★ A set of processes, at different protocol levels:

★ IP packet filtering
➢ screening of source & destination, only ‘clean’ packets proceed
➢ performed in OS kernel of router
★ TCP gateway
➢ monitors TCP connection requests
★ Application-level gateway
➢ runs proxy for an application on TCP gateway, e.g. Telnet
★ Bastion
➢ separate computer within intranet
➢ protected by IP packet filtering, runs TCP/application gateway
 30

Firewalls: How does it work?

★ Monitor and control all communication into and out of an intranet.
★ Service control
➢ filter requests for services on internal hosts
➢ e.g. reject HTTP requests unless to official web server
★ Behaviour control
➢ prevent illegal or anti-social behavior
➢ e.g. filter ‘spam’ messages
★ User control
➢ allow access to an authorized group of users
➢ e.g. dial-up services

Firewall Configurations

Key Distributions

★ Symmetric key cryptography

■ requires sender, receiver know shared secret key

★ Public key cryptography

■ sender, receiver do not share secret key
■ public encryption key known to all
■ private decryption key known only to receiver

Symmetric Key Cryptography

symmetric key crypto: Bob and Alice share same (symmetric) key: K
• e.g., key is knowing substitution pattern in mono alphabetic substitution cipher

Public Key Cryptography

 31

Cryptographic Algorithms

★ Encryption
➢ apply rules to transform plaintext to cipher text
➢ defined with a function F and key K
➢ denote message M encrypted with K by FK(M) = {M}K
★ Decryption
➢ uses inverse function
-1
★ F K({M}K) = M
➢ can be symmetric (based on secret key known to both parties)
➢ or asymmetric (based on public key)
➢ separate computer within intranet
➢ protected by IP packet filtering, runs TCP/application gateway

★ Symmetric (secret key): TEA, DES

➢ secret key shared between principals
➢ encryption with non-destructive opns (XOR) plus transpose
➢ decryption possible only if key known
➢ brute force attack (check {M}K for all values of key) hard (exponential in
no of bits in key)

★ Asymmetric (public key): RSA

➢ pair of keys (very large numbers), one public and one private
➢ encryption with public key
➢ decryption possible only if private key known
➢ factorizing large numbers (over 150 decimal digits) hard

Symmetric Cryptography

★ Tiny Encryption Algorithm(TEA)

➢ simple & concise, yet secure and reasonably fast

➢ Simple, symmetric (secret key) algorithm
➢ written in C [Wheeler & Needham 1994]
➢ key 128 bits (k[0]..k[3]),plaintext 64 bits (2 x 32 bits, text[0], text[1])
➢ in 32 rounds combines plaintext and key, swapping the two halves of
plaintext, XOR (^) and bitwise shift (<< >>)
 32

★ DES (The Data Encryption Standard 1977)

➢ US standard for business applications till recently

➢ 64-bit plaintext, 56 bit key
➢ cracked in 1997 (secret challenge message decrypted)
➢ triple-DES (key 112 bits) still secure, poor performance

★ AES (Advanced Encryption Standard)

➢ invitation for proposals 1997
➢ in progress
➢ key size 128, 192 and 256 bits

Asymmetric Cryptography

★ Trap-door functions
➢ pair of keys (e.g. large numbers)
➢ encryption function easy to compute (e.g. multiply keys)
➢ decryption function infeasible unless secret known (e.g. factorise the
product if one key not known)
★ Idea
➢ two keys produced: encryption key made public, decryption key kept
secret
➢ anyone can encrypt messages, only a participant with decryption key
can operate the trap door
★ Examples
➢ a few practical schemes: RSA

RSA(Rivest, Shamir, and Adelman)

➢ How it works
★ relies on N = P × Q (product of two very large primes)
★ factorization of N hard
★ choose keys e, d such that e × d = 1 mod Z where Z = (P-1) × (Q-1)
➢ It turns out...
★ can encrypt M by Me mod N
★ can decrypt by Cd mod N (C is encrypted message)
➢ Thus
★ Can freely make e and N public, while retaining d. In 1978 Rivest et al
thought factorizing numbers > 10200 would take more than four billion
years. Now (ca 2000)– faster computers, better methods numbers with
 33

155 (= 500 bits) decimal digits successfully factorized

Is 512-bit keys insecure? The future?
★ keys with 230 decimal digits (= 768 bits) recommended and 2048 bits
used in some applications (e.g. defense)

Digital Signatures

★ Why needed?
➢ alternative to handwritten signatures
➢ authentic, difficult to forge, and undeniable
★ How it works
➢ relies on secure hash functions which compress a message into a
so-called digest
➢ sender encrypts the digest and appends it to the message as a signature
➢ receiver verifies signature
➢ generally, public key cryptography is used, but a secret key is also
possible

Cryptographic Protocols

➢ Definition: A cryptographic Protocol is an abstract or concrete protocol that

performs security-related functions and applies cryptographic methods often as
the sequence of cryptographic primitives.
➢ A protocol describes how the data structures and algorithms should be used.
➢ Cryptographic protocols are widely used for secure applications-level data
transport. A cryptographic protocol usually incorporates at least some of these
aspects.
★ Key agreement on establishment
★ Entity authentication
★ Symmetric encryption and message authentication material
construction
★ Secured application-level data transport
★ Non-repudation
For example, TLS is a cryptographic protocol that is used to secure web

Authentication
 34

Definition
★ protocol for ensuring the authenticity of the sender

Secret-key protocol [Needham & Schroeder ‘78]

★ based on the secure key server that issues secret keys

★ flaw corrected ’81
★ implemented in Kerberos
Public-key protocol [Needham & Schroeder ‘78]
★ does not require secure key server (7 steps)
★ flaw discovered with CSP/FDR
★ SSL (Secure Sockets Layer) similar to it

Secure Communication

★ Secure communication provides a range of information security solutions to ensure

communications security over public and local network.
★ It includes IPsec,SSL ,PGP(Preety Good Protocols),Virtual Private
Network(VPNs).

Network Auditing

★ Network Auditing is the collective measures done to analyze, study, and gather
data about a network to ascertain its health by network /organization requirements.
★ It works through a systematic process where a network is analyzed for :
➢ Security
➢ Implementation of control
➢ Availability
➢ Management
➢ Performance
★ It uses both manual and automated techniques to gather data and review network
posture. It reviews:
➢ Each of node a network
➢ Network Control and security processes
➢ Network monitoring processes
➢ Other Data

Previous Year Question Solving:

1. Explain the network security model with a proper diagram.

 35

Answer: Securing the processes and the channels used for their interactions and
protecting the objects that they encapsulate against unauthorized access.
★ Protecting Objects: The server manages a collection of objects on behalf of
some users. The users can request the server to perform operations on the
objects. The server carries out the operation specified in each invocation and
sends the result to the client. Objects are intended to be used in different ways by
different users. For example, some objects may hold a user’s private data, such
as their mailbox and other objects may hold shared data such as web pages. To
support this, access rights specify who is allowed to perform the operations of an
object- for example, who is allowed to read or write its state.
★ The Enemy: Processes interact by sending messages. The messages are
exposed to attack because the network and the communication service that they
use are open. Enemy that is capable of sending any message to any process
and reading or copying any message sent between a pair of processes. Such
attacks can be made simply by using a computer connected to a network or a
program that generates messages that make false requests for services.

2. What are threats and attacks? Write down the types of threats.

Answer:
★ Threat: A potential for violation of security, which exists when there is a
circumstance, capability, action, or event that could breach security and cause
harm. That is, a threat is a possible danger that might exploit a vulnerability.

★ Attack: An assault on system security that derives from an intelligent threat; that
is, an intelligent act that is a deliberate attempt (especially in the sense of a
method or technique) to evade security services and violate the security policy of
a system.

Types of Threats
★ Eavesdropping: Obtaining copies of messages without authority
★ Masquerading: Sending/receiving messages using the identity of another
principal without their authority
★ Message tampering: Intercepting and altering messages
★ Replaying: Intercepting, storing, and replaying messages
★ Denial of service: Flooding a channel with requests to deny access to others

3. Write short notes on DES and RSA cryptographic algorithms.

Answer:
DES (The Data Encryption Standard 1977)
➢ US standard for business applications till recently
➢ 64-bit plaintext, 56-bit key
 36

➢ cracked in 1997 (secret challenge message decrypted)

➢ triple-DES (key 112 bits) still secure, poor performance
RSA: The RSA algorithm (

4. What do you mean by firewall? How can we configure firewalls?

5. Define network security. Explain the security model.
6. Define Network Auditing. Write down the strategies for Network security.
7. Briefly explain the security model for distributed computing. How can we defeat
the enemy in a network system?
8. Differentiate between threat and attack.
Answer:

Threat Attack

Threat can be intentional like hun=man The attack is a deliberate action. An

negligence or unintentional like natural attackr have a motive and plan the attack
disasters. accordingly.

A threat may or may not malicious. An attack is always malicious.

Change to demage or information The chance to damage or information

alteration varies from low to very high. alteration is very high.

A threat is difficult to detect. An attack is comparatively easy to detect.

A threat can be prevented by controlling An attack can’t be prevented by merely

the vulnerabilities. controlling the vulnerabilities. Other
measures like bsackup, detect and act etc
are required to handle a cyber attack.

9. How can we configure firewalls? To design a secure system what criteria should
follow?
 37

10. Briefly explain the security model for distributed computing. How can we defeat
the enemy in a network system?
11. What is network auditing? To design a secure system what criteria should follow?
12. What is Digital Signature? How does Digital Signature work?
13. Write short notes on different cryptographic algorithms: RSA, AES
14. Define Network Security. Briefly describe the network security model.

Chapter-04

Replication & Consistency

REPLICATION: Make copies of services on multiple machines.

Why?
★ Reliability
➢ Redundancy
★ Performance
➢ Increase processing capacity
➢ Reduce communication
★ Scalability (prevent centralisation)
➢ Prevent overloading of a single server (size scalability)
➢ Avoid communication latencies (geographic scalability)

REPLICATION ISSUES

★ Updates
➢ Consistency (how to deal with updated data)
➢ Update propagation
★ Replica placement
➢ How many replicas?
➢ Where to put them?
★ Redirection/Routing
➢ Which replica should clients use?

CONSISTENCY
 38

★ Non-distributed data store:

➢ Program order is maintained
➢ Data coherence is respected
★ Updates and concurrency result in conflicting operations
★ Conflicting Operations:
➢ Read-write conflict (only 1 write)
➢ Write-write conflict (multiple concurrent writes)
★ Consistency: The order in which conflicting operations are performed affects
consistency.
➢ partial order: order of a single client’s operations
➢ total order: interleaving of all conflicting operations

Example:
Client A: x = 1; x = 0;
Client B: print(x);
print(x);

Possible results:
- -, 11, 10, 00
How about 01?
What are the conflicting ops? What are the partial orders?
What are the total orders?

CONSISTENCY MODEL
Defines which interleavings of operations are valid(admissible)
Consistency Model:
★ Concerned with the consistency of a data store.
★ Specifies characteristics of valid total orderings
A data store that implements a particular model of consistency will provide a total ordering of
operations that is valid according to the model.

Data Coherence vs Data Consistency

Data Coherence ordering of operations for a single data item.
★ e.g. a read of x will return the most recently written value of x
Data Consistency ordering of operations for whole data store
★ implies data coherence
★ includes ordering of operations on other data items too

DATA-CENTRIC CONSISTENCY MODEL

A contract, between a distributed data store and clients, in which the data store specifies
precisely what the results of read and write operations are in the presence of concurrency.
★ Described consistency is experienced by all clients
★ Multiple clients accessing the same data store
 39

★ Client A, Client B, and Client C see the same kinds of orderings

★ Non-mobile clients (replica used doesn’t change)

STRONG ORDERING VS WEAK ORDERING

Strong Ordering (tight):
★ All writes must be performed in the order that they are invoked
★ Example: all clients must see: W(x)a W(x)b W(x)c
★ Strict (Linearisable) Sequential, Causal, FIFO (PRAM)
Weak Ordering (loose):
★ Ordering of groups of writes, rather than individual writes
★ Series of writes are grouped on a single replica
★ Only results of grouped writes propagated.
★ Example: {W(x)a W(x)b W(x)c} == {W(x)b W(x)a W(x)c}
★ Weak, Release, Entry

Feature Strong Ordering Weak Ordering

Consistency Stronger Weaker

guarantees

Programming Simpler More complex

model

Performance Potentially lower Potentially higher

Examples Sequential consistency, Eventual consistency, weak

strict consistency ordering, release consistency

STRICT CONSISTENCY
Any read on a data item x returns a value corresponding to the result of the most recent write
on x.
Absolute time ordering of all shared accesses

What is most recent in a distributed system?

 40

★ Assumes an absolute global time

★ Assumes instant communication (atomic operation)
★ Normal on a uniprocessor
★ Impossible in a distributed system

SEQUENTIAL CONSISTENCY
All operations are performed in some sequential order
★ More than one correct sequential order
★ All clients see the same order
★ Program order of each client maintained
★ Not ordered according to time

Performance: read time + write time >= minimal packet transfer time

CAUSAL CONSISTENCY
Potentially causally related writes are executed in the same order everywhere.
Causally Related Operations:
★ Read followed by a write (in same client)
★ W(x) followed by R(x) (in same or different clients)

WEAK CONSISTENCY
Shared data can be counted on to be consistent only after a synchronisation is done
Enforces consistency on a group of operations, rather than single operations
★ Synchronization variable (S)
★ Synchronise operation (synchronise(S))
★ Define ‘critical section’ with synchronise operations
Properties:
★ Order of synchronise operations sequentially consistent
★ Synchronise operation cannot be performed until all previous writes have completed
everywhere
★ Read or Write operations cannot be performed until all previous synchronise operations
have completed
Example:
★ synchronise(S) W(x)a W(y)b W(x)c synchronise(S)
★ Writes performed locally
★ Updates propagated only upon synchronisation
★ Only W(y)b and W(x)c have to be propagated

RELEASE CONSISTENCY
Explicit separation of synchronisation tasks
★ acquire(S) - bring local state up to date
 41

★ release(S) - propagate all local updates

★ acquire-release pair defines ’critical region’
Properties:
★ Order of synchronization operations are FIFO consistent
★ Release cannot be performed until all previous reads and writes done by the client have
completed
★ Read or Write operations cannot be performed until all previous acquires done by the
client have completed

Lazy Release Consistency

★ Don’t send updates on release
★ Acquire causes client to get newest state
★ Added efficiency if acquire-release performed by same client(e.g., in a loop)

CAP THEORY

★ C: Consistency: Linearisability
★ A: Availability: Timely response
★ P: Partition-Tolerance: Functions in the face of a partition

CAP CONSEQUENCES
For wide-area systems:
★ must choose: Consistency or Availability
★ choosing Availability: Eventual consistency
★ choosing Consistency: delayed (and potentially failing) operations

CONSISTENCY PROTOCOLS
Consistency Protocol: implementation of a consistency model
Primary-Based Protocols:
★ Remote-write protocols
★ Local-write protocols
Replicated-Write Protocols:
★ Active Replication
★ Quorum-Based Protocols

REMOTE-WRITE PROTOCOLS

Single Server:
★ All writes and reads executed at single server
★ No replication of data
 42

LOCAL-WRITE PROTOCOLS

Migration:
★ Data item migrated to local server on access
★ Distributed, non-replicated, data store

DYNAMIC REPLICATION

Situation changes over time

★ Number of users, Amount of data
★ Flash crowds
★ R/W ratio
Dynamic Replica Placement:
★ Network of replica servers
★ Keep track of data item requests at each replica
★ Deletion threshold
★ Replication threshold
★ Migration threshold
★ Clients always send requests to nearest server

PUSH VS PULL

Pull:
★ Updates propagated only on request
★ Also called client-based
★ R/W low
★ Polling delay

Push:
★ Push updates to replicas
★ Also called server-based
★ When low staleness required
★ R»W
★ Have to keep track of all replicas

Previous Year Questions Solving:

1. Mention two reasons for replication in a distributed system. Give an example
where replication is used.
Answer: Two reasons for distributed system:
1. Higher Availability: To ensure the availability of the distributed system(System
keeps on working even if one or fewer nodes fail).
 43

2. Reduced Latency: Replication assists in reducing the reducing the latency of

data queries(by keeping data geographically closer to a user).

Example of Replication: Data can be copied between two on-premises hosts, between
hosts in different locations, to multiple storage on the same host, or to form a cloud
based host.

2. What is consistency? Write about sequential and causal consistency models.

3. Compare the following: a)remote-write protocols vs local-write protocols b) Push
vs Pull.
Answer: a) remote-write protocols vs local-write protocols

Feature Remote-Write Protocol Local-Write Protocol

Write Operation All writes directed to Writes performed locally at

Flow primary replica any replica

Consistency Stronger (e.g. sequential Weaker (e.g. causal

Guarantees consistency) consistency)

Performance Slower due to Faster as updates are local

communication overhead

Availability Less available when More resilient to individual

primary fails replica failures

Examples Primary-backup protocol, Quorum-based protocols,

Paxos gossip protocols

b) Push vs Pull
 44

Feature Push Pull

Initiator Producer Consumer

Mechanism Sends data Requests data

Example Real-time notifications, streaming Web browsing, REST

services APIs

Server state Stateful Stateless

Scalability More scalable Less scalable

Latency Lower Higher

Resource More efficient for producer More efficient for

usage consumer

4. Write different types of design in replication.

5. What is replication? What are the benefits of replication?
 45

Chapter-05

Distributed Shared Memory(DSM)

DSM: shared memory + multicomputer

Definition: Distributed Shared Memory is the collection of shared memory and
multicomputer.

DSM = SM + MC

DSM is a mechanism for allowing user processes to access shared data without using
interprocess communications. This provides a virtual address space that is shared
among all components.

DSM consists of two components:

★ Shared address space
★ Replication and consistency of memory objects

Shared address space:

★ Shared addresses are valid in all processes

Transparent remote access:

Properties:
★ Remote access is expensive compared to local memory access
★ Individual operations can have very low overhead
★ Threads can distinguish between local and remote access

Why DSM?

★ Shared memory model: easiest to program to

★ Physically shared memory not possible on multicomputer
★ DSM emulates shared memory

Benefits of DSM
★ Ease of programming (shared memory model)
★ Eases porting of existing code
★ Pointer handling
➢ Shared pointers refer to shared memory
➢ Share complex data (lists, etc.)
 46

★ No marshaling

DSM IMPLEMENTATIONs
Typical Implementations
★ Provided by some research OSs(e.g. Mach and Chorus)
★ Most often implemented in user space (e.g., TreadMarks, CVM)
★ User space: what’s needed from the kernel?
➢ User-level fault handler [e.g., Unix signals]
➢ User-level VM page mapping and protection [e.g., mmap() and
mprotect()]
➢ Message passing layer [e.g., socket API]
Hardware
★ Multiprocessor
★ Example: MIT Alewife, DASH

OS with hardware support

★ SCI network cards (SCI = Scalable Coherent Interconnect)
★ SCI maps extended physical address space to remote nodes
★ OS maps shared virtual address space to SCI range

OS and Virtual Memory:

★ Virtual memory (page faults, paging)
★ Local address space vs Large address space
Middleware:
★ Library:
➢ Library routines to create/access shared memory
➢ Example: MPI-2, CRL
★ Language
➢ Shared memory encapsulated in language constructs
➢ Extend language with annotations
➢ Example: Orca, Linda, JavaSpaces, JavaParty, Jackal

DSM Models

1. Shared page (coarse-grained)

★ Traditional model
★ Ideal page size
★ False sharing
★ Examples: Ivy, TreadMarks
 47

2. Shared region (fine-grained)

★ More fine-grained than sharing pages
★ Prevent false sharing
★ Not regular memory access (transparency)
★ Examples: CRL (C Region Library), MPI-2 one-sided communication,
Shasta

3. Shared Variable
★ Release and Entry-based consistency
★ Annotations
★ Fine-grained
★ More complex for programmer
★ Examples: Munin, Midway

4. Shared structure
★ Encapsulate shared data
★ Access only through predefined procedures (e.g., methods)
★ Tightly integrated synchronization
★ Encapsulate (hide) consistency model
★ Lose familiar shared memory model
★ Examples: Orca (shared object), Linda (tuple space)

APPLICATIONS OF DSM

★ Scientific parallel computing

➢ Bioinformatics (gene sequence analysis)
➢ Simulations (climate modeling, economic modeling)
➢ Data processing (physics, astronomy)
★ Graphics (image processing, rendering)
★ Data server (distributed FS, Web server)
★ Data storage

DSM Environments

★ Multiprocessor
➢ NUMA
★ Multicomputer
➢ Supercomputer
 48

➢ Cluster
➢ Network of Workstations
➢ Wide-area

Requirements of DSM

Transparency
★ Location, migration, replication, concurrency

Reliability
★ Computations depend on availability of data

Performance
★ Important in high-performance computing
★ Important for transparency

Scalability
★ Important in wide-area
★ Important for large computations
★ Access to DSM should be consistent
★ According to a consistency model

Programmability
★ Easy to program
★ Communication transparency

Design Issues

★ Granularity
➢ Page-based, Page size: minimum system page size
★ Replication
➢ Lazy release consistency
★ Scalability
➢ Meant for cluster or NOW (Network of Workstations)
★ Synchronisation primitives
➢ Locks (acquire and release), Barrier
★ Heterogeneity
➢ Limited (doesn’t address endianness or mismatched word sizes)
★ Fault Tolerance
➢ Research
 49

★ No Security

DSM: A distributed shared memory (DSM) is an implementation of the shared memory

abstraction on a multicomputer architecture which has no physically shared memory.
There are several forms of memory coherence:

★ Strict Consistency: A read returns the most recently written value. This form of
consistency is what most programmers intuitively expect. However, it implies a
total ordering on all memory operations in the system so the most recent write
can be determined. This forced total ordering leads to inefficiency.

★ Sequential Consistency: A system is sequentially consistent if the result of any

execution is the same as if the operations of all the nodes were executed in
some sequential order, and the operations of each node appear in this sequence
in the order specified by its program. A sequentially consistent multiprocessor is
indistinguishable from a multi-threaded uniprocessor.

★ Concurrent Consistency: A system is concurrently consistent if it provides

sequential consistency except for programs that explicitly test for sequential
consistency or take access timings into account. Note that it is wasteful for a
program that expects sequential consistency to explicitly test for its existence.

★ Processor Consistency: Writes issued by a processor are never seen out of

order. This allows updates to propagate through the system in a pipelined
fashion, although two simultaneous reads of the same location from different
processors may yield different results.

★ Weak Consistency: Synchronization accesses are sequentially consistent, and

before a synchronization access can be performed all previous regular accesses
must be completed, and before a regular access can be performed all previous
synchronization accesses must be done. This leaves consistency up to the
programmer. The memory will only be consistent immediately after a
synchronization operation.

★ Release Consistency: weak consistency, but synchronization accesses must

only be processor consistent concerning each other. Synchronization operations
are broken down into acquire and release. All pending acquires must be done
before you can do a regular access, and all regular accesses must be done
before doing a release. This is just a further relaxation of weak consistency
without very much loss of coherence.
 50

Previous Year Question Solving:

1. Draw and explain the Distributed Shared Memory model preciously.

2. What are the requirements of DSM?
Answer:
Transparency
★ Location, migration, replication, concurrency

Reliability
★ Computations depend on the availability of data

Performance
★ Important in high-performance computing
★ Important for transparency

Scalability
★ Important in wide-area
★ important for large computations
★ Access to DSM should be consistent
★ According to a consistency model

Programmability
★ Easy to program
★ Communication transparency

3. Write the application areas of DSM. What are the benefits of DSM in terms of
shared memory?
Answer:
APPLICATIONS OF DSM

★ Scientific parallel computing

➢ Bioinformatics (gene sequence analysis)
➢ Simulations (climate modeling, economic modeling)
➢ Data processing (physics, astronomy)
★ Graphics (image processing, rendering)
★ Data server (distributed FS, Web server)
★ Data storage
Benefits of DSM:
★ Ease of programming (shared memory model)
★ Eases porting of existing code
 51

★ Pointer handling
➢ Shared pointers refer to shared memory
➢ Share complex data (lists, etc.)
★ No marshaling

4. What are the components of distributed shared memory?

Answer: DSM consists of two components:

★ Shared address space

★ Replication and consistency of memory objects
5. Why do we use DSM? Write the application areas of DSM.
Answer: Using DSM
★ Shared memory model: easiest to program to
★ Physically shared memory not possible on multicomputer
★ DSM emulates shared memory

APPLICATIONS OF DSM

★ Scientific parallel computing

➢ Bioinformatics (gene sequence analysis)
➢ Simulations (climate modeling, economic modeling)
➢ Data processing (physics, astronomy)
★ Graphics (image processing, rendering)
★ Data server (distributed FS, Web server)
★ Data storage

6. What is Distributed Shared Memory? What are the main requirements of this?

Answer: DSM: A distributed shared memory (DSM) is an implementation of the

shared memory abstraction on a multicomputer architecture that has no physically
shared memory.
Requirements of DSM

Transparency
★ Location, migration, replication, concurrency

Reliability
★ Computations depend on availability of data

Performance
 52

★ Important in high-performance computing

★ Important for transparency

Scalability
★ Important in wide-area
★ mportant for large computations
★ Access to DSM should be consistent
★ According to a consistency model

Programmability
★ Easy to program
★ Communication transparency

Chapter-06

Fault Tolerance

Dependability

★ Availability: The System is ready to be used immediately.

★ Reliability: The system can run continuously without failure.
★ Safety: When a system(temporarily) fails to operate correctly, nothing catastrophic
happens.
★ Maintainability: How easily a failed system can be repaired.

Building a dependable system comes down to controlling failure and faults.

Failure

Terminology:
 53

Failure: a system fails when it fails to meet its promises or cannot provide its services in the
specified manner
Error: part of the system state that leads to failure (i.e., it differs from its intended value)
Fault: the cause of an error (results from design errors, manufacturing faults, deterioration, or
external disturbance)
Recursive:
★ Failure can be a fault
★ Manufacturing fault leads to disk failure
★ Disk failure is a fault that leads to database failure
★ Database failure is a fault that leads to email service failure

TOTAL VS PARTIAL FAILURE***

Total Failure: All components in a system fail.

★ Typical in nondistributed system
Partial Failure: One or more (but not all) components in a distributed system fail.
★ Some components affected.
★ Other components are completely unaffected.
★ Considered as fault for the whole system.

CATEGORISING FAULTS AND FAILURES

Types of Faults:
★ Transient Fault: occurs once and then disappears
★ Intermittent Fault: occurs, vanishes, reoccurs, vanishes, etc.
★ Permanent Fault: persists until the faulty component is replaced.

Types of Failures:
★ Process Failure: process proceeds incorrectly or not at all.
★ Storage Failure: “stable” secondary storage is inaccessible.
★ Communication Failure: communication link or node failure.

FAILURE MODELS

Crash Failure: a server halts, but works correctly until it halts

★ Fail-Stop: The server will stop in a way that clients can tell that it has halted.
★ Fail-Resume: The server will stop, and then resume execution at a later time.
★ Fail-Silent: Clients do not know the server has halted
 54

Omission Failure: a server fails to respond to incoming requests

★ Receive Omission: fails to receive incoming messages
★ Send Omission: fails to send messages

Timing Failure: a server’s response lies outside the specified time interval.

Response Failure: a server’s response is incorrect

★ Value Failure: the value of the response is wrong
★ State Transition Failure: the server deviates from the correct flow of control

Arbitrary Failure: a server may produce an arbitrary response at arbitrary times (aka Byzantine
failure).

FAULT TOLERANCE

Fault Tolerance: The system can provide its services even in the presence of faults.

Goal:
★ Automatically recover from partial failure
★ Without seriously affecting overall performance

Techniques:
★ Prevention: prevent or reduce the occurrence of faults
★ Prediction: predict the faults that can occur and deal with them
★ Masking: hide the occurrence of the fault
★ Recovery: restore an erroneous state to an error-free state

FAILURE PREVENTION

Make sure faults don’t happen:

★ Quality hardware
★ Hardened hardware
★ Quality software

FAILURE PREDICTION

Deal with expected faults:

★ Test for error conditions
★ Error handling code
★ Error-correcting codes
1. checksums
2. erasure codes
 55

DETECTING FAILURE

Failure Detector:
★ Service that detects process failures
★ Answers queries about status of a process
Reliable:
★ Failed – crashed
★ Unsuspected – hint
Unreliable:
★ Suspected – may still be alive
★ Unsuspected – hint

Synchronous and Asynchronous System

Synchronous systems:
★ Timeout
★ Failure detector sends probes to detect crash failures
Asynchronous systems:
★ Timeout gives no guarantees
★ Failure detector can track suspected failures
★ Combine results from multiple detectors
★ How to distinguish communication failure from process failure?
★ Ignore messages from suspected processes
★ Turn an asynchronous system into a synchronous one

Feature Synchronous System Asynchronous System

Communicati Blocking, wait for response Non-blocking, send and

on continue

Timing Shared clock, global state Independent, no shared clock

awareness
 56

Complexity Easier to design and debug More complex to design and

debug

Performance Slower due to blocking calls Faster due to non-blocking

calls

Examples RPC, database transactions Messaging queues, web

applications

FAILURE MASKING

Try to hide the occurrence of failures from other processes

Redundancy:
★ Information redundancy
★ Time redundancy
★ Physical redundancy

RELIABLE COMMUNICATION

★ Communication channel experiences failure

★ Focus on masking crash (lost/broken connections) and omission (lost messages) failures

Two Army Problem//

Non-faulty processes but lossy communication.

★ 1 → 2 attack!
★ 2 → 1 ack
★ 2: did 1 get my ack?
★ 1 → 2 ack ack
★ 1: did 2 get my ack ack?
 57

★ etc.

Consensus with lossy communication is impossible.

Why does TCP work?

RELIABLE POINT-TO-POINT COMMUNICATION

★ Reliable transport protocol (e.g., TCP)

★ Masks omission failure
★ Not crash failure

Example: Failure and RPC:

Possible failures:
★ Client cannot locate server
★ Request message to server is lost
★ Server crashes after receiving a request
★ Reply message from the server is lost
★ Client crashes after sending a request
How to deal with the various kinds of failure?

REPLICATION

Create groups using replication

Primary-Based:
★ Primary-backup
★ Hierarchical group
★ If primary crashes others elect a new primary
Replicated-Write:
★ Active replication or Quorum
★ Flat group
★ Ordering of requests (atomic multicast problem)
k Fault Tolerance:
★ can survive faults in k components and still meet its specifications
★ k + 1 replicas enough if fail-silent (or fail-stop)
★ 2k + 1 required if it is byzantine

Byzantine Generals Problem

Reliable communication but faulty processes.

 58

★ n generals (processes)
★ m are traitors (will send incorrect and contradictory info)
★ Need to know everyone else’s troop strength gi
★ Each process has a vector: hg1, ...gni
★ (Note: this is interactive consistency)

Byzantine Generals Impossibility:

Faulty process
➜ If m faulty processes then 2m + 1 nonfaulty processes required for correct functioning

Byzantine agreement with Signatures:

★ Digitally sign messages
★ Cannot lie about what someone else said
★ Avoids the impossibility result
★ Can have an agreement with 3 processes and 1 faulty

FAILURE RECOVERY

Restoring an erroneous state to an error-free state

Issues:
★ Reclamation of resources: Locks, buffers held on other nodes.
★ Consistency: Undo partially completed operations before restart.
★ Efficiency: Avoid restarting the whole system from the start of computation.

FORWARD VS. BACKWARD ERROR RECOVERY

Forward Recovery:
★ Correct erroneous state without moving back to a previous state.
★ Example: erasure correction - missing packet reconstructed from successfully delivered
packets.
★ Possible errors must be known in advance
Backward Recovery:
★ Correct erroneous state by moving to a previously correct state
★ Example: packet retransmission when the packet is lost
★ General purpose technique.
★ High overhead
★ Error can reoccur
★ Sometimes impossible to roll back (e.g. ATM has already delivered the money)

BACKWARD RECOVERY

General Approach:
 59

★ Restore process to recovery point

★ Restore system by restoring all active processes
Specific Approaches:
★ Operation-based recovery:
➢ Keep log (or audit trail) of operations
➢ Restore to recovery point by reversing changes
★ State-based recovery:
➢ Store complete state at recovery point(checkpointing)
➢ Restore process state from checkpoint (rolling back)
Log or checkpoint recorded on stable storage

Operation Based Recovery - Logging: Update in-place together with write-ahead logging
★ Every change (update) of data is recorded in a log, which includes:
➢ Data item name (for identification)
➢ Old data item state (for undo)
➢ New data item state (for redo)
★ Undo log is written before update (write-ahead log).
★ Transaction semantics

State-Based Recovery - Checkpointing: Take frequent checkpoints during execution

Checkpointing:
★ Pessimistic vs Optimistic
➢ Pessimistic: assumes failure, optimised toward recovery
➢ Optimistic: assumes infrequent failure, minimises checkpointing overhead
★ Independent vs Coordinated
➢ Coordinated: processes synchronise to create global checkpoint
➢ Independent: each process takes local checkpoints independently of others
★ Synchronous vs Asynchronous
➢ Synchronous: distributed computation blocked while checkpoint taken
➢ Asynchronous: distributed computation continues while checkpoint taken

Checkpointing Overhead:
★ Frequent checkpointing increases overhead
★ Infrequent checkpointing increases recovery cost
Decreasing Checkpointing Overhead:
Incremental checkpointing: Only write changes since last checkpoint:
★ Write-protect whole address space
★ On write-fault mark page as dirty and unprotect
★ On checkpoint only write dirty pages
Asynchronous checkpointing: Use copy-on-write to checkpoint while execution continues
★ Easy with UNIX fork()
Compress checkpoints: Reduces storage and I/O cost at the expense of CPU time
 60

RECOVERY IN DISTRIBUTED SYSTEMS

★ Failed process may have causally affected other processes
★ Upon recovery of failed process, must undo effects on other processes
★ Must roll back all affected processes
★ All processes must establish recovery points
★ Must roll back to a consistent global state

Domino Effect://**

★ P2 fails → P2 y R22
Orphan message m is received but not sent → P1 y R12
★ P3 fails → P3 y R32 → P2 y R21 → P1 y R11, P3 y R31

Messaging dependencies plus independent checkpointing may force system to roll back to initial
state

Message Loss:

★ Failure of P2 → P2 y R21
★ Message m is now recorded as sent (by P1) but not received (by P2), and m will never
be received after rollback
★ Message m is lost
★ Whether m is lost due to rollback or due to imperfect communication channels is
indistinguishable!
★ Require protocols resilient to message loss

Livelock:

★ Pre-rollback message n1 is received after rollback

★ Forces another rollback P2 y R21, P1 y R11, can repeat indefinitely

Consistent Checkpointing

Consistent Cut

Idea: collect local checkpoints in a coordinated way.

★ A set of local checkpoints forms a global checkpoint.
★ A global checkpoint represents a consistent system state.
 61

★ {R11, R21, R31} form a strongly consistent checkpoint:

➢ No information flow during checkpoint interval
★ {R12, R22, R32} form a consistent checkpoint:
➢ All messages recorded as received must be recorded as sent

★ Strongly consistent checkpointing requires quiescent system

➢ Potentially long delays during blocking checkpointing
★ Consistent checkpointing requires dealing with message loss
➢ Not a bad idea anyway, as otherwise, each lost message would result in a global
rollback
➢ Note that a consistent checkpoint may not represent an actual past system state
How to take a consistent checkpoint?
★ Simple solution: Each process checkpoints immediately after sending a message
★ High overhead
★ Reducing this to checkpointing after n messages, n > 1, is not guaranteed to produce a
consistent checkpoint!
★ Require some coordination during checkpointing

ROLLBACK RECOVERY

First Phase:
★ Coordinator sends “r” messages to all other processes to ask them to roll back
★ Each process replies true, unless already in checkpoint or rollback
★ If all replies are true, coordinator decides to roll back, otherwise continue
Second Phase:
★ Coordinator sends decision to other processes
★ Processes receiving this message perform corresponding action

ASYNCHRONOUS CHECKPOINTING

Let processes checkpoint independently(unsynchronised) and construct a consistent state

during recovery.
★ The source of inconsistencies is orphan messages.
★ Consistent state can be obtained by:
1. Restarting the failed process from the latest checkpoint, and
2. rolling forward the restarted process past the point where the last
message was sent before failure.
★ All send operations during roll-forward are suppressed.
★ Except for timing, the result is indistinguishable from restarting from a (non-existent)
checkpoint taken after the last send.
★ Works as long as no message is lost.
 62

Previous Year Question Solving:

1. Define Fault Tolerance. Explain the techniques for fault tolerance.
Answer: Fault Tolerance: The system can provide its services even in the presence of
faults.

The techniques for fault tolerance are -

★ Prevention: prevent or reduce the occurrence of faults.
★ Prediction: predict the faults that can occur and deal with them.
★ Masking: hide the occurrence of the fault.
★ Recovery: restore an erroneous state to an error-free state.

2. What are total and partial failures? Briefly discuss about fault tolerance
techniques.

Answer: Total Failure: Total Failure means a fault which causes continuous and
complete loss of Service.

Partial Failure: A partial failure is less serious than a complete failure, and typically
causes a degradation of service, but not a complete loss of service.

The techniques for fault tolerance are -

★ Prevention: prevent or reduce the occurrence of faults.
★ Prediction: predict the faults that can occur and deal with them.
★ Masking: hide the occurrence of the fault.
★ Recovery: restore an erroneous state to an error-free state.

3. Explain failure models of distributed computing.

Answer:

Crash Failure: a server halts, but works correctly until it halts

★ Fail-Stop: The server will stop in a way that clients can tell that it has halted.
★ Fail-Resume: The server will stop, and then resume execution at a later time.
★ Fail-Silent: Clients do not know the server has halted

Omission Failure: a server fails to respond to incoming requests

★ Receive Omission: fails to receive incoming messages
 63

★ Send Omission: fails to send messages

Timing Failure: a server’s response lies outside the specified time interval.

Response Failure: a server’s response is incorrect

★ Value Failure: the value of the response is wrong
★ State Transition Failure: the server deviates from the correct flow of control

Arbitrary Failure: a server may produce arbitrary response at arbitrary times (aka
Byzantine failure).

4. Write the issues to restore an erroneous state to an error free state in failure
recovery. Discuss about different phases of rollback recovery.
Answer:
Restoring an erroneous state to an error-free state
Issues:
★ Reclamation of resources: Locks, buffers held on other nodes.
★ Consistency: Undo partially completed operations before restart.
★ Efficiency: Avoid restarting the whole system from the start of computation.
Different phases of rollback recovery:
First Phase:
★ Coordinator sends “r” messages to all other processes to ask them to roll back
★ Each process replies true, unless already in checkpoint or rollback
★ If all replies are true, coordinator decides to roll back, otherwise continue
Second Phase:
★ Coordinator sends decision to other processes
★ Processes receiving this message perform corresponding action

5. What are failures and fault? Illustrate different types of failures.

Answer:

Failure: a system fails when it fails to meet its promises or cannot provide its services in
the specified manner.
Fault: the cause of an error (results from design errors, manufacturing faults,
deterioration, or external disturbance).

Types of Failures:
★ Process Failure: process proceeds incorrectly or not at all.
★ Storage Failure: “stable” secondary storage is inaccessible.
★ Communication Failure: communication link or node failure.

6. Illustrate two army problem algorithms for reliable communication.

 64

Chapter-07

Synchronization and Coordination

DISTRIBUTED ALGORITHMS
Algorithms that are intended to work in a distributed environment.
are Used to accomplish tasks such as:
★ Communication
★ Accessing resources
★ Allocating resources
★ Consensus etc.
Synchronization and coordination inextricably linked to distributed algorithms
★ Achieved using distributed algorithms
★ Required by distributed algorithms

SYNCHRONOUS VS ASYNCHRONOUS DISTRIBUTED SYSTEMS

Timing model of a distributed system.

Affected by:
★ Execution speed/time of processes
★ Communication delay
★ Clocks & clock drift
★ (Partial) failure

Synchronous Distributed System: Time variance is bounded

Execution: bounded execution speed and time

Communication: bounded transmission delay
Clocks: bounded clock drift (and differences in clocks)
Effect:
★ Can rely on timeouts to detect failure
★ Easier to design distributed algorithms
★ Very restrictive requirements
➢ Limit concurrent processes per processor
➢ Limit concurrent use of network
 65

➢ Require precise clocks and synchronization

Asynchronous Distributed System: Time variance is not bounded

Execution: different steps can have varying duration

Communication: transmission delays vary widely
Clocks: arbitrary clock drift
Effect:
★ Allows no assumption about time intervals
★ Cannot rely on timeouts to detect failure
★ Most asynch DS problems hard to solve
★ Solution for asynch DS is also a solution for synch DS
★ Most real distributed systems are hybrid synch and asynch

EVALUATING DISTRIBUTED ALGORITHMS

General Properties:
★ Performance
➢ number of messages exchanged
➢ response/wait time
➢ delay
➢ throughput: 1/(delay + executiontime)
➢ complexity: O()
★ Efficiency
➢ resource usage: memory, CPU, etc.
★ Scalability
★ Reliability
➢ number of points of failure (low is good)

SYNCHRONIZATION AND COORDINATION

Important: Doing the right thing at the right time.
Two fundamental issues:
★ Coordination (the right thing)
★ Synchronization (the right time)

SYNCHRONIZATION: Ordering of all actions

★ Total ordering of events
★ Total ordering of instructions
★ Total ordering of communication
★ Ordering of access to resources
★ Requires some concept of time

COORDINATION: Coordinate actions and agree on values.

Coordinate Actions:
 66

★ What actions will occur

★ Who will perform actions
Agree on Values:
★ Agree on global value
★ Agree on environment
★ Agree on state

MAIN ISSUES

● Time and Clocks: synchronizing clocks and using time in distributed algorithms
● Global State: how to acquire knowledge of the system’s global state
● Concurrency Control: coordinating concurrent access to resources

TIME

Global Time:
★ ’Absolute’ time
➢ Einstein says no absolute time
➢ Absolute enough for our purposes
★ Astronomical time
➢ Based on earth’s rotation
➢ Not stable
★ International Atomic Time (IAT)
➢ Based on oscillations of Cesium-133
★ Coordinated Universal Time (UTC)
➢ Leap seconds
➢ Signals broadcast over the world

Local Time:
★ Not synchronised to Global source
★ Relative not ’absolute’

USING CLOCKS IN COMPUTERS

Computer Clocks:
★ Crystal oscillates at known frequency
★ Oscillations cause timer interrupts
★ Timer interrupts update clock
Clock Skew:
★ Crystals in different computers run at slightly different rates
★ Clocks get out of sync
★ Skew: instantaneous difference
★ Drift: rate of change of skew
 67

Timestamps:
★ Used to denote at which time an event occurred

PHYSICAL CLOCKS: Synchronization Using Physical Clocks:

Examples:
★ Performing events at an exact time (turn lights on/off, lock/unlock gates)
★ Logging of events (for security, for profiling, for debugging)
★ Tracking (tracking a moving object with separate cameras)
★ Make (edit on one computer build on another)

Based on actual time

★ Cp(t): current time (at UTC time t) on machine p
★ Ideally Cp(t) = t
★ Clock differences causes clocks to drift
★ Must regularly synchronize with UTC

SYNCHRONISING PHYSICAL CLOCKS

1. Internal Synchronisation:
★ Clocks synchronize locally
★ Only synchronized with each other
2. External Synchronisation:
★ Clocks synchronize to an external time source
★ Synchronise with UTC every δ seconds
3. Time Server:
★ Server that has the correct time
★ Server that calculates the correct time

BERKELEY ALGORITHM

The Berkeley algorithm assumes no machine has an accurate time source, and obtains
the average participating computers all clocks to average. Machine run time daemon
process that implements the protocol. One machine is elected(or designated) as the
server(master) the others are slaves. Master polls each machine periodically asks each
machine for time and can use Christian's algorithm to compensate for network latency.
When results are in computing average including master's time, average cancels out
individual clock tendencies to run fast or slow. It sends offset by which each clock needs
adjustment to each slave, avoiding problems with network delays if we want a time
stamp. The algorithm has pŕovisions for ignoring readings from clocks whose skew is too
great to compute a fault-tolerant average if the master fails any slave can take over.
 68

Example:

CRISTIAN’S ALGORITHM

Time Server:
★ Has UTC receiver
★ Passive
Algorithm:
★ Clients periodically request the time
★ Don’t set time backward
★ Take propagation and interrupt handling delay into account
➢ (T1 − T0)/2
➢ Or take a series of measurements and average the delay
★ Accuracy: 1-10 millisec (RTT in LAN)

NETWORK TIME PROTOCOL (NTP)

Hierarchy of Servers:
★ Primary Server: has UTC clock
★ Secondary Server: connected to primary etc.
Synchronization Modes:
★ Multicast: for LAN, low accuracy
★ Procedure Call: clients poll, reasonable accuracy
★ Symmetric: Between peer servers. highest accuracy

Synchronization
★ Estimate clock offsets and transmission delays between two nodes
★ Keep estimates for past communication
 69

★ Choose offset estimate for lowest transmission delay

★ Also determine unreliable servers
★ Accuracy 1 - 50 msec

LOGICAL CLOCKS
Event ordering is more important than physical time.
★ Events (e.g., state changes) in a single process are ordered
★ Processes need to agree on ordering of causally related events (e.g., message send
and receive)
Local ordering:
★ System consists of N processes pi, i ∈ {1, . . . , N}
★ Local event ordering →i:
If pi observes e before e′, we have e →i e′
Global ordering:
★ Leslie Lamport’s happened before relation →
★ Smallest relation, such that
➢ e →i e′ implies e → e′
➢ For every message m, send(m) → receive(m)
➢ Transitivity: e → e′ and e′ → e′′ implies e → e′′

The relation → is a partial order:

➜ If a → b, then a causally affects b
➜ We consider unordered events to be concurrent:
a 6→ b and b 6→ a implies a k b

Example:

★ Causally related: E11 → E12, E13, E14, E23, E24, . . .

E21 → E22, E23, E24, E13, E14, . . .
★ Concurrent: E11||21, E12||22, E13||23, E11||22, E13||24, E14||23, . . .

Lamport’s logical clocks:

★ Software counter to locally compute the happened-before relation →
★ Each process pi maintains a logical clock Li
★ Lamport timestamp:
➢ Li(e): timestamp of event e at pi
➢ L(e): timestamp of event e at process it occurred at
Implementation:
★ Before timestamping a local event pi executes Li := Li + 1
★ Whenever a message m is sent from pi to pj :
➢ pi executes Li := Li + 1 and sends Li with m
 70

➢ pj receives Li with m and executes Lj := max(Lj , Li) + 1 (receive(m) is annotated

with the new Lj )

Properties:
★ a → b implies L(a) < L(b)
★ L(a) < L(b) does not necessarily imply a → b

Example:

Total event ordering:

★ Complete partial to total order by including process identifiers
★ Given local time stamps Li(e) and Lj(e′), we define global time stamps hLi(e), ii and
hLj(e′), ji
★ Lexicographical ordering: hLi(e), ii < hLj(e′), ji iff
➢ Li(e) < Lj (e′) or
➢ Li(e) = Lj (e′) and i < j

CONSISTENT CUTS
Determining global properties:
★ We need to combine information from multiple nodes
★ Without global time, how do we know whether collected local information is consistent?
★ Local state sampled at arbitrary points in time surely is not consistent
★ We need a criterion for what constitutes a globally consistent collection of local
information

Cuts:
Picture need
Picture need
Consistent cut:
★ We call a cut consistent iff,
for all events e′ ∈ C, e → e′ implies e ∈ C
★ A global state is consistent if it corresponds to a consistent cut
★ Note: we can characterize the execution of a system as a sequence of consistent global
states
S0 → S1 → S2 → · · ·
 71

Chapter-08

Naming & Cloud Computing

WHAT IS NAMING?
Systems manage a wide collection of entities of different kinds. They are identified by different
kinds of names:
★ Files (/boot/vmlinuz), Processes (1, 14293), Users (chak, ikuz, cs9243), Hosts
(weill, facebook.com), . . .
//Examples of naming in distributed systems? What’s the difficulty?

BASIC CONCEPTS

Name:
★ String of bits or characters
★ Refers to an entity
Entity:
★ Resource, process, user, etc.
★ Operations performed on entities at access points
Address:
★ Access point named by an address
★ Entity address = address of entity’s access point
★ Multiple access points per entity
★ Entity’s access points may change

★ Name that uniquely identifies entity

★ Properties:
➢ Refers to at most one entity
➢ Entity referred to by at most one identifier
➢ Always refers to the same entity (i.e. no reuse)
★ Allows easy comparison of references

SYSTEM-ORIENTED VS HUMAN-ORIENTED NAMES

System-Oriented Names:
★ Represented in machine readable form (32 or 64-bit strings)
★ Structured or unstructured
★ Easy to store, manipulate, compare
 72

★ Not easy to remember, hard for humans to use

★ Example: inode (0x00245dad)

Human-Oriented Names:
★ Variable length character strings
★ Usually structured
★ Often many human-oriented names map onto a single system-oriented name
★ Easy to remember and distinguish between
★ Hard for machine to process
★ Example: URL

NAME SPACES: Container for a set of related names

Structure options:
★ Flat (only leaf nodes)
★ Hierarchical (Strictly hierarchical, DAG, Multiple root nodes)
★ Tag-based
Path Names (in hierarchies):
★ Sequence of edge labels
★ Absolute: if first node in path name is a root node
★ Relative: otherwise
Aliasing:
★ Alias: another name for an entity
★ Hard link: two or more paths to an entity in the graph
★ Soft link: leaf node stores a (absolute) path name to another node

Mounting
★ Directory node stores info about a directory node in another namespace
★ Need: protocol, server, path name, authentication, and authorization info, keys for
secure communication, etc.

Combining name spaces

★ Name Spaces: Protocol, DNS, File System

NAMING SERVICES: A naming service provides a name space

Name Server:
★ Naming service implemented by name servers
★ Implements naming service operations
Operations:
★ Lookup: resolve a path name, or element of a path name
 73

★ Add: add a directory or leaf node

★ Remove: remove a subtree or leaf node
★ Modify: modify the contents of a directory or leaf node
Client:
★ Invokes naming service operations

PARTITIONING: Split name space over multiple servers

Structured Partitioning:
★ split name space according to graph structure
★ Name resolution can use zone hints to quickly find appropriate server
★ Improved lookup performance due to knowledge of structure
★ Rigid structure

Structure-free Partitioning:
★ content placed on servers independent of name space
★ Flexible
★ Decreased lookup performance, increased load on root

DNS (DOMAIN NAME SYSTEM)

Structure:
★ Hierarchical structure (tree)
★ Top-level domains (TLD) (.com, .org, .net, .au, .nl, ...)
★ Zone: a (group of) directory node
★ Resource records: contents of a node
★ Domain: a subtree of the global tree
★ Domain name: an absolute path name

Table hey

★ Each zone implemented by a name server

Replication:
★ Each zone replicated on at least two servers
★ Updates performed on primary
★ Contents transferred to secondary using zone transfer
★ Higher levels have many more replicas (13 root servers: A-M.root-servers.net. Actually
386 replicas using anycast)
Caching:
★ Servers cache results of queries
★ Original entries have time-to-live field (TTL)
★ Cached data is non-authoritative, provided until TTL expires
Name Resolution:
 74

★ Query sent to local server

★ If cannot resolve locally then sent to root
★ Resolved recursively or iteratively

Cloud Computing

★ What is Cloud Computing?

★ X as a Service
★ Key Challenges
★ Developing for the Cloud

WHAT IS CLOUD COMPUTING?

A style of computing in which dynamically scalable and often virtualized resources are provided
as a service over the Internet.

Why is it called Cloud?

★ services provided on virtualized resources
★ virtual machines spawned on demand
★ location of services no longer certain
★ similar to network cloud

KEY CHARACTERISTICS OF CLOUD COMPUTING

SP 800-145. The NIST Definition of Cloud Computing:

★ On-demand, self-service
➢ get resources (CPU, storage, bandwidth, etc),
➢ automated: as needed, right now!
★ Network access
➢ Services accessible over the network, standard protocols
★ Pooled resources
➢ provider: multi-tenant pool of resources
➢ dynamically assigned and reassigned per customer demand
★ Elasticity
➢ Scalability: rapidly adjust resource usage as needed
★ Measured service
➢ monitor resource usage
➢ billing for resources used
 75

BENEFITS

Flexibility:
★ Flexible provisioning
★ Add machines on demand
★ Add storage on demand

Effort:
★ Low barrier to entry
★ Initial effort: no need to spec and set up physical infrastructure
★ Continuing effort: no need to maintain physical infrastructure

Speed: Most cloud computing services are provided self-service and on demand, so even vast
amounts of computing resources can be provisioned in minutes, typically with just a few mouse
clicks, giving businesses a lot of flexibility and taking the pressure off capacity planning.

Cost:
★ Cloud computing eliminates the capital expense of buying hardware and software
★ setting up and running on-site data centers—the racks of servers, the round-the-clock
electricity for power and cooling, and the IT experts for managing the infrastructure. It
adds up fast.
★ Low initial capital expenditure
★ Avoid costs of over-provisioning for scalability
★ Pay for what you use

Global scale: The benefits of cloud computing services include the ability to scale elastically. In
cloud speak, that means delivering the right amount of IT resources—for example,
more or less computing power, storage, and bandwidth—right when they’re needed, and
from the right geographic location.

Security
★ Many cloud providers offer a broad set of policies, technologies, and controls that
strengthen your security posture overall, helping protect your data, apps, and
infrastructure from potential threats.
★ Redundancy
★ Trust reliability of provider
★ Data backups

➢ Performance: The biggest cloud computing services run on a worldwide network of

secure data centers, which are regularly upgraded to the latest generation of fast and
efficient computing hardware. This offers several benefits over a single corporate data
center, including reduced network latency for applications and greater economies of
scale.
 76

➢ Reliability: Cloud computing makes data backup, disaster recovery, and business
continuity easier and less expensive because data can be mirrored at multiple redundant
sites on the cloud provider’s network.
➢ Productivity: On-site data centers typically require a lot of “racking and
stacking”—hardware setup, software patching, and other time-consuming IT
management chores. Cloud computing removes the need for many of these tasks, so IT
teams can spend time on achieving more important business goals.
➢ Speed: Most cloud computing services are provided self-service and on-demand, so
even vast amounts of computing resources can be provisioned in minutes, typically with
just a few mouse clicks, giving businesses a lot of flexibility and taking the pressure off
capacity planning.

Types of cloud computing

★ Public: open services available to everyone

★ Private: owned, operated, and available to specific organizations.
★ Hybrid: the system uses some private cloud services and some public cloud services.

Types of cloud services

Most cloud computing services fall into four broad categories: infrastructure as a service
(IaaS), platform as a service (PaaS), serverless, and software as a service (SaaS). These
are sometimes called the cloud computing "stack" because they build on top of one another.
Knowing what they are and how they’re different makes it easier to accomplish your business
goals.

★ Infrastructure as a service (IaaS): The most basic category of cloud computing

services. With IaaS, you rent IT infrastructure—servers and virtual machines (VMs),
storage, networks, and operating systems—from a cloud provider on a pay-as-you-go
basis.
★ Platform as a service (PaaS): Platform as a service refers to cloud computing services
that supply an on-demand environment for developing, testing, delivering, and managing
software applications. PaaS is designed to make it easier for developers to quickly
create web or mobile apps, without worrying about setting up or managing the
underlying infrastructure of servers, storage, network, and databases needed for
development.
★ Serverless computing: Overlapping with PaaS, serverless computing focuses on
building app functionality without spending time continually managing the servers and
infrastructure required to do so. The cloud provider handles the setup, capacity planning,
and server management for you. Serverless architectures are highly scalable and
event-driven, only using resources when a specific function or trigger occurs.
★ Software as a service (SaaS): Software as a service is a method for delivering software
applications over the Internet, on-demand, and typically on a subscription basis. With
 77

SaaS, cloud providers host and manage the software application and underlying
infrastructure, and handle any maintenance, like software upgrades and security
patching. Users connect to the application over the Internet, usually with a web browser
on their phone, tablet, or PC.

INFRASTRUCTURE AS A SERVICE: IAAS

Service provider provides

★ Server and network hardware
★ Virtual machines
★ IP addresses
★ Services to manage VMs (create, start, stop, migrate)
★ Optional: storage, database, synchronization, communication
Client provides:
★ OS and OS environment
★ Web server, DBMS, etc.
★ Middleware
★ Application software
Challenges – Client:
★ Transparency (naming, redirection)
★ Scalability: replication and load balancing decisions
★ Synchronization and coordination
★ Security
★ Fault tolerance
★ Software maintenance and sys admin
Challenges – Provider:
★ Hardware provisioning and maintenance
★ Load management
★ IP address management, DNS management
★ Infrastructure fault tolerance
★ Monitoring, logging, billing
★ Storage

EXAMPLE 1: AMAZON WEB SERVICES (AWS)

★ Elastic Compute Cloud (EC2)

★ Simple Storage Solution (S3)
★ Simple DB
★ Simple Queue Service

★ Instances: virtual cores, memory, storage

➢ instance types (cpu,memory,net, storage options):
 78

➢ t, m, c, p, g, x, r, i, d
➢ micro, small, medium, large, xlarge, ...
★ Cost:
➢ free tier: limited instances, free CPU hours
➢ on-demand: $0.007 - $39 per hour
➢ reserved: 1-3 years, discounted, fixed cost
★ Launch Amazon Machine Image (AMI) on instances
★ Preconfigured or custom images

PLATFORM AS A SERVICE

Service provider provides:

★ Hardware infrastructure
★ OS and platform software (middleware)
★ Distributed storage management
★ Load balancing, replication, migration
★ Management and Monitoring services
Client provides:
★ Application
Challenges – Client:
★ Learn new API and environment
★ Follow API
★ Optimise to limits of API and platform
★ Security for own app
Challenges – Provider:
★ Transparency (naming, redirection)
★ Scalability: replication and load balancing decisions
★ Synchronisation and coordination
★ Security
★ Fault tolerance
★ Monitoring
★ Software maintenance and sys admin

SOFTWARE AS A SERVICE

Service provider provides:

★ Hardware infrastructure
★ OS and platform software (middleware)
★ Distributed storage management
★ Load balancing, replication, migration
★ Management and Monitoring services
★ Application
 79

Client provides:
★ Data
Challenges– Client:
★ Learn new application
★ Deal with potential restrictions
➢ Web interface, restricted functionality
➢ No offline access, no local storage
Challenges – Provider:
★ Transparency (naming, redirection)
★ Scalability: replication and load balancing decisions
★ Synchronisation and coordination
★ Security
★ Fault tolerance
★ Monitoring
★ Software maintenance and sys admin
★ Application development and maintenance

KEY CHALLENGES OF CLOUD COMPUTING

Scalability:
★ Datacentre vs Global
★ Partitioning
➢ Services and Data
★ Replication
Consistency:
★ Dealing with the consequences of CAP Theorem
★ Dealing with un-usability of eventual consistency
Reliability:
★ SLA (Service Level Agreement): guarantees given by provider
➢ How reliable are the guarantees?
➢ What is the consequence if they aren’t met?
★ Redundancy and Replication
➢ within the same provider (e.g. Availability Zones, Regions, etc.)
➢ migration across providers
★ Geographically distributed architecture
★ Design for failure: Chaos Monkey
➢ Test how well the system deals with failure
➢ regularly and randomly kill system services

Security and Privacy:

★ External threats
➢ Denial of Service
➢ Infrastructure or platform service compromise
 80

➢ SaaS compromise: data theft

★ Co-located threats: other customers
➢ • Isolation: but, covert channels, bugs in isolation
★ Privacy: data collected by providers
➢ IaaS and PaaS providers: encryption only helps a bit
➢ SaaS providers: at mercy of service provider
➢ Governments and others: where is your data stored or processed? Which laws
apply?

Previous Year Question Solving:

1. Write about different types of cloud computing.

Answer:
★ Public: open services available to everyone
★ Private: owned, operated, and available to specific organizations.
★ Hybrid: the system uses some private cloud services and some public cloud
services.

2. What are the key characteristics of cloud computing? Describe the benefits of it.
Answer:
Key Characteristics of Cloud Computing:
SP 800-145. The NIST Definition of Cloud Computing:
★ On-demand, self-service
➢ get resources (CPU, storage, bandwidth, etc),
➢ automated: as needed, right now!
★ Network access
➢ Services accessible over the network, standard protocols
★ Pooled resources
➢ provider: multi-tenant pool of resources
➢ dynamically assigned and reassigned per customer demand
★ Elasticity
➢ Scalability: rapidly adjust resource usage as needed
★ Measured service
➢ monitor resource usage
➢ billing for resources used
Benefits of Cloud Computing:
Flexibility:
★ Flexible provisioning
★ Add machines on demand
★ Add storage on demand

Effort:
★ Low barrier to entry
★ Initial effort: no need to spec and set up physical infrastructure
 81

★ Continuing effort: no need to maintain physical infrastructure

Speed: Most cloud computing services are provided self-service and on demand, so even vast
amounts of computing resources can be provisioned in minutes, typically with just a few mouse
clicks, giving businesses a lot of flexibility and taking the pressure off capacity planning.

Cost:
★ Cloud computing eliminates the capital expense of buying hardware and software
★ setting up and running on-site data centers—the racks of servers, the round-the-clock
electricity for power and cooling, and the IT experts for managing the infrastructure. It
adds up fast.
★ Low initial capital expenditure
★ Avoid costs of over-provisioning for scalability
★ Pay for what you use

Global scale: The benefits of cloud computing services include the ability to scale elastically. In
cloud speak, that means delivering the right amount of IT resources—for example,
more or less computing power, storage, and bandwidth—right when they’re needed, and
from the right geographic location.

Security
★ Many cloud providers offer a broad set of policies, technologies, and controls that
strengthen your security posture overall, helping protect your data, apps, and
infrastructure from potential threats.
★ Redundancy
★ Trust reliability of provider
★ Data backups

3. Why is it called cloud in terms of cloud computing?

Answer:
★ services provided on virtualized resources
★ virtual machines spawned on demand
★ location of services no longer certain
★ similar to network cloud
4. Why is it called cloud in terms of cloud computing? Write the steps of finding
cloud services EC2 with examples.
Answer: Calling for Cloud Computing:
★ services provided on virtualized resources
★ virtual machines spawned on demand
★ location of services no longer certain
★ similar to network cloud
 82

5. Write about different types of cloud computing.

Answer:
★ Public: open services available to everyone
★ Private: owned, operated, and available to specific organizations.
★ Hybrid: the system uses some private cloud services and some public cloud
services.
6. Write the difference between system and human-oriented names.
Answer:

System-Oriented Names Human-Oriented Names

Represented in machine-readable Represented in variable-length character

form(32 or 64-bit strings) strings.

Structured or unstructured Structured

Not easy to remember hard for humans to Easy to remember and distinguish
use. between hard for a machine to process

Efficient manipulation and storage by Human comprehension and memorability

computer systems.

Bit patterns, unique identifiers. Character strings.

Machine-generated Human-generated

Use case: Internal communications. Data Use Case: User interactions, naming
management resources.

Example: inode(0x00245dad) Example: URL

7. Why is it called cloud in terms of cloud computing? Write down the types of cloud
computing with examples.
8. Explain cloud computing Infrastructure as a Service with example.
9. Describe the types of partitioning.
 83

Chapter-09

Distributed File System

INTRODUCTION

Distributed File System Paradigm:

★ File system that is shared by many distributed clients
★ Communication through shared files
★ Shared data remains available for long time
★ Basic layer for many distributed systems and applications
Clients and Servers:
★ Clients access files and directories
★ Servers provide files and directories
★ Servers allow clients to perform operations on the files and directories
★ Operations: add/remove, read/write
★ Servers may provide different views to different clients

CHALLENGES

Transparency
★ Location: a client cannot tell where a file is located
★ Migration: a file can transparently move to another server
★ Replication: multiple copies of a file may exist
★ Concurrency: multiple clients access the same file
Flexibility
★ Servers may be added or replaced
★ Support for multiple file system types
Dependability
★ Consistency: conflicts with replication & concurrency
★ Security: users may have different access rights on clients sharing files & network
transmission
 84

★ Fault tolerance: server crash, availability of files

★ Requests may be distributed across servers
★ Multiple servers allow higher storage capacity
Scalability
★ Handle increasing number of files and users
★ Growth over geographic and administrative areas
★ Growth of storage space
★ No central naming service
★ No centralized locking
★ No central file store

STATELESS VERSUS STATEFUL SERVERS

Advantages of stateless servers

★ Fault tolerance
★ No OPEN/CLOSE calls needed
★ No server space needed for tables
★ No limits on number of open files
★ No problems if server crashes
★ No problems if client crashes
Advantages of stateful servers
★ Shorter request messages
★ Better performance
★ Read ahead easier
★ File locking possible

CACHING

We can cache in three locations:

★ Main memory of the server: easy & transparent
★ Disk of the client
★ Main memory of the client (process local, kernel, or dedicated cache process)

Cache consistency:
★ Obvious parallels to shared-memory systems, but other trade offs
★ No UNIX semantics without centralized control
★ Plain write-through is too expensive; alternatives: delay
★ WRITEs and agglomerate multiple WRITEs
★ Write-on-close; possibly with delay (file may be deleted)
★ Invalid cache entries may be accessed if server is not contacted whenever a file is
opened

REPLICATION
Multiple copies of files on different servers
 85

★ Prevent data loss

★ Protect system against down time of a single server
★ Distribute workload
Three designs
★ Explicit replication: The client explicitly writes files to multiple servers(not transparent).
★ Lazy file replication: Server automatically copies files to other serversafter the file is
written.
★ Group file replication: WRITEs simultaneously go to a group of servers.

CASE STUDIES

★ Network File System (NFS)

★ Andrew File System (AFS) & Coda
★ Google File System (GFS)

NETWORK FILE SYSTEM (NFS)

Properties:
★ Introduced by Sun
★ Fits nicely into UNIX’s idea of mount points, but does not implement UNIX semantics
★ Multiple clients & servers (a single machine can be a client and a server)
★ Stateless servers (no OPEN & CLOSE) (changed in v4)
★ File locking through separate server
★ No replication
★ ONC RPC for communication
★ Caching: local files copies
➢ consistency through polling and timestamps
➢ asynchronous update of the file after close

ANDREW FILE SYSTEM (AFS) & CODA

Properties:
★ From Carnegie Mellon University (CMU) in the 1980s.
★ Developed as campus-wide file system: Scalability
★ Global namespace for file system (divided in cells, e.g. /afs/cs.cmu.edu, /afs/ethz.ch)
★ API same as for UNIX
★ UNIX semantics for processes on one machine, but globally write-on-close
★ Client: User-level process Venus (AFS daemon)
★ Cache on local disk
★ Trusted servers collectively called Vice
Scalability:
★ Server serves whole files. Clients cache whole files
★ Server invalidates cached files with a callback (stateful servers)
★ Clients do not validate cache (except on first use after booting)
 86

★ Result: Very little cache validation traffic

CODA

★ Successor of the Andrew File System (AFS)

➢ System architecture quite similar to AFS
★ Supports disconnected, mobile operation of clients
★ Supports replication

GOOGLE FILE SYSTEM(GFS)

Motivation:

★ 10+ clusters
★ 1000+ nodes per cluster
★ Pools of 1000+ clients
★ 350TB+ filesystems
★ 500Mb/s read/write load
★ Commercial and R&Dapplications

Assumptions:

★ Huge files (millions, 100+MB)

★ Large streaming reads
★ Small random reads
★ Large appends
★ Concurrent appends
★ Bandwidth more important than latency

No common standard like POSIX. Provides familiar file system interface:

★ Create, Delete, Open, Close, Read, Write
In addition:
★ Snapshot : low cost copy of a whole file with copy-on-write operation
★ Record append: Atomic append operation

★ Files split in fixed size chunks of 64 MByte

★ Chunks stored on chunk servers
★ Chunks replicated on multiple chunk servers
★ GFS master manages name space
★ Clients interact with master to get chunk handles
★ Clients interact with chunk servers for reads and writes
 87

★ No explicit caching

Picture hey

★ Single point of failure

★ Keeps data structures in memory (speed, easy background tasks)
★ Mutations logged to operation log
★ Operation log replicated
★ Checkpoint state when log is too large
★ Checkpoint has same form as memory (quick recovery)
★ Note: Locations of chunks not stored (master periodically asks chunk servers for list of
their chunks)

GFS Chunk servers:

★ Checksum blocks of chunks
★ Verify checksums before data is delivered
★ Verify checksums of seldom used blocks when idle
★ Write, atomic record append, snapshot
★ Master grants chunk lease to one of a chunk’s replicas
★ Replica with chunk becomes primary
★ Primary defines serial order for all mutations
★ Leases typically expire after 60 s, but are usually extended
★ Easy recovery from failed primary: master chooses another replica after the initial lease
expires

RE-EVALUATING GFS AFTER 10 YEARS

Workload has changed → changed assumptions

Single Master:
★ Too many requests for a single master
★ Single point of failure
★ Tune master performance
★ Multiple cells
★ Develop distributed masters
File Counts:
★ Too much meta-data for a single master
★ applications rely on Big Table (distributed)
★ Smaller files than expected
★ Reduce block size to 1MB

Throughput vs Latency:
★ Too much latency for interactive applications (e.g. Gmail)
★ Automated master failover
 88

★ Applications hide latency: e.g. multi-homed model

CHUBBY

Chubby is
★ Lock service
★ Simple FS
★ Name service
★ Synchronisation/consensus service
Architecture:
★ Cell: 5 replicas
★ Master:
➢ gets all client requests
➢ elected with Paxos
➢ master lease: no new master until lease expires
★ Write: Paxos agreement of all replicas
★ Read: local by master
★ Pathname: /ls/cell/some/file/name
★ Open (R/W), Close, Read, Write, Delete
★ Lock: Acquire, Release
★ Events: file modified, lock acquired, etc.

Using Chubby: electing a leader:

if (open("/ls/cell/TheLeader", W)) { write(my_id);

} else {
wait until "/ls/cell/TheLeader" modified;
leader_id = read();
}

Colossus:
★ follow up to GFS
BigTable:
★ Distributed, sparse, storage map
★ Chubby for consistency
★ GFS/Colossus for actual storage
Megastore:
★ Semi-relational data model, ACID transactions
★ BigTable as storage , synchronous replication (using Paxos)
★ Poor write latency (100-400 ms) and throughput
Spanner:
★ Structured storage, SQL-like language
★ Transactions with TrueTime, synchronous replication (Paxos)
 89

★ Better write latency (72-100ms)

Previous Year Question Solving:

★ Discuss about the challenges in distributed file system.

Answer:
CHALLENGES

Transparency
★ Location: a client cannot tell where a file is located
★ Migration: a file can transparently move to another server
★ Replication: multiple copies of a file may exist
★ Concurrency: multiple clients access the same file
Flexibility
★ Servers may be added or replaced
★ Support for multiple file system types
Dependability
★ Consistency: conflicts with replication & concurrency
★ Security: users may have different access rights on clients sharing files &
network transmission
★ Fault tolerance: server crash, availability of files
★ Requests may be distributed across servers
★ Multiple servers allow higher storage capacity
Scalability
★ Handle increasing number of files and users
★ Growth over geographic and administrative areas
★ Growth of storage space
★ No central naming service
★ No centralized locking
★ No central file store

2. Write short notes on Chubby, Google File System

Answer:
★ Chubby: Chubby is a highly available and persistent distributed lock service and
configuration manager for large-scale distributed systems. Client interface similar to
whole-file advisory locks with notification of various events(e.g. File modifications). The
primary goal of Chubby is reliability, availability, and easy-to-understand semantics. It is
used in Google: GFS, BigTable, etc. Elect leaders, store a small amount of metadata as
the root of the distributed data structures.
 90

★ Google File System: Google File System(GFS), a scalable distributed file system(DFS),
to meet the company’s growing data processing needs. GFS offers fault tolerance,
dependability, scalability, availability, and performance to big networks and connected
nodes. GFS is made up of several storage systems constructed from inexpensive
commodity hardware parts. The search engine, which creates enormous volumes of
data that must be kept, is only only one example of how it is customized to meet
Google’s various data use and storage requirements. The GFS reduced hardware flaws
while gaining commercially available servers.
3. Draw the upload/download model for the Network File System(NFS).
4. Write short notes on: Andrew File System, and Google File System.
Answer:
★ Andrew File System: AFS is a distributed file system. It uses the client/server model,
where all the files are stored on file server machines. Files are transferred to client
machines as necessary and cached on local disk. The server part of AFS is called the
AFS File Server, and the client part of AFS is called the AFS Cache Manager.
★ Google File System: Google File System(GFS), a scalable distributed file system(DFS),
to meet the company’s growing data processing needs. GFS offers fault tolerance,
dependability, scalability, availability, and performance to big networks and connected
nodes. GFS is made up of several storage systems constructed from inexpensive
commodity hardware parts. The search engine, which creates enormous volumes of
data that must be kept, is only only one example of how it is customized to meet
Google’s various data use and storage requirements. The GFS reduced hardware flaws
while gaining commercially available servers.

5. Write the key points on Stateless vs Stateful servers.

Answer:
Key Points of stateless servers
★ Fault tolerance
★ No OPEN/CLOSE calls needed
★ No server space needed for tables
★ No limits on number of open files
★ No problems if server crashes
★ No problems if client crashes
Key Points of stateful servers
★ Shorter request messages
★ Better performance
★ Read ahead easier
★ File locking possible
 91

Wow! Good …Carry On….Hurry up!!

So inspiring !!
Just go into the flow!!

Thank you for inspiring me. Take care.