DS Unit-3 Notes
DS Unit-3 Notes
Fault Tolerance
4. Classification of Failures
Issues
Ex: database slowdown, memory leak, blocked thread, bad data etc.
Fault tolerance is the property that enables the system to continue operating properly in the event of
failure of some of components.
1. Availability
2. Reliability
3. Safety
4. Maintainability
Availability: Availability is defined as the property that a system is ready to be used immediately
A high available system is one that will most likely be working at an given instance of
time(even after failure)
Reliability:
A highly reliable system is one that will most likely continue work without an interruption
Ex: If a system goes down on millisecond every one hour then the system is 99.99% available
but it is highly unreliable
Safety:
Refers to that situation that when a system temporarily fail, to operate correctly, nothing
bad/catastrophic/ dangerous happens
Ex: Sending people into space requires a high degree of safety, if such control system
temporarily fail for only a very brief moment the effect will be disastrous
Maintainability:
TYPES OF FAULTS
1. Transient Faults
2. Intermittent Faults
3. Permanent Faults
4. Processor Faults
Transient Faults
Ex: a bird flying through a beam of a microwave transmitter may cause loss of bits
Intermittent Faults
Fault occur, then variables of its own occurred, then reappear and so on
Permanent Faults
Fail silent faults: Faulty processor just stops and doesn’t respond at all
Byzantine faults: Faulty processor continues to run, issuing in the wrong answers/maliciously
Information Redundancy – add extra bits to allow for error detection/recovery (e.g.,
Hamming codes and the like).
Time Redundancy – perform operation and, if needs be, perform it again. Think about how
transactions work.
Physical Redundancy – add extra (duplicate) hardware and/or software to the system.
• Byzantine Agreement
4. Failure Detection
Ex: database slowdown, memory leak, blocked thread, bad data etc.
Processes can be made fault tolerant by arranging to have a group of processes, with each
member of the group being identical.
A message sent to the group is delivered to all of the “copies” of the process (the group
members), and then only one of them performs the required service.
If one of the processes fail, it is assumed that one of the others will still be able to
function (and service any pending request or operation).
FAILURE DETECTION
What it all boils down to is that for a group of processes, non faulty members should be able
to decide who is still a member, and who is not. In other words, we need to be able to detect
when a member has failed.
When it comes to detecting process failures, there are essentially only two mechanisms
i)Either processes actively send "are you alive?" messages to each other (for which they
obviously expect an answer) (usually followed)
ii) or passively waits until message come in from different process(practically this idea seems
to be good not used as if some process is not sending any message hence we cannot say its
dead)
TCP masks omission failure, which occur in the form of last messages, by using
acknowledgment and re-transmission(which are completely hidden from user).
How ever crash failures are not masked .It occurs when connection breaks, Here the client
informed that crash happened due to this reason
One way to mask this is to re-connect/re-establish connection automatically from server side
The goal of RPC is to hide communication by making remote procedure call, just like local ones
with few exceptions when server/client gets an issue
FEW FAILURES
Assume the server is down and client is requesting for the website then instead of loading
infinitely. We show an exception stating that page cannot be reached(currently in use today)
So with the concept of ack and timer we solve this lost request message
Whenever a client request for something, it will start a timer and if did not got any reply/ack it
will re-transmit again
Here depending on case of crash the operation in that server should do will depend
In case i) As the execution is done successfully by server then just re-sending ack/reply will be
enough
In case ii) here as execution is not done hence need to request again, based on this the client
cannot identify requirements, Hence to avoid these confusions we divide process into three
different events and based on the order we decide what append
3. crash (C).
1. M ~P ~C: A crash occurs after sending the completion message and printing the text.
2. M ~C (~P): A crash happens after sending the completion message, but before the text could
be printed.
3. p ~M ~C: A crash occurs after sending the completion message and printing the text.
4. P~C( ~M): The text printed, after which a crash occurs before the completion message could
be sent.
The obvious solution is just to rely on a timer again that has been set by the client's operating
system. If no reply is forthcoming within a reasonable period, just send the request once
more.
v)CLIENT CRASHES
What happens if a client sends a request to a server to do some work and crashes before the
server replies? At this point a computation is active and no parent is waiting for the result.
Such an unwanted computation is called an orphan.
Orphans can cause a variety of problems creating load on systems/server/waiting CPU cycles
Reliable multicast services guarantee that all messages are delivered to all members of a
process group.
The idea is simple but is become more and more tricky as no of clients/systems increases
This is a simple solution to reliable multicasting when all receivers are known and are assumed
not to fail. The sending process assigns a sequence number to outgoing messages (making it
easy to spot when a message is missing).
The main problem with the reliable multicast scheme just described is that it cannot support
large number of receivers.
If there are N receivers, the sender must be prepared to accept at least N acknowledgments.
With many receivers, the sender may be swamped with such feedback messages, which is also
referred to as a feedback implosion.
One solution to this problem is not to have receivers acknowledge the receipt of a message
Instead, a receiver returns a feedback message only to inform the sender it is missing a
message.
Feedback Suppression – reducing the number of feedback messages to the sender (as
implemented in the Scalable Reliable Multicasting Protocol).
Hierarchical reliable multicasting is another solution, the main characteristic being that it
supports the creation of very large groups.
Sub-groups within the entire group are created, with each local coordinator forwarding
messages to its children.
A local coordinator handles retransmission requests locally, using any appropriate multicasting
method for small groups.
Distributed COMMIT(SAVING)
General Goal: We want an operation to be performed by all group members or none at all.
1.single-phase commit
2. two-phase commit
3. three-phase commit.
Commit Protocols
An elected coordinator tells all the other processes to perform the operation in question.
All votes are collected by the coordinator. A GLOBAL_COMMIT is sent if all the group
members voted to commit. If one group member voted to abort, a GLOBAL_ABORT is
sent.
The group members then COMMIT or ABORT based on the last message received from
the coordinator.
It can lead to both the coordinator and the group members blocking, which may lead to the
dreaded deadlock.
If the coordinator crashes, the group members may not be able to reach a final decision,
and they may, therefore, block until the coordinator recovers …
c) Main point: although 3PC is generally regarded as better than 2PC, it is not applied often in
practice, as the conditions under which 2PC blocks rarely occur.
Once a failure has occurred , it is essential that the process where the failure happened to
recover to correct state
2.Forward Recovery: Bring a system into correct state, from which it can then continue to
execute
When an error occurs, the recovery mechanism then knows what to do bring the system
forward to a correct state
– CORBA
– DCOM
– GLOBE
CORBA
Background:
• Developed by the Object Management Group (OMG) in response to industrial demands for
object- based middleware
• Very much the work of a committee: there are over 800 members of the OMG and many of
them have a say in what CORBA should look like.
Essence: CORBA provides a simple distributed-object model, with specifications for many supporting
services it may be here to stay (for a couple of years)
CORBA Overview
Object Request Broker (ORB): CORBA’s object broker that connects clients, objects, and services
Proxy/Skeleton: Precompiled code that takes care of (un)marshaling invocations and results.
• Interface repository: Database containing interface definitions and which can be queried at
runtime
• Implementation repository: Database containing the implementation (code, and possibly also
state) of objects. Effectively: a server that can launch object servers.
SUTHOJU GIRIJA RANI, Assistant Professor, CSE, NGIT 1
Distributed Systems CSE
CORBA Object Model
Essence: CORBA has a “traditional” remote-object model in which an object residing at an object
server is remote accessible through proxies
Observation: All CORBA specifications are given by means of interface descriptions, expressed in an
IDL(Interface Definition Language). CORBA follows an interface-based approach to ob- jects:
• Not the objects, but interfaces are the really important entities
• An object may implement one or more interfaces
• Interface descriptions can be stored in an interface repository, and looked up at runtime
• Mappings from IDL to specific programming are part of the CORBA specification (languages
include C, C++, Smalltalk, Cobol, Ada, and Java.
CORBA Services
Service Description
Collection Facilities for grouping objects into lists, queue, sets, etc.
Query
Facilities for querying collections of objects in a declarative manner
Event
Facilities for asynchronous communication through
events
Notification
Advanced facilities for event-based asynchronous communication
Trading
Facilities to publish and find the services an object has to offer
Communication Models
Event communication: There are also additional fa- cilities by means of event channels:
Interceptors: A user written object that allows functionality to be inserted into CORBA. Adds security
& monitors components.
Message-level: Allows you to control message-passing between client and server (e.g., handle
reliability and fragmentation)
Naming
Specification level: An object reference is considered to be the same as a proxy for the referenced
object having an object reference means you can directly invoke methods; there is no separate client-
to-object binding phase
Implementation level: When a client gets an object reference, the implementation ensures that, one
way or the other, a proxy for the referenced object is placed in the client’s address space:
Naming Service
Essence: CORBA’s naming service allows servers to associate a name to an object reference, and have
clients subsequently bind to that object by resolving its name
Observation: In most CORBA implementations, ob- ject references denote servers at specific hosts;
nam- ing makes it easier to relocate objects
Observation: In the naming graph all nodes are ob- jects; there are no restrictions to binding names to
ob- jects CORBA allows arbitrary naming graphs
Observation: There is no single root; an initial con- text node is returned through a special call to the
ORB. Also: the naming service can operate across different ORBs interoperable naming service
Observation1: Recognizing that object references are implementation dependent, we need a separate
referencing mechanism to cross ORB boundaries
Solution: Object references passed from one ORB to another are transformed by the bridge through
which they pass (different transformation schemes can be implemented)
Observation2: Passing an object reference from ORB A to ORB B circumventing the A-to-B bridge may
be useless if ORB B doesn’t understand
Observation: To allow all kinds of different systems to communicate, we standardize the reference
that is passed between bridges:
Fault Tolerance
Essence: Mask failures through replication, by putting objects into object groups. Object groups are
transparent to clients: they appear as “normal” objects. This approach requires a separate type of
object reference: Interoperable Object Group Reference:
IOGRs have the same structure as IORs; the main difference is that they are used differently. In IORs
an additional profile is used as an alternative; in IOGR, it denotes another replica.
SUTHOJU GIRIJA RANI, Assistant Professor, CSE, NGIT 3
Distributed Systems CSE
Security
Essence: Allow the client and object to be mostly unaware of all the security policies, except perhaps
at binding time; the ORB does the rest. Specific policies are passed to the ORB as (local) objects and
are invoked when necessary:
• Evolved from early Windows versions to current NT-based systems (including Windows 2000)
DCOM Overview
DCOM is related to many things that have been introduced by Microsoft in the past couple of years:
SCM: Service Control Manager, responsible for activating objects (cf., to CORBA’s implementation
repository).
Proxy marshaler: handles the way that object references are passed between different machines
2. Each interface is typed, and therefore has a globally unique interface identifier
DCOM Services
Object invocations: Synchronous remote method calls with at-most-once semantics. Asynchronous
invocations are supported through a polling model, as in CORBA.
Observation: Objects are referenced by means of a local interface pointer. The question is how such
pointers can be passed between different machines:
Naming: Monikers
Observation: DCOM can handle only objects as temporary instances of a class. To accommodate
objects that can outlive their client, something else is needed.
Active Directory
Essence: a worldwide distributed directory service, but one that does not provide location
transparency.
Basics: Associate a directory service (called domain controller) with each domain; look up the
controller using a normal DNS query:
Fault Tolerance
Automatic transactions: Each class object (from which objects are created), has a transaction attribute
that determines how its objects behave as part of a transaction:
Declarative security: Register per object what the system should enforce with respect to
authentication. Authentication is associated with users and user groups. There are different
authentication levels:
Impersonation Description
IDENTIFY The server knows the client and can do access control
checks
IMPERSONATE The server can invoke local objects on behalf of the client
There is also support for programmatic security by which security levels can be set by an application,
as well as the required security services.
2. Prototype version up and running across multiple machines distributed in various networks.
Object Model
Essence: A Globe object is a physically distributed shared object: The object’s state may be physically
distributed across several machines.
Local object: A non-distributed object residing a single address space, often representing a
distributed shared object
Contact point: A point where clients can contact the distributed object; each contact point is
described through a contact address
Control subobject: Connects the user-defined inter- faces of the semantics subobject to the generic,
predefined interfaces of the replication subobject
Semantics subobject: Contains the methods that im-plements the functionality of the distributed
shared object
Client-to-Object Binding
Globe Services
Object References
Essence: Globe uses location-independent object handles which are to be resolved to contact
addresses (which describes where and how an object can be contacted):
Specifies (for example) a transport-level network address to which the object will listen
Contains an implementation handle, specifying exactly what the client should implement if it wants to
communicate through the contact point:
– “slave/master-slave/tcp/ip”
Observation: Objects in Globe have their own object-specific implementations; there is no “standard”
proxy that is implemented for all clients
Naming Objects
Observation1: Globe separates naming from locating objects. The current naming service is based on
DNS, using TXT records for storing object handles
Description
Called to synchronize replicas of the semantics subobjects, obtain locks if
necessary, etc.
Provide marshaled arguments of a specific method, and pass invocation to local
objects in other address spaces
Called after the control subobject has invoked a specific method at the
semantics subobject
The organization of a local object is such that replication is inherently part of each distributed shared
object GIRIJA RANI, Assistant Professor, CSE, NGIT
SUTHOJU 3
Distributed Systems CSE
All replication subobjects have the same interface:
Security
Essence: Additional security subobject checks for authorized communication, invocation, and
parameter values.
Comparison