Distributed Computing Paradigms

Paradigms for Distributed Applications

Paradigm means “a pattern, example, or model.”  In the study of any subject of

great complexity, it is useful to identify the basic patterns or models, and classify
the detail according to these models. 

The paradigms will be presented in the order of their level of abstraction as

shown in fig 3.1. At the lowest level of abstraction is Message passing which
encapsulates the least amount of detail and the Object Space occupies the
highest level of abstraction among all the paradigms.
 

Fig 3.1 Distributed Computing Paradigms and their level of abstraction

3.2 The Message Passing Paradigm

•    Message passing is the most fundamental paradigm for distributed applications. 

•    A process sends a message representing a request. 
•    The message is delivered to a receiver, which processes the request, and sends
a  message in response.

•    In turn, the reply may trigger a further request, which leads to a subsequent reply,
and so forth. 
•    The basic operations required to support the basic message passing paradigm
are send, and receive. 
•    For connection-oriented communication, the
operations connect and disconnect are also required.  

 
• With the abstraction provided by this model, the interconnected processes
perform input and output to each other, in a manner similar to file I/O.    The I/O
operations encapsulate the detail of network communication at the operating-
system level.

•  The socket application programming interface is based on this paradigm.

3.3 The Client-Server Paradigm

•    Perhaps the best known paradigm for network applications, the client-server
model assigns asymmetric roles to two collaborating processes. One process,
the server, plays the role of a service provider which waits passively for the
arrival of requests. The other, the client, issues specific requests to the server
and awaits its response.  
•    Simple in concept, the client-server model provides an efficient abstraction for the
delivery of network services.  Operations required include those for a server
process to listen and to accept requests, and for a client process to issue
requests and accept responses. 
•    By assigning asymmetric roles to the two sides, event synchronization is
simplified: the server process waits for requests, and the client in turn waits for
responses.
Many Internet services are client-server applications.  These services are often
known by the protocol that the application implements. Well known Internet
services include HTTP, FTP, DNS, finger, gopher, etc. 

3.4 The Peer-to-Peer System Architecture

•    In system architecture and networks, peer-to-peer is an architecture where

computer resources and services are direct exchanged between computer
systems. These resources and services include the exchange of information,
processing cycles, cache storage, and disk storage for files.
•     In such architecture, computers that have traditionally been used solely as clients
communicate directly among themselves and can act as both clients and servers,
assuming whatever roles are most efficient for the network.
•     In the peer-to-peer paradigm, the participating processes play equal roles, with
equivalent capabilities and responsibilities (hence the term “peer”).  Each
participant may issue a request to another participant and receive a response.
•    Where the client-server paradigm is an ideal model for centralized network service,
the peer-to-peer paradigm is more appropriate for applications such as instant
messaging, peer-to-peer file transfers, video conferencing, and collaborative
work. 
•    It is also possible for an application to be based on both the client-server model
and the peer-to-peer model. A well-known example of a peer-to-peer file transfer
service is Napster.com which allows audio files to be transmitted among
computers on the Internet along with a server for directory.
•    For web applications, the web agent  is a protocol promoted by the XNSORG (the
XNS Public Trust Organization) for peer-to-peer interprocess communication
•     Project JXTA is a set of open, generalized peer-to-peer protocols that allow any
connected device (cell phone, PDA, etc) on the network to communicate.

3.5 The Message System Paradigm

•    The Message System or Message-Oriented Middleware (MOM) paradigm is an
elaboration of the basic message-passing paradigm. In this paradigm, a message
system serves as an intermediary among separate, independent processes. 
•    A sender deposits a message with the message system, which forwards it to a
message queue associated with each receiver.  Once a message is sent, the
sender is free to move on to other tasks.

 
                                               Fig 3.4 The Message System Paradigm 
•    The message system acts as a switch for messages, through which processes
exchange messages asynchronously, in a decoupled manner.
There are two main subtypes of message system models
1)      The Point-To-Point Message Model
2)      The Publish/Subscribe Message Model

The Point-To-Point Message Model

•    In this model, a message system forwards a message from the sender to the
receiver’s message queue. Unlike the basic message passing model, the
middleware provides a message depository, and allows the sending and the
receiving to be decoupled.
•    Via the middleware, a sender deposits a message in the message queue of the
receiving process.  A receiving process extracts the messages from its message
queue, and handles each one accordingly.
Compared to the basic message-passing model, this paradigm provides the
additional abstraction for asynchronous operations.  To achieve the same effect
with basic message-passing, a developer will have to make use of threads or
child processes.

The Publish/Subscribe Message Model

•    In this model, each message is associated with a specific topic or event.
Applications interested in the occurrence of a specific event may subscribe to
messages for that event. 
•    When the awaited event occurs, the process publishes a message announcing
the event or topic. The middleware message system distributes the message to
all its subscribers.
•    The publish/subscribe message model offers a powerful abstraction for
multicasting or group communication.   The publish operation allows a process to
multicast to a group of processes, and thesubscribe operation allows a process to
listen for such multicast.
•    Using the publish/subscribe message model the auctioning system could be
implemented as follows:
     1) Each participant subscribes to begin-auction event message.
2) The auctioneer indicates the beginning of the auction by sending a begin-
auction event message.
3) Upon receiving the begin-auction event, the participant subscribes to an end
auction message.
4) The auctioner wishing to subscribe for new bidding sends the new bid event
message.
•      Message Queue Services (MQS) have been in use since the 1980’s. Other
existing supports for this include Microsoft’s Message Queue (MSQ) and Java’s
Message Service.

3.6 Remote Procedure Call (RPC)

•    As applications grew increasingly complex, it became desirable to have a
paradigm which allows distributed software to be run on a single processor.
•    The Remote Procedure Call (RPC) model provides such an abstraction. Using
this model, interprocess communications proceed as procedure, or function,
calls, which are familiar to application programmers.
•    In a program that involves only one process, a call to a procedure such as
someFunc(arg1, arg2) results in a change of the execution flow to the code in the
procedure as shown in fig 3.5. Arguments can be carried into the procedure as
parameters for its execution.
 Fig 3.5 Execution flow of code in a local procedure

•    A remote procedure call involves two independent processes, which may reside
on separate machines.  A process,  A, wishing to make a request to another
process, B, issues a procedure call to B, passing with the call a list of argument
values. At the completion of the procedure, process B returns a value to
processA.
•    Figure 3.6 illustrates the RPC paradigm. A procedure call is made by one process
to another, with data passed as arguments. Upon receiving a call, the actions
encoded in the procedure are executed as a result.
•    As a comparison, the message-passing model is data-oriented, with the actions
triggered by the mes
As a comparison, the message-passing model is data-oriented, with the actions
triggered by the message exchanged, while the RPC model is action-oriented,
with the data passed as arguments.
•    RPC allows programmers to build network applications using a programming
construct similar to the local procedure call, providing a convenient abstraction for
both interprocess communications and event synchronization

Fig 3.6 The Remote Procedure Call Paradigm

 3.7 The Distributed Objects Paradigms

•    The idea of applying object orientation to distributed applications is a natural

extension of object-oriented software development. Applications access objects
distributed over a network.
•    Objects provide methods, through the invocation of which an application obtains
access to services.
•     Object-oriented paradigms include:
  Remote method invocation (RMI)
  Network services
  Object request broker (ORB)
  Object spaces

Remote Method Invocation (RMI)

•    Remote method invocation is the object-oriented equivalent of remote method
calls.  In this model, a process invokes the methods in an object, which may
reside in a remote host. As with RPC, arguments may be passed with the
invocation.
•    The implementation of our auctioning system is essentially the same as with RPC,
except that object methods replace procedures:
•    The auctioning program provides a remote method for each participant register
itself and another method for a participant to make a bid.
•    Each participant program provides the following remote methods: (1) to allow the
auctioneer to call it to announce the onset of the session, (2) to allow the
auctioneer to inform the program of a new highest bid, and (3) to allow the
auctioneer to announce the end of the session.

        The Object Request broker Paradigm

•      In the object broker paradigm, an application issues requests to an object request
broker (ORB), which directs the request to an appropriate object that provides the
desired service.
•     The paradigm closely resembles the remote method invocation model in its
support for remote object access.  The difference is that the object request broker
in this paradigm functions as a middleware which allows an application, as an
object requestor, to potentially access multiple remote (or local) objects. 
•    The request broker may also function as a mediator for heterogeneous objects,
allowing interactions among objects implemented using different APIs and /or
running on different platforms.

Fig 3.8 The Object Request broker Paradigm

•     Implementation of the auctioning system using ORBs is similar to using RMI, with
the exception that each object (auctioneer or participant) must be registered with
the ORB and requested from the ORB. Each participant issues requests to the
auctioneer object to register for the session and to make bids.
•     Through the ORB, the auctioneer invokes the methods of each participant to
announce the start of the session, to update the bidding status, and to announce
the end of the session.
•     This paradigm is the basis of the Object Management Group's CORBA (Common
Object Request Broker Architecture).
•     Toolkits based on the architecture include Inprise's Visibroker, Java Interface
Definition Language (Java IDL), IONA's Orbix etc.
•    Component-based technologies, such as Microsoft COM, Microsoft DCOM, Java
Bean, and Enterprise Java Bean, are also based on distributed-object paradigms.

 The Object Space

•      Perhaps the most abstract of the object-oriented paradigms, the object space
paradigm assumes the existence of logical entities known as object spaces. The
participants of an application converge in a common object space.
•       A provider places objects as entries into an object space, and requesters who
subscribe to the space access the entries. 
•      In addition to the abstractions provided by other paradigms, the object space
paradigm provides a virtual space or meeting room among provides and
requesters of network resources or objects. 
•      This abstraction hides the detail involved in resource or object lookup needed in
paradigms such as remote method invocation, object request broker, or network
services.

 
 
Fig 3.9 The Object Space Paradigm

•      For the auctioning system, all participants as well as the service provider
subscribe to a common object space. Each participant deposits an object into the
object space to register for the session and to be notified when the auctioning
session starts.
•      At the onset of the session, the auctioneer deposits an object into the object
space. The object contains the item information and the bid history.
•      A participant wishing to place a bid retrieves the object from the space and, if
he/she so chooses, places a new bid in the object before returning the object to
the space.
•      At the end of the session, the auctioneer retrieves the object from the space and
contacts the highest bidder.

The Mobile Agent Paradigm

•      A mobile agent is a transportable program or object.

•      In this model, an agent is launched from an originating host. The agent travels
from host to host according to an itinerary that it carries.
•      At each stop, the agent accesses the necessary resources or services, and
performs the necessary tasks to accomplish its mission.
•      The paradigm offers the abstraction for a transportable program or object. Instead
of message exchanges, data is carried by the program/object as the program is
itself transported among the participants.
•      The mobile agent paradigm provides a novel way of implementing our auctioning
system. At the onset, each participant launches a mobile agent to the auctioneer.
•      The mobile agent carries with it the identity, including the network address, of the
participant that it represents.
•      Once the session starts, the auctioneer launches a mobile agent that carries with
it an itinerary of the participants, as well as the current highest bid.
•      The mobile agent circulates among the participants and the auctioneer until the
session ends, at which time the auctioneer launches the agent to make one more
round among the participants to announce the outcome.
•      Commercial packages that support the mobile agent paradigm include the
Mitsubishi’s Concordia system and the IBM’s Aglet system.

      Fig 3.10 The Mobile Agent Paradigm

 The Network Services Paradigm

•      In this paradigm, service providers register themselves with directory servers on a
network.  A process desiring a particular service contacts the directory server at
run time, and, if the service is available, will be provided a reference to the
service.  Using the reference, the process interacts with the service.
•      This paradigm is essentially an extension of the Remote Method Invocation
paradigm. The difference is that service objects are registered with a global
directory service, allowing them to be looked up and accessed by service
requestors on a federated network.
•      Ideally, services may be registered and located using a globally unique identifier,
in which case, the paradigm offers an extra abstraction: location transparency.
Location transparency allows a software developer to access an object or a
service without having to be concerned of the location of the object or service.
•      The implementation of our auctioning system is the same as under RMI paradigm,
except that the auctioneer registers itself with the directory service, allowing the
participants to locate it and, once the session has commenced, to make bids.
•      The participants provide callback methods to allow the auctioneer to announce the
start and the end of the session, and to update the status of the session.
•      Java's Jini technology is based on this paradigm.

   

                                       Fig 3.11 The Network Services Paradigm

The Collaborative Application (Groupware) Paradigm

•      In this model, processes participate in a collaborative session as a group. Each

participating process may contribute input to part or the entire group. 
•      Processes may do so using:
  multicasting to send data to all or part of the group, or they may use a
  virtual sketchpads or whiteboards which allows each participant to read and
write data to a shared display.
•      To implement the auctioning system using the message-based groupware
paradigm, the auctioneer initiates a group, to be joined by all interested
participants.
•      At the onset of the session, the auctioneer multicasts a message announcing the
start. During the session, each bid is multicast to all participants so that each one
may independently assess the status of the auction.
•      Finally, the auctioneer terminates the session by multicasting a message
announcing the outcome.
•      In the second method the auctioneer and the participants share a virtual
whiteboard. The auctioneer starts the bidding process by writing an
 announcement to the whiteboard. Subsequently, each participant may place a bid
by writing to the whiteboard.
•      Eventually, the auctioneer terminates the session by writing a final announcement.
•      Application program interfaces supporting the message-based shareware
paradigm include the Java multicast API and the Java Shared Data Toolkit
(JSDT).
•      The whiteboard paradigm is the basis for a number of applications such as the
SMARTBoard [smarttech.com], NetMeeting and Groove [groove.net].
•      The Notification Service Transfer Protocol (NSTP) has been proposed as "an
infrastructure for building synchronous groupware. It is based on the idea of a
coordinating notification server that is independent of any given synchronous
groupware application.
   

Fig 3.12 The Collaborative Application (Groupware) Paradigm

3.8 Trade-offs

The term trade-off means compromising one thing or feature of a process in

return for another to balance the working of that process. It can be said as a
technique of reducing or forgoing one or more desirable outcomes in exchange
for increasing or obtaining other desirable outcomes in order to maximize the
total return or effectiveness under given circumstances.
 
Level of Abstraction versus Overhead

We have already looked at the communication paradigms and their relative levels
of abstraction. At the lowest levels are the most basic paradigms: message
 passing and client server. At the highest levels are object spaces and
collaborative computing, which provide the most abstraction. The development of
a highly complex application may be greatly aided by a tool that offers a high
level of abstraction.

But abstraction comes at a price, which is overhead. Consider the remote method
abstraction, where software modules known as stubs and skeletons need to be
present at run time to handle details of interprocess communications. The run-
time support and additional software modules require additional system
resources and execution time.

An application written using RMI will require modern resources and will have a
longer run time than the one written using socket API. For this reason, the socket
API may be the most appropriate for application that calls for a fast response time
or for a system with minimum resources.

Scalability

•    The complexity of a distributed application increases significantly, as the number

of participants (processes or objects) increases.

•    Consider our example application, the auctioning system. As described, the

auctioneer must manage the addresses of the participants so that it may
announce the start and end of the session. Furthermore, the auctioneer needs to
repeatedly contact individual participants to inform them of the latest highest bid.

•    Using message passing, the developer must provide the coding to manage the
addresses and to contact the participants individually. The complexity grows as
the number of participants increases.

•    With a toolkit based on a high-level paradigm such as object spaces or a publish/

subscribe message system, the complexity of managing the participants is
handled by the system with no additional complexity.

•    Mobile-agent is another paradigm that allows an application to scale well, since

the number of participating hosts can increase without any significant impact on
the complexity of a program based on the paradigm.
 Cross-Platform Support

•    Paradigms, being abstract models, are inherently platform independent. Most of

the toolkits based on paradigms are also platform dependent. In order to provide
the generality, a tool that supports heterogeneous platforms necessarily incurs
complexity, compared to one that supports a single platform.

•    Many of the Java technologies, including Java RMI, API and JavaSpaces, by
choice run on Java virtual machines only. As a result, if these technologies are
employed, all participants in an application must be written in the Java language.

•    Likewise, COM/DCOM technologies are deployable on Microsoft platforms only.

By contrast, CORBA is an architecture designed for cross-platform support.
Hence tools based on the architecture can support programs written in different
languages and can also support processes running on different platforms.

•    Beyond these trade-offs, there are software-engineering issues that should be

considered when you are choosing a tool. Some of these are
  The maturity and stability of the tool
  The fault tolerance provided by the tool
  The availability of developer tools
  Maintainability
  Code reuse