Parallel and Distributed

Computing
Distributed Computing
Contents
• Definitions

• History of Distributed Computing

• Strengths and Weaknesses of Distributed Computing

• Architecture of Distributed Applications

Definitions
• A distributed system is one in which hardware or software components
located at networked computers communicate and coordinate their actions
only by message passing.

• In the term distributed computing, the word distributed means spread out
across space. Thus, distributed computing is an activity performed on a
distributed system.

• These networked computers may be in the same room, same campus, same
country, or in different country.
Definitions
Definitions
• The numbers of real applications are still somewhat limited, and the
challenges--particularly standardization--are still significant.
• But there's a new energy in the market, as well as some actual paying
customers, so it's about time to take a look at where distributed processing
fits and how it works.
• A distributed computing architecture consists of very lightweight software
agents installed on a number of client systems, and one or more dedicated
distributed computing management servers.
• There may also be requesting clients with software that allows them to
submit jobs along with lists of their required resources.
How it Works
• Distributed systems don’t have a single design or implementation pattern—
instead, they break down into patterns aligned with various types of business
problems.

• An example of this is the sharded pattern for stateful services like databases.
This allows you to horizontally scale your data tier to avoid performance
bottlenecks and single points of failure.

• This pattern is illustrated in Figure 1 below and is commonly referred to as a

distributed database.
How it Works
How it Works
History
• The idea of using idle computer power for big projects is old. Early programs
like Creeper explored this concept.

• The first internet project like this was in 1988, where people helped factor
large numbers by lending their computers' idle time.

• In the 1990s, projects with cash rewards for solving puzzles like cracking
encryption attracted more users.

• SETI@Home, launched in 1999, used idle computers to search for alien signals
and became the most famous project of this kind.
SETI@Home

• The project that truly popularized distributed computing and showed that it could
work was SETI@Home, an effort by the Search for Extraterrestrial Intelligence (SETI) at
the University of California at Berkeley.

• The project was started in May 1999 to analyze the radio signals that were being
collected by the Arecibo Radio Telescope in Puerto Rico.

• It has gained over three million independent users who volunteer their idle computers
to search for signals that may not have originated from Earth.

• This project has really brought the field to light, so that other groups and companies
are quickly following their lead.
Examples of distributed computing in action
• Here are some examples of where distributed computing is used. As you will
discover, this concept is so common that we are all beneficiaries in one way or
the other. Many of us encounter it without realizing it.
 Social media
 Online banking
 Price comparison
 Ride sharing
 Streaming services
 Search engines
 File sharing
 Email
Advantages and benefits of Distributed Computing
Here are 7 key advantages and benefits of distributed computing;

1. Cost-effective
2. Increased storage
3. Enhanced security
4. Improved performance
5. Increased reliability
6. Greater flexibility and scalability
7. Low latency
Drawbacks of Distributed Computing
While distributed computing delivers many benefits as we have seen, it also
has its fair share of drawbacks and disadvantages.

Here is a look at some of the drawbacks of distributed computing

1. Network transfer bottlenecks

2. System complexity
3. Standardization issues
4. Maintenance costs
5. Potential for component failure due to human error
6. Bandwidth limitations
7. Deploy challenges
Advantages and benefits of Distributed Computing
Different Types of Distributed Computing
• Applications that run on numerous computers as opposed to only one are made
for distributed computing.

• By developing the program, you may make it so that many computers work
together to complete various tasks and provide the desired outcome.

• The following list includes the four main types of distributed architecture;
 Client-server architecture
 Three-tier architecture
 N-tier architecture
 Peer-to-peer architecture
Most Popular Distributed Computing Frameworks And
Tools
• These are only a handful of the numerous distributed computing frameworks that
are accessible.
• It's critical to select the option that most closely matches the unique requirements
of your company because each has strengths and drawbacks of its own.
• The most common distributed computing tools are outlined below:
• Hadoop
• Spark
• Flink
• Storm
Key Components Of Distributed Architectures
• Distributed systems are complex and can vary greatly depending on their
specific use case and architecture. However, the following core components
are present in most distributed systems:
Nodes
• These are the individual computers or servers that make up the distributed
system. Each node runs its own instances of the applications and
services that make up the system.
Network
• This is the communication infrastructure that connects all nodes. The
network enables data and information to be exchanged between nodes.
Key Components Of Distributed Architectures
Middleware
• This software layer provides a programming model for developers and masks
the heterogeneity of the underlying network, hardware, and operating systems.
It provides useful abstractions and services which simplify the process of
creating complex distributed systems.
Shared Data/Database
• This is where the system stores and retrieves data. Depending on the specific
design of the distributed system, the data could be distributed across multiple
nodes, replicated, or partitioned.
Distributed Algorithms
• These are the rules and procedures nodes follow to communicate and
coordinate with each other. They enable nodes to work together, even when
some nodes fail or network connections are unreliable.
Key Components Of Distributed Architectures
System Management Tools
• These tools help manage, monitor, and troubleshoot the distributed system.
They provide functionality for load balancing, fault tolerance, system
configuration, and more.
Distributed Computing vs.
Related Concepts
Distributed Computing vs. Cloud Computing
Distributed computing involves a collection of independent computers
connected via a network, working together to perform tasks. Each computer
in a distributed system operates autonomously, and the system is designed to
handle failures of individual machines without affecting the entire system's
functionality.

Cloud computing is a model for delivering computing services over the

internet. It provides on-demand access to shared computing resources, such
as servers, storage, and applications, without direct active management by
the user. While distributed computing is about the system architecture, cloud
computing is more about service delivery.
Distributed Computing vs. Grid Computing
Grid computing involves using the unused processing power of computers
connected over a network (often the internet), to solve complex
computational problems. It is a decentralized form of distributed computing
where each node is independent, and there is no central coordinating system.

In contrast, distributed computing can be either centralized or decentralized,

depending on the architecture. It involves multiple computers sharing the
workload to achieve common goals. The computers in a distributed system
may be physically close together and connected by a local network, or they
may be geographically distant and connected by a wide area network.
Distributed Computing vs. Parallel Computing
Parallel computing is a type of computation in which many calculations or
processes are carried out simultaneously. It breaks down a large problem into
smaller, independent parts that can be solved concurrently.
While distributed computing and parallel computing share the objective of
processing tasks more quickly, they differ in how they achieve this. In parallel
computing, all processors may have access to shared memory to exchange
information, while in distributed systems, each node has its own memory.
Examples of Distributed Systems
• Examples of distributed systems and applications of distributed computing include the following:

• Telecommunication networks:

• Telephone networks and cellular networks

• Computer networks such as the Internet

• Network applications:

• World wide web and peer-to-peer networks

• Massively multiplayer online games and virtual reality communities

• Real-time process control:

• Aircraft control systems

• Industrial control systems

• Parallel computation:

• Scientific computing, including cluster computing and grid computing and various volunteer computing projects

• Distributed rendering in computer graphics