Scribd
Upload
2 views

Lecture 1.1 (Introduction, Goals, Issues)

The document provides an overview of Parallel and Distributed Computing, highlighting their definitions, advantages, and disadvantages. It explains the goals of these computing methods, such as improving computational speed and efficiency, and outlines the differences between them. Additionally, it discusses the challenges faced in implementing these systems, including communication overhead, synchronization issues, and security vulnerabilities.

Uploaded by

flyingthoughts861
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PPTX, PDF, TXT or read online on Scribd
2 views

Lecture 1.1 (Introduction, Goals, Issues)

The document provides an overview of Parallel and Distributed Computing, highlighting their definitions, advantages, and disadvantages. It explains the goals of these computing methods, such as improving computational speed and efficiency, and outlines the differences between them. Additionally, it discusses the challenges faced in implementing these systems, including communication overhead, synchronization issues, and security vulnerabilities.

Uploaded by

flyingthoughts861
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PPTX, PDF, TXT or read online on Scribd
You are on page 1/ 12

University Institute of Engineering

DEPARTMENT OF COMPUTER SCIENCE


& ENGINEERING
Bachelor of Engineering (Computer Science & Engineering)
Subject Name: Parallel and Distributed Computing
Subject Code: 22CSH-354/22ITH-354
Prepared by:
Er. Anupama Jamwal (E14665)

Parallel and Distributed Computing DISCOVER . LEARN . EMPOWER

1
Department of Computer Science and Engineering (CSE)

Content

•Introduction to Parallel and Distributed Computing


•Goals
•Issues

University Institute of Engineering (UIE) 2


Department of Computer Science and Engineering (CSE)

1 Introduction to Parallel and Distributed Computing


Parallel and Distributed Computing focuses on leveraging multiple computing units to
solve complex problems faster and more efficiently. In parallel computing, multiple
processors execute tasks simultaneously within a single system, often sharing memory.
Distributed computing, on the other hand, involves independent systems communicating
over a network to achieve a common goal. These approaches are foundational in modern
computing, enabling breakthroughs in big data, machine learning, and scientific
simulations. Both parallel and distributed computing have been around for a long time and
both have contributed greatly to the improvement of computing processes. However, they
have key differences in their primary function.

University Institute of Engineering (UIE) 3


Department of Computer Science and Engineering (CSE)

Parallel computing, also known as parallel processing, speeds up a computational


task by dividing it into smaller jobs across multiple processors inside one computer.
Parallel computing provides numerous advantages. Parallel computing helps to increase
the CPU utilization and improve the performance because several processors work
simultaneously. Moreover, the failure of one CPU has no impact on the other CPUs'
functionality. Furthermore, if one processor needs instructions from another, the CPU
might cause latency.

Examples
Artificial Intelligence and Machine Learning, Scientific Research and High-Performance
Computing, Financial Sectors, Energy and Environment sectors, Internet of Things,
Blockchain and Cryptocurrencies are the areas where distributed computing is used.

University Institute of Engineering (UIE) 4


Department of Computer Science and Engineering (CSE)

There are various advantages and disadvantages of parallel computing. Some of the
advantages and disadvantages are as follows:
Advantages
1. It saves time and money because many resources working together cut down on time
and costs.
2. It may be difficult to resolve larger problems on Serial Computing.
3. You can do many things at once using many computing resources.
4. Parallel computing is much better than serial computing for modeling, simulating, and
comprehending complicated real-world events.
Disadvantages
5. The multi-core architectures consume a lot of power.
Parallel solutions are more difficult to implement, debug, and prove right due to the
complexity of communication and coordination, and they frequently perform worse than
their serial equivalents

University Institute of Engineering (UIE) 5


Department of Computer Science and Engineering (CSE)

Distributed computing, on the other hand, uses a distributed system, such as the internet, to
increase the available computing power and enable larger, more complex tasks to be executed
across multiple machines. t comprises several software components that reside on different
systems but operate as a single system. A distributed system's computers can be physically close
together and linked by a local network or geographically distant and linked by a wide area
network (WAN). A distributed system can be made up of any number of different
configurations, such as mainframes, PCs, workstations, and minicomputers. The main aim of
distributed computing is to make a network work as a single computer.
Examples
Blockchains, Smartphones, Laptop computers, Internet of Things, Artificial intelligence and
machine learning, Space shuttle, Supercomputers are the technologies that uses Parallel
computing technology.

University Institute of Engineering (UIE) 6


Department of Computer Science and Engineering (CSE)

There are various advantages and disadvantages of distributed computing. Some of


the advantages and disadvantages are as follows:

Advantages
1. It is flexible, making it simple to install, use, and debug new services.
2. In distributed computing, you may add multiple machines as required.
3. If the system crashes on one server, that doesn't affect other servers.
4. A distributed computer system may combine the computational capacity of
several computers, making it faster than traditional systems.
Disadvantages
5. Data security and sharing are the main issues in distributed systems due to the
features of open systems
6. Because of the distribution across multiple servers, troubleshooting and
diagnostics are more challenging.
7. The main disadvantage of distributed computer systems is the lack of software
support.

University Institute of Engineering (UIE) 7


Department of Computer Science and Engineering (CSE)

Fig1: Parallel and Distributed Computing

University Institute of Engineering (UIE) 8


Department of Computer Science and Engineering (CSE)

Scope and Goals


The primary goal of parallel and distributed computing is to improve
computational speed, efficiency, and scalability. By dividing tasks across
multiple computing units, these systems aim to:
1. Reduce Computation Time: Enable faster execution for large-scale
problems.
2. Handle Large Datasets: Process massive amounts of data simultaneously.
3. Optimize Resource Utilization: Use available resources effectively across
distributed environments.
4. Enable Fault Tolerance: Ensure continuity even if some nodes fail.
Applications of these technologies include weather forecasting, simulations in
physics and biology, real-time analytics, cloud computing, and artificial
intelligence.

University Institute of Engineering (UIE) 9


Department of Computer Science and Engineering (CSE)

University Institute of Engineering (UIE) 10


Department of Computer Science and Engineering (CSE)

Differences Between Parallel and Distributed Computing:

Aspect Parallel Computing Distributed Computing

Shared memory among Independent memory


Memory
processors. systems.

Processors communicate
Communication occurs
Communication directly via shared
through a network.
memory.

High synchronization Limited synchronization;


Synchronization
between tasks. often asynchronous.

Designed for fault


Typically less fault-
Fault Tolerance tolerance and
tolerant.
redundancy.

Multi-core processors, Distributed databases,


Examples
GPUs. cloud platforms.

University Institute of Engineering (UIE) 11


Department of Computer Science and Engineering (CSE)

Issues and Challenges


1. Communication Overhead:
In distributed systems, nodes rely on a network for communication, which
introduces latency and can limit performance.
2. Synchronization:
Parallel systems require tight coordination, often causing bottlenecks if processes
wait for shared resources.
3. Fault Tolerance:
Failures in distributed systems are common due to network disruptions or node
crashes, requiring robust recovery mechanisms.
4. Scalability:
Scaling parallel systems often increases complexity, while distributed systems face
challenges in managing large networks.
5. Security:
Distributed systems are vulnerable to security threats such as data breaches and
unauthorized access.

University Institute of Engineering (UIE) 12

You might also like