Scribd
Upload
18 views28 pages

Chapter1 - CLO1

Parallel computing involves solving computational problems simultaneously using multiple processors or computers. It allows problems to be broken into discrete parts that can be solved concurrently, with instructions from each part executing simultaneously on different processors. The main reasons to use parallel computing are to save time and money, solve larger problems, provide concurrency, take advantage of non-local resources, and better utilize underlying parallel hardware. Parallel computing is used widely in science, engineering, industry, commerce, and for global applications such as modeling complex real-world phenomena.

Uploaded by

melvyn
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
18 views28 pages

Chapter1 - CLO1

Parallel computing involves solving computational problems simultaneously using multiple processors or computers. It allows problems to be broken into discrete parts that can be solved concurrently, with instructions from each part executing simultaneously on different processors. The main reasons to use parallel computing are to save time and money, solve larger problems, provide concurrency, take advantage of non-local resources, and better utilize underlying parallel hardware. Parallel computing is used widely in science, engineering, industry, commerce, and for global applications such as modeling complex real-world phenomena.

Uploaded by

melvyn
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 28

Chapter1 - CLO1

Introduction
Chapter Overview: Introduction

• Introduction to Parallel – What is Parallel Computing


• Motivation – Why use Parallel Computing
• Scope and application - Who is Using Parallel Computing?
What is Parallel Computing?

• Serial Computing:
Traditionally, software has been written for serial computation:
– A problem is broken into a discrete series of instructions
– Instructions are executed sequentially one after another
– Executed on a single processor
– Only one instruction may execute at any moment in time
What is Parallel Computing?

• exmple
What is Parallel Computing?

• Parallel Computing:
In the simplest sense, parallel computing is the simultaneous use of
multiple compute resources to solve a computational problem:
– A problem is broken into discrete parts that can be solved concurrently
– Each part is further broken down to a series of instructions
– Instructions from each part execute simultaneously on different
processors
– An overall control/coordination mechanism is employed
What is Parallel Computing?
What is Parallel Computing?
What is Parallel Computing?

• The computational problem should be able to:


– Be broken apart into discrete pieces of work that can be solved
simultaneously;
– Execute multiple program instructions at any moment in time;
– Be solved in less time with multiple compute resources than with a
single compute resource.
• The compute resources are typically:
– A single computer with multiple processors/cores
– An arbitrary number of such computers connected by a network
What is Parallel Computing?

• Parallel Computers:
Virtually all stand-alone computers today are parallel from a hardware
perspective:
– Multiple functional units (L1 cache, L2 cache, branch, prefetch, decode,
floating-point, graphics processing (GPU), integer, etc.)
– Multiple execution units/cores
– Multiple hardware threads
• IBM BG/Q Compute Chip with 18 cores (PU) and 16 L2 Cache units
(L2
What is Parallel Computing?

• Networks connect multiple stand-alone computers (nodes) to make


larger parallel computer clusters.
What is Parallel Computing?

For example, the schematic below shows a typical LLNL parallel


computer cluster:
– Each compute node is a multi-processor parallel computer in itself
– Multiple compute nodes are networked together with an Infiniband
network
– Special purpose nodes, also multi-processor, are used for other
purposes
What is Parallel Computing?
The majority of the world's large parallel computers
(supercomputers) are clusters of hardware produced by a handful
of (mostly) well known vendors.
Why Use Parallel Computing?

• The Real World is Massively Parallel:


– In the natural world, many complex, interrelated events are happening
at the same time, yet within a temporal sequence.
– Compared to serial computing, parallel computing is much better suited
for modeling, simulating and understanding complex, real world
phenomena.
– For example, imagine modeling these serially
Why Use Parallel Computing?
Why Use Parallel Computing?

• Main Reasons:

– SAVE TIME AND/OR MONEY:


– In theory, throwing more resources at a task will shorten its time to
completion, with potential cost savings.
– Parallel computers can be built from cheap, commodity components.
• SOLVE LARGER / MORE COMPLEX PROBLEMS:
– Many problems are so large and/or complex that it is impractical or
impossible to solve them on a single computer, especially given limited
computer memory.
– Example: "Grand Challenge Problems"
(en.wikipedia.org/wiki/Grand_Challenge) requiring PetaFLOPS and
PetaBytes of computing resources.
– Example: Web search engines/databases processing millions of
transactions every second
Why Use Parallel Computing?

• PROVIDE CONCURRENCY:
– A single compute resource can only do one thing at a time. Multiple
compute resources can do many things simultaneously.
– Example: Collaborative Networks provide a global venue where people
from around the world can meet and conduct work "virtually".
Why Use Parallel Computing?

• TAKE ADVANTAGE OF NON-LOCAL RESOURCES:


– Using compute resources on a wide area network, or even the Internet
when local compute resources are scarce or insufficient. Two examples
below, each of which has over 1.7 million contributors globally (May
2018):
– Example: SETI@home (setiathome.berkeley.edu)
– Example: Folding@home (folding.stanford.edu)
Why Use Parallel Computing?

• MAKE BETTER USE OF UNDERLYING PARALLEL HARDWARE:


– Modern computers, even laptops, are parallel in architecture with
multiple processors/cores.
– Parallel software is specifically intended for parallel hardware with
multiple cores, threads, etc.
– In most cases, serial programs run on modern computers "waste"
potential computing power.
Why Use Parallel Computing?

• The Future:
– During the past 20+ years, the trends indicated by ever faster networks,
distributed systems, and multi-processor computer architectures (even
at the desktop level) clearly show that parallelism is the future of
computing.
– In this same time period, there has been a greater than 500,000x
increase in supercomputer performance, with no end currently in sight.
– The race is already on for Exascale Computing!
– Exaflop = 1018 calculations per second
Who is Using Parallel Computing?

• Science and Engineering:


– Historically, parallel computing has been considered to be "the high end
of computing", and has been used to model difficult problems in many
areas of science and engineering:
– Atmosphere, Earth, Environment
– Physics - applied, nuclear, particle, condensed matter, high pressure,
fusion, photonics
– Bioscience, Biotechnology, Genetics
– Chemistry, Molecular Sciences
– Geology, Seismology
– Mechanical Engineering - from prosthetics to spacecraft
– Electrical Engineering, Circuit Design, Microelectronics
– Computer Science, Mathematics
– Defense, Weapons
Who is Using Parallel Computing?
Who is Using Parallel Computing?

• Industrial and Commercial:


– Today, commercial applications provide an equal or greater driving
force in the development of faster computers. These applications
require the processing of large amounts of data in sophisticated ways.
For example:
– "Big Data", databases, data mining
– Artificial Intelligence (AI)
– Web search engines, web based business services
– Medical imaging and diagnosis
– Pharmaceutical design
– Financial and economic modeling
– Management of national and multi-national corporations
– Advanced graphics and virtual reality, particularly in the entertainment
industry
– Networked video and multi-media technologies
Why Use Parallel Computing?
Who is Using Parallel Computing?

• Global Applications:
– Parallel computing is now being used extensively around the world, in a
wide variety of applications.

You might also like