CO - 1

COURSE NAME : PARALLEL & DISTRIBUTED COMPUTING

COURSE CODE : 22CS4106

TOPICS : TREE SUM PARALLEL ALGORITHM AND ITS APPLICATIONS

 WHAT IS PARALLEL ALGORITHM ?
• In computer science, a parallel algorithm, as opposed to a traditional
serial algorithm, is an algorithm which can do multiple operations in a
given time.
• It has been a tradition of computer science to describe serial
algorithms in abstract machine models, often the one known as
random-access machine.
• Similarly, many computer science researchers have used a so-called
parallel random-access machine (PRAM) as a parallel abstract machine
(shared-memory).

2
 WHAT IS PARALLEL ALGORITHM ?

• In general, a parallel algorithm can be defined as a set

of processes or tasks that may be executed
simultaneously and may communicate with each other
in order to solve a given problem.
• Parallel algorithms refer to a computational approach
where multiple solution points or populations of
solutions are utilized at each iteration of the algorithm.

3
 TREE SUM PARALLEL ALGORITHM
• A "tree sum parallel algorithm"
utilizes the concept of a parallel
"reduce" operation on a tree structure
to efficiently calculate the sum of all
node values in a tree by dividing the
computation across multiple
processors, typically by traversing the
tree in a bottom-up manner where
each node accumulates the sum of its
children's values, eventually reaching
the root with the total sum.

4
 TREE SUM PARALLEL ALGORITHM

• A tree sum parallel algorithm computes the sum of all

the elements stored in a tree data structure efficiently
using parallel processing.
• This algorithm divides the work across multiple
processors or threads, making it significantly faster than
sequential computation, particularly for large trees.

5
 KEY POINTS ABOUT TREE SUM PARALLEL ALGORITHM:
• Data structure:
• The input tree is usually represented as a binary tree for simplicity, but the concept can
be applied to any tree structure.
• Parallel paradigm:
• The algorithm employs a divide-and-conquer strategy, where each processor calculates
the sum of its subtree and then aggregates the results upwards towards the root.
• Up-sweep phase:
• This is the primary computation phase where nodes recursively calculate their partial
sums by adding the sums of their children.
• Down-sweep phase (optional):
• In some implementations, a down-sweep phase may be used to propagate the
calculated sum from the root back down to the leaves if necessary.

6
 KEY POINTS ABOUT TREE SUM PARALLEL ALGORITHM:

• Divide and Conquer Strategy:

• The algorithm breaks down the tree into smaller sub-trees by recursively
traversing the tree, with each processor calculating the sum of its assigned
subtree.
• Tree Traversal:
• The most common approach is to use a depth-first traversal, where each node
calculates the sum of its children's values and then adds its own value to that
sum.
• Parallel Reduction:
• Once the sums of all subtrees are calculated, a parallel reduction operation is
performed to combine these partial sums, eventually reaching the root node with
the final total sum.
7
 APPLICATIONS OF TREE SUM PARALLEL
• ImageALGORITHMS:
Processing:
• Calculating the sum of pixel intensities within a region of interest represented by a tree-like
structure.
• Graph Algorithms:
• Finding the total weight of a minimum spanning tree, where the tree structure represents the
connections between nodes.
• Data Analysis:
• Aggregating data from hierarchical structures like organizational charts or phylogenetic trees.
• Scientific Computing:
• Computing the total energy in a molecular simulation where the molecules are represented as
a tree structure.
• Network Analysis:
• Calculating the total traffic on a network where the network topology is represented as a tree.

8
 APPLICATIONS OF TREE SUM PARALLEL
• Data Aggregation:
ALGORITHMS:
• Used in hierarchical data structures like file systems or
organizational charts.
• E.g., summing the sizes of files in a directory tree.
• Scientific Computing:
• Frequently used in physics simulations, where spatial data is
represented as octrees or quadtrees.
• Machine Learning:
• Efficient processing of decision trees, random forests, or gradient-
boosted trees.

9
 APPLICATIONS OF TREE SUM PARALLEL
ALGORITHMS:
• Graphics and Rendering:
• In computer graphics, spatial partitioning trees (e.g., BSP trees,
quadtrees) are used to compute metrics like total illumination or
scene complexity.
• Database Query Optimization:
• When databases use hierarchical data structures like B-trees,
parallel algorithms speed up aggregation queries.
• Network Analysis:
• Summing up values in network hierarchies (e.g., summing traffic
or latency across routers).

10
 ALGORITHM
• Tree OVERVIEW
Representation:
• The tree can be a binary tree, general tree, or a
specific variant.
• Each node in the tree contains a value (e.g., an
integer).
• Parallel Strategy:
• The sum is computed recursively, where the work for
each subtree is distributed among parallel processors.
• Processors compute the sum for subtrees
independently and combine the results at their parent
nodes.
11
 ALGORITHM OVERVIEW
• Steps:
• Divide: Partition the tree into smaller subtrees at each
level.
• Compute in Parallel: Each processor computes the
sum of its assigned subtree.
• Combine: Combine the results from child nodes into the
parent node in parallel.
• Base Case:
• For a leaf node, the sum is simply the value of the node.
12
 PERFORMANCE
• The time complexity of a parallel tree sum algorithm
depends on the tree structure and the number of
processors.
• Assuming an ideal scenario (perfect load balancing and
no overhead), the time complexity is proportional to
the height of the tree: O(log⁡n), where n is the
number of nodes in a balanced binary tree.
• The space complexity includes the overhead of
managing parallel threads and the recursive stack.

13
 PREFIX SUM ARRAY – IMPLEMENTATION AND
APPLICATIONS IN COMPETITIVE PROGRAMMING

• Given an array arr[] of size N, find the prefix sum of the array. A prefix sum
array is another array prefixSum[] of the same size, such that the value
of prefixSum[i] is arr[0] + arr[1] + arr[2] . . . arr[i].

14
 PARALLEL ALGORITHM MODELS

• The need for a parallel algorithm model arises in order to

understand the strategy that is used for the partitioning of
data and the ways in which these data are being processed.
Therefore every model being used provides proper
structuring based on two techniques. They are as follows:

• Selection of proper partitioning and mapping techniques.

• Proper use of strategy in order to reduce the interaction.

15
 TYPES OF PARALLEL MODELS

• The Data-Parallel Model

• The data-parallel model algorithm is one of the simplest models of all other
parallel algorithm models.
• In this model, the tasks that need to be carried out are identified first and then
mapped to the processes.
• This mapping of tasks onto the processes is being done statically or semi-
statically.
• In this model, the task that is being performed by every process is the same or
identical but the data on which these operations or tasks are performed is
different.

16
 TYPES OF PARALLEL MODELS
• The Task Graph Model
• The task dependency graph is being used by the parallel algorithms for
describing the computations it performs. Therefore, the use of
interrelationships among the tasks in the task dependency graph can be
used for reducing the interaction costs.
• This model can be used effectively for solving problems in which tasks are
associated with a large amount of data as compared to that actual
computation.
• The parallelism that is described with the task dependency graph where
each task is an independent task is known as task parallelism.

17
Dense Matrix Finding the minimum
Multiplication number

18
 TYPES OF PARALLEL MODELS
• Work Pool Model​
• The work pool model is also
known as the task pool model.
This model makes use of a
dynamic mapping approach for
task assignment in order to
handle load balancing. The size
of some processes or tasks is
small and requires less time.

19
 TYPES OF PARALLEL MODELS

• Master-Slave Model
• Master Slave Model is also known as
Manager- worker model. The work is
being divided among the process. In
this model, there are two different
types of processes namely master
process and slave process. One or
more process acts as a master and
the remaining all other process acts
as a slave.

20
 TYPES OF PARALLEL MODELS
• The Pipeline Model
• The Pipeline Model is also known as the Producer-Consumer
model. This model is based on the passing of a data stream
through the processes that are arranged in succession.
• Hybrid Model
• A hybrid model is the combination of more than one parallel
model. This combination can be applied sequentially or
hierarchically to the different phases of the parallel algorithm.

21
Parallel LU factorization A combination of master-
algorithm slave, work pool, and data
graph model.

22
 THANK YOU

Team – Parallel & Distributed Computing

23