Department of Computer Science and Engineering (RA)

COURSE NAME: PARALLEL &DISTRIBUTED

COMPUTING
COURSE CODE: 22CS4106 R

Sorting on Parallel Computers: Sorting

Networks, Bitonic Sort, Merge sort
 SESSION OUTLINE
Understand the importance of Bitonic Merge sort in distributed systems
.
Explore the challenges of Bitonic Merge sort in Distributed systems.

Learn how the Bitonic Merge sort works and its execution.

Analyze practical use cases and real-world scenarios where Bitonic

Merge sort are applied.

Identify limitations and explore extensions of Bitonic Merge sort

techniques

2
 SESSION DESCRIPTION

• The bitonic sort algorithm is similar to quicksort, but it uses the

bitonic merge operation instead of the traditional quicksort
partitioning algorithm. The bitonic sort algorithm can be used to sort
an array of elements in any order (ascending or descending).
• The basic idea behind the bitonic sort algorithm is to divide the array
into two halves, sort each half independently, and then use the bitonic
merge operation to combine the two sorted halves into a single sorted
array.
• The number of threads used will depend on the size of the array being
sorted. The time complexity of the algorithms worst case scenario is
O(nlogn)

3
• Bitonic mergesort works by first dividing the array into two halves,
sorting each half, and then merging the two halves back together.
• The key to making this algorithm work efficiently is to use a bitonic
sequence when sorting the halves of the array.
• A bitonic sequence is an array where each element is either greater than
or less than all of the elements before it.
• By using a bitonic sequence, merge the two halves back together, all of
the elements will be in sorted order.
• This makes bitonic mergesort a very efficient way to sort an array

4
 BITONIC SORT
• The number of comparisons done by Bitonic sort is more than popular sorting
algorithms like Merge Sort [ does O(log N) comparisons], but Bitonic sort is
better for parallel implementation because we always compare elements in a
predefined sequence and the sequence of comparison doesn’t depend on data.
Therefore it is suitable for implementation in hardware and parallel processor
array.
• Bitonic Sort can only be done if the number of elements to sort is 2^n. The
procedure of bitonic sequence fails if the number of elements is not in the
aforementioned quantity precisely.

5
6
7
8
9
The bitonic merge sort algorithm is a recursive divide-and-conquer
algorithm that sorts an input array of numbers by first dividing it into
two halves, sorting each half using a bitonic sort, and then merging the
two sorted halves together using a bitonic merge operation.
The bitonic merge operation takes two sorted arrays (a and b) and
produces a single sorted array. The size of the arrays must be a power of
two (i.e., N = 2^k for some integer k). The bionic merge operation works
as follows:
1. Compare each element in array a with the corresponding element in
array b.
2. If the element in array a is less than or equal to the element in array b,
exchange them (i.e., swap their values).
3. Repeat step 2 for all remaining elements in arrays a and b
4. Now compare every other pair of elements in array

10
11
The bitonic merge sort algorithm begins by dividing the input
array into two subarrays, A and B. A is the left subarray and B is
the right subarray. If the size of the input array is not a power of
2, then the extra element will be added to A. Next, both
subarrays are sorted in a bitonic fashion. To do this, each
element in A compares itself with every other element in A and
exchanges values if it is larger than the other element. Similarly
for each element in B, it compares itself with every other
element in B and exchanges values if it is larger than the other
element. This process continues until all elements in A and B are
sorted in a bitonic fashion.

12
• A bitonic sorting network sorts n elements in Θ(log2n) time.
• A bitonic sequence has two tones - increasing and decreasing, or
vice versa. Any cyclic rotation of such networks is also considered
bitonic.
• 1,2,4,7,6,0 is a bitonic sequence, because it first increases and
then decreases. 8,9,2,1,0,4 is another bitonic sequence, because it
is a cyclic shift of 0,4,8,9,2,1.
• The kernel of the network is the rearrangement of a bitonic
sequence into a sorted sequence.

13
Let s = a0,a1,…,an-1 be a bitonic sequence such that a0 ≤ a1 ≤ ··· ≤ an/2-1 and an/2
≥ an/2+1 ≥ ··· ≥ an-1.

Consider the following subsequences of s:

s1 = min{a0,an/2},min{a1,an/2+1},…,min{an/2-1,an-1}
s2 = max{a0,an/2},max{a1,an/2+1},…,max{an/2-1,an-1}
(1)

Note that s1 and s2 are both bitonic and each element of s1 is less than every
element in s2.
We can apply the procedure recursively on s1 and s2 to get the sorted sequence.

14
Merging a 16-element bitonic sequence through a series
of log 16 bitonic splits.

15
• We can easily build a sorting network to implement this
bitonic merge algorithm.
• Such a network is called a bitonic merging network.
• The network contains log n columns. Each column
contains n/2 comparators and performs one step of the
bitonic merge.
• We denote a bitonic merging network with n inputs by
BM[n].
• Replacing the  comparators by Ө comparators results in
a decreasing output sequence; such a network is denoted
by ӨBM[n].

16
A bitonic merging network for n = 16. The input wires are numbered 0,1,…, n - 1, and the
binary representation of these numbers is shown. Each column of comparators is drawn
separately; the entire figure represents a BM[16] bitonic merging network. The network
takes a bitonic sequence and outputs it in sorted order.

17
 How do we sort an unsorted sequence using a bitonic
merge?

•We must first build a single bitonic sequence from the

given sequence.
•A sequence of length 2 is a bitonic sequence.
•A bitonic sequence of length 4 can be built by sorting the
first two elements using BM[2] and next two, using
ӨBM[2].
•This process can be repeated to generate larger bitonic
sequences

18
A schematic representation of a network that converts an input sequence into a bitonic sequence. In this
example, BM[k] and ӨBM[k] denote bitonic merging networks of input size k that use  and Ө
comparators, respectively. The last merging network (BM[16]) sorts the input. In this example, n = 16.

19
The comparator network that transforms an input sequence of 16 unordered numbers
into a bitonic sequence.

20
The depth of the network is Θ(log2 n).
Each stage of the network contains n/2 comparators. A
serial implementation of the network would have
complexity Θ(nlog2 n).

21
Bitonic sort is frequently used in:
•GPU-based Sorting: Graphics processing units can execute parallel tasks
efficiently.
•Distributed Systems: Where data is spread across multiple processors.
•Sorting Networks: Fixed networks with predetermined data movement
patterns
There are some disadvantages to bitonic merge sort as well. First, the
algorithm can be complex to implement. Second, it is not as effective at
sorting data that is not already partially sorted. However, these disadvantages
are outweighed by the advantages in most cases.

22
 SELF-ASSESSMENT QUESTIONS

Which architecture is best suited for Bitonic Sort?

. A) Single-core processor
B) Multi-core processor with SIMD capabilities
C) Stack-based machine
D) Virtual memory-based systems

What is the main disadvantage of Bitonic Sort?

A) It is non-deterministic.
B) It requires excessive memory.
C) It is inefficient for large datasets on sequential systems.
D) It cannot handle floating-point numbers.
What is the primary operation performed during the
merge phase of Bitonic Sort?
A) Merging sorted subarrays.
B) Swapping elements to satisfy bitonic conditions.
C) Dividing the sequence into smaller chunks.
D) Counting the elements in each subarray.

23
 REFERENCES
REFERENCES FOR
FOR FURTHER
FURTHER LEARNING
LEARNING OF
OF THE
THE
SESSION
SESSION

Reference Books:
. Attiya, Hagit, and Jennifer Welch. Distributed computing: fundamentals,
simulations, and
advanced topics. Vol. 19. John Wiley & Sons, 2004.
2. Ananth Grama, Vipin Kumar,”Introduction to Parallel Computing”, 2nd Edition,
Addison Wesley.
3. Tanenbaum, Andrew S Steen, Maarten van -Distributed systems: principles and
paradigms. Pearson, 4th Edition.
4. Coulouris GF, Dollimore J,
Kindberg T. Distributed systems: concepts and design. pearson education; 2005.
5. A.D. Kshemkalyani, M.
Singhal, Distributed Computing: Principles, Algorithms, and Systems, Cambridge
University Press, 2011
6.Peter_Pacheco,_Matthew_Malensek_An_Introduction_to_Parallel_Programming,
Elsevier Second Edition,2020

24
 THANK YOU

Team – Course Name

25