Complexity of conventional Sorting

Algorithms
Course Code: CSC2211 Course Title: Algorithms

Dept. of Computer Science

Faculty of Science and Technology

Lecturer No: 4.1 Week No: 04 Semester: Spring 22-23

Lecturer: Pritam Khan Boni; pritam.khan@aiub.edu
Lecture Outline

1. Sorting Algorithms
 Quick Sort
Quick Sort
• Quick sort is based on the divide-and-conquer approach.

• The idea is based on of choosing one element as a pivot

element and partitioning the array around it such that:
 Left side of pivot contains all the elements that are less
than the pivot element.
 Right side contains all elements greater than the pivot.

• It reduces the space complexity and removes the use of the

auxiliary array that is used in merge sort.
• Selecting a random pivot in an array results in an improved
time complexity in most of the cases.
Quick Sort Algorithm
 5

Quicksort
 A divide & conquer algorithm
 Efficient sorting algorithm
 Discovered by C.A.R. Hoare

 Basic idea:
 Pick an element (called pivot)
 Partition the array in two subsequences: those smaller than
the pivot and those larger or equal: Divide
 Sort each subsequence recursively: Conquer
 6

Quicksort: Partitioning an array

 Goal
 Move all elements smaller than or equal to the pivot to
the left of the array and all elements larger than or equal
to the pivot to the right of the array. In the end, the right
part of the array will contain elements that are larger than
or equal to those in the left part.
 Choosing the pivot is crucial

< pivot pivot > pivot

 7

Quicksort
 Place the pivot in its appropriate position

 Recursively the same algorithm is applied to each

part
 Until there is only single element i.e., no way to divide
in two part
< pivot > pivot

< pivot1 pivot1 > pivot1 pivot < pivot2 pivot2 > pivot2
 8

Quicksort: Code
void quicksort( int *a, int low, int high )
{
int pivot;
/* Termination condition! */
if ( high > low )
{
pivot = partition( a, low, high );
quicksort( a, low, pivot-1 );
quicksort( a, pivot+1, high );
}
int partition( int *a, int low, int high )
{
int left, right, pivot_item;
pivot_item = a[low];
pivot = left = low;
right = high;
while ( left < right )
{
/* Move left while item < pivot */
while( a[left] <= pivot_item )
left++;
/* Move right while item > pivot */
while( a[right] >= pivot_item )
right--;
if ( left < right )
SWAP(a[left],a[right]);
}
/* Right is final position for the pivot */
a[low] = a[right];
a[right] = pivot_item;
return right;
}
 10

Quicksort: Way of Execution

pivot_item = a[low];

Any item will do as the

pivot,
choose the leftmost one!

23 12 15 38 42 18 36 29 27

low high
 11

Quicksort: Way of Execution

pivot = left = low;
right = high;
Set left and right
markers

left right

23 12 15 38 42 18 36 29 27

low pivot: 23 high

 12

Quicksort: Way of Execution

Move the markers
while ( left <= right ) { until they cross
over
left++;

right--;

left right
23 12 15 38 42 18 36 29 27

low pivot: 23 high

 13

Quicksort: Way of Execution

Move the left pointer
while
it points
while( a[left] <= pivot_item ) to items <= pivot
left++;

left

23 12 15 38 42 18 36 29 27

low pivot: 23 high

 14

Quicksort: Way of Execution

Move the left pointer
while
it points
while( a[left] <= pivot_item ) to items <= pivot
left++;

left

23 12 15 38 42 18 36 29 27

low pivot: 23 high

 15

Quicksort: Way of Execution

Move the left pointer
while
it points
while( a[left] <= pivot_item ) to items <= pivot
left++;

left

23 12 15 38 42 18 36 29 27

low pivot: 23 high

 16

Quicksort: Way of Execution

Move the left pointer
while
it points
while( a[left] <= pivot_item ) to items <= pivot
left++;

left

23 12 15 38 42 18 36 29 27

low pivot: 23 high

 17

Quicksort: Way of Execution

Move the right pointer
while
it points
while( a[right] >= pivot_item )
to items >= pivot
right--;

left right

23 12 15 38 42 18 36 29 27

low pivot: 23 high

 18

Quicksort: Way of Execution

Move the right pointer
while
it points
while( a[right] >= pivot_item )
to items >= pivot
right--;

left right

23 12 15 38 42 18 36 29 27

low pivot: 23 high

 19

Quicksort: Way of Execution

Move the right pointer
while
it points
while( a[right] >= pivot_item )
to items >= pivot
right--;

left right

23 12 15 38 42 18 36 29 27

low pivot: 23 high

 20

Quicksort: Way of Execution

Move the right pointer
while
it points
while( a[right] >= pivot_item )
to items >= pivot
right--;

left right

23 12 15 38 42 18 36 29 27

low pivot: 23 high

 21

Quicksort: Way of Execution

Move the left pointer
while
it points
while( a[left] <= pivot_item ) to items <= pivot
left++;

while( a[right] >= pivot_item ) right--;

left right Move right

similarly
23 12 15 38 42 18 36 29 27

low pivot: 23 high

 22

Quicksort: Way of Execution

Swap the two items
on the wrong side of the
pivot
if ( left < right )
SWAP(a[left], a[right]);

left right

23 12 15 18 42 38 36 29 27
pivot: 23
low high
 23

Quicksort: Way of Execution

Swap the two items
on the wrong side of the
pivot
if ( left < right )
SWAP(a[left], a[right]);

left right

23 12 15 18 42 38 36 29 27
pivot: 23
low high
 24

Quicksort: Way of Execution

while ( left < right ) {

while( a[left] <= pivot_item ) left++;

left right

23 12 15 18 42 38 36 29 27

low high
pivot: 23
 25

Quicksort: Way of Execution

while ( left < right ) {

while( a[left] <= pivot_item ) left++;

left
right

23 12 15 18 42 38 36 29 27

low high
pivot: 23
 26

Quicksort: Way of Execution

while ( left < right ) {

while( a[right] >= pivot_item ) right--;

left
right

23 12 15 18 42 38 36 29 27

low high
pivot: 23
 27

Quicksort: Way of Execution

while ( left < right ) {

while( a[right] >= pivot_item ) right--;

left
right

23 12 15 18 42 38 36 29 27

low high
pivot: 23
 28

Quicksort: Way of Execution

right left

23 12 15 18 42 38 36 29 27

low pivot: 23 high

a[low] = a[right];
a[right] = pivot_item; Finally, swap the
pivot
and right
 29

Quicksort: Way of Execution

right
pivot: 23
18 12 15 23 42 38 36 29 27

low high

Return the
return right; position
of the pivot
 30

Quicksort: Way of Execution

pivo
t
pivot: 23
18 12 15 23 42 38 36 29 27

Recursively Recursively
sort left half sort right half

Conquer
Quicksort
23 12 15 38 42 18 36 29 27

18 12 15 42 38 36 29 27
23

38 36 29 27 42
12 15 18
36 29 27 38
12 15
29 27 36

27 29

12 15 18 23 27 29 36 38 42
 Quick Sort Analysis
Substitution Method
 33

Quicksort: Analysis

 Running time: depends on selection of pivot

 Best case (array partitioned in half every time)

 O(n log n)
int partition( int *a, int low, int high )
{
c1 + c2* (n/2) + c2*
int left, right, pivot_item; (n/2)
pivot_item = a[low]; =c2( n/2 + n/2)
pivot = left = low;
right = high; =c2 * n
while ( left < right )
{
/* Move left while item < pivot */
while( a[left] <= pivot_item )
left++;
/* Move right while item > pivot */
while( a[right] >= pivot_item )
right--;
if ( left < right )
SWAP(a[left],a[right]);
}
/* Right is final position for the pivot */
a[low] = a[right];
a[right] = pivot_item;
return right;
}
 Best Case T(n), to sort
n elements
void quicksort( void *a, int low, int high ) c1
{
int pivot;
/* Termination condition! */
if ( high > low )
{

pivot = partition( a, low, high ); c2 * n

quicksort( a, low, pivot-1 ); T(n/2)

quicksort( a, pivot+1, high ); T(n/2)
}
}

So, T(n) = O(1) when n = 1, and

2T(n/2) + c1+ c2*n when n > 1
So

Skipping the insignificant parts,

[]
39
 40

Quicksort: Analysis

 Worst case:
 Partitioning n elements results in two subsequences of
lengths 1 and n-1
What happens if we use quicksort on
data that’s already sorted (or nearly sorted)?
 Now Take a look
 41

Quicksort: Analysis
 Sorted Data
pivot

? 1 2 3 4 5 6 7 8 9

< pivot > pivot

 42

Quicksort: Analysis (Sorted Data)

 Each partition pivot
produces
 a problem of size 0 1 2 3 4 5 6 7 8 9

 and one of size n-1! > pivot

pivot

2 3 4 5 6 7 8 9

> pivot
 Worst Case T(n), to sort
n elements
void quicksort( void *a, int low, int high ) c1
{
int pivot;
/* Termination condition! */
if ( high > low )
{

pivot = partition( a, low, high ); c2 * n

quicksort( a, low, pivot-1 ); T(1) = c3

quicksort( a, pivot+1, high ); T(n-1)
}
}

So, T(n) = O(1) when n = 1, and

T(n-1) + c1+ c2*n + c3 when n > 1
 n–k=1
So k=n-1

0, 1, 3, 6
= 0, 1, (1+2), (1+2+3)
Kth term
1+2+3+....+(k-1) =
45
 46

Quicksort: Improve performance

 Quicksort’s O(n log n) behavior
 Depends on the partitions being nearly equal
 there are O( log n ) of them

 On average, this will nearly be the case

and quicksort is generally O(n log n)

 Can we do anything to ensure O(n log n) time?

 In general, no
 But we can improve our chances!!
 47

Quicksort: How to select pivot?

 Any pivot will work …

 But choose a different pivot …

 So that the partitions are equal
 Then we will see O(n log n) time

pivot

1 2 3 4 5 6 7 8 9

< pivot > pivot

 48

Quicksort: How to select pivot?

Median-of-3 pivot

 Take 3 positions and choose the median

 Say … First, middle, last

1 2 3 4 5 6 7 8 9

 Median is 5
 Perfect division of sorted data every time!
 O(n log n) time

 Since sorted (or nearly sorted) data is

common,
median-of-3 is a good strategy
 especially if you think your data may be sorted!
 49

Quicksort: How to select pivot?

Randomly selected pivot

 Choose a pivot randomly

 Different position for every partition
 On average, sorted data is divided evenly
 O(n log n) time

 Key requirement
• Pivot choice must take O(1) time

Ashiq, MIST
 50

Quicksort: Guaranteed O (nlogn)

 Never!!
 Any pivot selection strategy
could lead to O(n2) time

1 4 9 6 2 5 7 8 3
 Here median-of-3 chooses 2
 One partition of size 1 and 1 2 4 9 6 5 7 8 3
 One partition of size 7

 Next it chooses 4
 One partition of size 1 and
 One partition of size 5
 51

Key points

 Quicksort
 O(n log n) but ….
 Can be O(n2)

 Depends on pivot selection

 Median-of-3
 Random pivot
 Better but not guaranteed
Books

 Introduction to Algorithms, Thomas H. Cormen, Charle E.

Leiserson, Ronald L. Rivest, Clifford Stein (CLRS).

 Fundamental of Computer Algorithms, Ellis Horowitz, Sartaj

Sahni, Sanguthevar Rajasekaran (HSR)
References

 https://www.google.com/search?q=bubble+sort+step+by+step&sxsrf=AL
eKk01uxzgfT3Oy6k1Q3WxVnSpiIN8_4g:1587999728942&tbm=isch&sourc
e=iu&ictx=1&fir=vRwFsGwVfJ6pJM%253A%252CSzhhze6MPQr4cM%252C
_&vet=1&usg=AI4_-kSrEEXqwRL-PkHhVUtn7jNfF9dB6g&sa=X&ved=2ahU
KEwje0Pz974jpAhXRAnIKHWhMD2UQ_h0wAXoECAcQBg#imgrc=EN4Sdu7
veOWVoM&imgdii=eOqvCu85p9-eBM

 https://www.interviewcake.com/concept/java/counting-sort

 https://www.geeksforgeeks.org/counting-sort/

 https://www.hackerearth.com/practice/algorithms/sorting/quick-sort/tut
orial/