Complexity of conventional Sorting
Algorithms
Course Code: CSC2211 Course Title: Algorithms
Dept. of Computer Science
Faculty of Science and Technology
Lecturer No: 4.1 Week No: 04 Semester: Spring 22-23
Lecturer: Pritam Khan Boni;
[email protected]Lecture Outline
1. Sorting Algorithms
Quick Sort
Quick Sort
• Quick sort is based on the divide-and-conquer approach.
• The idea is based on of choosing one element as a pivot
element and partitioning the array around it such that:
Left side of pivot contains all the elements that are less
than the pivot element.
Right side contains all elements greater than the pivot.
• It reduces the space complexity and removes the use of the
auxiliary array that is used in merge sort.
• Selecting a random pivot in an array results in an improved
time complexity in most of the cases.
Quick Sort Algorithm
5
Quicksort
A divide & conquer algorithm
Efficient sorting algorithm
Discovered by C.A.R. Hoare
Basic idea:
Pick an element (called pivot)
Partition the array in two subsequences: those smaller than
the pivot and those larger or equal: Divide
Sort each subsequence recursively: Conquer
6
Quicksort: Partitioning an array
Goal
Move all elements smaller than or equal to the pivot to
the left of the array and all elements larger than or equal
to the pivot to the right of the array. In the end, the right
part of the array will contain elements that are larger than
or equal to those in the left part.
Choosing the pivot is crucial
< pivot pivot > pivot
7
Quicksort
Place the pivot in its appropriate position
Recursively the same algorithm is applied to each
part
Until there is only single element i.e., no way to divide
in two part
< pivot > pivot
< pivot1 pivot1 > pivot1 pivot < pivot2 pivot2 > pivot2
8
Quicksort: Code
void quicksort( int *a, int low, int high )
{
int pivot;
/* Termination condition! */
if ( high > low )
{
pivot = partition( a, low, high );
quicksort( a, low, pivot-1 );
quicksort( a, pivot+1, high );
}
int partition( int *a, int low, int high )
{
int left, right, pivot_item;
pivot_item = a[low];
pivot = left = low;
right = high;
while ( left < right )
{
/* Move left while item < pivot */
while( a[left] <= pivot_item )
left++;
/* Move right while item > pivot */
while( a[right] >= pivot_item )
right--;
if ( left < right )
SWAP(a[left],a[right]);
}
/* Right is final position for the pivot */
a[low] = a[right];
a[right] = pivot_item;
return right;
}
10
Quicksort: Way of Execution
pivot_item = a[low];
Any item will do as the
pivot,
choose the leftmost one!
23 12 15 38 42 18 36 29 27
low high
11
Quicksort: Way of Execution
pivot = left = low;
right = high;
Set left and right
markers
left right
23 12 15 38 42 18 36 29 27
low pivot: 23 high
12
Quicksort: Way of Execution
Move the markers
while ( left <= right ) { until they cross
over
left++;
right--;
left right
23 12 15 38 42 18 36 29 27
low pivot: 23 high
13
Quicksort: Way of Execution
Move the left pointer
while
it points
while( a[left] <= pivot_item ) to items <= pivot
left++;
left
23 12 15 38 42 18 36 29 27
low pivot: 23 high
14
Quicksort: Way of Execution
Move the left pointer
while
it points
while( a[left] <= pivot_item ) to items <= pivot
left++;
left
23 12 15 38 42 18 36 29 27
low pivot: 23 high
15
Quicksort: Way of Execution
Move the left pointer
while
it points
while( a[left] <= pivot_item ) to items <= pivot
left++;
left
23 12 15 38 42 18 36 29 27
low pivot: 23 high
16
Quicksort: Way of Execution
Move the left pointer
while
it points
while( a[left] <= pivot_item ) to items <= pivot
left++;
left
23 12 15 38 42 18 36 29 27
low pivot: 23 high
17
Quicksort: Way of Execution
Move the right pointer
while
it points
while( a[right] >= pivot_item )
to items >= pivot
right--;
left right
23 12 15 38 42 18 36 29 27
low pivot: 23 high
18
Quicksort: Way of Execution
Move the right pointer
while
it points
while( a[right] >= pivot_item )
to items >= pivot
right--;
left right
23 12 15 38 42 18 36 29 27
low pivot: 23 high
19
Quicksort: Way of Execution
Move the right pointer
while
it points
while( a[right] >= pivot_item )
to items >= pivot
right--;
left right
23 12 15 38 42 18 36 29 27
low pivot: 23 high
20
Quicksort: Way of Execution
Move the right pointer
while
it points
while( a[right] >= pivot_item )
to items >= pivot
right--;
left right
23 12 15 38 42 18 36 29 27
low pivot: 23 high
21
Quicksort: Way of Execution
Move the left pointer
while
it points
while( a[left] <= pivot_item ) to items <= pivot
left++;
while( a[right] >= pivot_item ) right--;
left right Move right
similarly
23 12 15 38 42 18 36 29 27
low pivot: 23 high
22
Quicksort: Way of Execution
Swap the two items
on the wrong side of the
pivot
if ( left < right )
SWAP(a[left], a[right]);
left right
23 12 15 18 42 38 36 29 27
pivot: 23
low high
23
Quicksort: Way of Execution
Swap the two items
on the wrong side of the
pivot
if ( left < right )
SWAP(a[left], a[right]);
left right
23 12 15 18 42 38 36 29 27
pivot: 23
low high
24
Quicksort: Way of Execution
while ( left < right ) {
while( a[left] <= pivot_item ) left++;
left right
23 12 15 18 42 38 36 29 27
low high
pivot: 23
25
Quicksort: Way of Execution
while ( left < right ) {
while( a[left] <= pivot_item ) left++;
left
right
23 12 15 18 42 38 36 29 27
low high
pivot: 23
26
Quicksort: Way of Execution
while ( left < right ) {
while( a[right] >= pivot_item ) right--;
left
right
23 12 15 18 42 38 36 29 27
low high
pivot: 23
27
Quicksort: Way of Execution
while ( left < right ) {
while( a[right] >= pivot_item ) right--;
left
right
23 12 15 18 42 38 36 29 27
low high
pivot: 23
28
Quicksort: Way of Execution
right left
23 12 15 18 42 38 36 29 27
low pivot: 23 high
a[low] = a[right];
a[right] = pivot_item; Finally, swap the
pivot
and right
29
Quicksort: Way of Execution
right
pivot: 23
18 12 15 23 42 38 36 29 27
low high
Return the
return right; position
of the pivot
30
Quicksort: Way of Execution
pivo
t
pivot: 23
18 12 15 23 42 38 36 29 27
Recursively Recursively
sort left half sort right half
Conquer
Quicksort
23 12 15 38 42 18 36 29 27
18 12 15 42 38 36 29 27
23
38 36 29 27 42
12 15 18
36 29 27 38
12 15
29 27 36
27 29
12 15 18 23 27 29 36 38 42
Quick Sort Analysis
Substitution Method
33
Quicksort: Analysis
Running time: depends on selection of pivot
Best case (array partitioned in half every time)
O(n log n)
int partition( int *a, int low, int high )
{
c1 + c2* (n/2) + c2*
int left, right, pivot_item; (n/2)
pivot_item = a[low]; =c2( n/2 + n/2)
pivot = left = low;
right = high; =c2 * n
while ( left < right )
{
/* Move left while item < pivot */
while( a[left] <= pivot_item )
left++;
/* Move right while item > pivot */
while( a[right] >= pivot_item )
right--;
if ( left < right )
SWAP(a[left],a[right]);
}
/* Right is final position for the pivot */
a[low] = a[right];
a[right] = pivot_item;
return right;
}
Best Case T(n), to sort
n elements
void quicksort( void *a, int low, int high ) c1
{
int pivot;
/* Termination condition! */
if ( high > low )
{
pivot = partition( a, low, high ); c2 * n
quicksort( a, low, pivot-1 ); T(n/2)
quicksort( a, pivot+1, high ); T(n/2)
}
}
So, T(n) = O(1) when n = 1, and
2T(n/2) + c1+ c2*n when n > 1
So
Skipping the insignificant parts,
[]
39
40
Quicksort: Analysis
Worst case:
Partitioning n elements results in two subsequences of
lengths 1 and n-1
What happens if we use quicksort on
data that’s already sorted (or nearly sorted)?
Now Take a look
41
Quicksort: Analysis
Sorted Data
pivot
? 1 2 3 4 5 6 7 8 9
< pivot > pivot
42
Quicksort: Analysis (Sorted Data)
Each partition pivot
produces
a problem of size 0 1 2 3 4 5 6 7 8 9
and one of size n-1! > pivot
pivot
2 3 4 5 6 7 8 9
> pivot
Worst Case T(n), to sort
n elements
void quicksort( void *a, int low, int high ) c1
{
int pivot;
/* Termination condition! */
if ( high > low )
{
pivot = partition( a, low, high ); c2 * n
quicksort( a, low, pivot-1 ); T(1) = c3
quicksort( a, pivot+1, high ); T(n-1)
}
}
So, T(n) = O(1) when n = 1, and
T(n-1) + c1+ c2*n + c3 when n > 1
n–k=1
So k=n-1
0, 1, 3, 6
= 0, 1, (1+2), (1+2+3)
Kth term
1+2+3+....+(k-1) =
45
46
Quicksort: Improve performance
Quicksort’s O(n log n) behavior
Depends on the partitions being nearly equal
there are O( log n ) of them
On average, this will nearly be the case
and quicksort is generally O(n log n)
Can we do anything to ensure O(n log n) time?
In general, no
But we can improve our chances!!
47
Quicksort: How to select pivot?
Any pivot will work …
But choose a different pivot …
So that the partitions are equal
Then we will see O(n log n) time
pivot
1 2 3 4 5 6 7 8 9
< pivot > pivot
48
Quicksort: How to select pivot?
Median-of-3 pivot
Take 3 positions and choose the median
Say … First, middle, last
1 2 3 4 5 6 7 8 9
Median is 5
Perfect division of sorted data every time!
O(n log n) time
Since sorted (or nearly sorted) data is
common,
median-of-3 is a good strategy
especially if you think your data may be sorted!
49
Quicksort: How to select pivot?
Randomly selected pivot
Choose a pivot randomly
Different position for every partition
On average, sorted data is divided evenly
O(n log n) time
Key requirement
• Pivot choice must take O(1) time
Ashiq, MIST
50
Quicksort: Guaranteed O (nlogn)
Never!!
Any pivot selection strategy
could lead to O(n2) time
1 4 9 6 2 5 7 8 3
Here median-of-3 chooses 2
One partition of size 1 and 1 2 4 9 6 5 7 8 3
One partition of size 7
Next it chooses 4
One partition of size 1 and
One partition of size 5
51
Key points
Quicksort
O(n log n) but ….
Can be O(n2)
Depends on pivot selection
Median-of-3
Random pivot
Better but not guaranteed
Books
Introduction to Algorithms, Thomas H. Cormen, Charle E.
Leiserson, Ronald L. Rivest, Clifford Stein (CLRS).
Fundamental of Computer Algorithms, Ellis Horowitz, Sartaj
Sahni, Sanguthevar Rajasekaran (HSR)
References
https://www.google.com/search?q=bubble+sort+step+by+step&sxsrf=AL
eKk01uxzgfT3Oy6k1Q3WxVnSpiIN8_4g:1587999728942&tbm=isch&sourc
e=iu&ictx=1&fir=vRwFsGwVfJ6pJM%253A%252CSzhhze6MPQr4cM%252C
_&vet=1&usg=AI4_-kSrEEXqwRL-PkHhVUtn7jNfF9dB6g&sa=X&ved=2ahU
KEwje0Pz974jpAhXRAnIKHWhMD2UQ_h0wAXoECAcQBg#imgrc=EN4Sdu7
veOWVoM&imgdii=eOqvCu85p9-eBM
https://www.interviewcake.com/concept/java/counting-sort
https://www.geeksforgeeks.org/counting-sort/
https://www.hackerearth.com/practice/algorithms/sorting/quick-sort/tut
orial/