CS11212 - Spring 2023

Data Structures &

Introduction to Algorithms

Analysis of Algorithms
Searching & Sorting: Part 2

Ibrahim Albluwi
Sorting: A Fundamental Problem

Problem. Given a list of n elements, order them in non-decreasing (or ascending) order.
Common variant. Order the elements in descending order.

Requirement. e meaning of <, >, == for the element type must be defined.
In C++, it is defined for int, double, char, string, etc.,
but not for user defined types (e.g. What does car1 > car2 mean?)

Inefficient, but easy to analyze!

(covered in this course)

Too many ways to sort!

Bubble Sort Quicksort MSD Radix Sort

Selection Sort Heapsort LSD Radix Sort
Insertion Sort Timsort Counting Sort
Exchange Sort Merge Sort Bucket Sort
Cocktail Sort Shell Sort Bitonic Sort
Stooge Sort BST Sort Bogo (Stupid) Sort
Comb Sort Cycle Sort ...
Sorting: A Fundamental Problem

Problem. Given a list of n elements, order them in non-decreasing (or ascending) order.
Common variant. Order the elements in descending order.

Requirement. e meaning of <, >, == for the element type must be defined.
In C++, it is defined for int, double, char, string, etc.,
but not for user defined types (e.g. What does car1 > car2 mean?)

Efficient, but harder to analyze!

(covered in the Algorithms course)

Too many ways to sort!

Bubble Sort Quicksort MSD Radix Sort

Selection Sort Heapsort LSD Radix Sort
Insertion Sort Timsort Counting Sort
Exchange Sort Merge Sort Bucket Sort
Cocktail Sort Shell Sort Bitonic Sort
Stooge Sort BST Sort Bogo (Stupid) Sort
Comb Sort Cycle Sort ...
Sorting Warmup

Problem. Sort a list of books alphabetically.

Restrictions. Can't place any book anywhere outside the shelf while sorting.

M N Z C B A T U V S

Which one is the best?

Let's count operations!

Idea 1. Select the min and place it in its correct position, then the second min, etc.
Idea 2. Go through each element and insert it in its correct position relative to its le.
Idea 3. Bubble Sort!
Selection Sort: Implementation

i
5 3 0 18 48 25 31 32 40 12
0 1 2 3 4 5 6 7 8 9

void selection(int a[], int n) {

for (int i = 0; i < n-1; i++) {
int min_index = i;
for (intfindj the
= index
i+1; ofj the
< n; j++)
ifminimum
(a[j] in < a[i,
a[min_index])
n-1]
min_index = j;

if (i != min_index)
place the minimum
in its right position
swap(a[i], a[min_index]);
}
}
Selection Sort: Implementation

i
5 3 0 18 48 25 31 32 40 12
0 1 2 3 4 5 6 7 8 9

void selection(int a[], int n) {

for (int i = 0; i < n-1; i++) { Search for the
int min_index = i; minimum n-1 times
for (intfindj the
= index
i+1; ofj the
< n; j++)
ifminimum
(a[j] in < a[i,
a[min_index])
n-1]
min_index = j;

if (i != min_index)
place the minimum
in its right position
swap(a[i], a[min_index]);
}
}
Selection Sort: Implementation

i
5 3 0 18 48 25 31 32 40 12
0 1 2 3 4 5 6 7 8 9

void selection(int a[], int n) {

for (int i = 0; i < n-1; i++) {
int min_index = i;
for (int j = i+1; j < n; j++) find the index of the
if (a[j] < a[min_index]) minimum in a[i, n-1]
min_index = j;

if (i != min_index) place the minimum

swap(a[i], a[min_index]); in its right position
}
}
Selection Sort: Implementation

i j
5 3 0 18 48 25 31 32 40 12
0 1 2 3 4 5 6 7 8 9

min_index

void selection(int a[], int n) {

for (int i = 0; i < n-1; i++) {
int min_index = i;
for (int j = i+1; j < n; j++) find the index of the
if (a[j] < a[min_index]) minimum in a[i, n-1]
min_index = j;

if (i != min_index) place the minimum

swap(a[i], a[min_index]); in its right position
}
}
Selection Sort: Tracing

i j
5 3 0 18 48 25 31 32 40 12
0 1 2 3 4 5 6 7 8 9

min_index

void selection(int a[], int n) {

for (int i = 0; i < n-1; i++) {
int min_index = i;
for (int j = i+1; j < n; j++) find the index of the
if (a[j] < a[min_index]) minimum in a[i, n-1]
min_index = j;

if (i != min_index)
swap(a[i], a[min_index]);
}
}
Selection Sort: Tracing

i j

5 3 0 18 48 25 31 32 40 12
0 1 2 3 4 5 6 7 8 9

min_index

void selection(int a[], int n) {

for (int i = 0; i < n-1; i++) {
int min_index = i;
for (int j = i+1; j < n; j++) find the index of the
if (a[j] < a[min_index]) minimum in a[i, n-1]
min_index = j;

if (i != min_index)
swap(a[i], a[min_index]);
}
}
Selection Sort: Tracing

i j

5 3 0 18 48 25 31 32 40 12
0 1 2 3 4 5 6 7 8 9

min_index

void selection(int a[], int n) {

for (int i = 0; i < n-1; i++) {
int min_index = i;
for (int j = i+1; j < n; j++) find the index of the
if (a[j] < a[min_index]) minimum in a[i, n-1]
min_index = j;

if (i != min_index)
swap(a[i], a[min_index]);
}
}
Selection Sort: Tracing

i j

5 3 0 18 48 25 31 32 40 12
0 1 2 3 4 5 6 7 8 9

min_index

void selection(int a[], int n) {

for (int i = 0; i < n-1; i++) {
int min_index = i;
for (int j = i+1; j < n; j++) find the index of the
if (a[j] < a[min_index]) minimum in a[i, n-1]
min_index = j;

if (i != min_index)
swap(a[i], a[min_index]);
}
}
Selection Sort: Tracing

i j j
5 3 0 18 48 25 31 32 40 12
0 1 2 3 4 5 6 7 8 9

min_index

void selection(int a[], int n) {

for (int i = 0; i < n-1; i++) {
int min_index = i;
for (int j = i+1; j < n; j++) find the index of the
if (a[j] < a[min_index]) minimum in a[i, n-1]
min_index = j;

if (i != min_index)
swap(a[i], a[min_index]);
}
}
Selection Sort: Tracing

i j j
5 3 0 18 48 25 31 32 40 12
0 1 2 3 4 5 6 7 8 9

min_index

void selection(int a[], int n) {

for (int i = 0; i < n-1; i++) {
int min_index = i;
for (int j = i+1; j < n; j++)
if (a[j] < a[min_index])
min_index = j;

if (i != min_index) place the minimum

swap(a[i], a[min_index]); in its right position
}
}
Selection Sort: Tracing

i j
0 3 5 18 48 25 31 32 40 12
0 1 2 3 4 5 6 7 8 9

min_index

void selection(int a[], int n) {

for (int i = 0; i < n-1; i++) {
int min_index = i;
for (int j = i+1; j < n; j++)
if (a[j] < a[min_index])
min_index = j;

if (i != min_index) place the minimum

swap(a[i], a[min_index]); in its right position
}
}
Selection Sort: Tracing

i j
0 3 5 18 48 25 31 32 40 12
0 1 2 3 4 5 6 7 8 9

min_index

void selection(int a[], int n) {

for (int i = 0; i < n-1; i++) {
int min_index = i;
for (int j = i+1; j < n; j++) find the index of the
if (a[j] < a[min_index]) minimum in a[i, n-1]
min_index = j;

if (i != min_index)
swap(a[i], a[min_index]);
}
}
Selection Sort: Tracing

i j j
0 3 5 18 48 25 31 32 40 12
0 1 2 3 4 5 6 7 8 9

min_index

void selection(int a[], int n) {

for (int i = 0; i < n-1; i++) {
int min_index = i;
for (int j = i+1; j < n; j++) find the index of the
if (a[j] < a[min_index]) minimum in a[i, n-1]
min_index = j;

if (i != min_index)
swap(a[i], a[min_index]);
}
}
Selection Sort: Tracing

i j j
0 3 5 18 48 25 31 32 40 12
0 1 2 3 4 5 6 7 8 9

min_index

void selection(int a[], int n) {

for (int i = 0; i < n-1; i++) {
int min_index = i;
for (int j = i+1; j < n; j++)
if (a[j] < a[min_index])
min_index = j;
No swap!
if (i != min_index) place the minimum
swap(a[i], a[min_index]); in its right position
}
}
Selection Sort: Tracing

i j

0 3 5 18 48 25 31 32 40 12
0 1 2 3 4 5 6 7 8 9

min_index

void selection(int a[], int n) {

for (int i = 0; i < n-1; i++) {
int min_index = i;
for (int j = i+1; j < n; j++) find the index of the
if (a[j] < a[min_index]) minimum in a[i, n-1]
min_index = j;

if (i != min_index)
swap(a[i], a[min_index]);
}
}
Selection Sort: Tracing

i j j
0 3 5 18 48 25 31 32 40 12
0 1 2 3 4 5 6 7 8 9

min_index

void selection(int a[], int n) {

for (int i = 0; i < n-1; i++) {
int min_index = i;
for (int j = i+1; j < n; j++) find the index of the
if (a[j] < a[min_index]) minimum in a[i, n-1]
min_index = j;

if (i != min_index)
swap(a[i], a[min_index]);
}
}
Selection Sort: Tracing

i j j
0 3 5 18 48 25 31 32 40 12
0 1 2 3 4 5 6 7 8 9

min_index

void selection(int a[], int n) {

for (int i = 0; i < n-1; i++) {
int min_index = i;
for (int j = i+1; j < n; j++)
if (a[j] < a[min_index])
min_index = j;
No swap!
if (i != min_index) place the minimum
swap(a[i], a[min_index]); in its right position
}
}
Selection Sort: Tracing

i j
0 3 5 18 48 25 31 32 40 12
0 1 2 3 4 5 6 7 8 9

min_index

void selection(int a[], int n) {

for (int i = 0; i < n-1; i++) {
int min_index = i;
for (int j = i+1; j < n; j++) find the index of the
if (a[j] < a[min_index]) minimum in a[i, n-1]
min_index = j;

if (i != min_index)
swap(a[i], a[min_index]);
}
}
Selection Sort: Tracing

i j j
0 3 5 18 48 25 31 32 40 12
0 1 2 3 4 5 6 7 8 9

min_index

void selection(int a[], int n) {

for (int i = 0; i < n-1; i++) {
int min_index = i;
for (int j = i+1; j < n; j++) find the index of the
if (a[j] < a[min_index]) minimum in a[i, n-1]
min_index = j;

if (i != min_index)
swap(a[i], a[min_index]);
}
}
Selection Sort: Tracing

i j j
0 3 5 18 48 25 31 32 40 12
0 1 2 3 4 5 6 7 8 9

min_index

void selection(int a[], int n) {

for (int i = 0; i < n-1; i++) {
int min_index = i;
for (int j = i+1; j < n; j++) find the index of the
if (a[j] < a[min_index]) minimum in a[i, n-1]
min_index = j;

if (i != min_index)
swap(a[i], a[min_index]);
}
}
Selection Sort: Tracing

i j j
0 3 5 18 48 25 31 32 40 12
0 1 2 3 4 5 6 7 8 9

min_index

void selection(int a[], int n) {

for (int i = 0; i < n-1; i++) {
int min_index = i;
for (int j = i+1; j < n; j++) find the index of the
if (a[j] < a[min_index]) minimum in a[i, n-1]
min_index = j;

if (i != min_index)
swap(a[i], a[min_index]);
}
}
Selection Sort: Tracing

i j
0 3 5 18 48 25 31 32 40 12
0 1 2 3 4 5 6 7 8 9

min_index

void selection(int a[], int n) {

for (int i = 0; i < n-1; i++) {
int min_index = i;
for (int j = i+1; j < n; j++)
if (a[j] < a[min_index])
min_index = j;

if (i != min_index) place the minimum

swap(a[i], a[min_index]); in its right position
}
}
Selection Sort: Tracing

i j
0 3 5 12 48 25 31 32 40 18
0 1 2 3 4 5 6 7 8 9

min_index

void selection(int a[], int n) {

for (int i = 0; i < n-1; i++) {
int min_index = i;
for (int j = i+1; j < n; j++)
if (a[j] < a[min_index])
min_index = j;

if (i != min_index) place the minimum

swap(a[i], a[min_index]); in its right position
}
}
Selection Sort: Tracing

i j
0 3 5 12 48 25 31 32 40 18
0 1 2 3 4 5 6 7 8 9

min_index

void selection(int a[], int n) {

for (int i = 0; i < n-1; i++) {
int min_index = i;
for (int j = i+1; j < n; j++) find the index of the
if (a[j] < a[min_index]) minimum in a[i, n-1]
min_index = j;

if (i != min_index)
swap(a[i], a[min_index]);
}
}
Selection Sort: Tracing

i j j
0 3 5 12 48 25 31 32 40 18
0 1 2 3 4 5 6 7 8 9

min_index

void selection(int a[], int n) {

for (int i = 0; i < n-1; i++) {
int min_index = i;
for (int j = i+1; j < n; j++) find the index of the
if (a[j] < a[min_index]) minimum in a[i, n-1]
min_index = j;

if (i != min_index)
swap(a[i], a[min_index]);
}
}
Selection Sort: Tracing

i j j
0 3 5 12 48 25 31 32 40 18
0 1 2 3 4 5 6 7 8 9

min_index

void selection(int a[], int n) {

for (int i = 0; i < n-1; i++) {
int min_index = i;
for (int j = i+1; j < n; j++) find the index of the
if (a[j] < a[min_index]) minimum in a[i, n-1]
min_index = j;

if (i != min_index)
swap(a[i], a[min_index]);
}
}
Selection Sort: Tracing

i j j
0 3 5 12 48 25 31 32 40 18
0 1 2 3 4 5 6 7 8 9

min_index

void selection(int a[], int n) {

for (int i = 0; i < n-1; i++) {
int min_index = i;
for (int j = i+1; j < n; j++) find the index of the
if (a[j] < a[min_index]) minimum in a[i, n-1]
min_index = j;

if (i != min_index)
swap(a[i], a[min_index]);
}
}
Selection Sort: Tracing

i j
0 3 5 12 48 25 31 32 40 18
0 1 2 3 4 5 6 7 8 9

min_index

void selection(int a[], int n) {

for (int i = 0; i < n-1; i++) {
int min_index = i;
for (int j = i+1; j < n; j++)
if (a[j] < a[min_index])
min_index = j;

if (i != min_index) place the minimum

swap(a[i], a[min_index]); in its right position
}
}
Selection Sort: Tracing

i j
0 3 5 12 18 25 31 32 40 48
0 1 2 3 4 5 6 7 8 9

min_index

void selection(int a[], int n) {

for (int i = 0; i < n-1; i++) {
int min_index = i;
for (int j = i+1; j < n; j++)
if (a[j] < a[min_index])
min_index = j;

if (i != min_index) place the minimum

swap(a[i], a[min_index]); in its right position
}
}
Selection Sort: Tracing

i j
0 3 5 12 18 25 31 32 40 48
0 1 2 3 4 5 6 7 8 9

min_index

void selection(int a[], int n) {

for (int i = 0; i < n-1; i++) {
int min_index = i;
for (int j = i+1; j < n; j++) find the index of the
if (a[j] < a[min_index]) minimum in a[i, n-1]
min_index = j;

if (i != min_index)
swap(a[i], a[min_index]);
}
}
Selection Sort: Tracing

i j j
0 3 5 12 18 25 31 32 40 48
0 1 2 3 4 5 6 7 8 9

min_index

void selection(int a[], int n) {

for (int i = 0; i < n-1; i++) {
int min_index = i;
for (int j = i+1; j < n; j++) find the index of the
if (a[j] < a[min_index]) minimum in a[i, n-1]
min_index = j;

if (i != min_index)
swap(a[i], a[min_index]);
}
}
Selection Sort: Tracing

i j
0 3 5 12 18 25 31 32 40 48
0 1 2 3 4 5 6 7 8 9

min_index

void selection(int a[], int n) {

for (int i = 0; i < n-1; i++) {
int min_index = i;
for (int j = i+1; j < n; j++)
if (a[j] < a[min_index])
min_index = j;
No swap!
if (i != min_index) place the minimum
swap(a[i], a[min_index]); in its right position
}
}
Selection Sort: Tracing

i j
0 3 5 12 18 25 31 32 40 48
0 1 2 3 4 5 6 7 8 9

min_index

void selection(int a[], int n) {

for (int i = 0; i < n-1; i++) {
int min_index = i;
for (int j = i+1; j < n; j++) find the index of the
if (a[j] < a[min_index]) minimum in a[i, n-1]
min_index = j;

if (i != min_index)
swap(a[i], a[min_index]);
}
}
Selection Sort: Tracing

i j j
0 3 5 12 18 25 31 32 40 48
0 1 2 3 4 5 6 7 8 9

min_index

void selection(int a[], int n) {

for (int i = 0; i < n-1; i++) {
int min_index = i;
for (int j = i+1; j < n; j++) find the index of the
if (a[j] < a[min_index]) minimum in a[i, n-1]
min_index = j;

if (i != min_index)
swap(a[i], a[min_index]);
}
}
Selection Sort: Tracing

i j
0 3 5 12 18 25 31 32 40 48
0 1 2 3 4 5 6 7 8 9

min_index

void selection(int a[], int n) {

for (int i = 0; i < n-1; i++) {
int min_index = i;
for (int j = i+1; j < n; j++)
if (a[j] < a[min_index])
min_index = j;
No swap!
if (i != min_index) place the minimum
swap(a[i], a[min_index]); in its right position
}
}
Selection Sort: Tracing

i j
0 3 5 12 18 25 31 32 40 48
0 1 2 3 4 5 6 7 8 9

min_index

void selection(int a[], int n) {

for (int i = 0; i < n-1; i++) {
int min_index = i;
for (int j = i+1; j < n; j++) find the index of the
if (a[j] < a[min_index]) minimum in a[i, n-1]
min_index = j;

if (i != min_index)
swap(a[i], a[min_index]);
}
}
Selection Sort: Tracing

i j j
0 3 5 12 18 25 31 32 40 48
0 1 2 3 4 5 6 7 8 9

min_index

void selection(int a[], int n) {

for (int i = 0; i < n-1; i++) {
int min_index = i;
for (int j = i+1; j < n; j++) find the index of the
if (a[j] < a[min_index]) minimum in a[i, n-1]
min_index = j;

if (i != min_index)
swap(a[i], a[min_index]);
}
}
Selection Sort: Tracing

i j
0 3 5 12 18 25 31 32 40 48
0 1 2 3 4 5 6 7 8 9

min_index

void selection(int a[], int n) {

for (int i = 0; i < n-1; i++) {
int min_index = i;
for (int j = i+1; j < n; j++)
if (a[j] < a[min_index])
min_index = j;
No swap!
if (i != min_index) place the minimum
swap(a[i], a[min_index]); in its right position
}
}
Selection Sort: Tracing

i j
0 3 5 12 18 25 31 32 40 48
0 1 2 3 4 5 6 7 8 9

min_index

void selection(int a[], int n) {

for (int i = 0; i < n-1; i++) {
int min_index = i;
for (int j = i+1; j < n; j++) find the index of the
if (a[j] < a[min_index]) minimum in a[i, n-1]
min_index = j;

if (i != min_index)
swap(a[i], a[min_index]);
}
}
Selection Sort: Tracing

i j
0 3 5 12 18 25 31 32 40 48
0 1 2 3 4 5 6 7 8 9

min_index

void selection(int a[], int n) {

for (int i = 0; i < n-1; i++) {
int min_index = i;
for (int j = i+1; j < n; j++) find the index of the
if (a[j] < a[min_index]) minimum in a[i, n-1]
min_index = j;

if (i != min_index)
swap(a[i], a[min_index]);
}
}
Selection Sort: Tracing

i j
0 3 5 12 18 25 31 32 40 48
0 1 2 3 4 5 6 7 8 9

min_index

void selection(int a[], int n) {

for (int i = 0; i < n-1; i++) {
int min_index = i;
for (int j = i+1; j < n; j++)
if (a[j] < a[min_index])
min_index = j;
No swap!
if (i != min_index) place the minimum
swap(a[i], a[min_index]); in its right position
}
}
Selection Sort: Tracing

i
0 3 5 12 18 25 31 32 40 48
0 1 2 3 4 5 6 7 8 9

void selection(int a[], int n) {

for (int i = 0; i < n-1; i++) {
int min_index = i;
for (int j = i+1; j < n; j++)
if (a[j] < a[min_index])
min_index = j;

if (i != min_index)
swap(a[i], a[min_index]);
}
}
void selection(int a[], int n) { Analysis
for (int i = 0; i < n-1; i++) {
int min_index = i;
for (int j = i+1; j < n; j++) find the index of the
if (a[j] < a[min_index]) minimum in a[i, n-1]
min_index = j;

if (i != min_index) place the minimum

swap(a[i], a[min_index]); in its right position
}
}

Data compares.

Counting only comparisons

between array elements
 void selection(int a[], int n) { Analysis
for (int i = 0; i < n-1; i++) {
int min_index = i;
for (int j = i+1; j < n; j++) find the index of the
if (a[j] < a[min_index]) minimum in a[i, n-1]
min_index = j;

if (i != min_index) place the minimum

swap(a[i], a[min_index]); in its right position
}
}

Data compares. e algorithm is insensitive to the arrangement of the elements in the array.
n−1
1
∑
= 1 + 2 + 3 + … + (n − 1) = i = 2 n(n − 1) data compares
i=1
 void selection(int a[], int n) { Analysis
for (int i = 0; i < n-1; i++) {
int min_index = i;
for (int j = i+1; j < n; j++) find the index of the
if (a[j] < a[min_index]) minimum in a[i, n-1]
min_index = j;

if (i != min_index) place the minimum

swap(a[i], a[min_index]); in its right position
}
}

Data compares. e algorithm is insensitive to the arrangement of the elements in the array.
n−1
1
∑
= 1 + 2 + 3 + … + (n − 1) = i = 2 n(n − 1) data compares
i=1

Data Moves.
Worst case. One swap per iteration, a total of n − 1 swaps (= 3(n − 1) data moves).
Best case. No swaps if the array is already sorted.

Counting only movements

of array elements
 void selection(int a[], int n) { Analysis
for (int i = 0; i < n-1; i++) {
int min_index = i;
for (int j = i+1; j < n; j++) find the index of the
if (a[j] < a[min_index]) minimum in a[i, n-1]
min_index = j;

if (i != min_index) place the minimum

swap(a[i], a[min_index]); in its right position
}
}

Data compares. e algorithm is insensitive to the arrangement of the elements in the array.
n−1
1
∑
= 1 + 2 + 3 + … + (n − 1) = i = 2 n(n − 1) data compares
i=1

Data Moves.
Worst case. One swap per iteration, a total of n − 1 swaps (= 3(n − 1) data moves).
Best case. No swaps if the array is already sorted.

Think!
Can you come up with an array
of size 6 that leads to 5 swaps?
 void selection(int a[], int n) { Analysis
for (int i = 0; i < n-1; i++) {
int min_index = i;
for (int j = i+1; j < n; j++) find the index of the
if (a[j] < a[min_index]) minimum in a[i, n-1]
min_index = j;

if (i != min_index) place the minimum

swap(a[i], a[min_index]); in its right position
}
}

Data compares. e algorithm is insensitive to the arrangement of the elements in the array.
n−1
1
∑
= 1 + 2 + 3 + … + (n − 1) = i = 2 n(n − 1) data compares
i=1

Data Moves.
Worst case. One swap per iteration, a total of n − 1 swaps (= 3(n − 1) data moves).
Best case. No swaps if the array is already sorted.

Total. O(n 2) operations in the best case and the worst case.
Insertion Sort: Implementation

i
0 -1 3 5 6 2 4 9 40 31
0 1 2 3 4 5 6 7 8 9

sorted not sorted

void insertion(int a[], int n) {

for (int i = 1; i < n; i++) {
int temp = a[i];
int j = i-1; Insert every element from
while (j >= 0 && temp < a[j]) { the unsorted part into its
a[j+1] = a[j]; correct position in the
j--; sorted part
}
a[j+1] = temp;
}
}
Insertion Sort: Implementation

temp = -1
i
0 -1 3 5 6 2 4 9 40 31
0 1 2 3 4 5 6 7 8 9

sorted not sorted

void insertion(int a[], int n) {

for (int i = 1; i < n; i++) {
int temp = a[i]; store element i

int j = i-1;
while (j >= 0 && temp < a[j]) {
a[j+1] = a[j];
j--;
}
a[j+1] = temp;
}
}
Insertion Sort: Implementation

temp = -1
j i
0 -1 3 5 6 2 4 9 40 31
0 1 2 3 4 5 6 7 8 9

sorted not sorted

void insertion(int a[], int n) {

for (int i = 1; i < n; i++) {
int temp = a[i]; store element i

int j = i-1;
while (j >= 0 && temp < a[j]) {
shi the elements until
a[j+1] = a[j]; the right position of
j--; element i is found
}
a[j+1] = temp;
}
}
Insertion Sort: Implementation

temp = -1
j i
0 0 3 5 6 2 4 9 40 31
0 1 2 3 4 5 6 7 8 9

sorted not sorted

void insertion(int a[], int n) {

for (int i = 1; i < n; i++) {
int temp = a[i]; store element i

int j = i-1;
while (j >= 0 && temp < a[j]) {
shi the elements until
a[j+1] = a[j]; the right position of
j--; element i is found
}
a[j+1] = temp;
}
}
Insertion Sort: Implementation

temp = -1
j i
0 0 3 5 6 2 4 9 40 31
0 1 2 3 4 5 6 7 8 9

sorted not sorted

void insertion(int a[], int n) {

for (int i = 1; i < n; i++) {
int temp = a[i]; store element i

int j = i-1;
while (j >= 0 && temp < a[j]) {
shi the elements until
a[j+1] = a[j]; the right position of
j--; element i is found
}
a[j+1] = temp;
}
}
Insertion Sort: Implementation

temp =

j i
-1 0 3 5 6 2 4 9 40 31
0 1 2 3 4 5 6 7 8 9

sorted not sorted

void insertion(int a[], int n) {

for (int i = 1; i < n; i++) {
int temp = a[i]; store element i

int j = i-1;
while (j >= 0 && temp < a[j]) {
shi the elements until
a[j+1] = a[j]; the right position of
j--; element i is found
}
place the element in
a[j+1] = temp; its right position
}
}
Insertion Sort: Implementation

j i
-1 0 3 5 6 2 4 9 40 31
0 1 2 3 4 5 6 7 8 9

sorted not sorted

void insertion(int a[], int n) {

for (int i = 1; i < n; i++) {
int temp = a[i]; store element i

int j = i-1;
while (j >= 0 && temp < a[j]) {
shi the elements until
a[j+1] = a[j]; the right position of
j--; element i is found
}
place the element in
a[j+1] = temp; its right position
}
}
Insertion Sort: Implementation

temp = 3
j i
-1 0 3 5 6 2 4 9 40 31
0 1 2 3 4 5 6 7 8 9

sorted not sorted

void insertion(int a[], int n) {

for (int i = 1; i < n; i++) {
int temp = a[i]; store element i

int j = i-1;
while (j >= 0 && temp < a[j]) {
shi the elements until
a[j+1] = a[j]; the right position of
j--; element i is found
}
place the element in
a[j+1] = temp; its right position
}
}
Insertion Sort: Implementation

temp =
j i
-1 0 3 5 6 2 4 9 40 31
0 1 2 3 4 5 6 7 8 9

sorted not sorted

void insertion(int a[], int n) {

for (int i = 1; i < n; i++) {
int temp = a[i]; store element i

int j = i-1;
while (j >= 0 && temp < a[j]) {
shi the elements until
a[j+1] = a[j]; the right position of
j--; element i is found
}
place the element in
a[j+1] = temp; its right position
}
}
Insertion Sort: Implementation

j i
-1 0 3 5 6 2 4 9 40 31
0 1 2 3 4 5 6 7 8 9

sorted not sorted

void insertion(int a[], int n) {

for (int i = 1; i < n; i++) {
int temp = a[i]; store element i

int j = i-1;
while (j >= 0 && temp < a[j]) {
shi the elements until
a[j+1] = a[j]; the right position of
j--; element i is found
}
place the element in
a[j+1] = temp; its right position
}
}
Insertion Sort: Implementation

temp = 5
j i
-1 0 3 5 6 2 4 9 40 31
0 1 2 3 4 5 6 7 8 9

sorted not sorted

void insertion(int a[], int n) {

for (int i = 1; i < n; i++) {
int temp = a[i]; store element i

int j = i-1;
while (j >= 0 && temp < a[j]) {
shi the elements until
a[j+1] = a[j]; the right position of
j--; element i is found
}
place the element in
a[j+1] = temp; its right position
}
}
Insertion Sort: Implementation

temp =

j i
-1 0 3 5 6 2 4 9 40 31
0 1 2 3 4 5 6 7 8 9

sorted not sorted

void insertion(int a[], int n) {

for (int i = 1; i < n; i++) {
int temp = a[i]; store element i

int j = i-1;
while (j >= 0 && temp < a[j]) {
shi the elements until
a[j+1] = a[j]; the right position of
j--; element i is found
}
place the element in
a[j+1] = temp; its right position
}
}
Insertion Sort: Implementation

j i
-1 0 3 5 6 2 4 9 40 31
0 1 2 3 4 5 6 7 8 9

sorted not sorted

void insertion(int a[], int n) {

for (int i = 1; i < n; i++) {
int temp = a[i]; store element i

int j = i-1;
while (j >= 0 && temp < a[j]) {
shi the elements until
a[j+1] = a[j]; the right position of
j--; element i is found
}
place the element in
a[j+1] = temp; its right position
}
}
Insertion Sort: Implementation

temp = 6
j i
-1 0 3 5 6 2 4 9 40 31
0 1 2 3 4 5 6 7 8 9

sorted not sorted

void insertion(int a[], int n) {

for (int i = 1; i < n; i++) {
int temp = a[i]; store element i

int j = i-1;
while (j >= 0 && temp < a[j]) {
shi the elements until
a[j+1] = a[j]; the right position of
j--; element i is found
}
place the element in
a[j+1] = temp; its right position
}
}
Insertion Sort: Implementation

temp =

j i
-1 0 3 5 6 2 4 9 40 31
0 1 2 3 4 5 6 7 8 9

sorted not sorted

void insertion(int a[], int n) {

for (int i = 1; i < n; i++) {
int temp = a[i]; store element i

int j = i-1;
while (j >= 0 && temp < a[j]) {
shi the elements until
a[j+1] = a[j]; the right position of
j--; element i is found
}
place the element in
a[j+1] = temp; its right position
}
}
Insertion Sort: Implementation

j i
-1 0 3 5 6 2 4 9 40 31
0 1 2 3 4 5 6 7 8 9

sorted not sorted

void insertion(int a[], int n) {

for (int i = 1; i < n; i++) {
int temp = a[i]; store element i

int j = i-1;
while (j >= 0 && temp < a[j]) {
shi the elements until
a[j+1] = a[j]; the right position of
j--; element i is found
}
place the element in
a[j+1] = temp; its right position
}
}
Insertion Sort: Implementation

temp = 2
j i
-1 0 3 5 6 2 4 9 40 31
0 1 2 3 4 5 6 7 8 9

sorted not sorted

void insertion(int a[], int n) {

for (int i = 1; i < n; i++) {
int temp = a[i]; store element i

int j = i-1;
while (j >= 0 && temp < a[j]) {
shi the elements until
a[j+1] = a[j]; the right position of
j--; element i is found
}
place the element in
a[j+1] = temp; its right position
}
}
Insertion Sort: Implementation

temp = 2
j i
-1 0 3 5 6 6 4 9 40 31
0 1 2 3 4 5 6 7 8 9

sorted not sorted

void insertion(int a[], int n) {

for (int i = 1; i < n; i++) {
int temp = a[i]; store element i

int j = i-1;
while (j >= 0 && temp < a[j]) {
shi the elements until
a[j+1] = a[j]; the right position of
j--; element i is found
}
place the element in
a[j+1] = temp; its right position
}
}
Insertion Sort: Implementation

temp = 2
j i
-1 0 3 5 6 6 4 9 40 31
0 1 2 3 4 5 6 7 8 9

sorted not sorted

void insertion(int a[], int n) {

for (int i = 1; i < n; i++) {
int temp = a[i]; store element i

int j = i-1;
while (j >= 0 && temp < a[j]) {
shi the elements until
a[j+1] = a[j]; the right position of
j--; element i is found
}
place the element in
a[j+1] = temp; its right position
}
}
Insertion Sort: Implementation

temp = 2
j i
-1 0 3 5 5 6 4 9 40 31
0 1 2 3 4 5 6 7 8 9

sorted not sorted

void insertion(int a[], int n) {

for (int i = 1; i < n; i++) {
int temp = a[i]; store element i

int j = i-1;
while (j >= 0 && temp < a[j]) {
shi the elements until
a[j+1] = a[j]; the right position of
j--; element i is found
}
place the element in
a[j+1] = temp; its right position
}
}
Insertion Sort: Implementation

temp = 2
j i
-1 0 3 5 5 6 4 9 40 31
0 1 2 3 4 5 6 7 8 9

sorted not sorted

void insertion(int a[], int n) {

for (int i = 1; i < n; i++) {
int temp = a[i]; store element i

int j = i-1;
while (j >= 0 && temp < a[j]) {
shi the elements until
a[j+1] = a[j]; the right position of
j--; element i is found
}
place the element in
a[j+1] = temp; its right position
}
}
Insertion Sort: Implementation

temp = 2
j i
-1 0 3 3 5 6 4 9 40 31
0 1 2 3 4 5 6 7 8 9

sorted not sorted

void insertion(int a[], int n) {

for (int i = 1; i < n; i++) {
int temp = a[i]; store element i

int j = i-1;
while (j >= 0 && temp < a[j]) {
shi the elements until
a[j+1] = a[j]; the right position of
j--; element i is found
}
place the element in
a[j+1] = temp; its right position
}
}
Insertion Sort: Implementation

temp = 2
j i
-1 0 3 3 5 6 4 9 40 31
0 1 2 3 4 5 6 7 8 9

sorted not sorted

void insertion(int a[], int n) {

for (int i = 1; i < n; i++) {
int temp = a[i]; store element i

int j = i-1;
while (j >= 0 && temp < a[j]) {
shi the elements until
a[j+1] = a[j]; the right position of
j--; element i is found
}
place the element in
a[j+1] = temp; its right position
}
}
Insertion Sort: Implementation

temp =

j i
-1 0 2 3 5 6 4 9 40 31
0 1 2 3 4 5 6 7 8 9

sorted not sorted

void insertion(int a[], int n) {

for (int i = 1; i < n; i++) {
int temp = a[i]; store element i

int j = i-1;
while (j >= 0 && temp < a[j]) {
shi the elements until
a[j+1] = a[j]; the right position of
j--; element i is found
}
place the element in
a[j+1] = temp; its right position
}
}
Insertion Sort: Implementation

j i
-1 0 2 3 5 6 4 9 40 31
0 1 2 3 4 5 6 7 8 9

sorted not sorted

void insertion(int a[], int n) {

for (int i = 1; i < n; i++) {
int temp = a[i]; store element i

int j = i-1;
while (j >= 0 && temp < a[j]) {
shi the elements until
a[j+1] = a[j]; the right position of
j--; element i is found
}
place the element in
a[j+1] = temp; its right position
}
}
Insertion Sort: Implementation

temp = 4
j i
-1 0 2 3 5 6 4 9 40 31
0 1 2 3 4 5 6 7 8 9

sorted not sorted

void insertion(int a[], int n) {

for (int i = 1; i < n; i++) {
int temp = a[i]; store element i

int j = i-1;
while (j >= 0 && temp < a[j]) {
shi the elements until
a[j+1] = a[j]; the right position of
j--; element i is found
}
place the element in
a[j+1] = temp; its right position
}
}
Insertion Sort: Implementation

temp = 4
j i
-1 0 2 3 5 6 6 9 40 31
0 1 2 3 4 5 6 7 8 9

sorted not sorted

void insertion(int a[], int n) {

for (int i = 1; i < n; i++) {
int temp = a[i]; store element i

int j = i-1;
while (j >= 0 && temp < a[j]) {
shi the elements until
a[j+1] = a[j]; the right position of
j--; element i is found
}
place the element in
a[j+1] = temp; its right position
}
}
Insertion Sort: Implementation

temp = 4
j i
-1 0 2 3 5 6 6 9 40 31
0 1 2 3 4 5 6 7 8 9

sorted not sorted

void insertion(int a[], int n) {

for (int i = 1; i < n; i++) {
int temp = a[i]; store element i

int j = i-1;
while (j >= 0 && temp < a[j]) {
shi the elements until
a[j+1] = a[j]; the right position of
j--; element i is found
}
place the element in
a[j+1] = temp; its right position
}
}
Insertion Sort: Implementation

temp = 4
j i
-1 0 2 3 5 5 6 9 40 31
0 1 2 3 4 5 6 7 8 9

sorted not sorted

void insertion(int a[], int n) {

for (int i = 1; i < n; i++) {
int temp = a[i]; store element i

int j = i-1;
while (j >= 0 && temp < a[j]) {
shi the elements until
a[j+1] = a[j]; the right position of
j--; element i is found
}
place the element in
a[j+1] = temp; its right position
}
}
Insertion Sort: Implementation

temp = 4
j i
-1 0 2 3 5 5 6 9 40 31
0 1 2 3 4 5 6 7 8 9

sorted not sorted

void insertion(int a[], int n) {

for (int i = 1; i < n; i++) {
int temp = a[i]; store element i

int j = i-1;
while (j >= 0 && temp < a[j]) {
shi the elements until
a[j+1] = a[j]; the right position of
j--; element i is found
}
place the element in
a[j+1] = temp; its right position
}
}
Insertion Sort: Implementation

temp =

j i
-1 0 2 3 4 5 6 9 40 31
0 1 2 3 4 5 6 7 8 9

sorted not sorted

void insertion(int a[], int n) {

for (int i = 1; i < n; i++) {
int temp = a[i]; store element i

int j = i-1;
while (j >= 0 && temp < a[j]) {
shi the elements until
a[j+1] = a[j]; the right position of
j--; element i is found
}
place the element in
a[j+1] = temp; its right position
}
}
Insertion Sort: Implementation

j i
-1 0 2 3 4 5 6 9 40 31
0 1 2 3 4 5 6 7 8 9

sorted not sorted

void insertion(int a[], int n) {

for (int i = 1; i < n; i++) {
int temp = a[i]; store element i

int j = i-1;
while (j >= 0 && temp < a[j]) {
shi the elements until
a[j+1] = a[j]; the right position of
j--; element i is found
}
place the element in
a[j+1] = temp; its right position
}
}
Insertion Sort: Implementation

temp = 9
j i
-1 0 2 3 4 5 6 9 40 31
0 1 2 3 4 5 6 7 8 9

sorted not sorted

void insertion(int a[], int n) {

for (int i = 1; i < n; i++) {
int temp = a[i]; store element i

int j = i-1;
while (j >= 0 && temp < a[j]) {
shi the elements until
a[j+1] = a[j]; the right position of
j--; element i is found
}
place the element in
a[j+1] = temp; its right position
}
}
Insertion Sort: Implementation

temp =

j i
-1 0 2 3 4 5 6 9 40 31
0 1 2 3 4 5 6 7 8 9

sorted not sorted

void insertion(int a[], int n) {

for (int i = 1; i < n; i++) {
int temp = a[i]; store element i

int j = i-1;
while (j >= 0 && temp < a[j]) {
shi the elements until
a[j+1] = a[j]; the right position of
j--; element i is found
}
place the element in
a[j+1] = temp; its right position
}
}
Insertion Sort: Implementation

j i
-1 0 2 3 4 5 6 9 40 31
0 1 2 3 4 5 6 7 8 9

sorted not sorted

void insertion(int a[], int n) {

for (int i = 1; i < n; i++) {
int temp = a[i]; store element i

int j = i-1;
while (j >= 0 && temp < a[j]) {
shi the elements until
a[j+1] = a[j]; the right position of
j--; element i is found
}
place the element in
a[j+1] = temp; its right position
}
}
Insertion Sort: Implementation

temp = 40

j i
-1 0 2 3 4 5 6 9 40 31
0 1 2 3 4 5 6 7 8 9

sorted not sorted

void insertion(int a[], int n) {

for (int i = 1; i < n; i++) {
int temp = a[i]; store element i

int j = i-1;
while (j >= 0 && temp < a[j]) {
shi the elements until
a[j+1] = a[j]; the right position of
j--; element i is found
}
place the element in
a[j+1] = temp; its right position
}
}
Insertion Sort: Implementation

temp =

j i
-1 0 2 3 4 5 6 9 40 31
0 1 2 3 4 5 6 7 8 9

sorted not sorted

void insertion(int a[], int n) {

for (int i = 1; i < n; i++) {
int temp = a[i]; store element i

int j = i-1;
while (j >= 0 && temp < a[j]) {
shi the elements until
a[j+1] = a[j]; the right position of
j--; element i is found
}
place the element in
a[j+1] = temp; its right position
}
}
Insertion Sort: Implementation

j i
-1 0 2 3 4 5 6 9 40 31
0 1 2 3 4 5 6 7 8 9

sorted not sorted

void insertion(int a[], int n) {

for (int i = 1; i < n; i++) {
int temp = a[i]; store element i

int j = i-1;
while (j >= 0 && temp < a[j]) {
shi the elements until
a[j+1] = a[j]; the right position of
j--; element i is found
}
place the element in
a[j+1] = temp; its right position
}
}
Insertion Sort: Implementation

temp = 31
j i
-1 0 2 3 4 5 6 9 40 31
0 1 2 3 4 5 6 7 8 9

sorted not sorted

void insertion(int a[], int n) {

for (int i = 1; i < n; i++) {
int temp = a[i]; store element i

int j = i-1;
while (j >= 0 && temp < a[j]) {
shi the elements until
a[j+1] = a[j]; the right position of
j--; element i is found
}
place the element in
a[j+1] = temp; its right position
}
}
Insertion Sort: Implementation

temp = 31
j i
-1 0 2 3 4 5 6 9 40 40
0 1 2 3 4 5 6 7 8 9

sorted not sorted

void insertion(int a[], int n) {

for (int i = 1; i < n; i++) {
int temp = a[i]; store element i

int j = i-1;
while (j >= 0 && temp < a[j]) {
shi the elements until
a[j+1] = a[j]; the right position of
j--; element i is found
}
place the element in
a[j+1] = temp; its right position
}
}
Insertion Sort: Implementation

temp = 31
j i
-1 0 2 3 4 5 6 9 40 40
0 1 2 3 4 5 6 7 8 9

sorted not sorted

void insertion(int a[], int n) {

for (int i = 1; i < n; i++) {
int temp = a[i]; store element i

int j = i-1;
while (j >= 0 && temp < a[j]) {
shi the elements until
a[j+1] = a[j]; the right position of
j--; element i is found
}
place the element in
a[j+1] = temp; its right position
}
}
Insertion Sort: Implementation

temp =

j i
-1 0 2 3 4 5 6 9 31 40
0 1 2 3 4 5 6 7 8 9

sorted not sorted

void insertion(int a[], int n) {

for (int i = 1; i < n; i++) {
int temp = a[i]; store element i

int j = i-1;
while (j >= 0 && temp < a[j]) {
shi the elements until
a[j+1] = a[j]; the right position of
j--; element i is found
}
place the element in
a[j+1] = temp; its right position
}
}
Insertion Sort: Implementation

i
-1 0 2 3 4 5 6 9 31 40
0 1 2 3 4 5 6 7 8 9

sorted

void insertion(int a[], int n) {

for (int i = 1; i < n; i++) {
int temp = a[i]; store element i

int j = i-1;
while (j >= 0 && temp < a[j]) {
shi the elements until
a[j+1] = a[j]; the right position of
j--; element i is found
}
place the element in
a[j+1] = temp; its right position
}
}
void insertion(int a[], int n) { Analysis
for (int i = 1; i < n; i++) { 5 4 3 2 1

int temp = a[i];

int j; Insert 4:
5 →5 3 2 1 1
for (j = i-1; j >= 0 && temp < a[j]; j--) shi
a[j+1] = a[j]; 4 5 3 2 1
a[j+1] = temp; +
} Insert 3:
4 5 → 5 2 1 2
} 4 → 4 5 2 1 shis
3 4 5 2 1
Worst Case. Reversely sorted arrays. +
n−1 Insert 2:
i = 12 n(n − 1)
∑
Data compares. 1 + 2 + 3 + … + (n − 1) = 3 4 5 → 5 1
3 4 → 4 5 1 3
i=1 shis
3 → 3 4 5 1
n−1
2 3 4 5 1
i = 12 n(n − 1)
∑
Number of shis. 1 + 2 + 3 + … + (n − 1) =
i=1 +
Insert 1:
Data moves. Number of shis + 2(n − 1) 2 3 4 5 → 5
For moving a[i] to temp and then temp to a[j+1] 2 3 4 → 4 5 4
2 3 → 3 4 5 shis
2 → 2 3 4 5
Total. O(n 2)
1 3 4 4 5
Best Case. Sorted arrays.
Analysis
Data compares. n − 1 (each element is compared to the one to its le) 1 2 3 4 5
Number of shis. 0 (all elements are in their place) 1 2 3 4 5
Data moves. Number of shis + 2(n − 1) 1 2 3 4 5
For moving a[i] to temp and then back to its place.
1 2 3 4 5
Total. O(n)
1 2 3 4 5

A Good Case. Partially sorted arrays

Total. O(n)
Intuition. If every element is either in its correct position or only a few steps away from it, we need
a few data compares and moves for every element, which makes the total O(n).

Example

1 2 3 5 4 6 7 10 8 9 11 13 12

[Optional Info] Insertion sort performs a number of shis that is equal to the number of
inversions. A sorted array has 0 inversions, a partially sorted array has a number of inversions
1
that is linear in the size of the array and a reversely sorted array has 2 n(n − 1) inversions.
Best Case. Sorted arrays.
Analysis
Data compares. n − 1 (each element is compared to the one to its le) 1 2 3 4 5
Number of shis. 0 (all elements are in their place) 1 2 3 4 5
Data moves. Number of shis + 2(n − 1) 1 2 3 4 5
For moving a[i] to temp and then back to its place.
1 2 3 4 5
Total. O(n)
1 2 3 4 5

A Good Case. Partially sorted arrays

Total. O(n)
Intuition. If every element is either in its correct position or only a few steps away from it, we need
a few data compares and moves for every element, which makes the total O(n).

Average Case. Random arrays.

Claim. Insertion sort requires for sorting a random array around half the amount of data moves
and data compares it needs for sorting a reversely sorted array.
Intuition. If elements are random, then each element moves around half the elements to its le before
1 1 1 1 1
being inserted in its position. I.e. 2 (1)+ 2 (2)+ 2 (3) + … + 2 (n − 1) = 4 n(n − 1) shis.
Bubble Sort: Implementation

i j
8 16 2 4 0 52 91
0 1 2 3 4 5 6

void bubble(int a[], int n) {

for (int i = 0; i < n-1; i++) {
bool swapped = false;
for (int j = n-1; j > i; j--) { compare adjacent
if (a[j] < a[j-1]) { elements and swap if
swap(a[j], a[j-1]); not in order
swapped = true;
}
}
if (!swapped)
break;
}
}
Bubble Sort: Implementation

i j-1 j
8 16 2 4 0 52 91
0 1 2 3 4 5 6

void bubble(int a[], int n) {

for (int i = 0; i < n-1; i++) {
bool swapped = false;
for (int j = n-1; j > i; j--) { compare adjacent
if (a[j] < a[j-1]) { elements and swap if
swap(a[j], a[j-1]); not in order
swapped = true;
}
}
if (!swapped)
break;
}
}
Bubble Sort: Implementation

i j-1 j
8 16 2 4 0 52 91
0 1 2 3 4 5 6

void bubble(int a[], int n) {

for (int i = 0; i < n-1; i++) {
bool swapped = false;
for (int j = n-1; j > i; j--) { compare adjacent
if (a[j] < a[j-1]) { elements and swap if
swap(a[j], a[j-1]); not in order
swapped = true;
}
}
if (!swapped)
break;
}
}
Bubble Sort: Implementation

i j-1 j
8 16 2 4 0 52 91
0 1 2 3 4 5 6

void bubble(int a[], int n) {

for (int i = 0; i < n-1; i++) {
bool swapped = false;
for (int j = n-1; j > i; j--) { compare adjacent
if (a[j] < a[j-1]) { elements and swap if
swap(a[j], a[j-1]); not in order
swapped = true;
}
}
if (!swapped)
break;
}
}
Bubble Sort: Implementation

i j-1 j
8 16 2 0 4 52 91
0 1 2 3 4 5 6

void bubble(int a[], int n) {

for (int i = 0; i < n-1; i++) {
bool swapped = false;
for (int j = n-1; j > i; j--) { compare adjacent
if (a[j] < a[j-1]) { elements and swap if
swap(a[j], a[j-1]); not in order
swapped = true;
}
}
if (!swapped)
break;
}
}
Bubble Sort: Implementation

i j-1 j
8 16 2 0 4 52 91
0 1 2 3 4 5 6

void bubble(int a[], int n) {

for (int i = 0; i < n-1; i++) {
bool swapped = false;
for (int j = n-1; j > i; j--) { compare adjacent
if (a[j] < a[j-1]) { elements and swap if
swap(a[j], a[j-1]); not in order
swapped = true;
}
}
if (!swapped)
break;
}
}
Bubble Sort: Implementation

i j-1 j
8 16 0 2 4 52 91
0 1 2 3 4 5 6

void bubble(int a[], int n) {

for (int i = 0; i < n-1; i++) {
bool swapped = false;
for (int j = n-1; j > i; j--) { compare adjacent
if (a[j] < a[j-1]) { elements and swap if
swap(a[j], a[j-1]); not in order
swapped = true;
}
}
if (!swapped)
break;
}
}
Bubble Sort: Implementation

i j-1 j
8 16 0 2 4 52 91
0 1 2 3 4 5 6

void bubble(int a[], int n) {

for (int i = 0; i < n-1; i++) {
bool swapped = false;
for (int j = n-1; j > i; j--) { compare adjacent
if (a[j] < a[j-1]) { elements and swap if
swap(a[j], a[j-1]); not in order
swapped = true;
}
}
if (!swapped)
break;
}
}
Bubble Sort: Implementation

i j-1 j
8 0 16 2 4 52 91
0 1 2 3 4 5 6

void bubble(int a[], int n) {

for (int i = 0; i < n-1; i++) {
bool swapped = false;
for (int j = n-1; j > i; j--) { compare adjacent
if (a[j] < a[j-1]) { elements and swap if
swap(a[j], a[j-1]); not in order
swapped = true;
}
}
if (!swapped)
break;
}
}
Bubble Sort: Implementation

j-1
i j
8 0 16 2 4 52 91
0 1 2 3 4 5 6

void bubble(int a[], int n) {

for (int i = 0; i < n-1; i++) {
bool swapped = false;
for (int j = n-1; j > i; j--) { compare adjacent
if (a[j] < a[j-1]) { elements and swap if
swap(a[j], a[j-1]); not in order
swapped = true;
}
}
if (!swapped)
break;
}
}
Bubble Sort: Implementation

j-1
i j
0 8 16 2 4 52 91
0 1 2 3 4 5 6

void bubble(int a[], int n) {

for (int i = 0; i < n-1; i++) {
bool swapped = false;
for (int j = n-1; j > i; j--) { compare adjacent
if (a[j] < a[j-1]) { elements and swap if
swap(a[j], a[j-1]); not in order
swapped = true;
}
}
if (!swapped)
break;
}
}
Bubble Sort: Implementation

i j-1 j
0 8 16 2 4 52 91
0 1 2 3 4 5 6

void bubble(int a[], int n) {

for (int i = 0; i < n-1; i++) {
bool swapped = false;
for (int j = n-1; j > i; j--) { compare adjacent
if (a[j] < a[j-1]) { elements and swap if
swap(a[j], a[j-1]); not in order
swapped = true;
}
}
if (!swapped)
break;
}
}
Bubble Sort: Implementation

i j-1 j
0 8 16 2 4 52 91
0 1 2 3 4 5 6

void bubble(int a[], int n) {

for (int i = 0; i < n-1; i++) {
bool swapped = false;
for (int j = n-1; j > i; j--) { compare adjacent
if (a[j] < a[j-1]) { elements and swap if
swap(a[j], a[j-1]); not in order
swapped = true;
}
}
if (!swapped)
break;
}
}
Bubble Sort: Implementation

i j-1 j
0 8 16 2 4 52 91
0 1 2 3 4 5 6

void bubble(int a[], int n) {

for (int i = 0; i < n-1; i++) {
bool swapped = false;
for (int j = n-1; j > i; j--) { compare adjacent
if (a[j] < a[j-1]) { elements and swap if
swap(a[j], a[j-1]); not in order
swapped = true;
}
}
if (!swapped)
break;
}
}
Bubble Sort: Implementation

i j-1 j
0 8 16 2 4 52 91
0 1 2 3 4 5 6

void bubble(int a[], int n) {

for (int i = 0; i < n-1; i++) {
bool swapped = false;
for (int j = n-1; j > i; j--) { compare adjacent
if (a[j] < a[j-1]) { elements and swap if
swap(a[j], a[j-1]); not in order
swapped = true;
}
}
if (!swapped)
break;
}
}
Bubble Sort: Implementation

i j-1 j
0 8 2 16 4 52 91
0 1 2 3 4 5 6

void bubble(int a[], int n) {

for (int i = 0; i < n-1; i++) {
bool swapped = false;
for (int j = n-1; j > i; j--) { compare adjacent
if (a[j] < a[j-1]) { elements and swap if
swap(a[j], a[j-1]); not in order
swapped = true;
}
}
if (!swapped)
break;
}
}
Bubble Sort: Implementation

j-1
i j
0 8 2 16 4 52 91
0 1 2 3 4 5 6

void bubble(int a[], int n) {

for (int i = 0; i < n-1; i++) {
bool swapped = false;
for (int j = n-1; j > i; j--) { compare adjacent
if (a[j] < a[j-1]) { elements and swap if
swap(a[j], a[j-1]); not in order
swapped = true;
}
}
if (!swapped)
break;
}
}
Bubble Sort: Implementation

j-1
i j
0 2 8 16 4 52 91
0 1 2 3 4 5 6

void bubble(int a[], int n) {

for (int i = 0; i < n-1; i++) {
bool swapped = false;
for (int j = n-1; j > i; j--) { compare adjacent
if (a[j] < a[j-1]) { elements and swap if
swap(a[j], a[j-1]); not in order
swapped = true;
}
}
if (!swapped)
break;
}
}
Bubble Sort: Implementation

i j-1 j
0 2 8 16 4 52 91
0 1 2 3 4 5 6

void bubble(int a[], int n) {

for (int i = 0; i < n-1; i++) {
bool swapped = false;
for (int j = n-1; j > i; j--) { compare adjacent
if (a[j] < a[j-1]) { elements and swap if
swap(a[j], a[j-1]); not in order
swapped = true;
}
}
if (!swapped)
break;
}
}
Bubble Sort: Implementation

i j-1 j
0 2 8 16 4 52 91
0 1 2 3 4 5 6

void bubble(int a[], int n) {

for (int i = 0; i < n-1; i++) {
bool swapped = false;
for (int j = n-1; j > i; j--) { compare adjacent
if (a[j] < a[j-1]) { elements and swap if
swap(a[j], a[j-1]); not in order
swapped = true;
}
}
if (!swapped)
break;
}
}
Bubble Sort: Implementation

i j-1 j
0 2 8 16 4 52 91
0 1 2 3 4 5 6

void bubble(int a[], int n) {

for (int i = 0; i < n-1; i++) {
bool swapped = false;
for (int j = n-1; j > i; j--) { compare adjacent
if (a[j] < a[j-1]) { elements and swap if
swap(a[j], a[j-1]); not in order
swapped = true;
}
}
if (!swapped)
break;
}
}
Bubble Sort: Implementation

i j-1 j
0 2 8 4 16 52 91
0 1 2 3 4 5 6

void bubble(int a[], int n) {

for (int i = 0; i < n-1; i++) {
bool swapped = false;
for (int j = n-1; j > i; j--) { compare adjacent
if (a[j] < a[j-1]) { elements and swap if
swap(a[j], a[j-1]); not in order
swapped = true;
}
}
if (!swapped)
break;
}
}
Bubble Sort: Implementation

j-1
i j
0 2 8 4 16 52 91
0 1 2 3 4 5 6

void bubble(int a[], int n) {

for (int i = 0; i < n-1; i++) {
bool swapped = false;
for (int j = n-1; j > i; j--) { compare adjacent
if (a[j] < a[j-1]) { elements and swap if
swap(a[j], a[j-1]); not in order
swapped = true;
}
}
if (!swapped)
break;
}
}
Bubble Sort: Implementation

j-1
i j
0 2 4 8 16 52 91
0 1 2 3 4 5 6

void bubble(int a[], int n) {

for (int i = 0; i < n-1; i++) {
bool swapped = false;
for (int j = n-1; j > i; j--) { compare adjacent
if (a[j] < a[j-1]) { elements and swap if
swap(a[j], a[j-1]); not in order
swapped = true;
}
}
if (!swapped)
break;
}
}
Bubble Sort: Implementation

i j-1 j
0 2 4 8 16 52 91
0 1 2 3 4 5 6

void bubble(int a[], int n) {

for (int i = 0; i < n-1; i++) {
bool swapped = false;
for (int j = n-1; j > i; j--) { compare adjacent
if (a[j] < a[j-1]) { elements and swap if
swap(a[j], a[j-1]); not in order
swapped = true;
}
}
if (!swapped)
break;
}
}
Bubble Sort: Implementation

i j-1 j
0 2 4 8 16 52 91
0 1 2 3 4 5 6

void bubble(int a[], int n) {

for (int i = 0; i < n-1; i++) {
bool swapped = false;
for (int j = n-1; j > i; j--) { compare adjacent
if (a[j] < a[j-1]) { elements and swap if
swap(a[j], a[j-1]); not in order
swapped = true;
}
}
if (!swapped)
break;
}
}
Bubble Sort: Implementation

j-1
i j
0 2 4 8 16 52 91
0 1 2 3 4 5 6

void bubble(int a[], int n) {

for (int i = 0; i < n-1; i++) {
bool swapped = false;
for (int j = n-1; j > i; j--) { compare adjacent
if (a[j] < a[j-1]) { elements and swap if
swap(a[j], a[j-1]); not in order
swapped = true;
}
}
if (!swapped)
break;
}
}
Bubble Sort: Implementation

i No Swaps!
This means that the
0 2 4 8 16 52 91 remaining elements
are already sorted
0 1 2 3 4 5 6

void bubble(int a[], int n) {

for (int i = 0; i < n-1; i++) {
bool swapped = false;
for (int j = n-1; j > i; j--) { compare adjacent
if (a[j] < a[j-1]) { elements and swap if
swap(a[j], a[j-1]); not in order
swapped = true;
}
}
if (!swapped)
break;
}
}
Bubble Sort: Implementation

i No Swaps!
This means that the
0 2 4 8 16 52 91 remaining elements
are already sorted
0 1 2 3 4 5 6

void bubble(int a[], int n) {

for (int i = 0; i < n-1; i++) {
bool swapped = false;
for (int j = n-1; j > i; j--) { compare adjacent
if (a[j] < a[j-1]) { elements and swap if
swap(a[j], a[j-1]); not in order
swapped = true;
}
}
if (!swapped)
break;
}
}
 void bubble(int a[], int n) { Analysis
for (int i = 0; i < n-1; i++) {
bool swapped = false;
for (int j = n-1; j > i; j--) {
if (a[j] < a[j-1]) {
swap(a[j], a[j-1]);
swapped = true;
}
}
if (!swapped)
break;
}
}
Worst Case. Reversely sorted arrays. n−1
i = 12 n(n − 1)
∑
Data compares. (n − 1) + (n − 2) + … + 3 + 2 + 1 =
i=1
Data moves. Swap with every compare = 3 × 12 n(n − 1)
Total. O(n 2)

Best Case. Sorted arrays.

Only one iteration of the outer loop (0 swaps and n − 1 data compares) = O(n)
 Best Worst Random Data Partially Sorted
DC DM DC DM DC DM DC DM
Bubble
Insertion
Selection
 Best Worst Random Data Partially Sorted
DC DM DC DM DC DM DC DM

O(n) O(1) O(n 2) O(n 2) 1

2
n(n − 1)
3
4
n(n − 1)
No general answer.
Bubble

O(n) It depends on when

O(n 2) the swapped flag
Sorted Arrays Reversely Sorted O(n 2) remains false
assuming the Arrays
swapped flag is used

1 1
O(n) O(n) O(n ) 2 O(n 2) n(n − 1) n(n − 1) O(n) O(n)
Insertion

4 4
shis
O(n) O(n 2)
O(n)
Sorted Arrays Reversely Sorted O(n 2)
Arrays

O(n 2) O(1) O(n 2) O(n) 1

n(n − 1) O(n) O(n 2) O(n)
Selection

O(n 2) O(n 2) O(n 2) O(n 2)

Sorted Arrays
 Best Worst Random Data Partially Sorted
DC DM DC DM DC DM DC DM

O(n) O(1) O(n 2) O(n 2) 1

2
n(n − 1)
3
4
n(n − 1)
No general answer.
Bubble

O(n) It depends on when

O(n 2) the swapped flag
Sorted Arrays Reversely Sorted O(n 2) remains false
assuming the Arrays
swapped flag is used

1 1
O(n) O(n) O(n ) 2 O(n 2) n(n − 1) n(n − 1) O(n) O(n)
Insertion

4 4
shis
O(n) O(n 2)
O(n)
Sorted Arrays Reversely Sorted O(n 2)
Arrays

O(n 2) O(1) O(n 2) O(n) 1

n(n − 1) O(n) O(n 2) O(n)
Selection

O(n 2) O(n 2) O(n 2) O(n 2)

Sorted Arrays

The overall running time for all of these algorithms is

asymptotically the same in the worst case
 Best Worst Random Data Partially Sorted
DC DM DC DM DC DM DC DM

O(n) O(1) O(n 2) O(n 2) 1

2
n(n − 1)
3
4
n(n − 1)
No general answer.
Bubble

O(n) It depends on when

O(n 2) the swapped flag
Sorted Arrays Reversely Sorted O(n 2) remains false
assuming the Arrays
swapped flag is used

1 1
O(n) O(n) 2
O(n ) O(n 2) n(n − 1) n(n − 1) O(n) O(n)
Insertion

4 4
shis
O(n) O(n 2)
O(n)
Sorted Arrays Reversely Sorted O(n 2)
Arrays

O(n 2) O(1) O(n 2) O(n) 1

n(n − 1) O(n) O(n 2) O(n)
Selection

O(n 2) O(n 2) O(n 2) O(n 2)

Sorted Arrays

Insertion Sort is expected to be a

bit more efficient on random data
 Best Worst Random Data Partially Sorted
DC DM DC DM DC DM DC DM

O(n) O(1) O(n 2) O(n 2) 1

2
n(n − 1)
3
4
n(n − 1)
No general answer.
Bubble

O(n) It depends on when

O(n 2) the swapped flag
Sorted Arrays Reversely Sorted O(n 2) remains false
assuming the Arrays
swapped flag is used

1 1
O(n) O(n) O(n ) 2 O(n 2) n(n − 1) n(n − 1) O(n) O(n)
Insertion

4 4
shis
O(n) O(n 2)
O(n)
Sorted Arrays Reversely Sorted O(n 2)
Arrays

O(n 2) O(1) O(n 2) O(n) 1

n(n − 1) O(n) O(n 2) O(n)
Selection

O(n 2) O(n 2) O(n 2) O(n 2)

Sorted Arrays

Insertion Sort is the winner

on partially sorted data
 Best Worst Random Data Partially Sorted
DC DM DC DM DC DM DC DM

O(n) O(1) O(n 2) O(n 2) 1

2
n(n − 1)
3
4
n(n − 1)
No general answer.
Bubble

O(n) It depends on when

O(n 2) the swapped flag
Sorted Arrays Reversely Sorted O(n 2) remains false
assuming the Arrays
swapped flag is used

1 1
O(n) O(n) O(n ) 2 O(n 2) n(n − 1) n(n − 1) O(n) O(n)
Insertion

4 4
shis
O(n) O(n 2)
O(n)
Sorted Arrays Reversely Sorted O(n 2)
Arrays

O(n 2) O(1) O(n 2) O(n) 1

n(n − 1) O(n) O(n 2) O(n)
Selection

O(n 2) O(n 2) O(n 2) O(n 2)

Sorted Arrays

Selection Sort is the only algorithm that does a linear

number of data moves in the worst case.
 Best Worst Random Data Partially Sorted
DC DM DC DM DC DM DC DM

O(n) O(1) O(n 2) O(n 2) 1

2
n(n − 1)
3
4
n(n − 1)
No general answer.
Bubble

O(n) It depends on when

O(n 2) the swapped flag
Sorted Arrays Reversely Sorted O(n 2) remains false
assuming the Arrays
swapped flag is used

1 1
O(n) O(n) O(n ) 2 O(n 2) n(n − 1) n(n − 1) O(n) O(n)
Insertion

4 4
shis
O(n) O(n 2)
O(n)
Sorted Arrays Reversely Sorted O(n 2)
Arrays

O(n 2) O(1) O(n 2) O(n) 1

n(n − 1) O(n) O(n 2) O(n)
Selection

O(n 2) O(n 2) O(n 2) O(n 2)

Sorted Arrays

Selection Sort is the only algorithm that does a

quadratic number of operations in the best case.
Advanced Exercises
Exercise # 1

Q. Consider an organ-pipe array made of two equal halves of size m each,

where elements increase then decrease:

1 2 3 4 5 6 7 8 8 7 6 5 4 3 2 1
m m

How many data compares does selection sort perform if run on such an array of size 2m ?
Exercise # 1

Q. Consider an organ-pipe array made of two equal halves of size m each,

where elements increase then decrease:

1 2 3 4 5 6 7 8 8 7 6 5 4 3 2 1
m m

How many data compares does selection sort perform if run on such an array of size 2m ?

1
Answer. Selection sort always does 2
n(n − 1) data compares if the array is of size n,
regardless of how the elements are ordered in the array.
1
e size of the array is 2m. erefore, selection sort performs 2
2m(2m − 1)
= m(2m − 1) = 2m 2 − m data compares.
Exercise # 2

Q. Consider an organ-pipe array made of two equal halves of size m each,

where elements increase then decrease:

1 2 3 4 5 6 7 8 8 7 6 5 4 3 2 1
m m

How many swaps does bubble sort perform if run on such an array of size 2m ?
Exercise # 2

Q. Consider an organ-pipe array made of two equal halves of size m each,

where elements increase then decrease:

1 2 3 4 5 6 7 8 8 7 6 5 4 3 2 1
m m

How many swaps does bubble sort perform if run on such an array of size 2m ?

Answer.
e 1st pass swaps the right-most 1 with 2m − 2 elements.
e 2nd pass swaps the right-most 2 with 2m − 4 elements.
e 3rd pass swaps the right-most 3 with 2m − 6 elements.
…
e right-most 6 is swapped with 4 elements.
e right-most 7 is swapped with 2 elements.
e right-most 8 is swapped with 0 elements. All the remaining elements
will not need extra swaps for them to get to their positions
(swaps from the previous passes of the algorithm get them to their positions).

e total is: 0 + 2 + 4 + 6 + … + (2m − 6) + (2m − 4) + (2m − 2)

= 2(0 + 1 + 2 + 3 + … + (m − 3) + (m − 2) + (m − 1))
1
= 2( 2 m(m − 1)) = m(m − 1) = m 2 − m swaps
Exercise # 3

Q. Consider an organ-pipe array made of two equal halves of size m each,

where elements increase then decrease:
1 2 3 4 5 6 7 8 8 7 6 5 4 3 2 1
m m
How many data compares does insertion sort perform if run on such an array of size 2m ?

Answer.
First half: m-1 compares. Each element is compared to the one to its le.
Second half: e 8 is compared to the 8 to its le (1 compare).
e 7 is compared to the 7, 8, 8 to its le (3 compares).
e 6 is compared to the 6, 7, 7, 8, 8 to its le (5 compares).
…
Finally, the 1 is compared to all the remaining 2m-1 elements.
e total is: 1 + 3 + 5 + ... + 2m-1
= (0+1) + (2+1) + (4+1) + ... + 2m-2+1
= m + 0 + 2 + 4 + ... + 2m-2
= m + 2(0 + 1 + 2 + ... + m-1)
= m + m(m-1) = m2

Adding the compares from the first half, we get a total of m 2 + m − 1 compares.
Exercise # 4

Q. Assume that selection sort knows how to find the minimum in a range of size m
in log2 m comparisons only. What would be the order of growth of the running time
of selection sort if run on an array of size n ?

A. O(n 2 log n) selection-sort(a[], n):

for every i from 0 to n-1:
B. O(n log n)
find the minimum from i to n-1
place the minimum at index i
C. O(n log m)

D. It is impossible to find the

minimum in logarithmic time.
Exercise # 4

Q. Assume that selection sort knows how to find the minimum in a range of size m
in log2 m comparisons only. What would be the order of growth of the running time
of selection sort if run on an array of size n ?

A. O(n 2 log n) selection-sort(a[], n):

for every i from 0 to n-1:
B. O(n log n)
find the minimum from i to n-1
place the minimum at index i
C. O(n log m)

D. It is impossible to find the

minimum in logarithmic time.

Total = log2(n − 1) + log2(n − 2) + log2(n − 3) + … + log2(3) + log2(2) + log2(1)

≤ log2(n!) = O(n log n)

Exercise # 5

Q. Assume that insertion sort uses binary search to find the insertion position in the
sorted portion of the array. Does this affect the worst case running time of the
algorithm?
insertion-sort(a[], n):
A. No.
for every i from 1 to n-1:
B. Affects the actual running insert a[i] in the range 0 to i-1
time but not the using linear search and shifts
asymptotic running time.

C. Affects both the actual and binary-insertion-sort(a[], n):

asymptotic running times.
for every i from 1 to n-1:
pos = binary_search(a, a[i], 0, i-1)
insert(a, a[i], pos, i-1)
Exercise # 5

Q. Assume that insertion sort uses binary search to find the insertion position in the
sorted portion of the array. Does this affect the worst case running time of the
algorithm?
insertion-sort(a[], n):
A. No.
for every i from 1 to n-1:
B. Affects the actual running insert a[i] in the range 0 to i-1
time but not the using linear search and shifts
asymptotic running time.

C. Affects both the actual and binary-insertion-sort(a[], n):

asymptotic running times.
for every i from 1 to n-1:
pos = binary_search(a, a[i], 0, i-1)
insert(a, a[i], pos, i-1)

Number of data compares becomes: O(lg(1) + lg(2) + lg(3) + … + lg(n − 1)) = O(n log n)
Number of data moves remains O(n 2)

Total = O(n log n) + O(n 2) = O(n 2) instead of O(n 2) + O(n 2) = O(n 2)