0% found this document useful (0 votes)
67 views15 pages

Insertion Sort: Analysis of Complexity: Georgy Gimel'farb

Uploaded by

Pavan Kumar
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
0% found this document useful (0 votes)
67 views15 pages

Insertion Sort: Analysis of Complexity: Georgy Gimel'farb

Uploaded by

Pavan Kumar
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 15

Outline Worst-case Average-case Inversions More Θ(n2 ) sorts

Insertion Sort: Analysis of Complexity

Georgy Gimel’farb

COMPSCI 220 Algorithms and Data Structures

1 / 15
Outline Worst-case Average-case Inversions More Θ(n2 ) sorts

1 Worst-case complexity of insertion sort

2 Average-case, or expected complexity of insertion sort

3 Analysis of inversions

4 Selection and bubble sort of complexity Θ(n2 )

2 / 15
Outline Worst-case Average-case Inversions More Θ(n2 ) sorts

Analysing Complexity of Insertion Sort


Iterative growth of a head (“sorted” sublist) of a list A:
a[0] a[1] . . . a[i − 1] a[i] a[i + 1] . . . a[n − 1]
| {z }| {z }
Head (sorted sublist) of size i Tail (unsorted sublist) of size n−i

n − 1 iterations (stages) i = 1, 2, . . . , n − 1;
j; 1 ≤ j ≤ i, comparisons and j or j − 1 moves per stage:
1 Initialisation: the head sublist of size 1.
2 Iteration: until the tail sublist is empty, repeat:
1 Choose the first element, x = a[i] in the tail sublist.
2 Find the last element, y = a[j]; 1 ≤ j ≤ i − 1, in the head
sublist not exceeding x.
3 Insert x after y in the head sublist.

Insertion sort is correct, since the head sublist is always sorted,


and eventually expands to include all elements of A.
3 / 15
Outline Worst-case Average-case Inversions More Θ(n2 ) sorts

Best- and Worst-case Complexity of Insertion Sort

The first element, a[i], of the tail is moved to the correct position
in the head by exhaustive backward search, comparing it to each
element, a[i − 1], . . ., of the head until finding the right place.

The best case, Θ(n): if the inputs A are already in sorted order:
a[0] < a[1] < . . . < a[n − 1], i.e. A = {1, 2, 3, 4}.
• One comparison and no moves per stage i; i = 1, . . . , n − 1.
• Comparisons in total: 1 + 1 + . . . + 1 = n − 1 ∈ Θ(n).
The worst case, Θ(n2 ): if the inputs A contain distinct items in
reverse order: a[0] > a[1] > . . . > a[n − 1], i.e. A = {4, 3, 2, 1}
• i comparisons and i moves per stage i; i = 1, . . . , n − 1.
• Comparisons in total:
2
1 + 2 + . . . + n − 1 = (n−1)n
2 = n 2−n ∈ Θ(n2 ).

4 / 15
Outline Worst-case Average-case Inversions More Θ(n2 ) sorts

Average-case Complexity of Insertion Sort

Lemma 2.3, p.30


The average-case time complexity of insertion sort is Θ(n2 )

The proof’s outline:


• Assuming all possible inputs are equally likely, evaluate the
average, or expected number C i of comparisons at each stage
i = 1, . . . , n − 1.
n−1
P
• Calculate the average total number C = C i.
i=1
• Evaluate the average-case complexity of insertion sort by
taking into account that the total number of data moves is at
least zero and at most the number of comparisons.

5 / 15
Outline Worst-case Average-case Inversions More Θ(n2 ) sorts

Average Complexity of Insertion Sort at Stage i


i + 1 positions in the already ordered head a[0], . . . , a[i − 1] of a
list A to insert the next unordered yet item a[i]:

a[0] a[1] a[2] ... a[i − 1] a[i]


↑j=0 ↑1 ↑2 ↑... ↑i−1 ↑j=i
Ci:0 = i Ci:1 = i Ci:2 = i − 1. . . Ci:i−1 = 2 Ci:i = 1
Mi:0 = i Mi:1 = i − 1 Mi:2 = i − 2 . . . Mi:i−1 = 1 Mi:i = 0

Ci:j = i − j + 1 comparisons and Mi:j = i − j moves to place a[i] into


each preceding position j = i, i − 1, . . . , 1.
• Ci:i = i comparisons and Mi:i = i moves for j = 0.
1
Pi
Average number, C i = i+1 j=0 Ci:j , of comparisons at stage i:

i(i+1)  
1 + 2 + ... + i + i 2+i i i i 1
Ci = = = + ≡ + 1−
i+1 i+1 2 i+1 2 i+1
6 / 15
Outline Worst-case Average-case Inversions More Θ(n2 ) sorts

Total Average Complexity for n Input Items


The total average number of comparisons for n − 1 stages:

C1 C2 C n−1
z
 }| { z }| { z
 }|
 {
1 1 2 1 n−1 1
C = + 1− + + 1− +... + + 1−
2 2 2 3 2 n
1
= 2 (1 + 2 + . . . + (n − 1)) +
| {z }
(n−1)n
  2   
1 1 1
1− + 1− + ... + 1 −
2 3 n
| {z }
(n−1)−(Hn −1)=n−Hn

(n−1)n
= 4 + n − Hn ∈ Θ(n2 )
n
1
P
where Hn = i ≈ ln n when n → ∞ is the n-th harmonic number.
i=1
7 / 15
Outline Worst-case Average-case Inversions More Θ(n2 ) sorts

Math Appendix: Evaluating Harmonic Numbers


n
P 1 1 1 1
Hn = i =1+ 2 + 3 + ... + i
i=1

1 Rn dx
Hn > x = ln n > Hn − 1
1

0.75 1 + ln n > Hn > ln n ⇒ Hn = Θ(log n)

0.5

0.25

0 1 2 3 4 5 6 7 8 9 10
8 / 15
Outline Worst-case Average-case Inversions More Θ(n2 ) sorts

Analysis of Inversions

The running time of insertion sort is strongly related to inversions


in a list A to be sorted.
Definition 2.5: An inversion in a list A = [a1 , a2 , . . . , an ] is any
ordered pair of positions (i, j) such that i < j but ai > aj .

Examples of inversions: [. . . , 2, . . . , 1] or [100, . . . , 35, . . .].

List A Number of Reverse list Arev Number of Total


inversions inversions
[3, 2, 5] 1 [5, 2, 3] 2 3
[3, 2, 5, 1] 4 [1, 5, 2, 3] 2 6
[1, 2, 3, 5, 7] 0 [7, 5, 3, 2, 1] 10 10

The number of inversions measures how far a list is from being sorted.

9 / 15
Outline Worst-case Average-case Inversions More Θ(n2 ) sorts

Analysis of Inversions

Number of inversions Ii , comparisons Ci and data moves Mi for


each element a[i] in A:
Element i 0 1 2 3 4 5 6
A 44 13 35 18 15 10 20
Ii 1 1 2 3 5 2 I = 14
Ci 1 2 3 4 5 3 C = 18
Mi 1 1 2 3 5 2 M = 14

n−1
P
Because Ii = Mi is always true, the total number I = Ii of
i=1
n−1
P
inversions is equal to the total number M = Mi of backward
i=1
moves of elements a[i] during the sort.

10 / 15
Outline Worst-case Average-case Inversions More Θ(n2 ) sorts

Analysis of Inversions

n−1
P
The total number of data comparisions C = Ci is also equal to
i=1
the total number of inversions plus at most n − 1.

Total number of inversions in both an arbitrary list A and its


reverse Arev is equal to the total number of the ordered pairs
(i < j) of integers i, j ∈ {1, . . . , n − 1}:
 
n−1 (n − 1)n
=
2 2

• A sorted list has no inversions.


• A reverse sorted list of size n has (n−1)n
2 inversions.
• In the average, all lists of size n have (n−1)n
4 inversions.
11 / 15
Outline Worst-case Average-case Inversions More Θ(n2 ) sorts

Complexity of Insertion Sort by Analysing Inversions


Exactly one inversion is removed by swapping two neighbours
being out of order: ai−1 > ai .
• If an original list has I inversions, insertion sort has to swap I
pairs of neighbours.
• A list with I inversions results in Θ(n + I) running time of
insertionSort because of Θ(n) other operations in the
algorithm.
• In the very rare best case of a nearly sorted list for which I is
Θ(n), insertion sort runs in linear time.
2
• The worst-case time: c n2 , or Θ(n2 ).
2
• The average-case, or expected time: c n4 , or still Θ(n2 ).

More efficient sorting algorithms must eliminate more than


just one inversion between neighbours per swap.

12 / 15
Outline Worst-case Average-case Inversions More Θ(n2 ) sorts

Implementation of Insertion Sort

The number of comparisons does not depend on how the list is


implemented, but the number of moves does.
• Backward moves in an array implementation of a list:
• Shifting elements to the right (linear time per stage) in the
worst and average case, or
• Successive swaps to move the element backward.
• Insertion operation in a linked list implementation of a list:
• Constant-time insertion of an element.
• Fewer swaps by simply scanning backward (but it may take
time for a singly linked list).

None of the implementation issues affect the asymptotic Θ(n2 ) running


time of the algorithm, just the hidden constants and lower order terms,
due to too many comparisons in the worst and average cases.

13 / 15
Outline Worst-case Average-case Inversions More Θ(n2 ) sorts

Quadratic Θ(n2 ) Selection Sort: Java Code


// Selection sort of an input array a of size n:
// building a head by successive minima selection in a tail
//
// Each leftmost unordered a[i] is swapped with the minimum element
// selected among the unordered yet elements a[i+1],...,a[n-1]

public static void selectionSort( int [ ] a ) {


for ( int i = 1; i < a.length - 1; i++ ) {
int posMin = i;
// for-loop for selecting position of the minimum element
for ( int k = i + 1; k < a.length; k++ ) {
if ( a[ posMin ] > a[ k ] ) posMin = k;
}
if ( posMin != i ) swap( a, i, posMin );
// swap a[i] with the minimum element selected
}
}
14 / 15
Outline Worst-case Average-case Inversions More Θ(n2 ) sorts

Quadratic Θ(n2 ) Bubble Sort: Java Code


// Bubble sort of an input array a of size n:
// n - 1 iterations to bubble up the maximum element
// among the unordered yet elements a[0],...,a[i]
//
// Each iteration i performs successive bottom-up swaps of
// the larger element in each adjacent pair of the elements
// for bubbling up the maximal element from a[0],...,a[i]

public static void bubbleSort( int [ ] a ) {


for ( int i = a.length - 1; i > 0; i-- ) {
for ( int k = 0; k < i; k++ ) {
if ( a[ k ] > a[ k + 1 ] )
swap( a, k, k + 1 );
bubble up the larger of the two adjacent elements
}
}
}
15 / 15

You might also like