DESIGN AND ANALYSIS OF

ALGORITHM

Sorting Algorithm
Prepared by
Murari Kumar
Singh
• Input:

– A sequence of n numbers a1, a2, . . . , an

• Output:

– A permutation (reordering) a1’, a2’, . . . , an’ of the input

sequence such that a1’ ≤ a2’ ≤ · · · ≤ an’

2
Structure of data
 Why Study Sorting Algorithms?
• There are a variety of situations that we can
encounter
– Do we have randomly ordered keys?
– Are all keys distinct?
– How large is the set of keys to be ordered?
– Need guaranteed performance?

• Various algorithms are better suited to some of

these situations
 Some Definitions
• Internal Sort
– The data to be sorted is all stored in the computer’s
main memory.
• External Sort
– Some of the data to be sorted might be stored in
some external, slower, device.
• In Place Sort
– The amount of extra space required to sort the data is
constant with the input size.
 Stability
• A STABLE sort preserves relative order of records with
equal keys

72 6 2 5 4 1
Unsorted List

12 2 4 5 6 7
Sorted List
 Insertion Sort
• Idea: like sorting a hand of playing cards
– Start with an empty left hand and the cards facing
down on the table.
– Remove one card at a time from the table, and insert
it into the correct position in the left hand
• compare it with each of the cards already in the hand, from
right to left
– The cards held in the left hand are sorted
• these cards were originally the top cards of the pile on the
table
 Insertion Sort

To insert 12, we need to

make room for it by moving
first 36 and then 24.
6 10 24 36

12
 Insertion Sort

6 10 24 36

12
 Insertion Sort

6 10 24 3
6

12
 Insertion Sort
input array

5 2 4 6 1
3
at each iteration, the array is divided in two sub-arrays:

left sub-array right sub-array

sorted unsorted
Insertion Sort
 INSERTION-SORT
Algo.: INSERTION-SORT(A) 1 2 3 4 5 6 7 8

a1 a2 a3 a4 a5 a6 a7 a8
for j ← 2 to n
do key ← A[ j ] key
Insert A[ j ] into the sorted sequence A[1 . . j -1]

i←j-1
while i > 0 and A[i] > key
do A[i + 1] ← A[i]
i←i–1
A[i + 1] ← key
• Insertion sort – sorts the elements in place
 Loop Invariant for Insertion Sort

Alg0.: INSERTION-SORT(A)
for j ← 2 to n
do key ← A[ j ]
Insert A[ j ] into the sorted sequence A[1 . . j -1]
i←j-1
while i > 0 and A[i] > key
do A[i + 1] ← A[i]
i←i–1
A[i + 1] ← key

Invariant: at the start of the for loop the elements in A[1 . . j-1]
are in sorted order
 Proving Loop Invariants
• Proving loop invariants works like induction
• Initialization (base case):
– It is true prior to the first iteration of the loop
• Maintenance (inductive step):
– If it is true before an iteration of the loop, it remains true before
the next iteration
• Termination:
– When the loop terminates, the invariant gives us a useful
property that helps show that the algorithm is correct
– Stop the induction when the loop terminates
 Loop Invariant for Insertion Sort
• Initialization:
– Just before the first iteration, j = 2:
the subarray A[1 . . j-1] = A[1],
(the element originally in A[1]) – is
sorted
 Loop Invariant for Insertion Sort
• Maintenance:
– the while inner loop moves A[j -1], A[j -2], A[j -3],
and so on, by one position to the right until the proper
position for key (which has the value that started out in
A[j]) is found
– At that point, the value of key is placed into this
position.
 Loop Invariant for Insertion Sort
• Termination:
– The outer for loop ends when j = n + 1  j-1 = n
– Replace n with j-1 in the loop invariant:
• the subarray A[1 . . n] consists of the elements originally in
A[1 . . n], but in sorted order
j-1 j

• The entire array is sorted!

Invariant: at the start of the for loop the elements in A[1 . . j-1]
are in sorted order
 Analysis of Insertion Sort
INSERTION-SORT(A) cost
for j ← 2 to n times
do key ← A[ j ] c1 n
c2
Insert A[ j ] into the sorted sequence A[1 . . j -1] n-1
i←j-1 0 n-1
 jn-1
n
while i > 0 and A[i] > key c4 t
2 j


n
(t j  1)
do A[i + 1] ← A[i] c5 j 2


n
i←i–1 (t j  1)
c6 j 2

A[i + 1] ← key c7
tj: # of times the while statement is executed at iterationcj n-
n 8
T (n)  c1n  c2 (n  1)  c4 (n  1)  c5  t j  c6  t j  1 c7 1t j  1 c8 (n  1)
n n

j 2 j 2 j 2
 Best Case Analysis
“while i > 0 and A[i] > key”
• The array is already sorted
– A[i] ≤ key upon the first time the while loop test is run
(when i = j -1)

– tj = 1

• T(n) = c1n + c2(n -1) + c4(n -1) + c5(n -1) + c8(n-1) =

(c1 + c2 + c4 + c5 + c8)n + (c2 + c4 + c5 + c8)

= an + b = (n) n
 c6  t j  1 c7  t j  1 c8 (n  1)
n n
T (n)  c n  c (n  1)  c (n  1)  c
1 2 4 5 t
j 2
j
j 2 j 2
 Worst Case Analysis
• The array is in reverse sorted order “while i > 0 and A[i] > key”
– Always A[i] > key in while loop test
– Have to compare key with all elements to the left of the j-th
position  compare with j-1 elements  tj = j
n
n(n  1) n
n(n  1) n
n(n  1)
using 
j 1
j
2
  j 
j 2 2
 1   ( j  1) 
j 2 2
we have:

 n ( n  1)  n ( n  1) n( n  1)
T ( n )  c1n  c2 ( n  1)  c4 ( n  1)  c5   1  c6  c7  c8 ( n  1)
 2  2 2

 an 2  bn  c a quadratic function of n

• T(n) = (n2) order nof growth in n 2

T (n)  c1n  c2 (n  1)  c4 (n  1)  c5  t j  c6  t j  1 c7  t j  1 c8 (n  1)

n n

j 2 j 2 j 2 21
 Comparisons and Exchanges in
Insertion Sort
INSERTION-SORT(A) cost times
c1 n
for j ← 2 to n
c2 n-1
do key ← A[ j ]
0 n-1
Insert A[ j ] into the sorted sequence A[1 . . j
-1] c4 n-1
n 2
/2 comparisons
c5
 t
n
i←j-1 j 2 j
c6
 (t
n
while i > 0 and A[i] > key j 2 j  1)
c7
do A[i + 1] ← A[i]  (t
n
2
n /2 exchanges j 2 j  1)
c8 n-1
i←i–1
A[i + 1] ← key
 Insertion Sort - Summary
• Advantages
– Good running time for “almost sorted” arrays (n)
• Disadvantages
 (n2) running time in worst and average case
  n2/2 comparisons and exchanges
THANK YOU
 Bubble Sort
• Idea:
– Repeatedly pass through the array
– Swaps adjacent elements that are out of order

i
1 2 3 n

8 4 6 9 2 3 1
j

• Easier to implement, but slower than Insertion

sort
 Example
8 4 6 9 2 3 1 1 8 4 6 9 2 3
i=1 j i=2 j

8 4 6 9 2 1 3 1 2 8 4 6 9 3
i=1 j i=3 j

8 4 6 9 1 2 3 1 2 3 8 4 6 9
i=1 j i=4 j

8 4 6 1 9 2 3 1 2 3 4 8 6 9
i=1 j i=5 j

8 4 1 6 9 2 3 1 2 3 4 6 8 9
i=1 j i=6 j

8 1 4 6 9 2 3 1 2 3 4 6 8 9
i=1 j i=7
j
1 8 4 6 9 2 3
i=1 j
 Bubble Sort
Alg.: BUBBLESORT(A)
for i  1 to length[A]
do for j  length[A] downto i + 1
do if A[j] < A[j -1]
then exchange A[j]  A[j-1]
i
8 4 6 9 2 3 1
i=1 j
 Bubble-Sort Running Time
Alg.: BUBBLESORT(A) c1

for i  1 to length[A] c2

do for j  length[A] downto i + 1 c3

Comparisons:  n2/2 do if A[j] < A[j -1]

c4

Exchanges:  n2/2 then exchange A[j]  A[j-1]

n n n

T(n) = c1(n+1) + c2  (n  i  1)  c3  (n  i )  c4  (n  i )
i 1
i 1 i 1
n

= (n) + (c2 + c2 + c4)  (n  i )

i 1
n n n
n ( n  1) n 2
n
where  (n  i )  n   i  n 2   
i 1 i 1 i 1 2 2 2
Thus,T(n) = (n2)
 Selection Sort
• Idea:
– Find the smallest element in the array
– Exchange it with the element in the first position
– Find the second smallest element and exchange it with
the element in the second position
– Continue until the array is sorted
• Disadvantage:
– Running time depends only slightly on the amount of
order in the file
 Example
8 4 6 9 2 3 1 1 2 3 4 9 6 8

1 4 6 9 2 3 8 1 2 3 4 6 9 8

1 2 6 9 4 3 8 1 2 3 4 6 8 9

1 2 3 9 4 6 8 1 2 3 4 6 8 9
 Selection Sort
Alg.: SELECTION-SORT(A)
n ← length[A] 8 4 6 9 2 3 1

for j ← 1 to n - 1
do smallest ← j
for i ← j + 1 to n
do if A[i] < A[smallest]
then smallest ← i
exchange A[j] ↔ A[smallest]
 Analysis of Selection Sort
Alg.: SELECTION-SORT(A) cost times

n ← length[A] c1 1

for j ← 1 to n - 1 c2 n

do smallest ← j c3 n-1
n2/2 c4
comparisons for i ← j + 1 to n 
n 1
j 1
(n  j  1)
c5
do if A[i] < A[smallest] 
n 1
j 1
(n  j )
n c6
then smallest ← i 
n 1
exchanges j 1
(n  j )
c7 n-1
exchange A[j] ↔ A[smallest]
n 1 n 1 n 1
T ( n)  c1  c2 n  c3 ( n  1)  c4  ( n  j  1)  c5   n  j   c6   n  j   c7 (n  1)  (n 2 )
j 1 j 1 j 2
THANK YOU