Lecture 2: Complexity Analysis

Malaka Walpola
 Analyzing Algorithms
 Analysis of Insertion Sort
 Analysis of Bubble Sort
 Asymptotic Notation

2
 After successfully studying contents covered
in this lecture, students should be able to,
 explain the major factors considered for
analyzing algorithms
 explain the use of asymptotic analysis to
examine the growth of functions
 express the time complexity of simple (non
recursive) algorithms using asymptotic notation

3
 What are the factors that affect the running
time of a program?
 The processing
 CPU speed

 Memory

 Size of input data set

 Nature of input data set

4
 Factors to Consider
 Speed/Amount of work done/Efficiency
 Space efficiency/ Amount of memory used

 Simplicity

 We want an analysis that does not depend

on
 CPU speed
 Memory

5
 Why?
 We want to have a hardware independent
analysis
 Thus, we use Asymptotic Analysis
 Ignore machine dependant constants
 Look at the growth of the running time

 Running time of an algorithm as a function of input

size n for large n
 Written using Asymptotic Notation

6
 Kinds of Analysis
 Worst-case: (usually)
 Average-case: (sometimes)

 Need assumption of statistical distribution of inputs.

 Best-case: (bogus?)

7
 Code Cost Times
INSERTION-SORT(A) ---- ---
1. for j = 2 to A.length C1 n
2. key = A[j] C2 n-1
3. //………… 0 n-1
4. i = j-1 C4 n-1
n
5. while i > 0 and A[i] > key C5 t
j 2
j

 t  1
n
6. A[i+1] = A[i] C6 j
j 2

 t  1
n

7. i = i-1 C7 j 2
j

8. A[i+1] = key C8 n-1

 n = A.length
 tj = the number of times the while loop test in line 5 is executed
8
 n
 T (n)  c1n  c2 (n  1)  c4 (n  1)  c5  t j
j 2

 c6  t j  1  c7  t j  1  c8 (n  1)
n n

j 2 j 2

 Best Case – Sorted Array

 tj = 1 for j=2, 3, ……, n

9
 n

T (n)  c1n  c2 (n  1)  c4 (n  1)  c5  t j
j 2

 c6  t j  1  c7  t j  1  c8 (n  1)
n n

j 2 j 2

 Worst Case – Reverse Sorted Array

 tj = j for j=2, 3, ……, n
n
n(n  1) n
n(n  1)
 j 1 &  ( j  1)  1
j 2 2 j 2 2

10
 Code Cost Times
BUBBLE-SORT(A)
1. do
2. swapped ← false
3. for i = 2 to A.length
4. if A[i-1] > A[i]
5. temp ← A[i]
6. A[i] ← A[i-1]
7. A[i-1] ← temp
8. swapped ← true
9. while swapped

11
 Θ-Notation
 Asymptotically tight bound
 Θ(g(n)) = {f(n) : there exist positive constants
c1, c2, and n0 such that 0 ≤ c1g(n) ≤ f(n) ≤ c2g(n)
for all n ≥ n0}

12
 O-Notation
 Asymptotic upper bound
 O(g(n)) = {f(n) : there exist positive constants c,
and n0 such that 0 ≤ f(n) ≤ cg(n) for all n ≥ n0}
 A General Guideline
 Ignore lower order terms
 Ignore leading constants

13
f(x) = 3x f(x) = 4+7x
g(x) = x g(x) = x 8*g(x)
C = 3, N =0 C = 8, N =4
f(x) ≤ 3*g(x) f(x) ≤ 8*g(x)
f(x) = 4+7x
f(x) = 3x = 3g(x)

g(x) = x
g(x) = x
N
3x = O(x) 4+7x = O(x)
14
f(x) = 3x f(x) = 4+7x
g(x) = x g(x) = x
C1 = 1, N =0, C2 = 3 C1 = 7, N =4, C2 = 8
Show Show
g(x) ≤ f(x) ≤ 3*g(x) 7*g(x) ≤ f(x) ≤ 8*g(x)
g(x) = x < f(x) 7*g(x) = 7x < f(x)
f(x) = 3x = 3g(x) f(x) = 7x+4 ≤ 8x (when x≥4)
Therefore Therefore
f(x) = 3x = θ(x) = θ(g(x))
f(x) = 7x+4 = θ(x) = θ(g(x))

15
f(x) = x+100x2 f(x) = 20+x+5x2+8x3
g(x) = x2 g(x) = x3
Show that Show that
f(x) = O(g(x)) f(x) = O(g(x))

16
f(x) = x+100x2 f(x) = 20+x+5x2+8x3
g(x) = x2 g(x) = x3
Show that Show that
f(x) = θ(g(x)) f(x) = θ(g(x))

17
 Example n
T ( n)  c1n  c2 ( n  1)  c4 ( n  1)  c5  t j
j 2

 c6  t j  1  c7  t j  1  c8 ( n  1)
n n

j 2 j 2

 Macro Substitution
 Convention: A set in a formula represents an
anonymous function in the set
 f(n) = n + O(n )
3 2

 f(n) = n3 + h(n) for some h(n) ∈ O(n2)

18
 O-Notation
 May or may not be asymptotically tight
 Other Notations (Self-Study)
 Ω-notation
 Asymptotic lower bound
 May or may not be asymptotically tight

 o-notation
 Asymptotic upper bound (Not asymptotically tight)
 ω-notation
 Asymptotic lower bound (Not asymptotically tight)

19
 Study the Ω, o and ω asymptotic notations
 What are the relationships between the Θ, O, Ω, o
and ω notations?
 Study the optimized Bubble sort algorithms
given in the slides
 Analyze the optimized versions of Bubble sort for
worst case time complexity
 Is there a difference in worst case time complexity?
 Is there an easy method to analyze only the worst
case time complexity?

20
OPTIMIZED-BUBBLE-SORT(A)
1. for j = A.length to 2
2. swapped = false
3. for i = 2 to j
4. if A[i-1] > A[i]
5. temp = A[i]
6. A[i] = A[i-1]
7. A[i-1] = temp
8. swapped = true
9. if (!swapped)
10. break;

21
OPTIMIZED-BUBBLE-SORT(A)
1. n = A.length
2. do
3. swapped = false
4. for i = 2 to n
5. if A[i-1] > A[i]
6. temp = A[i]
7. A[i] = A[i-1]
8. A[i-1] = temp
9. swapped = true
10. newLimit = i-1
11. n = newLimit
12. while swapped

22
[1] T.H. Cormen, C.E. Leiserson, R.L. Rivest and C. Stein, Introduction to
Algorithms, 3rd Ed. Cambridge, MA, MIT Press, 2009.
[2] S. Baase and Allen Van Gelder, Computer Algorithms: Introduction to
Design and Analysis, 3rd Ed. Delhi, India, Pearson Education, 2000.
[3] Wikipedia , “Insertion sort”, Oct. 09, 2012. [Online].
http://en.wikipedia.org/wiki/Insertion_sort. [Accessed: Oct. 22,
2012].
[4] Wikipedia , “Bubble sort”, Oct. 17, 2012. [Online].
http://en.wikipedia.org/wiki/Bubble_sort. [Accessed: Oct. 22, 2012].
[5] Lecture slides from Prof. Erik Demaine of MIT, available at
http://dspace.mit.edu/bitstream/handle/1721.1/37150/6-046JFall-
2004/NR/rdonlyres/Electrical-Engineering-and-Computer-
Science/6-046JFall-2004/B3727FC3-625D-4FE3-A422-
56F7F07E9787/0/lecture_01.pdf

23
 What we covered today
 IA –Sections 3.1
 Topic-wise

 Asymptotic Notation: Section 3.1 of IA

 Next class
 Recursion and Divide & Conquer Approaches
and Merge Sort: Section 2.3.1

24