Computational Complexity

Analyzing algorithms

Pablo Trinidad | @_pablotrinidad_

Efficient?
Why is this relevant?

• Allows to predict future performance

• Help us identify the limits of our computations

• Minimize resources
Empirical methods

• Are not platform-independent

• Depends on the implementation

• We only know what we measure

Search
Algorithm comparison

Given a sorted list of size N, find the

location of an arbitrary entry.
Search
Algorithm comparison

Linear search:
1. Read the elements in the list from left to right
keeping track of the index
2. If the entry is found, return the index
Search
Algorithm comparison

Binary search:
1. Position in the middle of the list (keeping track of the index)

2. If the entry is in the middle, return the index

3. If the value in the middle is greater than the entry, perform

binary search on the left half of the list.

4. Else, perform binary search in the right half of the list.

 Search
Algorithm comparison

Benchmarking:
• A state-of-the-art computer A will run linear search.
• A slower computer B will run binary search.

Source: https://en.wikipedia.org/wiki/Analysis_of_algorithms
 Search
Algorithm comparison

Source: https://en.wikipedia.org/wiki/Analysis_of_algorithms
 Search
Algorithm comparison

Source: https://en.wikipedia.org/wiki/Analysis_of_algorithms
Analysis of algorithms
Assumptions

• Lets define a model of computation on which:

• We have a single processing unit
• Instructions are executed in order, one after the other.
• Each instruction have its own cost.
• Instructions are primitive, no magic procedures exists in the processing unit.
 Running time



Number of primitive operations
executed
Running time

T(n)
Input size
• Depends on the problem being studied.

• Can involve more than one number.

Sample problem
Find if sum is possible

Given a list of numbers L and a number k return whether

any two numbers from the list add up to k.
Sample problem
Find if sum is possible

Given a list of numbers L and a number k return whether

any two numbers from the list add up to k.

Input: Output: YES!

L = [10, 15, 3, 7] Explanation: 10 + 7 = 17 = k

k = 17
Find if sum is possible
Sol #1

SUM-EXISTS(L, k):
1. for i = 0 up to L . length − 1:
2. for j = 0 up to L . length − 1:
3. if i ≠ j and L[i] + L[ j] = k:
4. return YES
5. return NO
Find if sum is possible
Sol #1

SUM-EXISTS(L, k): cost times

1. for i = 0 up to L . length − 1: c1
2. for j = 0 up to L . length − 1: c2
3. if i ≠ j and L[i] + L[ j] = k: c3
4. return YES c4
5. return NO c5
Find if sum is possible
Sol #1

SUM-EXISTS(L, k): cost times

1. for i = 0 up to L . length − 1: c1 n
2. for j = 0 up to L . length − 1: c2 n⋅n=n 2

3. if i ≠ j and L[i] + L[ j] = k: c3 n⋅n=n 2

4. return YES c4 1
5. return NO c5 1
Find if sum is possible
Sol #1

SUM-EXISTS(L, k): cost times

1. for i = 0 up to L . length − 1: c1 n
2. for j = 0 up to L . length − 1: c2 n⋅n=n 2

3. if i ≠ j and L[i] + L[ j] = k: c3 n⋅n=n 2

4. return YES c4 1
5. return NO c5 1

2 2
T(n) = c1n + c2n + c3n + c4 + c5
Find if sum is possible
Sol #1

2 2
T(n) = c1n + c2n + c3n + c4 + c5
2
T(n) = n (c2 + c3) + nc1 + (c4 + c5)
2
T(n) = an + bn + c
Find if sum is possible
Sol #2

SUM-EXISTS_2(L, k):
1. sort(L)
2. for x in L:
3. d←k−x
4. if find(L, d):
5. return YES
6. return NO
Find if sum is possible
Sol #2

SUM-EXISTS_2(L, k): cost

1. sort(L) n ⋅ log(n)
2. for x in L: c2
3. d←k−x c3
4. if find(L, d): log(n)
5. return YES c5
6. return NO c6
Find if sum is possible
Sol #2

SUM-EXISTS_2(L, k): cost times

1. sort(L) n ⋅ log(n) 1
2. for x in L: c2 n
3. d←k−x c3 n
4. if find(L, d): log(n) n
5. return YES c5 1
6. return NO c6 1
Find if sum is possible
Sol #2
SUM-EXISTS_2(L, k): cost times
1. sort(L) n ⋅ log(n) 1
2. for x in L: c2 n
3. d←k−x c3 n
4. if find(L, d): log(n) n
5. return YES c5 1
6. return NO c6 1

T(n) = n ⋅ log(n) + c2n + c3n + n ⋅ log(n) + c5 + c6

Find if sum is possible
Sol #2

T(n) = n ⋅ log(n) + c2n + c3n + n ⋅ log(n) + c5 + c6

T(n) = 2 ⋅ n log(n) + n(c2 + c3n) + (c5 + c6)

T(n) = a ⋅ n log(n) + bn + c
Find if sum is possible

Sol #1 Sol #2
Find if sum is possible

Sol #1
2
T(n) = an + bn + c

Sol #2
T(n) = a ⋅ n log(n) + bn + c
Find if sum is possible
Find if sum is possible
Sol #3

SUM-EXISTS_3(L, k):
1. store ← map[int]bool{}
2. for x in L:
3. if store.contains(x)
4. return YES
5. store.add(k − x)
6. return NO
Find if sum is possible
Sol #3

SUM-EXISTS_3(L, k): Total Cost

1. store ← map[int]bool{} c1
2. for x in L: c2n
3. if store.contains(x) c3n
4. return YES c4
5. store.add(k − x) c5n
6. return NO c6
Find if sum is possible
Sol #3

SUM-EXISTS_3(L, k): Total Cost

1. store ← map[int]bool{} c1
2. for x in L: c2n
3. if store.contains(x) c3n
4. return YES c4
5. store.add(k − x) c5n
6. return NO c6

T(n) = an + b
Find if sum is possible
Sol #1

2
T(n) = an + bn + c
Sol #2

T(n) = a ⋅ n log(n) + bn + c
Sol #3

T(n) = an + b
Order of growth
Cost function Order of growth

13n 2 + 1.2n + 9

1 3
n + 1010n 2 + 1
100

30n − 49

2n + n 10 + n 34 + n 2 + 1
Order of growth
Cost function Order of growth

13n 2 + 1.2n + 9 n2

1 3
n + 1010n 2 + 1 n 3
100

30n − 49 n

n 10 34
2 +n +n +n +1 2 2n
Asymptotic notation
Asymptotic notation

Ω Θ O
Best (lower bound) Range (lower & upper bound) Worst (upper bound)
Asymptotic notation
Big-O notation

For a given function g(n) we define O(g(n)) as the set of

functions:

O(g(n)) = {f(n) : there exist positive constants

c and n0 such that 0 ≤ f(n) ≤ cg(n) ∀n ≥ n0}
 Asymptotic notation
Big-O notation
O(g(n)) = {f(n) : there exist positive constants c
and n0 such that 0 ≤ f(n) ≤ cg(n) ∀n ≥ n0}

Source: Thomas H. Cormen, Charles E. Leiserson, Ronald L. Rivest, and Clifford Stein. 2009. Introduction to Algorithms, Third Edition (3rd. ed.). The MIT Press.
 Asymptotic notation
Big-O notation

Θ O Ω

Source: Thomas H. Cormen, Charles E. Leiserson, Ronald L. Rivest, and Clifford Stein. 2009. Introduction to Algorithms, Third Edition (3rd. ed.). The MIT Press.
Asymptotic notation
Conventions

2 3 n
• Even though O(n) ⊂ O(n ) ⊂ O(n ) ⊂ O(c ) ⊂ . . . is true, we use the
closest bound.

• f(n) ∈ O(g(n)) is abused and expressed as f(n) = O(g(n)).

2 2
• Constants are ignored, i.e: O(3n ) becomes O(n )
• If f(n) is constant, we express it as O(1).
Big-O comparison

Complexity/n

A LOT!
 Big-O comparison

Source: https://www.bigocheatsheet.com/
Big-O comparison
 We want to find efficient
procedures to solve large-scale
problems.
Computational Complexity
bit.ly/pm-comp-x2