CMPUT 175

Introduction to Foundations
of Computing
Algorithm Analysis

You should view the vignettes:

Objects and Classes
Post-test Fibonacci sequences

January 22, 2024 © Osmar R. Zaïane : University of Alberta 1

 Objectives
Realizing the difference in performance between
various algorithms solutions to the same problem.
Understanding algorithm analysis and realizing its
importance in programming.
Understanding the notion of “Big-O” used to
describe execution time for an algorithm.
Get a brief introduction to algorithm techniques.
Realizing the importance of right choices for data
structures and methods with implementations
with python.
January 22, 2024 © Osmar R. Zaïane : University of Alberta 2
 What is Algorithm?
Is a step-by-step procedure for solving a certain
problem.
Algorithm is any well-specified computational
procedure that takes some value, or set of
values, as input and produces some value, or set
of values, as output after processing.
Algorithm is thus a sequence of computational
steps that transform the input into the output.
It is like a recipe that describes how to transform
ingredients (input) into a dish (output).

January 22, 2024 © Osmar R. Zaïane : University of Alberta 3

 Example: What is an
Algorithm?
Problem: Input is a sequence of integers stored in a table.
Output: the smallest number.

INPUT Algorithm OUTPUT

An array of numbers The smallest number
m a[1];
25, 90, 53, 23, 11, 34 for i from 2 to size of input 11
if m > a[i] then ma[i];
return m
A sequence of Assume the first element is
instructions expressed in the smallest; store in m;
high level language then traverse the remaining
elements checking if any is
conveying the steps m smaller; store in m;
required to solve a Return m.
problem Data-Structure
January 22, 2024 © Osmar R. Zaïane : University of Alberta 4
 What is a program ?
A program is an implementation of an
algorithm in a programming language.
There are many programming languages
designed to communicate instructions to the
computer.
There are many of such artificial languages,
each with its own intricacies, specificities such
as grammar and syntax. Python is an example.
So a program is a set of instructions which the
computer will follow to solve a problem.
January 22, 2024 © Osmar R. Zaïane : University of Alberta 5
 Program vs. Algorithm
Without an algorithm there can be no program.
There is an algorithm in any program that does
something.
We should 1st express the algorithm in high level
language before writing the program.
An algorithm can be implemented as a program
in any programming language.
Example programming languages:
Old languages: Fortran, PL1, Cobol, Pascal, Basic, C, …
Common languages: Java, C++, Python, Javascript,
perl, …
January 22, 2024 © Osmar R. Zaïane : University of Alberta 6
 Algorithm
m a[1]; var Pascal
for i from 2 to size of input numbers : array[1..6] of integer;
if m > a[i] then ma[i]; m, i: integer;
begin
return m numbers[1]:=25; numbers[2]:=90; numbers[3]:= 53;
numbers[4]:=23; numbers[5]:=11; numbers[6]:=34;
Python m:=numbers[1];
for i:=2 to Length(numbers) do
>>> numbers=[25,90,53,23,11,34] begin
>>> m=numbers[0] if m > numbers[i] then Assembler 8085
begin
>>> for i in range(1,len(numbers)): m:=numbers[i]; LXI H,4200 ; Set pointer for array
if m>numbers[i]: end; MOV B,M ; Load the Count
INX H ; Set 1st element as largest data
m=numbers[i] end;
MOV A,M
writeln(m); DCR B ; Decremented the count
>>> print (m) int main() end;
11 {
C
LOOP: INX H
CMP M ; if A- reg < M go to AHEAD

int numbers[6]= {25,90,53,23,11,34} JC AHEAD

>>> min(numbers) int i, m = numbers[0]; MOV A,M ; Set the new value as smallest
AHEAD:DCR B
11 JNZ LOOP ; Repeat comparisons till coun
for ( i = 1 ; i < sizeof(numbers) ; i++ )
Perl {
STA 4300
HLT
; Store the largest value at 430

@numbers= (25,90,53,23,11,34); if (m > numbers[i])

{ GOAHEAD:
$m=$numbers[0]; m = numbers[i]; ADD SI,2
$m= $_<$m ? $_ : $m foreach (@numbers); INC AX
} CMP AX,6
print $m; } JL TESTMIN
printf(m);
print min(@numbers); return 0;
January 22, 2024 }© Osmar R. Zaïane : University of Alberta 7
m a[1];
Algorithm Visual Basic
for i from 2 to size of input Dim numbers = New Integer() {25,90,53,23,11,34}
if m > a[i] then ma[i];
return m m=numbers(0)
For index = 1 To numbers.GetUpperBound(0)
Python If m > numbers(index) Then
m = numbers(index)
>>> numbers=[25,90,53,23,11,34]
End If
>>> m=numbers[0] Next
>>> for i in range(1,len(numbers)):
if m>numbers[i]: MsgBox(m)
m=numbers[i] <?php PHP
>>> print (m) $numbers = array(25,90,53,23,11,34);
11 $m=$numbers[0];
foreach ($numbers as $value) {
>>> min(numbers) Fortran Java
if ($m > $value) {
11 integer, dimension(6) :: numbers $m = $value; int[] numbers= {25,90,53,23,11,34};
integer :: i, m } int m = numbers[0];
numbers = (/ 25,90,53,23,11,34 /) } for(int i = 1;i<numbers.length;i++)
m= numbers(1) echo $m; {
do i = 2, size(numbers) ?> if(m>numbers[i])
if (m> numbers(i)) then {
m = numbers(i) m = numbers[i];
endif }
end do }
January 22, 2024 © Osmar R. Zaïane : University of Alberta 8
print m System.out.println(m);
 Study of Algorithm
How to analyze algorithms
How to measure the performance of
algorithms
How to analyze an algorithm’s running time
without coding it
How to devise algorithms
Various techniques
How to validate algorithms
How to test programs
January 22, 2024 © Osmar R. Zaïane : University of Alberta 9
 What do we analyze about
them?
Programs consume resources.
Algorithms require time for execution
and space to store the data to be
processed.
Analysis pertains to:
Execution time: time complexity
Memory use: space complexity
Look for the optimal solution: Is it
possible to do better?
January 22, 2024 © Osmar R. Zaïane : University of Alberta 10
 Comparing Algorithms
Compare algorithms in terms of
computing resources that each
algorithm uses
One algorithm is better than the other
because it is more efficient in its use of
resources or uses less resources
Benchmark analysis
Execution time = running time
Memory usage
January 22, 2024 © Osmar R. Zaïane : University of Alberta 11
Problem: summing the first n numbers
def myfunction(something): def sum1(n):
me=0 theSum=0
for alpha in range(1,something+1): for i in range(1,n+1):
beta=me+alpha theSum=theSum+i
me=beta return theSum
return me

Which one is better?

In terms of algorithms they are the same.
One program is more readable than the
other, but their execution provides the same
result. Also 1st one uses extra variable.
January 22, 2024 © Osmar R. Zaïane : University of Alberta 12
Can we express it differently?

1+2=3 Rather than a loop

1+2+3=6 sum0
1 + 2 + 3 + 4 = 10 for i from 1 to n
1 + 2 + 3 + 4 + 5 = 15 sumsum+i
1 + 2 + 3 + 4 + 5 + 6 = 21
1 + 2 + 3 + 4 + 5 + 6 +7 = 28 We can have a formula
1 + 2 + 3 + 4 + 5 + 6 +7 + 8 = 36 sum(n)=sum(n-1)+n
1 + 2 + 3 + 4 + 5 + 6 +7 + 8 + 9 = 45
Or this formula
Sum(n)=n*(n+1)/2

January 22, 2024 © Osmar R. Zaïane : University of Alberta 13

 Benchmarking
def sum2(n): sum(n)=sum(n-1)+n
def sum1(n):
theSum=0 if n==0: return 0
for i in range(1,n+1): else: return sum2(n-1) + n
theSum=theSum+i
def sum3(n): Sum(n)=n*(n+1)/2
return theSum
return (n*(n+1))/2
import time
… Calling 100,000
10 trials times the function
n=990 Start time
totalTime=0.0
for i in range(10):
start=time.time()
for j in range(100000): x=sum2(n)
end time
end=time.time()
print("the sum is %d and it required %10.7f seconds"%(x,end-start))
totalTime=totalTime+end-start Averaging
over 10 trials
print("with sum2 the average time was %10.7f for n=%d"%(totalTime/10,n))

January 22, 2024 © Osmar R. Zaïane : University of Alberta 14

 Benchmarking 2
Running the sum function 100,000 times averaged over 10 trials for
n=990
def sum1(n):
theSum=0
for i in range(1,n+1):
theSum=theSum+i
return theSum
9.76 seconds

def sum2(n):
if n==0: return 0
else: return sum2(n-1) + n
44.48 seconds

0.05 seconds
def sum3(n):
return (n*(n+1))/2
January 22, 2024 © Osmar R. Zaïane : University of Alberta 15
 def sum1(n):

Benchmarking 3
n=100000000 theSum=0
totalTime=0.0 for i in range(1,n+1):
for i in range(5): theSum=theSum+i
start=time.time() return theSum
x=sum1(n)
end=time.time()
Repeating sum1 while varying n print("the sum is %d and it required %10.7f seconds"%(x,end-start))
totalTime=totalTime+end-start
print("with sum1 the average time was %10.7f for n=%d"%(totalTime/5,n))

n= 100,000 n= 1,000,000
the sum is 5000050000 and it required 0.0156250 seconds the sum is 500000500000 and it required 0.1250000 seconds
the sum is 5000050000 and it required 0.0156250 seconds the sum is 500000500000 and it required 0.1250000 seconds
the sum is 5000050000 and it required 0.0000000 seconds the sum is 500000500000 and it required 0.1250000 seconds
the sum is 5000050000 and it required 0.0156250 seconds the sum is 500000500000 and it required 0.1250000 seconds
the sum is 5000050000 and it required 0.0156250 seconds the sum is 500000500000 and it required 0.1250000 seconds
with sum1 the average time was 0.0125000 for n=100000 with sum1 the average time was 0.1250000 for n=1000000

n= 10,000,000 n= 100,000,000
the sum is 50000005000000 and it required 1.2343750 seconds the sum is 5000000050000000 and it required 12.5156250 seconds
the sum is 50000005000000 and it required 1.2343750 seconds the sum is 5000000050000000 and it required 12.5312500 seconds
the sum is 50000005000000 and it required 1.2500000 seconds the sum is 5000000050000000 and it required 12.3437500 seconds
the sum is 50000005000000 and it required 1.2656250 seconds the sum is 5000000050000000 and it required 12.3125000 seconds
the sum is 50000005000000 and it required 1.2187500 seconds the sum is 5000000050000000 and it required 12.5312500 seconds
with sum1 the average time was 1.2406250 for n=10000000 with sum1 the average time was 12.4468750 for n=100000000

Each time n is multiplied by 10, the execution time

is increased an order of magnitude

January 22, 2024 © Osmar R. Zaïane : University of Alberta 16

 Benchmarking 4
n=100000000 def sum3(n):
totalTime=0.0 return (n*(n+1))/2
for i in range(5):
start=time.time()
x=sum3(n)
end=time.time()
print("the sum is %d and it required %10.7f seconds"%(x,end-start))
Repeating sum3 while varying n totalTime=totalTime+end-start
print("with sum3 the average time was %10.7f for n=%d"%(totalTime/5,n))

n= 100,000 n= 1,000,000
the sum is 5000050000 and it required 0.0000000 seconds the sum is 500000500000 and it required 0.0000000 seconds
the sum is 5000050000 and it required 0.0000000 seconds the sum is 500000500000 and it required 0.0000000 seconds
the sum is 5000050000 and it required 0.0000000 seconds the sum is 500000500000 and it required 0.0000000 seconds
the sum is 5000050000 and it required 0.0000000 seconds the sum is 500000500000 and it required 0.0000000 seconds
the sum is 5000050000 and it required 0.0000000 seconds the sum is 500000500000 and it required 0.0000000 seconds
with sum3 the average time was 0.0000000 for n=100000 with sum3 the average time was 0.0000000 for n=1000000

n= 10,000,000 n= 100,000,000
the sum is 50000005000000 and it required 0.0000000 seconds the sum is 5000000050000000 and it required 0.0000000 seconds
the sum is 50000005000000 and it required 0.0000000 seconds the sum is 5000000050000000 and it required 0.0000000 seconds
the sum is 50000005000000 and it required 0.0000000 seconds the sum is 5000000050000000 and it required 0.0000000 seconds
the sum is 50000005000000 and it required 0.0000000 seconds the sum is 5000000050000000 and it required 0.0000000 seconds
the sum is 50000005000000 and it required 0.0000000 seconds the sum is 5000000050000000 and it required 0.0000000 seconds
with sum3 the average time was 0.0000000 for n=10000000 with sum3 the average time was 0.0000000 for n=100000000

Even though we increase n, the execution time

doesn’t change. It is actually too fast to measure the
time with time() on an AMD64 2.3Ghz machine
January 22, 2024 © Osmar R. Zaïane : University of Alberta 17
 Benchmarking 5
n=100000000 def sum3(n):
totalTime=0.0 return (n*(n+1))/2
for i in range(5):
start=time.time()
for j in range(100000): x=sum3(n)

Let’s slow down by calling sum3() end=time.time()

print("the sum is %d and it required %10.7f seconds"%(x,end-start))
totalTime=totalTime+end-start
100000 times print("with sum3 the average time was %10.7f for n=%d"%(totalTime/5,n))

n= 100,000 n= 1,000,000
the sum is 5000050000 and it required 0.0468750 seconds the sum is 500000500000 and it required 0.0468750 seconds
the sum is 5000050000 and it required 0.0468750 seconds the sum is 500000500000 and it required 0.0468750 seconds
the sum is 5000050000 and it required 0.0625000 seconds the sum is 500000500000 and it required 0.0468750 seconds
the sum is 5000050000 and it required 0.0468750 seconds the sum is 500000500000 and it required 0.0468750 seconds
the sum is 5000050000 and it required 0.0468750 seconds the sum is 500000500000 and it required 0.0468750 seconds
with sum3 the average time was 0.0500000 for n=100000 with sum3 the average time was 0.0468750 for n=1000000

n= 10,000,000 n= 100,000,000
the sum is 50000005000000 and it required 0.0468750 seconds the sum is 5000000050000000 and it required 0.0625000 seconds
the sum is 50000005000000 and it required 0.0468750 seconds the sum is 5000000050000000 and it required 0.0781250 seconds
the sum is 50000005000000 and it required 0.0468750 seconds the sum is 5000000050000000 and it required 0.0625000 seconds
the sum is 50000005000000 and it required 0.0468750 seconds the sum is 5000000050000000 and it required 0.0625000 seconds
the sum is 50000005000000 and it required 0.0468750 seconds the sum is 5000000050000000 and it required 0.0625000 seconds
with sum3 the average time was 0.0468750 for n=10000000 with sum3 the average time was 0.0656250 for n=100000000

Even though we increase n, the execution time

doesn’t change. It remains relatively constant at 0.04
to 0.06 seconds for calling 100,000 times sum3(n)
January 22, 2024 © Osmar R. Zaïane : University of Alberta 18
 Running time complexity
Time of sum1(n) increases linearly with n
def sum1(n):
theSum=0
for i in range(1,n+1):
theSum=theSum+i
return theSum

Time of sum3(n) remains constant

regardless of n def sum3(n):
time return (n*(n+1))/2
sum1

sum3
n
January 22, 2024 © Osmar R. Zaïane : University of Alberta 19
 8

13

1
2
1
5
3

0 1st 2nd 3rd 4th 5th 6th 7th …........…………… 35th ?

0, 1, 1, 2, 3, 5, 8, 13, ……..
January 22, 2024 © Osmar R. Zaïane : University of Alberta 20
 Fibonacci  Son of Bonacci
Leonardo Bonacci aka Fibonacci (Fillius Bonacci)
Pisa, Italy XII century

Leonardo Pisano

January 22, 2024 © Osmar R. Zaïane : University of Alberta 21

 Problem: to compute Fibonacci
Fn/Fn-1 ≈ 1.618number
3/2 = 1.5
The Fibonacci 5/3numbers
= 1.66666666667
are the integer sequence
0, 1, 1,8/5
2, 3, 5,=8,1.6
13, 21, 34, 55, 89,...,
in which each 13/8
number = 1.625
in the sequence is the sum of the
previous two.21/13 = 1.61538461538
34/21 = 1.61904761905
55/34 = 1.61764705882
89/55 = 1.61818181818
144/89 = 1.61797752809
…
F0=0   1  5  1.6180339887498948
Da Vinci’s
Vitruvian man:

F1=1 2 Golden ratio

related to
Fibonacci
Fn=Fn-1+Fn-2 sequence
Fn/Fn-1 ≈ 1.618

January 22, 2024 © Osmar R. Zaïane : University of Alberta 22

 def sum1(n):
theSum=0
for i in range(1,n+1):
Loop theSum=theSum+i
return theSum

def sum2(n):
Recursion if n==0: return 0
else: return sum2(n-1) + n

Formula def sum3(n):

return (n*(n+1))/2
January 22, 2024 © Osmar R. Zaïane : University of Alberta 23
Finding the nth number in the
Fibonacci sequence

0 1st 2nd 3rd 4th 5th 6th 7th …........ …………… 35th ?
0, 1, 1, 2, 3, 5, 8, 13, 21, 34, 55, 89, 144…..

a=b
b=c

January 22, 2024 © Osmar R. Zaïane : University of Alberta 24

An iteration program (bottom-up)
a b
0, 1, 1, 2, 3, 5, 8, 13, ……..
a b
Algorithm
def fibonacci(n): n=0: a=0; b=1
F(n) { a =n=20 n=1: c=1; a=1; b=1
a=0; b=1
a=0; b=1 b= 1 0 to 1
i from n=2: c=2; a=1; b=2
for i from 0 to n-1 { fori=0i in range(n): n=3: c=3; a=2; b=3
c=a+b c= 0+1=1
c=a+b n=4: c=5; a=3; b=5
a=1
a=b a=b
b=1 n=5: c=8; a=5; b=8
b=c i=1
b=c n=6: c=13; a=8; b=13
} returnc= 1+1=2
a n=7: c=21; a=13; b=21
a= 1
return a b= 2 n=8: c=34; a=21; b=34
} return a=1
print(fibonacci(35)) n=9: c=55; a=34; b=55
n=10: c=89; a=55; b=89
a moves n times (from 0 to n-1) 9227465 …
So at the end a is the nth number
January 22, 2024 © Osmar R. Zaïane : University of Alberta 25
 A recursive program
Algorithm
def fibonacci(n):
F(n) { if n == 0 or n == 1:
if n==0 or n==1 return n return n
else return F(n-1) + F(n-2) else:
return fibonacci(n-1) + fibonacci(n-2)
}

print(fibonacci(35))
For example for fibonacci(4) 9227465
fibonacci(4) calls fibonacci(3) and fibonacci(2)
fibonacci(3) calls fibonacci(2) and fibonacci(1) 4
fibonacci(2) calls fibonacci(1) and fibonacci(0) 3 2
fibonacci(1) terminates with 1
fibonacci(0) terminates with 0 2 1 1 0
fibonacci(1) terminates with 1 1 0
fibonacci(2) calls fibonacci(1) and fibonacci(0)
fibonacci(1) terminates with 1
fibonacci(0) terminates with 0
January 22, 2024 © Osmar R. Zaïane : University of Alberta 26
A recursive program with cache
Algorithm memFibo = {0:0, 1:1}
M[0]=0
M[1]=1 def fib(n):
F(n) { if not n in memFibo:
if not exist M[n] memFibo[n] = fib(n-1) + fib(n-2)
M[n]= F(n-1) + F(n-2) return memFibo[n]
return M[n]
} print(fib(35))
9227465
Avoids recalculating numbers in the
Fibonacci sequence that were already
calculated.
January 22, 2024 © Osmar R. Zaïane : University of Alberta 27
 Another solution
Algorithm

Fibonacii(n)=F(n) = clossest integer of

 n Fn/Fn-1 ≈ 1.618…

1 5
5
  1.6180339887 498948 also known as the Golden ratio
2
def fibonacci(n):
inverseSqrt5 = 0.44721359549995793928183473374626
phi = 1.6180339887498948482045868343656
x=pow(phi,n)
return int(round(x*inverseSqrt5))

print(fibonacci(35))
9227465
January 22, 2024 © Osmar R. Zaïane : University of Alberta 28
 Which program is better?
All four solutions are correct, but which
one is the best?  Measure the running time
fib1(): iterative fib2(): recursive fib3(): recursive with cache fib4(): Golden ratio

Fibbonaci(35) is 9227465. execution time with fib1() is 0.00000775187970 milliseconds

Fibbonaci(35) is 9227465. execution time with fib2() is 9.82390678195489 milliseconds
Fibbonaci(35) is 9227465. execution time with fib3() is 0.00000143609023 milliseconds
Fibbonaci(35) is 9227465. execution time with fib4() is 0.00000620676692 milliseconds
Fibbonaci(20) is 6765. execution time with fib1() is 0.00000614661654 milliseconds
Fibbonaci(20) is 6765. execution time with fib2() is 0.00727272180451 milliseconds
Fibbonaci(20) is 6765. execution time with fib3() is 0.00000140225564 milliseconds
Fibbonaci(20) is 6765. execution time with fib4() is 0.00000603759398 milliseconds
Fibbonaci(100) is 354224848179261915075. execution time with fib1() is 0.00001588345865 milliseconds
Fibbonaci(100) is 354224848179261915075. execution time with fib3() is 0.00000149248120 milliseconds
Fibbonaci(100) is 354224848179263111168. execution time with fib4() is 0.00000631954887 milliseconds
-1179648 difference Error due to machine precision for floating points as the mantissa is approximated
January 22, 2024 © Osmar R. Zaïane : University of Alberta 29
 Mantissa problem
def fibonacci(n):
inverseSqrt5 = 0.44721359549995793928183473374626
phi = 1.6180339887498948482045868343656
x=pow(phi,n)
return int(round(x*inverseSqrt5))

1 bit for sign

A Single-Precision floating-
8 bits for exponent point number occupies 32-bits
23 bits for mantissa
00000000000000000000000000000000
A Double-Precision floating-
11 bits for exponent point number occupies 64-bits
0000000000000000000000000000000000000000000000000000000000000000
52 bits for mantissa
January 22, 2024 © Osmar R. Zaïane : University of Alberta 30
 Another Theorem for Fibonacci
n
 Fn 1 Fn  1 1 
     Can be proven by induction on n
 Fn Fn 1  1 0 
1 1 
So we take the matrix 1  to the power of n, then take the top right corner as Fn
 0 

Assuming we have a function to calculate the power of a 2x2 matrix

matrixPower(). This function can be done by iterating, by
diagonalization or divide and conquer strategy as we shall see later.
def fibonacci(n):
(a,b,c,d) = matrixPower(1,1,1,0,n)
return b

print(fibonacci(35)) 9227465
January 22, 2024 © Osmar R. Zaïane : University of Alberta 31
 Divide and Conquer for the
Power function
Xn can be computed by multiplying X by itself n times  iteration
X * X *…..*X
n times or we can divide the problem to reduce the computation
n n
X  X X
n 2 2 if n is even
n 1 n 1
x4=x2 * x2
x7=x3 * x3 * x X 2
X 2
X if n is odd

When n is even we do the computation for

half the sequence then multiply the result
by itself. When n is odd we do almost the
same and get the same savings
January 22, 2024 © Osmar R. Zaïane : University of Alberta 32
 Reminder

2
a b a b a b aa+bc ab+bd
= c d c d =
c d ca+dc cb+dd

a b a b a b a b a b a b a b a b
c d c d c d c d c d c d c d c d

January 22, 2024 © Osmar R. Zaïane : University of Alberta 33

 Fibonacci with matrices
def matrixPower(a,b,c,d,n): def fibonacii(n):
if n<=1: return (1,1,1,0) (a,b,c,d) = matrixPower(1,1,1,0,n)
elif n%2==0: return b
# x^n = x^(n/2) * x^(n/2)
(a1,b1,c1,d1)=matrixPower(a,b,c,d,n/2) F(5)=?
a3=a1*a1+b1*c1 1 1 5 1 1 1 1 1 1 1 1 1 1
b3=a1*b1+b1*d1 1 0 = 1 0 1 0 1 0 1 0 1 0
aa+bc ab+bd
c3=c1*a1+d1*c1
d3=c1*b1+d1*d1
ca+dc cb+dd
2 1 2 1
else:
1 1 1 1
# x^n = x^((n-1)/2) * x^((n-1)/2) * x
(a1,b1,c1,d1)=matrixPower(a,b,c,d,(n-1)/2)
a2=a1*a1+b1*c1 # x^((n-1)/2) * itself 5 3 8 5
b2=a1*b1+b1*d1 3 2 5 3
c2=c1*a1+d1*c1 F(5)=5
d2=c1*b1+d1*d1
a3=a2*a+b2*c # x^n = x^(n-1) * x import timeit
b3=a2*b+b2*d …
c3=c2*a+d2*c t=timeit.Timer("fib5(35)","from __main__ import fib5")
d3=c2*b+d2*d f=fib5(35)
print("Fibbonaci(35) is %d. execution time with fib5() is"%(f), end=" ")
return a3,b3,c3,d3 print (" %17.14f milliseconds"%(t.timeit(number=1)))

Fibbonaci(35) is 9227465. execution time with fib5() is 0.00001665037594 milliseconds

January 22, 2024 © Osmar R. Zaïane : University of Alberta 34
 Which program is better?
All four solutions are correct, but which
one is the best?  Measure the running time
fib1(): iterative fib2(): recursive fib3(): recursive with cache fib4(): Golden ratio

Fibbonaci(35) is 9227465. execution time with fib1() is 0.00000775187970 milliseconds

Fibbonaci(35) is 9227465. execution time with fib2() is 9.82390678195489 milliseconds
Fibbonaci(35) is 9227465. execution time with fib3() is 0.00000143609023 milliseconds
Fibbonaci(35) is 9227465. execution time with fib4() is 0.00000620676692 milliseconds

fib5(): matrix multiplication

Fibbonaci(35) is 9227465. execution time with fib5() is 0.00001665037594 milliseconds

January 22, 2024 © Osmar R. Zaïane : University of Alberta 35

 Performance of an Algorithm
The performance (time) of an algorithm
Determined by the number of basic
operations needed to solve the problem
Not by the actual running time of a
program using the algorithm

Determined by the size of the problem

Typically the number of data points n

the size of the input

January 22, 2024 © Osmar R. Zaïane : University of Alberta 36
Growth Rate of an Algorithm
The time performance is specified by
T( n)
n is the size of the problem
T(n) is a function of n, representing the number of basic
operations for the problem with size n
The performance of an algorithm is determined by
its behaviour at the large size of the problem (or as
the size grows)
The performance of an algorithm is determined by
the growth rate of the number of operations with
respect to the increase of the sizes of the problem
January 22, 2024 © Osmar R. Zaïane : University of Alberta 37
 Different growth rates
Consider the following functions
T1(n) = 10
T2(n) = n
T3(n) = log2(n)
T4(n) = n2
T5(n) = 2n
Which one is best?
Which one is worst? fib1(): iterative  Tfib1(n)=n
fib2(): recursive  Tfib2(n)=φn
fib3(): recursive with cache Tfib3(n)=n
fib4(): Golden ratio  Tfib4(n)=log(n)
fib5(): Matrix power  Tfib5(n)=log(n)

January 22, 2024 © Osmar R. Zaïane : University of Alberta 38

 Comparative results
n=10 n=20 n=30 n=40 n=50 n=60 n=70 n=80 n=90 n=100 n=1000 n=10000
fib1() 0.00000473 0.00000592 0.00000717 0.00000823 0.00000961 0.00001081 0.00001305 0.00001337 0.00001485 0.00001566 0.00015460 0.00368752
fib2() 0.00005934 0.00720515 0.90731057 109.52110087
fib3() 0.00000133 0.00000138 0.00000139 0.00000139 0.00000132 0.00000136 0.00000129 0.00000137 0.00000135 0.00000132
fib4() 0.00000567 0.00000582 0.00000612 0.00000600 0.00000595 0.00002308 0.00000605 0.00000623 0.00000610 0.00000620 0.00000765
fib5() 0.00001074 0.00001360 0.00001494 0.00001551 0.00001684 0.00001801 0.00001875 0.00001763 0.00002058 0.00001939 0.00003592 0.00029137

120.00000000
fib1(): iterative 0.00018000

100.00000000 fib2(): recursive 0.00016000

fib3(): recursive with cache 0.00014000
80.00000000
fib1() fib4(): Golden ratio 0.00012000
fib1()
60.00000000
fib2() fib5(): Matrix power 0.00010000 fib3()
fib3()
0.00008000 fib4()
fib4()
40.00000000 fib5()
fib5() 0.00006000

0.00004000
20.00000000
0.00400000 0.00002000
0.00000000 0.00000000
0.00350000
10 20 = 30 = 40 = 50 = 60 = 70 = 80 = 90 100 000

00
n= n=

n= 0
10

20

30
40

50

60

70

80
90
n n n n n n n n= n= 1

10
10
n=

n=

n=
n=

n=

n=

n=

n=
n=
0.00300000

n=
0.00250000
fib1()
0.00200000
fib1(): Tfib1(n)=n fib5()
0.00150000 Some could not continue
fib2(): Tfib2(n)=φn 0.00100000 fib2(): Exponential
fib3(): Tfib3(n)=n 0.00050000 fib3(): recursion depth too deep
fib4(): Tfib4(n)=log(n) 0.00000000 fib4(): error +
0
n= 00
n= 0
10

20

30

40

50

60

70

80

90

number too large

00
10
10
n=

n=

n=

n=

n=

n=

n=

n=

n=

fib5(): Tfib5(n)=log(n)
10
n=

January 22, 2024 © Osmar R. Zaïane : University of Alberta 39

 Multipliers
Consider the following functions
T1(n) = 10 T6(n) = 40  Constant time
T2(n) = n T7(n) = 3 * n  linear
T3(n) = log2(n) T8(n) = 5 * log2(n)  logarithmic
T4(n) = n2 T9(n) = 50 * n2  polynomial
T5(n) = 2n T10(n) = 10 * 2n  exponential

Any big difference between the left and the right

columns?

January 22, 2024 © Osmar R. Zaïane : University of Alberta 40

 Order of magnitude functions
Time

n
(size)
January 22, 2024 © Osmar R. Zaïane : University of Alberta 41
 Function of Growth rate

January 22, 2024 © Osmar R. Zaïane : University of Alberta 42

 Asymptotic performance

Big-O notation

January 22, 2024 © Osmar R. Zaïane : University of Alberta 43

 The time performance of
algorithms
The number of basic operations as a
function of the problem size
Represented as functions in order of
increasing growth rate
Denoted by
O(f(n))

January 22, 2024 © Osmar R. Zaïane : University of Alberta 44

 Types of problems
Easy problems
O(n)
Polynomial
Challenging problems
No known polynomial algorithms
Cannot prove that there are no polynomial
algorithms
Hard problems
There exist no polynomial algorithms

January 22, 2024 © Osmar R. Zaïane : University of Alberta 45

 Algorithm Techniques
Dynamic programming
Use a table to store intermediate results to avoid
repeat computation
Divided and conquer
Decompose a problem into two or more smaller problems
and solve them separately
Combine solutions of smaller problems to solve the given
problem

Greedy Algorithm
Take the best one can get in each step
Recursion
January 22, 2024 © Osmar R. Zaïane : University of Alberta 46
 Dynamic programming
Use a table to store intermediate results
Avoid repeat computation
The Fibonacci numbers

def fibonacci(n):
if n == 0 or n == 1:
return n
else:
return fibonacci(n-1) + fibonacci(n-2)
January 22, 2024 © Osmar R. Zaïane : University of Alberta 47
 The function The number of calls
F(5) 1
F(4) 2
F(3) 3
F(2) 5
F(1) 7
F(0) 5

January 22, 2024 © Osmar R. Zaïane : University of Alberta 48

 Using a cache
What if we use a table to store all the
intermediate results of f(m) for m < n?
No more repeat computation
Compromise Space for timing
memFibo = {0:0, 1:1}

def fib(n):
if not n in memFibo:
memFibo[n] = fib(n-1) + fib(n-2)
return memFibo[n]

January 22, 2024 © Osmar R. Zaïane : University of Alberta 49

 f(3)
f(4)
f(2)

January 22, 2024 © Osmar R. Zaïane : University of Alberta 50

 Divide and Conquer

1. Divide instance of a problem into two or

more smaller instances of the same type
2. Solve smaller instances recursively
3. Obtain solution to original (larger)
instance by combining these solutions

January 22, 2024 © Osmar R. Zaïane : University of Alberta 51

Divide and Conquer (cont.)

January 22, 2024 © Osmar R. Zaïane : University of Alberta 52

 Divide and Conquer (cont.)
Fibonacci(11)
1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1
1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0

1 1 1 1 1 1 1 1 1 1
1 0 1 0 1 0 1 0 1 0

2 1 2 1 1 1
1 1 1 1 1 0

8 5 8 5
5 3 5 3

89 55 1 1
55 34 1 0
Only four 2x2 matrix multiplications
0, 1, 1, 2, 3, 5, 8, 13, 21, 34, 55, 89,..., 144 89
in a sequence of 11 89 55
January 22, 2024 © Osmar R. Zaïane : University of Alberta 53
 Fibonacci with matrices
def fibonacii(n):
def matrixPower(a,b,c,d,n):
(a,b,c,d) = matrixPower(1,1,1,0,n)
if n<=1: return (1,1,1,0)
return b
elif n%2==0:
# x^n = x^(n/2) * x^(n/2)
(a1,b1,c1,d1)=matrixPower(a,b,c,d,n/2)
a3=a1*a1+b1*c1
b3=a1*b1+b1*d1 aa+bc ab+bd
c3=c1*a1+d1*c1 I didn’t call matrixPower
d3=c1*b1+d1*d1
ca+dc cb+dd
else:
again. I simply multiplied
# x^n = x^((n-1)/2) * x^((n-1)/2) * x the matrix result (a1, b1, c1, d1)
(a1,b1,c1,d1)=matrixPower(a,b,c,d,(n-1)/2)
a2=a1*a1+b1*c1 # x^((n-1)/2) * itself by itself
b2=a1*b1+b1*d1
c2=c1*a1+d1*c1
d2=c1*b1+d1*d1
a3=a2*a+b2*c # x^n = x^(n-1) * x
The same here
b3=a2*b+b2*d
c3=c2*a+d2*c
d3=c2*b+d2*d

return a3,b3,c3,d3

January 22, 2024 © Osmar R. Zaïane : University of Alberta 54

 Greedy Algorithm
Optimization problems
An optimization problem is one in which
one wants to find, not just a solution, but
the best solution possible
A greedy algorithm sometimes works
well for optimization problems

January 22, 2024 © Osmar R. Zaïane : University of Alberta 55

 Greedy Algorithm (cont.)
A greedy algorithm works in steps.
At each step
take the best one can get right now,
without regarding the eventual
optimization
Hope that by choosing a local optimum at
each step, one will end up at a global
optimum

January 22, 2024 © Osmar R. Zaïane : University of Alberta 56

 Example: counting money
Suppose you want to count out a certain
amount of money, using the fewest possible
bills and coins
A greedy algorithm:
At each step, take the largest possible bill or
coin that does not overshoot
Example: To make $6.39, you can choose:
one $5 bill
one $1 coin, to make $6
one 25¢ coin, to make $6.25
one 10¢ coin, to make $6.35
four 1¢ coins, to make $6.39
January 22, 2024 © Osmar R. Zaïane : University of Alberta 57
 Running time and data
structures
The choice of data structure can make a
difference in the running time of the
implementation of an algorithm
How the data structure is manipulated can
also make a difference in the execution time
Anything you have to iterate over  O(n)
Accessing an indexed container  O(1)
Sometimes it is subtile like when pop(0) at
the beginning of a list requires shifting all
elements  O(n)
January 22, 2024 © Osmar R. Zaïane : University of Alberta 58
 Example: ways to create a list
# Append elements 1 by 1
# Concatenate elements 1 by 1 def list2():
def list1(): myList = []
myList= [] for i in range(1000):
for i in range(1000): myList.append(i)
myList = myList + [i]
# List range
# Comprehension def list4():
def list3(): myList = list(range(1000))
myList= [i for i in range(1000)]
They all do the same thing: creating a list myList with
integers from 0 to 999
Do they take the same time to initialize myList?
Let’s measure it
January 22, 2024 © Osmar R. Zaïane : University of Alberta 59
 Benchmarking Lists
import timeit

…
t=timeit.Timer("list1()","from __main__ import list1")
print("Concatenation: %17.14f milliseconds"%(t.timeit(number=1000)))

t=timeit.Timer("list2()","from __main__ import list2")

print("Append : %17.14f milliseconds"%(t.timeit(number=1000)))

t=timeit.Timer("list3()","from __main__ import list3")

print("Comprehension: %17.14f milliseconds"%(t.timeit(number=1000)))

t=timeit.Timer("list4()","from __main__ import list4")

print("List Range : %17.14f milliseconds"%(t.timeit(number=1000)))

January 22, 2024 © Osmar R. Zaïane : University of Alberta 60

 Big-O Efficiency of Python List
Operations
Operation Big-O Efficiency

index [] O(1)
index assignment O(1)
append O(1)
pop() O(1)
pop(i) O(n)
insert(i,item) O(n)
del operator O(n)
iteration O(n)
contains (in) O(n)
get slice [x:y] O(k)
del slice O(n)
set slice O(n+k)
reverse O(n)
concatenate O(k)
sort O(n log n)
multiply O(nk)
January 22, 2024 © Osmar R. Zaïane : University of Alberta 61