Design and Analysis of Algorithms
Lecture 6:
Efficiency Analysis
Tauseef Iftikhar
Department of Computer Science
Government College University, Lahore.
Todays Agenda
What is efficiency analysis?
Measuring time efficiency
Best case, worst case and average case analysis
Efficiency Analysis
The amount of computing resources needed to execute the
algorithm
it is also known as complexity analysis
Efficiency Analysis
The amount of computing resources needed to execute the
algorithm
it is also known as complexity analysis
Computing resources:
Memory space: space required to store the data processed by
algorithm
CPU time: also known as running time. It is time needed to
execute the operations of the algorithm.
An efficient algorithm uses a minimum amount of computing
resources.
Efficiency Analysis
There are two types of efficiency analysis
I
The key strategy in efficiency analysis is the amount of
computing resources depends on the input size (problem size)
I
Space analysis: how much space an algorithm required to store
the data in memory?
Running time analysis: it refers to how fast an algorithm runs
Input size: the number of elements belonging to the input
data.
Hence the main question to be answered by efficiency analysis
is how depends the time and/or space needed by algorithm on
the input size?
Space and time trade off
Often we have to make a compromise between space
efficiency and time efficiency
Example: adjacency matrix vs adjacency list for graph
representation.
I
A sparse graph can be represented by an adjacency matrix
which would be time efficient for traversing an edge but it will
be at the cost of space
A sparse graph can be represented by an adjacency list which
would be space efficient and but it would take longer time for
traversing the edges.
How can be time efficiency measured?
Our efficiency measure for running time must be independent of
I
machine
programming language
programmer
How can be time efficiency measured?
Our efficiency measure for running time must be independent of
I
machine
programming language
programmer
Running time is a function of input size denoted by T (n) where n
is input size.
RAM: Computational Model
However to estimate running time we must use some
computational model.
RAM: Computational Model
However to estimate running time we must use some
computational model.
Computational model is an abstract machine having some
properties.
RAM: Computational Model
However to estimate running time we must use some
computational model.
Computational model is an abstract machine having some
properties.
For this purpose we will use RAM computational model(Random
Access Machine) having the following properties
1. All processing steps are sequentially executed (there is no
parallelism in the execution of the algorithm)
2. The time of executing the basic operations does not depend
on the values of the operands (there is no time difference
between computing 1+2 and computing 12433+4567)
3. The time to access data does not depend on their address
(there are no differences between processing the first element
of an array and processing the last element)
4. All basic operations, assignment, arithmetic, logical, relational
take unit time.
Example1: running time
Now we see how running time expresses the dependence of the
number of executed operations on the input size
Example: swapping
aux x
x y
y x
T (n) =
iterations
1
1
1
3
cost
c1
c1
c1
c1
where c1 is some constant and 3c1 is also some constant. We
conclude that running time of this algorithm is some constant.
Hence we can understand that the running time of above algorithm
is independent of the input size.
Example2: running time
Example 2: Compute sum of the series s = 1 + 2 + . . . + n
precondition: n 1
postcondition: s = 1 + 2 + . . . + n
input: n
1:
2:
3:
4:
5:
6:
s0
i 1
while i n
do
s s +i
i i +1
end while
line no.
1
2
3
4
5
cost
c1
c1
c2
c3
c3
iterations
1
1
n+1
n.1
n.1
Example2: running time
Example 2: Compute sum of the series s = 1 + 2 + . . . + n
precondition: n 1
postcondition: s = 1 + 2 + . . . + n
input: n
1:
2:
3:
4:
5:
6:
s0
i 1
while i n
do
s s +i
i i +1
end while
line no.
1
2
3
4
5
T (n) =
cost
c1
c1
c2
c3
c3
2c1 + c2(n + 1) + 2c3n
iterations
1
1
n+1
n.1
n.1
Example2: running time
Example 2: Compute sum of the series s = 1 + 2 + . . . + n
precondition: n 1
postcondition: s = 1 + 2 + . . . + n
input: n
1:
2:
3:
4:
5:
6:
s0
i 1
while i n
do
s s +i
i i +1
end while
line no.
1
2
3
4
5
T (n) =
T (n) =
cost
c1
c1
c2
c3
c3
2c1 + c2(n + 1) + 2c3n
2c1 + c2.n + c2 + 2c3.n
iterations
1
1
n+1
n.1
n.1
Example2: running time
Example 2: Compute sum of the series s = 1 + 2 + . . . + n
precondition: n 1
postcondition: s = 1 + 2 + . . . + n
input: n
1:
2:
3:
4:
5:
6:
s0
i 1
while i n
do
s s +i
i i +1
end while
line no.
1
2
3
4
5
T (n) =
T (n) =
T (n) =
cost
c1
c1
c2
c3
c3
2c1 + c2(n + 1) + 2c3n
2c1 + c2.n + c2 + 2c3.n
a.n + b
iterations
1
1
n+1
n.1
n.1
Example2: running time
Example 2: Compute sum of the series s = 1 + 2 + . . . + n
precondition: n 1
postcondition: s = 1 + 2 + . . . + n
input: n
1:
2:
3:
4:
5:
6:
s0
i 1
while i n
do
s s +i
i i +1
end while
line no.
1
2
3
4
5
T (n) =
T (n) =
T (n) =
cost
c1
c1
c2
c3
c3
2c1 + c2(n + 1) + 2c3n
2c1 + c2.n + c2 + 2c3.n
a.n + b
Running time is linear function of input size
iterations
1
1
n+1
n.1
n.1
Example3: running time
Example 3: Find the minimum in non-empty array x[1 . . . n]
P: n 1
Q: m = min{x[j]|j = 1, 2, . . . n}n
input: x[1..n]
1:
2:
3:
4:
5:
6:
7:
m x[1]
for i 2, n do
if x[i] < m
then
m x[i]
end if
end for
return m
line no.
1
2
3
4
cost
1
1
1
1
iterations
1
2n
n1
h(n)
Example3: running time
Example 3: Find the minimum in non-empty array x[1 . . . n]
P: n 1
Q: m = min{x[j]|j = 1, 2, . . . n}n
input: x[1..n]
1:
2:
3:
4:
5:
6:
7:
m x[1]
for i 2, n do
if x[i] < m
then
m x[i]
end if
end for
return m
line no.
1
2
3
4
T (n) =
cost
1
1
1
1
1 + 2n + n 1 + h(n)
iterations
1
2n
n1
h(n)
Example3: running time
Example 3: Find the minimum in non-empty array x[1 . . . n]
P: n 1
Q: m = min{x[j]|j = 1, 2, . . . n}n
input: x[1..n]
1:
2:
3:
4:
5:
6:
7:
m x[1]
for i 2, n do
if x[i] < m
then
m x[i]
end if
end for
return m
line no.
1
2
3
4
T (n) =
T (n) =
cost
1
1
1
1
1 + 2n + n 1 + h(n)
3n + h(n)
iterations
1
2n
n1
h(n)
Example3: running time
Example 3: Find the minimum in non-empty array x[1 . . . n]
P: n 1
Q: m = min{x[j]|j = 1, 2, . . . n}n
input: x[1..n]
line no.
cost
iterations
m x[1]
1
1
1
for i 2, n do
2
1
2n
3:
if x[i] < m
3
1
n1
then
4
1
h(n)
4:
m x[i]
T (n) = 1 + 2n + n 1 + h(n)
5:
end if
T (n) =
3n + h(n)
6: end for
7: return m
The running time depends not only on n but also on the properties
of input data
1:
2:
Best-case analysis and worst-case analysis
Whenever analysis of an algorithm not only depends on the input
size but also on some property of input data then we have to
perform analysis in more details
I
Worst-case analysis: gives the longest running time for any
input of size n.
Best-case analysis: gives us the minimum running time for
any input of size n.
I
worst-case running time of an algorithm gives us an upper
bound on the running time for any input.
Knowing it provides a guarantee that the algorithm will never
take any longer.
best-case running time of an algorithm gives us lower bound
on the running time for any input.
Knowing it provides a guarantee that the algorithm will never
take more less time.
Example 4: sequential search
Preconditions: x[1..n], n >= 1, v a value
Postconditions: found = TRUE when v x[1..n]
Input: x[1..n], v
Algorithm 1 search
1: found true
2: i 1
3: while (found = false) and (i n)
do
4:
if x[i] = v then
5:
found true
6:
else
7:
i i +1
8:
end if
9: end while
Example 4: sequential search
Preconditions: x[1..n], n >= 1, v a value
Postconditions: found = TRUE when v x[1..n]
Input: x[1..n], v
Algorithm 2 search
1: found true
2: i 1
3: while (found = false) and (i n)
do
4:
if x[i] = v then
5:
found true
6:
else
7:
i i +1
8:
end if
9: end while
line no
1
2
3
4
5
7
cost
1
1
f (n) + 1
f (n)
g (n)
h(n)
T(n)=
3 + 2f (n) + g (n) + h(n)
Example 4: sequential search
Preconditions: x[1..n], n >= 1, v a value
Postconditions: found = TRUE when v x[1..n]
Input: x[1..n], v
Algorithm 3 search
1: found true
2: i 1
3: while (found = false) and (i n)
do
4:
if x[i] = v then
5:
found true
6:
else
7:
i i +1
8:
end if
9: end while
line no
1
2
3
4
5
7
cost
1
1
f (n) + 1
f (n)
g (n)
h(n)
T(n)=
3 + 2f (n) + g (n) + h(n)
Example 4: sequential search
The running time depends on the properties of the array.
Case 1: the value v is in the array (let k be the first position of v)
Case 2: the value v is not in the array
Example 4: sequential search
The running time depends on the properties of the array.
Case 1: the value v is in the array (let k be the first position of v)
Case 2: the value v is not in the array
(
k, if v x[i..n]
f (n) =
n, if v
/ x[1..n]
Example 4: sequential search
The running time depends on the properties of the array.
Case 1: the value v is in the array (let k be the first position of v)
Case 2: the value v is not in the array
(
k, if v x[i..n]
f (n) =
n, if v
/ x[1..n]
(
1,
g (n) =
0,
if v x[i..n]
if v
/ x[1..n]
Example 4: sequential search
The running time depends on the properties of the array.
Case 1: the value v is in the array (let k be the first position of v)
Case 2: the value v is not in the array
(
k, if v x[i..n]
T (n) = 3 + 2f (n) + g (n) + h(n)
f (n) =
n, if v
/ x[1..n]
(
1, if
g (n) =
0, if
(
k 1,
h(n) =
n,
v x[i..n]
v
/ x[1..n]
if v x[i..n]
if v
/ x[1..n]
Example 4: sequential search
The running time depends on the properties of the array.
Case 1: the value v is in the array (let k be the first position of v)
Case 2: the value v is not in the array
(
k, if v x[i..n]
T (n) = 3 + 2f (n) + g (n) + h(n)
f (n) =
n, if v
/ x[1..n]
Best case: x[1] = v
f (n) = 1, g (n) = 1, h(n) = 0
(
T (n) = 6
1, if v x[i..n]
g (n) =
0, if v
/ x[1..n]
(
k 1, if v x[i..n]
h(n) =
n,
if v
/ x[1..n]
Example 4: sequential search
The running time depends on the properties of the array.
Case 1: the value v is in the array (let k be the first position of v)
Case 2: the value v is not in the array
(
k, if v x[i..n]
T (n) = 3 + 2f (n) + g (n) + h(n)
f (n) =
n, if v
/ x[1..n]
Best case: x[1] = v
f (n) = 1, g (n) = 1, h(n) = 0
(
T (n) = 6
1, if v x[i..n]
g (n) =
Worst case: v
/ x[1..n]
0, if v
/ x[1..n]
f
(n)
=
n,
g
(n)
= 0, h(n) = n
(
T
(n)
=
3
+
2n
+ n = 3 + 3n
k 1, if v x[i..n]
h(n) =
n,
if v
/ x[1..n]
