CSI247:

Data Structures

Lecture 04 –Time complexity

of algorithms

Department of Computer Science

B. Gopolang (247-275)
 Previous lesson
• Defined a Data Structure
• Broad categories of DSs
• Examples under each category
• Common operations performed on DSs

2
Today …

3
 Learning Objectives
● By the end of this lecture you must be able to:
• Determine growth rate of an algorithm.
• Determine Big-O values for a given algorithm

4
 Introduction
● Programs are written to solve problems
● A program: Implementation of an algorithm
• Algorithm: A well-defined instruction set to
solve a specific problem in a finite amount of
time
● Notes:
• A problem can have several algorithms
• Algorithm implemented using a programming
language (various options available)
• Different implementation platforms available
5
 Efficiency of an algorithm
● Algorithm efficiency is affected by available resources
and its performance
a) Storage space
• Data to be processed must be stored somewhere
• Qs - Which data structure(s) to use?
- Its effect(s) on the algorithm’s efficiency?
b) Time it takes (this handout’s focus)
• Instructions takes time to execute
• Qs: - Is our algorithm efficient?
- What things affect the algorithm’s
runtime? 6
 Time complexity of an algorithm
● Time complexity!
• Amount of time an algorithm takes to run
given input of size n (number of input items)
● Used to estimate algorithm’s runtime

● Notes
i. Runtime depends on the input
• E.g.: Consider 2 arrays:
• an ordered (sorted) & an unsorted one
• Which one is easier and quicker to search?
7
 ii. Algorithm’s runtime increases when input
size, n, increases
• More time needed to process large data sets
● Suppose:
• We have 2 potential algorithms for 1 problem
• Q: How can we know which one is efficient?
• A: We need to estimate time taken by each
• Then compare them!
● NB:
• Good analysis must be h/w & data set
independent 8
a) Option 1: Experimentation
● Write programs to implement both algorithms
(e.g. using Java)
● Run the 2 programs, using identical data sets of
different sizes (n)
● Record their runtime
• Try using System.currentTimeMillis() for a
measure of the actual execution time
● Compare the runtimes
• The shorter the runtime, the better!
● Can plot the results in a graph
9
Limitations
● Must implement algorithms
● Same environment (h/w and s/w) required for both
● Will we get same results for all data sets?
• Experiments usually use limited input sets
• So, algorithm may be less efficient on some data
● Other factors likely to affect runtime worth considering:
• Is the h/w used representative of all h/w?
• Some more powerful than others
• Programming style
• Clean code vs spaghetti code 10
b) Option 2: Methodological approach

● Uses Mathematical techniques

• No need for implementation of the algorithm

● Runtime defined as a function of input size, n.

• Expressed using some standard notation

● Based on the number of primitive operations in

the algorithm (pseudo code or program)

11
● Primitive operations:
• Basic low-level actions, easily identifiable
• Independent of the language & h/w
• Assumption:
• Each takes some constant amount of time,
c, to execute
• E.g.:
• Arithmetic operations, assignment
statements, evaluating expressions, array
indexing, method calls, etc. 12
Possible analysis:
a) Best-case
• Shortest possible runtime
b) Average-case
• Expected runtime over input size n
• Difficult to calculate
c) Worst-case
• Most commonly used
• Gives maximum runtime for an input size n
• The most crucial in some application domains
(e.g. surgery, air traffic control, etc)
13
● How to estimate runtime:
• Count number of primitive operations
• Express the efficiency using growth functions
• How runtime increases when size increases
Notes:
● Convention:
• Parameterize running time with input size
• E.g.:
• Ta(n): estimated time of algorithm a for input
size n
14
 Calculating algorithm’s
estimated runtimes
● Each primitive operation takes some time
● Examples
a) 1 statement
i = 1; // c1, time: 1 (constant time)

b) Sequential statements:
i = 1; // c1, time: 1
val += I; // c2, time: 1

Total time, T(n) = 1*c1 + 1*c2 15

c) Selection
if ( a > o) // c1, time: 1
val = 10; // c2, time: 1
else
val = 1; // c3, time: 1
Total time, T(n) = 1*c1 + max(1*c2, 1*c3)

=1 + max(1, 1)
T(n) is constant 16
d) Repetition block
tot = 0; // c1, 1
c = 1; // c2, 1
while (c <= n) { // c3, executed n+1 times
tot = tot + c; // c4, n times
c = c + 2; // c5, n times
}
Total time, T(n) = c1 + c2 + (n+1)* c3 + n* c4 +
n* c5
T(n) is proportional to n
17
e) Nested repetition block
c=1;
while (c <= n) {
i=1;
while (i <= n) {
print i;
i = i+1;
}
c = c+1;
}
18
● From previous slide:
T(n) = c1 + (n+1)* c2 + n* c3 + n*(n+1)* c4 +
n*n*c5 + n*n*c6 +n*c7

T(n) is proportional to n2

19
● Estimation rules
• Sequential statements: sum of runtime of
the statements.
• if-else: At most, runtime of condition + max
between runtimes of if-block and else-block
• Loops: runtime of loop statements * no. of
iterations

20
 Algorithm Growth Rates
● Assume algorithms have these runtimes for size n:
• Algorithm A: 2*n
• Algorithm B: 4 + 3*n2
● From these, we can derive growth rates
• Refers to increase in runtime as size increases
• A: time is proportional to n.
• B: time is proportional to n2.
● Growth rates can help compare efficiency of
algorithms
21
 Big-Oh and Growth Rate
● We can say “f(n) is O(g(n))”
• It means growth rate of f(n) does not exceed growth
rate of g(n)
● This notation is called the Big O notation
• It uses capital O
● Provides a formal way to say 10n - 2 ≈ n
● We can rank algorithms according to their growth rate
using Big-Oh notation
• An algorithm whose time is proportional to n2, is said
to be O(n2).
• Algorithm requiring time proportional to n, it is O(n).
22
 Big-Oh Rules
● When using Big-O, we should:
• Focus on the dominant term
• Drop constants, coefficients & lower-order terms
• Rem: Worst-case performance
● Why?
• They do not affect growth rate
• Increasing n does not alter them!
● Examples (Big-O for some runtimes)
f(n) = 7n-3 ➔ O (n)
f(n) = 8n2log n + 5n2 + n ➔ O (n2log n)
f(n) = 3n2 + 5n ➔ O (n2) 23
Common orders of Big-O magnitude

● Common time
complexities:
• Constant  1
• Logarithmic  log n
• Linear  n
• N-Log-N  n log n
• Quadratic  n2
• Cubic  n3
• Exponential  2n

24
 Comparison of time complexities
Time complexity Comments
O(1) Excellent
O(log N) Good
O(N) Fair
O(N log N) Bad
O(N2) Worst
O(2N) Worst
O(N!) Worst
25
 Big-Oh Exercises
● Determine Big-Oh for each of these growth
rates:
• 7n - 2
• 3n3 + 20n2 + 5
• 3 log n + 5
• 3n3 + 20n2 + 5

26
● What is the time complexity, using big-O
notation, of each of the following?
• An algorithm that determines the average
of n numbers
• An algorithm that prints all characters in a
string of length n
• An iterative algorithm for computing n!

27
 Summary
• In this lesson, we looked at how to:
• Determine growth rate of an algorithm
• Determine Big-O values for a given algorithm

14
 Next lesson

• Recursion!!!

15
Q &A

16