DSAL-210 LECT 2:

UNDERSTANDING ALGORITHM
COMPLEXITY
By
W.G. Gondwe
Lecturer, Computer Networks
CS&IT Department
Malawi University of Science & Technology
Overview
• Revisting Time and Space Complexities
• Why We Analyse Algorithm Complexity
• Factors Affecting Complexity
• Best, Average and Worst Case Complexity
• How We Analyze Complexity
• Class Exercise

DSAL-210 Lecture 2 - Understanding Complexity 2

Time and Space Complexity
• Recap: We saw from the last lecture that any non-trivial algorithm
consumes resources
• From this point forward, we will refer to computer algorithms and
stay away from trivial examples e.g. cooking nsima
• The most important resources consumend by an algorithm are:
1. Time - any non-trivial algorithm takes time to execute and different
algorithms take different times. In computation, time translates directly to
CPU execution cycles.
- Recall from COMP-111 that CPU cycles are an important and scarce resource
1. Space - an algorithm requires working memory to carry out its job.
Memory is another scarce resource

DSAL-210 Lecture 2 - Understanding Complexity 3

Time and Space Complexity cont...
3 6 8 4 100 5 10

• For instance: Consider the search algorithm example from

Lecture 1 and note the following:
• The algorithm works on a list of integers - here the list is the data structure. If each
integer consumes 2KB of memory then our data structure consumes a total of 7x2KB
= 14KB
• While searching, our algorithm may require additional working memory e.g. 20KB.
This totally depends on how we design the algorithm
• We can (informally) say that the space complexity of the algorithm is 20KB
(Note that a different algorithm may require 30KB to finish the search. This one is more
complex in terms of space i.e. requires more space)

DSAL-210 Lecture 2 - Understanding Complexity 4

Time and Space Complexity cont...
3 6 8 4 100 5 10

• Similarly:
• While searching, our algorithm may require 500 CPU cycles, each
lasting 2 nanoseconds (a total of 1000ns)
• A different algorithm may require 300 CPU cycles, depending on how
we design it
• In this case, the second algorithm is less complex in terms of time
than the first one

DSAL-210 Lecture 2 - Understanding Complexity 5

Why Analyze Complexity?
• At this point, it must be clear that most computational problems can be
solved in different ways (algorithms)
• Each of these algorithms will consume its own set of resources e,g. one
might be faster than the other
• The more the resources consumed by an algorithm, the more complex
it is
• The best algorithm is the one that produces the desired output using
minimal running time and memory space i.e. minimum time and space
complexities
• We analyze algorithms to identify the 'best' algorithm for a given
problem so that we can produce efficient software programs

DSAL-210 Lecture 2 - Understanding Complexity 6

Factors Affecting Complexity
• Generally speaking, the following factors are the determiners
drivers of algorithm complexity:
1. Algorithm design - the type of algorithm and its design approach determines its
complexity e.g. for certain applications brute force algorithms are more complex
than divide-and-conquer algorithms
2. Data structure used - each data structure is efficient for given applications and
algorithms e.g. a binary-searc-tree (BST) data structure is efficient with search
algorithms
3. Input size - Generally, the larger the input, the more time & space the algorithm
requires
4. Scenario - An algorithm can execute in best, average or worst scenarios based
on status of input. Generally, best case scenarios are less complex than worst
case. We will dig further into this later in the lecture

DSAL-210 Lecture 2 - Understanding Complexity 7

Important Things to Note
• In literature and in this course, we are only concerned with time
complexity of algorithms and mostly disregard space complexity
(As a student, you should find out why)
• Hence, from this point forward, complexity will implicitly mean
time complexity
• We do not express complexity in absolute values (e.g. 500 CPU
cycles or 1000ns). The reason for this will be seen as we
progress in the course

DSAL-210 Lecture 2 - Understanding Complexity 8

Best, Average and Worst Case
• In the previous slide, we discovered that algorithm complexity can be
determined by the particular scenario under consideration
• Here, scenario generally refers to the status of our input data
• Going back to our list search example, the intention is to search for a
given element and report Yes if found and No otherwise
• The speed with which we can report Yes or No depends on where the
element is located. In this case:
1. Best case - element is in the first position that we look
2. Average case - element is somewhere in the middle
3. Worst case - element is in the last position that we look

DSAL-210 Lecture 2 - Understanding Complexity 9

Best, Average, Worst cont...
• On more imporant thing to note: In complexity theory and in this
course, we focus on the worst case scenario when analyzing all our
algorithms
• In this case, we can conclude for all the scenarios
• In other words, we prepare for the worst and hope for the best
• More concretely, if we discover that our algorithm has complexity X in the
worst case scenario, then we know that the best and average case
scenarios are always better than X
• Hence, worst case complexity speaks for all possible scenarios

DSAL-210 Lecture 2 - Understanding Complexity 10

How We Analyze Complexity
• The major objective of this course is for us to be able to design and
analyze algorithms for different problems so that we can choose the best
• Hence, knowing how to analyze algorithms forms a major part of the
skills required for completion of the course
• How do we analyze algorithms (general steps):
1. Given an algorithm, determine its input size (we usually express it as N)
2. Look for the non-trivial steps in the algorithm
3. Express the non-trivial steps as a function (f) of N (input size)
4. Consider the growth of rate of f as N grows (to infinity)
5. Reason and make conclusion based on the growth rate on f(N)

DSAL-210 Lecture 2 - Understanding Complexity 11

How We Analyze Complexity
• A common misconception:
• One would ask, given algorithm A, why can't we write a program for the algorithm,
run it on a computer and measure the time it takes to complete? (Benchmarking)
• In the same vein, given algorithm A and B, why can't we benchmark both and
compare them to find the better algorithm?
- This is the wrong approach for measuring algorithm complexity. The reasons will
be discussed in a new class forum.

DSAL-210 Lecture 2 - Understanding Complexity 12

Class Exercise
• Consider the following example:
• Suppose you are visiting a neighbourhood for the first time and you are
required to locate a particula residence among a series of houses.
1. How would you approach this problem?
2. Would you find the residence faster if you randomized your knocking?
3. What is the best case scenario?
4. What is the worst case scenario?
5. What is the average case scenario?

DSAL-210 Lecture 2 - Understanding Complexity 13

Next: Introduction To Asymptotic Notation

14