Design and Analysis of

Algorithms
Week-1
 Instructor Information

• Name
– Zain Ul Islam Adil

• Contact information
– [email protected]

• Research field
– Network Security
 Course Information
• Name
– Design and Analysis of Algorithm
• Course Code
– CSAA-242
• Lectures
– 16
• Quizzes
– 4
• Assignments
– 3
• Attendance
– 75%
 What is course about?
The theoretical study of design and analysis of
computer algorithms
Basic goals for an algorithm:
• always correct
• always terminates
• performance
 Performance often draws the line between
them
Design and Analysis of Algorithms
• Analysis: predict the cost of an algorithm in
terms of resources and performance

• Design: design algorithms which minimize the

cost
 Outline
1. Program cost and asymptotic analysis
2. Divide and conquer
3. Data structures - heaps
4. Dynamic programming
5. Depth-First-Search
6. Shortest paths in graphs
7. Greedy algorithms
Textbooks:
8. T. H. Cormen, C. E. Leiserson, R. L. Rivest and C. Stein.
Introduction to Algorithms
Why is this course
required?
• Algorithms are fundamental to CS.
• Algorithms are useful.
Why is this course
required?
• Algorithms are fundamental to CS.
• Algorithms are useful.
Algorithms are
fundamental

Operating Systems Machine learning

Compilers
Networking Computational Biology
Why is this course
required?
• Algorithms are fundamental to CS.
• Algorithms are useful.
Algorithms are useful
• As we get more and more
data and problem sizes get
bigger and bigger,
algorithms become more
and more important.
 Algorithms

Algorithm: a finite set of well-defined instructions for

accomplishing some task.

Algorithm is a well defined computational procedure

that takes some value (s) as input, and produces some
value (s) as output.

Algorithm is finite number of computational

statements that transform input into the output
Evaluating algorithms
Two questions we ask about an algorithm
1. Is it correct?
2. Is it efficient?

Correctness is of utmost importance.

It is easy to design a highly efficient but incorrect algorithm.

Effi ciency with respect to:

 Running time
 Space (amount of memory used)
 Network traffic
 Other features (e.g. number of times secondary storage is
accessed)
 Measuring running time
On an actual computer, the running time of a program depends on
many factors:
1. The running time of the algorithm.
2. The input of the program.
3. The quality of the implementation
(e.g. quality of the code generated by the compiler).
4. The machine running the program.
We are concerned with 1.
Running Time of an
Algorithm
 Depends upon
 Input Size
 Nature of Input
 Generally time grows with size of input, so running
time of an algorithm is usually measured as function
of input size.

 Running time is measured in terms of number of

steps/primitive operations performed

 Independent from machine, OS

 Fast computers vs efficient algorithms

Many recent innovations rely on

 fast computers
 efficient algorithms.

Which is more important?

 Fast computers vs efficient algorithms

Many recent innovations rely on

 fast computers
 efficient algorithms.

Which is more important?

But first we need to learn natural log (ln)

 Ln(x)
• Ln(x) can be computed using:

ln( x) log e x 2.303log10 x

• [] https://www.youtube.com/watch?v=Z2AY-bWsezk
 how ? log e x 2.303log10 x

• We will make use of following basic info:

ln( x) log e x
log10 x
log e x 
log10 e
log10 x

log10 2.7182
log10 x

0.4342
2.303log10 x
 Natural Log (ln) Examples
• Verify following output:
Natural Log (ln) Identities
 Fast computers vs efficient algorithms

Many recent innovations rely on

 fast computers
 efficient algorithms.

Which is more important?

 The importance of efficient algorithms
The cost of an algorithm can be quantified by the number of steps T (n)
in which the algorithm solves a problem of size n.
Imagine that a certain problem can be solved by four different
algorithms, with T (n) = n, n 2 , n 3 , and 2 n , respectively.
Question: what is the maximum problem size that the algorithm can
solve in a given time?
Assume that a computer is capable of 1010 steps per second.

Cost T (n) Maximum problem size solvable in

(Complexity) 1 second 1 hour 1 year
n 1010 13 17
3.6 × 10 3 × 10
n2 105
6 8
6× 5×
10 10
n3 2154 33000 680000
2n 33 45 58
[] https://www.cs.ox.ac.uk/files/13277/asymptotics.pdf
 Faster computers vs more efficient algorithms

Suppose a faster computer is capable of 1016 steps per second.

Cost T (n) Max. size before Max. size now

n s1 106 × s1
n2 s2 1000 × s2
n3 s3 100 × s3
2n s4 s 4 + 20

A 106× increase in speed results in only a factor-of-100 improvement

if cost is n 3 , and only an additive increase of 20 if cost is 2 n .
Conclusions As computer speeds increase ...
1. ... it is algorithmic efficiency that really determines the increase
in problem size that can be achieved.
2. ... so does the size of problems we wish to solve.
Thus, designing efficient algorithms becomes even more
important!
 Algorithms as a technology
• Even if computers are infinitely fast and memory is
free. Still need to study algorithms? The answer is
Yes.
• The reason is:
– You need to demonstrate that method terminates
– Terminates with a correct answer
– Algorithm within bounds of a good software practice
– Computer are fast, but not infinitely fast
– Memory is cheap, but it is not free
 Efficiency: Algorithm
• Algorithms devised to solve the same problem differ
in efficiency.
• For a concrete example:

Insertion Sort Merge Sort

Computer Speed Faster Slower
Number to sort 1 million 1 million
Instructions/ second 1 billion 10 million
Programmer Expert Average
Time 2n2 50 N log(n)
 Efficiency: Algorithm
Insertion Sort Merge Sort

2(106 ) 2 50 106 ln106

109 107
 Efficiency: Algorithm
Insertion Sort Merge Sort
50 106 ln106
2(106 ) 2
107
109
50 ln106
2 103 
10
2000 seconds
5 ln106
5 6 ln10
5 6 ln10
30 2.303
69.09 100 seconds
 Comparing Growth Rate
• Ideally, we would like our algorithms to run in linear or n-log-
n time. Algorithms with quadratic or cubic running times are
less practical, but algorithms with exponential running times
are infeasible for all but the smallest sized inputs

constant logarithmic linear N-Log-N Quadratic cubic exponential

n O(1) O(lg n) O(n) O(n lg n) O(n2) O(n3) O(2n)
4 1 2 4 8 16 64 16
16 1 3 16 48 256 4096 65536

• [] https://www.cpp.edu/~ftang/courses/CS240/lectures/analysis.htm
• [] https://www.calculator.net/log-calculator.html?xv=4&base=e&yv=&x=23&y=15
 Comparing Growth Rate
• Ideally, we would like our algorithms to run in linear or n-
log-n time. Algorithms with quadratic or cubic running
times are less practical, but algorithms with exponential
running times are infeasible for all but the smallest sized
inputs

• [] https://course.ccs.neu.edu/cs5002f18-seattle/lects/cs5002_lect9_fall18_notes.pdf