Analysis of Algorithms By

Hina Ali
Lecture # 1
Discussion Of Course outline
 The Algorithm What is your take on it
• Designing an Algorithm ?
• For example, lets see you need to add 3 numbers
• A problem can be written in C, python etc
• Before writing code for problem a blueprint is required • prerequisites
this is termed as algorithm 1. The problem that is to be solved by this algorithm:
• we can devise multiple algorithm for one problem Add 3 numbers and print their sum.
Properties of Algorithm: 2. The constraints of the problem that must be
1. It should terminate after a finite time. considered while solving the problem:
2. It should produce at least one output. The numbers must contain only digits and no other
characters.
3. It should take zero or more input.
3. The input to be taken to solve the problem:
4. It should be deterministic means giving the same
output for the same input case. The three numbers to be added.
5. Every step in the algorithm must be effective i.e. 4. The output to be expected when the problem is solved:
every step should do some work. The sum of the three i.e., a single integer value.
A detailed explanation is provided on referenced page for 5. The solution to this problem, in the given constraints:
interested students
The solution consists of adding the 3 numbers. It can be
done with the help of the ‘+’ operator, or bit-wise, or any
• Source = https://www.geeksforgeeks.org/introduction-to-algorithms/
other method.
• Useful Link : https://aofa.cs.princeton.edu/80strings/
How to write Algorithm
Natural Language
Pseudocode
Flowchart
Natural Language
As described in designing algorithm example
Pseudocode
High-level description of an algorithm
More structured than English prose
Less detailed than a program
Preferred notation for describing algorithms

Hides program design issues

Flowchart
Will be displayed from research paper
 problem

Algo 1 Algo 2 Algo 3 …..

Which one to choose?

1. Analyze All algorithms in term 2 factors i.e. of time(Faster execution) and Space(Less Memory)
2. Design and Analysis of Algorithm
• Design (How many algorithm are there for problem)
• Analysis (what will be time taken by each of them and what are memory requirements)
Then we chose algorithm
Analysis of Algorithm involves Evaluating
Following
• Memory : Units generalized as words
• CPU time: units are cycles
• Correctness: Desired output
 Analyzing Algorithm Types of Analysis
• Goal of the Analysis of Algorithms: Defines the input for which the algorithm takes
• To compare algorithms (or solutions) mainly
• In terms of running lime but also in terms of other factors (e.g., memory, developer
effort, ..) Worst case
• Running Time Analysis: How processing time increases as the size of the problem • a long time.
(input size) increases.
• Input types
• runs the slowest.
• Size of an array • Best case
• • Polynomial degree
• • Number of elements in a matrix • the least time.
•
•
• Number of bits in the binary representation of the input
• Vertices and edges in a graph.
• runs the fastest.
• Types of Analysis of Algorithm • Average case
• Provides a prediction about the running time or the
1. Priori Analysis:
• Running time of a given algorithm as a function of the
algorithm.
inputssize n i.e. f(n) • Assumes that the input is random.
• Running time increases as a function of input is called rate of lower Bound <= Average Time <= Upper Bound
growth The best. worst and average cases in the form of expressions.
• Does not depend on language, hardware As an
2. Posterior Analysis: example, let f(n) be the function which represents the given
• Dependent on the language algorithm.
• Hardware
f (n) = n2 + 500, for worst case
• Empirical Analysis :
• Algorithm Specification, RAM Computational Model
f (n) = n + 100n + 500, for best case
Problem set 1
The Problem Definition
Given an array of integers nums and an integer target, return indices of the two numbers such that they add up to target.
You may assume that each input would have exactly one solution, and you may not use the same element twice.
You can return the answer in any order.
Example 1:
Input: nums = [2,7,11,15], target = 9 Output: [0,1] Explanation: Because nums[0] + nums[1] == 9, we return [0, 1].
Example 2:
Input: nums = [3,2,4], target = 6 Output: [1,2]
Example 3:
Input: nums = [3,3], target = 6 Output: [0,1]

Constraints:
•2 <= nums.length <= 104
•-109 <= nums[i] <= 109
•-109 <= target <= 109
•Only one valid answer exists.

Follow-up: Can you come up with an algorithm that is less than O(n2) time complexity?
Notation for Analyzing Algorithm(Asymptotic
Notation, for comparing algorithms )
1)Big Oh (O) notation (Worst)

• Big O notation is used to explain the complexity of algorithm in form of algebraic terms.
• “f(n) is O(g(n))” iff for some constants c and n ₀,
0 ≤ f(n) ≤ cg(n) for all n > n₀
1. if f(n) = n2 +1 and g(n) = n3 for n₀>1 we can say f(n) = O(g(n))
2. If g(n) =c n2 can we say that f(n) = O(g(n))? If no, why if yes why? What will be value of c and n₀
3. What about O(n4), O(n4 +1 ), O(n5), O(n6)…….. ?
4. What if we create a graph of f(n)=n3 and g(n)=n4 ?

What about minimum g(n)?

10000
9000 10000
8000 9000
n2 cn3 n4 8000
7000
6000 7000
g(n) = n4 6000 O(g(n)) = n4
5000
4000 5000
4000
3000 h(n) = cn3 3000
2000
f(n) = n3 2000 f(n) =
1000 n3
1000
0
1 2 3 4 5 6 7 8 9 10 0
n2 n4
 Big Ω Omega Notation (Best case or Lower bound)

2) Ω Notation notation is used to explain the complexity of algorithm in form of algebraic

terms.
• “f(n) is Ω (g(n))” iff for some constants c and n₀,
0 ≤ cg(n) ≤ f(n) for all n > n₀
• f(n) = Ω(g(n)) where f(n) = cn2 + n2 and g(n) = n2 • f(n) = Ω(k(n)) where f(n) = cn2 + n and k(n) = n
• f(n) = Ω(h(n)) where f(n) = cn2 + cn and h(n) = n • f(n) = Ω(1) or Ω(l(n) ) where f(n) = cn2 + 1 and l(n) = 1

Omega Notation Graph

100
n Cn N2 cn2
n cn n2 cn2
90 1 3 1 2
80
f(n) = cn2 2 6 4 8
70 3 9 9 18
g(n) = n2 4 12 16 32
60
5 15 25 50
50
6 18 36 72
40 h(n) = cn 7 21 49 98
30 8 24 64 128
20 k(n) = n 9 27 81 162
10 10 30 100 200
0
1 2 3 4 5 6 7 8 9 10
3) Big θ theta Notation (Average)
• “f(n) is θ(g(n))” iff for some constants c1 c2 and n₀,
0 ≤ f(n) ≤ c1g(n) and 0 ≤ c2g(n) ≤ f(n) for all n > n₀
OR c2g(n) ≤ f(n) ≤ c1g(n)
• A function f(n) is said to be big theta θ of g(n) if it is big Oh and
omega of f(n) it bounds algorithm from upper and lower end and is
informally known as tighter bound.