Sanjivani Rural Education Society’s

Sanjivani College of Engineering, Kopargaon-423 603

(An Autonomous Institute, Affiliated to Savitribai Phule Pune University, Pune)
NAAC ‘A’ Grade Accredited, ISO 9001:2015 Certified

Department of Computer Engineering

(NBA Accredited)

Subject- Design and Analysis of Algorithms (DAA) [CO 301)]

Unit 1:- Problem Solving &Basic of Algorithm

Prof. Abhijit S. Bodhe

Assistant Professor
Department of Computer Engineering
E-mail : [email protected]
Contact No: 7709 340 570
 Unit 1:- Problem Solving &Basic of Algorithm

• Problem Solving: Definition of Problem,

• Problem solving principles,
• Classification & Strategies to solve problems,
• Algorithm: Definition,
• Asymptotic Notations, Time Complexities, Best, Worst & Average Case
Analysis.
• Types of algorithms: Randomized, Approximate & Exact.
• Case study: Brute Force Method.
• Application: Medical Domain Problem (MRI Scanner) and Algorithm
for the MRI Scanner

DEPARTMENT OF COMPUTER ENGINEERING, Sanjivani COE, Kopargaon 2

 Problem & Solving
• A problem is a question raised to reveal difficulty, stress, or doubt due to a
matter or situation OR Thing that is difficult to deal with or to understand.
• Problem solving definition is the use of ideas, skills, or facts to achieve
the solution to a problem, so a desired outcome can be reached.
• problem solving is:-
1. Defined as the process in which people attempt to overcome
difficulties.-Psychology
2. It’s a Mental process people, go through to discover, analyze, and solve
problems.-Cognitive Science
3. Using logical thinking to develop resolutions, and using these
resolutions until one of them works – Logic Problem
DEPARTMENT OF COMPUTER ENGINEERING, Sanjivani COE, Kopargaon 3
 Problems in computer science

• Slow Performance. ...

• Loud Noises From Computer. ...
• Computer Constantly Freezes. ...
• Internet Browsing Is Slow or Unresponsive. ...
• You're Seeing a BSOD (Blue Screen of Death) ...
• Screen Display Is Truncated or Has Low Resolution.. .
• So On . .!!

DEPARTMENT OF COMPUTER ENGINEERING, Sanjivani COE, Kopargaon 4

 Problem solving in Engineering

• Problem solving is an essential skill for an engineer to have, is defined

as a methodical and systematic step approach to determine the best
feasible action and resolution when confronted with a problem.
• These steps typically include: identifying the problem, collecting
relevant data, listing and ranking possible solutions, developing a
detailed plan for the top possible solutions (mathematical models and
computational methods commonly used), reevaluating the plan,
implementing the plan, and checking results.

DEPARTMENT OF COMPUTER ENGINEERING, Sanjivani COE, Kopargaon 5

 Principles of problem solving

1. Each problem should be seen as unique

2. Focus on the reasons for solving the problem
3. Seek solutions using a long-term perspective
4. View the problem(s) from a systems approach
5. Learn to work with minimal information
6. Keep the people factor in mind when developing solutions
7. Incorporate the timeline principle

DEPARTMENT OF COMPUTER ENGINEERING, Sanjivani COE, Kopargaon 6

 Steps of problem solving in computer science
1: Identify the Problem
2: Analyze the Problem
3: Describe the Problem
4: Look for Root Causes
5: Develop Alternate Solutions
6: Implement the Solution
7: Measure the Results

DEPARTMENT OF COMPUTER ENGINEERING, Sanjivani COE, Kopargaon 7

 Classification of problem
• Simple classification is when the information and assumptions of a
problem are in the realm of "known knowns“
• Complicated classification has "known unknowns" that are sensed, but
that need to be analyzed before a response can be given. An example
of this could be reducing the amount of developers and LOCs relied
upon in a project.
• Complex classification is in the realm of "unknown unknowns" where
the environment is constantly changing with multiple factors at play.
• Chaotic classification is for "critical" problems that have no order and
there is no obvious cause and effect relationship.

DEPARTMENT OF COMPUTER ENGINEERING, Sanjivani COE, Kopargaon 8

 Classification of problem in computer science
• Complexity theory is a subfield of computer science that deals with
classifying problems into a set of categories that specify the solvability
of these problems.
1. Easy Problems (P)
2. Hard Problems (EXP)
3. Solvable Problems (R)
4. Deterministic and nondeterministic problems
5. NP problems
6. NP-hard vs. NP-complete
• https://towardsdatascience.com/complexity-theory-101-problems-
classification-9793f05e42e1
DEPARTMENT OF COMPUTER ENGINEERING, Sanjivani COE, Kopargaon 9
 Problem Solving Strategies(Generalized)
1. Define the Problem.
What are you trying to solve?
2. Brainstorm Ideas.
What are some ways to solve the problem?
3. Decide on a Solution.
What are you going to do?
4. Implement the Solution.
What are you doing?
5. Review the Results.
• What did you do?
DEPARTMENT OF COMPUTER ENGINEERING, Sanjivani COE, Kopargaon 10
 What are Algorithms
• An algorithm is a procedure used for solving a problem or performing a
computation.
• Algorithms act as an exact list of instructions that conduct specified actions step
by step in either hardware- or software-based routines.
• A problem-solving algorithm is a procedure that is guaranteed to produce a
solution if it is followed strictly.

DEPARTMENT OF COMPUTER ENGINEERING, Sanjivani COE, Kopargaon 11

 Time Complexities
• Time complexity estimates how an algorithm performs regardless of
the kind of machine it runs on.
• Finding out the time complexity of your code can help you develop
better programs that run faster.
• One can get the time complexity, by “counting” the number of
operations performed by your code.
• To calculate the time complexity of our code when we have the
following elements:
1. Basic operations like assignments, bit, and math operators.
2. Loops and nested loops
3. Function invocations and recursion
DEPARTMENT OF COMPUTER ENGINEERING, Sanjivani COE, Kopargaon 12
Time Complexities Calculation with example

DEPARTMENT OF COMPUTER ENGINEERING, Sanjivani COE, Kopargaon 13

 Classification of Time Complexities
1.(1) - Constant time
2.(n) - Linear time
3.(n^2) - Quadratic time
4.(n^c) - Polynomial time
5.(log n) - Logarithmic time
6.(n log n) - Linearithmic
7.(2^n) - Exponential time
8.(n!) - Factorial time

DEPARTMENT OF COMPUTER ENGINEERING, Sanjivani COE, Kopargaon 14

 Asymptotic Notations
• Informally, the term asymptotic means approaching a value or curve arbitrarily
closely. formally, let be a continuous variable tending to some limit.
• Asymptotic notations are, the mathematical notations used to describe the
running time of an algorithm when the input tends towards a particular value or
a limiting value.
• For example: In bubble sort, when the input array is already sorted, the time
taken by the algorithm is linear i.e. the best case. O(n)
• But, when the input array is in reverse condition, the algorithm takes the
maximum time (quadratic) to sort the elements i.e. the worst case. O(N2)
• When the input array is neither sorted nor in reverse order, then it takes
average time. O(N2)
• These durations are denoted using asymptotic notations.
DEPARTMENT OF COMPUTER ENGINEERING, Sanjivani COE, Kopargaon 15
 Sorting algorithm & Comparison of algorithms

Data Time Time

Time complexity: Space complexity:
Algorithm struct complexity: complexity:
Average Worst
ure Best Worst
Quick sort Array O(n log(n)) O(n log(n)) O(n2) O(n)
Merge sort Array O(n log(n)) O(n log(n)) O(n log(n)) O(n)
Heap sort Array O(n log(n)) O(n log(n)) O(n log(n)) O(1)
Smooth sort Array O(n) O(n log(n)) O(n log(n)) O(1)
Bubble sort Array O(n) O(n2) O(n2) O(1)
Insertion sort Array O(n) O(n2) O(n2) O(1)
Selection sort Array O(n2) O(n2) O(n2) O(1)
Bogo sort Array O(n) O(n n!) O(∞) O(1)

DEPARTMENT OF COMPUTER ENGINEERING, Sanjivani COE, Kopargaon 16

 Asymptotic analysis (Notations)

• Asymptotic analysis is input bound i.e., if there's no input to the

algorithm, it is concluded to work in a constant time. Other than the
"input" all other factors are considered constant.
• Asymptotic analysis refers to computing the running time of any
operation in mathematical units of computation.
• For example, the running time of one operation is computed as f(n) and
may be for another operation it is computed as g(n2).
• This means the first operation running time will increase linearly with the
increase in n and the running time of the second operation will increase
exponentially when n increases.

DEPARTMENT OF COMPUTER ENGINEERING, Sanjivani COE, Kopargaon 17

 Asymptotic Notations

• Following are the commonly used asymptotic notations to calculate

the running time complexity of an algorithm.
1. Big-O notation Ο Notation
2. Omega notation Ω Notation
3. Theta notation θ Notation

DEPARTMENT OF COMPUTER ENGINEERING, Sanjivani COE, Kopargaon 18

 Big-O notation
• Big-O notation represents the upper bound of the
running time of an algorithm. Therefore, it gives
the worst-case complexity of an algorithm.
• If f(n) describes the running time of an
algorithm,
Then, f(n) is O(g(n)) if there exist a positive
constant C and n0
such that, 0 ≤ f(n) ≤ cg(n) for all n ≥ n0

DEPARTMENT OF COMPUTER ENGINEERING, Sanjivani COE, Kopargaon 19

 Omega notation
• Big-Omega (Ω) notation gives a lower bound
for a function f(n) to within a constant factor.
• Let g and f be the function from the set of
natural numbers to itself.
The function f is said to be Ω(g), if there is a
constant c > 0 and a natural number n0
• such that c*g(n) ≤ f(n) for all n ≥ n0

DEPARTMENT OF COMPUTER ENGINEERING, Sanjivani COE, Kopargaon 20

 Theta Notation
• Let g and f be the function from the set of
natural numbers to itself.
• The function f is said to be Θ(g),
• if there are constants c1, c2 > 0 and a natural
number n0
• such that c1* g(n) ≤ f(n) ≤ c2 * g(n)
for all n ≥ n0

DEPARTMENT OF COMPUTER ENGINEERING, Sanjivani COE, Kopargaon 21

 Best case, Worst case, Average case analysis
• Best case is the function which performs the minimum number of
steps on input data of n elements.
• Worst case is the function which performs the maximum number of
steps on input data of size n.
• Average case is the function which performs an average number of
steps on input data of n elements.
• Best case = fastest time to complete, with optimal inputs chosen. For
example, the best case for a sorting algorithm would be data that's
already sorted.
• Worst case = slowest time to complete, with pessimal inputs chosen. ...
• Average case = arithmetic mean.
DEPARTMENT OF COMPUTER ENGINEERING, Sanjivani COE, Kopargaon 22
 lower bound and upper bound of algorithm
• The Lower and Upper Bound Theory provides a way to find the lowest
complexity algorithm to solve a problem.
1. Lower Bound –
Let L(n) be the running time of an algorithm A(say), then g(n) is the Lower
Bound of A if there exist two constants C and N such that L(n) >= C*g(n)
for n > N.
Lower bound of an algorithm is shown by the asymptotic notation Omega
2. Upper Bound –
Let U(n) be the running time of an algorithm A(say), then g(n) is the Upper
Bound of A if there exist two constants C and N such that U(n) <= C*g(n) for n
> N.
Upper bound of an algorithm is shown by the asymptotic notation Big Oh(O).
DEPARTMENT OF COMPUTER ENGINEERING, Sanjivani COE, Kopargaon 23
 Sorting algorithm & Comparison of algorithms

Data Time Time

Time complexity: Space complexity:
Algorithm struct complexity: complexity:
Average Worst
ure Best Worst
Quick sort Array O(n log(n)) O(n log(n)) O(n2) O(n)
Merge sort Array O(n log(n)) O(n log(n)) O(n log(n)) O(n)
Heap sort Array O(n log(n)) O(n log(n)) O(n log(n)) O(1)
Smooth sort Array O(n) O(n log(n)) O(n log(n)) O(1)
Bubble sort Array O(n) O(n2) O(n2) O(1)
Insertion sort Array O(n) O(n2) O(n2) O(1)
Selection sort Array O(n2) O(n2) O(n2) O(1)
Bogo sort Array O(n) O(n n!) O(∞) O(1)

DEPARTMENT OF COMPUTER ENGINEERING, Sanjivani COE, Kopargaon 24

 Types of Algorithms & Applications

1. Randomized Algorithms
2. Approximate Algorithms
3. Exact Algorithms
• Applications of Algorithms

DEPARTMENT OF COMPUTER ENGINEERING, Sanjivani COE, Kopargaon 25

 Randomized Algorithms
• Imagined as tossing coins during the computation, Depending on the
outcome of the toss, the algorithm may split up its computation path.
• Definition :-An algorithm that uses random numbers to decide what to do
next anywhere in its logic is called Randomized Algorithm.
• Typically, this randomness is used to reduce time complexity or space
complexity in algorithms & most times helpful to avoid infinite looping.
• Example:- Freivalds algorithm is a probabilistic randomized algorithm, use
to verify matrix multiplication, other hand, randomness can be useful in
quicksort in pivot selection.
• two main types of randomized algorithms: Las Vegas algorithms and Monte-
Carlo algorithms.
DEPARTMENT OF COMPUTER ENGINEERING, Sanjivani COE, Kopargaon 26
 Las Vegas algorithms
• In Las Vegas algorithms, the algorithm may use the randomness to
speed up the computation, but the algorithm must always return the
correct answer to the input.
• The usual definition of a Las Vegas algorithm includes the restriction
that the expected runtime be finite, where the expectation is carried out
over the space of random information.
• Las Vegas algorithms arise frequently in search problems, For
example, one looking for some information online might search
related websites for the desired information. The time complexity thus
ranges from getting "lucky" and finding the content immediately, to
being "unlucky" and spending large amounts of time. Once the right
website is found, then there is no possibility of error.
DEPARTMENT OF COMPUTER ENGINEERING, Sanjivani COE, Kopargaon 27
 Monte-Carlo algorithms
• Monte-Carlo algorithms do not have the former restriction, that is,
they are allowed to give wrong return values. However, returning a
wrong return value must have a small probability,
• Las Vegas algorithms can be contrasted with Monte Carlo algorithms,
in which the resources used are bounded but the answer may be
incorrect with a certain (typically small) probability.
• Two examples of such algorithms are Karger–Stein algorithm and
Monte Carlo algorithm for minimum Feedback arc set
• Karger's algorithm is a randomized algorithm to compute a minimum
cut of a connected graph. It was invented by David Karger and first
published in 1993.
DEPARTMENT OF COMPUTER ENGINEERING, Sanjivani COE, Kopargaon 28
 Exact Algorithms
• An exact algorithm finds the solution to the problem asked.
• exact algorithms are algorithms, that always solve an optimization
problem to optimality.
• This is by contrast with an approximate algorithm, which only gets
close to the solution.
• There are practical situations such that exact algorithms cannot be
used and one must content oneself with approximations.
• Why do we need approximation algorithms?
• when finding an optimal solution is intractable, but can also be used in
some situations where a near-optimal solution can be found quickly
and an exact solution is not needed.
DEPARTMENT OF COMPUTER ENGINEERING, Sanjivani COE, Kopargaon 29
 Approximate Algorithms
• approximation algorithms are efficient algorithms that find
approximate solutions to optimization problems, with provable
guarantees on the distance of the returned solution to the optimal one.
• This technique does not guarantee the best solution. The goal of the
approximation algorithm is to come as close as possible to the optimal
solution in polynomial time(P).
• we have to optimize a certain variable in accordance to some other
variables. Optimization means finding maximum or minimum.
• Examples:-
1.Finding the shortest path between two vertices in a graph.
2.For the traveling salesperson problem, the optimization problem is to find the
shortest cycle, and the approximation problem is to find a short cycle.
DEPARTMENT OF COMPUTER ENGINEERING, Sanjivani COE, Kopargaon 30
Case study: Brute Force Method

DEPARTMENT OF COMPUTER ENGINEERING, Sanjivani COE, Kopargaon 31

 Medical Domain Problem
• (MRI Scanner) and Algorithm for the MRI Scanner:-

DEPARTMENT OF COMPUTER ENGINEERING, Sanjivani COE, Kopargaon 32

• Done. . .!!

DEPARTMENT OF COMPUTER ENGINEERING, Sanjivani COE, Kopargaon 33

 Unit 2:-Divide-&-Conquer and Greedy Strategy
• Divide and Conquer Strategy: Principle, Control Abstraction,
• Time complexity Analysis, Binary search algorithm.
• Case study: Merge Sort.
• Application: Google's Binary Search to Identify Malware.

• Greedy Strategy: Principle, Control Abstraction,

• Time Complexity Analysis, Knapsack Problem,
• Case study: Scheduling Algorithms-Job Scheduling.
• Application: Finding the Shortest Path on Google Ma

DEPARTMENT OF COMPUTER ENGINEERING, Sanjivani COE, Kopargaon 34

DEPARTMENT OF COMPUTER ENGINEERING, Sanjivani COE, Kopargaon 35