Module-2

Analysis & Design of Algorithms(BCS401)

By :
Vishal Sathawane
Assistant Professor
Department of Computer Science and Engineering
Module-2
Syllabus:-
✔ BRUTE FORCE APPROACHES (contd..): Exhaustive Search (Travelling Salesman probem
and Knapsack Problem).
✔ DECREASE-AND-CONQUER: Insertion Sort, Topological Sorting.
✔ DIVIDE AND CONQUER: Merge Sort, Quick Sort, Binary Tree Traversals, Multiplication of
Large Integers and Strassen’s Matrix Multiplication.

Text Book:
✔ Introduction to the Design and Analysis of Algorithms, By Anany Levitin, 3rd Edition
(Indian), 2017, Pearson. Chapter 3(Section 3.4), Chapter 4 (Sections 4.1,4.2), Chapter 5
(Section 5.1,5.2,5.3, 5.4)

Reference Book:
✔ Computer Algorithms/C++, Ellis Horowitz, SatrajSahni and Rajasekaran, 2nd Edition, 2014,
Universities Press.
✔ Introduction to Algorithms, Thomas H. Cormen, Charles E. Leiserson, Ronal L. Rivest,
Clifford Stein, 3rd Edition, PHI.
✔ Design and Analysis of Algorithms, S. Sridhar, Oxford (Higher Education)
Brute Force Approach (Contd….)
Exhaustive search:
Exhaustive search is a problem-solving algorithm that checks every possible solution
to find the best one. It's also known as brute force, generate and test, or direct
search.

How it works

▪ Checks every possible combination of input values

▪ Evaluates all peaks if the response surface has multiple peaks
▪ Guaranteed to find the best solution from the specified domain

Strengths and weaknesses

▪ Guaranteed to find the best solution

▪ Evaluates all peaks if the response surface has multiple peaks

▪ The number of combinations to be evaluated can become very large

Travelling Salesman Problem
Given n cities a salesman starts at specified city and visit all the n-1 cities only once
and return back to the starting city.

Objective: To find the route through the cities that minimizes the cost(Maximize
profit)

TSP problem can be solved using

Brute Force
Dynamic Programming
Travelling Salesman Problem (Brute Force)

Get all the routes from one city to other city by taking various permutations
Compute route cost for each permutation and select the shortest one

Exhaustive Sequential Search

Compute cost of all tour
Select the tour with minimum cost
Travelling Salesman Problem (Brute Force)
TSP (Brute Force Analysis)
TSP (Brute Force: Advantages and Disadvantages )
Advantages:
• Always gives Optimal solution
• Straight Forward approach
• Simple Technique

Disadvantages:
• Inefficient Method ((n-1)! Possible routes)
• Time consuming to find optimal route
Knapsack Problem
You are given the following-
•A knapsack (kind of shoulder bag) with limited weight capacity.
•Few items each having some weight and value.

The problem states-

Which items should be placed into the knapsack such that-
•The value or profit obtained by putting the items into the knapsack is maximum.
•And the weight limit of the knapsack does not exceed.

Knapsack problem can be solved using

Brute Force
Dynamic Programming
Knapsack Problem
Knapsack Problem Variants-

Knapsack problem has the following two variants-

1. Fractional Knapsack Problem
2. 0/1 Knapsack Problem

In 0/1 Knapsack Problem,

• As the name suggests, items are indivisible here.
• We can not take the fraction of any item.
• We have to either take an item completely or leave it completely.
• It is solved using dynamic programming approach.
Knapsack Problem (Brute Force)
0-1 Knapsack problem
• 1. Given a knapsack with maximum capacity W, and a set S consisting of n items.
• 2. Each item i has some weight wi and benefit value vi (all wi and W are integer
values)
• 3. Problem: How to pack the knapsack to achieve maximum total value of packed
items?
Problem, in other words, is to find
max vi subject to wi<=W
• The problem is called a "0-1" problem, because each item must be entirely accepted
or rejected.
Knapsack Problem (Brute Force: Problem)
Item I1 I2 I3
Weight 20 25 10
Profit 30 40 35

W = 40, n=3
Knapsack Problem (Brute Force: Efficiency)

• There are 16 possible subsets including null set for 4

items.
• Hence for n items, there are 2^n possible subsets.
• Therefore efficiency is 2^n
DECREASE-AND-CONQUER
Decrease and conquer is a technique used to solve problems by reducing the size of the
input data at each step of the solution process. This technique is similar to
divide-and-conquer, in that it breaks down a problem into smaller subproblems, but the
difference is that in decrease-and-conquer, the size of the input data is reduced at each step.
The technique is used when it’s easier to solve a smaller version of the problem, and the
solution to the smaller problem can be used to find the solution to the original problem.
DECREASE-AND-CONQUER
“Divide-and-Conquer” vs “Decrease-and-Conquer”:
The name “divide and conquer” should be used only when each problem may generate two
or more subproblems. The name decrease and conquer has been proposed instead for the
single-subproblem class. According to this definition, Merge Sort and Quick Sort comes under
divide and conquer (because there are 2 sub-problems) and Binary Search comes under
decrease and conquer (because there is one sub-problem).

Implementations of Decrease and Conquer :

This approach can be either implemented as top-down or bottom-up.
• Top-down approach : It always leads to the recursive implementation of the problem.
• Bottom-up approach : It is usually implemented in iterative way, starting with a solution to
the smallest instance of the problem.
DECREASE-AND-CONQUER
Advantages of Decrease and Conquer:

1. Simplicity: Decrease-and-conquer is often simpler to implement compared to

other techniques like dynamic programming or divide-and-conquer.

2. Efficient Algorithms: The technique often leads to efficient algorithms as the size
of the input data is reduced at each step, reducing the time and space complexity
of the solution.

3. Problem-Specific: The technique is well-suited for specific problems where it’s

easier to solve a smaller version of the problem.

Disadvantages of Decrease and Conquer:

1. Problem-Specific: The technique is not applicable to all problems and may not be
suitable for more complex problems.

2. Implementation Complexity: The technique can be more complex to implement

when compared to other techniques like divide-and-conquer, and may require
more careful planning.
Variations of Decrease and Conquer
There are three major variations of decrease-and-conquer:

1. Decrease by a constant : Ex. Insertion Sort, Topological Sorting BFS, DFS

2. Decrease by a constant factor: Ex. Binary Search

3. Variable size decrease: Ex. Interpolation Search

Insertion Sort Algorithm
Insertion sort is a simple sorting algorithm that works by iteratively inserting each element of
an unsorted list into its correct position in a sorted portion of the list. It is like sorting playing
cards in your hands. You split the cards into two groups: the sorted cards and the unsorted
cards. Then, you pick a card from the unsorted group and put it in the right place in the
sorted group.

• We start with second element of the array as first element in the array is assumed to be
sorted.
• Compare second element with the first element and check if the second element is
smaller then swap them.
• Move to the third element and compare it with the first two elements and put at its
correct position
• Repeat until the entire array is sorted.
Insertion Sort Algorithm
ALGORITHM InsertionSort(A[0..n − 1])
//Sorts a given array by insertion sort
//Input: An array A[0..n − 1] of n orderable elements
//Output: Array A[0..n − 1] sorted in nondecreasing order
for i ←1 to n − 1 do
v ←A[i]
j ←i − 1
while j ≥ 0 and A[j ]> v do
A[j + 1]←A[j ]
j ←j − 1
A[j + 1]←v
Insertion Sort Program
Complexity Analysis of Insertion Sort :
Time Complexity of Insertion Sort

• Best case: O(n), If the list is already sorted, where n is the number of
elements in the list.

• Average case: O(n2), If the list is randomly ordered

• Worst case: O(n2), If the list is in reverse order

Space Complexity of Insertion Sort

•Auxiliary Space: O(1), Insertion sort requires O(1) additional space, making it a
space-efficient sorting algorithm.
Topological Sort Algorithm
It is linear ordering of graph vertices such that for every
directed edge uv from vertex u to vertex v, u comes before
v in the ordering.

• Applicable on DAG(Direct Acyclic Graph)

• Linear running time complexity.
• Topological sort is not unique.
Topological Sort Algorithm
A
B C

D
Topological Sort Algorithm Using DFS
A

B C

G E

H
Topological Sort Algorithm Using DFS
Algorithm
1. Start with an empty list to store the topological order.
2. Pick any unvisited node and perform a DFS from that node.
3. During the DFS traversal, if you encounter a node with node unvisited neighbours,
add it to the topological order list.
4. Repeat this process for all unvisited nodes in the graph.
Topological Sort Algorithm Using BFS (Kahn’s)
L <- Empty list that will contain the sorted elements
S <- Set of all nodes with no incoming edge
while S is not empty do
remove a node n from S
add n to L
for each node m with an edge e from n to m do
remove edge e from the graph
if m has no other incoming edges then insert m into S
if graph has edges then
return error (graph has at least one cycle)
else
return L (a topologically sorted order)
Topological Sort Algorithm Using BFS (Kahn’s)
A

B C

G E

H
Topological Sort Algorithm Using BFS (Kahn’s)
The time complexity of performing the topological sort

The time complexity of topological sort is O(V + E), where V is the number of
vertices and E is the number of edges in a graph. This is because the algorithm
visits each vertex and edge only once.

Explanation

• The time complexity of building the graph from the given edges is O(E).
• The topological sort algorithm iterates through all vertices in the input graph
exactly once, which takes O(V) time.
• The dfs function is called for each unvisited vertex, and within dfs, all the
neighbors of the vertex are explored. This results in a total of E (the number of
edges) work being done during the DFS traversal.
Topological Sort Algorithm
A B

C D E

F G
Topological Sort Algorithm
Divide-and-conquer algorithms
The divide-and-conquer strategy solves a problem by:
1. Breaking it into subproblems that are themselves smaller instances of the
same type of problem
2. Recursively solving these subproblems
3. Appropriately combining their answers
Integer multiplication
Input: Two positive n digit integers, a and b.
Output: a*b
The naive approach is to simply multiply the two numbers, which
takes O(n2) time because we need to multiply each digit in one
number with those in the second number, put them in the correct
position and add them as shown below.
Integer multiplication using Divide and
Conquer (Algorithm)
Integer multiplication using Divide and
Conquer (Algorithm)
Matrix multiplication
Direct Method:Suppose we want to multiply two n x n matrices, A and B.Their product,
C=AB, will be an n by n matrix and will therefore have n^2 elements.
The number of multiplications involved in producing the product in this way is Θ(n^3).

Divide and Conquer method: Using the divide and conquer matrix achieved through
multiplying the submatrices recursively.

multiplication is In the above method, we do 8 multiplications for matrices of size n/2 x n/2
and 4 additions.
Addition of two matrices takes O(n^2) time. So the time complexity can be written as
T(n)= 8T(n/2) + O(n^2) which happen to be O(n^3); same as the direct method
Strassen’s Matrix multiplication
Divide and Conquer through Strassen's Method: By using the divide-and-conquer approach
proposed by Strassen in 1969, we can reduce the number of multiplications.
Multiplication of 2×2 matrices: The principal insight of the algorithm lies in the discovery that
we can find the product C of two 2 × 2 matrices A and B with just 7 multiplications as opposed
to the eight required by the brute-force algorithm. This is accomplished by using the following
formulas:

Thus, to multiply two 2×2 matrices, Strassen’s algorithm makes seven multiplications and 18
additions/subtractions, whereas the brute-force algorithm requires eight multiplications and
four additions.
Maximum of the height of Tree (Algorithm)
ALGORITHM Height(Tn)
//Computes recursively the height of a binary tree
//Input: A binary tree Tn
//Output: The height of Tn
if Tn = ∅ return −1
else return max{Height(Tnlef t ), Height(Tnright)} + 1
Binary Search(Algorithm)
ALGORITHM binary_search(A, n, T)
// A: array of sorted element
// n: No. of elements in array
// T: Value you want to search
L := 0
R := n − 1
while L ≤ R do
m := floor((L + R) / 2)
if A[m] < T then
L := m + 1
else if A[m] > T then
R := m − 1
else:
return m
return unsuccessfulss
Merge Sort Algorithm :
ALGORITHM Mergesort(A[0..n − 1])
//Sorts array A[0..n − 1] by recursive mergesort
//Input: An array A[0..n − 1] of orderable elements
//Output: Array A[0..n − 1] sorted in nondecreasing order
if n > 1
copy A[0..n/2 − 1] to L[0..n/2 − 1]
copy A[n/2..n − 1] to R[0..n/2 − 1]
Mergesort(L[0..n/2 − 1])
ALGORITHM Merge(L[0..p − 1], R[0..q − 1], A[0..p + q − 1])
Mergesort(R[0..n/2 − 1])
//Merges two sorted arrays into one sorted array
Merge(L, R, A) //see below
//Input: Arrays L[0..p − 1] and R[0..q − 1] both sorted
//Output: Sorted array A[0..p + q − 1] of the elements of L
and R
i ←0; j ←0; k←0
while i <p and j <q do
if L[i]≤ R[j ]
A[k]←L[i]; i ←i + 1
else A[k]←R[j ]; j ←j + 1
k←k + 1
if i = p
copy R[j..q − 1] to A[k..p+ q − 1]
else copy L[i..p − 1] to A[k..p + q − 1]
Important Questions
1. What is Divide and Conquer? Develop the quick sort algorithm and write it’s best
case. Make use of this algorithm to sort the list of characters: E,X,A,M,P,L,E

2. Distinguish between Decrease and Conquer and Divide and Conquer algorithm
design techniques with block diagrams. Develop insertion sort algorithm to sort a
list of integers and estimate the efficiency.

3. Define Topological Sorting. List the two approaches of topological sorting,

illustrate with an example.

4. Explain Strassen’s matrix multiplication approach with an example and derive its
time complexity.

5. Explain the concept of divide and conquer. Design an algorithm for merge sort
and derive its time complexity

6. Design an insertion sort algorithm and obtain its time complexity. Apply insertion
sort on these elements. 25,75,40,10,20.
Important Questions
7. Design an algorithm for quick sort algorithm. Apply quick sort on these elements.
25,75,40,10,20,05,15

8. What is topological sorting? Apply DFS for below graph to solve topological
sorting.

A B

C D E

F G
Important Questions
7. Design an algorithm for quick sort algorithm. Apply quick sort on these elements.
25,75,40,10,20,05,15

8. What is topological sorting? Apply DFS for below graph to solve topological
sorting.

2 5

1 4

3 6
11 Institutions, Infinite Possibilities

We provide 100+ programs across 50 academic

streams.