MODULE V

Greedy Algorithm
MODULE 5
• A greedy algorithm is an approach for solving a problem by
selecting the best option available at the moment. It doesn't
worry whether the current best result will bring the overall
optimal result.

• The algorithm never reverses the earlier decision even if the

choice is wrong. It works in a top-down approach.

• This algorithm may not produce the best result for all the
problems. It's because it always goes for the local best
choice to produce the global best result.
• The term "greedy" in the context of algorithms refers to
the strategy of making the locally optimal choice at
each stage with the hope of finding a global optimum.
In other words, a greedy algorithm makes the best
possible decision at each step without considering the
consequences of that decision on future steps.
General structure of Greedy Algorithm

1. Identify the problem as an optimization problem where we need to find the best
solution among a set of possible solutions.

2. Determine the set of feasible solutions for the problem.

3. Identify the optimal substructure of the problem, meaning that the optimal solution to
the problem can be constructed from the optimal solutions of its subproblems.

4. Develop a greedy strategy to construct a feasible solution step by step, making the
locally optimal choice at each step.
Prove the correctness of the algorithm by showing that the locally optimal choices at
each step lead to a globally optimal solution.
Fractional Knapsack Problem

• The Fractional Knapsack Problem is a classic optimization problem in computer

science and mathematics. It is one of the most popular problems that take greedy
approach to be solved.

The problem can be stated as follows:

• Problem Statement: Given a set of items, each with a weight and a Profit,
determine the maximum profit that can be obtained by selecting fractions of
items, such that the total weight does not exceed a given limit.
Input:

• n: the number of items.

• wi​: the weight of the ith item.

• Pi​: the value of the ith item.

• W: the maximum weight capacity of the knapsack.

Output: Maximize the total value of items in the knapsack without exceeding the
weight capacity W
Algorithm
• Consider all the items with their weights and profits mentioned respectively.

• Calculate Pi/Wi of all the items and sort the items in descending order based on their

Pi/Wi values.

• Without exceeding the limit, add the items into the knapsack.

• If the knapsack can still store some weight, but the weights of other items exceed the
limit, the fractional part of the next time can be added.

• Hence, giving it the name fractional knapsack problem .

Example

• For the given set of items and the knapsack capacity of 10 kg, find the
subset of the items to be added in the knapsack such that the profit is
maximum.
• Step 1

• Step 2
• Step 3

• Without exceeding the knapsack capacity, insert the items in the knapsack with maximum profit.

• If 5 kg weight of 4th item is added, it will exceed the capacity. Therefore, only 4 kg weight of the
5 kg will be added in the knapsack.

• Hence, the knapsack holds the weights = [(1 * 1) + (1 * 3) + (1 * 2) + (4/5 * 5)] = 10, with
maximum profit of [(1 * 8) + (1 * 15) + (1 * 10) + (4/5 * 20)] = 37.
Advantages & Disadvantages
Characteristics of a Greedy Algorithm

• The algorithm solves its problem by finding an optimal solution. This solution can be a
maximum or minimum value. It makes choices based on the best option available.

• The algorithm is fast and efficient with time complexity of O(n log n) or O(n).
Therefore applied in solving large-scale problems.

• The search for optimal solution is done without repetition – the algorithm runs once.

• It is straightforward and easy to implement.

• The greedy algorithm doesn't always produce the optimal solution. This is the major
disadvantage of the algorithm
• Applications of Greedy Algorithm

• It is used in finding the shortest path.

• It is used to find the minimum spanning tree using the

prim's algorithm or the Kruskal's algorithm.

• It is used in a job sequencing with a deadline.

• This algorithm is also used to solve the fractional

knapsack problem
Divide and Conquer
Algorithm
Divide and Conquer Algorithm
• A divide and conquer algorithm is a strategy of
solving a large problem by
1.breaking the problem into smaller sub-problems
2.solving the sub-problems, and
3.combining them to get the desired output.
• To use the divide and conquer algorithm, recursion is
used.
Divide and Conquer Applications
• Binary Search
• Merge Sort
• Quick Sort
• Strassen's Matrix multiplication
• Karatsuba Algorithm
Advantages of Divide and Conquer Algorithm

• It efficiently uses cache memory without occupying much space because it solves simple subproblems within

the cache memory instead of accessing the slower main memory.

• Divide and conquer simplifies complex problems by breaking them into smaller, more manageable

subproblems. Each subproblem is easier to solve than the original problem.

• Because the subproblems are independent of each other, divide and conquer algorithms are often suitable for

parallelization. Different processors or threads can work on different subproblems simultaneously, leading to

potential performance improvements.

Disadvantages of Divide and Conquer

• Recursion is incorporated into the majority of its algorithms; hence it requires intensive memory management.

• The space could be overused by an explicit stack.

• If the recursion is carried through rigorously beyond the CPU stack, the system can even crash.
Algorithm for Divide and Conquer
DAC (P)
{
If(small(P)
{
S(P)
}
else
{
Divide P into P1, P2, P3…..Pk
Apply DAC(P1), DAC(P2),………..
Combine(DAC(P1), DAC(P2)
}
}
Dynamic Programming

The following are the steps that the dynamic programming follows:

• It breaks down the complex problem into simpler subproblems.

• It finds the optimal solution to these sub-problems.

• It stores the results of subproblems (memoization). The process of storing the

results of subproblems is known as memoization.

• It reuses them so that same sub-problem is calculated more than once.

• Finally, calculate the result of the complex problem.

• The Fibonacci sequence is a classic example to illustrate dynamic
programming because it has overlapping subproblems that can be
solved more efficiently by storing intermediate results.
• We want to compute f(2) as the second call from f(4), but in the
recursive tree we had already computed f(2) once (in the first
recursive call of f(3) ) .
1. Initialize the list
Start by initializing a list with the first two Fibonacci
numbers, 0 and 1.
2. Iterate
Iterate from 2 to n, calculating each Fibonacci number by
adding the two preceding numbers in the list.
3. Store the values
Store each calculated value in the program's memory so
it can be reused later. This is called memoization.
• Reduces computation time from exponential to linear, because we only
calculate each Fibonacci number once.

• Saves us from repeatedly solving the same subproblems, making the solution
more efficient.

• This process exemplifies the core idea of dynamic programming: breaking

down a complex problem into simpler subproblems, solving each
subproblem only once, and storing the results for future use.
Example of 0/1 knapsack problem

Consider the problem having weights and profits are:

• Weights: {3, 4, 6, 5}

• Profits: {2, 3, 4,1}

• The weight of the knapsack is 8 kg

• Create a matrix shown as below:

• In the above matrix, columns represent the weight, i.e., 8.

• The rows represent the profits and weights of items.
• Here we have not taken the weight 8 directly, problem is divided into
sub-problems, i.e., 0, 1, 2, 3, 4, 5, 6, 7, 8.
• The solution of the sub-problems would be saved in the cells and answer
to the problem would be stored in the final cell.
• First, we write the weights in the ascending order and profits according
to their weights shown as below:
• wi = {3, 4, 5, 6}
• = 3; Since we have only one item in the set having weight
equal to 3, and weight of the knapsack is 8; therefore, we
can fill the knapsack with an item of weight equal to 3. We
put profit corresponding to the weight 3, i.e., 2 at M[1][8]
shown as below:
• When i =2, W = 4

• The weight corresponding to the value 2 is 4, i.e., w2 = 4. Since we have

two items in the set having weights 3 and 4, and the weight of the
knapsack is 4. We can put item of weight 4 in a knapsack as the profit
corresponding to weight 4 is more than the item having weight 3, so we
add 3 at M[2][4] shown as below:
• When i = 2, W = 7

• The weight corresponding to the value 2 is 4, i.e., w 2 = 4. Since we

have two items in the set having weights 3 and 4, and the weight of
the knapsack is 7. We can put item of weight 4 and 3 in a knapsack
and the profits corresponding to weights are 2 and 3; therefore, the
total profit is 5, so we add 5 at M[2][7] shown as below:
• When i = 4, W = 7

• The weight corresponding to the value 4 is 6, i.e., w4 = 6. Since we

have four items in the set of weights 3, 4, 5, and 6 respectively, and the
weight of the knapsack is 7. Here, if we add two items of weights 3 and
4 then it will produce the maximum profit, i.e., (2 + 3) equals to 5, so
we add 5 at M[4][7] shown as below:
• As we can observe in the above table that 5 is the maximum
profit among all the entries. The pointer points to the last row
and the last column having 5 value. Now we will compare 5
value with the previous row; if the previous row, i.e., i = 3
contains the same value 5 then the pointer will shift upwards.
Since the previous row contains the value 5 so the pointer
will be shifted upwards as shown in the below table:

• Again, we will compare the value 5 from the above row, i.e., i
= 2. Since the above row contains the value 5 so the pointer
will again be shifted upwards
• Again, we will compare the value 5 from the above row,
i.e., i = 1. Since the above row does not contain the
same value so we will consider the row i=2, and the
weight corresponding to the row is 4. Therefore, we
have selected the weight 4 and we have rejected the
weights 5 and 6 shown below:
• x = { 1, 0, 0}
• The profit corresponding to the weight is 3. Therefore,
the remaining profit is (5 - 3) equals to 2. Now we will
compare this value 2 with the row i = 2. Since the row (i
= 1) contains the value 2; therefore, the pointer shifted
upwards shown below:
• Again we compare the value 2 with a above row, i.e., i = 1.
Since the row i =0 does not contain the value 2, so row i =
1 will be selected and the weight corresponding to the i = 1
is 3 shown below:

• X = {1, 1, 0, 0}

• The profit corresponding to the weight is 2. Therefore, the

remaining profit is 0.
Back Tracking
• Many problems are difficult to solve algorithmically. Backtracking makes it possible to
solve at least some large instances of difficult combinatorial problems.

• Suppose we have to make a series of decisions among various choices,

where

• We don’t have enough information to know what to choose

• Each decision leads to a new set of choices.

• Some sequence of choices (more than one choices) may be a solution to your problem.
• A backtracking algorithm works by recursively exploring all possible solutions to
a problem.

• It starts by choosing an initial solution, and then it explores all possible extensions
of that solution.

• If an extension leads to a solution, the algorithm returns that solution.

• If an extension does not lead to a solution, the algorithm backtracks to the previous
solution and tries a different extension.

• It is commonly used in situations where you need to explore multiple possibilities

to solve a problem, like searching for a path in a maze or solving puzzles
like Sudoku.
• The following is a general outline of how a backtracking algorithm works:
1. Choose an initial solution.

2. Explore all possible extensions of the current solution.

3. If an extension leads to a solution, return that solution.

4. If an extension does not lead to a solution, backtrack to the previous solution and
try a different extension.

5. Repeat steps 2-4 until all possible solutions have been explored.
N-Queens Problem
• N - Queens problem is to place n - queens in such a manner on an n
x n chessboard that no queens attack each other by being in the
same row, column or diagonal.
• Consider the 4 queens problem
• Given a 4 x 4 chessboard and number the rows and column of the
chessboard 1 through 4.
Illustration of 4 Queens
Solution:
• Step 0: Initialize a 4×4 board.
• Step 1:
• Put our first Queen (Q1) in the (0,0) cell .
• ‘x‘ represents the cells which is not safe i.e. they are
under attack by the Queen (Q1).
• After this move to the next row [ 0 -> 1 ].
• Step 2:
• Put our next Queen (Q2) in the (1,2) cell .
• After this move to the next row [ 1 -> 2 ].
• Step 4:
• There is still a safe cell in the row 1 i.e. cell ( 1, 3 ).
• Put Queen ( Q2 ) at cell ( 1, 3).
• Step 5:
• Put queen ( Q3 ) at cell ( 2, 1 ).
• Step 6:
• There is no any cell to place Queen ( Q4 ) at row 3.
• Backtrack and remove Queen ( Q3 ) from row 2.
• Again there is no other safe cell in row 2, So backtrack
again and remove queen ( Q2 ) from row 1.
• Queen ( Q1 ) will be remove from cell (0,0) and move
to next safe cell i.e. (0 , 1).
• Step 7:
• Place Queen Q1 at cell (0 , 1), and move to next row.
• Step 8:
• Place Queen Q2 at cell (1 , 3), and move to next row.
• Step 9:
• Place Queen Q3 at cell (2 , 0), and move to next row.
• Step 10:
• Place Queen Q4 at cell (3 , 2), and move to next row.
• This is one possible configuration of solution
Follow the steps mentioned below to implement the idea:
•Start in the leftmost column
•If all queens are placed return true
•Try all rows in the current column. Do the following for every row.
•If the queen can be placed safely in this row
•Then mark this [row, column] as part of the solution and
recursively check if placing queen here leads to a solution.
•If placing the queen in [row, column] leads to a solution then
return true.
•If placing queen doesn’t lead to a solution then unmark this [row,
column] then backtrack and try other rows.
•If all rows have been tried and valid solution is not found
return false to trigger backtracking.