Design of efficient algorithms

Greedy Method
The greedy method suggests that one can device an algorithm which works in stages considering one
input at a time.At each stage a decision is made regarding whether or not a particular input is an
optimal solution. That means an input is considered and if that input is feasible, will include it in
solution and from all feasible solutions we will get an optimal solution.

For a problem, there are many solutions in which some solution satisfies the condition
(constraint), such solutions are called feasible solution. A solution which is already feasible and give
best result then that solution is called as optimal solution.

Problem which require minimum or maximum result is called optimization problem.

Example: Suppose a problem ‘P’ is to move from “A’ to ‘B’.

Here S4 &S5 are feasible because to cover the distance by 12 hrs requires either train or flight. If we
need to cover the journey in minimum cost , problem demands the result should be minimum. This
problem is minimization problem.

General method:

Algorithm greedy(a,n)
{
for i=1to n do
{
x=select(a)
if feasible(x) then
solution=solution+x
}
}
Example Problem: Change making

A child buys candy values 52 cents . He pays $1 to the cashier. The cashier wishes to return change
48 cents using fewest number of coins. Assume that an unlimited supply of quarters(25 cents],
dimes[10 cents],nickels[5 cents], pennies[ 1 cent] is available.
Solution:
The cashier construct the change in stages. In each stage a coin is selected using greedy
criteria(select largest coin, fewest coins). To assure feasibility ( the change given exactly equals the
desired amount of the solution, the selected coin should not cause the total amount of change given
to exceed the final desired amount.
Stage 1: Choose 1 quarter, then 23 cents remaining.
Stage 2: Choose 1 dime which is feasible. Now 13 cents remaining
Stage 3: Again choose 1 more dime. Now 3 cents remaining.
Stage 4: Select 1 penny, balance 1 cent.
Stage 5: Again select 1 penny
Stage 6: Again select 1 more penny to get the requires change.

On each stage the choice made must be

1. Feasible: It has to satisfy the condition
2. Locally optimal: it has to be the best local choice out of all other choices
3. Irrevocable: once made it cannot be changed.

Problems using greedy approach

1. Fractional Knapsack problem
2. Job sequencing with deadlines
3. Optimal merge pattern
4. Optimal binary search tree
5. Minimum Cost spanning Tree
6. Activity Selection Problem
7. 0-1 knapsack problem
8. Travelling Salesman problem using greedy approach

1. Fractional Knapsack Problem:

We are given n objects and a knapsack(bag) with capacity ‘m’ . object i has weight ‘w’ and profit ‘p’.
A fraction of ‘x’ of object is placed in knapsack without exceeding capacity ‘m’ and earn maximum
profit. This is stated as

Maximum

Subject to
 and

Solutions that satisfy 2&3 is feasible and the feasible solution that satisfy 1is optimal. This is a
maximization problem.
Consider n objects where n=3,and a knapsack with capacity m=20.
Item(i) profit (p) weight (w)

Case 1. Let’s choose item with largest profit. Ie, item 1 with profit 25 and weight 18. Now the
remaining knapsack capacity is 2.Next item with largest profit is item 2with profit 24 and weight 15.
Knapsack cannot hold full of it , so a fraction of it is added (2/15). Now item 3 cannot be added.
Case 2: Lets choose the item with smallest weight.

In the above 2 solutions the maximum earned is solution 2. So the optimal solution is solution 2
since it maximize the profit.

Algorithm:

Void greedy knapsack(m,n)

//m-knapsack capacity, n-no: of objects, w-weight, p-profit, x-fraction of object.

for i=1 to n do

if (w[i]>m) break;

else

w[i]=1

m=m-w[i]

if(i<=n)

x[i]=m/w[i]

2. Job sequencing with deadlines

We are given set of n jobs associated with deadlines. For job i deadline is di>=0, and profit pi
is earned iff job is completed by its deadline. To complete the job one has to process the job on a
machine for a unit of time and only one machine is available for processing.
Example: 5 jobs are given, associated with each job there is profit and deadline to finish the job.
Assume there is a machine. Each job has to be processed and each job takes one unit of time, and
we have 3 unit of time slots.

Jobs J1 J2 J3 J4 J5
Profit 20 15 10 5 1
Deadline 2 2 1 3 3

We want set of jobs that should be completed within deadline such that profit earned should be
maximum. This is a maximization problem.

Condition: job should be completed within deadline.

To do this first sort the job according to their profit in decreasing order. Then find the maximum
deadline value and execute the jobs within deadline.

Maximum Profit = 15+20+ 5=40

Algorithm:

1.Sort all the jobs in decreasing order of profit.

2.Iterate on jobs in decreasing order of profit. For each job, do the following:
a. Fins a time slot i, such that slot is empty and i<deadline and i is greatest. Put the job
in this slot and mark this slot filled.
b. If no such i exists, then ignore the job.
3.Optimal Merge Pattern
Huffmann Coding:
Huffman Coding Algorithm:

1. Create a priority queue Q consisting of each unique character.

2. Sort then in ascending order of their frequencies
3. For all the unique characters:
a. Create a newnode
b. Extract minimum value from Q and assign it to left child of newnode
 c. Extract minimum value from Q and assign it to right child of newnode
d. Calculate the sum of these two minimum values and assign it the value of newnode
e. Insert this newnode into the tree.
4. Return newnode

4.Optimal BST
Another method below :
 5.Minimum Cost Spanning Tree
A spanning tree is a subset of an undirected Graph that has all the vertices
connected by minimum number of edges.
If all the vertices are connected in a graph, then there exists at least one spanning
tree. In a graph, there may exist more than one spanning tree.

Properties
• A spanning tree does not have any cycle.
• Any vertex can be reached from any other vertex.
Minimum Spanning Tree
A Minimum Spanning Tree (MST) is a subset of edges of a connected weighted
undirected graph that connects all the vertices together with the minimum possible
total edge weight. To derive an MST, Prim’s algorithm or Kruskal’s algorithm can be
used.

Example:

Spanning trees are:

Cost= 5+1=6 cost=2+1=3

Here (b) is minimum spanning tree

 Cost=2+5=7

Prim's Algorithm
It is a greedy algorithm. It starts with an empty spanning tree. The idea is to maintain
two sets of vertices:

o Contain vertices already included in MST.

o Contain vertices not yet included.

At every step, it considers all the edges and picks the minimum weight edge. After
picking the edge, it moves the other endpoint of edge to set containing MST.

Algorithm
1. Step 1: Select a starting vertex
2. Step 2: Repeat Steps 3 and 4 until there are fringe vertices
3. Step 3: Select an edge 'e' connecting the tree vertex and fringe vertex that has
minimum weight
4. Step 4: Add the selected edge and the vertex to the minimum spanning tree T
5. [END OF LOOP]
6. Step 5: EXIT

Example of prim's algorithm

Now, let's see the working of prim's algorithm using an example. It will be easier to
understand the prim's algorithm using an example.

Suppose, a weighted graph is -

Step 1 - First, we have to choose a vertex from the above graph. Let's choose B.

Step 2 - Now, we have to choose and add the shortest edge from vertex B. There are
two edges from vertex B that are B to C with weight 10 and edge B to D with weight
4. Among the edges, the edge BD has the minimum weight. So, add it to the MST.

Step 3 - Now, again, choose the edge with the minimum weight among all the other
edges. In this case, the edges DE and CD are such edges. Add them to MST and explore
the adjacent of C, i.e., E and A. So, select the edge DE and add it to the MST.

Step 4 - Now, select the edge CD, and add it to the MST.
Step 5 - Now, choose the edge CA. Here, we cannot select the edge CE as it would
create a cycle to the graph. So, choose the edge CA and add it to the MST.

So, the graph produced in step 5 is the minimum spanning tree of the given graph.
The cost of the MST is given below -

Cost of MST = 4 + 2 + 1 + 3 = 10 units.

Complexity of Prim's algorithm

The time complexity of the prim's algorithm is O(E logV) or O(V logV), where E is the no. of
edges, and V is the no. of vertices.

Kruskals Algorithm:
Kruskal's Algorithm is used to find the minimum spanning tree for a connected weighted
graph. The main target of the algorithm is to find the subset of edges by using which we can
traverse every vertex of the graph.
Example of Kruskal's algorithm
Now, let's see the working of Kruskal's algorithm using an example. It will be easier to
understand Kruskal's algorithm using an example.

Suppose a weighted graph is -

The weight of the edges of the above graph is given in the below table -

Edge AB AC AD AE BC CD DE

Weight 1 7 10 5 3 4 2

Now, sort the edges given above in the ascending order of their weights.

Edge AB DE BC CD AE AC AD

Weight 1 2 3 4 5 7 10

Now, let's start constructing the minimum spanning tree.

Step 1 - First, add the edge AB with weight 1 to the MST.

Step 2 - Add the edge DE with weight 2 to the MST as it is not creating the cycle.

Step 3 - Add the edge BC with weight 3 to the MST, as it is not creating any cycle or
loop.
Step 4 - Now, pick the edge CD with weight 4 to the MST, as it is not forming the
cycle.

Step 5 - After that, pick the edge AE with weight 5. Including this edge will create the
cycle, so discard it.

Step 6 - Pick the edge AC with weight 7. Including this edge will create the cycle, so
discard it.

Step 7 - Pick the edge AD with weight 10. Including this edge will also create the
cycle, so discard it.

So, the final minimum spanning tree obtained from the given weighted graph by using
Kruskal's algorithm is -
The cost of the MST is = AB + DE + BC + CD = 1 + 2 + 3 + 4 = 10.

Algorithm
1. Step 1: Create a forest F in such a way that every vertex of the graph is a separ
ate tree.
2. Step 2: Create a set E that contains all the edges of the graph.
3. Step 3: Repeat Steps 4 and 5 while E is NOT EMPTY and F is not spanning
4. Step 4: Remove an edge from E with minimum weight
5. Step 5: IF the edge obtained in Step 4 connects two different trees, then add it
to the forest F
6. (for combining two trees into one tree).
7. ELSE
8. Discard the edge
9. Step 6: END
Complexity of Kruskal's algorithm
Now, let's see the time complexity of Kruskal's algorithm.

o Time Complexity
The time complexity of Kruskal's algorithm is O(E logE) or O(V logV), where E is the no.
of edges, and V is the no. of vertices.
6.Activity Selection Problem
7.0-1 Knapsack problem
8.Travelling Salesman Problem