Sanjivani Rural Education Society’s

Sanjivani College of Engineering, Kopargaon-423 603

(An Autonomous Institute, Affiliated to Savitribai Phule Pune University, Pune)
NAAC ‘A’ Grade Accredited, ISO 9001:2015 Certified

Department of Computer Engineering

(NBA Accredited)

Subject- Design and Analysis of Algorithms (DAA) [CO 301)]

Unit 2:- Divide -&- Conquer And Greedy Strategy

Prof. Abhijit S. Bodhe

Assistant Professor
Department of Computer Engineering
E-mail : [email protected]
Contact No: 7709 340 570
 Unit 2:-Divide-&-Conquer and Greedy Strategy
• Divide and Conquer Strategy: Principle, Control Abstraction,
• Time complexity Analysis, Binary search algorithm.
• Case study: Merge Sort.
• Application: Google's Binary Search to Identify Malware.

• Greedy Strategy: Principle, Control Abstraction,

• Time Complexity Analysis, Knapsack Problem,
• Case study: Scheduling Algorithms-Job Scheduling.
• Application: Finding the Shortest Path on Google Ma

DEPARTMENT OF COMPUTER ENGINEERING, Sanjivani COE, Kopargaon 2

 Divide and Conquer strategy
This problem solving technique can be divided into the three parts:
1.Divide: This involves dividing the problem into smaller sub-problems.
2.Conquer: Solve sub-problems by calling recursively until solved.
3.Combine: Combine the sub-problems to get the final solution of the whole problem.
• Advantages of Divide and Conquer Algorithm:
1. The difficult problem can be solved easily.
2. It divides the entire problem into subproblems thus can be solved parallelly ensuring multiprocessing.
3. Efficiently uses cache memory without occupying much space.
4. Reduces time complexity of the problem.
• Disadvantages of Divide and Conquer Algorithm:
1. It involves recursion which is sometimes slow.
2. Efficiency depends on the implementation of logic
3. It may crash the system, If the recursion is performed rigorously/Infinite loping problem
DEPARTMENT OF COMPUTER ENGINEERING, Sanjivani COE, Kopargaon 3
 Control Abstraction (Technically)
• Control abstraction is, the process by which programmers define new
control constructs, specifying a statement ordering separately from an
implementation of that ordering.
• control abstraction is, a procedure whose flow of control is clear but
whose primary operations are specified by other procedures whose
precise meanings are left undefined.
• It is argued that, control abstraction can and should play a central role in
parallel programming.
• For example, a car and its functions are described to the buyer and the
driver also learns how to drive using the steering wheel and the
accelerators but the inside mechanisms of the engine are not displayed to
the buy.
DEPARTMENT OF COMPUTER ENGINEERING, Sanjivani COE, Kopargaon 4
 Control Abstraction of D&C
• The control abstraction for divide and conquer technique is DANDC(P),
where P is the problem to be solved.

• SMALL (P) is a Boolean valued function

• otherwise, the problem ‘p’ into smaller sub problems. These sub
problems p1, p2, . . . , pk are solved by recursive application of DANDC.
DEPARTMENT OF COMPUTER ENGINEERING, Sanjivani COE, Kopargaon 5
 Time complexity of D&C
• If the sizes of the two sub problems are approximately equal then the
computing time of DANDC is:

• Where, T (n) is the time for DANDC on ‘n’ inputs g (n) is the time to
complete the answer directly for small inputs and f (n) is the time for
Divide and Combine
• https://www.geeksforgeeks.org/introduction-to-divide-and-conquer-algorithm-data-structure-and-
algorithm-tutorials/
• https://stackoverflow.com/questions/25858500/solving-recurrence-tn-2tn-2-%CE%981-by-
substitution

DEPARTMENT OF COMPUTER ENGINEERING, Sanjivani COE, Kopargaon 6

 Binary search algorithm
• Compare the middle element of the search space with the key.
• If the key is found at middle element, the process is terminated.
• If the key is not found at middle element, choose which half will be
used as the next search space.
• If the key is smaller than the middle element, then the left side is used for next
search.
• If the key is larger than the middle element, then the right side is used for next
search.
• This process is continued until the key is found or the total search
space is exhausted.
• Consider an array arr[] = {2, 5, 8, 12, 16, 23, 38, 56, 72, 91}, and
the target = 23.
DEPARTMENT OF COMPUTER ENGINEERING, Sanjivani COE, Kopargaon 7
 Merge Sort
• Merge sort is based on the divide-and-conquer paradigm
• The basic concept of merge sort is divides the list into two
smaller sub-lists of approximately equal size.
• Recursively repeat this procedure till only one element is left
in the sub-list.
• After this, various sorted sub-lists are merged to form sorted
parent list.
• This process goes on recursively till the original sorted list
arrived.
• Example & Program execution. . .
DEPARTMENT OF COMPUTER ENGINEERING, Sanjivani COE, Kopargaon 8
 Merge Sort Algorithm
• MERGE-SORT (A, p, r)
1. IF p < r // Check for base case
2. THEN q = FLOOR[(p + r)/2] // Divide step
3. MERGE (A, p, q) // Conquer step.
4. MERGE (A, q + 1, r) // Conquer step.
5. MERGE (A, p, q, r) // Conquer step.
recurrences that we’ll be dealing with Merge sort comes from D& C look like:-
T(n) = aT(n/b) + f(n) => T(n)= 2T(n/2) + C
Where, a is the number of subproblems that the original problem is divided into,
n/b is the size of each subproblem, and f(n) is how long it takes to divide into
subproblems and combine the results of the subproblems.

DEPARTMENT OF COMPUTER ENGINEERING, Sanjivani COE, Kopargaon 9

 Merge Sort Time Complexity
• https://www.digitalocean.com/community/tutorials/merge-sort-
algorithm-java-c-python Implementation on 3 languages.
• Merge Sort is a recursive algorithm and time complexity can be
expressed as following recurrence relation as discussed,
• T(n) = 2T(n/2) + O(n)
• Merge Sort is quite fast, and has a time complexity of O(n*log n).
• It is also a stable sort, which means the "equal" elements are ordered
in the same order in the sorted list.

DEPARTMENT OF COMPUTER ENGINEERING, Sanjivani COE, Kopargaon 10

Merge sort time complexity calculation

DEPARTMENT OF COMPUTER ENGINEERING, Sanjivani COE, Kopargaon 11

 Unit 2:-Greedy Strategy
• Principle of greedy approach
• control abstraction for greedy method.
• time analysis of control abstraction,
• Application 1:-knapsack problem,
• Application 2:- scheduling algorithms-Job scheduling
• Case study: activity selection problems using greedy approach

DEPARTMENT OF COMPUTER ENGINEERING, Sanjivani COE, Kopargaon 12

 Greedy approach (Generalized)
• Greedy is an algorithmic paradigm that builds up a solution piece by
piece, always choosing the next piece that offers the most obvious and
immediate benefit.
• So the problems where choosing locally optimal also leads to global
solution are the best fit for Greedy.
• In Greedy Algorithm a set of resources are recursively divided based
on the maximum, immediate availability of that resource at any given
stage of execution.
• To solve a problem based on the greedy approach, there are two stages
1.Scanning the list of items,
2.Optimization.
DEPARTMENT OF COMPUTER ENGINEERING, Sanjivani COE, Kopargaon 13
 Principle of Greedy approach
• To understand the greedy approach, One need to have a working
knowledge of recursion and context switching.
• Two conditions define the greedy paradigm:-
1. Each stepwise solution must structure a problem towards its
best-accepted solution.
2. It is sufficient if the structuring of the problem can halt in a finite
number of greedy steps.
• Example:- protocols that run the web, such as the open-
shortest-path-first (OSPF) and many other network packet
switching protocols use the greedy strategy to minimize time spent
on a network.
DEPARTMENT OF COMPUTER ENGINEERING, Sanjivani COE, Kopargaon 14
 Characteristics of the Greedy Algorithm

1. The greedy approach has a few tradeoffs, which may make it

suitable for optimization.
2. You will take the maximum quantity of resources in the time a
constraint applies.
3. achieve the most feasible solution immediately.
4. Divide a problem recursively based on a condition, with no need to
combine all the solutions.
5. Uses context switching:- to change from one task (or process) to
another while ensuring that the tasks do not conflict.

DEPARTMENT OF COMPUTER ENGINEERING, Sanjivani COE, Kopargaon 15

 Greedy Vs. DAC

DEPARTMENT OF COMPUTER ENGINEERING, Sanjivani COE, Kopargaon 16

 Applications of Greedy Method
1. CPU Scheduling algorithms. ...
2. Minimum spanning trees. ...
3. Dijkstra shortest path algorithm. ...
4. Fit algorithm in memory management. ...
5. Travelling salesman problem. ...
6. Fractional knapsack problem. ...
7. Egyptian fraction. ...
8. Bin packing problem . .

DEPARTMENT OF COMPUTER ENGINEERING, Sanjivani COE, Kopargaon 17

control abstraction for greedy method

DEPARTMENT OF COMPUTER ENGINEERING, Sanjivani COE, Kopargaon 18

Important activities in greedy algorithm

DEPARTMENT OF COMPUTER ENGINEERING, Sanjivani COE, Kopargaon 19

 Time analysis of control abstraction for greedy
approach
Algorithm Greedy (a, n)
The time complexity is O(n),
// a(1 : n) contains the ‘n’ inputs
{
because with each step of the loop, at
solution := NULL; // initialize the solution to empty for i:=1 to n do
least one locally optimal solution is find
for i= 1 to n do then combined together to give optimal
{ solution.
x := select (a);
if feasible (solution, x) then Some time it can be Reduce to,
solution := Union (Solution, x); O(N*LogN)
} As union function also needs log n time
return solution; to execute for bigger inputs
} IFF n tends to infinity.

DEPARTMENT OF COMPUTER ENGINEERING, Sanjivani COE, Kopargaon 20

Example - Greedy Approach

DEPARTMENT OF COMPUTER ENGINEERING, Sanjivani COE, Kopargaon 21

Solution to Coin problem

DEPARTMENT OF COMPUTER ENGINEERING, Sanjivani COE, Kopargaon 22

 knapsack problem
• Problem Statement:-
• We are given ‘n’ objects and a knapsack.
• The object ‘i’ has a weight wi and the knapsack has a capacity ‘m’.
• If a fraction xi, 0 < Xi < 1 of object i is placed into the knapsack
• then a profit of (Pi*Xi) is earned
• The objective is to fill the knapsack that maximizes the total profit
earned

DEPARTMENT OF COMPUTER ENGINEERING, Sanjivani COE, Kopargaon 23

 knapsack problem-Technically
• Since the knapsack capacity is ‘m’, we require the total weight of all chosen
objects to be at most ‘m’. The problem is stated as:

• The profits and weights are positive numbers.

DEPARTMENT OF COMPUTER ENGINEERING, Sanjivani COE, Kopargaon 24

 Algorithm(Control Abstraction) GreedyKnapsack

Algorithm GreedyKnapsack (m, n)

// P[1 : n] and w[1 : n] contain the profits and weights respectively of
// Objects ordered so that p[i] / w[i] > p[i + 1] / w[i + 1].
// m is the knapsack size and x[1: n] is the solution vector.
{
for i := 1 to n do x[i] := 0.0 // initialize U := m;
for i := 1 to n do
{
if (w(i) > U) then break;
x [i] := 1.0; U := U – w[i];
}
if (i < n) then x[i] := U / w[i];
}

DEPARTMENT OF COMPUTER ENGINEERING, Sanjivani COE, Kopargaon 25

 Running time of knapsack.

• The objects are to be sorted into non-decreasing order of pi / wi ratio.

But if we disregard the time to initially sort the objects, the algorithm
requires only O(n) time.

• If the objects are already been sorted into non-increasing order of p[i] /
w[i] then the algorithm given below obtains solutions corresponding to
this strategy.

DEPARTMENT OF COMPUTER ENGINEERING, Sanjivani COE, Kopargaon 26

 knapsack problem instance

• Consider the following instance of the knapsack problem:

• n = 3, m = 20,
• (p1, p2, p3) = (25, 24, 15) and
• (w1, w2, w3) = (18, 15, 10).

Solve for maximize the profit

DEPARTMENT OF COMPUTER ENGINEERING, Sanjivani COE, Kopargaon 27

 knapsack problem instance

• First, we try to fill the knapsack by selecting the objects in some order:

DEPARTMENT OF COMPUTER ENGINEERING, Sanjivani COE, Kopargaon 28

 knapsack problem instance
• Considering the objects in the order of non-decreasing weights wi.

DEPARTMENT OF COMPUTER ENGINEERING, Sanjivani COE, Kopargaon 29

 knapsack problem instance
• Select the object with the maximum profit first (p = 25). So, x1 = 1 and profit
earned is 25.
• Now, only 2 units of space is left, select the object with next largest profit (p =
24). So, x2 = 2/15
• X3=0

DEPARTMENT OF COMPUTER ENGINEERING, Sanjivani COE, Kopargaon 30

 knapsack problem instance-Optimized
• Considered the objects in the order of the ratio pi / wi .
• p1/w1

DEPARTMENT OF COMPUTER ENGINEERING, Sanjivani COE, Kopargaon 31

 knapsack problem instance-Optimized
• Sort the objects in order of the non-increasing order of the ratio pi / xi. Select the
object with the maximum pi / xi ratio, so, x2 = 1 and profit earned is 24.
• Now, only 5 units of space is left, select the object with next largest pi / xi ratio,
so x3 = ½ and the profit earned is 7.5.

• This solution is the optimal solution.

• Profit = 31.5; When m=20.

DEPARTMENT OF COMPUTER ENGINEERING, Sanjivani COE, Kopargaon 32

 knapsack problem instance 2
• n=7, M=15,
(p1, p2, p3, p4, p5, p6, p7) = (10, 5, 15, 7, 6, 18, 3),
(w1, w2, w3, w4, w5, w6, w7) = (2, 3, 5, 7, 1, 4, 1)

DEPARTMENT OF COMPUTER ENGINEERING, Sanjivani COE, Kopargaon 33

 knapsack problem instance 2
• Case-1: maximum profit

X6=1, x3=1, x1=1, x4=4/7, x5=0, x2=0, x7=0

∑ Pi Xi= p1x1+p2x2+p3x3+ p4x4+p5x5+p6x6+p7x7

=10*1+5*0+15*1+7*4/7+6*0+18*1+3*0
=10+15+4+18
=47

DEPARTMENT OF COMPUTER ENGINEERING, Sanjivani COE, Kopargaon 34

 knapsack problem instance 2

• Case-2: Minimum weight

X7=1, x5=1,x1=1,x2=1, x6=1, x3=4/5,x4=0

∑PiXi=p1x1+p2x2+p3x3+p4x4+p5x5+p6x6+p7x7
=10*1+5*1+15*4/5+7*0+6*1+18*1+3*1
=10+5+12+6+18+3
=54

DEPARTMENT OF COMPUTER ENGINEERING, Sanjivani COE, Kopargaon 35

 knapsack problem instance 2
• Case-3: Maximum profit per unit weight
P1/w1=10/2=5
P2/w2=5/3=1.6
P3/w3=15/5=3
P4/w4=7/7= 1
P5/w5=6/1=6
P6/w6=18/4=4.5
P7/w7=3/1=3
X5=1, x1=1, x6=1, x3=1, x7=1, x2=2/3, x4=0
• ∑PiXi = p1x1+p2x2+p3x3+ p4x4+p5x5+p6x6+p7x7
= 10*1+5*2/3+15*1+7*0+6*1+18*1+3*1
=10+3.3+15+0+6+18+3
=55.3
DEPARTMENT OF COMPUTER ENGINEERING, Sanjivani COE, Kopargaon 36
 knapsack problem instance 2
• he Three feasible solutions .are:

(x1,x2,x3,x4,x5,x6,x7) ∑wixi ∑ pixi

1. ( 1, 0, 1, 4/7, 0, 1, 0) 15 47
2. (1, 1, 4/5, 0, 1, 1, 1) 15 54
3. (1, 2/3, 1, 0, 1, 1, 1) 15 55.33

The optimal solution is maximum profit is 55.33 and the

inputs are (1,2/3,1,0,1,1,1)

DEPARTMENT OF COMPUTER ENGINEERING, Sanjivani COE, Kopargaon 37

 Scheduling algorithms-Job scheduling
• Job scheduling is - Job Sequencing with deadlines
• When we are given a set of ‘n’ jobs. Associated with each Job i, deadline di > 0
and profit Pi > 0.
• For any job ‘i’ the profit pi is earned iff the job is completed by its deadline. Only
one machine is available for processing jobs.
• An optimal solution is the feasible solution with maximum profit.

DEPARTMENT OF COMPUTER ENGINEERING, Sanjivani COE, Kopargaon 38

 Job Sequencing with deadlines Control Abstraction

1. Sort the jobs in ‘j’ ordered by their deadlines.

2. The array d [1 : n] is used to store the deadlines of the order of their
p-values.
3. The set of jobs j [1 : k] such that j [r], 1 ≤ r ≤ k are the jobs in ‘j’ and
d (j [1]) ≤ d (j[2]) ≤ . . . ≤ d (j[k]).
4. To test whether J U {i} is feasible, we have just to insert i into J
preserving the deadline ordering and then
5. verify that d [J[r]] ≤ r, 1 ≤ r ≤ k+1.

DEPARTMENT OF COMPUTER ENGINEERING, Sanjivani COE, Kopargaon 39

 Job Sequencing with deadlines algorithm
• Algorithm:
• The algorithm constructs an optimal set J of jobs that can be processed by their deadlines.
Algorithm GreedyJob (d, J, n)
// J is a set of jobs that can be completed by their deadlines.
{
J := {1};
for i := 2 to n do
{
if (all jobs in J U {i} can be completed by their dead lines) then J := J U {i};
}
}

DEPARTMENT OF COMPUTER ENGINEERING, Sanjivani COE, Kopargaon 40

 Job Sequencing with deadlines Example

• Let n = 4, (P1, P2, P3, P4,) = (100, 10, 15, 27) and (d1 d2 d3 d4) = (2, 1, 2, 1).
• Find the feasible solutions and their values:

DEPARTMENT OF COMPUTER ENGINEERING, Sanjivani COE, Kopargaon 41

DEPARTMENT OF COMPUTER ENGINEERING, Sanjivani COE, Kopargaon 42
JSD more more Example

DEPARTMENT OF COMPUTER ENGINEERING, Sanjivani COE, Kopargaon 43

JSD more Example- Solution

DEPARTMENT OF COMPUTER ENGINEERING, Sanjivani COE, Kopargaon 44

 JSD more Example- Solution

• Let n=7, Profits(p1, p2. ….p7)={3, 5, 20, 18, 1, 6, 30}

Deadlines(d1, d2,….d7)={1, 3, 4, 3, 2, 1, 2}

DEPARTMENT OF COMPUTER ENGINEERING, Sanjivani COE, Kopargaon 45

JSD more Example- Solution

DEPARTMENT OF COMPUTER ENGINEERING, Sanjivani COE, Kopargaon 46

 Activity selection problems using greedy approach

• Problem Statement:- You are given n activities with their start

and finish times. Select the maximum number of activities that
can be performed by a single person, assuming that a person
can only work on a single activity at a time.

• https://www.geeksforgeeks.org/activity-selection-problem-greedy-
algo-1/

DEPARTMENT OF COMPUTER ENGINEERING, Sanjivani COE, Kopargaon 47

Activity selection problems using greedy approach

DEPARTMENT OF COMPUTER ENGINEERING, Sanjivani COE, Kopargaon 48

 Unit 3:- Dynamic Programming
• Principle,
• Control abstraction,
• time analysis of control abstraction,
• Application 1:- binomial coefficients,
• Application 2:- OBST,
• Application 3:- 0/1 knapsack,
• Case Study :- Chain Matrix Matrix Multiplication.

DEPARTMENT OF COMPUTER ENGINEERING, Sanjivani COE, Kopargaon 49

• Done. . .!!

DEPARTMENT OF COMPUTER ENGINEERING, Sanjivani COE, Kopargaon 50