0% found this document useful (0 votes)
8 views20 pages

Branch and Bound & Backtracking

Branch and Bound & Backtracking algorithm

Uploaded by

Yash Arya
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
0% found this document useful (0 votes)
8 views20 pages

Branch and Bound & Backtracking

Branch and Bound & Backtracking algorithm

Uploaded by

Yash Arya
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 20

Branch and Bound:

• An algorithm design technique, primarily for solving hard optimization


problems
• Guarantees that the optimal solution will be found
• Does not necessarily guarantee worst case polynomial time complexity
-- But tries to ensure faster time on most instances
• Basic Idea
-- Model the entire solution space as a tree
-- Search for a solution in the tree systematically, eliminating parts of the tree
from the search intelligently
• Important technique for solving many problems for which efficient algorithms
(worst case polynomial time) are not known
Optimization Problems:
• Set of input variables I
• Set of output variables O
-- Values of the output variables define the solution space
• Set of constraints C over the variables
• Set of feasible solutions S
-- Set of solutions that satisfy all the constraints
• Objective function F: S → R
-- Gives a value F(s) for each solution s S
• Optimal solution
-- A solution s S for which F(s) is maximum among all s S (for a
maximization problem) or minimum (for a minimization problem)
Example :
• Many problems:
-- Minimum weight spanning tree, matrix chain multiplication, fractional
knapsack problem, ….
-- All these (above) have known efficient algorithms
Graph Coloring Problem:
-- Input: an undirected graph
-- Output: a color assigned to each vertex
-- Constraint: no two adjacent vertices have the same color
-- Objective function: no of colors used by a feasible solution
-- Goal: Find a coloring that uses the minimum number of colors
-- No efficient algorithm known for general graphs
0-1 Knapsack Problem:
• Given a set of items with weights and
values
paci
ty find the largest subset of items that can be packed into the
knapsack such that the total value gained is maximized.
• Solution format
-- if item i is chosen in the subset, 0 otherwise
• Feasible solution:
• Objective function F:
• Optimal solution: feasible solution with maximum value of
• Solution space size
State Space Tree:
• Represent the solution space as a tree
-- Each edge represents a choice of one
- Level 0 to Level 1 edges show choice of
- Level 1 to Level 2 edges show choice of
- Level to Level edges show choice of
-- Each internal node represents a partial solution
- Partitions the solution space into disjoint subspaces
-- Leaf nodes represent the complete solution (may or may not be
feasible)
-- Models the complete solution being built by choosing one
component at a time
State Space Tree: (Knapsack using Branch and Bound)
State Space Tree: (Example: 0-1 Knapsack)
• Level to Level edges show choice of
• Level to Level edges show choice of
• Level to Level edges show choice of
• Level (root) node partitions the solution space into those that
contain and those that do not contain
• For the subtree which has chosen, Level nodes partitions
the subspace ( present) further into ( and present) and
( present but not present)
• Leaf nodes represent the solutions (the path from root to leaf
shows what items are chosen (edges marked 1 along the path)
State Space Tree: (Example: 0-1 Knapsack)
• Level to Level edges show choice of
• Level to Level edges show choice of
• Level to Level edges show choice of
• Level (root) node partitions the solution space into those that
contain and those that do not contain
• For the subtree which has chosen, Level nodes partitions
the subspace ( present) further into ( and present) and
( present but not present)
• Leaf nodes represent the solutions (the path from root to leaf
shows what items are chosen (edges marked 1 along the path)
Finding the Optimal Solution :
• One possible approach
-- Generate the state space tree, look at all solutions (leaf nodes), find the
feasible solutions, apply objective function on each feasible solution, and
choose the optimal
• But generating the tree is as good as doing brute force search
-- Will take huge space (to store the tree) and huge time (to generate all
nodes)
-- Need to do something better
- To reduce space, do not generate all nodes at once
- To reduce time, do not generate all nodes (how to decide?)
• We will first look at the problem of finding just one feasible solution to
understand a basic technique
Finding One Feasible Solution:
Basic Approach :
--Expand the tree systematically in parts, stop when you find a feasible solution
- Reduces space as whole tree is not generated at one go
- The parts that have been looked at already are not stored
- May reduce time for many instances as feasible solution may be found
without generating the whole tree
- But in worst case, you may still have to generate all the nodes
Finding One Feasible Solution:
How to expand the tree:
• Start with root node
• Generate other nodes in some order
-- DFS, BFS, …..
• Live node: a node which is generated but all children of which has not
been generated
• Dead node: A generated node which is not to be expanded (will see why
later) or whose all children have been generated
• The different node generation orders will pick one live node to expand
(generate its children) at one time
• E-node: The live node being expanded currently
Finding One Feasible Solution:
How to expand the tree:
Example: n-Queens Problem :
• Consider a board
• You have to place queens on the squares so that no two queens
can attack each other, i.e., no two queens should be
-- In the same row
-- In the same column
-- In the same diagonal
One Solution for 8-queens :
• Consider a board
• You have to place queens on the squares so that no two queens
can attack each other, i.e., no two queens should be
-- In the same row
-- In the same column
-- In the same diagonal

Courtesy: Computer Algorithms: E. Horowitz, S. Sahni, and S. Rajasekharan


How to find a solution:
• 4 queen solution space, nodes are numbered as in depth first search

Courtesy: Computer Algorithms: E. Horowitz, S. Sahni, and S. Rajasekharan


Backtracking :
• Systematic search of the state space tree with pruning
-- Pruning done using bounding functions
• Tree generated using DFS order
-- When a child C of the current E-node R is generated, C becomes the new
E-node
-- R will become the E-node again after subtree rooted at C is fully explored
• At every step, apply the bounding function (a predicate) on a node to check
if the subtree rooted at the node needs to be explored
-- Do not generate the subtree rooted at the node if the bounding function
returns false
• Find all feasible solutions (or stop at the first one if only one is needed)
Notes on Backtracking :
• While most definitions of backtracking specify DFS order generation of
state-space tree, other definitions do not restrict the order to DFS
• Backtracking can be applied to problems other than optimization problems
also
-- Just answer yes/no (decision problem)
-- Find one/all feasible solution
-- Some definitions actually restrict backtracking to non-optimization
problems only, with branch and bound for optimization problems
• Forms the basis for state space search for optimization problems also
• Branch and bound uses backtracking with different tree generation order
and augmented bounding function to prune the tree further for finding an
optimal solution
Branch and Bound:
• Use similar methods as backtracking to generate the state space tree
-- But need not be DFS order
• Use the bounding function to prune off parts of the tree
-- Parts that do not contain any feasible solutions as before
-- Current partial solution cannot lead to a feasible solution
-- Parts that may contain feasible solutions but cannot contain an
optimal solution
-- Current partial solution cannot be extended to an optimal
solution
-- Use the value of the solution (objective function applied to the
solution) for deciding this
Tree Generation Orders :
• DFS order (mostly used in backtracking)
• FIFO Branch and Bound/BFS
-- Same as BFS. A E-node node remains an E-node until all its children are
either generated or killed (due to pruning)
-- Implemented the same way as BFS with a queue. Node at head of queue
is the next E-node
• LIFO Branch and Bound/D-Search
-- Similar to BFS in that all nodes of an E-node are generated first
-- Different from BFS in that the generated nodes are placed in a stack
instead of in a queue
Tree of 0-1 Knapsack with LIFO B&B :

Courtesy: Computer Algorithms: E. Horowitz, S. Sahni, and S. Rajasekharan

You might also like