We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 20
Branch and Bound:
• An algorithm design technique, primarily for solving hard optimization
problems • Guarantees that the optimal solution will be found • Does not necessarily guarantee worst case polynomial time complexity -- But tries to ensure faster time on most instances • Basic Idea -- Model the entire solution space as a tree -- Search for a solution in the tree systematically, eliminating parts of the tree from the search intelligently • Important technique for solving many problems for which efficient algorithms (worst case polynomial time) are not known Optimization Problems: • Set of input variables I • Set of output variables O -- Values of the output variables define the solution space • Set of constraints C over the variables • Set of feasible solutions S -- Set of solutions that satisfy all the constraints • Objective function F: S → R -- Gives a value F(s) for each solution s S • Optimal solution -- A solution s S for which F(s) is maximum among all s S (for a maximization problem) or minimum (for a minimization problem) Example : • Many problems: -- Minimum weight spanning tree, matrix chain multiplication, fractional knapsack problem, …. -- All these (above) have known efficient algorithms Graph Coloring Problem: -- Input: an undirected graph -- Output: a color assigned to each vertex -- Constraint: no two adjacent vertices have the same color -- Objective function: no of colors used by a feasible solution -- Goal: Find a coloring that uses the minimum number of colors -- No efficient algorithm known for general graphs 0-1 Knapsack Problem: • Given a set of items with weights and values paci ty find the largest subset of items that can be packed into the knapsack such that the total value gained is maximized. • Solution format -- if item i is chosen in the subset, 0 otherwise • Feasible solution: • Objective function F: • Optimal solution: feasible solution with maximum value of • Solution space size State Space Tree: • Represent the solution space as a tree -- Each edge represents a choice of one - Level 0 to Level 1 edges show choice of - Level 1 to Level 2 edges show choice of - Level to Level edges show choice of -- Each internal node represents a partial solution - Partitions the solution space into disjoint subspaces -- Leaf nodes represent the complete solution (may or may not be feasible) -- Models the complete solution being built by choosing one component at a time State Space Tree: (Knapsack using Branch and Bound) State Space Tree: (Example: 0-1 Knapsack) • Level to Level edges show choice of • Level to Level edges show choice of • Level to Level edges show choice of • Level (root) node partitions the solution space into those that contain and those that do not contain • For the subtree which has chosen, Level nodes partitions the subspace ( present) further into ( and present) and ( present but not present) • Leaf nodes represent the solutions (the path from root to leaf shows what items are chosen (edges marked 1 along the path) State Space Tree: (Example: 0-1 Knapsack) • Level to Level edges show choice of • Level to Level edges show choice of • Level to Level edges show choice of • Level (root) node partitions the solution space into those that contain and those that do not contain • For the subtree which has chosen, Level nodes partitions the subspace ( present) further into ( and present) and ( present but not present) • Leaf nodes represent the solutions (the path from root to leaf shows what items are chosen (edges marked 1 along the path) Finding the Optimal Solution : • One possible approach -- Generate the state space tree, look at all solutions (leaf nodes), find the feasible solutions, apply objective function on each feasible solution, and choose the optimal • But generating the tree is as good as doing brute force search -- Will take huge space (to store the tree) and huge time (to generate all nodes) -- Need to do something better - To reduce space, do not generate all nodes at once - To reduce time, do not generate all nodes (how to decide?) • We will first look at the problem of finding just one feasible solution to understand a basic technique Finding One Feasible Solution: Basic Approach : --Expand the tree systematically in parts, stop when you find a feasible solution - Reduces space as whole tree is not generated at one go - The parts that have been looked at already are not stored - May reduce time for many instances as feasible solution may be found without generating the whole tree - But in worst case, you may still have to generate all the nodes Finding One Feasible Solution: How to expand the tree: • Start with root node • Generate other nodes in some order -- DFS, BFS, ….. • Live node: a node which is generated but all children of which has not been generated • Dead node: A generated node which is not to be expanded (will see why later) or whose all children have been generated • The different node generation orders will pick one live node to expand (generate its children) at one time • E-node: The live node being expanded currently Finding One Feasible Solution: How to expand the tree: Example: n-Queens Problem : • Consider a board • You have to place queens on the squares so that no two queens can attack each other, i.e., no two queens should be -- In the same row -- In the same column -- In the same diagonal One Solution for 8-queens : • Consider a board • You have to place queens on the squares so that no two queens can attack each other, i.e., no two queens should be -- In the same row -- In the same column -- In the same diagonal
Courtesy: Computer Algorithms: E. Horowitz, S. Sahni, and S. Rajasekharan
How to find a solution: • 4 queen solution space, nodes are numbered as in depth first search
Courtesy: Computer Algorithms: E. Horowitz, S. Sahni, and S. Rajasekharan
Backtracking : • Systematic search of the state space tree with pruning -- Pruning done using bounding functions • Tree generated using DFS order -- When a child C of the current E-node R is generated, C becomes the new E-node -- R will become the E-node again after subtree rooted at C is fully explored • At every step, apply the bounding function (a predicate) on a node to check if the subtree rooted at the node needs to be explored -- Do not generate the subtree rooted at the node if the bounding function returns false • Find all feasible solutions (or stop at the first one if only one is needed) Notes on Backtracking : • While most definitions of backtracking specify DFS order generation of state-space tree, other definitions do not restrict the order to DFS • Backtracking can be applied to problems other than optimization problems also -- Just answer yes/no (decision problem) -- Find one/all feasible solution -- Some definitions actually restrict backtracking to non-optimization problems only, with branch and bound for optimization problems • Forms the basis for state space search for optimization problems also • Branch and bound uses backtracking with different tree generation order and augmented bounding function to prune the tree further for finding an optimal solution Branch and Bound: • Use similar methods as backtracking to generate the state space tree -- But need not be DFS order • Use the bounding function to prune off parts of the tree -- Parts that do not contain any feasible solutions as before -- Current partial solution cannot lead to a feasible solution -- Parts that may contain feasible solutions but cannot contain an optimal solution -- Current partial solution cannot be extended to an optimal solution -- Use the value of the solution (objective function applied to the solution) for deciding this Tree Generation Orders : • DFS order (mostly used in backtracking) • FIFO Branch and Bound/BFS -- Same as BFS. A E-node node remains an E-node until all its children are either generated or killed (due to pruning) -- Implemented the same way as BFS with a queue. Node at head of queue is the next E-node • LIFO Branch and Bound/D-Search -- Similar to BFS in that all nodes of an E-node are generated first -- Different from BFS in that the generated nodes are placed in a stack instead of in a queue Tree of 0-1 Knapsack with LIFO B&B :
Courtesy: Computer Algorithms: E. Horowitz, S. Sahni, and S. Rajasekharan