Backtracking Algorithm

A backtracking algorithm is a problem-solving algorithm that uses a brute force approach for finding
the desired output.

The Brute force approach tries out all the possible solutions and chooses the desired/best solutions.

The term backtracking suggests that if the current solution is not suitable, then backtrack and try
other solutions. Thus, recursion is used in this approach.

This approach is used to solve problems that have multiple solutions. If you want an optimal solution,
you must go for dynamic programming.

State Space Tree

A space state tree is a tree representing all the possible states (solution or nonsolution) of the
problem from the root as an initial state to the leaf as a terminal state.

State Space Tree

Backtracking Algorithm

Backtrack(x)

if x is not a solution

return false

if x is a new solution

add to list of solutions

backtrack(expand x)

Example Backtracking Approach

Problem: You want to find all the possible ways of arranging 2 boys and 1 girl on 3 benches.
Constraint: Girl should not be on the middle bench.

Solution: There are a total of 3! = 6 possibilities. We will try all the possibilities and get the possible
solutions. We recursively try all the possibilities.

All the possibilities are:

 All the possibilities

The following state space tree shows the possible solutions.

State tree with all the solutions

The backtracking algorithm is a problem-solving approach that tries out all the possible solutions
and chooses the best or desired ones. Generally, it is used to solve problems that have multiple
solutions. The term backtracking suggests that for a given problem, if the current solution is not
suitable, eliminate it and then backtrack to try other solutions.

When do we use backtracking algorithm?

Backtracking algorithm can be used for the following problems −

 The problem has multiple solutions or requires finding all possible solutions.

 When the given problem can be broken down into smaller subproblems that are similar to
the original problem.

 If the problem has some constraints or rules that must be satisfied by the solution

How does backtracking algorithm work?

The backtracking algorithm explores various paths to find a sequence path that takes us to the
solution. Along these paths, it establishes some small checkpoints from where the problem can
backtrack if no feasible solution is found. This process continues until the best solution is found.
In the above figure, green is the start point, blue is the intermediate point, red are points with no
feasible solution, grey is the end solution.

When backtracking algorithm reaches the end of the solution, it checks whether this path is a
solution or not. If it is the solution path, it returns that otherwise, it backtracks to one step behind in
order to find a solution.

Algorithm

Following is the algorithm for backtracking −

1. Start

2. if current_position is goal, return success.

3. else

4. if current_position is an end point, return failed.

5. else-if current_position is not end point, explore and repeat above steps.

6. Stop

Complexity of Backtracking

Generally, the time complexity of backtracking algorithm is exponential (0(kn)). In some cases, it is
observed that its time complexity is factorial (0(N!)).

Types of Backtracking Problem

The backtracking algorithm is applied to some specific types of problems. They are as follows −

 Decision problem − It is used to find a feasible solution of the problem.

 Optimization problem − It is used to find the best solution that can be applied.

 Enumeration problem− It is used to find the set of all feasible solutions of the problem.

Here’s a purely theoretical explanation of the Backtracking algorithm, ideal for notes or exams:

🔁 Backtracking Algorithm – Theory

Backtracking is a general algorithmic technique used to solve combinatorial problems, constraint

satisfaction problems, and optimization problems.

It incrementally builds candidates for the solution and abandons ("backtracks") a candidate as soon
as it determines that the candidate cannot lead to a valid solution.

🔷 Definition:

Backtracking is a depth-first search method for solving problems recursively by trying partial
solutions and then abandoning them if they are not feasible (i.e., if constraints are violated).
🔷 Working Principle:

1. Choose: Make a choice from available options.

2. Check: Verify if the choice violates any constraints.

3. Explore: Proceed to the next level (next decision).

4. Backtrack: If the solution is invalid or complete, undo the last choice and try another.

🔷 General Structure:

 Start from an empty solution.

 Explore all possible options recursively.

 If a partial solution is invalid, prune the search tree (i.e., skip further exploration).

 Continue until all valid solutions are found or the best one is obtained.

🔷 Key Concepts:

 Solution space tree: A tree that represents all possible solutions. Each level denotes a
decision.

 Constraint checking: Ensures the current solution remains valid.

 Backtracking point: The moment where we undo a decision and try a new path.

🔷 Applications:

Problem Explanation

N-Queens Problem Place N queens on a chessboard such that no two attack each other.

Fill a grid so that rows, columns, and boxes contain unique

Sudoku Solver
numbers.

Assign minimum colors to vertices so no adjacent nodes share a

Graph Coloring
color.

Hamiltonian Path/Cycle Find a path/cycle that visits each vertex exactly once.

Subset Sum / Combinatorial

Find subsets that meet a specific condition (like sum = K).
Sum

Maze Solving Find a path from start to finish in a grid.

Word Search / Crossword Filling Find if a word can be placed in a grid following rules.

🔷 Advantages:
  Simple to implement.

 Works well with pruning techniques (like constraint propagation).

 Can be adapted to a wide range of problems.

🔷 Limitations:

 Time-consuming for large input (combinatorial explosion).

 Inefficient without pruning or optimization techniques.

 Not suitable for real-time or very large data problems unless optimized.

⏱ Time Complexity (Theoretical):

 Worst-case time complexity: Often exponential, depends on the number of choices per level
and depth of recursion.

O(kn)where k=choices per step, n=stepsO(k^n) \quad \text{where } k = \text{choices per step},\ n = \
text{steps}

The 8 Queen Problem is a puzzle in which 8 queens must be placed on an 8x8 chessboard so that no
two queens threaten each other. It is a classic problem in computer science and mathematics. There
are 92 solutions to this problem. The eight queens puzzle problem was first posed in the mid-19th
century.

Backtracking is a recursive approach for solving any problem where we must search among all the
possible solutions following some constraints. More precisely, we can say that it is an improvement
over the brute force technique. In this blog, we will learn one popular DSA problem: 8 queens
problem using Backtracking.

Problem Statement

Given an 8x8 chess board, you must place 8 queens on the board so that no two queens attack each
other. Print all possible matrices satisfying the conditions with positions with queens marked with '1'
and empty spaces with '0'. You must solve the 8 queens problem using backtracking.

 Note 1: A queen can move vertically, horizontally and diagonally in any number of steps.

 Note 2: You can also go through the N-Queen Problem for the general approach to solving
this problem.

Sample Example

Example: One possible solution to the 8 queens problem using backtracking is shown below. In the
first row, the queen is at E8 square, so we have to make sure no queen is in column E and row 8 and
also along its diagonals. Similarly, for the second row, the queen is on the B7 square, thus, we have to
secure its horizontal, vertical, and diagonal squares. The same pattern is followed for the rest of the
queens.
Output:

00001000

01000000

00010000

00000010

00100000

00000001

00000100

10000000

Bruteforce Approach

One brute-force approach to solving this problem is as follows:

 Generate all possible permutations of the numbers 1 to 8, representing the columns on the
chessboard.

 For each permutation, check if it represents a valid solution by checking that no two queens
are in the same row or diagonal.

 If a valid solution is found, print the board layout.

While this approach works for small numbers, it quickly becomes inefficient for larger sizes as the
number of permutations to check grows exponentially. More efficient algorithms, such as
backtracking or genetic algorithms, can be used to solve the problem in a more optimized way.

Backtracking Approach

This approach rejects all further moves if the solution is declined at any step, goes back to the
previous step and explores other options.

Algorithm

Let's go through the steps below to understand how this algorithm of solving the 8 queens problem
using backtracking works:

 Step 1: Traverse all the rows in one column at a time and try to place the queen in that
position.

 Step 2: After coming to a new square in the left column, traverse to its left horizontal
direction to see if any queen is already placed in that row or not. If a queen is found, then
move to other rows to search for a possible position for the queen.
  Step 3: Like step 2, check the upper and lower left diagonals. We do not check the right side
because it's impossible to find a queen on that side of the board yet.

 Step 4: If the process succeeds, i.e. a queen is not found, mark the position as '1' and move
ahead.

 Step 5: Recursively use the above-listed steps to reach the last column. Print the solution
matrix if a queen is successfully placed in the last column.

 Step 6: Backtrack to find other solutions after printing one possible solution.

Step 1: Start with one column

We place one queen column by column, starting from the leftmost column (column 0).

We go row by row in that column and try to place a queen in a safe position.

Step 2: Check the Row

Before placing the queen in a row of the current column, we look to the left in the same row (i.e., in
previous columns) to see if any queen is already there.

If there is already a queen in that row, we can’t place another one there — they’ll attack each other.

Step 3: Check the Diagonals

Then we also check:

 The upper-left diagonal (↖)

 The lower-left diagonal (↙)

Why only on the left side?

 Because we are placing queens from left to right.

 So any queen that could attack the new one must already be on the left.

If there's a queen on any of these diagonals, we skip this row and try the next one.

Step 4: Place the Queen

If no queen is found in the same row or diagonals, we found a safe spot!

So we place the queen there and mark it in the board matrix (like with a 1 or Q).

Step 5: Move to the Next Column

Now, we go to the next column on the right and repeat the same steps (1 to 4) to place the next
queen.

We keep doing this recursively (calling the same steps again and again) until all 8 queens are placed.
Step 6: Backtrack if Stuck

If we reach a point where:

 No safe spot is found in the current column,

 We go back to the previous column (i.e., backtrack),

 Remove the previously placed queen from her position,

 And try the next row for her.

This helps us explore all possible combinations and not miss any solution.

Here is the theory-only explanation of the 8-Queens Problem using Backtracking, suitable for exams
and notes:

♛ 8-Queens Problem (Theory)

🟣 Problem Statement:

Place 8 queens on an 8×8 chessboard such that no two queens threaten each other. This means:

 No two queens should be in the same row,

 No two queens should be in the same column,

 No two queens should be on the same diagonal.

🔁 Approach: Backtracking

The 8-Queens problem is a classic example of backtracking, where we build the solution step by step
and backtrack when a conflict arises.

🟢 Steps Involved:

1. Start from the first row (row = 0).

2. Try placing a queen in each column of the current row.

3. For every placement:

o Check whether it's safe (i.e., doesn't conflict with queens in previous rows).

4. If it is safe:

o Place the queen and move to the next row recursively.

5. If no valid column is found in a row:

o Backtrack to the previous row and try the next column.

 6. Repeat this process until:

o A solution is found (all 8 queens placed), or

o All possibilities are exhausted.

🔶 Safety Check:

For placing a queen at position (row, col):

 No other queen should be in the same column.

 No other queen should be on the major diagonal (row - col) same.

 No other queen should be on the minor diagonal (row + col) same.

🧠 Solution Space:

 There are 888^8 total ways to place queens (8 choices per row).

 But we restrict it to only one queen per row, and prune invalid placements using safety
checks.

📌 Key Properties:

 Type of problem: Constraint Satisfaction Problem (CSP)

 Technique used: Recursive Backtracking

 Solution count: There are 92 distinct solutions for the 8-Queens problem (not counting
symmetric solutions).

⏱ Time Complexity (Worst-case):

 Exponential: O(N!)O(N!) for N-Queens (due to N! permutations)

o For 8-Queens, up to 40320 combinations in the worst case.

🔍 Applications of 8-Queens:

 Demonstrates constraint satisfaction techniques.

 Basis for solving N-Queens problem in AI and competitive coding.

 Teaches recursion, pruning, and backtracking logic.

What is a Hamiltonian Cycle?

A Hamiltonian Cycle or Circuit is a path in a graph that visits every vertex exactly once and returns to
the starting vertex, forming a closed loop. A graph is said to be a Hamiltonian graph only when it
contains a hamiltonian cycle, otherwise, it is called non-Hamiltonian graph.

A graph is an abstract data type (ADT) consisting of a set of objects that are connected via links.

The practical applications of hamiltonian cycle problem can be seen in the fields like network design,
delivery systems and many more. However, the solution to this problem can be found only for small
types of graphs, and not for larger ones.

Input Output Scenario

Suppose the given undirected graph G(V, E) and its adjacency matrix are as follows −

The backtracking algorithm can be used to find a Hamiltonian path in the above graph. If found, the
algorithm returns the path. If not, it returns false. For this case, the output should be (0, 1, 2, 4, 3, 0).

Finding Hamiltonian Cycle using Backtracking Approach

The naive way to solve Hamiltonian cycle problem is by generating all possible configurations of
vertices and checking if any configuration satisfies the given constraints. However, this approach is
not suitable for large graphs as its time complexity will be (O(N!)).

The following steps explain the working of backtracking approach −

 First, create an empty path array and add a starting vertex 0 to it.

 Next, start with vertex 1 and then add other vertices one by one.

 While adding vertices, check whether a given vertex is adjacent to the previously added
vertex and hasn't been added already.

 If any such vertex is found, add it to the path as part of the solution, otherwise, return false.

What is a Hamiltonian Cycle?

A Hamiltonian cycle is a path in a graph that visits every vertex exactly once and returns to the
starting vertex, forming a closed loop. For a Hamiltonian cycle to exist, the graph must be connected,
meaning there is a path between every pair of vertices.

Here are the key properties of a Hamiltonian cycle:

1. Visits every vertex in the graph exactly once

 2. Returns to the starting vertex to form a closed loop

3. The graph must be connected for a Hamiltonian cycle to exist

For example, consider a graph with 4 vertices labeled A, B, C, & D. One possible Hamiltonian cycle in
this graph could be: A -> B -> C -> D -> A. This path visits each vertex once and loops back to the start.

Not all graphs have Hamiltonian cycles. Some graphs may have a Hamiltonian path but no cycle. And
some graphs have neither paths nor cycles that visit each vertex exactly once.

Determining if a Hamiltonian cycle exists in a given graph is an NP-complete problem, meaning it is

computationally challenging for large graphs. But for small graphs, we can use algorithms like
backtracking to find Hamiltonian cycles, which we will see later in this article.

What is a Hamiltonian Path?

A Hamiltonian path is similar to a Hamiltonian cycle, but instead of forming a closed loop, it is a path
that visits each vertex in the graph exactly once without returning to the starting vertex.

The key properties of a Hamiltonian path are:

1. Visits every vertex in the graph exactly once

2. Does not return to the starting vertex

3. The graph must be connected for a Hamiltonian path to exist

Let's consider the same example graph from before with 4 vertices A, B, C, & D. An example of a
Hamiltonian path (but not a cycle) in this graph would be: A -> B -> C -> D. This path visits each vertex
once but does not loop back to A.

Every Hamiltonian cycle is also a Hamiltonian path, but not every Hamiltonian path is a cycle. If a
graph has a Hamiltonian cycle, it must also have a Hamiltonian path. However, a graph with a
Hamiltonian path may or may not have a cycle.

Finding Hamiltonian paths is also an NP-complete problem, just like finding Hamiltonian cycles. But
for small graphs, we can use algorithms like backtracking to find paths & cycles efficiently.

Hamiltonian Cycle using Backtracking Algorithm

Now that we understand what Hamiltonian cycles are, let's see how we can find them in a graph
using a backtracking algorithm.

Backtracking is a general algorithmic technique that explores all possible solutions by incrementally
building candidates to the solution and abandoning a candidate ("backtracking") as soon as it
determines that the candidate cannot lead to a valid solution.

Here's how we can apply backtracking to find Hamiltonian cycles:

1. Start with an empty path and choose any vertex as the starting point.
2. Add the starting vertex to the path.

3. Recursively build the path by choosing the next unvisited vertex and adding it to the path.

4. If the path contains all vertices and the last vertex has an edge to the starting vertex, we have
found a Hamiltonian cycle.

5. If the path does not meet the criteria for a Hamiltonian cycle, backtrack by removing the last
vertex from the path and trying a different unvisited vertex.

6. Continue this process until a Hamiltonian cycle is found or all possibilities have been exhausted.

The backtracking algorithm explores all possible paths in the graph and finds a Hamiltonian cycle if
one exists. If no Hamiltonian cycle is found after exploring all paths, the graph does not have one.

Introduction

In graph coloring problems, we are asked to color the parts of the graph.

Vertex coloring is one of the most common graph coloring problems. In this problem, we are given a
graph and ‘m’ colors. We need to find a number of ways to color a graph with these m colors such
that no two adjacent nodes are colored the same. Some of the other graph coloring problems,
like Edge Coloring (No vertex is incident to two edges of the same color) and Face
Coloring (Geographical Map Coloring), can be transformed into vertex coloring.

Another famous graph coloring problem is Chromatic Number. In this problem, we need to find the
minimum number of colors are required to color the graph such that no adjacent nodes are colored
the same.

In the above graph, we can see that we need 3 colors to color the entire graph such that no two
nodes are colored the same.

Chromatic Number

The chromatic number of a graph is the smallest number of colors needed to color the vertices of
the graph such that no two adjacent vertices share the same color. It is a crucial concept in graph
theory, particularly in problems involving scheduling, register allocation, and frequency assignment
in networks. The chromatic number provides insights into the complexity of a graph and can help
determine optimal coloring strategies for various applications.

Algorithm of Graph Coloring using Backtracking

Graph coloring is an algorithmic technique for assigning colors to the vertices of a graph. The
backtracking approach systematically explores all possibilities to find a valid coloring solution. The
algorithm works by assigning colors to vertices one at a time and backtracking if a conflict arises,
ensuring that adjacent vertices do not share the same color.

Steps of the Backtracking Algorithm

1. Select a Vertex: Choose a vertex that has not yet been colored.

2. Assign Colors: Try assigning different colors to the selected vertex.

3. Check for Validity: For each color, check if it conflicts with the colors assigned to adjacent
vertices.

4. Backtrack: If a conflict arises, revert to the previous vertex and try the next color.

5. Repeat: Continue the process until all vertices are colored or all options are exhausted.

Here's a pure theory explanation of the Graph Colouring Problem using Backtracking—suitable for
exams, class notes, and conceptual clarity:

🎨 Graph Colouring Problem (Theory)

🔷 Problem Statement:

The Graph Colouring Problem involves assigning colours to the vertices of a graph such that no two
adjacent vertices have the same colour, using at most M colours.

🟡 Formal Definition:

Given:

 A graph G=(V,E)G = (V, E) with V vertices and E edges

 An integer M, the number of available colours

Objective:
Assign a colour to each vertex such that:

 No two connected (adjacent) vertices share the same colour.

 The total number of colours used does not exceed M.

🧩 Approach: Backtracking Algorithm

1. Start from the first vertex.

2. Try assigning it a colour from 1 to M.

3. Move to the next vertex and assign a valid colour.

4. Before assigning, check whether the colour is safe (i.e., not used by any adjacent vertex).

5. If safe:

o Assign the colour and proceed to the next vertex.

6. If no colour is safe:

o Backtrack to the previous vertex and try a different colour.

7. Repeat until:

o All vertices are coloured successfully (solution found), or

o No valid colouring is possible with M colours (no solution).

🛡 Safety Check:

A colour is considered safe for a vertex if none of its adjacent vertices have the same colour.

⏱ Time Complexity:

 Worst-case: O(MV)O(M^V)

o Where V = number of vertices

o M = number of colours

 This is exponential because every vertex has M choices.

🧠 Key Points:

 Type: NP-Complete Problem

 Technique: Backtracking / Constraint Satisfaction

 Special Case: When M=2M = 2, the problem checks if the graph is bipartite.

📌 Applications:

 Register allocation in compilers

 Scheduling problems (e.g., exams without student overlaps)

 Map colouring
  Frequency assignment in mobile radio systems

 Sudoku solver (can be modeled as graph colouring)

Here is a detailed explanation of real-world applications of the Graph Colouring Problem, covering
diverse domains with clarity:

🎯 Applications of Graph Colouring (Elaborated)

1. 🗺 Map Colouring

Objective: Assign colours to regions on a map so that no two adjacent regions (countries/states)
have the same colour.

 Each region is a vertex, and a border between two regions is an edge.

 Helps in visual distinction.

 Four Colour Theorem: Any map can be coloured with at most 4 colours such that no
adjacent regions share the same colour.

✅ Use Case: Geography textbooks, political maps, and digital mapping software.

2. 🧠 Register Allocation in Compilers

Objective: Assign limited CPU registers to variables without conflicts during execution.

 Each variable is a vertex.

 An edge exists if two variables are live at the same time (interfere).

 Colouring the graph means assigning registers such that no two interfering variables share a
register.

✅ Use Case: Optimizing code in compilers for better memory management and performance.

3. 🧪 Timetable Scheduling (Exam or Course)

Objective: Assign time slots to exams or classes such that no student or teacher has overlapping
sessions.

 Each exam/class is a vertex.

 An edge exists if a common student/teacher is involved.

 Colouring assigns a time slot (colour) ensuring no conflicts.

✅ Use Case: University exam scheduling, school timetable creation, conference planning.

4. 📶 Frequency Assignment in Mobile/Radio Networks

Objective: Assign frequencies to towers/transmitters such that adjacent towers do not interfere.

 Each transmitter/tower is a vertex.

 An edge represents interference risk (close physical proximity).

 Colours represent frequencies assigned.

✅ Use Case: Mobile network planning, radio broadcasting, satellite communications.

5. 🧩 Puzzle Solving (e.g., Sudoku)

Objective: Assign numbers to Sudoku cells such that no row, column, or box contains duplicates.

 Cells are treated as vertices.

 Edges connect cells in the same row, column, or block.

 Colours represent numbers (1–9 in 9x9 Sudoku).

✅ Use Case: Sudoku solvers, AI puzzle applications.

6. 🧰 Job Scheduling

Objective: Schedule jobs that share common resources without overlap.

 Each job is a vertex.

 An edge between two jobs means they share a resource and cannot run simultaneously.

 Colouring assigns time slots or processors to jobs.

✅ Use Case: Manufacturing systems, cloud computing, task scheduling.

7. 📚 Course Scheduling

Objective: Assign different time slots to courses with overlapping student enrollments.

 Each course is a vertex.

 An edge connects two courses if a student is enrolled in both.

 Colouring assigns non-conflicting time slots.

✅ Use Case: Timetabling systems in colleges, student registration systems.

8. 🧠 Artificial Intelligence (AI)

Objective: Represent and solve constraint satisfaction problems (CSPs).

 Many CSPs (like n-queens, Sudoku, map colouring) can be modeled as graph colouring.

 Used in backtracking-based AI algorithms for decision-making.

✅ Use Case: AI problem solving, robotics path planning, resource allocation in AI agents.

What is Branch and Bound Algorithm?

The Branch and Bound algorithm is a technique used in combinatorial optimization problems to
systematically search through the space of possible solutions. It is particularly useful for solving
problems where exhaustive search is not feasible due to the large number of possible solutions.

The basic idea behind Branch and Bound is to divide the problem into smaller subproblems, called
branches, and then solve each subproblem using some kind of systematic search method, such as
depth-first search or breadth-first search. As solutions are found for the subproblems, the algorithm
keeps track of the best solution found so far, and uses this information to prune the search tree by
eliminating branches that cannot possibly lead to a better solution than the current best solution.

The algorithm typically follows these steps:

1. Branching: The original problem is divided into smaller subproblems, typically by making a
series of decisions or choices.

2. Bounding: For each subproblem, an upper bound on the possible solutions is calculated. This
upper bound is used to determine whether the subproblem can be pruned (i.e., eliminated
from further consideration) without further exploration.

3. Searching: The algorithm systematically explores the space of possible solutions, typically
using depth-first search, breadth-first search, or some other search strategy.

4. Backtracking: If a subproblem cannot be pruned and does not lead to a feasible solution, the
algorithm backtracks to the previous decision point and explores a different branch.

5. Updating: As solutions are found for the subproblems, the algorithm updates the current
best solution found so far and adjusts the upper bounds accordingly.

The Branch and Bound algorithm continues this process until all branches have been explored or
pruned, at which point the best solution found is returned as the optimal solution to the original
problem.

Core Principles of the Branch and Bound Method

 Divide and Conquer Approach: The original problem is divided into smaller subproblems,
making it more manageable.

 Systematic Exploration: Subproblems are explored systematically using techniques like

depth-first search or breadth-first search.

 Upper Bound Calculation: An upper bound on the possible solutions is calculated for each
subproblem, aiding in pruning the search tree.

 Pruning: Subproblems that cannot possibly lead to a better solution than the current best
solution are eliminated from further consideration.

 Backtracking: If a subproblem does not lead to a feasible solution, the algorithm backtracks
to explore other branches.

Essential Features of Branch and Bound Integer Programming

  Discrete Decision Variables: Branch and Bound is particularly useful for problems where
decision variables are discrete.

 Objective Function: It deals with optimization problems where an objective function needs
to be maximized or minimized.

 Constraints: It considers constraints on decision variables, such as capacity constraints in

knapsack problems.

Applications of the Branch and Bound Method in Integer Programming

Branch and Bound is extensively used in various areas of integer programming, including:

1. Traveling Salesman Problem (TSP): In the TSP, the goal is to find the shortest possible route
that visits each city exactly once and returns to the original city. Branch and Bound can be
used to systematically explore the space of possible routes, pruning branches that are
guaranteed to be suboptimal. It helps in finding an optimal solution among the exponentially
large number of possible routes.

2. Knapsack Problem: The knapsack problem involves selecting a subset of items to maximize
the total value while staying within a given weight constraint. Branch and Bound can be
applied to search through the combinations of items, pruning branches that exceed the
weight constraint or cannot lead to an optimal solution. It efficiently explores the space of
possible item selections to find the optimal solution.

3. Job Scheduling: Job scheduling problems involve assigning tasks to resources over time,
considering constraints such as resource availability, precedence relationships, and job
durations. Branch and Bound can be used to search through the space of possible schedules,
pruning branches that violate constraints or cannot lead to an optimal schedule. It helps in
finding an optimal schedule that minimizes completion time or maximizes resource
utilization.

4. Network Flow Optimization: Network flow optimization problems involve determining the
flow of resources through a network, subject to capacity constraints and other restrictions.
Branch and Bound can be employed to search for the optimal flow configuration, pruning
branches that violate capacity constraints or cannot lead to an optimal flow. It helps in
finding the most efficient utilization of resources in transportation, communication, and
other network-based systems.

5. Facility Location Problems: Facility location problems involve deciding the locations of
facilities to minimize costs or maximize coverage, considering factors such as demand,
transportation costs, and facility capacities. Branch and Bound can be utilized to search for
the optimal facility locations, pruning branches that exceed capacity constraints or cannot
lead to an optimal solution. It assists in making strategic decisions regarding the placement
of facilities such as warehouses, factories, or service centers to optimize overall system
performance.

Importance of Branch and Bound in Algorithm Development

 Efficiency: Branch and Bound efficiently handles problems with large search spaces by
pruning the search tree.
  Optimality: It ensures that the optimal solution is found, given the constraints and objective
function.

 Flexibility: Branch and Bound can be adapted to various optimization problems, making it a
versatile technique.

Linking Branch and Bound and Decision Tree in Solving Complex Problems

 Decision Tree Representation: Branch and Bound can be conceptualized as traversing a

decision tree, where each node represents a decision point.

 Pruning Techniques: Similar pruning techniques are used in both Branch and Bound and
decision tree algorithms to reduce the search space.

Real-world Use-cases for Branch and Bound

 Logistics Optimization: Optimizing routes for delivery vehicles.

 Resource Allocation: Allocating resources efficiently in project management.

 Manufacturing Optimization: Scheduling production to minimize costs.

 Telecommunications Network Design: Optimizing the layout of communication networks.

 Portfolio Optimization: Selecting an optimal investment portfolio given various constraints

and objectives.

The branch and bound algorithm is a technique used for solving combinatorial optimization
problems. First, this algorithm breaks the given problem into multiple sub-problems and then using a
bounding function, it eliminates only those solutions that cannot provide optimal solution.

Combinatorial optimization problems refer to those problems that involve finding the best solution
from a finite set of possible solutions, such as the 0/1 knapsack problem, the travelling salesman
problem and many more.

When Branch and Bound Algorithm is used?

The branch and bound algorithm can be used in the following scenario −

 Whenever we encounter an optimization problem whose variables belong to a discrete set.

These types of problems are known as discrete optimization problems.

 As discussed earlier, this algorithm is also used to solve combinatorial optimization problem.

 If the given problem is a mathematical optimization problem, then the branch and bound
algorithm can also be applied.

How does Branch and Bound Algorithm work?

The branch and bound algorithm works by exploring the search space of the problem in a systematic
way. It uses a tree structure (state space tree) to represent the solutions and their extensions. Each
node in the tree is part of the partial solution, and each edge corresponds to an extension of this
solution by adding or removing an element. The root node represents the empty solution.

The algorithm starts with the root node and moves towards its children nodes. At each level, it
evaluates whether a child node satisfies the constraints of the problem to achieve a feasible solution.
This process is repeated until a leaf node is reached, which represents a complete solution.
Searching techniques in Branch and Bound

There are different approaches to implementing the branch and bound algorithm. The
implementation depends on how to generate the children nodes and how to search the next node to
expand. Some of the common searching techniques are −

 Breadth-first search − It maintains a queue of nodes to expand, which means this searching
technique uses First in First out order to search next node.

 Least cost search − This searching technique works by computing bound value of each node.
The algorithm selects the node with the lowest bound value to expand next.

 Depth-first search − It maintains a stack of nodes to expand, which means this searching
technique uses Last in First out order to search the next node.

Types of Branch and Bound Solutions

The branch and bound algorithm can produce two types of solutions. They are as follows −

 Variable size solution − This type of solution is represented by a subset which is the optimal
solution to the given problem.

 Fixed-size solution − This type of solution is represented by 0s and 1s.

Advantages of Branch and Bound Algorithm

The advantages of the branch and bound algorithm are as follows −

 It can reduce the time complexity by avoiding unnecessary exploration of the state space
tree.

 It has different search techniques that can be used for different types of problems and
preferences.

Disadvantages of Branch and Bound Algorithm

Some of the disadvantages of the branch and bound algorithm are as follows −

 In the worst case scenario, it may search for all the combinations to produce solutions.

 It can time consuming if the state space tree is too large.

The Travelling Salesman Problem (TSP) using the Branch and Bound technique is a systematic way of
reducing the search space by pruning unpromising paths, making it more efficient than brute force.
🧭 Travelling Salesman Problem (TSP): Brief Recap

Given a set of n cities and the cost of travel between each pair of cities, the goal is to find the
shortest possible route that visits each city exactly once and returns to the starting city.

🌳 Branch and Bound Approach (Step-by-Step)

✅ Key Concepts:

1. State Space Tree: Each node represents a partial path.

2. Cost Matrix: A matrix where cost[i][j] represents the cost to travel from city i to city j.

3. Bounding Function: Estimates the lower bound (minimum cost) for a node.

4. Pruning: Discard a path if its lower bound > best known solution.

🔢 Algorithm Steps

Step 1: Initial Cost Matrix Reduction

 Reduce the rows and columns of the cost matrix to get the initial lower bound.

 Row reduction: Subtract the minimum value in each row.

 Column reduction: Subtract the minimum value in each column.

 Sum of all subtracted values = Initial lower bound.

Step 2: Create the Root Node

 Path so far: Only the starting city (e.g., City 0).

 Remaining cities: All others.

 Lower bound: From Step 1.

 Push to priority queue (min-heap) based on lower bound.

Step 3: Branching

 Pick the node with the lowest bound from the queue.

 For each unvisited city from the current city, create a child node:

o Add cost from current city to the new city.

o Update cost matrix: Set visited rows and columns to ∞.

o Reduce the new cost matrix.

o Calculate total cost (bound + path so far).

 o If it's better than the best known solution, enqueue the child.

Step 4: Bounding and Pruning

 If a node's total estimated cost is greater than or equal to the best known cost, prune
(discard) it.

 Otherwise, keep exploring its children.

Step 5: Termination

 When all cities are visited and the last city returns to the start, update the best solution.

 When the queue is empty, the best solution is the shortest route.

✍️Example (Illustrated Briefly)

Given 4 cities with the following cost matrix:

A B C D

A ∞ 10 15 20

B 10 ∞ 35 25

C 15 35 ∞ 30

D 20 25 30 ∞

Initial lower bound (after matrix reduction): 80

Start from A → branch to B, C, D → compute new bounds and proceed recursively.
Eventually, the optimal path: A → B → D → C → A with cost: 80

⏱ Time Complexity

 Worst Case: O(n!), similar to brute force.

 Average Case: Much better due to pruning.

 Depends on how effective the bounding function is in eliminating paths.

✅ Advantages

 Much faster than brute force for moderate n.

 Guarantees optimal solution.

❌ Disadvantages
  Still exponential in worst case.

 Not practical for very large n (e.g., > 20).

In Design and Analysis of Algorithms (DAA), a lower bound is a fundamental concept used to
describe the minimum amount of work any algorithm must do to solve a problem. It helps in
understanding how efficient an algorithm can possibly be — no algorithm can perform better than
this bound in the worst case.

✅ What is a Lower Bound?

A lower bound of a problem is the minimum number of operations or time complexity that any
algorithm must take to solve the problem, in the worst-case scenario.

📘 Why Lower Bounds Matter

 Helps determine how close an algorithm is to optimal.

 If an algorithm matches the lower bound, it's asymptotically optimal.

 Guides researchers to know whether trying to improve an algorithm is worthwhile.

🧠 Types of Bounds in DAA

Type of
Meaning
Bound

Maximum time an algorithm may take — denotes the algorithm’s worst-case time
Upper Bound
(Big-O).

Lower Bound Minimum time any algorithm must take — represents the best possible performance.

Tight Bound When an algorithm's upper and lower bounds are the same (Θ-notation).

🧮 How is Lower Bound Calculated?

Usually by using mathematical proof techniques, such as:

1. Adversary Argument

 Assume an adversary tries to make the algorithm do the maximum amount of work.

 Used to prove comparison-based lower bounds.

Example: In comparison-based sorting (e.g., Merge Sort, Heap Sort):

 You must make comparisons between elements.

 It can be shown that Ω(n log n) is the lower bound for any comparison sort.

2. Decision Tree Model

  Tree represents all possible outcomes of comparisons.

 Depth of the tree = number of comparisons in the worst case.

 Used to prove bounds in sorting, searching, etc.

3. Information Theory

 Measures how much information must be gathered to solve a problem.

 Helps in deriving lower bounds for problems where we must "discover" hidden information.

📊 Lower Bounds for Common Problems

Problem Lower Bound

Comparison Sorting Ω(n log n)

Searching (Unsorted List) Ω(n)

Searching (Sorted List with Binary Search) Ω(log n)

Merging 2 sorted arrays Ω(n)

Matrix Multiplication (Standard) Ω(n³)

Convex Hull (2D) Ω(n log n)

🧩 Example: Sorting Lower Bound

 Suppose you want to sort n elements using comparisons.

 There are n! possible orderings.

 A comparison sort must distinguish between all of them.

 The height of the decision tree is ≥ log₂(n!) = Ω(n log n).

 So, no comparison-based algorithm can be better than that.

🧠 Summary

 A lower bound tells you the best time complexity you can hope for any algorithm solving a
problem.

 Helps evaluate the quality of your algorithm.

 If your algorithm matches the lower bound ⇒ it’s optimal.

An algorithm is a sequence of steps that take inputs from the user and after some computation,
produces an output. A parallel algorithm is an algorithm that can execute several instructions
simultaneously on different processing devices and then combine all the individual outputs to
produce the final result.
Concurrent Processing

The easy availability of computers along with the growth of Internet has changed the way we store
and process data. We are living in a day and age where data is available in abundance. Every day we
deal with huge volumes of data that require complex computing and that too, in quick time.
Sometimes, we need to fetch data from similar or interrelated events that occur simultaneously. This
is where we require concurrent processing that can divide a complex task and process it multiple
systems to produce the output in quick time.

Concurrent processing is essential where the task involves processing a huge bulk of complex data.
Examples include − accessing large databases, aircraft testing, astronomical calculations, atomic and
nuclear physics, biomedical analysis, economic planning, image processing, robotics, weather
forecasting, web-based services, etc.

What is Parallelism?

Parallelism is the process of processing several set of instructions simultaneously. It reduces the total
computational time. Parallelism can be implemented by using parallel computers, i.e. a computer
with many processors. Parallel computers require parallel algorithm, programming languages,
compilers and operating system that support multitasking.

In this tutorial, we will discuss only about parallel algorithms. Before moving further, let us first
discuss about algorithms and their types.

What is an Algorithm?

An algorithm is a sequence of instructions followed to solve a problem. While designing an

algorithm, we should consider the architecture of computer on which the algorithm will be executed.
As per the architecture, there are two types of computers −

 Sequential Computer

 Parallel Computer

Depending on the architecture of computers, we have two types of algorithms −

 Sequential Algorithm − An algorithm in which some consecutive steps of instructions are

executed in a chronological order to solve a problem.

 Parallel Algorithm − The problem is divided into sub-problems and are executed in parallel to
get individual outputs. Later on, these individual outputs are combined together to get the
final desired output.

It is not easy to divide a large problem into sub-problems. Sub-problems may have data dependency
among them. Therefore, the processors have to communicate with each other to solve the problem.

It has been found that the time needed by the processors in communicating with each other is more
than the actual processing time. So, while designing a parallel algorithm, proper CPU utilization
should be considered to get an efficient algorithm.

To design an algorithm properly, we must have a clear idea of the basic model of computation in a
parallel computer.

Model of Computation
Both sequential and parallel computers operate on a set (stream) of instructions called algorithms.
These set of instructions (algorithm) instruct the computer about what it has to do in each step.

Depending on the instruction stream and data stream, computers can be classified into four
categories −

 Single Instruction stream, Single Data stream (SISD) computers

 Single Instruction stream, Multiple Data stream (SIMD) computers

 Multiple Instruction stream, Single Data stream (MISD) computers

 Multiple Instruction stream, Multiple Data stream (MIMD) computers

SISD Computers

SISD computers contain one control unit, one processing unit, and one memory unit.

In this type of computers, the processor receives a single stream of instructions from the control unit
and operates on a single stream of data from the memory unit. During computation, at each step,
the processor receives one instruction from the control unit and operates on a single data received
from the memory unit.

SIMD Computers

SIMD computers contain one control unit, multiple processing units, and shared memory or
interconnection network.

Here, one single control unit sends instructions to all processing units. During computation, at each
step, all the processors receive a single set of instructions from the control unit and operate on
different set of data from the memory unit.
Each of the processing units has its own local memory unit to store both data and instructions. In
SIMD computers, processors need to communicate among themselves. This is done by shared
memory or by interconnection network.

While some of the processors execute a set of instructions, the remaining processors wait for their
next set of instructions. Instructions from the control unit decides which processor will
be active (execute instructions) or inactive (wait for next instruction).

MISD Computers

As the name suggests, MISD computers contain multiple control units, multiple processing
units, and one common memory unit.

Here, each processor has its own control unit and they share a common memory unit. All the
processors get instructions individually from their own control unit and they operate on a single
stream of data as per the instructions they have received from their respective control units. This
processor operates simultaneously.

MIMD Computers

MIMD computers have multiple control units, multiple processing units, and a shared
memory or interconnection network.
Here, each processor has its own control unit, local memory unit, and arithmetic and logic unit. They
receive different sets of instructions from their respective control units and operate on different sets
of data.

Note

 An MIMD computer that shares a common memory is known as multiprocessors, while

those that uses an interconnection network is known as multicomputers.

 Based on the physical distance of the processors, multicomputers are of two types −

o Multicomputer − When all the processors are very close to one another (e.g., in the
same room).

o Distributed system − When all the processors are far away from one another (e.g.- in
the different cities)