UNIT 5
NP-HARD AND NP-COMPLETE PROBLEMS
1. Basic Concepts
• P (Polynomial Time):
• Refers to the class of decision problems that can be solved by a deterministic
algorithm in polynomial time. Examples include sorting and searching problems like
Merge Sort and Binary Search.
• NP (Non-deterministic Polynomial Time):
Refers to decision problems for which a solution can be verified in polynomial time, even if
finding the solution may not necessarily be in polynomial time. For example, Sudoku:
Given a solution, it’s easy to verify its correctness.
• Decision Problem:
A problem with a "yes" or "no" answer. For example, "Is there a path in a graph with a cost
less than 10?"
• Reduction:
The process of transforming one problem into another. If problem A can be reduced to
problem B, solving B helps solve A.
2. Non-deterministic Algorithms
• A non-deterministic algorithm is a theoretical model of computation where
multiple paths or choices can be explored simultaneously.
• Such algorithms involve two main steps:
o Guessing: Non-deterministically guess a solution.
o Verification: Verify if the guessed solution is correct in polynomial time.
Example:
For the Hamiltonian Path Problem (finding a path that visits each vertex exactly once in a
graph):
o Guess: Randomly choose a path.
o Verify: Check if the chosen path satisfies the conditions in polynomial time.
3. NP-Hard and NP-Complete Classes
NP-Hard Problems
• Definition: A problem is NP-hard if every problem in NP can be reduced to it in
polynomial time.
• Characteristics:
o It may or may not belong to the class NP.
o Solving an NP-hard problem efficiently implies we can solve all NP problems
efficiently.
o Example: The Traveling Salesman Problem (TSP)
▪ Given a list of cities and distances between them, find the shortest
possible route visiting each city once and returning to the start.
NP-Complete Problems
• Definition: A problem is NP-complete if:
o It belongs to NP (solutions can be verified in polynomial time).
o It is NP-hard (every NP problem can be reduced to it).
• These problems are the "hardest" in NP. If any NP-complete problem can be solved
in polynomial time, then every NP problem can be solved in polynomial time.
• Examples:
o Satisfiability Problem (SAT): Given a Boolean formula, determine if there’s
an assignment of truth values to variables that makes the formula true.
o Subset Sum Problem: Determine if a subset of numbers exists that sums to
a given value.
4. Cook's Theorem in Detail
Cook’s Theorem is a foundational result in computational complexity theory that
identifies the first problem proven to be NP-complete, namely the Boolean Satisfiability
Problem (SAT). The theorem, proposed by Stephen Cook in 1971, states:
"The Boolean Satisfiability Problem (SAT) is NP-complete."
This theorem introduced the concept of NP-completeness and showed that all problems in
the class NP can be transformed into SAT in polynomial time.
Boolean Satisfiability Problem (SAT)
The SAT problem asks:
• Given a Boolean formula in conjunctive normal form (CNF), is there an
assignment of truth values (true/false) to the variables that makes the formula
true?
Example:
• Formula: (x1∨¬x2)∧(x2∨x3)
• Solution: Assign x1=True,x2=False,x3=True
Proof Overview of Cook’s Theorem
The proof shows that SAT is NP-complete by demonstrating:
1. SAT is in NP:
a. Given a truth assignment for a Boolean formula, verifying whether it satisfies
the formula can be done in polynomial time.
2. Every problem in NP can be reduced to SAT in polynomial time:
a. Any problem in NP can be solved by a non-deterministic Turing machine
(NDTM) in polynomial time.
b. The computation of the NDTM can be represented as a Boolean formula.
c. The Boolean formula is satisfiable if and only if the NDTM accepts the input.
Steps in the Proof:
1. Represent the computation of a Turing machine (solving an NP problem) using
Boolean variables.
2. Encode the transition rules of the Turing machine into Boolean logic.
3. Show that the resulting Boolean formula is satisfiable if and only if the Turing
machine accepts the input.
Implications of Cook’s Theorem
• SAT as a Benchmark: SAT serves as the first NP-complete problem, providing a
basis for proving other problems NP-complete.
• P vs NP Problem: If SAT can be solved in polynomial time, then P=NP, solving one
of the greatest open questions in computer science
UNIT 4
Backtracking
Backtracking is a general algorithmic technique for solving problems recursively by trying
to build a solution incrementally, removing those solutions that fail to satisfy the
constraints of the problem at any point.
General Method
1. Recursive Approach:
a. Use recursion to explore all possible solutions.
b. At each step, make a decision and recursively proceed to the next step.
2. Backtrack:
a. If a decision leads to an invalid solution, undo the decision (backtrack) and
try another.
3. Key Components:
a. State Space Tree: Represents all possible states of the solution.
b. Pruning: Cut off branches of the tree that cannot possibly lead to a valid
solution.
Applications of Backtracking
1. N-Queens Problem
The N-Queens problem involves placing N queens on an N× N chessboard so that no two
queens threaten each other (no two queens can be in the same row, column, or diagonal).
Algorithm:
1. Start with the first row.
2. Place a queen in a column.
3. Check if it’s safe (not threatened).
4. If safe, proceed to the next row.
5. If not safe or no valid position, backtrack.
Example (4-Queens):
• Chessboard size: 4×4
• Steps:
o Place a queen in row 1, column 1.
o In row 2, place a queen in the first safe column.
o Continue until either all queens are placed or backtrack if stuck.
2. Sum of Subsets Problem
Find all subsets of a set whose sum is equal to a given target.
Algorithm:
1. Consider each element either included or excluded.
2. Keep track of the current sum.
3. If the current sum equals the target, record the solution.
4. Backtrack if the current sum exceeds the target.
Example:
• Set: [10, 7, 5, 18, 12, 20, 15]
• Target Sum: 35
• Solution: [10,5,20],[7,12,15]
3. Graph Coloring
Assign colors to vertices of a graph such that no two adjacent vertices have the same color
using the minimum number of colors.
Algorithm:
1. Start with the first vertex.
2. Assign a color to the vertex that does not conflict with its neighbors.
3. Proceed to the next vertex.
4. Backtrack if no valid color is available.
Example:
• Graph: V={A,B,C,D},E={(A,B),(B,C),(C,D),(D,A),(A,C)}
• Solution: 3 colors (A=1,B=2,C=3,D=1).
4. Hamiltonian Cycle
Find a cycle in a graph that visits each vertex exactly once and returns to the starting
vertex.
Algorithm:
1. Start at a vertex.
2. Add a vertex to the path only if it hasn’t been visited and it connects to the current
vertex.
3. Check if the path forms a cycle.
4. Backtrack if stuck.
Example:
• Graph: V={A,B,C,D},E={(A,B),(B,C),(C,D),(D,A),(A,C)}
• Solution: A→B→C→D→A.
Branch and Bound
Branch and Bound (B&B) is a systematic method for solving optimization and decision
problems, particularly those involving combinatorial optimization. It explores the solution
space by branching into smaller subproblems and using bounds to prune suboptimal
branches.
General Method
1. Branching:
a. Divide the problem into smaller subproblems (branches).
b. Explore one branch at a time.
2. Bounding:
a. Compute an upper or lower bound for the objective function for each
subproblem.
b. Use this bound to decide whether a branch can potentially lead to an optimal
solution.
3. Pruning:
a. Discard branches that cannot yield a better solution than the current best
solution (prune).
4. Search Strategies:
a. LC (Least Cost): Explore the branch with the smallest cost first.
b. FIFO (First In, First Out): Explore branches in the order they are generated
(queue-based).
Applications of Branch and Bound
1. Travelling Salesperson Problem (TSP)
Problem: Find the shortest possible route that visits every city exactly once and returns to
the starting city.
Steps:
1. Represent the problem as a graph where nodes are cities and edges are distances.
2. Use a cost matrix to represent distances between cities.
3. Apply B&B:
a. Branch: Divide the problem by fixing the next city in the route.
b. Bound: Calculate a lower bound for the route using the reduced cost matrix.
c. Prune: Discard branches with bounds higher than the current best solution.
Example:
• Cities: A,B,C,D
• Cost Matrix: A B C D
A - 10 15 20
B 10 - 35 25
C 15 35 - 30
D 20 25 30 -
• Branch: Fix A→B, calculate the remaining routes.
• Bound: Compute lower bound by reducing rows and columns of the cost matrix.
2. 0/1 Knapsack Problem
Problem: Given n items with weights w[i] and values v[i], and a knapsack with capacity W,
determine the maximum value that can be obtained by selecting items such that the total
weight does not exceed W.
LC Branch and Bound Solution (Least Cost):
• Use a priority queue to explore nodes with the highest bound first.
• Steps:
o Start with the root node representing no items in the knapsack.
o Branch: Include or exclude the current item.
o Bound: Compute the upper bound for the node using the fractional knapsack
approach.
o Prune: Discard nodes whose bound is less than the current best solution.
Example:
• Items: v = [40, 100, 50], w = [2, 3, 1], W=5
• Steps:
o Sort items by v/w: (100,3),(50,1),(40,2).
o Branch: Include/exclude items.
o Bound: Use the fractional method for unconsidered items.
FIFO Branch and Bound Solution:
• Use a queue to explore nodes in the order they are generated.
• Steps:
o Initialize the queue with the root node.
o Branch: Generate child nodes by including or excluding the current item.
o Bound: Calculate bounds for child nodes.
o Enqueue child nodes that are not pruned.
Example:
• Items: v=[40,100,50], w=[2,3,1], W=5
• Steps:
o Start with the root node (value = 0, weight = 0).
o Enqueue nodes for item1 included/excluded.
o Continue processing nodes in FIFO order.
UNIT 3
Graphs: Algorithm and Analysis
Graphs are data structures consisting of nodes (vertices) and edges connecting them.
They are widely used in computer science to represent relationships and solve problems
like traversal, connectivity, and optimization.
1. Breadth-First Search (BFS)
Definition:
BFS is an algorithm for traversing or searching a graph level by level, exploring all
neighbors of a vertex before moving to the next level. It works for both directed and
undirected graphs.
Algorithm:
1. Start from a source vertex.
2. Use a queue to maintain the order of exploration.
3. Mark each visited vertex and enqueue it.
4. Dequeue a vertex, process it, and enqueue its unvisited neighbors.
Example:
Graph:
V={A,B,C,D,E},E={(A,B),(A,C),(B,D),(C,D),(D,E)}
BFS from A:
1. Start: A (enqueue: A).
2. Explore A: Enqueue B,C.
3. Explore B: Enqueue D.
4. Explore C: D already visited.
5. Explore D: Enqueue E.
6. Explore E: Done.
Output: A→B→C→D→E
2. Depth-First Search (DFS)
DFS explores as far as possible along a branch before backtracking. It uses a stack (or
recursion) to explore.
Algorithm:
1. Start from a source vertex.
2. Use a stack (or recursive calls) to explore a path.
3. Mark each visited vertex and explore its unvisited neighbors.
4. Backtrack when no more neighbors exist.
Example:
Graph:
V={A,B,C,D,E},E={(A,B),(A,C),(B,D),(C,D),(D,E)}
DFS from A:
1. Start: A (push: A).
2. Explore A: Push B, then C.
3. Explore C: Push D.
4. Explore D: Push E.
5. Backtrack: Done.
Output: A→C→D→E→B
3. Spanning Trees
A spanning tree of a graph is a subgraph that:
1. Includes all the vertices.
2. Is a tree (connected and acyclic).
Properties:
• A graph with nn vertices has n−1 edges in its spanning tree.
• A connected graph can have multiple spanning trees.
Example:
Graph:
V={A,B,C,D},E={(A,B),(B,C),(C,D),(A,D),(B,D)}
One Spanning Tree:
Edges: (A,B),(B,C),(C,D).
4. Connected Components
Definition:
A connected component of an undirected graph is a maximal subgraph in which every pair
of vertices is connected.
Algorithm:
• Use BFS or DFS to explore all vertices of a graph. Each DFS/BFS call identifies one
connected component.
Example:
Graph:
V={A,B,C,D,E,F},E={(A,B),(B,C),(E,F)}
Output:
• Connected Components: {{A,B,C},{D},{E,F}}.
5. Bi-Connected Component:
A graph is bi-connected if:
1. It is connected.
2. It has no articulation points (vertices whose removal disconnects the graph).
Algorithm:
• Use DFS to identify articulation points and determine bi-connected components.
6. Articulation Points
An articulation point in a graph is a vertex whose removal increases the number of
connected components.
Algorithm (Tarjan’s Algorithm):
1. Perform DFS on the graph.
2. Track discovery time and lowest reachable vertex for each vertex.
3. A vertex u is an articulation point if:
a. u is the root of the DFS tree with at least two children.
b. u has a child v such that no vertex in the subtree rooted at v connects to u's
ancestor.
Example:
Graph:
V={A,B,C,D,E},E={(A,B),(B,C),(B,D),(C,E)}
Articulation Points:
• B: Removing B disconnects A from C,D,E.
Dynamic Programming (DP)
Dynamic Programming is a method for solving complex problems by breaking them down
into simpler subproblems. It solves each subproblem only once and stores the results for
future use, avoiding redundant computations.
General Method
1. Characterize the Structure:
a. Break the problem into smaller overlapping subproblems.
2. Recursive Solution:
a. Define the problem in terms of smaller subproblems.
3. Optimal Substructure:
a. Ensure the global optimal solution depends on the optimal solutions of its
subproblems.
4. Overlapping Subproblems:
a. Solve each subproblem only once and store the results.
5. Bottom-Up Approach:
a. Solve problems iteratively starting from the smallest subproblem and store
results in a table.
Applications of Dynamic Programming
1. Optimal Binary Search Trees (OBST)
Problem:
Given a set of nn keys with access probabilities p[1…n], construct a Binary Search Tree
(BST) such that the expected search cost is minimized.
Approach:
• Cost of a BST:
The cost is the weighted sum of depths of all nodes, weighted by their access
probabilities.
• Use DP to calculate the minimum cost for subproblems (trees for subsets of keys).
Steps:
1. Compute cumulative probabilities P[i, j] for subsets of keys.
2. Use a DP table to store the optimal costs for trees of different subsets.
3. Iteratively calculate the cost for larger subsets using smaller subsets.
Example:
Keys: {10,20,30}, Probabilities: {0.2,0.5,0.3}.
• Calculate the cost of all possible BST configurations using DP.
2. 0/1 Knapsack Problem
Problem:
Given n items with weights w[i] and values v[i], and a knapsack with capacity W, find the
maximum value obtainable by selecting items such that the total weight does not exceed
W.
Approach:
• Recursive Relation:
Example:
• Items: v = [60, 100, 120], w = [10, 20, 30],W = 50.
• DP Table tracks the maximum value for each capacity from 0 to W.
3. All-Pairs Shortest Path Problem (Floyd-Warshall Algorithm)
Problem:
Find the shortest paths between all pairs of vertices in a weighted graph.
Steps:
1. Initialize dp[i][j] with the graph's adjacency matrix.
2. Update distances for each intermediate vertex k.
4. Travelling Salesperson Problem (TSP)
Problem:
Given a set of nn cities and the distance between every pair of cities, find the shortest
possible route that visits each city exactly once and returns to the starting city.
Approach:
• Represent the problem as a graph.
• Use a bitmask dp[mask][i] where:
o mask represents visited cities.
o i represents the current city.
5. Reliability Design
Problem:
Design a reliable system by selecting components for nn subsystems such that the overall
reliability is maximized while staying within a cost constraint.
