EC9600: Applied Algorithm
Introduction to Algorithms II
Department of Computer Engineering
Faculty of Engineering
University of Jaffna
Mr Nishankar S
BSc Eng (Hons)
Algorithm Techniques
Classification of Algorithms
● Content:
○ By Purpose:
■ Searching Algorithms
■ Sorting Algorithms
■ Optimization Algorithms
○ By Approach:
■ Brute Force
■ Greedy
■ Divide and Conquer
■ Dynamic Programming
■ Backtracking
Brute Force
● Tries all possible solutions to find the correct one.
● Example: String Matching (Naïve Approach)
● Pseudocode for String Matching
Pros:
● Simple to implement
Cons:
● Highly inefficient for large input sizes
Brute Force - String Matching
1. String Matching (Naive Approach)
The brute-force approach to string matching involves checking every possible position of the pattern in the text to see if it matches.
● Problem: Given a text T and a pattern P, find all occurrences of P in T.
● Steps:
1. Start with the first character in the text.
2. Compare the pattern with the substring of the text.
3. If they match, return the position. If not, move to the next character and repeat.
4. Continue until the end of the text.
● Time Complexity: O(n * m), where n is the length of the text and m is the length of the pattern.
Brute Force - Traveling Salesman
2. Traveling Salesman Problem (TSP)
In the brute-force solution to the TSP, we try every possible route to find the shortest path that visits all cities and returns to the starting
city.
● Problem: Given a set of cities and distances between them, find the shortest possible route that visits each city exactly once
and returns to the origin city.
● Steps:
1. Generate all possible permutations of the cities.
2. Calculate the total distance for each permutation.
3. Return the permutation with the smallest total distance.
● Time Complexity: O(n!), where n is the number of cities.
Brute Force Knapsack Problem
3.Knapsack Problem (0/1 Knapsack)
The brute-force approach to the knapsack problem tries all possible combinations of items to determine the maximum value that can
be carried in the knapsack without exceeding its weight limit.
● Problem: Given n items with weights and values, find the maximum value you can carry in a knapsack of capacity W.
● Steps:
1. Generate all possible combinations of items.
2. For each combination, check if the total weight is less than or equal to the knapsack's capacity.
3. If it is, compute the total value and track the maximum value.
● Time Complexity: O(2^n), where n is the number of items.
Divide & Conquer
● Content:
○ Break a problem into subproblems, solve each recursively, and combine results.
○ Example: Merge Sort
○ Pseudocode for Merge Sort
○ Pros:
■ Efficient for large datasets
■ Parallelizable
○ Cons:
■ Recursion overhead
Divide & Conquer - Merge Sort
Problem: Sort an array of integers in ascending order.
Divide and Conquer Approach:
1. Divide: Split the array into two halves.
2. Conquer: Recursively sort each half.
3. Combine: Merge the two sorted halves into one sorted array.
Divide & Conquer - Merge Sort
● Divide: Splitting the array into two halves takes O(1).
● Conquer: Recursively sorting the two halves, where each recursive call
processes a subarray of size n/2, takes T(n/2)time.
● Combine: Merging the two sorted halves takes O(n).
The recurrence relation for merge sort is:
T(n) = 2T(n/2) + O(n)
Solving this recurrence using the master theorem, we get:
T(n)=O(nlogn)
● Best Case: O(nlogn)
● Average Case: O(nlogn)
● Worst Case: O(nlogn)
Dynamic Programming
Content:
● Solving complex problems by breaking them into simpler overlapping subproblems.
● Memoization (Top-down) vs. Tabulation (Bottom-up)
● Example: Fibonacci Sequence
● Pseudocode for Fibonacci using DP
● Pros:
○ Optimizes time complexity by avoiding recomputation
● Cons:
○ Higher space complexity due to storing results
Dynamic Programming - Longest Subsequence
Given two strings, find the longest subsequence present in both.
Problem: Given strings X and Y, find the length of their longest common subsequence.
Dynamic Programming Approach: Use a 2D table to store the lengths of LCS of substrings of X and Y.
DP - Longest Palindromic SubString
LeetCode Problem
DP - 0/1 Knapsack Problem
Dynamic Programming Approach: Use a 2D table to store the maximum value for each subproblem (considering a subset of items
with a given weight capacity).
Time Complexity:
● Time Complexity: O(n×W), where n is the number of items, and W is the weight capacity.
● Space Complexity: O(n×W)
Greedy Algorithm
Greedy Algorithm: A problem-solving approach that makes a sequence of choices, each of which looks the best at the moment.
Greedy Choice Property: A global optimum can be arrived at by selecting a local optimum.
Optimal Substructure: An optimal solution to the problem contains optimal solutions to subproblems.
Examples of Greedy Algorithms
1. Activity Selection Problem
2. Fractional Knapsack Problem
3. Huffman Coding
4. Minimum Spanning Tree (Kruskal's and Prim's algorithms)
5. Dijkstra's Shortest Path Algorithm
Greedy Algorithms - Fractional Knapsack
Problem:
Given n items with weights and values, maximize the
value of items that fit into a knapsack of capacity W. You
can take fractional parts of an item.
Greedy Approach:
1. Calculate value-to-weight ratio for each item.
2. Sort items based on this ratio.
3. Add items to the knapsack starting with the highest
ratio.
Greedy Algorithms - Activity Selection
Problem:- Given n activities with start and finish times, select the maximum
number of activities that don't overlap.
Greedy Algorithm - Activity Selection
Back Tracking
Backtracking: A systematic method for solving constraint satisfaction problems by trying out various possibilities and undoing invalid
choices.
Concept: It incrementally builds candidates for a solution and abandons a candidate as soon as it determines that the candidate
cannot lead to a valid solution.
Applications:
● Puzzle solving (e.g., Sudoku, N-Queens)
● Combinatorial search problems
● Pathfinding (e.g., mazes)
Key Concepts of Backtracking
● Recursive Algorithm: Uses recursion to explore all possible solutions.
● Candidate Solution: A partial solution to a problem at any stage of the recursion.
● Constraints: Conditions that must be met for a valid solution.
● Pruning: Discarding a candidate solution as soon as it violates the problem's constraints.
Steps in Backtracking
1. Start with an empty solution.
2. Generate candidates for the solution.
3. Check constraints:
○ If valid, proceed to the next step.
○ If invalid, backtrack and try another candidate.
4. Repeat until a valid solution is found or all options have been explored.
Backtracking N-Queen Problem
Problem: Place N queens on an N×N chessboard such that no two queens threaten each other.
Backtracking Approach:
1. Place one queen in each row, ensuring no conflict with previously placed queens.
2. Use recursion to explore possibilities.
3. If a placement is invalid, backtrack and try another position.
Backtracking N-Queen Problem
GeeksForGeeks
Real World Applications
Divide and Conquer
1. Sorting Algorithms (Merge Sort, Quick Sort): Dividing an unsorted list into smaller sublists, sorting those, and then merging
them back together. This technique efficiently sorts data in various applications, like database management systems.
2. Image Processing (Convolution): Breaking down an image into smaller sections, processing them individually (e.g., applying
filters), and then combining the results. This is used in applications like facial recognition or medical imaging.
Dynamic Programming
1. Resource Allocation Problem: Optimizing the distribution of resources among competing projects to maximize overall profit or
minimize costs. This approach is used in project management and finance.
2. Inventory Management: Determining the optimal stock levels for various products to minimize holding costs while meeting
customer demand. Dynamic programming can help in calculating the most cost-effective inventory strategy.
Real World Applications
Greedy Algorithm
1. Network Routing: Determining the shortest path for data packets to travel across a network. The greedy approach selects the
nearest neighbor node at each step to minimize latency.
2. Change Making Problem: Finding the minimum number of coins needed to make a specific amount of change using available
denominations. This is relevant in financial applications and vending machines.
Backtracking
1. Sudoku Solver: Filling a Sudoku grid by trying various placements for numbers and backtracking when encountering conflicts.
This is applicable in developing intelligent gaming software.
2. Scheduling Tasks in a Manufacturing Plant: Consider a factory where different machines must complete a set of tasks. Each
machine can perform only one task at a time, and some tasks must be done in a specific order (due to dependencies between
tasks).
