Class: MSc(Computer Science Sem I)

Subject: Design and Analysis of Algorithm

Ms. Sujata Patil

Dr. D.Y.Patil ACS College, Pimpri.
MSc(Computer Science)
Course Name: Design and Analysis of Algorithm
Chapter 1: Basics of Algorithms
❖Algorithm definition and characteristics
❖Space complexity
❖Time complexity
❖worst case-best case-average case complexity, asymptotic notation
❖Recursive and non-recursive algorithms
❖Sorting algorithms (insertion sort, heap sort, bubble sort)
❖Sorting in linear time: counting sort, concept of bucket and radix sort
❖Searching algorithms: Linear, Binary
 Chapter 2: Divide and conquer strategy
❖General method, control abstraction
❖Binary search
❖Merge sort, Quick sort
❖Comparison between Traditional Method of
❖Matrix Multiplication vs. Strassen’s Matrix Multiplication
Chapter 3: Greedy Method
❖ Knapsack problem
❖ Job sequencing with deadlines,
❖ Minimum-cost spanning trees: Kruskal and Prim’s algorithm
❖ Optimal storage on tapes
❖ Optimal merge patterns
❖ Huffman coding
❖ Shortest Path :Dijkstra’s Algorithm
 Chapter 4: Dynamic Programming
❖Principle of optimality
❖Matrix chain multiplication
❖0/1 Knapsack Problem
❖ i)Merge & Purge
❖ ii)Functional Method
❖ Bellman Ford Algorithm
❖All pairs Shortest Path
❖Floyd- Warshall Algorithm
❖Longest common subsequence,
❖String editing, Travelling Salesperson problem
 Chapter 5 Decrease and Conquer
❖Definition of Graph Representation of Graph
❖By Constant - DFS and BFS
❖Topological sorting
❖Connected components and spanning trees
❖By Variable Size decrease Euclid’s algorithm
❖Articulation Point and Bridge edge
Chapter 6 Backtracking
❖ General method
❖ Fixed Tuple vs. Variable Tuple Formulation
❖ n- Queen’s problem
❖ Graph coloring problem
❖ Hamiltonian cycle
❖ Sum of subsets
 Chapter 7 Branch and Bound
❖Introduction
❖FIFO BB Search, LIFO Search
❖Definitions of LCBB Search
❖Bounding Function, Ranking Function
❖Traveling Salesman problem Using Variable tuple Formulation using LCBB
❖0/1 knapsack problem using LCBB

Chapter 8 Problem Classification

❖Nondeterministic algorithm
❖The class of P, NP, NP-hard and NP –Complete problems
❖Cook’s theorem
 Chapter 1: Basics of Algorithms
❖Algorithm definition and characteristics:
Algorithm: An algorithm is a finite set of instructions which if followed, accomplish a
particular task. It is method that can be used by computer for the solution of user
requirement.
The word algorithm comes from the name of a Persian mathematician, Abu Ja’far
Mohammed ibn Musa al Khowarizmi.
❖Characteristics of algorithm:
All algorithm must satisfy the following criteria.
1. Input: Zero or more quantities are externally supplied. If there are 0 external inputs
then the input will be taken internally.
2. Output: At least one quantity is produced.
3. Definiteness: Each instruction is clear and unambiguous e.g. add 5 to x or y is
not clear statement and “Divide 5 by 0” is indefinite.
4. Finiteness: If we trace out the instructions of an algorithm then for all cases the
algorithm terminates after a finite number of steps.
e.g. operating system is exceptional. It is made to run infinitely till power is
not off.
5. Effectiveness: Every instruction must be very basic so that it can be carried out in
principle by a person using only paper and pencil.
Advantages of Algorithms:
❖ It is easy to understand.
❖ Algorithm is a step-wise representation of a solution to a given problem.
❖ In Algorithm the problem is broken down into smaller pieces or steps hence, it is
easier for the programmer to convert it into an actual program.
Algorithm Analysis:
How to Design an Algorithm?
• In order to write an algorithm, following things are needed as a pre-requisite:
• The problem that is to be solved by this algorithm.
• The constraints of the problem that must be considered while solving the problem.
• The input to be taken to solve the problem.
• The output to be expected when the problem is solved.
• The solution to this problem, in the given constraints.
❖How to devise an algorithm:
Creating an algorithm is an art which may never be fully automated.
Various design techniques are provided that have proven to be useful in giving good
algorithms. By mastering these design strategy it will become easier for you to devise
new and useful algorithms.
❖Different design strategies are:
a. Divide and Conquer
b. Greedy Method
c. Dynamic Programming
d. Backtracking
e. Branch Bound
Some of the techniques are especially useful in fields other than computer science as
operational research and electrical engineering.
❖How to validate the algorithm?
Once an algorithm is devised it is necessary to show that it computes the correct output
for all legal input. The purpose of validation is to assure that the algorithm will perform
correctly independent of the programming language in which it will be eventually
implemented.
❖How to analyze the algorithm:
Analysis refers to the task of determining how much computing time and storage the
algorithm requires. It also contains efficiency constraints such as how well does the
algorithm perform for best case, worst case and average case of input.
❖How to test an program:
Testing involves 2 phases.
In debugging and profiling the program is executed on sample data. If the result is fault
the program is corrected.
Debugging: can give the presence of errors but not their absence.
Profiling: Profiling means executing a correct program on the data and measuring the
time and space required to compute the result.
Profiling is useful for optimization.
Complexity:
The complexity of algorithm is measured in two terms.
1. Space Complexity:
Deals with storage requirement.
2. Time Complexity:
Deals with computing time.
❖Space complexity:
It is defined as the amount of memory required for running an algorithm.
Space complexity of an algorithm is sum of the following two components.
1. Fixed Part: It is memory required by that part of the program, which is independent
of input and output characteristics. It includes space required for instruction (code),
space required for simple and fixed size variables and space required for constants.
2. Variable Part:
It is memory required by that part of the program which is dependent on particular
problem instance. It includes space needed for stack in case of recursion, which also
depends on instance characteristics.
Space increases or decreases depending upon whether the program uses iterative or
recursive procedures.
The space complexity of an algorithm P is denoted by S(P).
S(P)=C + Sp
Where C is constant which is fixed part of the memory.
Sp is instance characteristics which is variable part.
Time Complexity:
❖Time complexity of an algorithm (P) is denoted by t(P).
❖Time complexity of algorithm is sum of compilation time and execution time of an
algorithm P. But compile time do not depend upon the instance characteristics, also once
compiled program can be executed several times without recompilation.
❖ Therefore runtime of program is considered in calculation of time complexity
which is denoted by t(P).
❖ t(P) depends upon every operation.
❖ Example: addition, multiplication, division, loads etc.
Tp (n) =Ca Add (n)+ Cb Sub (n) + Cm Mul(n) +…………
Where n denotes instance characteristics. C denotes time required for respective
operations. Obtaining certain formula for finding time complexity of each algorithm is
impossible because time needed for each algorithm is depends on the input.
❖If the execution time is physically clocked and programs are executed on particular
machine then we may get the exact time. But again some difficulties are there, such as
in multi user system execution time is dependent on some factors like load time and
time required for other running programs.
❖If we assume that each step takes one unit of time and is dependent on instance
characteristics then it is possible to calculate tp.
❖Each step – one unit of time
❖It is the amount of time taken by the algorithm to execute.
❖It depends upon number of times a statement executes, which is Frequency of the
statement.
❖In this method, we count number of times one instruction is executed.
❖The number of steps any program statement is assigned depends on the kind of
statement.
❖Example comment count as zero steps. Assignment count as one step.
➢Measures of Time Best, Worst and Average Case
Best Case:
Suppose we want to search an element from the array given below.
Assume that array is int a[4]={10,20,30,40};
Assume you need to search an element 10 from the array.
This is the best case of the problem.

10 20 30 40

▪ How many Comparisons have we done here?

Only 1 Comparison and the element is searched.
▪ Why it is the best case of the problem?
Because you need to compare the element only once.
Average Case:
10 20 30 40

Suppose we are asked to find element 20 from the array.

We need to compare this element with the array two times. Total Comparisons needed
are two. This is Average Case of the algorithm.
▪ How many comparisons are needed?
2 Comparisons i.e.: In general if there are n elements we require n/2 comparisons.
▪ Why it is called as Average Case of the Algorithm?
It is Average Case because we need to do 2 Comparisons i.e.:- half of the total no of
elements. In general if we have n elements total number of comparisons needed are n/2.
Worst Case:
10 20 30 40

Suppose you are asked to search element 50 in the array. You need to compare the element
50 with all elements in the array from beginning. But even after the comparison you are
not able to find the same. This is the worst case of the algorithm.

▪ How many Comparisons have we done here?

4 Comparisons i.e.: in general if we have n elements in the array we need to do n
comparisons.
▪ Why this is called as Worst Case of the Algorithm?
It is called as Worst Case because we are doing maximum number of Comparisons
here.ie: in general if we have n elements we need to do n comparisons.
❖Best Case Complexity:
The best case is a measure of minimum time that the algorithm will require for an
input of size n. When the situation is ideal. For example Already sorted input for
sorting algorithm.
Lower bound on running time.
❖ Average Case Complexity:
The amount of time the algorithm taken on an average set of inputs. Any random
input taken.
❖ Worst Case Complexity:
The amount of time the algorithm takes on the worst possible set of inputs or
maximum time required for n inputs. Reverse sorted input for a sorting algorithm.
❖Asymptotic Notations:
❖We have discuss space and time complexity of program where we can analyze the
exact counts rather than estimate. Asymptotic notation is used to make statements
about program performance when the instance characteristics are large.
❖Mathematical Tools
❖The Big-oh notation is most popular asymptotic notation used in the performance
analysis of program, the omega, theta and little-oh notations also are in common use.
❖A problem may have numerous algorithmic solutions. In order to choose the best
algorithm for particular task you need to be able to judge how long particular solution
will take to run or more accurately, you need to be able to judge how long two
solutions will take to run and choose the better of two. You don’t need to know how
many minutes and seconds they will take, but you do need some way to compare
against one another.
❖Big O Notation: (Worst Case running of algorithm)
❖This notation is used to denote upper bounds. It is expressed as O(g(n)) which means
that time taken never exceeds roughly f(n) operations.
❖Using Big (O) notation we can express the time complexity of a program in terms of
the order of magnitude of frequency count.
❖If we are going to compare the performance characteristics of different algorithms, we
need some benchmark to measure them against.
❖A standard notation has been developed to represent functions which bound the
computing time for algorithm.
❖This describes how the behavior of function is affected by the size of the dataset.
• O(1) indicates that time is constant means it is unrelated to the size of the input.
• O(n) says that the time it takes increases linearly with the size of the input.
• O(n^2) indicates that time grows as square of number of times.
❖An algorithm which is in class O(n) will run faster than one in the class O(n^2).
❖Big –O is used to specify upper bound on speed.
Example:
𝒇(𝒏) = 𝒏𝟑 + 𝟐𝒏𝟑+3
then the term with upper bound i.e. 𝒏 𝟑 determines the time complexity of the
Function f(n) and it is denoted as O(𝒏 𝟑 )
Definition:
f(n)=O(g(n) ( read as f of n equals big-oh of g of n) is true iff there exists two possible
constants c and no such that
❖Heap Sort:
▪ A heap is a complete binary tree and it is implemented in an array as an sequential
representation.
▪ Heap is a special tree-based data structure. A binary tree is said to follow a heap data
structure if it is a complete binary tree (every level, except possibly the last , is
completely filled and all the nodes are as far left as possible.
▪ It is comparison based sorting.
▪ All nodes in the tree follow the property that they are greater than their children i.e. the
largest element is at the root and both its children smaller than the root and so on. Such
a heap is called a max-heap.
▪ A heap is called max heap or descending heap if every node of heap has value greater
than or equal to the value of every child of that node.
▪ So in max heap root will keep the highest value.
▪ Similarly a heap is called min heap or ascending heap if every node of heap has a
value less than or equal to the value of every child of that node.
• The following example diagram shows Max-Heap and Min-Heap.
❖Sorting in Linear Time:
We have sorting algorithms that can sort "n" numbers in O (n log n) time. E.g. Heap
Sort, bubble Sort. All these methods determine the sorted order only by using
comparisons between the input elements. Such sorting algorithms are called
comparison sort.
The radix sort, sorts n numbers without any comparison between the elements. Such
sorting algorithms are called non-comparison sort. All these methods are possible in
linear time using certain assumptions.
1. Radix Sort:
▪ It is used to sort the data (numbers and alphabets)
▪ Data is not sorted by using any comparison. Here data is sorted digit by digit. Data
sorted according to place value of digit or position of that digit in that number.
▪ It sorts the number from least significant digit to most significant digit.
Radix is base
• For decimal number base is 10
• For alphabets base is 26
• For binary number base is 2
▪ If we have a list of names and we want to sort them alphabetically then first we take
the names which start with alphabet A and then we take names which start from
alphabet B and so on. In list of names the radix is 26(all alphabets)
▪ If the both names are same then in second pass it will be sorted by using second
alphabet.
▪ Similarly if we have list of numbers then there will be 10 buckets from 0 to 9 because
radix is 10.
▪ In the first pass we take the numbers on the basis of unit digit(least significant digit)
▪ In the second pass the number sorted on the basis of ten’s digit. If the largest number
has 3 digits then number will be sorted in maximum 3 passes.
➢Analysis:
▪ Radix sort is dependent on three things
1. Radix
2. Number of digits in largest element
3. Size of array elements
Number of passes is the number of digits in largest element i.e. d. In each pass we will
compare the digits of elements. So the maximum number of comparisons will be d*r*n
What is the running time of Radix Sort?
Radix Sort takes O(d* (n+b)) time where b is the base for representing numbers, for
example, for the decimal system, b is 10.
n is number of elements and d is number of digits in largest elements.
❖Bucket Sort:
▪ Bucket sort is mainly useful when input is uniformly distributed over a range.
▪ Bucket Sort is a sorting technique that sorts the elements by first dividing the elements
into several groups called buckets.
▪ The elements inside each bucket are sorted using any of the suitable sorting
algorithms (insertion sort ) to sort the elements.
▪ Finally combine the elements from each bucket and place in array. The array gets
sorted.
bucketSort(arr[], n)
1) Create n empty buckets
2) Do following for every array element arr [i].
Insert arr[i] into bucket
3) Sort individual buckets using insertion sort.
4) Concatenate all sorted buckets.
• Time Complexity: O(n + k) for best case and average case and O(n^2) for the
worst case.
❖Counting Sort:
▪ This is a simple algorithm for sorting an array.
▪ Counting sort assumes that each of the elements is an integer in the range 1 to k where
k=O(n), the counting sort runs in O(n) time.
▪ The basic idea of counting sort is to determine , for each input elements x, the number
of elements less than x. This information can be used to place directly into its correct
position.
▪ Consider the first element of the array. If number of elements less than the first
element in the complete array is 5. It indicates that the first element must be located
at 6th position in the array.
▪ For counting sort we are given array A[1..n] of length n. we required two more arrays,
the array B[1..n] holds the sorted output and array C[1..k] provides temporary working
storage.
• Algorithm: CountingSort(A,n,k)
// A[1..n] is an array of n integers to be sorted.
//’k’ is the range of integer values.
{for i= 1 to k do
C[i]=0;
for i=1 to n do
C[A[i]]=C[A[i]]+1;
//C[i] now contains the number of elements equal to i
for i=2 to k do
C[i]=C[i]+C[i-1];
//C[i] now contains the number of elements <=i
for i=2 to k do
C[i]=C[i]+C[i-1];
//C[i] now contains the number of elements <=i
for j=n to 1 do
{ B[C[A[j]]]=A[j];
// Putting sorted element into correct position
C[A[j]]=C[A[j] ] -1; //decrement the value by 1
}}
❖Searching Algorithm:
Searching Algorithms are designed to check for an element or retrieve an element from
any data structure where it is stored. Based on the type of search operation, these
algorithms are generally classified into two categories:
1. Linear Search
In Computer Science , a linear search or sequential search is a method for finding
An element within a list. It sequentially checks each element of the list until the match
is found or the whole list has been searched. Linear search is a very basic and simple
search algorithm. In Linear search, we search an element or value in a given array by
traversing the array from the starting, till the desired element or value is found.
It compares the element to be searched with all the elements present in the array and
when the element is matched successfully,
Worst Case Complexity and Average Case Complexity = O(n)
• It is used for unsorted and unordered small list of elements.
• It has a time complexity of O(n), which means the time is linearly dependent on the
number of elements.
2. Binary Search
▪ Binary search is the search technique which works efficiently on the sorted lists.
Hence, in order to search an element into some list by using binary search technique,
we must ensure that the list is sorted.
▪ Binary search follows divide and conquer approach in which, the list is divided into
two halves and the item is compared with the middle element of the list.
▪ If the match is found then, the location of middle element is returned otherwise, we
search into either of the halves depending upon the result produced through the match.
Tuesday, January 26, 2021 9:50 PM

Counting Sort:
▪ This is a simple algorithm for sorting an array.
▪ Counting sort assumes that each of the elements is an integer in the range 1 to
k where k=O(n), the counting sort runs in O(n) time.
▪ The basic idea of counting sort is to determine , for each input elements x, the
number of elements less than x. This information can be used to place directly
into its correct position.
▪ Consider the first element of the array. If number of elements less than the first
element in the complete array is 5. It indicates that the first element must be
located at 6th position in the array.
▪ For counting sort we are given array A[1..n] of length n. we required two more
arrays, the array B[1..n] holds the sorted output and array C[1..k] provides
temporary working storage.
▪ Algorithm: CountingSort(A,n,k)
▪ // A[1..n] is an array of n integers to be sorted.
//'k’ is the range of integer values.
{ for i= 1 to k do
C[i]=0;
for i=1 to n do
C[A[i]]=C[A[i]]+1;

//C[i] now contains the number of elements equal to i

for i=2 to k do
C[i]=C[i]+C[i-1];
//C[i] now contains the number of elements <=i
for j=n to 1 do
{
B[C[A[j]]]=A[j];
// Putting sorted element into correct position
Counting Sort Page 1
 // Putting sorted element into correct position
C[A[j]]=C[A[j] ] -1; //decrement the value by 1
}
}

❖ Searching Algorithm:
Searching Algorithms are designed to check or search for an element or
retrieve an element from any data structure where it is stored.
The Search is said to be successful if the required element is found otherwise it
Is unsuccessful.
Based on the type of search operation, these algorithms are generally classified
into two categories:
❖ Linear Search
In Computer Science , a linear search or sequential search is a method for
finding an element within a list. It sequentially checks each element of the list
until the match is found or the whole list has been searched.
Counting Sort Page 2
 until the match is found or the whole list has been searched.
Linear search is a very basic and simple search algorithm. In Linear search, we
search an element or value in a given array by traversing the array from the
starting, till the desired element or value is found.
It compares the element to be searched with all the elements present in the
array and when the element is matched successfully,
Best Case Complexity = O(1)
Worst Case Complexity and Average Case Complexity = O(n)
It is used for unsorted and unordered small list of elements.
It has a time complexity of O(n), which means the time is linearly dependent
on the number of elements.

for i=0; i<n;i++)

{ if (arr[i]==x)
{printf("element found"
flag=1;
break;
}}
if( flag==0) { printf("element not found");}

❖ Binary Search:
▪ The sequential search situation will be in worst case if the element is at
the end of the list.
▪ For eliminating this problem, we have one efficient searching technique called
binary search.
▪ The condition for binary search is that all data should be in a Sorted array.
▪ If it is less than the middle element then we search it in the left portion of the
array and if it is greater than the middle element then search will be in the right
portion of the array. Now we will take that portion only for search and
compare with middle element of that portion. This process will be in iteration
until we find the element or middle element has no left or right portion to
Counting Sort Page 3
 compare with middle element of that portion. This process will be in iteration
until we find the element or middle element has no left or right portion to
search.
❖ Non-Recursive Binary Search Algorithm:
Algorithm : BinSearch(a, n, x)
//Given an array a[1..n] of elements in non-decreasing order, n>=0,
Determine whether x is present, and if so return j such that x=a[j];
else return 0
{
low =1;
high=n;
While low<=high do
{
mid=(low + high)/2
if a[mid]=x then
return mid;
else if
a[mid] > x then
high=mid-1;
else
low=mid+1;
}
return 0;
}

Counting Sort Page 4

Counting Sort Page 5
Counting Sort Page 6
Counting Sort Page 7
Chapter 1 Basics of Algorithm
Wednesday, January 6, 2021 7:20 AM

Design and Analysis of Algorithm

Asymptotic_Big_O_Omega Page 1
Asymptotic_Big_O_Omega Page 2
Asymptotic_Big_O_Omega Page 3
❖ Asymptotic Notations:
• We have discuss space and time complexity of program where we can
analyze the exact counts rather than estimate. Asymptotic notation is used
to make statements about program performance when the instance
characteristics are large.
• Mathematical Way of representing the time complexity.
• The Big-oh notation is most popular asymptotic notation used in the
performance analysis of program, the omega, theta notations also are
in common use.
• A problem may have numerous algorithmic solutions. In order to choose
the best algorithm for particular task you need to be able to judge how long
particular solution will take to run or more accurately, you need to be able
to judge how long two solutions will take to run and choose the better of two.
• You don’t need to know about how many minutes and seconds they will
take, but you need some way to compare algorithms against one another.
• Asymptotic notations are mathematical tools to represent time complexity of
algorithms for asymptotic analysis. The following 3 asymptotic notations are
mostly used to represent time complexity of algorithms.
1. Big O Notation:
▪ It defines upper bound of an algorithm.
▪ Big-Oh (O) notation gives an upper bound for a function f(n) to within a
constant factor.
▪ It calculates the Worst Case Complexity of an Algorithm.
▪ It is a notation used to find out the efficiency of an algorithm.
▪ It gives the algorithm complexity in terms of input size n.
▪ Big – O notation focuses on Worst Case Complexity of the Algorithm.

Asymptotic_Big_O_Omega Page 4
▪ Big – O notation focuses on Worst Case Complexity of the Algorithm.

▪ Standard notation has been developed to represent functions which bound

the computing time for algorithm.
This describes how the behavior of function is affected by the size of the
dataset.
▪ O(1)- Constant time required
e.g. finding even or odd number
▪ O(n): Time increases linearly
e.g. Finding maximum element from unsorted array.
▪ O(n2) :Finding Duplicate elements from array.
Definition:-
The function f(n)= O (g(n)) [read as f of n is Big-O of g of n] if and only if
there exist positive constants C and such that
f(n) < = C*g(n) where C > 0 and n >= and >=1

❖ Big Omega Notation: Ω

It is a notation used to find out the efficiency of an algorithm.
○ It indicates the minimum time required for any algorithm for all input
values
○ Omega notation focuses on Best Case Complexity of the Algorithm.

Asymptotic_Big_O_Omega Page 5
○
○ It defines lower bound of an algorithm.
○ In simple words, when we represent a time complexity for any algorithm in
the form of big-Ω, we mean that the algorithm will take at least this much
time to complete it's execution.
Definition :- The function f(n)= Ω(g(n)) [read as f of n is Ω of g of n]
if and only if there exist positive constants C and no such that
f(n)> =C*g(n) where C>0 and n>= and >=1

Asymptotic_Big_O_Omega Page 6
Asymptotic_Big_O_Omega Page 7
Asymptotic_Big_O_Omega Page 8
Examples Page 1
Examples Page 2
Examples Page 3
Tuesday, January 26, 2021 11:40 AM

Theta Notation: Θ
• The theta notation bounds the function from above and below, so it defines
exact asymptotic behavior.
• This notation denotes both upper and lower bounds of f(n) and written as
f(n)=Θ (g(n))
• It is a notation used to find out the efficiency of an algorithm.
• It represents the Average Case Time Complexity.
• It is also called as tight bound of an algorithm
• It shows upper and lower bound of the algorithm.
Definition :- The function f(n)= Θ(g(n)) [read as f of n is Θ of g of n] if and
only if there exist positive constants C and such that
C1*g(n) <= f(n) < =C2 *g(n)
where C1 and C2 constants are>0 and n>= and >=1

Theta Notation Page 1

Theta Notation Page 2
Theta Notation Page 3
Theta Notation Page 4
Theta Notation Page 5
Tuesday, January 26, 2021 2:09 PM

Sorting algorithm:
▪ Sorting refers to arranging data in a particular format.
▪ Sorting algorithm specifies the way to arrange data in a particular order.
▪ Sorting algorithms may require some extra space for comparison and
temporary storage of few data elements.
▪ The algorithms do not require any extra space and sorting is said to happen
in-place, or for example, within the array itself.
▪ This is called in-place sorting. Bubble sort is an example of in-place sorting.
❖ Bubble Sort:
▪ Bubble sort is a simple sorting algorithm. This sorting algorithm is
comparison-based algorithm in which each pair of adjacent element is
compared and the elements are swapped if they are not in order. (if the first
element is greater than second element then swapping of element is
required)
▪ Bubble Sort compares all the element one by one and sort them based on
their values.
▪ If the given array has to be sorted in ascending order, then bubble sort will
start by comparing the first element of the array with the second element, if
the first element is greater than the second element, it will swap both the
elements, and then move on to compare second and third element, and so
on.
▪ If we have total n elements, then we need to repeat this process
for n-1 times.
▪ It is known as bubble sort, because with every complete iteration the largest
element in the given array, bubbles up towards the last place or the highest
index.
▪ Sorting takes place by stepping through all the elements one-by-one and
comparing it with the adjacent element and swapping them if required.
▪ Implementation:
Bubble_Sort Page 1
▪ Implementation:
Following are the steps involved in bubble sort(for sorting a given array in
ascending order):
1. Starting with the first element(index = 0), compare the current element
with the next element of the array.
2. If the current element is greater than the next element of the array, swap
them.
3. If the current element is less than the next element, move to the next
element. Repeat Step 1.
Algorithm
▪ We assume list is an array of n elements. We further assume that swap
function swaps the values of the given array elements.
Bubble Sort(n)
for (i=0;i<n-1;i++)
{
for (j=0;j<n-1-i;j++)
{
If (a[j] > a[j+1])
{
temp=a[j];
a[j]=a[j+1];
a[j+1]=temp;
}}}

Bubble_Sort Page 2
This algorithm is not suitable for large data sets as its average and worst
case complexity are of Ο(n2 )where n is the number of items.
Worst Case complexity= O(n2)
Average Case Complexity = O(n2)
Best Case Complexity= O(n)

Bubble_Sort Page 3
Tuesday, January 26, 2021 2:13 PM

❖ Insertion Sort Algorithm:

▪ Insertion sort is a simple sorting algorithm that works similar to the way
you sort playing cards in your hands.
▪ The array is virtually split into a sorted and an unsorted part.
▪ Values from the unsorted part are picked and placed at the correct position
in the sorted part.
▪ The insertion sort inserts each element in proper place.
▪ An element which is to be 'inserted in this sorted sub-list, has to find its
appropriate place and then it has to be inserted there. Hence the name,
insertion sort.
▪ The array is searched sequentially and unsorted items are moved and
inserted into the sorted sub-list.

Algorithm:
Insertion Sort(A)
{ for j=2 to A.length
key=A[j];
// Insert A[j] into sorted sequence A[1…..j-1]
i=j-1;
While(i>0 and A[i] > key)
A[i+1]=A[i];
i=i-1;
A[i+1]=key;
}

❖ Analysis:
In insertion sort we insert the element before or after and we start Comparison
from first element. Since first element has no other elements before it. So it
Insertion Sort Page 1
 from first element. Since first element has no other elements before it. So it
does not require any comparison.
Second element requires 1 comparison, third element requires 2 comparisons,
Fourth requires 3 comparisons and so on.
The last element requires n-1 comparisons. So total number of comparisons
will be
= 1+2+3+……………………+n-1
= n(n-1) / 2
= O(n2)

❖ Stable Sort:
▪ A sorting algorithm is said to be stable if two objects with equal keys appear
in the same order in sorted output as they appear in the input unsorted array.
▪ Some Sorting Algorithm is stable by nature like Insertion Sort and Bubble Sort
etc.

❖ In place sorting:
▪ It is in-place because this algorithm does not require extra memory space).
Insertion Sort Page 2
▪ It is in-place because this algorithm does not require extra memory space).

Insertion Sort Page 3

Chapter 2 Divide and Conquer
Monday, February 1, 2021 7:25 PM

Chapter 2: Divide and conquer strategy

• General method, control abstraction
• Binary search
• Merge sort, Quick sort
• Comparison between Traditional Method of Matrix Multiplication vs.
Strassen’s Matrix Multiplication
❖ General method:
▪ Divide and Conquer is a design approach that can be applied to several set
of problems.
▪ Given a function to compute on n inputs the divide and conquer strategy
suggests that splitting the inputs into k distinct subsets, 1<k<=n
subproblems.
▪ These subproblems must be solved, and then method must found to
combine sub solutions into a solution of a whole.
▪ If the subproblems are still relatively large, then the divide and conquer
strategy can possibly reapplied.
▪ Often the subproblems resulting from a divide and conquer design are of
the same type as the original problem.
❖ The three steps used are:
1. Divide: the problem into subproblems.
2. Conquer: Small subproblems are solved using some efficient method
while for larger subproblems divide and conquer approach is recursively
applied.
3. Combine: the solutions of the subproblems is combined to get the solution
of the original problem.
A design strategy can be applied to a variety of problems.
❖ Control abstraction for Divide and Conquer
Control abstraction means procedure, whose flow of control is clear but
whose primary operations are specified by other procedures.
Algorithm: DAndC(P)
{
if Small(P) then return S(P);
Ch 2 Divide and Conquer _General_method_Bin_Search Page 1
 if Small(P) then return S(P);
else
{
Divide P into smaller instances p1,p2,……pk, k>=1
Apply DAndC to each of these subproblem.
return Combine (DAndC (p1), DAndC (p2)…..DAndC (pk));
}
}
Control Abstraction for divide and Conquer
1. DAndC is initially invoked as DAndC(P), where P is the problem to be
solved.
2. Small (P) is a Boolean-valued function that determines whether the input
size is small enough that the answer can be computed without splitting.
If this is so, the function S is invoked. Otherwise the problem P is divided
into smaller subproblems.
These sub-problems p1,p2…..pk are solved by using recursive applications
of DAndC.
3. Combine is a function that determines the solution to P using the solutions
to the k subproblems.
❖ Computing Time of DAndC
If size of P is n and the sizes of the k subproblems are n1,n2,…..nk
respectively, then the computing time of DAndC is described by the
recurrence relation.
T(n)= g(n) n small
T(n1) + T(n2)+……Tnk + f(n) otherwise

Where T(n) is the time for DAndC on any input size n and g(n) is the time
to compute the answer directly for small inputs.
The function f(n) is the time for dividing P and combining the solutions to
subproblems. Assume that problem is of size of n.

Ch 2 Divide and Conquer _General_method_Bin_Search Page 2

❖ Binary Search:
▪ The sequential search situation will be in worst case if the element is at
the end of the list.
▪ For eliminating this problem, we have one efficient searching technique
called binary search.
▪ The condition for binary search is that all data should be in a Sorted array.
▪ If it is less than the middle element then we search it in the left portion of
the array and if it is greater than the middle element then search will be in
the right portion of the array. Now we will take that portion only for search
and compare with middle element of that portion. This process will be in
iteration until we find the element or middle element has no left or right
portion to search.
❖ Non-Recursive Binary Search Algorithm:
Algorithm : BinSearch(a, n, x)
//Given an array a[1..n] of elements in non-decreasing order, n>=0
Determine whether x is present, and if so return j such that x=a[j];

Ch 2 Divide and Conquer _General_method_Bin_Search Page 3

 Determine whether x is present, and if so return j such that x=a[j];

else return 0
{ low =1;
high=n;
While low<=high do
{ mid=(low + high)/2;
if a[mid]=x then
return mid;
else if
a[mid] > x then
high=mid-1;
else
low=mid+1;
}
return 0;
}

❖ Recursive BinarySearch algorithm:

Algorithm: Binsrch(a, i, l, x)
//Given an array a[i…..l] of elements in non-decreasing order
// Determine whether x is present and if so return j such that x=a[j];
else return 0;
if l=i then //If Small (P)
{
if x=a[i] then return i;
else return 0;
}
else

Ch 2 Divide and Conquer _General_method_Bin_Search Page 4

 { //Reduce P into a smaller subproblems
mid=(i+l)/2;
if a[mid]=x then return mid;
else if a[mid]>x then return
BinSrch(a,i,mid-1,x);
else return
BinSrch (a, mid+1,l, x);
}
}
❖ Performance Analysis of Binary Search:
▪ In binary search we are going to divide the considered problem into two
parts, which are approximately of the same size. K is the maximum number
of divisions performed.
If we have input size n such that we can divide problem into 2k
subproblems.
1. For best case the search element will be present at first mid position.
In such case the algorithm will require some constant time. So its time
complexity is O(1).
2. For average case, the number of divisions is less than k but it is not possible
to predict the amount of work saved in the average case. Thus time
complexity is O(log2 n)
3. For worst case and unsuccessful search, the number of divisions is exactly
K. Thus time complexity is O(log2 n)

Ch 2 Divide and Conquer _General_method_Bin_Search Page 5

Ch 2 Divide and Conquer _General_method_Bin_Search Page 6
Saturday, February 6, 2021 10:29 AM

❖ Merge Sort
▪ It is another example of divide and conquer.
▪ We assume the elements are sorted in non-decreasing order.
▪ Given a sequence of n elements a[1]…….a[n] the general idea is to split the
array into two sets a[1]……..a[n/2] and a[n/2+1,………a[n]
▪ Each set is individually sorted and the resulting sorted sequences are merged
to produce single sorted sequence of n elements.
▪ Thus we are splitting array into two equal sized sets and the combining
operation is the merging of two sorted sets into one.

Algorithm MergeSort (low, high)

// a[low: high] is a global array to be sorted.
// Small (P) is true if there is only one element to sort. In this case list is
already sorted.
{
if (low<high) then //if there are more than one element.
{
//divide P into subproblems
//Find where to split the set
mid=(low + high)/2;
// Solve the subproblems
MergeSort(low, mid);
MergeSort(mid+1, high);
//Combine the solutions
Merge(low, mid, high);
}
}

Algorithm Merge (low, mid, high)

Ch 2 Divide and Conquer_Merge Sort Page 1
 Algorithm Merge (low, mid, high)
//a[low: high] is a global array containing two sorted subsets in a[low: mid]
and a[mid+1,high]. The goal is to merge these two sets into single set
residing in a a[low: high].
b[] is an auxiliary global array.
{
h=low;
i=low;
j=mid+1;
While ((h<=mid) and (j<=high)) do
{
if (a[h] <= a[j]) then
{
b[i]=a[h];
h=h+1;
}
else
{
b[i]=a[j];
j=j+1;
}
i=i+1;
if(h > mid) then
for k=j to high do
{
b[i]=a[k];
i=i+1;
}
else
for k=h to mid do
{
b[i]=a[k];
i+i+1;
}
for k =low to high do a[k]=b[k];
}

Ch 2 Divide and Conquer_Merge Sort Page 2

Ch 2 Divide and Conquer_Merge Sort Page 3
❖ Performance analysis of Merge Sort:
▪ For Merge Sort, the best case will be a special case where n=1.
▪ For average case and worst case, we have to perform exactly k divisions
and k calls will be given to merge.
▪ The number of comparisons required for average case will be less than the
number of comparisons required for worst case. But still both cases will have
complexity O(n log n)
▪ The drawback of MergeSort algorithm is the extra space required for
auxiliary array b and the stack space required for handling the recursion.
▪ We are going to have k divisions, which means log n and for each division
▪ We are going to generate two calls to procedure MergeSort. Thus the stack
space required is 2 log n.

Complexity Computation for Merge Sort:

Ch 2 Divide and Conquer_Merge Sort Page 4

Ch 2 Divide and Conquer_Merge Sort Page 5
Saturday, February 6, 2021 11:29 AM

Quick Sort:
▪ The divide and Conquer approach can be used to arrive at an efficient sorting
▪ Method different from merge sort.
▪ In merge sort the file a[1..n] was divided at its midpoint into subarrays which are
independently sorted and later merged.
▪ In quick sort, the division into two subarrays is made. So that the sorted
subarrays do not need to be merged later.
Algorithm Partition(a, m, p)
// a is array.
//m is low value
// p is high value
{
V=a[m]; i=m; j=p;
repeat
{
repeat
i=i+1;
until (a[i] >= V);
repeat
j=j-1;
until(a[j]<=V);

if(i< j) then Interchange (a, i, j);

}
until (i>=j);
a[m]=a[j];
a[j]=V;
return j;
}
Algorithm Interchange(a, i, j )
// Exchange a[i] with a[j]
{
p=a[i];
a[i]=a[j];
a[j]=p;
}

Ch 2 Divide and Conquer_ Quick Sort Page 1

}

Ch 2 Divide and Conquer_ Quick Sort Page 2

 Algorithm QuickSort(p, q)
{
if (p<q) then // if there are more than one element
{
//divide P into two subproblems
j=partition(a, p, q+1);
// j is the position of the partitioning element.
// Solve the subproblems.
QuickSort(p,j-1);
QuickSort(j+1,q);
// There is no need to for combining solutions.
}
}

❖ Performance analysis of Quick Sort:

Let us suppose that the pivot we just choose has divided the array into two parts.
One of size k and the other of size n-k. Notice that both these parts still need to be
sorted. This gives us the following relation.

T(n) = T(k) +T(n-k) + cn

Where T(n) refers to the time taken by the algorithm to sort n elements.
❖ Worst Case Complexity Computation
In the worst case of quick sort algorithm the pivot element will be positioned at one
of the extremities and the array will be divided into 2 subarrays of size 1 and n-1.
❖ Recurrence Relation:

T(n) = a n=1
T(n-1) +cn n>1

Ch 2 Divide and Conquer_ Quick Sort Page 3

❖ Best Case Complexity Computation:
In the best case of quick sort the pivot element is partitioned at the middle most
position in the array. The array is partitioned into 2 subarrays of equal size. If the
Size of considered array is n then there are 2 subarrays of size n/2.
Thus recurrence relation becomes

T(n)= a if n=1
2T(n/2) + cn n>1

= 2[2T (n/4) +c(n/2)] +cn

=4T(n/4)+ 2cn

=8T(n/8)+3 cn

= 2k T(n/2k)+kcn
Ch 2 Divide and Conquer_ Quick Sort Page 4
 =8T(n/8)+3 cn

= 2k T(n/2k)+kcn

= n.T(n/n)+kcn
=n.(1)+kcn
=n.a+log2 n.c.n
=n(a+c.log2 n)

T(n)= O(n log n)

The average case of time complexity is same. i.e. O(n log n)

Ch 2 Divide and Conquer_ Quick Sort Page 5

Sunday, February 14, 2021 7:38 PM

❖ Strassen's Matrix Multiplication

Let A and B be two n × n matrices. The product matrix C=AB is also an n × n matrix
whose i, jth element is formed by taking the elements in the ith row of A and jth
column of B and multiplying them we get

C(i, j ) = ∑ A(i, k) B (k, j)

for all i and j between 1 to n

To compute C(i, j) using this formula, we need n multiplications.
As the matrix C has n2 elements, the time for the resulting matrix multiplication
algorithm, which we refer to as conventional method is O(n3).

❖ Divide and Conquer:

The divide and Conquer strategy suggests another way to compute the product of
n ×n matrices. For simplicity we assume that n is a power of 2 , that is there exists
a non-negative integer k such that n=2k . In case n is not a power of two,
then enough rows and columns of zeros can be added to both A and B so that the
resulting dimensions are A and B. So that the resulting dimensions are power of two.
Ch 2 Strassen's Matrix Multiplication_1 Page 1
resulting dimensions are A and B. So that the resulting dimensions are power of two.
Imagine that A and B are each partitioned into four square submatrices, each
submatrix having dimensions as n/2 × n/2.
Then the product AB can be computed by using the above formula for the product
of 2 × 2 matrices. If dimensions are 4 × 4 or 8 × 8 we can apply divide strategy.

Ch 2 Strassen's Matrix Multiplication_1 Page 2

Ch 2 Strassen's Matrix Multiplication_1 Page 3
Monday, February 15, 2021 8:49 AM

❖ Using Strassen's Method:

Volker Strassen has discovered a way to compute the Cij using only 7
multiplication and 18 additions or subtractions. The method involves first
computing the seven n/2 × n/2 matrices. P,Q,R,S,T,U,V can be computed using 7
matrix multiplication and 10 matrix addition or subtractions.
P= (A11+A22) (B11+B22)
Q= B11(A21+A22)
R= A11(B12-B22)
S= A22(B21-B11)
T= B22 (A11+A12)
U= (A21-A11)(B11+B12)
V= (B21+B22)(A12-A22)

C11= P+S-T+V
C12=R+T
C21=Q+S
C22=P+R-Q+U

The resulting recurrence relation for T(n) is

T(n)= b n<=2
7T(n/2)+an 2 n>2
Where a and b are constants.

Ch 2 TC_Derivation_Strassens_2 Page 1
Ch 2 TC_Derivation_Strassens_2 Page 2
Wednesday, February 24, 2021 11:12 AM

P= (A11+A22) (B11+B22)
Q= B11(A21+A22)
R= A11(B12-B22) C11= P+S-T+V
S= A22(B21-B11) C12=R+T
T= B22 (A11+A12) C21=Q+S
U= (A21-A11)(B11+B12) C22=P+R-Q+U
V= (B21+B22)(A12-A22)

Ch 2 Strassens Method Example_3 Page 1

Chapter 3 Greedy Method
Sunday, February 21, 2021 3:45 AM

❖ General Method:
▪ The Greedy method is the simplest possible design strategy.
▪ It is used to solve optimization problem.
▪ The problem which requires either minimum or maximum result.
It deals with the problem that have 'n' inputs and require us to obtain a subset
that satisfies some constraints.
▪ Any Subset that satisfies these constraints is called a feasible solution.
▪ The solution which satisfies the constraints.
We need to find a feasible solution that either maximizes or minimizes a
given objective function. A feasible solution that does this is called an
optimal solution. Satisfying the objective of problem is optimal solution

▪ The greedy method suggests that one can devise an algorithm that works in
stages, considering one input at a time.
▪ At each stage decision is made regarding whether a particular input is an
optimal solution. This is done by considering one input at a time and input
order determined by some selection procedure.
▪ If the inclusion of the next input to partially constructed optimal solution
will be result in an infeasible solution, then this input is not added to the
partial solution otherwise it is added.
▪ The selection procedure itself is based on some optimization measure. This
measure may be objective function.

The general algorithm for greedy method will be as follows:

Algorithm: Greedy (a, n)
// a[1..n] contains the n inputs.
{
solution = Ø // Initialize the solution
for i=1 to n do
{
x=select (a);
if feasible(solution, x) then
solution= union(solution, x);
}
return solution;
Ch 3 Greedy Method_1 Page 1
 return solution;
}
Greedy Method Control abstraction
1. Select: The function select selects an input from a[] and remove it. The
selected input's value is assigned to x.
2. Feasible: Feasible is a Boolean-valued function that determines whether x
can be included into solution vector.
3. Union: The function union combines x with the solution and updates the
objective function.

❖ Knapsack Problem:
▪ Suppose a student is going on a trip. He has a single knapsack that can carry
items of weights at most 'm'.
▪ In addition, he is allowed to break items into fractions arbitrarily. Each item i
has some utility value say pi. He wants to fill his knapsack with items most
useful to him with total weight at most m.
▪ Let us try to apply the greedy method to solve knapsack problem.
We are given n objects and a knapsack or bag.
Object i has a weight wi and the knapsack has capacity m. If a fraction xi,
of object i is placed into the knapsack, then profit of pixi is earned.
▪ The objective is to obtain a filling of the knapsack that maximizes the total
profit earned. Since the knapsack capacity is m, we require a total weight
of all chosen objects to be at most m.

Formally the problem can be stated as

maximize ∑ pixi
Subject to
∑ wixi <= m

The profit and weights are positive numbers.

An optimal solution is a feasible solution for which pixi is maximized.
There are following three approaches to generate feasible solutions.
1. Greedy by profit
2. Greedy by weight
3. Greedy by density

1. Greedy by Profit:
At each step select from the remaining items the one with the highest
profit.(provided the capacity of the knapsack is not exceeded). This
Ch 3 Greedy Method_1 Page 2
1. Greedy by Profit:
At each step select from the remaining items the one with the highest
profit.(provided the capacity of the knapsack is not exceeded). This
approach tries to maximize the profit by choosing the most profitable
items first.
2. Greedy by Weight:
At each step select from the remaining items the one with the least
weight. (the capacity of knapsack not exceeded). This approach tries to
maximize the profit by putting as many items into the knapsack as
possible.
3. Greedy by profit density:
At each step select from the remaining items the one with the highest
or largest profit density, pi/wi the capacity of knapsack is not exceeded.
This approach tries to maximize the profit by choosing items with the
largest profit per unit of weight.

Algorithm: Greedy_knapsack(m, n)
//p[1..n] and w[1..n] contain the profits and weights respectively of the
n objects ordered such that p[i]/w[i] >= p[i+1]/ w[i+1]
m is knapsack size and x[1..n] is the solution vector.
{
for i=1 to n do
x[i]=0.0; // Initialize x
U=m;
for i=1 to n do
{
If (w[i] > U) then break;
x[i]=1.0;
U=U-w[i]; // decreasing the available capacity
After putting the object into knapsack.
}
If (i<=n) then x[i]=U/w[i];
}
Algorithm for greedy strategies for knapsack problem

▪ Greedy by Profit:
Find an optimal solution to the knapsack instance n=7, m=23

p1,p2,p3,p4,p5,p6,p7 = (15, 19, 7, 5, 2, 11, 7)

Ch 3 Greedy Method_1 Page 3
Find an optimal solution to the knapsack instance n=7, m=23

p1,p2,p3,p4,p5,p6,p7 = (15, 19, 7, 5, 2, 11, 7)

w1,w2,w3,w4,w5,w6,w7 = (5, 6, 3, 4, 1, 5, 2)

Ch 3 Greedy Method_1 Page 4

Ch 3 Greedy Method_1 Page 5
What is the optimal solution for fractional knapsack problem with 4 objects, m=11,
P=(5,11,6,13) and W=(3,5,4,8)?

Ch 3 Greedy Method_1 Page 6

Tuesday, March 2, 2021 10:19 AM

❖ Job Sequencing with Deadline:

▪ We are given a set of n job. Associated with job i is an integer deadline di >= 0
and a profit pi>0. For any job i the profit pi is earned if the job is completed by
its deadline.
▪ To complete a job, one has to process the job on a machine for one unit of time.
▪ Only one machine is available to process the jobs.
▪ A feasible solution for this problem is a subset of jobs such that each job in this
subset can be completed by its deadline.
▪ The value of a feasible solution is the sum of profits of the jobs.
▪ An optimal solution is a feasible solution with maximum value.

Example:
▪ Which is the optimal sequence in case of Job Sequencing problem with 4 jobs
where P=(20,10,30,50) and D=(2,3,1,3)

▪ Which is the optimal sequence in case of Job Sequencing problem with 4 jobs
where P=(8,6,5,4,3) and D=(2,3,4,4,1)

Ch 3 Job Sequencing With Deadline Page 1

▪ Which is the optimal sequence in case of Job Sequencing problem with 4 jobs
where P=(8,6,5,4,3) and D=(2,3,4,4,1)

Ch 3 Job Sequencing With Deadline Page 2

Sunday, March 7, 2021 4:12 PM

❖ Minimum cost Spanning Tree:

A spanning tree of connected graph G contains all the nodes and has the
edges, which connects all the nodes. So number of edges will be 1 less than
the number of nodes. Let us take a connected graph G.

In all below trees are spanning tree. The number of edges is 3 which is less
than the number of nodes.

❖ Example:
If the nodes of G represents cities and the edges represent possible
communication links connecting two cities, then the minimum number
of links needed to connect the n cities is n-1. The spanning tree of G
represents all feasible solutions.

❖ Minimum Spanning Tree:

▪ Minimum spanning tree is the spanning tree in which the sum of weights
on edges is minimum. In the above example (d) is the minimum spanning
tree with weight g.
▪ A minimum spanning tree MST is a spanning tree that has the minimum
weight than all other spanning trees of the graph.
▪ A graph can have one or more number of spanning trees.
▪ If the graph has n vertices then spanning tree will have n-1 edges.
▪ Spanning tree should not contain cycle.
▪ There are many ways for creating minimum spanning tree but the most
Ch 3 Minimum Cost Spanning Tree Page 1
 ▪ There are many ways for creating minimum spanning tree but the most
famous method are Prim's and Kruskal algorithm.

❖ Kruskal Algorithm:
▪ Kruskal's Algorithm uses greedy method.
▪ Kruskal's Algorithm is used to find the minimum spanning tree for a
connected weighted graph.
▪ The main target of the algorithm is to find the subset of edges by using
which, we can traverse every vertex of the graph.
▪ Kruskal's algorithm follows greedy approach which finds an optimum
solution at every stage instead of focusing on a global optimum.
▪ Kruskal Algorithm Steps:
1. Arrange all the edges in increasing order of weight or cost.
Construct heap out of edge costs using Heapify;
Heapify procedure which creates the heap of edges such that edge with
minimum cost is at the root.
2. Choose edge with lowest cost and delete from edges group and add it into
Spanning tree. If the edge creates cycle then that edge will be rejected.
3. Adjust: Adjust is procedure which again built the heap so that edge
with minimum cost from the remaining edge will be selected as root.
4. This procedure should be applied until the heap becomes empty.

Ch 3 Minimum Cost Spanning Tree Page 2

Ch 3 Minimum Cost Spanning Tree Page 3
Sunday, March 7, 2021 5:31 PM

❖ Kruskal Algorithm:
▪ Kruskal's Algorithm uses greedy method.
▪ Kruskal's Algorithm is used to find the minimum spanning tree for a connected
weighted graph.
▪ The main target of the algorithm is to find the subset of edges by using which,
we can traverse every vertex of the graph.
▪ Kruskal's algorithm follows greedy approach which finds an optimum solution
at every stage instead of focusing on a global optimum.
❖ Kruskal Algorithm Steps:
1. Arrange all the edges in increasing order of weight or cost.
Construct heap out of edge costs using Heapify;
Heapify procedure which creates the heap of edges such that edge with
minimum cost is at the root.
2. Adjust: Adjust is procedure which again built the heap so that edge
with minimum cost from the remaining edge will be selected as root.
3. Find: Find is the procedure which finds the root of the tree to which
Parameter node belongs.
4. Choose edge with lowest cost and delete from edges group and add it into
Spanning tree. If the edge creates cycle then that edge will be rejected.
5. This procedure should be applied until the heap becomes empty.

Ch 3 Kruskal Algorithm Page 1

Wednesday, March 10, 2021 9:43 AM

❖ Prims Algorithm:
▪ Prims algorithm is Greedy Method.
▪ Prims algorithm the minimum weighted edges associated with particular
vertices are considered one by one.
▪ The algorithm will start with a tree that includes only a minimum cost
edge of G.
▪ Then edges are added to this tree one by one .
▪ The next edge (i, j) to be added in such way that i is a vertex already
included in the tree, j is a vertex not yet included and the cost of (i,j) is
minimum among all edges.
▪ To determine this edge (i, j) efficiently we associate with each vertex j
not yet included in the tree and find j in such way that the new edge should
contains minimum weight.

We start from one vertex and keep adding edges with the lowest weight until
we reach our goal.
The steps for implementing Prim's algorithm are as follows:
Step 1: Select any connected vertices with min weight.
Step 2: Select unvisited vertex which is adjacent of visited vertices with min
weight.
Step 3: Repeat step 2 until all vertices are visited and we will get the
spanning tree.

Ch 3 Prims Algorithm Page 1

Ch 3 Prims Algorithm Page 2
Monday, March 15, 2021 7:17 PM

❖ Optimal Merge Pattern:

Two sorted files containing n and m records respectively could merge together to
obtain one sorted file in time O(n + m) when more than two sorted files are to be
merged together, the merge can be accomplished by repeatedly merging sorted files
in pairs.
Thus if files x1,x2,x3 and x4 are to be merged.
We could first merge x1 and x2 to get a file y1. Then we could merge y1 and x3 to get
y2. Finally, we could merge y2 and x4 to get desired sorted file.

Alternatively we could first merge x1 and x2 getting y1, then merge x3 and x4 and get
y2 and finally merge y1 and y2 and get the desired sorted file.

Given n sorted files, there are many ways in which to pairwise merge them into
single sorted file.
Different pairings require different amounts of computing time.
The problem we address ourselves to now is that of determining an optimal way
To pairwise merge n sorted files. In optimal merge pattern number of moves are
minimized.

Example:
The files x1, x2 and x3 are three sorted files of length 30, 20 and 10 records each.
Merging x1 and x2 requires 50 moves. Merging the result with x3 requires another 60
moves. The total number of record moves required to merge the three files this way is
110.

Ch 3 Optimal merge Pattern Page 1

moves. The total number of record moves required to merge the three files this way is
110.

If instead we first merge x2 and x3 (taking 30 moves) and then x1 (taking 60 moves)
the total record moves made is only 90. Hence, the second merge pattern is faster
than first.

▪ A greedy attempt to obtain an optimal merge pattern is easy to formulate since

merging an n-record file and an m-record file requires possibly n + m record
moves. The obvious choice for a selection criteria is at each step merge the two
smallest size files together.
▪ In two way merge pattern each merge step involves the merging of two files.
▪ The two way merge pattern can be represented by binary merge trees.
▪ The leaf nodes are drawn as squares and represent the given files.
▪ These nodes are drawn as squares and represent the given files. These nodes
are called external nodes.
▪ The remaining nodes are drawn as circles and are called internal nodes.
▪ Each internal node has exactly two children and it represents the file obtained by
merging the files represented by its two children.
▪ The number in each node is the length (i.e. number of records) of the file
represents by that node.

❖ Generation of optimal Binary Merge Tree:

In the two way merge pattern of the binary tree the external nodes of the tree
Ch 3 Optimal merge Pattern Page 2
❖ Generation of optimal Binary Merge Tree:
In the two way merge pattern of the binary tree the external nodes of the tree
represent the given files, whereas the internal nodes represents the merged files.
In this method to find the total number of record moves we use the formula
n
∑ di r i
i=1
Where di= depth of the file in tree or distance of the file from
root node.
ri= number of records in the file.

Find the optimal merge pattern for merging the files of sizes 10, 5 , 7, 20, 12

Ch 3 Optimal merge Pattern Page 3

Ch 3 Optimal merge Pattern Page 4
Ch 3 Optimal merge Pattern Page 5
Wednesday, March 17, 2021 10:02 PM

❖ Optimal Storage on Tapes:

There are n programs that are to be stored on a computer tape of length l. Associated
with each program i is a length li, 1<=i<=n
All the programs can be stored on a tape if and only if the sum of lengths of the
programs is at most l. such that

l >= ∑ li

The tape is a sequential device so every program on the tape is stored one after
the other.
If a program pi is to be retrieved from the tape, we first have to retrieve the
programs p1,p2……pi-1
Therefore the total time needed to retrieve pi = time needed to retrieve p1+
time needed to retrieve p2 + ………+ time needed to retrieve pi-1.

The arrangement of programs on the tape is expected to be done in such a way

that the total retrieval time of each program from the tape is minimum.
If we assume that all the programs are accessed only once, then the mean retrieval
time (MRT) is
n
MRT= ∑ d(pi) / n
i=1
Where d(pi) is the retrieval time of program pi .

In this method our objectives is to find minimum MRT.

Ch 3 Optimal Storage on Tapes Page 1

Ch 3 Optimal Storage on Tapes Page 2
Monday, March 22, 2021 2:17 PM

❖ Dijkstra’s shortest path algorithm:

▪ Dijkstra's algorithm is an algorithm for finding the shortest paths between
nodes in a graph, which may represent, for example, road networks.
Graphs can be used to represent the highway structure of a state or country with
vertices representing cities and edges representing sections of highway. The edges
can be then be assigned weights which may be either distance between the two cities
connected by the edge or the average time to drive along that section of highway.
Application:
Graphs are directly applicable to real-world scenarios.

Example: A motorist wishing to drive from city A to B

Is there a path from A to B?
If there are more than one path from A to B which is the smallest path?
The length of a path is now defined to be the sum of the weights of the edges
on that path.
The starting vertex is referred as Source and Last vertex is the Destination.

In the problem we consider, we are given a directed graph G=(V,E), and source
vertex V0. The problem is to determine the shortest path from V0 to all the
remaining vertices of G.
It is assumed that all the weights are positive.

The numbers on the edges are the weights. If node 1 is the source vertex, then the
Shortest path from 1 to 2 is 1, 4, 5, 2
The length of this path is 10+15+20=45
Even though there are three edges on this path, it is shorter than 1->2
which is of length 50. There is no path from 1 to 6.
Ch 3 Dijkstra’s shortest path Page 1
 which is of length 50. There is no path from 1 to 6.
The following table shows shortest paths from 1

Path Length
1) 1,4 10
2) 1,4,5 25
3) 1,4,5,2 45
4) 1,3 45
The shortest path from 1 to nodes 4, 5, 2, and 3

❖ Working of Dijkstra's Algorithm:

▪ Dijkstra's Algorithm can only work with graphs that have positive weights.
▪ Dijkstra's Algorithm basically starts at the node that you choose (the source node) and it
analyzes the graph to find the shortest path between that node and all the
other nodes in the graph.
▪ The algorithm keeps track of the currently known shortest distance from
each node to the source node and it updates these values if it finds a shorter path.
▪ Once the algorithm has found the shortest path between the source node and
another node, that node is marked as "visited" and added to the path.
▪ The process continues until all the nodes in the graph have been added to the path.
▪ This way, we have a path that connects the source node to all other nodes following
the shortest path possible to reach each node.

Ch 3 Dijkstra’s shortest path Page 2

Ch 3 Dijkstra’s shortest path Page 3
Friday, March 19, 2021 11:50 AM

❖ Expected Retrieval Time:

Let p1,p2…..pn be the n programs having lengths l1,l2,l3……..ln.
These programs are arranged on a tape . The frequencies of retrieving these programs
from the tape are respectively f1,f2,f3…..fn
The greedy method of arranging the program in ascending order of their lengths
fails in this case.
Also storing the programs in non-increasing/decreasing order of their frequencies
does not give minimum ERT.
It has been observed that the ERT is minimized if the programs are arranged on the
tape in non-increasing order of the ratio fi/li

Here, j
d(pj)= fj ∑ li
n i=1
and ERT = ∑ d(pj)/N
j=1 where N= ∑ fi

For finding ERT we can use methods:

1. Greedy by Length:
Arranging programs in ascending order of their length.
2. Greedy by frequency:
Arranging all programs in decreasing order of their frequency.
3. Greedy by Ratio:
Arranging the programs in decreasing order of the ratio fi/li

Let n=5 the programs P1 to p5 have length l=(2,3,3,6,10) and frequency of their
retrieval is f=(6,3,4,11,5)

1. Greedy by Length:

Ch 3 Expected Retrieval Time Page 1

Ch 3 Expected Retrieval Time Page 2
 '

Ch 3 Expected Retrieval Time Page 3

Wednesday, March 17, 2021 9:31 PM

Huffman Code:
Another application of binary tree with minimal weighted external path length
is to obtain an optimal set of codes for messages M1…..Mn+1 . Each code is
binary string that is used for transmission of the corresponding message. At the
receiving end the code is decoded using decode tree.
▪ Huffman coding is a lossless data compression algorithm.
▪ Compression technique
▪ Reducing the size of the data
▪ Used to store data in compresses form
▪ When data is sent over the network then data is compresses and then
transmitted to reduce the cost of transmission.

▪ A decode tree is a binary tree in which external nodes represent messages.

▪ Given n messages say M1,M2…Mn and the relative frequencies of these
messages i.e. f1,f2,f3……fn
▪ We have to construct optimal binary tree with respect to the frequencies of
the messages.
▪ Generally we assign '0' to the left branch of the tree and '1' to the right branch
of the tree. The leaf nodes of the tree are messages.
▪ There are mainly two parts. First one to create a Huffman tree, and another one
to traverse the tree to find codes.
Example: A network company uses a compression technique to encode the
message before transmitting over the network for security.
The Huffman coding method is used for encoding the messages.

Ch 3 Huffman Coding Page 1

Ch 3 Huffman Coding Page 2
Ch 3 Huffman Coding Page 3
Ch 3 Huffman Coding Page 4
Ch 3 Huffman Coding Page 5
Monday, March 22, 2021 4:09 PM

Chapter 4 Dynamic Programming

▪ The principle should be satisfied by the problem, when it is to be solved using
dynamic programming method is principle of optimality.
▪ Dynamic programming is another algorithm design strategy used in the problems
where optimization (maximization/minimization) of result is expected.
▪ It gives the optimal solution.
▪ It has optimal sub structure and uses recurrence relation to find optimal solution.
▪ In dynamic programming all feasible solutions of the problem are found and
among them optimal solution is considered.
▪ This design strategy is popular in the problems where solution to the problem
can be viewed as a result of sequence of decisions.
▪ Dynamic programming is mostly suitable for the problems where recursive
algorithm is required.
▪ In this technique the smallest subproblems are solved first, then there results
are combined to solve larger subproblems and so on till we get the solution
for the given problem.
▪ It is not faster than greedy strategy.
▪ We can use dynamic programming method when solution has optimal
structure.
▪ It divides the larger problem into subproblems. optimal substructure is problem
is divisible.
▪ Finding optimal solution to these subproblems and store the result.
Dynamic programming uses Memoization.
▪ If the problem can be broken into subproblems which are reused several times,
the problem possesses overlapping subproblems property.
❖ Properties of Dynamic Programming:
1. Optimal structure
2. Overlapping subproblems

Ch 4 Dynamic Programming Page 1

When we apply Top-Down approach of dynamic programming then it decreases
time complexity but increases space complexity.

Ch 4 Dynamic Programming Page 2

❖ Matrix Chain Multiplication:
Let M=M1×M2× …….Mn
be the product of n matrices. Each matrix Mi consists of ri columns. The
orders in which the matrices are multiplied together have a significant effect
on the total number of scalar multiplication required to evaluate M.
For example Let M1,M2,M3, and M4 be four matrices.
To find out the product M=M1×M2×M3×M4 we can parenthesize in one of the
following distinct ways.

1. (M1 (M2 (M3 M4)))

2. (M1 ((M2 M3) M4))
3. ((M1 M2) (M3 M4))
4. ((M1 (M2 M3)) M4)
5. (((M1 M2) M3) M4)

If A is a matrix of order p × q and B is another matrix of order q × r .

Then, to find the product AB the number of scalar multiplication done
p×q×r

Given a chain M1,M2 …Mn of n matrices, where for i=1,2,……n matrix Mi has
ri-1 rows and ri columns.
▪ M=M1M2M3……Mn in a such way that minimizes the number of scalar
multiplications.
▪ However dynamic programming provides an algorithm with time complexity
O(n3).

Let mij be the minimum cost of computing the product.

mij= 0 if i=j
min (mik+mk+1,j + ri-1rkrj) j>i

mik is the minimum cost of evaluating the product.

Ch 4 Matrix Chain Multiplication Page 1

 mik is the minimum cost of evaluating the product.

Recurrence Relation:
dp[i, j] = 0 if i=j
dp[i, j] = min{dp[i, k] + dp[k+1, j]} + mat[i-1]*mat[k]*mat[j]

Suppose A is of order 2×3 and B is of order 3 × 4 then the total number of

scalar multiplication are 2×3×4=24

Let M1, M2, M3, M4 are four matrices with dimensions 2X3, 3X4,4X2, 2X5.
Find the number of minimum scalar multiplications required to multiply
matrices M1M2M3M4
1. (M1 (M2 (M3 M4)))
2. (M1 ((M2 M3) M4))
3. ((M1 M2) (M3 M4))
4. ((M1 (M2 M3)) M4)
5. (((M1 M2) M3) M4)

(M1 (M2 (M3 M4)))

Ch 4 Matrix Chain Multiplication Page 2

Ch 4 Matrix Chain Multiplication Page 3
Friday, April 2, 2021 10:03 AM

Let M1, M2, M3, M4 are four matrices with dimensions 2X3, 3X4,4X2, 2X5.
Find the number of minimum scalar multiplications required to multiply matrices
M1M2M3M4

Matrix_Multiplication Page 1
Friday, March 26, 2021 10:41 AM

❖ 0/1 Knapsack Merge and Purge Method:

▪ This is another method for finding the optimal solution of 0/1 knapsack
problem using dynamic programming strategy.
▪ In the previous method (0/1 knapsack function method ) the optimal solution
is the value of fn(m) whereas in 0/1 knapsack merge and purge method the
optimal solution is the value of Sn.
▪ Every (Pj,Wj) is a state where wi is the total weight of objects included in the
Knapsack and Pj is corresponding profit.

❖ The Dominance Rule: (Purging Rule)

▪ Two pairs (pj ,wj) and (pk, wk) with property that pj<=pk and wj>=wk then
the pair (pj, wj) is discarded. The pair (pk, wk) dominates (pj, wj).
The dominated pair gets purged.
▪ While generating Si's all the pairs (p, w) with w > m are also purged.
Since the knapsack capacity is m. In this way continuing the generation of sets
Si, Si 1 and Hence Si+1 obtain the set Sn. This is the last set generated.
The optimum solution is given by the P value of the last pair in Sn.

Example: Find an optimal solution for 0/1 knapsack problem by using merge
and purge method. n=3 m=6 P=(1,2,5) and W=(2,3,4)

Ch 4 0 1 Knapsack Merge and Purge Method Page 1

Ch 4 0 1 Knapsack Merge and Purge Method Page 2
Ch 4 0 1 Knapsack Merge and Purge Method Page 3
Friday, March 26, 2021 10:16 AM

❖ 0/1 Knapsack:
▪ 0/1 Knapsack is based on the dynamic programming method.
▪ 0/1 knapsack problem is solved using function method and merge and purge
method.
A thief robbing a store finds n items, the ith item is worth Vi Rs. and weights
Wi are integers. He wants to take a valuable as load possible but he can carry at
most m kgs in his knapsack( a bag) for some integer m. what items should take?
Here thief can select or leave behind an item. He cannot take fractional amount
of an item or cannot take item more than once.
This is 0/1 knapsack problem.
We have already defined the greedy knapsack.
▪ The only variation is that xi=0 or xi=1 and not a fraction.
▪ It is a maximization problem.
▪ An optimal solution is a feasible solution for which
n
∑ pixi is maximum
i=1
❖ 0/1 knapsack with function method:
▪ In 0/1 knapsack problem items are indivisible and in fractional knapsack items
are divisible.
fn(m)= max {fn-1(m), fn-1(m-wn) + pn}
f0(m)=0;
f1(m)= P1; w1<=m
= 0; w1>m
fn(-m)=-∞;

❖ Solve the following 0/1 knapsack problem. n=3, m=14, W=(8,6,4)

P=(10,11,9). Find Maximum Profit (Use Function Method)

Ch 4 0 1 Knapsack_Function Method Page 1

f1(m)= P1; w1<=m
= 0; w1>m

0/1 knapsack problem using with n=3, m=7, W=(2,3,4) P=(11,12,15).

Find Solution Vector (Use Function Method)

Ch 4 0 1 Knapsack_Function Method Page 2

Monday, March 29, 2021 2:33 AM

❖ All pairs Shortest Path (Floyd- Warshall Algorithm)

▪ Dynamic programming approach is being followed in Floyd Warshall
Algorithm to solve all pair shortest path problem.
▪ Suppose we wish to compute the shortest path distance between every pair of
vertices in directed graph G with n vertices and m edges.
Let G=(V,E) be a directed graph with n vertices. Let cost be a cost adjacency
matrix for G such that Cost (i, i)=0 length of shortest path from i to j.
If k is intermediate vertex on this shortest path, then sub must be shortest paths
from i to k and k to j respectively.
The all pair shortest path is to determine a matrix A such that A< i, j> is the
shortest path.
Once this decision has been made we need to find two shortest path, one from i to
k and other from k to j.
Using Ak (i , j) to represent the length of shortest path from i to j going though
no vertex of index greater than k we obtain.

▪ Floyd-Warshall Algorithm is used to find all pair shortest distances using

Dynamic Programming method.
▪ Floyd Warshall’s Algorithm is used for solving all pair shortest path
problems. It means the algorithm is used for finding the shortest paths between
all pairs of vertices in a graph.
▪ Floyd Warshall’s Algorithm can be applied on directed graph.

1. Which algorithm is used to find all pair shortest distances using Dynamic
Programming ?
2. Floyd Warshall’s Algorithm is used for solving all pair shortest distances
problem.

Ch 4 All Pair Shortest Path_Floyd_Warshall Page 1

Ch 4 All Pair Shortest Path_Floyd_Warshall Page 2
Monday, March 29, 2021 3:07 AM

❖ Bellman Ford Algorithm:

▪ Bellman ford algorithm provides solution for Single source shortest path
problems.
▪ Which algorithm is used to find single source shortest path when the graph
contains negative weighted edges?
▪ Bellman ford algorithm is used to indicate whether the graph has negative
weight cycles or not.
▪ Bellman Ford Algorithm can be applied for directed and weighted graphs.

Ch 4 Bellman Ford Algorithm Page 1

Ch 4 Bellman Ford Algorithm Page 2
Friday, April 2, 2021 12:08 PM

Ch 4 Function Method Page 1

Ch 4 Function Method Page 2
Friday, April 2, 2021 12:08 PM

Ch 4 Function Method_Example Page 1

Ch 4 Function Method_Example Page 2
Tuesday, March 23, 2021 6:54 PM

❖ Longest Common Subsequence:

▪ Longest Common Subsequence (LCS) is a good example of the technique of
dynamic programming.
▪ It is one of the string matching problem.
▪ This technique is also used in file comparison. We can compare two different
versions of the same file to determine what changes have been made to the
file. This comparison is made by line by line.
Application : For DNA matching LCS with dynamic programming is used.

Definition:
Given a sequence x=<x1,x2….xm> y= <y1,y2….yn> and a sequence
z=<z1,z2…..zk>
Given two sequences X and Y, a sequence Z is a common subsequence of X
and Y if Z is a subsequence of both sequences X and Y.

a. If xi = yj then
C[i, j] = C[i-1, j-1] + 1
i.e. Upper Diagonal element + 1 and arrow " Cross Arrow"

b. If Xi Yj then
check the upper and left side of the element to be computed.
If upper element is greater than or equal to left element then new element is
upper element with arrow "↑" otherwise new element is left element is left
element and arrow is "←"
Ch 4 Longest Common Subsequence Page 1
element and arrow is "←"

Find LCS of the sequences X and Y where

X= <A,B,C,B,D,A,B>
Y= <B,D,C,A,B,A>

j 0 1 2 3 4 5 6
i Yi B D C A B A
0 Xi 0 0 0 0 0 0 0
1 A 0
2 B 0
3 C 0
4 B 0
5 D 0
6 A 0
7 B 0

Algorithm LCS_length (X, Y, m, n)

// m is the length of sequence X.
//n is the length of sequence Y.
{
m=length(X);
n=length(Y);
for i=0 to m do
Ci,0 =0;
for j=0 to n do
C0,j =0;
for i=1 to m do
for j=1 to n do
if xi=yj then
{
Cij =Ci-1,j-1 +1;
bij= " Cross arrow";
}
else
if Ci-1,j >= Ci,j-1 then
//upper element is greater than or equal to left
write upper element as it is and "↑"
Ch 4 Longest Common Subsequence Page 2
 write upper element as it is and "↑"
{
cij=Ci-1,j;
bij="↑";
else
{
Cij=Ci,j-1; //else write left element as it is & "←"
bij= "←";
}
return c and b;
}

Algorithm printLCS
Algorithm printLCS (b, x, i, j)
{
if i=0 or j=0 then
return;
if bij= "cross arrow" then
{
printLCS (b,x,i-1,j-1);
Print xi;
}
else
{
if bij="↑" then
printLCS(b, x, i-1, j);
else
printLCS(b, x, i, j-1);
}
printLCS

Find length of LCS of two sequences X=<0,1,0,1,0> and Y = <1,0,1,1,0>

j 0 1 2 3 4 5
i Yj 1 0 1 1 1
0 Xi
1 0
Ch 4 Longest Common Subsequence Page 3
 0 Xi
1 0
2 1
3 0
4 1
5 0

Length of common subsequence is:

Given two sequences X and Y , a maximum length common subsequence of X

and Y is called Longest Common Subsequence of X and Y.

Ch 4 Longest Common Subsequence Page 4

Thursday, March 25, 2021 10:47 AM

❖ String Editing:
▪ We are given two strings X= x1,x2,……xn and Y= y1,y2….ym where xi,
1<=i<=n and yj, 1<=j<=m are members of a finite set of symbols known as the
alphabet.
▪ We want to transform X into Y using a sequence of edit operations on X.
▪ The permissible edit operations are insert, delete and change and there is cost
associated with performing each.
The cost of sequence of operations is the sum of the costs of the individual
operations in the sequence.
▪ The problem of string editing is to identify minimum cost sequence of edit
operations that will transform X into Y.
▪ Let D (xi) be the cost of deleting the symbol xi from X,
▪ I (Yj) be the cost of inserting the symbol Yj into X,
▪ C (xi, yj) be the cost of changing the symbol Xi of X into Yj

Example: X be the sequence of x1,x2,x3,x4,x5=a, a, b, a, b and Y be the

sequence of y1,y2,y3,y4= b, a, b, b
The cost of each insertion and deletion operation is 1 and cost of changing any
symbol to any other symbol is 2. Find number of edit operations for
transforming string X into Y.
If i ≠ j
C D
I
Add C=+2 D=+1 I=+1
i=j Write diagonal element as it is.

j b a b b
i yj 0 1 2 3 4
xi 0
a 1
a 2
b 3
a 4
b 5
▪ The minimum cost of edit sequence is:

Ch 4 String Editing Page 1

 b 5
▪ The minimum cost of edit sequence is:

x1 x2 x3 x4 x5
a a b a b

y1,y2,y3,y4= b, a, b, b

▪ String Editing is used to identify minimum cost sequence of edit operations that
will transform X into Y.
▪ Find number of edit operations for transforming string SSAB to AAB. The cost
of each insertion and deletion operation is 1 and cost of changing any symbol to
any other symbol is 2.

X= SSAB
Y= AAB
j A A B
i yj 0 1 2 3
xi 0
S 1
S 2
A 3
B 4

X = abc
Y = abc

j A B C
i yj 0 1 2 3
xi 0
A 1
B 2
C 3
The edit distance between two strings be zero when two strings are equal.
When the lengths of the two strings are equal edit distance is not zero.
Ch 4 String Editing Page 2
When the lengths of the two strings are equal edit distance is not zero.
Both dynamic programming and recursion is can be used to solve the edit
distance problem.

Ch 4 String Editing Page 3

Saturday, March 27, 2021 8:42 AM

❖ Travelling Salesman problem:

▪ Travelling Salesman Problem is solved by using dynamic programming method.
▪ It consists of Salesman and set of cities. The salesman travels along all the cities
and cost of travelling should be minimum.
▪ The salesman has to visit each city starting from one city and returning to same
city. The main challenge to this is minimize the total distance in this trip.
We have find out we should have minimum cost/distance in this trip.
Let G=(V,E) be a directed graph on n vertices. The cost of tour is sum of cost of
the edges on the tour.

Look at the figure:

A,B,C,D are four cities. These are connected by different roads. The distance of
the road are written (in kms) in the figure. A salesman starts his tour at A and
travels along the roads in such way that he visits every other city only once and
comes back to A. The distance travelled during his tour is minimum.
We observe different tours like
A->C->D->B->A , A->B->C->D->A
out of which in the first tour distance travelled is 16 km and in that of second is 11
km minimum.
Naturally the salesperson prefers the tour with minimum distance travelled.
▪ This is minimization problem. It is the problem of finding minimum distance.

g(i, s )=min {cij + g(j, s-{j})}

Ch 4 TSP Page 1
▪ It gives shortest path traversed by salesperson and path includes every vertex.
Shortest path is always cycle.
▪ The traveling salesman problem involves visiting each city only once.

Ch 4 TSP Page 2
Saturday, March 27, 2021 9:02 AM

Chapter 5 Decrease and Conquer

▪ Reduce the problem instance to smaller instances.
▪ Solve the smaller instance.
❖ Definition of Graph Representation of Graph:
Graph is one of the most important data structure, very widely used in
computer Science. Many real problems can be represented by graph.
Graph is ordered pair of two sets (V, E) where, V is vertices and E is edges.
▪ A graph can also be represented pictorially or using matrices, adjacency
matrix or incidence matrix.

❖ Graph Traversal:
Graph traversal is systematic procedure for exploring the graph by visiting all its
Vertices. To solve the many problems dealing with the graph we need to search
and visit the vertices of the graph in a systematic fashion.
1. Depth First Search:
This is traversal technique. Depth First Search (DFS) algorithm traverses a graph
in a depthward motion and uses a stack to remember to get the next vertex to start
a search.
It employs the following rules.
Rule 1 − Visit the adjacent unvisited vertex. Mark it as visited. Display it. Push it
in a stack.
Ch 5 Decrease and Conquer Page 1
in a stack.
Rule 2 − If no adjacent vertex is found, pop up a vertex from the stack. (It will
pop up all the vertices from the stack, which do not have adjacent vertices.)
Rule 3 − Repeat Rule 1 and Rule 2 until the stack is empty.
▪ DFS uses LIFO technique. In DFS, how many times a node is visited? Once
▪ The Data structure used in standard implementation of Depth First Search is
Stack. That is use of stack when we traverse the graph.
▪ A vertex is pushed on the stack when it is visited first time and it is poped
from the stack when explored.
▪ A vertex which itself is visited and all its adjacent vertices are also visited is
said to be explored.
▪ Note:
▪ The Depth First Search traversal of a graph will result into tree.
▪ Depth first Search is equivalent to Pre-order Traversal in the binary Trees.
▪ The maximum number of edges in the tree generated by DFS from an
undirected graph with n vertices is n-1.
▪ DFS uses backtracking and visits all the vertices in the graph.
▪ Time Complexity of DFS is? (V – number of vertices, E – number of edges)
O(V+E)
▪ The output generated by BFT or DFT of a directed graph can be either tree or
forest. (Forest is a collection of disjoint trees.)
◊ Applications Of DFS
1. Topological Sorting
2. Finding Strongly Connected components
3. Finding Articulation Point and Bridge edge

Ch 5 Decrease and Conquer Page 2

Ch 5 Decrease and Conquer Page 3
Ch 5 Decrease and Conquer Page 4
Ch 5 Decrease and Conquer Page 5
Monday, March 29, 2021 6:05 AM

❖ Breadth First Search:

▪ In breadth first search we start at vertex V and mark it as having been reached
(visited). The vertex V is at this time said to be unexplored. A vertex is said to
have been explored by an algorithm when the algorithm has visited all
vertices adjacent from it. All unvisited vertices adjacent from V are visited next.
Exploration continues until no unexplored vertex is left. The list of unexplored
vertices operates as queue.
▪ BFS is a graph search algorithm that begins at root node and explores all the
neighboring vertices. Then for each of those nearest nodes it explores their
unexplored neighbor nodes.
▪ The Data structure QUEUE used in standard implementation of Breadth First
Search.
▪ In BFS, how many times a node is visited? Once
▪ Breadth first Search is equivalent to Level-order Traversal in the binary Trees.
Application of Breadth First Search
a) Finding shortest path between two nodes or shortest path from
Source to every vertex in the graph.
b) GPS Navigation systems: Breadth First Search is used to find all
neighboring locations.
c) Peer to Peer Networks. In Peer to Peer Networks, Breadth First Search
is used to find all neighbor nodes. (DFS is not used in peer to peer
network).
▪ Finding shortest distance between every pair vertices is not an application of
BFS.
▪ The tree generated by BFS traversal from a source node A in an unweighted,
connected, undirected graph is used to find the shortest path from S to every
vertex in the graph.
▪ Given two vertices in graph P, Q. Which of the following traversal is used to find
path from P to Q?
▪ We can use both BFS and DFS traversal method.
▪ The output generated by Breadth First Traversal or Depth First Traversal of a
directed graph can be either tree or forest. (Forest is a collection of disjoint
trees.)

Ch 5 BFS_Theory Page 1
Sunday, April 4, 2021 1:26 PM

Ch 5 BFS_Example Page 1
Ch 5 BFS_Example Page 2
Monday, March 29, 2021 9:41 AM

❖ Topological Sorting:
▪ A topological sort or topological ordering of a directed acyclic graph G=(V,E)
is a linear ordering of its vertices such that for every edge <u, v>, u comes
before v in the linear ordering.
▪ Graph should be directed acyclic graph.(DAG)
▪ For instance the vertices of the graph may represent tasks to be performed and
the edges may represents constraints that one task must be performed before
another.
▪ The topological sorting is a valid sequence for the tasks.
▪ A topological ordering is possible if and if the graph has no directed cycles, that
is, if it is directed acyclic graph.(DAG)

❖ Algorithm for topological sort:

a. Call DFS(V,E) to compute finishing time for each vertex V.
b. When each vertex is finished, insert into front of a linked list.
c. Return linked list of vertices.

a. Topological sort can be applied to which of the following graphs?

Directed Acyclic Graphs (DAG)
b. Topological sort can be implemented by Using both Depth and Breadth
First Search.

Ch 5 Toplogical Sorting_Theory_4 Page 1

Ch 5 Toplogical Sorting_Theory_4 Page 2
Sunday, April 4, 2021 1:46 PM

❖ Algorithm for topological sort:

a. Call DFS(V,E) to compute finishing time for each vertex V.
b. When each vertex is finished, insert into front of a linked list.
c. Return linked list of vertices.

Topological sort can be implemented by Using both Depth and Breadth First
Search.

Ch 5 Topological Sorting_Example Page 1

Monday, March 29, 2021 12:12 PM

❖ Connected components and spanning trees

Strongly Connected Component:
▪ A directed graph is called strongly connected if there is a path in each direction
between each pair of vertices of the graph. That is, a path exists from the first
vertex in the pair to the second, and another path exists from the second vertex to
the first.
Or
▪ A graph having a path from each vertex to every other vertex is called as
Strongly Connected Component.

▪ In the mathematical theory of directed graphs, a graph is said to be strongly

connected if every vertex is reachable from every other vertex.
▪ A directed graph is acyclic if and only if it has no strongly connected
subgraphs because a cycle is strongly connected and every strongly
connected graph contains at least one cycle.
 A graph having a path from each vertex to every other vertex is called as
Strongly Connected Component.
 To find Strongly connected components Depth First Search is used.
 What should be type of graph to find Strongly connected Components?
Directed Graph
 Directed graphs are required for finding Strongly Connected Components.
 Strongly Connected Component of a directed graph G=(V,E) is a maximal set
of vertices S such that for every pair of vertices u and v in S, both vertices u and
v are reachable from each other.

▪ Find strongly connected components for below graph.

Ch 5 Strongly Connected Component Page 1

Ch 5 Strongly Connected Component Page 2
Ch 5 Strongly Connected Component Page 3
Tuesday, April 6, 2021 9:54 AM

❖ By Variable Size decrease Euclid’s algorithm

GCD of two numbers is the largest number that divides both of them. A simple
way to find GCD is to factorize both numbers and multiply common prime factors.

Ch 5 Euclid’s algorithm Page 1

Tuesday, March 30, 2021 2:24 PM

❖ Articulation Point and Bridge edge:

▪ Articulation Point:
A vertex 'V' in an undirected connected graph G is an articulation point iff
deletion of vertex V along with the edges incident to it disconnects the graph into 2
or more non-empty components.

▪ Bridge Edge:
An edge of graph G is a bridge edge if its deletion from graph disconnects the
Graph into 2 or more non-empty components.

Ch 5 Articulation Point and Bridge edge Page 1

Ch 5 Articulation Point and Bridge edge Page 2
Tuesday, March 30, 2021 3:15 PM

Chapter 6 Backtracking
❖ Introduction:
▪ Backtracking is design strategy/ approach used to solve puzzles or
problems include such as eight queens puzzle, four queens puzzle, Sudoku.
▪ Backtracking can be defined as a general algorithmic technique that considers
searching every possible combination in order to solve a computational problem.
▪ It removes the solutions that doesn't give rise to the solution of the problem based on
the constraints given to solve the problem.
▪ Backtracking is not used for optimization.
▪ It is used for finding multiple solutions.
▪ Useful technique for optimizing search under some constraints.
▪ Express the desired solution as an n-tuple (x1, . . . , xn) where each xi ∈ Si , Si being
a finite set .
▪ The solution is based on finding one or more vectors that maximize, minimize, or
satisfy a criterion function P(x1, . . . , xn)
▪ Many of the problems we solve using backtracking require that all the solutions
satisfy the set of constraints.

▪ Backtrack approach
▪ Requires less than m trials to determine the solution.
▪ Form a solution (partial vector) one component at a time, and check at every step
if this has any chance of success.
▪ If the solution at any point seems not-promising, ignore it.
If the partial vector (x1, x2, . . . , xi) does not yield an optimal solution, ignore mi+
1 · · · mn possible test vectors even without looking at them.

For any problem these constraints can be divided into two categories.

1. Explicit Constraint
These are the rules which restrict each xi to take value from a given set.
▪ Si = {1, 2, 3, 4, 5, 6, 7, 8}, 1 ≤ i ≤ 8
▪ Xi=0 or xi=1 or Si= {0,1}
▪ Xi>=0 or Si={all nonnegative real numbers}

2. Implicit Constraint
These are rules that determine which of the tuples in the solution space of I satisfy
the criterion function.

Ch 6 Backtracking Page 1
❖ n-queen Problem:
In n queens problem there is n × n chessboard and we have to place n queen on
n × n chessboard so that no two queens should attack. That is no two queens should
be on the same row, same column and same diagonal. This is generalized problem.
This is generalized problem to understand this we consider 4 queen problem. In
three directions the queens can attack each other.

❖ 4 - queen Problem:
Given a 4×4 chessboard and we have to arrange 4 queens in such way that
No two queens attack each other. No two queens should be on the same row and
column and on the same diagonal.

▪ Explicit Constraint:
These are the rules which restrict each xi to take value from a given set.
▪ Si = {1, 2, 3, 4} or 1 ≤ i ≤ 4

▪ Implicit Constraint:
a. No two queens should be on the same row.
b. No two queens should be on the same column.
c. No two queens should be on the same diagonal.

▪ This backtracking algorithm is implemented by constructing a tree of choices called

as State-space tree.
▪ The problem of placing n queens in a chessboard such that no two queens attack
each other is called as n queen problem.
▪ Find correct solutions for 4-queens problem.

Ch 6 Backtracking Page 2
Saturday, April 3, 2021 3:57 PM

Ch 6 4 queen problem Page 1

Thursday, April 8, 2021 7:53 AM

❖ 8 queen Problem:
You are given an 8x8 chessboard, find a way to place 8 queens such that no queen can attack
any other queen on the chessboard. A queen can only be attacked if it lies on the same row, or
same column, or the same diagonal of any other queen. Print all the possible configurations.
▪ Explicit constraints for 8 queen's problem are:
All solutions to the 8-queens problem can be represented as an 8-tuple where queen i is on
column . The explicit constraints are the solution space consists of 8-tuples.
The implicit constraints are that no two 's can be the same (as queens must be on different
columns) and no two queens can be on the same diagonal.

All solutions to 8-queens problem can now be represented as 8-tuples (x1, x2,...,x8 ) where xi
is the column on which queen i is placed.
The first of these constraints implies that all solutions are permutations of 8-tuple (1, 2, 3, 4, 5,
6, 7, 8).
This realization reduces the size of the problem space to 8! tuples.

▪ Identify explicit constraints:

Explicit constraints using 8-tuple formulation are Si = {1, 2, 3, 4, 5, 6, 7, 8}, 1 ≤ i ≤ 8
We can arrange 8 queens on 8 × 8 chessboard 8^8 8 tuples.
▪ Implicit constraints for 8 queen's:
All solutions are permutations of the 8-tuple (1, 2, 3, 4, 5, 6, 7, 8) reduced to 8! Ways.
No two queens can be on the same diagonal, column and same row.

Ch 6 8 queen Problem Page 1

Thursday, April 8, 2021 2:23 PM

❖ Graph Coloring Problem:

▪ Let G=(V,E) be a graph and 'm' be the given positive integer. We want to discover
whether nodes of G can be colored in a such way that no two adjacent nodes have the
same color yet only m colors are used. This is termed the m-colorability decision
problem.
▪ If d is degree of given graph, then it can be colored with d+1 colors.
▪ The m- colorablity optimization problem asks for the smallest integer m for which the
graph G can be colored. This integer is referred as chromatic number of graph.
▪ The below graph can be colored with three colors 1, 2 and 3.

▪ The color of each node is indicated next to it. It can also been seen that three colors are
needed to color this graph. Hence graph chromatic number is 3.
Minimum number of unique colors required for vertex coloring of a graph is
called?

▪ vertex coloring of a graph:

A condition where any two vertices having a common edge should not have same
color.
FD

For the following graph find out all possible solutions with m=3.

Ch 6 Graph Coloring Problem Page 1

Ch 6 Graph Coloring Problem Page 2
Thursday, April 8, 2021 2:53 PM

Ch 6 Graph_Colouring_Example Page 1
Thursday, April 8, 2021 7:38 AM

❖ Hamiltonian Cycles:
▪ Hamiltonian circuit problem is solved by using backtracking.
▪ Let G =(V,E) be a connected graph with n vertices. A Hamiltonian cycle
(Suggested by Sir William Hamilton) is a round-trip path along n edges of G
That visits every vertex once and returns to its starting position.
▪ A Hamiltonian cycle is a closed loop on a graph where every node (vertex) is
visited exactly once. A loop is just an edge that joins a node to itself; so a
Hamiltonian cycle is a path traveling from a point back to itself, visiting every
node on route.
▪ Hamiltonian Path in an undirected graph is a path that visits each vertex exactly
once. A Hamiltonian cycle (or Hamiltonian circuit) is a Hamiltonian Path
such that there is an edge (in the graph) from the last vertex to the first
vertex of the Hamiltonian Path. Determine whether a given graph contains
Hamiltonian Cycle or not.
▪ It is used in various fields such as Computer Graphics, electronic circuit design.
▪ Live node is a generated node for which all of the children have not been
generated yet.

Ch 6 Hamiltonian Cycles Page 1

Ch 6 Hamiltonian Cycles Page 2
Wednesday, March 31, 2021 4:18 PM

❖ Sum of subset Problem:

Suppose we are given n distinct positive numbers w1,w2, ….. wn usually called as
weights and an integer number m.
We have to find out all subsets of W ={w1,w2,….wn} such that the sum of subset of
elements is equal to m or desire to find all the combinations of these numbers whose
sums are m.
This is called sum of subset problem.
The sum of subset problem is used for checking for the presence of a subset that has
sum of elements equal to a given number m and printing true or false based on the
result.

X Y Z

Where x=sum of weights included

y=index of next object considered.
z=total remaining weight to be considered.

Procedure for generating state space tree:

Initially for root node of the state space tree.
x=0 No weight included yet
y=1 1st object weight to be considered.
z= ∑ wi Remaining weight is equal to total weight since none of the
weight included

The node indicating inclusion of next object will have

x=x + wi, y=i +1 and z= z - wi

A solution node is said to obtained if x=m ( whatever may be the value of y and z)

The branch is discarded if any one of the following is true.

x+wi+1 > m
y+1 > n
x + z <m

Ch 6 Sum of Subset Problem Page 1

Ch 6 Sum of Subset Problem Page 2
Friday, April 9, 2021 2:35 PM

Chapter 7 Branch and Bound

 Introduction:
▪ Branch-and-Bound is used for solving Optimization Problem.
▪ We convert maximization problem into minimization problem.
▪ Branch and Bound is a Problem Solving technique.
▪ In backtracking, the state space tree is searched until the solution is obtained.
▪ In Branch-and-Bound as the optimum solution may be present anywhere in the
state space tree, so the tree need to be searched completely.
▪ Branch-and-Bound is less efficient.
▪ It is also used to solve greedy knapsack problem for minimizing the loss.
▪ In branch and bound the state space tree can be traversed in both ways by using
DFS as well as BFS.
▪ In branch and bound, after generating tree representing a feasible solutions,
bounding functions are computed for each node.
▪ The nodes which are capable of giving the solutions are explored and the nodes
which are not capable of giving the solutions are discarded.
There are two bounding functions associated with every node.
c^(x)= Lower bound on x
u(x)= Upper bound on x

 Difference between Backtracking and Branch and Bound:

▪ Backtracking is used to find all possible solutions available to a problem. When it
realizes that it has made a bad choice, it undoes the last choice by backing it up. It
searches the state space tree until it has found a solution for the problem.
▪ Branch-and-Bound is used to solve optimization problems. It completely searches
the state space tree to get optimal solution.
▪ Backtracking traverses the state space tree by DFS (Depth First Search) manner.
▪ Branch-and-Bound traverse the tree in any manner, DFS or BFS.

 How the solution is generated?

▪ For generating state space tree we can use fixed tuple size formulation or variable
tuple size formulation.
▪ The nodes can be visited in BFS like manner or DFS like manner.
▪ BFS is also known as FIFOBB(First In First Out branch and bound) and
DFS is known as Last in first out branch and bound. They don’t give any preference
to the node which has highest probability of giving optimal solution.
▪ To overcome that drawback new search method is defined which is known as
LCBB (Least Cost branch and Bound)
 Types of Searching:
Ch 7 Branch and Bound_Lecture Page 1
 ▪ To overcome that drawback new search method is defined which is known as
LCBB (Least Cost branch and Bound)
 Types of Searching:
1. LIFOBB:
In data structure such as stack we use the concept of last in first out(LIFO).
LIFOBB uses depth first search(DFS).
2. FIFOBB:
In data structure such as queue we use the concept of first in first out(FIFO).
FIFOBB uses breadth first search (BFS).

 Basic Terminologies used in Branch and Bound:

1. Live node: The node which has been generated but none of its children nodes
have been generated in state space tree is called Live node.
2. Dead node: The node that is not to be expanded or explored any further. All
children of a dead node have already been expanded is called dead node.
3. E-node: The node whose children are currently being explored is called E-
node.

 Least Cost search (LC search):

▪ This search is preferable over FIFOBB and LIFOBB.
▪ The method for solving a problem by LCBB is same as FIFOBB.
The only difference is the way for poping the node from the queue.
▪ In FIFOBB we sequentially remove the nodes from the queue whereas in LCBB
the node which has minimum value of c^ is considered for been exploring
further. This makes mechanism faster.
▪ It is faster than FIFOBB and LIFOBB since it concentrates on that path which has
maximum probability of reaching the optimal solution.
▪ We can use fixed tuple size formulation or variable tuple size formulation to
generate SST.
 0/1 Knapsack problem using Branch and Bound:
▪ In 0/1 knapsack problem we are given 'n' objects their weights and profit associated
with their inclusion.
▪ We have to include subset of object in such way that total profit obtained will be
maximized.
▪ Branch and bound is not capable of maximizing the objective function to minimize
the minus profit.
▪ In knapsack problem the objective function is ∑ pi xi which we replace by
-∑pi xi.
▪ Clearly ∑pi xi is maximized iff -∑pi xi is minimized.
▪ This modified knapsack problem can be stated as
n
Minimize -∑ pi xi (Ranking Function)
i=1
Ch 7 Branch and Bound_Lecture Page 2
 n
Minimize -∑ pi xi (Ranking Function)
i=1
n
Subject to ∑ wi xi <=m
i=1
 Steps for 0/1 knapsack problem:
Step 1: Construct SST (either VTS or FTS ) which gives all feasible solutions of
the problem.
Consider an empty queue initially.
Step 2: Calculate c^(1) and u(1). Also make initialization u bar = u(1).
Push all the children of root node (i.e. node 1) in the queue.
Step 3: While queue is not empty do
x=delete ( q ) i.e.
Pop x from queue and calculate C^(x).
 If C^(x) > U bar, kill the node x
 Compute u(x)
If u(x) <= u bar set u bar =u(x)
Step 4: Repeat above process until queue becomes empty.
Step 5: The node x for which u bar is lastly updated is the solution node.
Step 6: Represent solution vector and optimal solution.

Example:
Solve the following instance of 0/1 knapsack using FIFOBB with variable tuple
size formulation method.

Ch 7 Branch and Bound_Lecture Page 3

Ch 7 Branch and Bound_Lecture Page 4
Ch 7 Branch and Bound_Lecture Page 5
Ch 7 Branch and Bound_Lecture Page 6
Saturday, April 10, 2021 6:35 AM

 0/1 Knapsack problem using Branch and Bound:

▪ In 0/1 knapsack problem we are given 'n' objects their weights and profit associated
with their inclusion.
▪ We have to include subset of object in such way that total profit obtained will be
maximized.
▪ Branch and bound is not capable of maximizing the objective function to minimize
the minus profit.
▪ In knapsack problem the objective function is ∑ pi xi which we replace by
-∑pi xi.
▪ Clearly ∑pi xi is maximized iff -∑pi xi is minimized.
▪ This modified knapsack problem can be stated as
n
Minimize -∑ pi xi (Ranking Function)
i=1
n
Subject to ∑ wi xi <=m
i=1
 Steps for 0/1 knapsack problem:
Step 1: Construct SST (either VTS or FTS ) which gives all feasible solutions of
the problem.
Consider an empty queue initially.
Step 2: Calculate c^(1) and u(1). Also make initialization u bar = u(1).
Push all the children of root node (i.e. node 1) in the queue.
Step 3: While queue is not empty do
x=delete ( q ) i.e.
Pop x from queue and calculate C^(x).
 If C^(x) > U bar, kill the node x
 Compute u(x)
If u(x) <= u bar set u bar =u(x)
Step 4: Repeat above process until queue becomes empty.
▪ If node is not killed then from all nodes select such node which is having
minimum C^ value and expand that node further.
▪ Insert its child nodes in the queue.
▪ Again calculate C^ value for inserted child nodes.
▪ From remaining all nodes (including inserted child nodes also) select such node
for expansion having minimum C^ value.
Repeat this process until queue becomes empty.
Step 5: The node x for which u bar is lastly updated is the solution node.
Step 6: Represent solution vector and optimal solution.
Ch 7 0-1 Knapsack Using Branch and Bound LCBB Meth Page 1
 Step 6: Represent solution vector and optimal solution.

Example:
Solve the following instance of 0/1 knapsack using LCBB with variable tuple
size formulation method.

Ch 7 0-1 Knapsack Using Branch and Bound LCBB Meth Page 2

Ch 7 0-1 Knapsack Using Branch and Bound LCBB Meth Page 3
Live node is a generated node for which all of the children have not been generated yet.
E-node is a live node whose children are currently being generated or explored.
Bounding functions are used to kill live nodes to make them dead nodes.

Ch 7 0-1 Knapsack Using Branch and Bound LCBB Meth Page 4

Saturday, April 10, 2021 7:17 AM

 Traveling Salesman problem Using Variable tuple Formulation using LCBB

Here we apply branch and bound technique to solve TSP to find minimum cost tour
for given weighted directed graph G=(V,E)
we construct the cost matrix C[i, j] where each entry in matrix represents the weight
of the edge between ith and jth vertex.

Cij= C[i, j] = weight (edge(i, j)) ; if ( i, j ) ∈ E

= ∞ ; if (i, j) ∉ E

To obtain the value of c^(x) we have to reduce the cost matrix.

▪ A cost matrix is said to be reduced when we subtract minimum element from
every row and every column of the matrix.
▪ We always assume that every tour in the TSP started and ends at vertex 1.
▪ Each vertex is visited exactly once.

 Steps:
▪ Let A be the reduced cost matrix for node R
 Perform row reduction.
 Perform column reduction.
 Make all the entries in ith row and jth column of matrix A as ∞.
 Set A(j,1)=∞
▪ If r is the total amount subtracted
C^(s)= C^ (R ) +A(i, j) + r

Solve the given instance of TSP using branch and bound technique.
∞ 20 30 10 11
15 ∞ 16 4 2
3 5 ∞ 2 4
19 6 18 ∞ 3
16 4 7 16 ∞

Ch 7 TSP using Variable Tuple LCBB Page 1

Ch 7 TSP using Variable Tuple LCBB Page 2
Ch 7 TSP using Variable Tuple LCBB Page 3
Ch 7 TSP using Variable Tuple LCBB Page 4
Ch 7 TSP using Variable Tuple LCBB Page 5
Monday, January 3, 2022 5:10 PM

 Least Cost search (LC search):

▪ This search is preferable over FIFOBB and LIFOBB.
▪ The method for solving a problem by LCBB is same as FIFOBB.
The only difference is the way for poping the node from the queue.
▪ In FIFOBB we sequentially remove the nodes from the queue whereas in LCBB
the node which has minimum value of c^ is considered for been exploring further.
This makes mechanism faster.
▪ It is faster than FIFOBB and LIFOBB since it concentrates on that path which has
maximum probability of reaching the optimal solution.
▪ We can use fixed tuple size formulation or variable tuple size formulation to generate
SST.

Solve the following instance of 0/1 knapsack using LCBB with variable tuple size
formulation method.

LCBB_Lecture Page 1
LCBB_Lecture Page 2
LCBB_Lecture Page 3
Wednesday, January 5, 2022 11:17 AM

LIFOBB Page 1
LIFOBB Page 2
Wednesday, April 14, 2021 8:45 AM

Chapter 8 Problem Classification

❖ Computational Problem:
In theoretical computer science, a computational problem is a mathematical object
representing a question that computers might want to solve.
e.g. given any number x, determine whether x is prime or not is a computational problem.
❖ Computational Complexity theory:
Computational complexity theory is the branch of theory of computation that studies
the cost of computation required to solve given computational problem.
This cost is measured in terms of time and space.
Time represents the number of steps taken to solve a problem and space represents
how much memory program takes.
❖ Decision Problem:
A computational problem which only returns a yes or no answer is called a decision
problem.
e.g. given an integer n, determine whether n is prime and given two numbers x and y
determine whether x evenly divides y.
❖ Optimization Problem:
Any problem that involves the identification of an optimal (either maximum or minimum)
value of given cost function is known as optimization problem.

❖ Polynomial and Non Polynomial Algorithm:

▪ The decision problem that can be solved in short time (polynomial) are called
Polynomial algorithm.
e.g. Search O(log n), Sort, String editing
▪ The decision problem that require vast amounts of time (non-polynomial) because
of the numerous possibilities are called as non-polynomial algorithm.
e.g. TSP and Knapsack Problem.

❖ Classification of Polynomial time algorithm:

Polynomial time algorithms are further classified into two types:

1. Deterministic Algorithm:
Algorithms in which the result of every operation is uniquely defined are called
deterministic algorithm.
The group of deterministic polynomial time algorithm is denoted by P.
Thus P is set of all decision problems solvable by deterministic algorithms in polynomial
Chapter 8 Problem Classification Page 1
 Thus P is set of all decision problems solvable by deterministic algorithms in polynomial
time.
e.g. finding number whether it is odd or even., Sorting, searching, MST
▪ A deterministic algorithm is (essentially) one that always computes the correct answer
2. Nondeterministic algorithm: (non-deterministic Polynomial)
A non-deterministic algorithm contains operations whose outcomes are not uniquely
defined but are limited to certain specific sets of possibilities. The machine executing such
operations is allowed to choose any one of these outcomes subject to a termination condition.
NP: the class of decision problems that are solvable in polynomial time on a
nondeterministic machine (or with a nondeterministic algorithm) – (A deterministic
computer is what we know) – A nondeterministic computer is one that can “guess” the right
answer or solution.
Example: Satisfiability (SAT) • the problem of deciding whether a given Boolean formula is
satisfiable.

These algorithms are specified by using three functions.

1) Choice(S)
2) Failure ( )
3) Success()
▪ Choice(S): The function 'Choice' chooses arbitrarily one of the elements of set S.
▪ Failure: The function failure signals an unsuccessful completion and success
Signals successful completion.

A non-deterministic algorithm terminates unsuccessfully if and only if there exist no set of

choices leading to a successful signal or successful completion.

Non deterministic algorithm to search an element x in an array A[1..n] of elements where

n>=1

Algorithm Search (A, n, x)

j= Choice(1,n);
If (A[j] = x) then
Success();
Failure();
search algorithm is written non-deterministically its time complexity is O(1)
sorting algorithm is written non-deterministically its time complexity is O(n)

Chapter 8 Problem Classification Page 2

Saturday, April 17, 2021 7:45 AM

❖ Decidable Problem:
A problem is said to be Decidable if we can always construct a corresponding algorithm
that can answer the problem correctly. Suppose we are asked to compute all the prime
numbers in the range of 1000 to 2000. To find the solution of this problem, we can easily
devise an algorithm that can enumerate all the prime numbers in this range.
❖ The Halting problem –
Halting problem is example of undecidable problem.
Given a program/algorithm will ever halt or not? Can we design machine or algorithm
which tells about the given program will always halt or not on particular input?
Halting means that the program on certain input will accept it and halt or reject
and it would never go into an infinite loop.
Basically halting means terminating. So can we have an algorithm that will tell that the
given program will halt or not.
In terms of Turing machine, will it terminate when run on some machine with some
particular given input string.
The answer is no we cannot design a generalized algorithm which can appropriately say
that given a program will ever halt or not? Not NP complete
The only way is to run the program and check whether it halts or not.
We can refrain the halting problem question in such a way also:

Given a program written in some programming language(c/c++/java) will it ever get into
an infinite loop (loop never stops) or will it always terminate(halt)?

This is an undecidable problem because we cannot have an algorithm which will tell us
whether a given program will halt or not in a generalized way i.e. by having specific
program/algorithm.
▪ In general we can’t always know that’s why we can’t have a general algorithm.
▪ The best possible way is to run the program and see whether it halts or not.
▪ In this way for many programs we can see that it will sometimes loop and always halt.
▪ The problems that cannot be solved by any algorithm are called Undecidable problem.
In computability theory, an undecidable problem is a type of computational problem that
requires a yes/no answer, but where there cannot possibly be any computer program that
always gives the correct answer; that is, any possible program would sometimes give the
wrong answer or run forever without giving any answer.
Halting Problem is not NP complete.
A halting problem is not an NP class problem. An NP problem can be solved in a finite
amount of time, though this time period may exceed the age of the universe for
sufficiently large inputs.
The halting problem is to determine whether a give piece of code will stop executing at
some point or keep running indefinitely.

Ch 8 Halting Problem Page 1

Ch 8 Halting Problem Page 2
Wednesday, April 14, 2021 11:51 AM

❖ Nondeterministic Polynomial:
▪ Non deterministic polynomial algorithm (NP) is divided into NP hard and NP complete.
▪ NP class is the class of decision problems that can be solved by non-deterministic
polynomial algorithms.
Explanation: NP problems are called as non-deterministic polynomial problems. They are a
class of decision problems that can be solved using NP algorithms.
Problems which can be solved using polynomial time algorithms are called as tractable
(easy).
Problems that can be solved using super polynomial time algorithms are called intractable
(hard)
1. NP Hard:
Non deterministic polynomial time hard in computational theory is a class of problems that
are informally at least as hard as the hardest problems in NP.
e.g. of NP-hard problem is the decision sum of subset problem.

2. NP Complete: NP-complete problem, any of a class of computational problems for which

no efficient solution algorithm has been found. Many significant computer-science
problems belong to this class—e.g. graph-covering problems.
This problem require vast amount of time to execute.
NP complete problems are defined as the hardest problems in NP.
Let S be a set of algorithms. An algorithm A is said to be S-hard if A is harder than all
members of S .
An algorithm A is said to be C-complete if it is C -hard and also a member of C. So A is
C-Complete imply A is hardest problem in C.

Example : Calculating chromatic number of graph is NP complete Problem.

A problem L1 reduces to problem L2 written as L1 L2 iff there exists a way to solve L1 by a

deterministic polynomial time algorithms using deterministic algorithm that solves L2 in
Polynomial time.
A problem L is NP hard if and only if reduces to L.

If we have non deterministic algorithm then it is NP complete.

Cook's Theorem:

Satisfiability problem is in P if and only if P=NP

Ch 8 NP Complete and Hard Page 1

Sunday, April 18, 2021 4:25 PM

❖ CNF Satisfiability : Conjunctive Normal Form

Propositional calculus formula using Boolean variables. Clauses are formed using
disjunction and they are joined with conjunction.
Let x1,x2…. denotes Boolean variables (their value is either true or false )
Let xi bar denotes the negation of xi. A literal is either a variable or its negation.
A formula in the propositional calculus is an expression that can be constructed
using literals and the operations (and & or).
e.g.
CNF=

The satisfiability problem is to determine or to find for what values of x i this formula
is true.

e.g. of such formulas are ( x1 ^ x2 ) V (x3 ^ x4) and (x3 v x4) (x1 v x2).

The symbol V denotes or and ^ denotes and.

Algorithm Eval (E, n)

{
for i=1 to n do
Xi= choice (true, false);
If € (x1,x2 ……xn) then
Success();
else
Failure( ); }
Xi possible values are
8 values 23
for n variables 2n exponential time
Ch 8 Satisfiability Page 1
 Failure( ); }
Xi possible values are
8 values 23
for n variables 2n exponential time
X1 X2 X3
0 0 0
0 0 1
0 1 0
0 1 1
1 0 0
1 0 1
1 1 0
1 1 1

Ch 8 Satisfiability Page 2