1

Introduction to Algorithms and Growth of

Functions

TOPICS TO BE COVERED:

● Introduction

● Fundamentals of Analysis of Algorithms,

● Analysis Framework, Asymptotic notations, Standard notations and

common functions,

● Important problem types – Searching, sorting, string processing, graph

problems, combinatorial problems, Recurrences, Mathematical

Analysis of Recursive and Non Recursive Algorithms

Teena A James, Dept. of CSE, NHCE

(I) ALGORITHM:

An algorithm is finite set of instructions that is followed, accomplishes a particular task. In

addition, all algorithms must satisfy the following criteria:

1. Input. Zero or more quantities are externally supplied.

2. Output. At least one quantity is produced.

3. Definiteness. Each instruction is clear and produced.

4. Finiteness. If we trace out the instruction of an algorithm,
then for all cases, the algorithm terminates after a finite
number of steps.
5. Effectiveness. Every instruction must be very basic so that it
can be carried out, in principal, by a person using only pencil
and paper. It is not enough that each operation be definite as in
criterion 3; it also must be feasible.

An algorithm is composed of a finite set of steps, each of which may require one or more
operations. The possibility of a computer carrying out these operations necessitates that certain
constraints be placed on the type of operations an algorithm can include.

The fourth criterion for algorithms is that they terminate after a finite number of
operations.
The fifth criterion requires that, each operation be effective; each step must be such
that it can, at least in principal, be done by a person using pencil and paper in a finite
amount of time. Performing arithmetic on integers is an example of effective
operation, but arithmetic with real numbers is not, since some values may be
expressible only by infinitely long decimal expansion. Adding two such numbers
would violet the effectiveness property.

• Algorithms that are definite and effective are also called computational procedures.

• The same algorithm can be represented in several ways.

• Several algorithms to solve the same problem

• Different ideas different speed

Example: Problem: GCD of Two numbers m, n

Input specification: Two inputs, nonnegative, not both zero

Euclids algorithm
-gcd(m,n)=gcd(n,m mod n)

Teena A James, Dept. of CSE, NHCE

Untill m mod n =0,since gcd(m,0) =m

Another way of representation of the same algorithm

Euclids algorithm

Step1:if n=0 return val of m & stop else proceed step 2

Step 2:Divide m by n & assign the value of remainder to r

Step 3:Assign the value of n to m,r to n,Go to step1.

Another algorithm to solve the same problem

Euclids algorithm

Step1:Assign the value of min(m,n) to t

Step 2:Divide m by t.if remainder is 0,go to step3 else goto step4

Step 3: Divide n by t.if the remainder is 0,return the value of t as the

answer and stop,otherwise proceed to step4

Step4 :Decrease the value of t by 1. go to step 2

(II) FUNDAMENTAL OF ALGORITHM ANALYSIS

Teena A James, Dept. of CSE, NHCE

 ● Algorithm analysis, is a study for determining the amount of resources (time and memory)

needed for executing an algorithm. Efficiency or complexity of an algorithm is always

stated in terms of time and space complexity.

● The time complexity of an algorithm is defined as the running time of a program as a

function of the input size. Similarly, the space complexity of an algorithm is defined as the

amount of computer memory that is required during the program execution as a function

of the input size.

● In general, the time taken by an algorithm grows with the size of the input, so it is

important to describe the running time of a program as a function of the size of its input.

To do so, need to define the terms "running time" and "size of input.

● Since, running time requirements are more critical than memory requirements. Therefore

basically the running time efficiency of algorithms is categorized as follows: (i) Worst-

case (ii) Average-case (iii) Best-case Time Complexity

● Worst-case time complexity: It is the behaviour of an algorithm with respect to the worst

possible input instance. It denotes the upper bound on the running time for any input.

Therefore, the algorithm will never go beyond this time limit.

● Average-case time complexity: The average-case running time of an algorithm is an

estimate of the running time for an ‘average’ input. It specifies the expected behaviour of

the algorithm when the input is randomly drawn from a given distribution. Average-case

running time assumes that all inputs of a given size are equally likely.

● Best-case running time”" The term ‘best-case performance’ is used to analyse an

algorithm under optimal conditions. It denotes the lower bound on the running time for any

input. Therefore, the algorithm will never go athis time limit. For example, the best case

Teena A James, Dept. of CSE, NHCE

 for a simple linear search on an array occurs when the desired element is the first in the

list.

● Algorithm Analysis uses two approaches for resolving time complexity: Iterative approach

and Recursive approach

● Iterative Approach: If an algorithm contains loops, then the efficiency of that algorithm

may vary depending on the number of loops and the running time of each loop in the

algorithm.

● Recursive Approach: If an algorithm contains recursive call, then the efficiency of that

algorithm may vary depending on the number of calls being made

Need of Time and Space Complexity

The time and space complexity is expressed using a function f(n) where n is the input size for a

given instance of the problem being solved. Complexity is required when

● We want to predict the rate of growth of complexity as the input size of the problem

increases.

● There are multiple algorithms that find a solution to a given problem and we need to find

the algorithm that is most efficient

(III) FRAMEWORK FOR ANALYSIS

We use a hypothetical model with following assumptions
• Total time taken by the algorithm is given as a function on its input size
• Logical units are identified as one step

• Every step require ONE unit of time

• Total time taken = Total Num. of steps executed
Input’s size: Time required by an algorithm is proportional to size of the problem instance. For
e.g., more time is required to sort 20 elements than what is required to sort 10 elements.
Units for Measuring Running Time: Count the number of times an algorithm‘s basic operation

Teena A James, Dept. of CSE, NHCE

is executed. (Basic operation: The most important operation of the algorithm, the operation
contributing the most to the total running time.) For e.g., The basic operation is usually the most
time-consuming operation in the algorithm‘s innermost loop. Consider the following example:

ALGORITHM sum_of_numbers ( A[0… n-1] )

// Functionality : Finds the Sum

// Input : Array of n numbers

// Output : Sum of „n‟ numbers

i=0
sum=0
while i < n
sum=sum + A[i]
n
i=i + 1
return sum

Total number of steps for basic operation execution, C (n) = n

NOTE: Constant of fastest growing term is insignificant: Complexity theory is an
Approximation theory. Not interested in exact time required by an algorithm to solve the problem,
rather interested in order of growth. i.e How much faster will algorithm run on computer that is
twice as fast? How much longer does it take to solve problem of double input size? We can crudely
estimate running time by
T (n) ≈ Cop *C (n) , Where, T (n): running time as a function of n, Cop : running time of a
single operation. , C (n): number of basic operations as a function of n.

Teena A James, Dept. of CSE, NHCE

(IV) ASYMPTOTIC NOTATION:

Order of Growth:

● It describes how the running time of an algorithm increase with size of the input. It is

estimated by taking into account the dominant terms of the running time expression. This

term, overpasses negligible values when the input size is large.

Eg. Let T(n) = an+b; where (a>0)

If “size of the input” is multiplied by “k”, then dominant term in t(n) is also

multiplied by “k”

T(kn) = k(an)+b;

Thus the growth rate T(n) has linear manner

● Running Time of an algorithm on a particular input is the number of primitive operations

or steps executed

● Asymptotic can be defined as a way to describe the behaviour of functions in the limit or

without bounds

● This model assumes, time taken for simple instructions such as addition, multiplication

takes exactly one time unit, thus makes convenient for describing running time

BIG O NOTATION:

● It provides asymptotically upper bound f(n) and gives the worst case complexity

Teena A James, Dept. of CSE, NHCE

 ● Let “f” be any non-negative function. Then f(n)= O(g(n)), if there exists certain constants

“c” and “n0” such that

0 ⩽ f(n) ⩽ c.g(n), for all large input n, n ⩾0. Therefore it can be said that g(n) is

the upper bound of f(n)

Example:

(1) Let f(n) = 3n+2; g(n) = n

Prove: f(n)= O(g(n)) (i.e f(n) ⩽ c.g(n)) where c>0 and n0⩾0

Proof:

If f(n) ⩽ c.g(n) then

3n+2 ⩽ c.g(n)

Evaluate “n” by substituting c=4, we get

3n+2 ⩽4n

i.e n⩾2. So for every n⩾2 where c=4, f(n) ⩽ c.g(n))[∴ f(n)= O(g(n)) ]

Thus, Big OH -> Least Upper Bound

OMEGA NOTATION (Ω)

● It provides asymptotically lowest bound f(n) and gives the best case complexity

● Let “f” be any non-negative function. Then f(n)= Ω(g(n)), if there exists certain constants

“c” and “n0” such that

0 ⩽ c.g(n) ⩽f(n), for all large input n, n ⩾0. Therefore it can be said that g(n) is the

lower bound of f(n)

Example:

(2) Let f(n) = 3n+2; g(n) = n

Prove: f(n)= Ω(g(n)) (i.e f(n) ⩾c.g(n)) where c>0 and n0⩾0

Proof:

Teena A James, Dept. of CSE, NHCE

 If f(n) ⩾c.g(n) then

3n+2 ⩾c.g(n)

Evaluate “n” by substituting c=1, we get

3n+2 ⩾n

i.e n⩾1. So for every n⩾1 where c=1, f(n) ⩾ c.g(n))[∴ f(n)= Ω(g(n)) ]

Thus, Big Omega-> Closest Lower Bound

THETA NOTATION (Q)

● For any two functions f(n) and g(n) we have f(n)=Θg(n), if and only if

f(n)= O(g(n)) And f(n)= Ω(g(n))

● Let “f” be any non-negative function. Then f(n)= Θ(g(n)), if there exists certain constants

“c1”,”c2” and “n0” such that

0⩽ c1.g(n)⩽ f(n) ⩽ c2.g(n), for all large input n, n ⩾0. Therefore it can be said that

g(n) is asymptotically tight bound for f(n) and it gives the average case time complexity

Example:

(3) Let f(n) = 3n+2; g(n) = n

Prove: f(n)= Θ(g(n)) (i.e c1.g(n)⩽f(n) ⩽ c2.g(n)) where c1,c2>0 for n⩾n0, n0⩾1

Proof:

As per the asymptotic upper and lower bound, proved f(n) can be

n⩽ 3n+2 ⩽ 4n

Thus n0⩾1, c1=1, c2=4

c1.g(n)⩽f(n) ⩽ c2.g(n) [∴ f(n)= Θ(g(n)) ]

Teena A James, Dept. of CSE, NHCE

Teena A James, Dept. of CSE, NHCE
(V) Mathematical Analysis of Non-Recursive and Recursive Algorithms

Mathematical analysis (Time Efficiency) of Non-recursive Algorithms General plan for analyzing
efficiency of non-recursive algorithms:

1. Decide on parameter n indicating input size

2. Identify algorithm‘s basic operation

3. Check whether the number of times the basic operation is executed depends only on
the input size n. If it also depends on the type of input, investigate worst, average, and
best case efficiency separately.
4. Set up summation for C(n) reflecting the number of times the algorithm‘s basic
operation is executed.
5. Simplify summation using standard formulas

Example: Finding the largest element in a given array

ALOGORITHM MaxElement(A[0..n-1])

Teena A James, Dept. of CSE, NHCE

 //Determines the value of largest element in a given array //nput: An array A[0..n-1] of
real numbers //Output: The value of the largest element in A
currentMax ← A[0]
for i ← 1 to n - 1 do

if A[i] > currentMax

currentMax ← A[i]

return currentMax

Analysis:

1. Input size: number of elements = n (size of the array)

2. Basic operation:
a) Comparison

b) Assignment

3. NO best, worst, average cases.

4. Let C (n) denotes number of comparisons: Algorithm makes one comparison on
each execution of the loop, which is repeated for each value of the loop‘s variable i within the
bound between 1 and n – 1.
Example: Element uniqueness problem
Algorithm UniqueElements (A[0..n-1])
//Checks whether all the elements in a given array are distinct //Input: An array A[0..n-1]

//Output: Returns true if all the elements in A are distinct and false otherwise

for i 0 to n - 2 do

for j i + 1 to n – 1 do

if A[i] = = A[j]

Teena A James, Dept. of CSE, NHCE

 return false

return true

Analysis

1. Input size: number of elements = n (size of the array)

2. Basic operation: Comparison
3. Best, worst, average cases EXISTS.
Worst case input is an array giving largest comparisons.
• Array with no equal elements
• Array with last two elements are the only pair of equal elements

4. Let C (n) denotes number of comparisons in worst case: Algorithm makes one
comparison for each repetition of the innermost loop i.e., for each value of the
loop‘s variable j between its limits i + 1 and n – 1; and this is repeated for each
value of the outer loop i.e, for each value of the loop‘s variable i between its limits
0 and n – 2

(V) Mathematical analysis (Time Efficiency) of recursive Algorithms

General plan for analyzing efficiency of recursive algorithms:

1. Decide on parameter n indicating input size
2. Identify algorithm‘s basic operation
3. Check whether the number of times the basic operation is executed depends only on
the input size n. If it also depends on the type of input, investigate worst, average, and best
case efficiency separately.
times the up
4. Set algorithm‘s basic
recurrence operation
relation, withisanexecuted.
appropriate initial condition, for the number of
5. Solve the recurrence.

Example: Factorial function

Teena A James, Dept. of CSE, NHCE

ALGORITHM Factorial (n)
//Computes n! recursively
//Input: A nonnegative integer n
//Output: The value of n!

if n = = 0

return 1

else

return Factorial (n – 1) * n

Analysis:

1. Input size: given number = n

2. Basic operation: multiplication
3. NO best, worst, average cases.
4. Let M (n) denotes number of multiplications.
M (n) = M (n – 1) + 1 for n > 0
M (0) = 0 initial condition
Where: M (n – 1) : to compute Factorial (n – 1) and 1 :to multiply Factorial (n – 1) by n
5. Solve the recurrence: Solving using “
Backward substitution method”: M (n) = M (n – 1) + 1

= [M(n – 2) + 1 ] + 1 = M (n – 2) + 2

= [ M (n – 3) + 1 ] + 3 = M (n – 3) + 3

In the ith recursion, we have = M (n – i ) + i

When i = n, we have

= M (n – n ) + n = M (0 ) + n Since M (0) = 0
=n
M (n) ∈ Θ (n)

Teena A James, Dept. of CSE, NHCE

 Example: Find the number of binary digits in the binary representation
of a positive decimal integer

ALGORITHM BinRec (n)

//Input: A positive decimal integer n

//Output: The number of binary digits in n‟s binary representation
if n = = 1
return 1

else

return BinRec (└ n/2 ┘) + 1

Analysis:

1. Input size: given number = n

2. Basic operation: addition

3. NO best, worst, average cases.
4. Let A (n) denotes number of additions.
A (n) = A (└ n/2 ┘) + 1 for n > 1
A (1) = 0 initial condition
Where: A (└ n/2 ┘) : to compute BinRec (└ n/2 ┘)

1 : to increase the returned value by 1

5 Solve the recurrence:

.

f> 1
n/2 ┘) + 1
A (n) = A (└ k
Assume n = 2 (smoothness rule)
k k-1
A (2 ) = A (2 ) + 1 for k > 0; A (20) = 0
k k-1 k-2
Solving using “Backward substitution method”: A (2 ) = A (2 ) + 1 = [A (2 )+

Teena A James, Dept. of CSE, NHCE

 1] + 1(2
k-3 k-2
= A (2) + 1]) ++ 22 = A (2
k-3
= [A )+3
In the ith recursion, we have

k-i
= A (2 )+i

When i = k, we have

k-k 0 0
= Ak(2 ) + k = A (2 ) + k Since A (2 ) = 0
A (2 ) = k
k
Since n = 2 , HENCE k = log2 n

A (n) = log2 n

A (n) ∈ Θ ( log n)

MASTER’S THEOREM

The master method provides a "cookbook" method for solving recurrences of the form

T(N)= a T(n/b) + f(n)

Where, a ≥ 1 and b > 1 are constants

and f (n) is an asymptotically positive function.

Then T (n) can be bounded asymptotically as follows:

1. If f(n) = Օ(n b )
log a-ε
for some constant ε > 0, then

T(n) = Θ (n log ba )

2. If f(n) = Θ (n log ba ) , then the solution to the recurrence is given by

T(n) = Θ (n log ba log n )

Note: In this case, f(n) is "equal" to n log ba, i.e. neither term dominates thus the extra

Teena A James, Dept. of CSE, NHCE

 term to get the bound.

3. If f (n) = Ω(n log ba+ε ) , for some constant > 0,

and if af (n/b) ≤ cf (n) for some constant c <1 and all

sufficiently large n, then

T(n) = Θ (f(n) )

In all three cases it is important to compute the recursive term run time and compare

it asymptotically to the combine term f(n) run time to determine which case holds.

T(n) = Θ (n2)

Examples

(1) Solve the recursive equation

T(n) = 9(T(n/3)) + n

For this equation a = 9, b = 3, and f(n) = n.

Computing (n log ba) = n2

Then f(n) = (n log ba ) is satisfied for any ε ≤ 1,

i.e. choose ε = 1. Hence the equation satisfies Case 1 so the solution is

T(n) = Θ (n log ba ) = Θ (n 2)

(2) Solve the recursive equation

T(n) = 3(T(n/4)) + n logn

For this equation a = 3, b = 4, and f(n) = n logn

Teena A James, Dept. of CSE, NHCE

 Computing (n log43 ) = n0.79

Here, f(n) ≠ (n log ba) i.e to make it equal, need to add some constant ε to (n log

a
b )

f(n)= n logn = Ω(n) = Ω(n0.79+ε ) , which is satisfied for any ε ≤ 0.21,

e.g. choose ε = 0.2. Hence the equation might satisfy Case3 so checking regularity

a f(n/b) = 3(n/4) log(n/4)

⩽(¾) nlogn

⩽(¾) f(n)

Thus regularity holds by choosing c = 3/4 < 1. Therefore the equation satisfies Case

3 and the solution is

T(n) = Θ (f(n)) = Θ (n logn )

(VI) IMPORTANT PROBLEMS:

● Sorting

● Searching

● String Processing

● Graph Problems

● Combinatorial Problems

● Geometric Problems

● Numerical Problems

(A) SORTING

Teena A James, Dept. of CSE, NHCE

 ● Rearrange the items of a given list in ascending order.

•Input: A sequence of n numbers <a1, a2, …, an>

•Output: A reordering <a´1, a´2, …, a´n> of the input sequence such that a´1≤
a´2 ≤ … ≤ a´n.

● Why sorting?

•Help searching

•Algorithms often use sorting as a key subroutine.

● Sorting key

•A specially chosen piece of information used to guide sorting. E.g., sort student
records by names.

● Examples of sorting algorithms

•Selection sort
•Bubble sort
•Insertion sort
•Merge sort
•Heap sort …
● Evaluate sorting algorithm complexity: the number of key comparisons.
● Two properties
•Stability: A sorting algorithm is called stable if it preserves the relative order of
any two equal elements in its input.
•In place : A sorting algorithm is in place if it does not require extra memory,
except, possibly for a few memory units.

SelectionSort(A[0..n - 1])

//Input: An array A[0..n - 1] of orderable elements

//Output: Array A[0..n - 1] sorted in ascending order

for i=0 to n - 2 do

min=i

Teena A James, Dept. of CSE, NHCE

 for j=i + 1 to n - 1 do

if A[j ]<A[min] min=j

swap A[i] and A[min]

(B) SEARCHING

● Find a given value, called a search key, in a given set.

● Examples of searching algorithms

•Sequential search

•Binary search …

Input: sorted array a_i < … < a_j and key x;

m �(i+j)/2;

while i < j and x != a_m do

if x < a_m then j � m-1

else i � m+1;

if x = a_m then output a_m;

Time: O(log n)

( C ) STRING PROCESSING

●A string is a sequence of characters from an alphabet.

●Text strings: letters, numbers, and special characters.

●String matching: searching for a given word/pattern in a text.

Teena A James, Dept. of CSE, NHCE

 Examples:

●searching for a word or phrase on WWW or in a Word document

●searching for a short read characters in the genomic sequence

( D ) GRAPH PROBLEMS

●Informal definition

A graph is a collection of points called vertices, some of which are connected by

line segments called edges.

●Modeling real-life problems

Modeling WWW

Communication networks

Project scheduling …

●Examples of graph algorithms

Graph traversal algorithms

Shortest-path algorithms

Topological sorting

Teena A James, Dept. of CSE, NHCE

 2
BRUTE FORCE & DIVIDE & CONQUER

TOPICS TO BE COVERED :

BRUTE FORCE:

● Brute force Selection Sort and Bubble Sort

● String matching algorithms –NAÏVE string matching algorithms, Rabin
Karp algorithm & Knuth Morris Pratt algorithm,
● Exhaustive Search – Travelling Salesman problem, Knapsack problem,
Assignment problem

DIVIDE & CONQUER:

● Merge Sort and its analysis,

● Quick sort and its analysis

Teena A James, Dept. of CSE, NHCE

INTRODUCTION

● Brute force is a straightforward approach to problem solving, usually directly

based on the problem‘s statement and definitions of the concepts involved.
● This approach yields reasonable algorithms, for some important problems like
computing an (a > 0, n a nonnegative integer), multiplying two matrices,
searching, sorting etc. with no limitation on the input size
● In general, a brute-force algorithm can be useful for solving small-size
instances of a problem.

(I) Selection Sort and Bubble Sort

Problem:Given a list of n orderable items (e.g., numbers, characters from some

alphabet, character strings), rearrange them in nondecreasing order.

ALGORITHM

SelectionSort(A[0..n - 1])

//Input: An array A[0..n - 1] of orderable elements

//Output: Array A[0..n - 1] sorted in ascending order

for i=0 to n - 2 do

min=i

for j=i + 1 to n - 1 do

if A[j ]<A[min] min=j

swap A[i] and A[min]

Teena A James, Dept. of CSE, NHCE

Example:

Seconditeration

Teena A James, Dept. of CSE, NHCE

Performance Analysis of the selection sort algorithm:

The input‘s size is given by the number of elements n.

The algorithm‘s basic operation is the key comparison A[j]<A[min]. The

number of times it is executed depends only on the array‘s size and is given
by.

Thus,
selection sort is a O(n2) algorithm on all inputs. The number of key swaps is
only O(n) or, more precisely, n-1. This property distinguishes selection sort
positively from many other sorting algorithms.

Teena A James, Dept. of CSE, NHCE

Bubble Sort

Compare adjacent elements of the list and exchange them if they are out of order.
Then repeat the process,by doing it repeatedly, and end up bubbling up the largest
element to the last position on the list

ALGORITHM

BubbleSort(A[0..n - 1])

//The algorithm sorts array A[0..n - 1] by bubble sort

//Input: An array A[0..n - 1] of orderable elements

//Output: Array A[0..n - 1] sorted in ascending order

for i=0 to n - 2 do

for j=0 to n - 2 - i do

if A[j + 1]<A[j ] swap A[j ] and A[j + 1]

Teena A James, Dept. of CSE, NHCE

Example

Teena A James, Dept. of CSE, NHCE

Bubble Sort Analysis

Teena A James, Dept. of CSE, NHCE

Clearly, for bubble sort, irrespective of the nature of input,
the number of passes to be made is n − 1. Further, the number
of comparisons during i th pass is n − i. By the end of every
pass, at least one element is placed in its right position. In
case of best case input, there is no swapping done and for
every other input swapping may be required during each
pass. Since the underlying model focuses on the number of
comparisons, the number of comparisons is n − 1 + n − 2 + . .
. + 2 + 1 = O(n 2 ) for all inputs.

Bubble sort is not very good for big set of input. However bubble sort is very
simple to code.

(II) Brute Force String Matching

PROBLEM : Find a substring of the text that matches the pattern.( find i—the index
of the leftmost character of the first matching substring in the text—such that

ti = p0, . . . , ti+j = pj , . . . , ti+m-1 = pm-1)

Given : a string of n characters called the text and

a string of m characters (m = n) called the pattern,

Teena A James, Dept. of CSE, NHCE

 where, ti is the ‘i’th character of ‘text’ and pj is the jth character of pattern)

Example:

T=bacbababababacab

P=ababaca

The algorithm shifts the pattern always after a single character comparison. In the
worst case, the algorithm may have to make all ‘m’ comparisons before shifting the

Teena A James, Dept. of CSE, NHCE

pattern, and this can happen for each of the (n - m + 1) tries. Thus, in the worst case,
the algorithm is in θ(nm).

Naive String Matcher is inefficient because it entirely ignores information gained

about the text for one value of ‘s’ when it considers other values of ‘s’.

Rabin Karp

All characters are interpreted as radix-d digits. It calculates a hash value(high-

order digit position of an m-digit window.) for the pattern, and for each M-character
subsequence of text to be compared. If the hash values are unequal, the algorithm
will calculate the hash value for next M-character sequence. If the hash values are
equal, the algorithm will compare the pattern and the M-character sequence. In this
way, there is only one comparison per text subsequence, and character matching is
only needed when hash values match

ALGORITHM

Teena A James, Dept. of CSE, NHCE

 Example:

Teena A James, Dept. of CSE, NHCE

 Each character is a decimal digit, and compute values
modulo 13. (a) A text string. A window of length 5 is shaded.
The numerical value of the shaded number, computed modulo
13, yields the value 7. (b) The same text string with values
computed modulo 13 for each possible position of a length-
5 window. Assuming the pattern P = 31415, whose value modulo
13 is 7, since 31415 ≣ 7 (mod 13). The algorithm finds two such
windows, shown shaded in the figure. The first, beginning at
text position 7, is indeed an occurrence of the pattern, while
the second, beginning at text position 13, is a spurious hit.
(c) How to compute the value for a window in constant time,
given the value for the previous window. The first window has
value 31415. Dropping the high-order digit 3, shifting left
(multiplying by 10), and then adding in the low-order digit 2
gives the new value 14152. Because all computations are
performed modulo 13, the value for the first window is 7, and
the value for the new window is 8.

In worst case, RABIN-KARP-MATCHER takes Θ(m) preprocessing time,

and its matching time is Θ(n-m+1)m

KNUTH MORRIS PRATT:

A linear time algorithm that solves the string matching problem by

preprocessing P in Θ(m) time. The idea is to skip some comparisons by using an
auxiliary array π that is defined as the following: π[i] is the largest integer smaller

Teena A James, Dept. of CSE, NHCE

than i such that P1 . . . Pπ[i] is a suffix of P1 . . . Pi. Π indicates how much of the last
comparison can be reused.

Two Components of KMP algorithm: The prefix function, Π for pattern encapsulates
knowledge about how the pattern matches against shifts of itself. This information
can be used to avoid useless shifts of the pattern ‘p’. In other words, this enables
avoiding backtracking on the string ‘S’.The KMP Matcher with string ‘S’, pattern
‘p’ and prefix function ‘Π’ as inputs, finds the occurrence of ‘p’ in ‘S’ and returns
the number of shifts of ‘p’ after which occurrence is found.

ALGORITHM

Teena A James, Dept. of CSE, NHCE

Teena A James, Dept. of CSE, NHCE
Example

Teena A James, Dept. of CSE, NHCE

Analysis

The running time of Knuth-Morris-Pratt algorithm is proportional to the time needed

to read the characters in text and pattern. The worst-case running time of the
algorithm is O(m + n)

(III) Exhaustive Search

Exhaustive Search:
1. Travelling Salesman Problem
2. Knapsack Problem
3. Assignment Problem
Exhaustive Search : A brute force solution to a problem involving search for an
element with a special property, usually among combinatorial objects such as
permutations, combinations, or subsets of a set.

Method:

● Generate a list of all potential solutions to the problem in a systematic manner

(see algorithms in Sec. 5.4)
● Evaluate potential solutions one by one, disqualifying infeasible ones and, for
an optimization problem, keeping track of the best one found so far
● When search ends, announce the solution(s) found

Travelling Salesman Problem

● The traveling salesman problem (TSP) has been intriguing researchers for the
last 150 years by its seemingly simple formulation, In layman’s terms, the
problem asks to find the shortest tour through a given set of n cities that visits
each city exactly once before returning to the city where it started.

Teena A James, Dept. of CSE, NHCE

 ● The problem can be conveniently modeled by a weighted graph, with the
graph’s vertices representing the cities and the edge weights specifying the
distances. Then the problem can be stated as the problem of finding the
shortest Hamiltonian circuit of the graph. (A Hamiltonian circuit is defined
as a cycle that passes through all the vertices of the graph exactly once.
● Thus, we can get all the tours by generating all the
permutations of n − 1 intermediate cities, compute the
tour lengths, and find the shortest among them.

Example

Tour Cost

a→b→c→d→a 2+3+7+5 = 17

a→b→d→c→a 2+4+7+8 = 21

a→c→b→d→a 8+3+4+5 = 20

a→c→d→b→a 8+7+4+2 = 21

a→d→b→c→a 5+4+3+8 = 20

Teena A James, Dept. of CSE, NHCE

 a→d→c→b→a 5+7+3+2 = 17

The total number of permutations needed is still (n − 1)!,

which makes the exhaustive-search approach impractical
for all but very small values of n.

Knapsack Problem

Given n items of known

weights w1, w2, . . . , wn and
values v1, v2, . . . , vn and
a knapsack of capacity W ,
● Find the most valuable subset of the items that fit into the knapsack. Think
about a transport plane that has to deliver the most valuable set of items to a
remote location without exceeding the plane’s capacity.
● The exhaustive-search approach to this problem leads to generating all the
subsets of the set of n items given, computing the total weight of each subset
in order to identify feasible subsets (i.e., the ones with the total weight not
exceeding the knapsack capacity), and finding a subset of the largest value
among them.

Example:

item weight value

1 2 $20

Teena A James, Dept. of CSE, NHCE

 2 5 $30

3 10 $50

4 5 $10

Knapsack capacity W=16

Subset Total weight Total value

{1} 2 $20

{2} 5 $30

{3} 10 $50

{4} 5 $10

{1,2} 7 $50

{1,3} 12 $70

{1,4} 7 $30

{2,3} 15 $80

{2,4} 10 $40

{3,4} 15 $60

{1,2,3} 17 not feasible

{1,2,4} 12 $60

{1,3,4} 17 not feasible

{2,3,4} 20 not feasible

{1,2,3,4} 22 not feasible

Teena A James, Dept. of CSE, NHCE

Thus, for both the traveling salesman and knapsack problems considered above,
exhaustive search leads to algorithms that are extremely inefficient on every input.
In fact, these two problems are the best-known examples of so-called NP-hard
problems.

Assignment Problem
● There are n people who need to be assigned to execute n jobs, one person per
job. (That is, each person is assigned to exactly one job and each job is
assigned to exactly one person.)
● The cost if the ith person is assigned to the j th job is a known quantity C[i, j
] for each pair i, j = 1, 2, . . . , n.
The problem is to find an assignment with the minimum total cost.

Example:
Job 0 Job 1 Job 2 Job 3
Person 0 9 2 7 8
Person 1 6 4 3 7
Person 2 5 8 1 8
Person 3 7 6 9 4

9 2 7 8
C = 6 4 3 7
5 8 1 8
7 6 9 4

Assignment (col.#s) Total Cost

1, 2, 3, 4 9+4+1+4=18

Teena A James, Dept. of CSE, NHCE

 1, 2, 4, 3 9+4+8+9=30
1, 3, 2, 4 9+3+8+4=24
1, 3, 4, 2 9+3+8+6=26
1, 4, 2, 3 9+7+8+9=33
1, 4, 3, 2 9+7+1+6=23
etc.

● Feasible solutions to the assignment problem as n-tuples

j1, . . . , jn in which the ith component,
i = 1, . . . , n, indicates the column of the element selected in the ith row (i.e.,
the job number assigned to the ith person).
Therefore, the exhaustive-search approach to the assignment problem would
require generating all the permutations of integers 1, 2, . . . , n, computing the
total cost of each assignment by summing up the corresponding elements of
the cost matrix, and finally selecting the one with the smallest sum.
● Since the number of permutations to be considered for the general case of the
assignment problem is n!, exhaustive search is impractical for all but very
small instances of the problem.

(IV) DIVIDE & CONQUER

Definition:

Divide & conquer is a general algorithm design strategy with a general plan as
follows:

1. DIVIDE: A problem‘s instance is divided into several smaller

instances of the same problem

Teena A James, Dept. of CSE, NHCE

 2. RECUR: Solve the sub-problem recursively.

3. CONQUER: If necessary, the solutions obtained for the smaller

instances are combined to get a solution to the original instance.

Advantages of Divide & Conquer technique:

● For solving conceptually difficult problems like Tower Of Hanoi, divide &
conquer is a powerful tool
● Results in efficient algorithms
● Divide & Conquer algorithms are adapted for execution in multi-processor
machines
● Results in algorithms that use memory cache efficiently.

Limitations of divide & conquer technique:

Teena A James, Dept. of CSE, NHCE

 ● Recursion is slow
● Simple problem may be more complicated than an iterative approach.

General Method: An instance of size n can be divided into b

instances of size n/b, with ―a‖ of them needing to be
solved. [ a ≥ 1, b > 1]. Assume size n is a power of b. The
recurrence for the running time T(n) is as follows:

T(n) = aT(n/b) + f(n)

where:f(n) – a function that accounts for the time spent on dividing the problem
into smaller ones and on combining their solutions. Therefore, the order of growth
of T(n) depends on the values of the constants a & b and the order of growth of the
function f(n)

Master theorem

Theorem: If f(n) Є Θ (nd) with d ≥ 0 in recurrence equation

T(n) = aT(n/b) + f(n),

then,

Θ (n^d) if a < b^d

T(n) = Θ(n^d log n) if a = b^d

Teena A James, Dept. of CSE, NHCE

 Θ (n^logba ) if a > b^d

Example: Let T(n) = 2T(n/2) + 1, solve using master theorem.

Solution:Here: a = 2, b = 2, f(n) = Θ(1), d = 0

Therefore: a > bd i.e., 2 > 20

Case 3 of master theorem holds good.

Therefore:

T(n) Є=Θ (nlogba )

Є =(nlog22 )

Є Θ (n)

(a) Binary Search

It is dichotomic divide and conquer search algorithm. Ti inspects the middle element
of the sorted list. If equal to the sort value, then the position has been found.
Otherwise, if the key is less than the middle element, do a binary search on the first
half, else on the second half.

Algorithm:

Algorithm can be implemented as recursive or non-recursive algorithm.

BinSrch ( A[0 … n-1], key)

l→0

Teena A James, Dept. of CSE, NHCE

 r→n-1

while l ≤ r do

m→( l + r) / 2

if key = = A[m]

return m

Else

if key < A[m]

R→m-1

Else

l→m+1

return -1

Analysis:

• Input size: Array size, n

• Basic operation: key comparison

• Depend on Best – key matched with mid element worst – key not found or
key sometimes in the list

Teena A James, Dept. of CSE, NHCE

• Let C(n) denotes the number of times basic operation is executed. Then
Cworst(n) = Worst case efficiency. Since after each comparison the algorithm
divides the problem into half the size, we have
Cworst(n) = Cworst(n/2) + 1 for n > 1

C(1) = 1
• Solving the recurrence equation using master theorem, to give the number of
times the search key is compared with an element in the array, we have
C(n) = C(n/2) + 1 a = 1
b=2
f(n) = n0 ; d = 0
case 2 holds:
C(n) = Θ (ndlog n)
= Θ (n0log n)
= Θ ( log n)

Applications of binary search:

• Number guessing game

• Word lists/search dictionary etc

Advantages:

• Efficient on very big list

• Can be implemented iteratively/recursively

Teena A James, Dept. of CSE, NHCE

Limitations:

• Interacts poorly with the memory hierarchy

• Requires given list to be sorted

• Due to random access of list element, needs arrays instead of linked list.

(b) Merge Sort

Definition: Merge sort is a sort algorithm that splits the items to be sorted
into two groups, recursively sorts each group, and merges them into a final sorted
sequence.

● This is a classic divide and conquer sorting algorithm that splits an array into

two pieces, sorts each piece (recursively!), then merges the results back together.

APPROACH

Let there there be n data elements stored in an 1-D array.

1. If n = 1, it is already sorted

2. If n > 1,

● Split array A[0..n-1] into about equal halves and make copies of
each half in arrays L and R

Teena A James, Dept. of CSE, NHCE

 ● Sort arrays L and R recursively
● Merge sorted arrays Land Rinto array A as follows:
○ Repeat the following until no elements remain in one of
the arrays:

– compare the first elements in the remaining unprocessed

portions of the arrays

– copy the smaller of the two into A, while incrementing

the index indicating the unprocessed portion of that array

● Once all elements in one of the arrays are processed, copy the
remaining unprocessed elements from the other array into A.

Teena A James, Dept. of CSE, NHCE

Teena A James, Dept. of CSE, NHCE
Example:

Analysis:

• Input size: Array size, n

• Basic operation: key comparison

• Best, worst, average case exists:

Teena A James, Dept. of CSE, NHCE

 Worst case: During key comparison, neither of the two arrays becomes
empty before the other one contains just one element.
• Let C(n) denotes the number of times basic operation is executed. Then
C(n) = 2C(n/2) + Cmerge(n) for n > 1
C(1) = 0
where, Cmerge(n) is the number of key comparison made during the
merging stage.

In the worst case:

Cmerge(n) = 2 Cmerge(n/2) + n-1 for n > 1

Cmerge(1) = 0

• Solving the recurrence equation using master theorem: C(n) = 2C(n/2) + n-1
for n > 1
C(1) = 0

Here a = 2; b = 2
f(n) = n; d = 1
Therefore 2 = 21, case 2 holds
C(n) = Θ (ndlog n)
= Θ (n1log n)
= Θ (n log n)

Advantages:

• Number of comparisons performed is nearly optimal.

Teena A James, Dept. of CSE, NHCE

• Mergesort will never degrade to O(n2)

• It can be applied to files of any size

Limitations:

• Uses O(n) additional memory.

(C) Quick Sort

Definition: Quick sort is a well –known sorting algorithm, based on divide &
conquer approach. The steps are:

1. Pick an element called pivot from the list

2. Reorder the list so that all elements which are less than the pivot come
before the pivot and all elements greater than pivot come after it. After this
partitioning, the pivot is in its final position. This is called the partition
operation

3. Recursively sort the sub-list of lesser elements and sub-list of greater

elements.

Features:

● Efficient algorithm
● NOT stable sort
● Significantly faster in practice, than other algorithms

ALGORITHM

Teena A James, Dept. of CSE, NHCE

Quicksort (A[ l …r ])

//sorts by quick sort

//i/p: A sub-array A[l..r] of A[0..n-1],defined by its left and right indices l and r

//o/p: The sub-array A[l..r], sorted in ascending order

if l < r

Partition (A[l..r]) // s is a split position

Quicksort(A[l..s-1])

Quicksort(A[s+1..r]

Partition (A[l ..r])

//Partitions a sub-array by using its first element as a pivot

//i/p: A sub-array A[l..r] of A[0..n-1], defined by its left and right indices l and r (l
< r)

//o/p: A partition of A[l..r], with the split position returned as this function‘s value

p→A[l]

i→l

j→r + 1;

Repeat

Teena A James, Dept. of CSE, NHCE

 repeat i→i + 1 until A[i] >=p //left-right scan

repeat j→j – 1 until A[j] < p //right-left scan

if (i < j) //need to continue with the scan

swap(A[i], a[j])

until i >= j //no need to scan

swap(A[l], A[j])

return j

Example: Sort by quick sort the following list: 5, 3, 1, 9, 8, 2, 4, 7, show recursion

tree.

5 3 1 9 8 2 4 7

2 3 1 4 5 8 9 7

1 2 3 4 5 7 8 9

1 2 3 4 5 7 8 9

1 2 3 4 5 7 8 9

1 2 3 4 5 7 8 9

Analysis:

● Input size: Array size, n

● Basic operation: key comparison
● Best, worst, average case exists: Best case: when partition happens in the
middle of the array each time. Worst case: When input is already sorted.

Teena A James, Dept. of CSE, NHCE

 During key comparison, one half is empty, while remaining n-1 elements are
on the other partition.
● Let C(n) denotes the number of times basic operation is executed in worst
case: Then

C(n) = C(n-1) + (n+1) for n > 1

C(1) = 1

Best case:

C(n) = 2C(n/2) + Θ(n)

Solving the recurrence equation using backward substitution/ master

theorem, we have:

C(n) = C(n-1) + (n+1) for n > 1; C(1) = 1

C(n) = Θ (n2)

C(n) = 2C(n/2) + Θ(n).

= Θ (n1log n)

= Θ (n log n)

NOTE:

The quick sort efficiency in average case is Θ( n log n) on random input.

Teena A James, Dept. of CSE, NHCE

Teena A James, Dept. of CSE, NHCE
 3
GREEDY AND DYNAMIC PROGRAMMING

TOPICS TO BE COVERED :

GREEDY METHOD:

● Introduction, Job scheduling problem,

● Minimum Spanning tree algorithms – Kruskals & Prims,
● Shortest Path algorithm – Dijkstra’s,
● Huffman Trees,
● Knapsack problems, Travelling Salesman problem

DYNAMIC PROGRAMMING:

● Introduction, Computing Binomial Coefficients,

● Transitive closure - Warshall’s
● Floyds algorithm
INTRODUCTION

● Constructs a solution to any problem stages by stages through a sequence of

choices that are:
○ satisfying the constraints
○ locally optimal : leads to globally optimal solution
○ greedy and irrevocable
● It considers one input at a time. At each stage, the decision is made
regarding whether a particular input chosen gives an optimal solution or not.
● Greedy algorithms are typically used to solve an optimization problem,
which are required to be either maximized or minimized w. r. t. some
constraints or conditions.

Example : Change-Making Problem

(I) MINIMUM SPANNING TREE (MST) PROBLEM

● Spanning tree: Let G=(V,E) be an undirected connected graph. A subgraph

T=(V,E’) of G is a spanning tree of G if and only if T is a tree (i.e. no cycle
exist in T) and contains all the vertices of G.
● In general, a complete graph with n vertices has total n​n-2 ​spanning trees. For
example, if n=4 then total number of spanning tree is 16.

Teena A James, Dept. of CSE, NHCE

 ● A minimum spanning tree (MST) of a weighted connected graph G is that
spanning tree whose sum of length (or weight) of all its edges is minimum,
among all the possible spanning tree of G.
● For example: consider the following weighted connected graph G There are
so many spanning trees for G. Out of all possible spanning trees, two spann
ing trees of G are shown

● To find a MST of a given graph G, one of the following algorithms is used:

1. Kruskal’s algorithm 2. Prim’s algorithm

KRUSKAL ALGORITHM

1. Sort all the edges in non-decreasing order of their weight.

2. Pick the smallest edge. Check if it forms a cycle with the spanning tree formed
so far. If cycle is not formed, include this edge. Else, discard it.

3. Repeat step-2 until there are (V-1) edges in the spanning tree.

The algorithm is a ​Greedy Algorithm​. The Greedy Choice is to pick the smallest weight
edge that does not cause a cycle in the MST constructed so far. Let us understand it with
an example:

Teena A James, Dept. of CSE, NHCE

Example​:

After sorting:
Weight Src Dest
1 h g
2 i c
2 g f
4 a b
4 c f
6 i g
7 c d
7 h i
8 a h
8 b c
9 d e
10 f e
11 b h
14 d f

The graph contains 9 vertices and 14 edges. So, the minimum spanning tree formed will
be having (9 – 1) = 8 edges.

Now pick all edges one by one from sorted list of edges

(A) Pick edge h-g:​ No cycle is formed, include it.

(B) Pick edge i-c:​ No cycle is formed, include it.

Teena A James, Dept. of CSE, NHCE

 (C) ​Pick edge g-f:​ No cycle is formed, include it.
(D) Pick edge a-b:​ No cycle is formed, include it.

(E) Pick edge c-f:​ No cycle is formed, include it.

(F) Pick edge i-g: ​Since including this edge results in cycle, discard it

(G) Pick edge c-d :​ No cycle is formed, include it.

(H) Pick edge h-i:​ Since including this edge results in cycle, discard it.

Teena A James, Dept. of CSE, NHCE

PSUEDOCODE:

ANALYSIS :

● Let us assume a graph with e number of edges and n number of vertices. Kruskal’s
algorithm starts with sorting of edges.
● Time complexity of sorting algorithm= O (e log e)
● In Kruskal’s algorithm, we have to add an edge to the spanning tree, in each
iteration. This involves merging of two components.
● Time complexity of merging of components= O (e log n)
● Overall time complexity of the algorithm= O (e log e) + O (e log n)

Teena A James, Dept. of CSE, NHCE

 PRIM’S

● PRIM’s algorithm has the property that the edges in the set A always form
a single tree, i.e. at each step we have only one connected component.
● It starts with one starting vertex (say v) of a given graph G(V,E)Then, in
each iteration, a minimum weight edge (u, v) connecting a vertex v in the
set A to the vertices in the set is chosen . That is, it always finds an edge (u,
v) of minimum weight such that v∊ A and u∊ V-A. Then it modify the set A
by adding u i.e. This process is repeated until , i.e. until all the vertices are
not in the set A. Following pseudo-code is used to constructing a MCST,
using PRIM’s algorithm

PSEUDOCODE​:

Teena A James, Dept. of CSE, NHCE

EXAMPLE:

Initially empty and keys assigned to vertices are {0, INF, INF, INF, INF, INF, INF, INF} where
INF indicates infinite. Pick the vertex with minimum key value. The vertex A is picked, After
including to ​mstSet,​ update key values of adjacent vertices. Adjacent vertices of A are B and H.
The key values of B and H are updated as 4 and 8.

Pick the vertex with minimum key value and not Visited. The vertex B is picked and added to
mstSet. So mstSet now becomes {0, 1}. Update the key values of adjacent vertices of B. The key
value of vertex C becomes 8.

Teena A James, Dept. of CSE, NHCE

Similarly, Repeat the above steps until ​mstSet ​includes all vertices of a given graph. Finally, will
get the MST of above graph.

ANALYSIS​:

Thus, the total time for Prim’s algorithm is O(V lg V+ E lgV ) = O(E lgV ), which is
asymptotically the same as of Kruskal’s algorithm.

HUFFMAN ENCODING:

● It finds the minimum length bit string which can be used to encode a string of symbols
based on the frequencies of corresponding characters. The most frequent character
gets the smallest code and the least frequent character gets the largest code
● In a fixed-length code each codeword has the same length. In a variable-length
code codewords may have different lengths.

E.g. {a,b,c,d} can be coded as {00,01,10,11}

There are mainly two major parts in Huffman Coding

Teena A James, Dept. of CSE, NHCE

1) Build a Huffman Tree from input characters.

2) Traverse the Huffman Tree and assign codes to characters.

EXAMPLE:

Teena A James, Dept. of CSE, NHCE

PSEUDOCODE:

ANALYSIS:

As, for loop in lines 3–8 executes exactly n - 1 times, and since each heap
operation requires time O(lg n). The total running time of HUFFMAN on a set of n
characters is O(n lg n).

Teena A James, Dept. of CSE, NHCE

DIJKSTRA

● Dijkstra, is a greedy algorithm that solves the single-source shortest path problem
for a directed graph G=(V,E) with non-negative edge weights.
● It generates a SPT (shortest path tree) with given source as root.
● It also maintains two sets, one set contains vertices included in shortest path tree,
other set includes vertices not yet included in shortest path tree. At every step of
the algorithm, it finds a vertex which is in the other set and has minimum distance
from source.

DIJKSTRA(G, w, s)

1 INITIALIZE-SINGLE-SOURCE(G, s)

2S←Ø

3 Q ← V[G]

4 while Q ≠ Ø

5 do u ← EXTRACT-MIN(Q)

6 S ← S ∪{u}

7 for each vertex v ∈ Adj[u]

8 do RELAX(u, v, w)

● An example of Dijkstra’s
algorithm.

Teena A James, Dept. of CSE, NHCE

Teena A James, Dept. of CSE, NHCE
Teena A James, Dept. of CSE, NHCE
Teena A James, Dept. of CSE, NHCE
ANALYSIS:

Time Complexity of the implementation is O(V^2). If the input graph is represented

using adjacency list, it can be reduced to O(E log V) with the help of binary heap.

Dijkstra’s algorithm doesn’t work for graphs with negative weight edges.

KNAPSACK PROBLEM:

● The Objective is to fill a knapsack (up to its maximum capacity M) which

maximizes the total profit earned.
● Note that the value of will be any value between 0 and 1 (inclusive). If any
object is completely placed into a knapsack then its value is 1 , if we do not
pick (or select) that object to fill into a knapsack then its value is 0 .
Otherwise if we take a fraction of any object then its value will be any value
between 0 and 1.

Teena A James, Dept. of CSE, NHCE

 ● To solve this problem, Greedy method may apply any one of the following
strategies:
○ From the remaining objects, select the object with maximum profit
that fit into the knapsack.
○ From the remaining objects, select the object that has minimum
weight and also fits into knapsack.
○ From the remaining objects, select the object with maximum that fits
into the knapsack.

Example:

item weight value

1 18 $25

2 15 $24

3 10 $15

Knapsack capacity W=20

When applied all above 3 approaches on the given knapsack instance

Teena A James, Dept. of CSE, NHCE

 Thus from above all 3 approaches, it may be noticed that Greedy approaches
do not always yield an optimal solution. In such cases the greedy
method is frequently the basis of a heuristic approach. In the example
above approach3 (Selection of object in decreasing order of the ratio )
gives a optimal solution for knapsack problem.
JOB SEQUENCING WITH DEADLINES:
● The problem is stated as: There are n jobs to be processed on a machine.
Each job i has a deadline di ≥ 0 and profit pi ≥0 . Pi is earned iff the job is
completed by its deadline. The job is completed if it is processed on a
machine for unit time
● Other constraints:
Only one machine is available for processing jobs. Only one job is processed
at a time on the machin​e.
● A optimal solution is a subset of jobs J such that each job is completed by its
deadline.

Teena A James, Dept. of CSE, NHCE

EXAMPLE​:
Let n = 4, (p1 ,p2 ,p3 ,p4,p5 ) = (5,1,10,15,20), (d1 ,d2 ,d3 ,d4, d5 ) = (3,3,1,2,2)

Consider the jobs in the non increasing order of profits subject and maximum
deadline given = 3

i. Now start adding the job with J = 0 and Σ i ej Pi= 0.

Timeslot 1 2 3
Status

ii. Job 5 is added to J as it has the largest profit and thus J = {5} is a feasible one.

Timeslot 1 2 3
Status J5

iii. Now Job 4 is considered. The solution J = {4, 5} is a feasible one with
processing sequence (4, 5)

Timeslot 1 2 3
Status J4 J5

Teena A James, Dept. of CSE, NHCE

iv. The next job, Job 1 is considered. The solution J = {4, 5, 1} is a feasible one
with processing sequence (4, 5, 1).

Timeslot 1 2 3
Status J4 J5 J1

Therefore the sequence is : J4, J5, J1

Teena A James, Dept. of CSE, NHCE

II. DYNAMIC PROGRAMMING:

● Dynamic Programming is a general algorithm design technique f l i bl d fi d

b ith l i for solving problems defined by recurrences with overlapping
subproblems.
● Main idea is to set up a recurrence relating a solution to a larger instance to
solutions of some smaller instances solve smaller instances once, record
solutions in a table - extract solution to the initial instance from that table
BINOMIAL COEFFICIENT
● Binomial coefficients are represented by ​C(​ ​n​, ​k)​ or (​n​k)​ and can be used to
represent the coefficients of a binomail:
​ ​b)​ n​ ​ = ​C(​ ​n​, 0)​an​ ​ + ... + ​C​(​n,​ ​k​)a​ n​ -​ ​kbk​ + ... + ​C(​ ​n​, ​n​)​bn​
(​a +
The recursive relation is defined by the prior power
C​(​n,​ ​k)​ = ​C​(​n​-1, ​k​-1) + ​C(​ ​n​-1, ​k​) for n​ ​ > ​k​ > 0
IC ​C​(​n,​ 0) = ​C(​ ​n,​ ​n)​ = 1

Teena A James, Dept. of CSE, NHCE

 PSUEDOCODE:

Example: C (5, 2)

Your function should return 10 for n = 5 and k = 2.

It should be noted that the above function computes the same sub problems again and
again. See the following recursion tree for n = 5 and k = 2.

The function C (3, 1) is called two times. For large values of n, there will be many
common sub problems.

Teena A James, Dept. of CSE, NHCE

 C(5, 2)

/ \

C(4, 1) C(4, 2)

/ \ / \

C(3, 0) C(3, 1) C(3, 1) C(3, 2)

/ \ / \ / \

C(2, 0) C(2, 1) C(2, 0) C(2, 1) C(2, 1) C(2, 2)

/ \ / \ / \

C(1, 0) C(1, 1) C(1, 0) C(1, 1) C(1, 0) C(1, 1)

Since same sub problems are called again, this problem has Overlapping Sub problems
property. So the Binomial Coefficient problem has both properties of a dynamic
programming problem and the ​Time Complexity: O (n*k)

WARSHALL’S ALGORITHM : TRANSITIVE CLOSURE OF A GRAPH

Given a directed graph, find out if a vertex j is reachable from another vertex i for all
vertex pairs (i, j) in the given graph. Here reachable mean that there is a path from vertex
i to j. The reach-ability matrix is called transitive closure of a graph.

Consider below graph

Transitive closure of above graphs is

Teena A James, Dept. of CSE, NHCE

 1111

1111

1111

0001

On the kth iteration, the algorithm determine if a path exists between two vertices i, j
using just vertices among 1,…,k allowed as intermediate

PSUEDOCODE​:

Recurrence Relation:

R(k) [i,j] = R(k-1)[i,j] or (R(k-1)[i,k] and R(k-1)[k,j])

Rules for generating R (k) from R(k-1):

Teena A James, Dept. of CSE, NHCE

Rule 1- If an element in row i and column j is 1 in R (k-1), it remains 1 in R (k)

Rule 2- If an element in row i and column j is 0 in R (k-1) , it has to be changed to 1 in

R(k) it has to be changed to 1 in R if and only if (k) if and only if the element in its row i
and column k and the element in its column j and row k are both 1’s in R(k-1)

Example:

Teena A James, Dept. of CSE, NHCE

FLOYD’S ALGORITHM:

Used to find shortest paths in a weighted graph

• Travel maps containing driving distance from one point to another – Represented by
tables – Shortest distance from point A to point B given by intersection of row and
column – Route may pass through other cities represented in the table

• Navigation systems All-pairs shortest-path problem

• Graph G = (V, E) Weighted and directed graph

• Problem: Find the length of the shortest path between every pair of vertices – Length
of the path is strictly determined by the weight of its edges – It is not based on the
number of edges traversed

• Representation of weighted directed graph by adjacency matrix – n × n matrix for a

graph with n vertices – adjacency matrix ∗ Chosen for constant time access to every edge
– Nonexistent edges may be assigned a value​ ∞

Teena A James, Dept. of CSE, NHCE

PSUEDOCODE:

EXAMPLE:

a. Given Di-Graph

b. Weighted Matrix

c. Final Distance Matrix(D​4​)

Teena A James, Dept. of CSE, NHCE

Teena A James, Dept. of CSE, NHCE