BCS401-Module1 | Anooplal KS

1.1 What Is an Algorithm?

Although there is no universally agreed-on wording to describe this notion, there is general
agreement about what the concept means: An algorithm is a sequence of unambiguous
instructions for solving a problem, i.e., for obtaining a required output for any legitimate
input in a finite amount of time.

Characteristics of an algorithm:
Input: Zero / more quantities are externally supplied.
Output: At least one quantity is produced.
Definiteness: Each instruction is clear and unambiguous.
Finiteness: If the instructions of an algorithm is traced then for all cases the algorithm must
terminates after a finite number of steps.
Efficiency: Every instruction must be very basic and runs in short time.

Euclids algorithm for gcd(m,n)

Algorithm:Euclid’s algorithm for computing gcd(m, n)
Step 1: If n 0, return the value of m as the answer and stop;
otherwise, proceed to Step 2.
Step 2: Divide m by n and assign the value of the remainder to r.
Step 3: Assign the value of n to m and the value of r to n. Go to
Step 1.
Pseudocode
• ALGORITHM: Euclid(m, n)
• Input: Two nonnegative, not-both-zero integers m and n
• Output: Greatest common divisor of m and n
while n /= 0 do
r ← m mod n
m←n
n←r
return m

Consecutive integer checking algorithm for computing gcd(m, n)

Step 1 Assign the value of min{m, n} to t.
Step 2 Divide m by t. If the remainder of this division is 0, go to Step 3;otherwise, go to Step
4.

1
 BCS401-Module1 | Anooplal KS

Step 3 Divide n by t. If the remainder of this division is 0, return the value of t as the answer
and stop; otherwise, proceed to Step 4.
Step 4 Decrease the value of t by 1. Go to Step 2
unlike Euclid’s algorithm, this algorithm, in the form presented, does not work correctly
when one of its input numbers is zero.

Middle-school procedure for computing gcd(m, n)

Step 1 Find the prime factors of m.
Step 2 Find the prime factors of n.
Step 3 Identify all the common factors in the two prime expansions found in Step 1 and Step
2. (If p is a common factor occurring pm and pn times in m and n, respectively, it should be
repeated min{pm, pn} times.)
Step 4 Compute the product of all the common factors and return it as the greatest common
divisor of the numbers given.

Thus, for the numbers 60 and 24, we get 60 = 2 . 2 . 3 . 5

24 = 2 . 2 . 2 . 3
gcd(60, 24) = 2 . 2 . 3 = 12.

Algorithm for generating consecutive primes not exceeding any given

integer n> 1.
The algorithm starts by initializing a list of prime candidates with consecutive integers from 2
to n. Then,on its first iteration, the algorithm eliminates from the list all multiples of 2, i.e., 4,6,
and so on. Then it moves to the next item on the list, which is 3, and eliminates its multiples.
(In this straightforward version, there is an overhead because some numbers, such as 6, are
eliminated more than once.) No pass for number 4 is needed: since 4 itself and all its multiples
are also multiples of 2, they were already eliminated on a previous pass. The next remaining
number on the list, which isused on the third pass, is 5. The algorithm continues in this fashion
until no more numbers can be eliminated from the list. The remaining integers of the list are
the primes needed. As an example, consider the application of the algorithm to finding the list
of primes not exceeding n = 25:

2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25
2 3 5 7 9 11 13 15 17 19 21 23 25
2 3 5 7 11 13 17 19 23 25
2 3 5 7 11 13 17 19 23

ALGORITHM Sieve(n)
//Implements the sieve of Eratosthenes
//Input: A positive integer n > 1
//Output: Array L of all prime numbers less than or equal to n
for p→2 to n do A[p ] ←p
for p ← 2 to⌊ √n⌋ do //see note before pseudocode
if A[p]≠ 0 //p hasn’t been eliminated on previous passes
j←p∗p
while j ≤ n do
A[j] ← 0 //mark element as eliminated

2
 BCS401-Module1 | Anooplal KS

j→j+p
//copy the remaining elements of A to array L of the primes
i ←0
for p← 2 to n do
if A[p] ≠ 0
L[i] ← A[p]
i←i+1
return L

1.2 Fundamentals of Algorithmic Problem Solving:

Following are the steps in designing and analysing an algorithm:

1. Understanding the Problem: The first thing you need to do before designing an
algorithm is to understand completely the problem given. Read the problem’s
description carefully and ask questions if you have any doubts about the problem, An
input to an algorithm specifies an instance of the problem the algorithm solves. It is
very important to specify exactly the set of instances the algorithm needs to handle.
2. Ascertaining the Capabilities of the Computational Device:
Ascertain the capability of a computational device
• How the objects would be represented
Is it RAM model or concurrent execution
• How the operations would be implemented?
Is it sequential algorithm or parallel algorithm
3. Choosing between Exact and Approximate Problem Solving
The next principal decision is to choose between solving the problem exactly or
solving it approximately. In the formeru case, an algorithm is called an exact
algorithm; in the latter case, an algorithm is called an approximation algorithm.

3
 BCS401-Module1 | Anooplal KS

Why would one opt for an approximation algorithm?

• First, there are important problems that simply cannot be solved exactly for
most of their instances; examples include extracting square roots, solving
nonlinear equations, and evaluating definite integrals
• Second, available algorithms for solving a problem exactly can be
unacceptably slow because of the problem’s intrinsic complexity..
4. Algorithm Design Techniques:
An algorithm design technique (or “strategy” or “paradigm”) is a general approach to
solving problems algorithmically that is applicable to a variety of problems from
different areas of computing
• First, they provide guidance for designing algorithms for new problems.
• Second, algorithms are the cornerstone of computer science. Algorithm design
techniques make it possible to classify algorithms according to an underlying
design idea
5. Designing an Algorithm and Data Structures
designing an algorithm for a particular problem may still be a challenging task. Some
design techniques can be simply inapplicable to the problem in question. Sometimes,
several techniques need to be combined, and there are algorithms that are hard to
pinpoint as applications of the known design techniques: Algorithms + Data
Structures = Programs
one should pay close attention to choosing data structures appropriate for the
operations performed by the algorithm.
6. Methods of Specifying an Algorithm
These are the two options that are most widely used nowadays for specifying
algorithms. Using a natural language has an obvious appeal; however, the inherent
ambiguity of any natural language makes a succinct and clear description of
algorithms surprisingly difficult. Nevertheless, being able to do this is an important
skill that you should strive to develop in the process of learning algorithms.
Pseudocode is a mixture of a natural language and programming languagelike
constructs
7. Proving an Algorithm’s Correctness .Once an algorithm has been specified, you
have to prove its correctness. That is, you have to prove that the algorithm yields a
required result for every legitimate input in a finite amount of time.
8. Analyzing an Algorithm:
After correctness, by far the most important is efficiency.
time efficiency, indicating how fast the algorithm runs, and space efficiency,
indicating how much extra memory it uses. Yet another desirable characteristic of an
algorithm is generality. Range of inputs the algorithm works. Optimality-No other
algorithm can do better
9. Coding an Algorithm:
Most algorithms are destined to be ultimately implemented as computer programs.
Programming an algorithm presents both a peril and an opportunity. The peril lies in
the possibility of making the transition from an algorithm to a program either
incorrectly or very inefficiently. Some influential computer scientists strongly believe
that unless the correctness of a computer program is proven with full mathematical
rigor, the program cannot be considered correct.

4
 BCS401-Module1 | Anooplal KS

1.3 Fundamentals of the Analysis of Algorithm

Efficiency
The Analysis Framework
It is a general framework for analysing the efficiency of algorithms. There are two kinds of
efficiency: time efficiency and space efficiency.
The algorithm analysis framework consists of the following:
1. Measuring an Input’s Size
2. Units for Measuring Running Time
3. Orders of Growth

4. Worst-Case, Best-Case, and Average-Case Efficiencies

i) Measuring an Input’s Size:

• An algorithm’s efficiency is defined as a function of some parameter n
indicating the algorithm’s input size. In most cases, selecting such a parameter
is quite straightforward. For example, it will be the size of the list for problems
of sorting, searching.
• For the problem of evaluating a polynomial p(x) = anx n + . . . + a0 of degree n,
the size of the parameter will be the polynomial’s degree or the number of its
coefficients, which is larger by 1 than its degree.
• In computing the product of two n × n matrices, the choice of a parameter
indicating an input size does matter.
• Consider a spell-checking algorithm. If the algorithm examines individual
characters of its input, then the size is measured by the number of characters.
• In measuring input size for algorithms solving problems such as checking
primality of a positive integer n. the input is just one number.
• The input size by the number b of bits in the n’s binary representation is b=(log2
n)+1.

ii) Units for Measuring Running Time

• Some standard unit of time measurement such as a second, or millisecond, and so
on can be used to measure the running time of a program after implementing the
algorithm.
• Drawbacks
o Dependence on the speed of a particular computer.
o Dependence on the quality of a program implementing the algorithm.
o The compiler used in generating the machine code.
o The difficulty of clocking the actual running time of the program.

5
 BCS401-Module1 | Anooplal KS

So, we need metric to measure an algorithm’s efficiency that does not depend on these
extraneous factors. One possible approach is to count the number of times each of the
algorithm’s operations is executed. This approach is excessively difficult. The most important
operation (+, -, *, /) of the algorithm, called the basic operation. Computing the number of
times the basic operation is executed is easy. The total running time is determined by basic
operations count.

iii) Orders of Growth

• A difference in running times on small inputs is not what really distinguishes
efficient algorithms from inefficient ones.
• For example, the greatest common divisor of two small numbers, it is not
immediately clear how much more efficient Euclid’s algorithm is compared to
the other algorithms, the difference in algorithm efficiencies becomes clear for
larger numbers only.
• For large values of n, it is the function’s order of growth that counts just like the
Table 1.1, which contains values of a few functions particularly important for
analysis of algorithms.
n log2 n n n log2 n n 2 n3 2n n!
1 1 2 3
10 3.3 10 3.3*10 10 10 103 3.6*106
102 6.6 102 6.6*102 104 106 1.3*1030 9.3*10157
3 3 3 6 9
10 10 10 1.0*10 10 10
4 4 5 8
10 13 10 1.3*10 10 1012
5 5 6
10 17 10 1.7*10 1010 1015
6 6 7 12
10 20 10 2.0*10 10 1018

iv) Worst-Case, Best-Case, and Average-Case Efficiencies:

Consider Sequential Search algorithm some search key K

6
 BCS401-Module1 | Anooplal KS

Worst-case efficiency : The worst-case efficiency of an algorithm is its efficiency for the
worst case input of size n. The algorithm runs the longest among all possible inputs of that
size. For the input of size n, the running time is Cworst(n) = n.
Best case efficiency :The best-case efficiency of an algorithm is its efficiency for the best
case input of size n.
• The algorithm runs the fastest among all possible inputs of that size n.
• In sequential search, If we search a first element in list of size n. (i.e. first element
equal to a search key), then the running time is Cbest(n) = 1
Average case efficiency
• The Average case efficiency lies between best case and worst case.
• To analyze the algorithm’s average case efficiency, we must make some
assumptions about possible inputs of size n.
• The standard assumptions are that
o The probability of a successful search is equal to p (0 ≤ p ≤ 1) and
o The probability of the first match occurring in the ith position of the list is
the same for every i.

1.4 Asymptotic Notations and Basic Efficiency

Classes
The efficiency analysis framework concentrates on the order of growth of an algorithm’s
basic operation count as the principal indicator of the algorithm’s efficiency. To compare and
rank such orders of growth, computer scientists use three notations:
1. O(big oh),
2. Ω(big omega),
3. Θ (big theta) and
4. o(little oh)

7
 BCS401-Module1 | Anooplal KS

1. Big-Oh notation
Definition: A function t(n) is said to be in O(g(n)), denoted t(n)∈O(g(n)), if t (n) is bounded
above by some constant multiple of g(n) for all large n, i.e., if there exist some positive
constant c and some nonnegative integer n0 such that
t(n) ≤ c g(n) for all n ≥ n0

Example:1

Example2:

2. Ω - Big omega notation

A function t(n) is said to be in Ω(g(n)), denoted t(n) ∈ Ω(g(n)), if t(n) is bounded below by
some positive constant multiple of g(n) for all large n, i.e., if there exist some positive
constant c and some nonnegative integer n0 such that
t (n) ≥ cg(n) for all n ≥ n0.
Where t(n) and g(n) are nonnegative functions defined on the set of natural numbers.

8
 BCS401-Module1 | Anooplal KS

Ω = Asymptotic lower bound = Useful for best case analysis = Loose bound

Example:
Prove the assertions: n3 +10n2 +4n+2 ∈ Ω (n2 ).
Proof: n3 +10n2 +4n+2 ≥ n2 (for all n ≥ 0)
i.e., by definition t(n) ≥ cg(n), where c=1 and n0=0

(iii) Θ - Big theta notation

A function t(n) is said to be in Θ(g(n)), denoted t(n) ∈ Θ(g(n)), if t(n) is bounded both above
and below by some positive constant multiples of g(n) for all large n, i.e., if there exist some
positive constants c1 and c2 and some nonnegative integer n0 such that

c2g(n) ≤ t (n) ≤ c1g(n) for all n ≥ n0.

Where t(n) and g(n) are nonnegative functions defined on the set of natural numbers.
Θ = Asymptotic tight bound = Useful for average case analysis

9
 BCS401-Module1 | Anooplal KS

Useful Property Involving the Asymptotic Notations

Using Limits for Comparing Orders of Growth

method for comparing the orders of growth of two specific functions

10
 BCS401-Module1 | Anooplal KS

1.5 Basic efficiency classes

Class Name Description

1 constant The algorithm takes the same amount of time to complete, no
matter how big the input is.
log n logarithmic Running time is logarithmic
Problem size is reduced by constant factor in each iteration
Eg. Binary search
n linear Algorithms that scan a list of size n (e.g., sequential search)
belong to this class. Running time is linear
n * log n Linear Many divide-and-conquer algorithms , including mergesort and
arithmatic quicksort in the average case, fall into this category.
2
quadratic Typically, characterizes efficiency of algorithms with two
n
embedded loops. Elementary sorting algorithms and certain
operations on n x n matrices are standard examples.
3
cubic typically, characterizes efficiency of algorithms with three
n
embedded loops. Several nontrivial algorithms from linear
algebra fall into this class
n
2 exponential Typical for algorithms that generate all subsets of an n-element
set. Often. the term “exponential” is used in a broader sense to
include this and larger orders Ofgrowth as well.
n! factorial Typical for algorithms that generate all permutations of an n-
element set.

11
 BCS401-Module1 | Anooplal KS

1.6 Mathematical Analysis of Non recursive

Algorithms
General Plan for Analyzing the Time Efficiency of Nonrecursive Algorithms
1. Decide on a parameter (or parameters) indicating an input’s size.

2. Identify the algorithm’s basic operation (in the innermost loop).

3. Check whether the number of times the basic operation is executed depends only on the
size of an input. If it also depends on some additional property, the worst-case, average-case,
and, if necessary, best-case efficiencies have to be investigated separately.
4. Set up a sum expressing the number of times the algorithm’s basic operation is executed.

5. Using standard formulas and rules of sum manipulation either find a closed form
formula for the count or at the least, establish its order of growth.
Consider the problem of finding the value of the largest element in a list of n numbers.
Assume that the list is implemented as an array for simplicity.
ALGORITHM :MaxElement(A[0..n − 1])
//Determines the value of the largest element in a given array
//Input: An array A[0..n − 1] of real numbers
//Output: The value of the largest element in A
maxval ←A[0]
for i ←1 to n − 1 do
if A[i]>maxval
maxval←A[i]
return maxval
Algorithm analysis
• The measure of an input’s size. here is the number of elements in the array, i.e., n.
• There are two operations in the for loop’s body:
• The comparison A[i]> maxval and
• The assignment maxval←A[i].
• Therefore the basic operation is comparison
• The number of comparisons will be the same for all arrays of size n; therefore, there is
no need to distinguish among the worst, average, and best cases here.
• Let C(n) denotes the number of times this comparison is executed. The algorithm
makes one comparison on each execution of the loop, which is repeated for each
value of the loop’s variable i within the bounds 1 and n- 1, inclusive.

12
 BCS401-Module1 | Anooplal KS

EXAMPLE 2: Consider the element uniqueness problem: check whether all the
Elements in a given array of n elements are distinct.

• The natural measure of the input’s size here is again n (the number of elements in the
array).
• the comparison of two elements is the algorithm’s basic operation.
• The number of element comparisons depends not only on n but also on whether there
are equal elements in the array and, if there are, which array positions they occupy.
We will limit our investigation to the worst case only.
• One comparison is made for each repetition of the innermost loop, i.e., for each value
of the loop variable j between its limits i + 1 and n − 1; this is repeated for each value
of the outer loop, i.e., for each value of the loop variable i between its limits 0 and n −
2.

13
 BCS401-Module1 | Anooplal KS

Example 3:The following algorithm finds the number of binary digits in the binary
representation of a positive decimal integer.

Since the value of n is about halved on each repetition of the loop, the answer should be
about log2 n. The exact formula for the number of times the comparison n> 1 will be
executed is actually log2 n +1
Example4: Given two n n matrices A and B, find the time efficiency of the definition-based
algorithm for computing their product C AB. By definition, C is an n *n matrix whose
elements are computed as the scalar (dot) products of the rows of matrix A and the columns
of matrix B:

Efficiency:

14
 BCS401-Module1 | Anooplal KS

• The total number of multiplications M(n) is expressed by the following triple sum:

The algorithm computes n2 elements of the product matrix. Each of the product’s elements is
computed as the scalar (dot) product of an n-element row of the first matrix and an n-element
column of the second matrix, which takes n multiplications. So the total number of
multiplications is n . n2 n3. To estimate the running time of the algorithm on a particular
machine,

where cm is the time of one multiplication on the machine.Total time taken is given by

where ca is the time of one addition.

1.7 Mathematical Analysis of Recursive Algorithms

General Plan for Analyzing the Time Efficiency of Recursive Algorithms

1. Decide on a parameter (or parameters) indicating an input’s size.

2. Identify the algorithm’s basic operation.
3. Check whether the number of times the basic operation is executed can vary on different
inputs of the same size; if it can, the worst-case,average-case, and best-case efficiencies
must be investigated separately.
4. Set up a recurrence relation, with an appropriate initial condition, for the number of times
the basic operation is executed.
5. Solve the recurrence or, at least, ascertain the order of growth of its solution. Compute the
factorial function F(n) = n! for an arbitrary nonnegative integer n.

15
 BCS401-Module1 | Anooplal KS

Example:Compute the factorial function F(n) = n! for an arbitrary nonnegative integer

n.
• Since n!= 1*. ...... * (n − 1) * n = (n − 1)! * n, for n ≥ 1 and 0!= 1 by definition, we can
compute F(n) = F(n − 1) * n with the following recursive algorithm.

Algorithm analysis
• For simplicity, we consider n itself as an indicator of this algorithm’s input size. i.e. 1.
• The basic operation of the algorithm is multiplication, whose number of executions
we denote M(n). M(0)=0(no multiplications done)

• Since the function F(n) is computed according to formula F(n) = F(n −1)*n for n > 0.
F(0)=1.

EXAMPLE 2: consider educational workhorse of recursive algorithms: the Tower of

Hanoi puzzle.
• We have n disks of different sizes that can slide onto any of three pegs. Consider A
(source), B (auxiliary), and C (Destination). Initially, all the disks are on the first peg
in order of size, the largest on the bottom and the smallest on top. The goal is to move
all the disks to the third peg, using the second one as an auxiliary.

16
 BCS401-Module1 | Anooplal KS

Algorithm analysis
• The number of moves M(n) depends on n only, and we get the following recurrence
equation for it:
M(n) = M(n − 1) + 1+ M(n − 1) for n > 1.
• With the obvious initial condition M(1) = 1, we have the following recurrence relation
for the number of moves
M(n): M(n) = 2M(n − 1) + 1 for n > 1, M(1) = 1.
EXAMPLE 3:A recursive version of the algorithm for binary representation of positive
decimal number
ALGORITHM: BinRec(n)
//Input: A positive decimal integer n
//Output: The number of binary digits in n’s binary representation
if n = 1 return 1
else return BinRec( n/2]) + 1
A(n) = A( n/2]) + 1 for n> 1. (2.4) Since the recursive calls end when n is equal to 1 and there are
no additions made then, the initial condition is A(1) = 0.

A(2k) = A(2k−1) + 1 for k> 0, A(20) = 0. Now backward substitutions

17
 BCS401-Module1 | Anooplal KS

Now backward substitutions encounter no problems:

A(2k) = A(2k−1) + 1 substitute A(2k−1) = A(2k−2) + 1

= [A(2k−2) + 1] + 1 = A(2k−2) + 2 substitute A(2k−2) = A(2k−3) + 1

= [A(2k−3) + 1] + 2 = A(2k−3) + 3

... . . .

Thus, we end up with

= A(2k−i) + i

= A(2k−k) + k.

A(2k) = A(1) + k = k, or, after returning to the original variable n = 2k and hence k = log2 n,

A(n) = log2 n ∈ Θ (log n).

1.8 Brute Force Approaches

In this section, we consider the application of the brute-force approach to the problem of
sorting: given a list of n orderable items (e.g., numbers, characters from some alphabet,
character strings), rearrange them in nondecreasing order.

Selection Sort
We start selection sort by scanning the entire given list to find its smallest element and exchange it
with the first element, putting the smallest element in its final − position in the sorted list. Then we
scan the list, starting with the second element, to find the smallest among the last n 1 elements and
exchange it with the second element, putting the second smallest element in its final
position.Generally the algorithm searches for the smallest item among the last n − i elements and
swaps it with Ai :

ALGORITHM SelectionSort(A[0..n − 1])

//Sorts a given array by selection sort

//Input: An array A[0..n − 1] of orderable elements

//Output: Array A[0..n − 1] sorted in nondecreasing order

for i ← 0 to n − 2 do
min←i
for j← i +1 to n -1 do
if A[j ] < A[min]
min← j
swap A[i] and A[min]

18
 BCS401-Module1 | Anooplal KS

| 89 45 68 90 29 34 17
17 | 45 68 90 29 34 89
17 29 | 68 90 45 34 89
17 29 34 | 90 45 68 89
17 29 34 45 | 90 68 89
17 29 34 45 68 | 90 89
17 29 34 45 68 89 | 90

Thus,selection sort is a Θ(n2) algorithm on all inputs

Bubble sort algorithm

Here we end up “bubbling up” the largest element to the last position on the list. The next pass
bubbles up the second largest element, and so on, until after n 1 passes the list is sorted.

19
 BCS401-Module1 | Anooplal KS

Linear/sequential search(discussed above)

Advantages of Linear Search:

• Linear search can be used irrespective of whether the array is sorted or not. It can be used
on arrays of any data type.

• Does not require any additional memory.

• It is a well-suited algorithm for small datasets.

Disadvantages of Linear Search:

• Linear search has a time complexity of O(N)(Worst Case), which in turn makes it slow for
large datasets.

• Not suitable for large arrays.

Brute-Force String Matching

Brute-force: Scan text LR, compare chars, looking for pattern,

Step 1 Align pattern at beginning of text

Step 2 Moving from left to right, compare each character of pattern to the corresponding character
in text until

- all characters are found to match (successful search);

- or a mismatch is detected
Step 3 While pattern is not found and the text is not yet exhausted, realign pattern one position to
the right and repeat Step 2

ALGORITHM: BruteForceStringMatch(T [0..n − 1],P [0..m − 1])

//Implements brute-force string matching

//Input: An array T [0..n 1] of n characters representing a text andan array P [0..m 1]
of m characters representing a pattern
//Output: The index of the first character in the text that starts a matching substring
or − 1 if the search is unsuccessful
for i ← 0 to n − m do
j← 0
while j < m and P [j] = T [i + j] do
j←j+1
if j = m return I
return − 1

• the algorithm shifts the pattern almost always after a single character comparison.
• The worst case is much worse:

20
 BCS401-Module1 | Anooplal KS

• the algorithm may have to make all m comparisons before shifting the pattern,
and this can happen for each of the m(n-m+1) tries.Hence O(nm)

21