BCS401 ADA Notes

MODULE-1 INTRODUCTION

1.1 WHAT IS AN ALGORITHM?

An algorithm is a sequence of unambiguous instructions for solving a problem, i.e., for
obtaining a required output for any legitimate input in a finite amount of time. In addition, all
algorithms must satisfy the following criteria:
1. Input: Zero or more quantities are externally supplied.
2. Output: At least one quantity is produced.
3. Definiteness: Each instruction is clear and unambiguous.
4. Finiteness: If we trace out the instructions of an algorithm, then for all cases, the
algorithm terminates after a finite number of steps.
5. Effectiveness: Every instruction must be very basic so that it can be carried out, in
principle, by a person using only pencil and paper. It is not enough that each operation
be definite as in criterion3; it also must be feasible

The study of algorithms includes many important and active areas of research. There are four
distinct areas of study one can identify:
1. How to devise algorithms: The act of creating an algorithm is an art which may
never be fully automated. A major goal is to study various design techniques which
have proven to be useful in that they have often yielded good algorithms.

2. How to validate algorithms: Once an algorithm is devised it is necessary to show

that it computes the correct answer for all possible legal inputs. We refer to this
process as algorithm validation. The purpose of the validation is to assure us that this
algorithm will work correctly independent of the issues concerning the programming
language it will eventually be written in. Once the validity of the method has been
shown, a program can be written and a second phase begins. This phase is referred to
as program proving or sometimes as program verification.

3. How to analyze algorithms: This field of study is called analysis of algorithms. As

an algorithm is executed, it uses the computer's central processing unit (CPU) to
perform operations and its memory to hold the program and data. Analysis of
algorithms or performance analysis refers to the task of determining how much
computing time and storage an algorithm requires. This is a challenging area which
sometimes requires great mathematical skill. An important result of this is that it
allows you to predict whether the software will meet any efficiency constraints that
exist.

4. How to test a program: Testing a program really consists of two phases: debugging
and profiling. Debugging is the process of executing programs on sample data sets to

Dept. of CSE, HKBKCE 1 2023-24

BCS401 ADA Notes
determine if faulty results occur and, if so, to correct them. Profiling is the process of
executing a correct program on data sets and measuring the time and space it takes to
compute the results.

1.2 ALGORITHM SPECIFICATION

1.2.1 Pseudocode Conventions

1. Comments begin with // and continue until the end of line.

2. Blocks are indicated with matching braces :{ and}. A compound statement (i.e., a
collection of simple statements) can be represented as a block. Statements are
delimited by ;.

3. An identifier begins with a letter. The data types of variables are not explicitly
declared. The types will be clear from the context. Whether a variable is global or
local to a procedure will also be evident from the context. We assume simple data
types such as integer, float, char,boolean, and so on. Compound data types can be
formed with records. Here is an example:
node= record
{
data type_1 data 1;
.
.
data type _n data_n;
node *link;
}

4. Assignment of values to variables is done using the assignment statement

(variable) (expression);

5. There are two Boolean values true and false. In order to produce these values, the
logical operators and, or, and not and the relational operators<, <, <=and >= are
provided.

6. Elements of multidimensional arrays are accessed using [and]. For example, if A is a

two dimensional array, the (i,j)th element of the array is denoted as A[i,j].

7. The following looping statements are employed: for, while, and repeat-until. The
while loop takes the following form:
while (condition)do
{
(statement 1)
.
.
(statement n)
}
As long as (condition) is true, the statements get executed. When (condition) becomes
false, the loop is exited. The value of (condition) is evaluated at the top of the loop.
The general form of a for loop is
for variable value l to value n do
{

Dept. of CSE, HKBKCE 2 2023-24

BCS401 ADA Notes
(statement 1)
.
.
(statement n)
}
A repeat-until statement is constructed as follows:
repeat
(statement 1)
.
.
(statement n)
until (condition)

The statements are executed as long as {condition) is false. The value of {condition)is
computed after executing the statements.

Dept. of CSE, HKBKCE 3 2023-24

 BCS401 ADA Notes

8. A conditional statement has the following forms:

if (condition)then(statement)
if (condition)then(statement1)else(statement2)
Here (condition) is a boolean expression and (statement), (statement1) and (statement2)
are arbitrary statements (simple or compound)

9. Input and output are done using the instructions read and write. No format is used to
specify the size of input or output quantities.

10. There is only one type of procedure: Algorithm. An algorithm consists of a heading
and a body. The heading takes the form
AlgorithmName((parameterlist))
where Name is the name of the procedure and ((parameter list)) is a listing of the
procedure parameters.

Example: Illustrating the notion of algorithm, we consider three methods for solving
the same problem: computing the greatest common divisor of two integers.

Euclid’s Algorithm:

Dept. of CSE, HKBKCE 4 2023-24

BCS401 ADA Notes

FUNDAMENTALS OF ALGORITHMIC PROBLEM SOLVING

We can consider algorithms to be procedural solutions to problems. These solutions are not
answers but specific instructions for getting answers. It is this emphasis on precisely defined
constructive procedures that makes computer science distinct from other disciplines.
The sequence of steps one typically goes through in designing and analyzing an algorithm
(Figure 1.2).

Dept. of CSE, HKBKCE 5 2023-24

BCS401 ADA Notes

Understanding the Problem:

The first thing you need to do before designing an algorithm is to understand completely the
problem given. Read the problem's description carefully and ask questions if you have any
doubts about the problem, do a few small examples by hand, think about special cases, and
ask questions again if needed.
There are a few types of problems that arise in computing applications quite often. An
input to an algorithm specifies an instance of the problem the algorithm solves. It is very
important to specify exactly the range of instances the algorithm needs to handle. (As an
example, recall the variations in the range of instances for the three greatest common divisor
algorithms.) If you fail to do this, your algorithm may work correctly for a majority of inputs
but crash on some "boundary" value. Remember that a correct algorithm is not one that works
most of the time, but 'One that works correctly for all legitimate inputs.

Ascertaining the Capabilities of a Computational Device:

The vast majority of algorithms in use today are still destined to be programmed for a
computer closely resembling the von Neumann machine. The essence of this architecture is
captured by the so-called random-access machine. Its central assumption is that instructions
are executed one after another, one operation at a time. Accordingly, algorithms designed to be
executed on such machines are called sequential algorithms. The central assumption of the
RAM model does not hold for some newer computers that can execute operations concurrently,
i.e., in parallel. Algorithms that take advantage of this capability are called parallel algorithms.
It is imperative to be aware of the speed and memory available on a particular computer

Dept. of CSE, HKBKCE 6 2023-24

BCS401 ADA Notes
system.
Choosing between Exact and Approximate Problem Solving:
The next principal decision is to choose between solving the problem exactly or solving it
approximately. In the former case, an algorithm is called an exact algorithm; in the latter case,
an algorithm is called an approximation algorithm. First, there are important problems that
simply cannot be solved exactly for most of their instances; examples include extracting square
roots, solving nonlinear equations, and evaluating definite integrals. Second, available
algorithms for solving a problem exactly can be unacceptably slow because of the problem's
intrinsic complexity. Third, an approximation algorithm can be a part of a more sophisticated
algorithm that solves a problem exactly.
Deciding on Appropriate Data Structures:
The fundamental importance of both algorithms and data structures for computer programming
by its very title:
Algorithms + Data Structures = Programs.
In the new world of object-oriented programming, data structures remain crucially important
for both design and analysis of algorithms.
Algorithm Design Techniques:
An algorithm design technique (or "strategy" or "paradigm") is a general approach to solving
problems algorithmically that is applicable to a variety of problems from different areas of
computing.
Methods of Specifying an Algorithm:
These are the two options that are most widely used nowadays for specifying algorithms. Using
a natural language has an obvious appeal; however, the inherent ambiguity of any natural
language makes a succinct and clear description of algorithms surprisingly difficult. A
pseudocode is a mixture of a natural language and programming language like constructs. A
pseudocode is usually more precise than a natural language, and its usage often yields more
succinct algorithm descriptions. The state of the art of computing has not yet reached a point
where an algorithm's description-whether in a natural language or a pseudocode-can be fed into
an electronic computer directly. Instead, it needs to be converted into a computer program
written in a particular computer language. We can look at such a program as yet another way of
specifying the algorithm, although it is preferable to consider it as the algorithm's
implementation.
Proving an Algorithm's Correctness:
Once an algorithm has been specified, you have to prove its correctness. That is, you have to
prove that the algorithm yields a required result for every legitimate input in a finite amount of
time. For some algorithms, a proof of correctness is quite easy; for others, it can be quite
complex. A common technique for proving correctness is to use mathematical induction
because an algorithm's iterations provide a natural sequence of steps needed for such proofs.
Analyzing an Algorithm:
We usually want our algorithms to possess several qualities. After correctness, by far the most
important is efficiency. In fact, there are two kinds of algorithm efficiency: time efficiency and
space efficiency. Time efficiency indicates how fast the algorithm runs; space efficiency
indicates how much extra memory the algorithm needs. Another desirable characteristic of an

Dept. of CSE, HKBKCE 7 2023-24

BCS401 ADA Notes
algorithm is simplicity. Unlike efficiency, which can be precisely defined and investigated with
mathematical rigor, simplicity, like beauty, is to a considerable degree in the eye of the beholder.
Yet another desirable characteristic of an algorithm is generality. There are, in fact, two issues
here: generality of the problem the algorithm solves and the range of inputs it accepts.
Coding an Algorithm:
Most algorithms are destined to be ultimately implemented as computer programs.
Programming an algorithm presents both a peril and an opportunity. The peril lies in the
possibility of making the transition from an algorithm to a program either incorrectly or very
inefficiently. Some influential computer scientists strongly believe that unless the correctness of
a computer program is proven with full mathematical rigor, the program cannot be considered
correct.
In conclusion,
let us emphasize again the main lesson of the process depicted in Figure 1.2:
As a rule, a good algorithm is a result of repeated effort and rework.
Even if you have been fortunate enough to get an algorithmic idea that seems perfect, you
should still try to see whether it can he improved.

THE ANALYSIS FRAMEWORK

There are two kinds of efficiency:

1. Time efficiency: Time efficiency, also called time complexity, indicates how fast an
algorithm in question runs.
2. Space efficiency: Space efficiency, also called space complexity, refers to the amount
of memory units required by the algorithm in addition to the space needed for its input
and output.

Measuring an Input’s Size: It is preferable to measure size by the number b of bits in the n’s
binary representation:

Units for Measuring Running Time:

Let cop be the execution time of an algorithm’s basic operation on a particular computer, and
let C(n) be the number of times this operation needs to be executed for this algorithm. Then
we can estimate the running time T (n) of a program implementing this algorithm on that
computer by the formula

Orders of Growth
A difference in running times on small inputs is not what really distinguishes efficient
algorithms from inefficient ones. For large values of n, it is the function’s order of growth
that counts: just look at Table 1.1, which contains values of a few functions particularly
important for analysis of algorithms.

Dept. of CSE, HKBKCE 8 2023-24

BCS401 ADA Notes

TABLE 1.1: Values (some approximate) of several functions important for

analysis of algorithms

TABLE 1.2: Basic asymptotic efficiency classes

Worst-Case, Best-Case, and Average-Case Efficiencies
There are many algorithms for which running time depend not only on an input size but also
on the specifics of a particular input. Consider, as an example, sequential search. This is a

Dept. of CSE, HKBKCE 9 2023-24

BCS401 ADA Notes
straightforward algorithm that searches for a given item (some search key K) in a list of n
elements by checking successive elements of the list until either a match with the search key
is found or the list is exhausted. Here is the algorithm’s pseudocode, in which, for simplicity,
a list is implemented as an array.

ALGORITHM SequentialSearch(A[0..n − 1], K)

//Searches for a given value in a given array by sequential search
//Input: An array A[0..n − 1] and a search key K
//Output: The index of the first element in A that matches K or −1 if there are
no matching elements
for i ←0 to n-1 do
if (key=A[i]) do
return i
else
return −1

Algorithm efficiency depends on the input size n. And for some algorithms efficiency
depends on type of input. We have best, worst & average case efficiencies.

Worst-case efficiency: Efficiency (number of times the basic operation will be

executed) for the worst case input of size n. i.e. The algorithm runs the longest
among all possible inputs of size n. The algorithm makes the largest number of key
comparisons among all possible inputs of size n:
Cworst(n) = n.

Best-case efficiency: Efficiency (number of times the basic operation will be

executed) for the best case input of size n. i.e. The algorithm runs the fastest among
all possible inputs of size n. The best-case inputs for sequential search are lists of size
n with their first element equal to a search key; accordingly, Cbest(n) = 1 for this
algorithm.

Average-case efficiency: Average time taken (number of times the basic operation
will be executed) to solve all the possible instances (random) of the input. In the
case of a successful search, the probability of the first match occurring in the ith
position of the list is p/n for every i, and the number of comparisons made by the
algorithm in such a situation is obviously i. In the case of an unsuccessful search, the
number of comparisons will be n with the probability of such a search being (1− p).
Therefore,
Cavg(n) = Probability of successful search + Probability of unsuccessful search

Recapitulation of the Analysis Framework

Let us summarize the main points of the framework outlined above.
Both time and space efficiencies are measured as functions of the algorithm’s input
size.
Time efficiency is measured by counting the number of times the algorithm’s basic
operation is executed. Space efficiency is measured by counting the number of extra

Dept. of CSE, HKBKCE 10 2023-24

 BCS401 ADA Notes
memory units consumed by the algorithm.
The efficiencies of some algorithms may differ significantly for inputs of the same
size. For such algorithms, we need to distinguish between the worst-case, average-
case, and best-case efficiencies.
The framework’s primary interest lies in the order of growth of the algorithm’s
running time (extra memory units consumed) as its input size goes to infinity.

Asymptotic Notation (O, Ɵ, Ώ)

To compare and rank order of growth, computer scientists use three notations O (big oh), Ω(big
omega), and Θ (big theta).

Let t (n) and g(n) can be any nonnegative functions defined on the set of natural numbers.
● t (n) - algorithm’s running time (usually indicated by its basic operation count C(n)), and
● g(n) will be some simple function to compare the count with.
Big Oh Notation:
DEFINITION: A function t (n) is said to be in O(g(n)), denoted t (n) ∈ O(g(n)), if t (n) is
bounded above by some constant multiple of g(n) for all large n, i.e., if there exist some
positive constant c and some nonnegative integer n0 such that
t (n) ≤ cg(n) for all n ≥ n0.

FIGURE 1.3: Big-oh notation: t (n) ∈ O(g(n)).

As an example, let us formally prove one of the assertions made in the introduction:
100n + 5 ∈ O(n2). Indeed,
100n + 5 ≤ 100n + n (for all n ≥ 5) = 101n ≤ 101n2.

Thus, as values of the constants c and n0 required by the definition, we can take 101 and 5,
respectively. Note that the definition gives us a lot of freedom in choosing specific values for
constants c and n0. For example, we could also reason that 100n + 5 ≤ 100n + 5n (for all n ≥
1) = 105n to complete the proof with c = 105 and n0 = 1.

PROBLEMS:
1. Let f(n) =3n+2. Express f(n) in Big-oh [O(n)]
Given f(n)=3n+2
To find g(n): Let us express g(n) in terms of higher order term of f(n) as shown
g(n)=f(n)
g(n)=3n+2
so, g(n)=4n where n=2

The constraint to be satisfied is f(n) ≤ c.g(n)

i.e., 3n+2 ≤ 4n for n ≥ 2

Dept. of CSE, HKBKCE 11 2023-24

BCS401 ADA Notes
It is clear from the above relations that c=4,g(n)=n and n0=2. So by definition
f(n)=O(g(n))
i.e., f(n)=O(n)

2. Let f(n) =10n2+4n+2. Express f(n) in Big-oh [O(n2)]

Given f(n)= 10n2+4n+2
To find g(n): Let us express g(n) in terms of higher order term of f(n) as shown
g(n)=f(n)
g(n)= 10n2+4n+2
g(n)=10n2+n2 (Replacing 4n with n2)
g(n)=11n2

so, g(n)= 11n2 where n2=4n and n=5

The constraint to be satisfied is f(n) ≤ c.g(n)

i.e., 10n2+4n+2≤ 11n2 for n ≥ 5
It is clear from the above relations that c=11,g(n)=n2 and n0=5. So by definition
f(n)=O(g(n))
i.e., f(n)=O(n )
2

3. Let f(n) =6*2n+n2. Express f(n) in Big-oh[O( 2n)]

Given f(n)= 6*2n+n2
To find g(n): Let us express g(n) in terms of higher order term of f(n) as shown
g(n)=f(n)
g(n)= 6*2n+n2
g(n)= 6*2n+2n (Replacing n2 with 2n)
g(n)=7*2 n

so, g(n)= 7*2n

The constraint to be satisfied is f(n) ≤ c.g(n)

i.e., 6*2n+n2 ≤ 7*2n for n ≥ 4
It is clear from the above relations that c=7, g (n) =2n and n0=0. So by definition

f(n)=O(g(n))

i.e., f(n)=O(2n)

Big Omega notation

DEFINITION: A function t (n) is said to be in Ω(g(n)), denoted t (n) ∈ (g(n)), if t (n) is
bounded below by some positive constant multiple of g(n) for all large n, i.e., if there exist
some positive constant c and some nonnegative integer n0 such that
t (n) ≥ cg(n) for all n ≥ n0.
The definition is illustrated in Figure 1.4.

Dept. of CSE, HKBKCE 12 2023-24

BCS401 ADA Notes

FIGURE 1.4: Big-omega notation: t (n) ∈ Ω(g(n)).

Here is an example of the formal proof that n3 ∈ Ω(n2):

n3 ≥ n2 for all n ≥ 0,
i.e., we can select c = 1 and n0 = 0.

PROBLEMS:
4. Let f(n) =3n+2. Express f(n) in Big-Omega [Ω (n)]
Given f(n)=3n+2
To find g(n): Let us express g(n) in terms of higher order term of f(n) as shown
g(n)=f(n)
g(n)=3n+2 (replacing 2 with 0)
so, g(n)=3n where n=2

The constraint to be satisfied is f(n) ≥ c.g(n)

i.e., 3n+2 ≥ 3n for n ≥ 0
It is clear from the above relations that c=3,g(n)=n and n0=0. So by definition
f(n)=Ω(g(n))
i.e., f(n)=Ω(n)
5. Let f(n) =10n2+4n+2. Express f(n) in Big-Omega [Ω (n2)]
Given f(n)= 10n2+4n+2
To find g(n): Let us express g(n) in terms of higher order term of f(n) as shown
g(n)=f(n)
g(n)= 10n2+4n+2
g(n)=10n2+0 (Replacing 4n+2 with 0)
g(n)=10n2

so, g(n)= 10n2

The constraint to be satisfied is f(n) ≥ c.g(n)

i.e., 10n2+4n+2 ≥ 10n2 for n ≥ 0
It is clear from the above relations that c=10,g(n)=n2 and n0=0. So by definition
f(n)=Ω(g(n))
i.e., f(n)=Ω(n2)

Big theta notation

DEFINITION: A function t (n) is said to be in (g(n)), denoted t (n) ∈ ɵ(g(n)), if t (n) is
bounded both above and below by some positive constant multiples of g(n) for all large
n, i.e., if there exist some positive constants c1 and c2 and some nonnegative integer n0
such that

Dept. of CSE, HKBKCE 13 2023-24

BCS401 ADA Notes
c2g(n) ≤ t (n) ≤ c1g(n) for all n ≥ n0.
The definition is illustrated in Figure 1.5.

FIGURE 1.5: Big-theta notation: t (n) ∈ ɵ(g(n)).

For example, let us prove that 1/2 n (n − 1) ∈ ɵ(n2).

First, we prove the right inequality (the upper bound):
1/2 n(n − 1) = 1/2n2 – 1/2n ≤ 1/2n2 for all n ≥ 0.

Second, we prove the left inequality (the lower bound):

1/2n(n − 1) = 1/2n2 – 1/2n ≥ 1/2n2 – 1/2n<=1/2n (for all n ≥ 2) = 1/4n2.

Hence, we can select c2 = 1/4, c1 = 1/2 , and n0 = 2.

PROBLEMS:
6. Let f(n) =3n+2. Express f(n) in Big-theta [ɵ(n)]

The constraint to be satisfied is c2g(n) ≤ t (n) ≤ c1g(n)

i.e., 3n ≤ 3n+2 ≤ 4n for n ≥ 1
It is clear from the above relations that c2=3, c1=4, g(n)=n and n0=1.
So by definition f(n)= ɵ (g(n))
i.e., f(n)= ɵ (n)

7. Let f(n) =6*2n+n2. Express f(n) in Big-theta

Given f(n)= 6*2n+n2

The constraint to be satisfied is c2g(n) ≤ t (n) ≤ c1g(n)

i.e., 6*2n ≤ 6*2n+n2 ≤ 7*2n

It is clear from the above relations that c2=7,c1=6,g(n)=2n and n0=4. So by definition
f(n)= ɵ (g(n))
i.e., f(n)= ɵ (2n)

THEOREM: If t1(n) ∈ O(g1(n)) and t2(n) ∈ O(g2(n)), then t1(n) + t2(n) ∈ O(max{g1(n),
g2(n)}).

Dept. of CSE, HKBKCE 14 2023-24

BCS401 ADA Notes
(The analogous assertions are true for the Ω and ɵ notations as well.)

PROOF: The proof extends to orders of growth the following simple fact about four arbitrary
real numbers a1, b1, a2, b2: if a1 ≤ b1 and a2 ≤ b2, then a1 + a2 ≤ 2 max{b1, b2}.
Since t1(n) ∈ O(g1(n)), there exist some positive constant c1 and some nonnegative
integer n1 such that
t1(n) ≤ c1g1(n) for all n ≥ n1.
Similarly, since t2(n) ∈ O(g2(n)),
t2(n) ≤ c2g2(n) for all n ≥ n2.

Let us denote c3 = max{c1, c2} and consider n ≥ max{n1, n2} so that we can use both
inequalities. Adding them yields the following:
t1(n) + t2(n) ≤ c1g1(n) + c2g2(n)
≤ c3g1(n) + c3g2(n) = c3[g1(n) + g2(n)]
≤ c3 [2 max{g1(n), g2(n)}].

Hence, t1(n) + t2(n) ∈ O(max{g1(n), g2(n)}), with the constants c and n0 required by the O
definition being 2c3 = 2 max{c1, c2} and max{n1, n2}, respectively.

Mathematical Analysis of Non recursive Algorithms

EXAMPLE 1: Consider the problem of finding the value of the largest element in a list of n
numbers. For simplicity, we assume that the list is implemented as an array. The following is
pseudocode of a standard algorithm for solving the problem.
ALGORITHM MaxElement(A[0..n − 1])
//Determines the value of the largest element in a given array
//Input: An array A[0..n − 1] of real numbers
//Output: The value of the largest element in A
maxval ←A[0]
for i ←1 to n − 1 do
if A[i]>maxval
maxval←A[i]
return maxval

The obvious measure of an input’s size here is the number of elements in the array, i.e., n.
The operations that are going to be executed most often are in the algorithm’s for loop. There
are two operations in the loop’s body: the comparison A[i]> maxval and the assignment
maxval←A[i]. Since the comparison is executed on each repetition of the loop and the
assignment is not, we should consider the comparison to be the algorithm’s basic operation.
Note that the number of comparisons will be the same for all arrays of size n; therefore, in
terms of this metric, there is no need to distinguish among the worst, average, and best cases
here.

Let us denote C(n) the number of times this comparison is executed and try to find a formula
expressing it as a function of size n. The algorithm makes one comparison on each execution
of the loop, which is repeated for each value of the loop’s variable i within the bounds 1 and
n − 1, inclusive. Therefore, we get the following sum for C(n):

This is an easy sum to compute because it is nothing other than 1 repeated n – 1 times. Thus,

Dept. of CSE, HKBKCE 15 2023-24

BCS401 ADA Notes

Thus the efficiency of above algorithm is ɵ (n).

General Plan for Analyzing the Time Efficiency of Non-recursive Algorithms
1. Decide on a parameter (or parameters) indicating an input’s size.
2. Identify the algorithm’s basic operation. (As a rule, it is located in the innermost loop.)
3. Check whether the number of times the basic operation is executed depends only on
the size of an input. If it also depends on some additional property, the worst-case,
average-case, and, if necessary, best-case efficiencies have to be investigated
separately.
4. Set up a sum expressing the number of times the algorithm’s basic operation is
executed.
5. Using standard formulas and rules of sum manipulation either find a closed form
formula for the count or, at the very least, establish its order of growth.

EXAMPLE 2: Consider the element uniqueness problem: check whether all the elements in
a given array of n elements are distinct. This problem can be solved by the following
straightforward algorithm.

ALGORITHM UniqueElements(A[0..n − 1])

//Determines whether all the elements in a given array are distinct
//Input: An array A[0..n − 1]
//Output: Returns “true” if all the elements in A are distinct and “false”
otherwise
for i ←0 to n − 2 do
for j ←i + 1 to n − 1 do
if A[i]= A[j ] return false
return true

The natural measure of the input’s size here is again n, the number of elements in the array.
Since the innermost loop contains a single operation (the comparison of two elements), we
should consider it as the algorithm’s basic operation. Note, however, that the number of
element comparisons depends not only on n but also on whether there are equal elements in
the array and, if there are, which array positions they occupy. We will limit our investigation
to the worst case only.

By definition, the worst case input is an array for which the number of element comparisons
Cworst(n) is the largest among all arrays of size n. An inspection of the innermost loop reveals
that there are two kinds of worst-case inputs—inputs for which the algorithm does not exit
the loop prematurely: arrays with no equal elements and arrays in which the last two elements
are the only pair of equal elements. For such inputs, one comparison is made for each
repetition of the innermost loop, i.e., for each value of the loop variable j between its limits i
+ 1 and n − 1; this is repeated for each value of the outer loop, i.e., for each value of the loop
variable i between its limits 0 and n − 2. Accordingly, we get

Dept. of CSE, HKBKCE 16 2023-24

 BCS401 ADA Notes

EXAMPLE 3: Given two n × n matrices A and B, find the time efficiency of the definition-
based algorithm for computing their product C = AB. By definition, C is an n × n matrix
whose elements are computed as the scalar (dot) products of the rows of matrix A and the
columns of matrix B:

where C[i, j ]= A[i, 0]B[0, j]+ . . . + A[i, k]B[k, j]+ . . . + A[i, n − 1]B[n − 1, j] for every pair of
indices 0 ≤ i, j ≤ n − 1.

ALGORITHM MatrixMultiplication(A[0..n − 1, 0..n − 1], B[0..n − 1, 0..n − 1])

//Multiplies two square matrices of order n by the definition-based algorithm
//Input: Two n × n matrices A and B
//Output: Matrix C = AB
for i ←0 to n − 1 do
for j ←0 to n − 1 do
C[i, j ]←0.0
for k←0 to n − 1 do
C[i, j ]←C[i, j ]+ A[i, k] ∗ B[k, j]
return C

Dept. of CSE, HKBKCE 17 2023-24

BCS401 ADA Notes
Obviously, there is just one multiplication executed on each repetition of the algorithm’s
innermost loop, which is governed by the variable k ranging from the lower bound 0 to the
upper bound n − 1. Therefore, the number of multiplications made for every pair of specific
values of variables i and j is

and the total number of multiplications M(n) is expressed by the following triple sum:

Now, we can compute this sum by using formula (S1) and rule (R1) given above. Starting
with the innermost sum , which is equal to n, we get

EXAMPLE 4: The following algorithm finds the number of binary digits in the binary representation of
a positive decimal integer.
ALGORITHM Binary(n)
//Input: A positive decimal integer n
//Output: The number of binary digits in n’s binary representation
count ←1
while n > 1 do
count ←count + 1
n←n/2
return count

First, notice that the most frequently executed operation here is not inside the while loop but
rather the comparison n > 1 that determines whether the loop’s body will be executed. Since
the number of times the comparison will be executed is larger than the number of repetitions
of the loop’s body by exactly 1, the choice is not that important.

A more significant feature of this example is the fact that the loop variable takes on only a
few values between its lower and upper limits; therefore, we have to use an alternative way of
computing the number of times the loop is executed. Since the value of n is about halved on
each repetition of the loop, the answer should be about log2 n. The exact formula for the
number of times the comparison n>1 will be executed is actually [log2 n] + 1—the number of
bits in the binary representation of n.

Mathematical Analysis of Recursive Algorithms

The general framework for analysis of algorithms to recursive algorithms. We start with an
example often used to introduce novices to the idea of a recursive algorithm.

Dept. of CSE, HKBKCE 18 2023-24

BCS401 ADA Notes
EXAMPLE 1: Compute the factorial function F(n) = n! for an arbitrary nonnegative
integer n.
Since
n!= 1 . . . . . (n − 1) . n = (n − 1)! . n for n ≥ 1

and 0!= 1 by definition, we can compute F(n) = F(n − 1) . n with the following recursive
algorithm.

ALGORITHM F(n)
//Computes n! recursively
//Input: A nonnegative integer n
//Output: The value of n!
if n = 0 return 1
else return F(n − 1) ∗ n

For simplicity, we consider n itself as an indicator of this algorithm’s input. The basic
operation of the algorithm is multiplication, whose number of executions we denote M(n).
Since the function F(n) is computed according to the formula
F(n) = F(n − 1) . n for n > 0,

the number of multiplications M(n) needed to compute it must satisfy the equality
M(n) = M(n − 1) + 1 for n > 0,

to compute to multiply
F(n−1) F(n−1) by n

Indeed, M(n − 1) multiplications are spent to compute F(n − 1), and one more multiplication
is needed to multiply the result by n.
The last equation defines the sequence M(n) that we need to find. This equation defines M(n)
not explicitly, i.e., as a function of n, but implicitly as a function of its value at another point,
namely n − 1. Such equations are called recurrence relations or, for brevity, recurrences.
Recurrence relations play an important role not only in analysis of algorithms but also in
some areas of applied mathematics.

To determine a solution uniquely, we need an initial condition that tells us the value with
which the sequence starts. We can obtain this value by inspecting the condition that makes
the algorithm stop its recursive calls:
if n = 0 return 1.

This tells us two things. First, since the calls stop when n = 0, the smallest value of n for
which this algorithm is executed and hence M(n) defined is 0. Second, by inspecting the
pseudocode’s exiting line, we can see that when n = 0, the algorithm performs no
multiplications. Therefore, the initial condition we are after is
M(0) = 0.

the calls stop when n = 0 no multiplications when n = 0

Thus, we succeeded in setting up the recurrence relation and initial condition for the
algorithm’s number of multiplications M(n):
M(n) = M(n − 1) + 1 for n > 0,
(Equation 1)
M(0) = 0.

Dept. of CSE, HKBKCE 19 2023-24

BCS401 ADA Notes
The first is the factorial function F(n) itself; it is defined by the recurrence
F(n) = F(n − 1) . n for every n > 0,
F(0) = 1.

The second is the number of multiplications M(n) needed to compute F(n) by the recursive
algorithm whose pseudocode was given at the beginning of the section

The several techniques available for solving recurrence relations, we use what can be called
the method of backward substitutions. The method’s idea (and the reason for the name) is
immediately clear from the way it applies to solving our particular recurrence:
M(n) = M(n − 1) + 1 substitute M(n − 1) = M(n − 2) + 1
= [M(n − 2) + 1]+ 1= M(n − 2) + 2 substitute M(n − 2) = M(n − 3) + 1
= [M(n − 3) + 1]+ 2 = M(n − 3) + 3.

After inspecting the first three lines, we see an emerging pattern, which makes it possible to
predict not only the next line (what would it be?) but also a general formula for the pattern:
M(n) = M(n − i) + i.

Since it is specified for n = 0, we have to substitute i = n in the pattern’s formula to get the
ultimate result of our backward substitutions:
M(n) = M(n − 1) + 1= . . . = M(n − i) + i = . . . = M(n − n) + n = n.

The issue of time efficiency is actually not that important for the problem of computing n!,
the function’s values get so large so fast that we can realistically compute exact values of n!
only for very small n’s.

General Plan for Analyzing the Time Efficiency of Recursive Algorithms

1. Decide on a parameter (or parameters) indicating an input’s size.
2. Identify the algorithm’s basic operation.
3. Check whether the number of times the basic operation is executed can vary on
different inputs of the same size; if it can, the worst-case, average-case, and best-case
efficiencies must be investigated separately.
4. Set up a recurrence relation, with an appropriate initial condition, for the number of
times the basic operation is executed.
5. Solve the recurrence or, at least, ascertain the order of growth of its solution.

EXAMPLE 2: The Tower of Hanoi puzzle.

The problem has an elegant recursive solution, which is illustrated in Figure 1.7. To move
n>1 disks from peg 1 to peg 3 (with peg 2 as auxiliary), we first move recursively n − 1 disks
from peg 1 to peg 2 (with peg 3 as auxiliary), then move the largest disk directly from peg 1
to peg 3, and, finally, move recursively
n − 1 disks from peg 2 to peg 3 (using peg 1 as auxiliary). Of course, if n = 1, we simply
move the single disk directly from the source peg to the destination peg.

Dept. of CSE, HKBKCE 20 2023-24

BCS401 ADA Notes

FIGURE 1.7: Recursive solution to the Tower of Hanoi puzzle.

Let us apply the general plan outlined above to the Tower of Hanoi problem. The number of
disks n is the obvious choice for the input’s size indicator, and so is moving one disk as the
algorithm’s basic operation. Clearly, the number of moves M(n) depends on n only, and we
get the following recurrence equation for it:
M(n) = M(n − 1) + 1+ M(n − 1) for n > 1.

With the obvious initial condition M(1) = 1, we have the following recurrence relation for the
number of moves M(n):

When a recursive algorithm makes more than a single call to itself, it can be useful for
analysis purposes to construct a tree of its recursive calls. In this tree, nodes correspond to
recursive calls, and we can label them with the value of the parameter (or, more generally,
parameters) of the calls. For the Tower of Hanoi example, the tree is given in Figure 1.7. By

Dept. of CSE, HKBKCE 21 2023-24

BCS401 ADA Notes
counting the number of nodes in the tree, we can get the total number of calls made by the
Tower of Hanoi algorithm:

,where l is the level in the tree= 2n-1

FIGURE 1.7: Tree of recursive calls made by the recursive algorithm for the Tower of
Hanoi puzzle.

EXAMPLE 3: As our next example, we investigate a recursive version of the algorithm

discussed at the end of Section 2.3.
ALGORITHM BinRec(n)
//Input: A positive decimal integer n
//Output: The number of binary digits in n’s binary representation
if n = 1 return 1
else return BinRec(n/2)+ 1

Let us set up a recurrence and an initial condition for the number of additions A(n) made by
the algorithm. The number of additions made in computing BinRec(n/2) is A(n/2), plus one
more addition is made by the algorithm to increase the returned value by 1. This leads to the
recurrence
A(n) = A(_n/2_) + 1 for n > 1.

Since the recursive calls end when n is equal to 1 and there are no additions made then, the
initial condition is
A(1) = 0.

The presence of n/2 in the function’s argument makes the method of backward substitutions
stumble on values of n that are not powers of 2. Therefore, the standard approach to solving
such a recurrence is to solve it only for n = 2k and then take advantage of the theorem called
the smoothness rule (see Appendix B), which claims that under very broad assumptions the
order of growth observed for n = 2k gives a correct answer about the order of growth for all
values of n. (Alternatively, after getting a solution for powers of 2, we can sometimes fine-
tune this solution to get a formula valid for an arbitrary n.) So let us apply this recipe to our
recurrence, which for n = 2k takes the form

A(2k) = A(2k-1) + 1 for k > 0,

A(20) = 0.

Now backward substitutions encounter no problems:

A(2k) = A(2k−1) + 1 substitute A(2k-1) = A(2k-2) + 1

Dept. of CSE, HKBKCE 22 2023-24

 BCS401 ADA Notes
= [A(2k−2) + 1]+ 1= A(2k−2) + 2 substitute A(2k−2) =
A(2k−3) + 1
= [A(2k−3) + 1]+ 2 = A(2k−3) + 3 . . .. . .
= A(2k−i) + i
...
= A(2k−k) + k.

Thus, we end up with

A(2k) = A(1) + k = k,
or, after returning to the original variable n = 2k and hence k = log2 n,
A(n) = log2 n ∈ ɵ(log n).
The exact solution for an arbitrary value of n is given by just a slightly more refined formula
A(n) = log2 n.

BRUTE FORCE DESIGN TECHNIQUE

Brute force is a straightforward approach to solving a problem, usually directly based on the problem
statement and definitions of the concepts involved.

Selection Sort:
We start selection sort by scanning the entire given list to find its smallest element and exchange it
with the first element, putting the smallest element in its final position in the sorted list. Then we scan
the list, starting with the second element, to find the smallest among the last n - 1 elements and
exchange it with the second element, putting the second smallest element in its final position.
Generally, on the ith pass through the list, which we number from 0 ton - 2, the algorithm searches
for the smallest item among the last n - i elements and swaps it with A;

After n - 1 passes, the list is sorted. Here is a pseudocode of this algorithm, which, for simplicity,
assumes that the list is implemented as an array.

Example:

Dept. of CSE, HKBKCE 23 2023-24

BCS401 ADA Notes

Dept. of CSE, HKBKCE 24 2023-24

 BCS401 ADA Notes

Sequential Search:
We have already encountered a brute-force algorithm for the general searching problem: it is called
sequential search (see Section 2.1). To repeat, the algorithm simply compares successive elements
of a given list with a given search key until either a match is encountered (successful search) or the
list is exhausted without finding a match (unsuccessful search). A simple extra trick is often
employed in implementing sequential search: if we append the search key to the end of the list, the
search for the key will have to be successful, and therefore we can eliminate a check for the list's
end on each iteration of the algorithm. Here is a pseudocode for this enhanced version, with its
input implemented as an array.

Dept. of CSE, HKBKCE 25 2023-24

 BCS401 ADA Notes

Another straightforward improvement can be incorporated in sequential search if a given list is

known to be sorted: searching in such a list can be stopped as soon as an element greater than or
equal to the search key is encountered. Sequential search provides an excellent illustration of the
brute-force approach, with its characteristic strength (simplicity) and weakness (inferior
efficiency). The efficiency results obtained in Section 2.1 for the standard version of sequential
search change for the enhanced version only very slightly, so that the algorithm remains linear in
both worst and average cases. We discuss later in the book several searching algorithms with a
better time efficiency.

Brute-Force String Matching

Dept. of CSE, HKBKCE 26 2023-24

 BCS401 ADA Notes

Note that for this example, the algorithm shifts the pattern almost always after a single character
comparison. However, the worst case is much worse: the algorithm may have to make all m
comparisons before shifting the pattern, and this can happen for each of the n - m + 1 tries.
(Problem 6 asks you to give a specific example of such a situation.) Thus, in the worst case, the
algorithm is in 8(nm). For a typical word search in a natural language text, however, we should
expect that most shifts would happen after very few comparisons (check the example again).
Therefore, the average-case efficiency should be considerably better than the worst-case efficiency.
Indeed it is: for searching in random texts, it has been shown to be linear, i.e., Ɵ (n + m) = Ɵ (n).

Dept. of CSE, HKBKCE 27 2023-24