lOMoARcPSD|30897286

lOMoARcPSD|308 972 86

ASAN MEMORIAL COLLEGE OF ENGINEERING & TECHNOLOGY

(Approved by AICTE, Affiliated to Anna University,Chennai)
Asan Nagar,Thandarai,Oragadam main road,Chengalpattu-603105

DEPARTMENT OF ARTIFICIAL INTELLIGENCE AND DATA

SCIENCE

Course Code/Course Name: AD3351 / DESIGN AND ANALYSIS OF ALGORITHM.

Course Instructor: Mrs.V.RevathyVadivel,AP/ADS

UNIT I INTRODUCTION
Notion of an Algorithm – Fundamentals of Algorithmic Problem Solving – Important Problem Types –
Fundamentals of the Analysis of Algorithm Efficiency – Analysis Framework – Asymptotic Notations and
their properties – Empirical analysis – Mathematical analysis of recursive and Non-recursive algorithms –
Visualization.

 Give the Euclid’s algorithms for computing gcd (m,n) [Part A - MAY/JUNE 2016]
 What is an algorithm? [Part A – APR/MAY/2017]
 Write an algorithm to compute the greatest common divisor of two numbers.[Part A –
APR/MAY/2017& 2018]
 What is basic Operation[Part A- APR/MAY 2018]
 Define Best, Worst Average case time complexity[Part A- NOV/DEC 2018]

1. NOTION OF AN ALGORITHM

An algorithm is a sequence of unambiguous instructions for solving a problem,i.e., forobtaining

a required output for any legitimate input in a finite amount of time.

Problem to be solved

Algorithm

Input Computer Program Output

FIGURE 1.1 The notion of the algorithm.

It is a step by step procedure with the input to solve the problem in a finite amountof timeto obtain
the required output.
 lOMoARcPSD|308 972 86

The notion of the algorithm illustrates some important points:

 The non-ambiguity requirement for each step of an algorithm cannot becompromised.
 The range of inputs for which an algorithm works has to be specified carefully.
 The same algorithm can be represented in several different ways.
 There may exist several algorithms for solving the same problem.
 Algorithms for the same problem can be based on very different ideas and cansolve
theproblem with dramatically different speeds.
Characteristics of an algorithm:
Input: Zero / more quantities are externally
supplied.
Output: At least one quantity is produced.
Definiteness: Each instruction is clear and unambiguous.
Finiteness: If the instructions of an algorithm is traced then for all cases thealgorithm
mustterminates after a finite number of steps.
Efficiency: Every instruction must be very basic and runs in short time.
Steps for writing an algorithm:
1. An algorithm is a procedure. It has two parts; the first part is head and the secondpart is
body.
2. The Head section consists of keyword Algorithm and Name of the algorithmwith
parameter list. E.g. Algorithm name1(p1, p2,…,p3)
The head section also has the following:
//Problem Description:
//Input:
//Output:
3. In the body of an algorithm various programming constructs like if, for, while
and somestatements like assignments are used.
4. The compound statements may be enclosed with { and } brackets. if, for, while can beclosed by
endif, endfor, endwhile respectively. Proper indention is must for block.
5. Comments are written using // at the beginning.
6. The identifier should begin by a letter and not by digit. It contains alpha numeric lettersafter
first letter. No need to mention data types.
7. The left arrow “←” used as assignment operator. E.g. v←10
8. Boolean operators (TRUE, FALSE), Logical operators (AND, OR, NOT) and
Relational operators (<,<=, >, >=,=, ≠, <>) are also used.
9. Input and Output can be done using read and write.
10. Array[ ], if then else condition, branch and loop can be also used in algorithm.
Example:
The greatest common divisor(GCD) of two nonnegative integers m and n (not- both-zero),
denoted gcd(m, n), is defined as the largest integer that divides both m and n evenly, i.e., with a remainder
of zero.

Euclid’s algorithm is based on applying repeatedly the equality gcd(m, n) = gcd(n, m mod n), where m mod
n is the remainder of the division of m by n, until m mod n is equal to 0. Since gcd(m,0) = m, the last value of
m is also the greatest common divisor of the initial mand n.

gcd(60, 24) can be computed as follows:

 lOMoARcPSD|308 972 86

gcd(60, 24) = gcd(24, 12) = gcd(12, 0) = 12.

Euclid’s algorithm for computing gcd(m, n) in simple steps

Step 1 If n = 0, return the value of m as the answer and stop; otherwise, proceed to Step 2.
Step 2 Divide m by n and assign the value of the remainder to r.
Step 3 Assign the value of n to m and the value of r to n. Go to Step 1.

Euclid’s algorithm for computing gcd(m, n) expressed in pseudocode

ALGORITHM Euclid_gcd(m, n)
//Computes gcd(m, n) by Euclid’s algorithm
//Input: Two nonnegative, not-both-zero integers m and n
//Output: Greatest common divisor of m and n
while n ≠ 0 do
r ←m mod nm←n
n←r
return m

Euclid’s algorithm for computing gcd(m,n)

Step 1: If n = 0, return the value of m as the answer and stop;otherwise,
proceed to Step 2.
Step 2: Divide m by n and assign the value of the remainder to r.
Step 3: Assign the value of n to m and the value of r to n. Go to Step 1.

Pseudocode for Euclid’s Algorithm

//Computes gcd(m, n) by Euclid’s algorithm

//Input: Two nonnegative , not both zero integers m and n
//Output: Greatest common divisor of m and n
While n ≠ 0 do
r⟵m mod n
m⟵n
n⟵r
return m

Consecutive integer checking algorithm for computing gcd(m,n)Step 1: Assign the

value of min(m, n) to t.
Step 2: Divide m by t. If the remainder of this division is 0, go to Step3;Otherwise, go
to Step 4.
Step 3: Divide n by t . If the remainder of this division is 0,
return the value of t as the answer and stop; otherwise , proceed to Step 4.
Step 4: Decrease the value of t by 1. Go to Step 2.
This algorithm does not work correctly when of its input numbers is zero. This is important
to specify the range of an algorithm’s inputs explicitly and carefully.

Middle School procedure for computing gcd(m,n)

Step 1: Find the prime factrs of m.

Step 2: Find the prime factors of n.
Step 3: Identify all the common factors in two prime expansions found in Step1 and Step2.(If p is a
common factor occurring pm and pn times in m and n,respectively. It should be repeated
min{pm and pn} times.)
Step 4: Compute the product of all the common factors and return it as the greatest common divisor
of the numbers given.
Eg: For the numbers 60 and 24, we get
 lOMoARcPSD|308 972 86

60 = 2·2·3·4
24 = 2·2·2·3
gcd(60,24)= 2·2·3 = 12
Practice problem Find
the GCD ( 12,8)Find GCD
( 80,24)

 Explain in detail about the fundamentals of algorithmic Problem Solving [PartB- APR/MAY
2016]
 List the steps in the fundamentals of algorithmic Problem Solving[Part B-APR/MAY 2013]

2. FUNDAMENTALS OF ALGORITHMIC PROBLEM SOLVING

consider algorithms to be procedural solutions to problems. These solutions are not answers but
specific instructions for getting answers . A sequence of steps involvedin designing and analyzing an
algorithm is shown in the figure below.

FIGURE 1.2 Algorithm design and analysis process.

(i) Understanding the Problem
 This is the first step in designing of algorithm.
 The first thing you need to do before an algorithm is to understand completely theproblem given.
 Read the problem’s description carefully to understand the problem statementcompletely. Ask
questions for clarifying the doubts about the problem.
 Identify the problem types and use existing algorithm to find solution.
 Input (instance) to the problem and range of the input get fixed.
 Read the problem’s description carefully and ask questions if you any doubts aboutthe problem, do
a few small examples by hand, think about special cases, and ask question s again if needed.
 It helps to understand how such an algorithm works and to know its strengths and
 lOMoARcPSD|308 972 86

weaknesses, especially if you have to choose among several available algorithms.

 It is very important to specify exactly the range of instances the algorithm needs tohandle.

(ii) Decision making

The Decision making is done on the following:
(a) Ascertaining the Capabilities of the Computational Device
 In random-access machine (RAM), instructions are executed one after another (The
central assumption is that one operation at a time). Accordingly, algorithms designed to
be executed on such machines are called sequential algorithms.
 In some newer computers, operations are executed concurrently, i.e., in parallel.
Algorithms that take advantage of this capability are called parallel algorithms.
 Choice of computational devices like Processor and memory is mainly based on space
and time efficiency
(b) Choosing between Exact and Approximate Problem Solving
 The next principal decision is to choose between solving the problemexactly orsolving it
approximately.
 An algorithm used to solve the problem exactly and produce correct result is calledan
exact algorithm.
 If the problem is so complex and not able to get exact solution, then we have tochoose an
algorithm called an approximation algorithm. i.e., produces an
approximate answer. E.g., extracting square roots, solving nonlinear equations, and
evaluating definite integrals.
(c) Algorithm Design Techniques
 An algorithm design technique (or “strategy” or “paradigm”) is a general approach to
solving problems algorithmically that is applicable to a variety of problems from different
areas of computing.
Algorithms+ Data Structures = Programs
 Though Algorithms and Data Structures are independent, but they are combined together to
develop program. Hence the choice of proper data structure is required before designing the
algorithm.
 Implementation of algorithm is possible only with the help of Algorithms andData Structures
 Algorithmic strategy / technique / paradigm are a general approach by which many
problems can be solved algorithmically. E.g., Brute Force, Divide and Conquer, Dynamic
Programming, Greedy Technique and so on.

(iii) Methods of Specifying an Algorithm

There are three ways to specify an algorithm. They are:
a. Natural language
b. Pseudocode
c. Flowchart

Algorithm Specification

Natural Language Pseudocode Flowchart

 lOMoARcPSD|308 972 86

FIGURE 1.3 Algorithm Specifications

Pseudocode and flowchart are the two options that are most widely used nowadays for specifying
algorithms.

a. Natural Language
It is very simple and easy to specify an algorithm using natural language. But many times specification of
algorithm by using natural language is not clear and thereby we get brief specification.
Example: An algorithm to perform addition of two numbers.
Step 1: Read the first number, say a.
Step 2: Read the first number, say b.
Step 3: Add the above two numbers and store the result in c.
Step 4: Display the result from c.
Such a specification creates difficulty while actually implementing it. Hence manyprogrammersprefer to
have specification of algorithm by means of Pseudocode.
b. Pseudocode
 Pseudocode is a mixture of a natural language and programming language
constructs.Pseudocode is usually more precise than natural language.
 For Assignment operation left arrow “←”, for comments two slashes “//”,if
condition, for,while loops are used.

ALGORITHM Sum(a,b)
//Problem Description: This algorithm performs addition of two numbers
//Input: Two integers a and b
//Output: Addition of two integers
c←a+b
return c

This specification is more useful for implementation of any language.

 lOMoARcPSD|308 972 86

c. Flowchart
In the earlier days of computing, the dominant method for specifying algorithms was a
flowchart,this representation technique has proved to be inconvenient.
Flowchart is a graphical representation of an algorithm. It is a a method of expressing an algorithmby a
collection of connected geometric shapes containing descriptions of the algorithm’s steps.

Symbols Example: Addition of a and b

Start Start state

Start
Transition / Assignment
Input the value of a
Processing /
Input read
Input the value of b
Input and c=a+b
Output

Condition / Decision Display the value of c

Flow
Stop
connectiv
ity Stop
Stop
state
FIGURE 1.4 Flowchart symbols and Example for two integer addition.

(iv) Proving an Algorithm’s Correctness

 Once an algorithm has been specified then its correctness must be proved.
 An algorithm must yields a required result for every legitimate input in a finite amount oftime.

 For example, the correctness of Euclid’s algorithm for computing the greatestcommon
divisor stems from the correctness of the equality gcd(m, n) = gcd(n, m mod n).
 A common technique for proving correctness is to use mathematical induction because an
algorithm’s iterations provide a natural sequence of steps needed for such proofs.

 The notion of correctness for approximation algorithms is less straightforwardthan it is for

exact algorithms.
(v) Analyzing an Algorithm
 For an algorithm the most important is efficiency. In fact, there are two kinds of
algorithmefficiency. They are:
 Time efficiency, indicating how fast the algorithm runs, and
 Space efficiency, indicating how much extra memory it uses.
 The efficiency of an algorithm is determined by measuring both time efficiencyand
spaceefficiency.
 So factors to analyze an algorithm are:
 Time efficiency of an algorithm
 Space efficiency of an algorithm
 lOMoARcPSD|308 972 86

 Simplicity of an algorithm
 Generality of an algorithm

(vi) Coding an Algorithm

 The coding / implementation of an algorithm is done by a suitable programming languagelike C,
C++, JAVA.
 The transition from an algorithm to a program can be done either incorrectly or very inefficiently.
Implementing an algorithm correctly is necessary. The Algorithm power should not reduced by
inefficient implementation.
 Standard tricks like computing a loop’s invariant (an expression that does not change its value)
outside the loop, collecting common subexpressions, replacing expensive operations by cheap
ones, selection of programming language and so onshould be known tothe programmer.
 Typically, such improvements can speed up a program only by a constant factor, whereas a better
algorithm can make a difference in running time by orders of magnitude.
 It is very essential to write an optimized code (efficient code) to reduce the burden of compiler.

3. IMPORTANT PROBLEM TYPES

The most important problem types
 Sorting
 Searching
 String processing
 Graph problems
 Combinatorial problems
 Geometric problems
 Numerical problems
(i) Sorting
 The sorting problem is to rearrange the items of a given list in nondecreasing (ascending)order.
 Sorting can be done on numbers, characters, strings or records.
 To sort student records in alphabetical order of names or by student number or by studentgrade-
point average. Such a specially chosen piece of information is called a key.
 The most important when we use the searching of records. There are different types of sorting
algorithms. There are some algorithms that sort an arbitrary of size n using nlog2n comparisons,
 On the other hand, no algorithm that sorts by key comparisons can do better thanthat.
 Although some algorithms are better than others, there is no algorithm that wouldbe the best in all
situations.
 Some algorithms are simple but relatively slow while others are faster but more complex.
 Some are suitable only for lists residing in the fast memory while others can be adapted for
sorting large files stored on adisk, and so on.
 There are two important properties.
 The first is called stable, if it preserves the relative order ofany two equal elements in its input. For
example, if we sort the student list based on their GPA and if two student’s GPA are the same, then
the elements are stored or sorted based on its position.
 The second is said to be ‘in place’ if it does not require extra memory. There are some sorting
algorithms that are in place and those that are not.
 An algorithm is said to be in-place if it does not require extra memory, E.g., Quick sort.
 A sorting algorithm is called stable if it preserves the relative order of any two
equalelements in its input.
 lOMoARcPSD|308 972 86

(ii) Searching
 The searching problem deals with finding a given value, called a search key, in agiven set.
 E.g., Ordinary Linear search and fast binary search.
 The searching can be either a straightforward algorithm or binary search algorithmwhich is a
differentform.
 These algorithms play an important role in real-life applications because they areused for
storingand retrieving information from large databases.
 Some algorithms work faster but require morememory, some are very fast butapplicable only
to sorted arrays.

 Searching, mainly deals with addition and deletion of records. In such cases, the data structures
and algorithms are chosen to balance among the required set of operations.

(iii) String processing

 A string is a sequence of characters from an alphabet.
 Strings comprise letters, numbers, and special characters; bit strings, which comprise zeros and
ones; and gene sequences, which can be modeled by strings of characters from the four- character
alphabet {A, C, G, T}. It is very useful in bioinformatics.
 Searching for a given word in a text is called string matching

(iv) Graph problems

 A graph is a collection of points called vertices, some of which are connected by linesegments
called edges.
 Some of the graph problems are graph traversal, shortest path algorithm, topological
sort,traveling salesman problem and the graph-coloring problem and so on.

(v) Combinatorial problems

 These are problems that ask, explicitly or implicitly, to find a combinatorial objectsuch as
apermutation, a combination, or a subset that satisfies certain constraints.
 A desired combinatorial object may also be required to have some additionalproperty suchs a
maximum value or a minimum cost.
 In practical, the combinatorial problems are the most difficult problems incomputing.
 The traveling salesman problem and the graph coloring problem are
examples of combinatorial problems.

(vi) Geometric problems

 Geometric algorithms deal with geometric objects such as points, lines, andpolygons.
 Geometric algorithms are used in computer graphics, robotics, and tomography.
 The closest-pair problem and the convex-hull problem are comes under thiscategory.

(vii) Numerical problems

 Numerical problems are problems that involve mathematical equations, systems ofequations,
computing definite integrals, evaluating functions, and so on. The majority of such mathematical
problems can be solved only approximately.
 lOMoARcPSD|308 972 86

 How do you measure the efficiency of an algorithm [Part A – Apr/May 2019]

3. FUNDAMENTALS OF THE ANALYSIS OF ALGORITHM EFFICIENCY
The efficiency of an algorithm can be in terms of time and space. The algorithmefficiencycan be
analyzed by the following ways.
a. Analysis Framework.
b. Asymptotic Notations and its properties.
c. Mathematical analysis for Recursive algorithms.
d. Mathematical analysis for Non-recursive algorithms.

a. Analysis Framework
There are two kinds of efficiencies to analyze the efficiency of any algorithm. They are:
i. Time efficiency, indicating how fast the algorithm runs, and
ii. Space efficiency, indicating how much extra memory it uses.

The algorithm analysis framework consists of the following:

iii. Measuring an Input’s Size
iv. Units for Measuring Running Time
v. Orders of Growth
vi. Worst-Case, Best-Case, and Average-Case Efficiencies

(i) Measuring an Input’s Size

 An algorithm’s efficiency is defined as a function of some parameter n indicating the algorithm’s
input size. In most cases, selecting such a parameter is quite straightforward. For example, it will
be the size of the list for problems of sorting,searching.
 For the problem of evaluating a polynomial p(x) = anxn + . . . + a0 of degree n, the size of the
parameter will be the polynomial’s degree or the number of its coefficients, which is larger by 1
than its degree.
 In computing the product of two n × n matrices, the choice of a parameter indicating an input size
does matter.
 Consider a spell-checking algorithm. If the algorithm examines individual characters of its input,
then the size is measured by the number of characters.
 In measuring input size for algorithms solving problems such as checking primality of a positive
integer n. the input is just one number.
 The input size by the number b of bits in the n’s binary representation is b=(log2 n)+1.

(ii) Units for Measuring Running Time

Some standard unit of time measurement such as a second, or millisecond, and soon can beused to
measure the running time of a program after implementing the algorithm.
 lOMoARcPSD|308 972 86

Drawbacks
 Dependence on the speed of a particular computer.
 Dependence on the quality of a program implementing the algorithm.
 The compiler used in generating the machine code.
The difficulty of clocking the actual running time of the program. One possible
approach is to count the number of times each of the algorithm’soperations is executed. This
approach is excessively difficult. The most importantoperation (+, -, *, /) of the algorithm,
called the basic operation.Computing thenumber of times, the basic operation is executed is
easy. The total running time is basics
of unit time.
(iii) Orders of Growth
 A difference in running times on small inputs is not what really distinguishes efficient algorithms
from inefficient ones.

For example, the greatest common divisor of two small numbers, it is not immediately clear how
much more efficient Euclid’s algorithm is compared to the other algorithms, thedifference in
algorithm efficiencies becomes clear for larger numbers only.
TABLE 1.1 Values (approximate) of several functions important for analysis of algorithms

n log2n n n log2n n2 n3 2n n!

√𝑛
1 1 0 1 0 1 1 2 1
2 1.4 1 2 2 4 4 4 2
4 2 2 4 8 16 64 16 24
8 2.8 3 8 2.4•101 64 5.1•10 2.6•102 4.0•104
2
10 3.2 3.3 10 3.3•101 102 103 103 3.6•106
16 4 4 16 6.4•101 2.6•10 4.1•10 6.5•104 2.1•101
2 3 3
102 10 6.6 10 6.6•102 104 106 1.3•10 9.3•101
2 30 57
103 31 10 103 1.0•104 106 109
Very big
104 102 13 104 1.3•105 108 1012 computation

105 3.2•10 17 105 1.7•106 1010 1015

2

106 103 20 106 2.0•107 1012 1018

 lOMoARcPSD|308 972 86

(iv) Worst-Case, Best-Case, and Average-Case EfficienciesConsider

Sequential Search algorithm some search key K ALGORITHM
SequentialSearch(A[0..n - 1], K)
//Searches for a given value in a given array by sequential search
//Input: An array A[0..n - 1] and a search key K
//Output: The index of the first element in A that matches K or -1 if there are no
// matching elements
i ←0
while i < n and A[i] ≠ K do
i
←i + 1
if i < n
return ielse
return -1
Clearly, the running time of this algorithm can be quite different for the same list size n.

In the worst case, there is no matching of elements or the first matching element can foundat last
on the list. In the best case, there is matching of elements at first on the list.

Worst-case efficiency
 The worst-case efficiency of an algorithm is its efficiency for the worst case inputof size n.
 The algorithm runs the longest among all possible inputs of that size.
 For the input of size n, the running time is Cworst(n) = n.
Best case efficiency
 The best-case efficiency of an algorithm is its efficiency for the best case input ofsize n.
 The algorithm runs the fastest among all possible inputs of that size n.
 In sequential search, If we search a first element in list of size n. (i.e. first elementequal toa search
key), then the running time is Cbest(n) = 1

Average case efficiency

 The Average case efficiency lies between best case and worst case.
 To analyze the algorithm’s average case efficiency, we must make someassumptions about
possible inputs of size n.
 The standard assumptions are that
o The probability of a successful search is equal to p (0 ≤ p ≤ 1) and
o The probability of the first match occurring in the ith position of the list is

the samefor every i.

For example , if p=1(successful) , the average number of key comparisons made by
sequential search is 𝑛(𝑛+1) i.e., the algorithm will inspect , on average , about half of the
2
list’s elements. If p=0(unsuccessful), the average number of key comparisons will be n
 lOMoARcPSD|308 972 86

because the algorithm will inspect all n elements on all such inputs.
(𝑛+1)
(v)CAvg(n) =
2

Yet another type of efficiency is called amortized efficiency. It applies not to a single run ofan
algorithm but rather to a sequence of operations performed on the same data structure.
It turns out that in some situations a single operation can be expensive, but the total timefor an entire
sequence of n such operations is always significantly better than the worst- case efficiency of that single
operation multiplied by n.

 Prove that the if f(n)=O(g(n)) and g(n)=O(f(n)), then f(n)=⍬g(n)[Part A- APR/MAY 2019]
 State the transpose symmetry of theta and omega [Part A- NOV/DEC 2019]
 Compare the orders of growth of n(n-1)/2 and n2. [Part A - MAY/JUNE 2016]
 Elaborate asymptotic analysis of an algorithm with an example. [Part B- NOV/DEC2019]
 Give the definition and graphical representation of O-Notation. [Part B -MAY/JUNE 2016]
 Briefly explain Big oh Notation, Omega Notation, Theta Notation, Give examples. [PartB-
APR/MAY 2017 & 2018 & 2019]

4. ASYMPTOTIC NOTATIONS AND ITS PROPERTIES

 Asymptotic notation is a notation, which is used to take meaningful statement about

theefficiency of a program.
 The efficiency analysis framework concentrates on the order of growth of an algorithm’sbasic
operation count as the principal indicator of the algorithm’s efficiency.

 To compare and rank such orders of growth, computer scientists use threenotations, they are:
o
i. O - Big oh notation
ii. Ω - Big omega notation
iii. Θ - Big theta notation
Let t(n) and g(n) can be any nonnegative functions defined on the set of natural numbers. The
algorithm’s running time t(n) usually indicated by its basic operation count C(n), and g(n), some
simple function to compare with the count.

Example 1:

where g(n) = n2.

(i) O - Big oh notation
A function t(n) is said to be in O(g(n)), denoted 𝑛 (𝑛) ∈ 𝑛(𝑛(𝑛)), if t (n) is bounded above by some
constant multiple of g(n) for all large n, i.e., if there exist some positive constant c and some nonnegative
integer n0 such that
𝑛 (𝑛) ≤ 𝑛(𝑛) fo𝑛 𝑛𝑛𝑛 𝑛 ≥ 𝑛0.
 lOMoARcPSD|308 972 86

Where t(n) and g(n) are nonnegative functions defined on the set of natural numbers.O =
Asymptotic upper bound = Useful for worst case analysis = Loose bound

FIGURE 1.5 Big-oh notation: 𝑛 (𝑛) ∈ (𝑛(𝑛)).

Example 2: Prove the assertions 100𝑛 + 5 ∈
2
(𝑛 ).
Proof: 100n + 5 ≤ 100n + n (for all n ≥ 5)
= 101n

≤ 101n2 (Ӭ 𝑛 ≤ 𝑛2)
Since, the definition gives us a lot of freedom in choosing specific values for constants c
and n0. We have c=101 and n0=5
Example 3: Prove the assertions 100𝑛 + 5 ∈ (𝑛). Proof: 100n
+ 5 ≤ 100n + 5n (for all n ≥ 1)
= 105n
i.e., 100n + 5
≤ 105ni.e., t(n)
≤ cg(n)
±100𝑛 + 5 ∈ (𝑛) with c=105 and n0=1

(ii) Ω - Big omega notation

A function t(n) is said to be in Ω(g(n)), denoted t(n) ∈ Ω(g(n)), if t(n) is bounded below by some
positive constant multiple of g(n) for all large n, i.e., if there exist some positive constant c and some
nonnegative integer n0 such that
t (n) ≥ cg(n) for all n ≥ n0.
Where t(n) and g(n) are nonnegative functions defined on the set of natural numbers.
Ω = Asymptotic lower bound = Useful for best case analysis = Loose bound

FIGURE 1.6 Big-omega notation: t (n) ∈ Ω (g(n)).

 lOMoARcPSD|308 972 86

Example 4: Prove the assertions n3+10n2+4n+2 ∈ Ω (n2).

Proof: n3+10n2+4n+2 ≥ n2 (for all n ≥ 0)
i.e., by definition t(n) ≥ cg(n), where c=1 and n0=0

(iii) Θ - Big theta notation

A function t(n) is said to be in Θ(g(n)), denoted t(n) ∈ Θ(g(n)), if t(n) is bounded both aboveand
below by some positive constant multiples of g(n) for all large n, i.e., if there exist some positive constants
c1 and c2 and some nonnegative integer n0 such that
c2g(n) ≤ t (n) ≤ c1g(n) for all n ≥ n0.
Where t(n) and g(n) are nonnegative functions defined on the set of natural numbers.
Θ = Asymptotic tight bound = Useful for average case analysis

FIGURE 1.7 Big-theta notation: t (n) ∈ Θ(g(n)).

Note: asymptotic notation can be thought of as "relational operators" for functionssimilar to

thecorresponding relational operators for values.
= ⇒ Θ(), ≤ ⇒ O(), ≥ ⇒ Ω(), < ⇒ o(), >⇒
ω()

5. Analysis Framework – Empirical analysis

Useful Property Involving the Asymptotic Notations

The following property, in particular, is useful in analyzing algorithms that comprise
twoconsecutively executed parts.

THEOREM: If t1(n) ∈ O(g1(n)) and t2(n) ∈ O(g2(n)), then t1(n) + t2(n) ∈ O(max{g1(n),g2(n)}).(The
analogous assertions are true for the Ω and Θ notations as well.)

PROOF: The proof extends to orders of growth the following simple fact about fourarbitrary realnumbers
a1, b1, a2, b2: if a1 ≤ b1 and a2 ≤ b2, then a1 + a2 ≤ 2 max{b1, b2}.
Since t1(n) ∈ O(g1(n)), there exist some positive constant c1 and some nonnegative integern1 such
that
t1(n) ≤ c1g1(n) for all n ≥ n1.
 lOMoARcPSD|308 972 86

Similarly, since t2(n) ∈ O(g2(n)),

t2(n) ≤ c2g2(n) for all n ≥ n2.
Let us denote c3 = max{c1, c2} and consider n ≥ max{n1, n2} so that wecan useboth
inequalities. Adding them yields the following:
t1(n) + t2(n) ≤ c1g1(n) + c2g2(n)
≤ c3g1(n) + c3g2(n)
= c3[g1(n) + g2(n)]
≤ c32 max{g1(n), g2(n)}.
Hence, t1(n) + t2(n) ∈ O(max{g1(n), g2(n)}), with the constants c and n0 requiredby
thedefinition O being 2c3 = 2 max{c1, c2} and max{n1, n2}, respectively.

The property implies that the algorithm’s overall efficiency will be determined bythe part with a
higher order of growth, i.e., its least efficient part.
± t1(n) ∈ O(g1(n)) and t2(n) ∈ O(g2(n)), then t1(n) + t2(n) ∈ O(max{g1(n), g2(n)}).

Basic rules of sum manipulation

Summation formulas

Properties of Asymptotic Notation

Practice

 Define Recursion [Part A - NOV/DEC 2019]

 Solve the following recurrence equation using iteration method or tree [Part B - NOV/DEC 2019]

 Write an algorithm using recursion that determines the GCD of two numbers. Determine the
time and space Complexity.

 Solve the following recurrence equation: [Part B – Apr/May 2019]

t(n) = t(n/2)+1 where n= 2k for all k>=0

T(n)=T(n/3)+T(2n/3)+cn where ‘c’ is a constant and ‘n’ is the input size.
 Give the recursive algorithm for finding the number of binary digit in n’s binary representation
where n is a positive decimal integer. Find the recurrence relation and complexity. [Part A -
MAY/JUNE 2016]
 lOMoARcPSD|308 972 86

 Briefly explain the mathematical analysis of recursive and non-recursive algorithm. [Part B-
APR/MAY 2019 & 2022]
 Give the General plan for analysing the time efficiency of recursive algorithms and use recurrence
to find number of moves for Towers of Hanoi Problem. [Part B- APR/MAY 2019]
6. MATHEMATICAL ANALYSIS FOR RECURSIVE ALGORITHMS
General Plan for Analyzing the Time Efficiency of Recursive Algorithms
1. Decide on a parameter (or parameters) indicating an input’s size.
2. Identify the algorithm’s basic operation.
3. Check whether the number of times the basic operation is executed can vary on different inputs of
the same size; if it can, the worst-case, average-case, and best- case efficiencies must be
investigated separately.
4. Set up a recurrence relation, with an appropriate initial condition, for the number of times the
basic operation is executed.
5. Solve the recurrence or, at least, ascertain the order of growth of its solution.
EXAMPLE 1: Compute the factorial function F(n) = n! for an arbitrary nonnegative integer n.Since n!= 1•.
. . . • (n − 1) • n = (n − 1)! • n, for n ≥ 1 and 0!= 1 by definition,we can computeF(n) = F(n − 1) • n with
the following recursive algorithm.
(ND 2015)
ALGORITHM F(n)
//Computes n! recursively
//Input: A nonnegative integer n
//Output: The value of n!
if n = 0 return 1
else return F(n − 1) * n

Algorithm analysis
 For simplicity, we consider n itself as an indicator of this algorithm’s input size. i.e. 1.
 The basic operation of the algorithm is multiplication, whose number of executions we denote
M(n). Since the function F(n) is computed according to the formula F(n) = F(n −1)•nfor n > 0.
 The number of multiplications M(n) needed to compute it must satisfy the equality M(n) = M(n-1)
+ 1 for n > 0

To To
comp multip ly
ute F(n-1) b
F(n-
1)
 lOMoARcPSD|308 972 86

M(n − 1) multiplications are spent to compute F(n − 1), and one more multiplication is needed to multiply
the result by n.

Recurrence relations
The last equation defines the sequence M(n) that we need to find. This equation defines M(n)
not explicitly, i.e., as a function of n, but implicitly as a function of its value at another point, namely n −
1. Such equations are called recurrence relations or recurrences.

Solve the recurrence relation (𝑛) = 𝑛(𝑛 − 1) + 1, i.e., to find an explicit formula for
M(n) in terms of n only.
To determine a solution uniquely, we need an initial condition that tells us thevalue with
which the sequence starts.
We can obtain this value by inspecting the condition that makes the algorithm stopits recursive
calls:
if n = 0 return 1.
This tells us two things. First, since the calls stop when n = 0, the smallest value of n for which
this algorithm is executed and hence M(n) defined is 0.
Second, by inspecting the pseudocode’s exiting line, we can see that when n = 0,the algorithm
performs no multiplications.

Thus, the recurrence relation and initial condition for the algorithm’s number ofmultiplications
M(n):
M(n) = M(n − 1) + 1 for
n > 0,M(0) = 0 for n
= 0.

Method of backward substitutions

M(n) = M(n − 1) + 1 substitute M(n − 1) = M(n − 2) + 1
= [M(n − 2) + 1]+ 1
= M(n − 2) + 2 substitute M(n − 2) = M(n − 3) + 1
= [M(n − 3) + 1]+ 2
= M(n − 3) + 3
…

= M(n − i) + i
…
= M(n − n) + n
= n.
Therefore M(n)=n

EXAMPLE 2: consider educational workhorse of recursive algorithms: the Tower of Hanoipuzzle. We

have n disks of different sizes that can slide onto any of three pegs. Consider A (source), B (auxiliary),
and C (Destination). Initially, all the disks are on the first peg in order of size, the largest on the bottom
and the smallest on top. The goal is to move all the disks to the third peg, using the second one as an
auxiliary.
18
 lOMoARcPSD|308 972 86

FIGURE 1.8 Recursive solution to the Tower of Hanoi puzzle.

ALGORITHM
TOH(n, A, C, B)
//Move disks from source to destination recursively
//Input: n disks and 3 pegs A, B, and C
//Output: Disks moved to destination as in the source order.
if n=1
Move disk from A to C
else
Move top n-1 disks from A to Busing
CTOH(n - 1, A, B, C)
Move top n-1 disks from B to C usingATOH(n -
1, B, C, A)

Algorithm analysis
The number of moves M(n) depends on n only, and we get the followingrecurrenceequation for it:
M(n) = M(n − 1) + 1+ M(n − 1) for n > 1.
With the obvious initial condition M(1) = 1, we have the following recurrence relationfor thenumber of
moves M(n):
M(n) = 2M(n − 1) + 1 for
n > 1,M(1) = 1.
We solve this recurrence by the same method of backward substitutions:
M(n) = 2M(n − 1) + 1 sub. M(n − 1) = 2M(n − 2) + 1
= 2[2M(n − 2) + 1]+ 1
= 22M(n − 2) + 2 + 1 sub. M(n − 2) = 2M(n − 3) + 1

19
 lOMoARcPSD|308 972 86

= 22[2M(n − 3) + 1]+ 2 + 1
= 23M(n − 3) + 22 + 2 + 1 sub. M(n − 3) = 2M(n − 4) + 1
= 24M(n − 4) + 23 + 22 + 2 + 1
…
= 2iM(n − i) + 2i−1 + 2i−2 + . . . + 2 + 1= 2iM(n − i) + 2i − 1.
…
Since the initial condition is specified for n = 1, which is achieved for i = n
– 1, M(n) = 2n−1M(n − (n − 1)) + 2n−1 – 1 = 2n−1M(1) + 2n−1 − 1= 2n−1 + 2n−1 −
1= 2n − 1.
Thus, we have an exponential time algorithm

EXAMPLE 3: An investigation of a recursive version of the algorithm which finds thenumber ofbinary
digits in the binary representation of a positive decimal integer.

ALGORITHM BinRec(n)
//Input: A positive decimal integer n
//Output: The number of binary digits in n’s binary representation
if n = 1 return 1
else return BinRec(‫ہ‬n/2])+ 1

Algorithm analysis
The number of additions made in computing BinRec(‫ہ‬n/2]) is A(‫ہ‬n/2]), plus one more addition
is made by the algorithm to increase the returned value by 1. This leads tothe recurrence A(n) = A(‫ہ‬n/2]) +
1 for n > 1.
then, the initial condition is A(1) = 0.
The standard approach to solving such a recurrence is to solve it only for n =
2kA(2k) = A(2k−1) + 1 for k > 0,
A(20) = 0.

backward substitutions
A(2k) = A(2k−1) + 1 substitute A(2k−1) = A(2k−2) + 1
= [A(2k−2) + 1]+ 1= A(2k−2) + 2 substitute A(2k−2) = A(2k−3) + 1
= [A(2k−3) + 1]+ 2 = A(2k−3) + 3 ...
...
= A(2k−i) + i
...
= A(2k−k) + k.
Thus, we end up with A(2k) = A(1) + k = k, or, after returning to the original variable n
= 2k andhence k = log2 n,A(n) =
log2 n ϵ Θ (log2 n).

 Write an algorithm for determining the uniqueness of an array. Determine the time complexity
of your algorithm [Part b– Apr/May 2019]
 Give the algorithm to check whether all the elements in a given array of n elements are distinct.
Find Worst case complexity of the same. [Part B - MAY/JUNE 2016]

20
 lOMoARcPSD|308 972 86

7. MATHEMATICAL ANALYSIS FOR NON-RECURSIVE ALGORITHMS

General Plan for Analyzing the Time Efficiency of NonrecursiveAlgorithms
1. Decide on a parameter (or parameters) indicating an input’s size.
2. Identify the algorithm’s basic operation (in the innermost loop).
3. Check whether the number of times the basic operation is executed depends only on the sizeof an
input. If it also depends on some additional property, the worst- case, average-case, and, if
necessary, best-case efficiencies have to be investigatedseparately.
4. Set up a sum expressing the number of times the algorithm’s basic operation is executed.
5. Using standard formulas and rules of sum manipulation either find a closed form formulafor the
count or at the least, establish its order of growth.

EXAMPLE 1: Consider the problem of finding the value of the largest element in alist of nnumbers.
Assume that the list is implemented as an array for simplicity.
ALGORITHM MaxElement(A[0..n − 1])
//Determines the value of the largest element in a given array
//Input: An array A[0..n − 1] of real numbers

//Output: The value of the largest element in Amaxval

←A[0]
for i ←1 to n − 1 do
if A[i]>maxval
maxval←A[i]
return maxval

Algorithm analysis
 The measure of an input’s size here is the number of elements in the array, i.e., n.
 There are two operations in the for loop’s body:
o The comparison A[i]> maxval and
o The assignment maxval←A[i].
 The comparison operation is considered as the algorithm’s basic operation, because the
comparison is executed on each repetition of the loop and not the assignment.
 The number of comparisons will be the same for all arrays of size n; therefore, there is no need to
distinguish among the worst, average, and best cases here.
 Let C(n) denotes the number of times this comparison is executed. The algorithm makesone
comparison on each execution of the loop, which is repeated for each value of the loop’s variable i
within the bounds 1 and n − 1, inclusive. Therefore, the sum for C(n) is calculated as follows:
𝑛−
𝑛(𝑛) = ∑
=
i.e., Sum up 1 in repeated n-1 times
𝑛−
𝑛(𝑛) = ∑ = 𝑛 − ∈ ()

21
 lOMoARcPSD|308 972 86

EXAMPLE 2: Consider the element uniqueness problem: check whether all theElements in agiven
array of n elements are distinct.
ALGORITHM UniqueElements(A[0..n − 1])
//Determines whether all the elements in a given array are distinct
//Input: An array A[0..n − 1]
//Output: Returns “true” if all the elements in A are distinct and “false” otherwise
for i ←0 to n − 2 do
for j ←i + 1 to n − 1 do
if A[i]= A[j ] return false

return true

Algorithm
analysis
 The natural measure of the input’s size here is again n (the number of elements inthe array).
 Since the innermost loop contains a single operation (the comparison of twoelements), we
should consider it as the algorithm’s basic operation.
 The number of element comparisons depends not only on n but also on whether there are equal
elements in the array and, if there are, which array positions they occupy. We will limit our
investigation to the worst case only.
 One comparison is made for each repetition of the innermost loop, i.e., for each value of the loop
variable j between its limits i + 1 and n − 1; this is repeated for each value of the outer loop, i.e.,
for each value of the loop variable i between its limits 0 and n − 2.

EXAMPLE 3: Consider matrix multiplication. Given two n × n matrices A and B, find the timeefficiency of
the definition-based algorithm for computing their product C = AB. By definition, C
is an n × n matrix whose elements are computed as the scalar (dot) products of the rows of matrix Aand
the columns of matrix B:

22
 lOMoARcPSD|308 972 86

where C[i, j ]= A[i, 0]B[0, j]+ . . . + A[i, k]B[k, j]+ . . . + A[i, n − 1]B[n − 1, j] for every pairofindices 0 ≤ i, j ≤
n − 1.

ALGORITHM MatrixMultiplication(A[0..n − 1, 0..n − 1], B[0..n − 1, 0..n − 1])

//Multiplies two square matrices of order n by the definition-based algorithm
//Input: Two n × n matrices A and B
//Output: Matrix C = AB
for i ←0 to n − 1 do for
j ←0 to n − 1 do C[i, j
]←0.0
for k←0 to n − 1 do
C[i, j ]←C[i, j ]+ A[i, k] ∗ B[k, j]
return C
Algorithm analysis
 An input’s size is matrix order n.
 There are two arithmetical operations (multiplication and addition) in the innermost loop. But
we consider multiplication as the basic operation.
 Let us set up a sum for the total number of multiplications M(n) executed by the algorithm. Since
this count depends only on the size of the input matrices, we do not have to investigate the worst-
case, average-case, and best-case efficiencies separately.
 There is just one multiplication executed on each repetition of the algorithm’s innermost loop,
which is governed by the variable k ranging from the lower bound0 to the upper bound n − 1.
 Therefore, the number of multiplications made for every pair of specific values of

variablesi and j is
The total number of multiplications M(n) is expressed by the following triple sum:

Now, we can compute this sum by using formula (S1) and rule (R1)

The running time of the algorithm on a particular machine m, we can do it by the product. If
we consider, time spent on the additions too, then the total time on the machine is basis of
running time.

23
 lOMoARcPSD|308 972 86

EXAMPLE 4 The following algorithm finds the number of binary digits in the binaryrepresentation
of a positive decimal integer.

• ALGORITHM Binary(n)
//Input: A positive decimal integer n
//Output: The number of binary digits in n’s binary
representationcount ←1
while n > 1 do
count
←count + 1
n←‫ہ‬n/2]
return count
Algorithm
analysis
 An input’s size is n.
 The loop variable takes on only a few values between its lower and upper limits.
 Since the value of n is about halved on each repetition of the loop, the answershould beabout
log2 n.
 The exact formula for the number of times.
 The comparison > 1 will be executed is actually ‫ہ‬log2 n] + 1.

8.ALGORITHM VISUALIZATION
 In addition to the mathematical and empirical analyses of algorithms, there is yet a third way to study
algorithms. It is called algorithm visualization and can be defined as the use of images to convey some
useful information about algorithms.

 That information can be a visual illustration of an algorithm’s operation, of its performance on different
kinds of inputs, or of its execution speed versus that of other algorithms for the same problem.

 To accomplish this goal, an algorithm visualization uses graphic elements—points, line segments, two-
or three-dimensional bars, and so on—to represent some “interesting events” in the algorithm’s
operation.

Static algorithm visualization

o It shows an algorithm’s progress through a series of still images. Algorithm animation, on the other hand,
shows a continuous, movie-like presentation of an algorithm’s operations.
 Animation is an arguably more sophisticated option, which, of course, is much more difficult to
implement.
 Early efforts in the area of algorithm visualization go back to the 1970s. The watershed event happened
in 1981 with the appearance of a 30-minute color sound film titled Sorting Out Sorting.

 This algorithm visualization classic was produced at the University of Toronto by RonaldBaecker with the
assistance of D. Sherman.

 The success of Sorting Out Sorting made sorting algorithms a perennial favorite for algorithm animation.
Indeed, the sorting problem lends itself quite naturally to visual presentation via vertical or horizontal
bars or sticks of different heights or lengths, which need to be rearranged according to their sizes.

 This presentation is convenient, however, only for illustrating actions of a typical sorting algorithm on
small inputs.

24
 lOMoARcPSD|308 972 86

 For larger files, Sorting Out Sorting used the ingenious idea of presenting data by a scatterplot of points
on a coordinate plane, with the first coordinate representing an item’s position in the file and the second
one representing the item’s value; with such a representation, the process of sorting looks like a
transformation of a “random” scatterplot of points into the points along a frame’s diagonal .

 In addition, most sorting algorithms work by comparing and exchanging two given items at a time—an
event that can be animated relatively easily.

 Since the appearance of Sorting Out Sorting, a great number of algorithm animations have been created,
especially after the appearance of Java and the World Wide Web in the 1990s.

 They range in scope from one particular algorithm to a group of algorithms for the same problem (e.g.,
sorting) or the same application area (e.g., geometric algorithms) to general purpose animation systems.
At the end of 2010, a catalog of links to existing visualizations, maintained under the NSF-supported
AlgoVizProject, contained over 500links.

 Unfortunately, a survey of existing visualizations found most of them to be of low quality, with the content
heavily skewed toward easier topics such as sorting [Sha07].
 There are two principal applications of algorithm visualization:

 research and
 education.

 Potential benefits for researchers are based on expectations that algorithm visualizationmay help uncover
some unknown features of algorithms. For example, one researcher used a visualization of the recursive
Tower of Hanoi algorithm in which odd- and even-numbered disks were colored in two different colors.

 He noticed that two disks of the same color never came in direct contact during the
algorithm’s execution.

 This observation helped him in developing a better non-recursive version of the classicalgorithm.

 To give another example, Bentley and McIlroy [Ben93] mentioned using an algorithm animation system
in their work on improving a library implementation of a leading sorting algorithm.
 The application of algorithm visualization to education seeks to help students learning algorithms.
 The available evidence of its effectiveness is decisively mixed. Although some experiments did register
positive learning outcomes, others failed to do so.
 The increasing body of evidence indicates that creating sophisticated software systems is not going to be
enough.
 In fact, it appears that the level of student involvement with visualization might be more important than
specific features of visualization software.
 In some experiments, low-tech visualizations prepared by students were more effective than passive
exposure to sophisticated software systems.
 To summarize, although some successes in both research and education have been reported in the
literature, they are not as impressive as one might expect.
 A deeper understanding of human perception of images will be required before the true potential of
algorithm visualization is fulfilled.

25