PAVAAI ENGINEERINGCOLLEGE

ACADEMIC YEAR 2024-2025

PRESENTER’ S MANUAL

DEPARTMENT OF ARTIFICIAL INTELLIGENCE AND DATA

SCIENCE
COURSE NAME: DESIGN AND ANALYSIS OF ALGORITHM
COURSE CODE: AD3351
YEAR/SEM: II YEAR – III SEMESTER
What is Paavai Teaching Methodology?
At Paavai Educational Institutions, inclusive, flexible and insightful learning aims to provide engaging
educational experiences and meet the needs of learners from all the backgrounds. Teachers should align their
teaching with everyday life which will make learning meaningful.
This semester is based on the teaching methodology following three cardinal components:
1. Concept Class
2. Lab Manual
3. Directed Learning Class

Concept class: This is a theory class that will focus on the concepts. Whenever required this session will also
demonstrate how these concepts get translated into analyse the efficiency of algorithm.
Lab Manual: The lab manual follows the concept class. The students learn to implement the concepts learnt
in the concept class.
Directed Learning Class: Learning and application may be challenging for some students. One of the oldest
and most comprehensive ways of delivery information, self-directed class allows the student to apply
themselves in a manner that makes understanding content more accessible. In this process, learners take
initiative in their own learning by planning, implementing and evaluating their learning.
The entire purpose of this methodology is to make the students more:
 Concept focused
 Adapted to real life work environment

Introduction to the Course

Need to study algorithms:

 There are both practical and theoretical reasons to study algorithms.

 From the practical point of view

- to know a standard set of important algorithms from different areas of computing.

- able to design new algorithms and analyze their efficiency.

 From theoretical point of view

- study of algorithms, sometimes called algorithmic, has come to be recognized as the

cornerstone of computer science.

 Another reason for studying algorithms is their usefulness in developing analytical

skills.

 Algorithms can be seen as special kinds of solutions to problems-not answers but rather
precisely defined procedures for getting answers.

 Well trained computer science professionals knows how to deal with algorithms:
 - How to construct them.

- Manipulate them.

- Understand them.

- Analyze them.

The knowledge is preparation for much more than writing good computer programs.

 Actually, a person does not really understand something until after teaching it to someone else.
Similarly, a person does not understand until after teaching it to a computer.

i.e., Expressing it as an algorithms.

Prerequisites for taking this course

 Understanding of fundamental engineering concepts.
 Proficiency in at least one programming language.(eg. Python, Java etc)
 Familiar with basic Data Structure.

Course Objectives:

a.To critically analyze the efficiency of alternative algorithmic solutions for the same
problem

b.To illustrate brute force and divide and conquer design techniques.
c.To explain dynamic programming and greedy technique for solving various problems.
d.To apply iterative improvement technique to solve optimization problems
e.To examine the limitations of algorithmic power and handling it in different problems.

SYLLABUS
AD3351 DESIGN AND ANALYSIS OF ALGORITHM T T P C
3024
Unit I: Introduction 8+3
Notion of an Algorithm –Fundamentals of Algorithmic Problem Solving–Important Problem Types –
Fundamental of the Analysis of Algorithmic Efficiency –Asymptotic Notations and their properties.
Analysis Framework -Empirical analysis-Mathematical analysis for Recursive and Non-recursive
algorithms–Visualization.
Unit II: Brute Force And Divide-And-Conquer 10+3
Brute Force – String Matching - Exhaustive Search-Travelling Salesman- Problem-Knapsack
Problem-Assignment problem. Divide and Conquer Methodology-Multiplication of Large Integers
and Strassen’s Matrix Multiplication–Closest-Pair and Convex Hull Problems. Decrease and Conquer
Method: Topological Sorting-Transform and Conquer Method: Presorting – Heaps and Heap Sort.
 Unit III: Dynamic Programming And Greedy Technique 11+3
Dynamic programming – Principle of optimality - Coin changing problem, Warshall’s and Floyd‘s
algorithm-Optimal Binary Search Trees–Multistage graph-Knapsack Problem and Memory functions.
Greedy Technique- Dijkstra’s algorithm – Huffman Trees and codes-0/1 Knapsack problem.
Unit IV: Iterative Improvement 7+3
The Simplex Method- The Maximum –Flow Problem–Maximum Matching in Bipartite Graphs-The
Stable marriage Problem.
Unit V: Coping with the limitations of algorithm power 9+3
Lower - Bound Arguments - P, NP NP- Complete and NP Hard Problems. Backtracking – n-Queen
problem - Hamiltonian Circuit Problem – Subset Sum Problem. Branch and Bound –LIFO Search
and FIFO search - Assignment problem –Knapsack Problem – Travelling Salesman Problem -
Approximation Algorithms for NP-Hard Problems – Travelling Salesman problem–Knapsack
problem.

Total hours: 45

Student Learning Outcomes:

Clearly written student learning outcomes are the foundation upon which effective courses are designed.
Outcomes inform both the ways students are evaluated in a course and the way a course will be organized.
Effective learning outcomes are student-centered, measurable, concise, meaningful, achievable and outcome-
based (rather than task-based). The course contents are designed for those who are keen on getting
themselves a career in the field of computer science, specifically for those who want to be involved in the
design and implementation of algorithm.
At the end of the course, the students will be able to:
 Analyze the efficiency of recursive and non-recursive algorithms
Mathematically.
 Analyze the efficiency of brute force, divide and conquer, decrease
and conquer, Transform and conquer algorithmic techniques.
 Implement and analyze the problems using dynamic programming
and greedy technique algorithmic techniques.
 Solve the problems using iterative improvement technique for
optimization.
 Compute the limitations of algorithmic power and solve the problems
using backtracking and branch and bound technique.

TEXT BOOKS PRESCRIBED:

Anany Levitin, Introduction to the Design and Analysis of Algorithms, Third Edition, Person Education,
2012.

REFERENCE BOOKS:
 1. Ellis Horowitz, Sartaj Sahni and Sanguthevar Rajasekaran, Computer Algorithms/ C++, Second
Edition, Universities Press, 2019.
2. Thomas H. Cormen, Charles E. Leiserson, Ronald L.Rivest and Clifford Stein, Introduction to
Algorithms, Third Edition, PHI Learning Private Limited, 2012.
3. S.Sridhar, Design and Analysis of Algorithms, Oxford University Press, 2014.
4. Alfred, Data Structures and Algorithms, Pearson Eductaion, Reprint 2006.

CO’s- PO’s & PSO’s MAPPING

CO’s PO’s PSO’s

1 2 3 4 5 6 7 8 9 10 11 12 1 2 3
1 3 3 3 1 1 - - - 1 1 2 2 3 2 1
2 2 1 1 3 2 - - - 2 2 1 2 2 2 2
3 3 2 1 2 2 - - - 2 1 1 2 1 3 3
4 3 2 3 2 2 - - - 3 3 3 2 2 1 2
5 3 1 2 3 3 - - - 2 2 2 2 3 1 3
AVG 3 2 2 2 2 - - - 2 2 2 2 2 2 2
1 - low, 2 - medium, 3 - high, ‘-' - no correlation

* For Entire Course, PO & PSO Mapping

POs & PSO REFERENCE:

PO Engineering Knowledge PO7 Environment & Sustainability PSO1 Professional Skills
1
PO Problem Analysis PO8 Ethics PSO2 Problem-Solving Skills
2
PO Design & Development PO9 Individual & Team Work PSO3 Successful Career and
3 Entrepreneurship
PO Investigations PO10 Communication Skills
4
PO Modern Tools PO11 Project Mgt. & Finance
5
PO Engineer & Society PO12 Life Long Learning
6

UNIT WISE HOURS ALLOCATION

Course Concept Class Dir. Learning Class Total
Hrs. Hrs. Hrs.
Unit - 1 7 2 9
Unit - 2 7 2 9
Unit - 3 7 2 9
Unit - 4 7 2 9
Unit - 5 7 2 9
Total 35 10 45
 I. CONCEPT CLASS
LESSON PLAN

UNIT 1: INTRODUCTION 9 hours

Notion of an Algorithm –Fundamentals of Algorithmic Problem Solving–Important Problem Types –
Fundamental of the Analysis of Algorithmic Efficiency –Asymptotic Notations and their properties.
Analysis Framework -Empirical analysis-Mathematical analysis for Recursive and Non-recursive
algorithms–Visualization.

Method Lecture
S.No Proposed notes – Teaching
Topics to be covered Ref
. Date Page Method
Number
Notion of an CC
Algorithm – Fundamentals of
1. 15.07.24 1- 2 T1 CB/L
Algorithmic Problem Solving-
Important Problem Types
Fundamentals of the Analysis CC
2. 16.07.24 3-7 T1 CB/L
of Algorithmic Efficiency
Asymptotic Notations and their CC
3. 18.07.24 8 - 11 T1 CB/L
properties
Analysis Framework CC
4. 19.07.24 11 - 12 T1 CB/L
– Empirical analysis
Mathematical analysis for CC
5. Recursive 20.07.24 13 - 15 T1 CB/L
Algorithms
Mathematical CC
6. analysis for Non- 22.07.24 16 - 18 T1 CB/L
Recursive Algorithms
7. Visualization CC 23.07.24 19 - 20 T1 CB/L
Example Problems in Recursive CC
8. 24.07.24 23 - 24 T1 CB/L
Algorithm Analysis
Example Problems in Non- CC
9. 25.07.24 25- 27 T1 CB/L
Recursive Algorithm Analysis

*T1 – Textbook 1, CC – Concept Class, DL – Directed Learning

 PAAVAI COLLEGE OF ENGINEERING
DEPARTMENT OF ARTIFICIAL INTELLIGENCE & DATA SCIENCE
Lecture no 1
Subject Name Design and analysis of Algorithm
Date 12.08.2024
Subject Code AD3351 Day 1

Unit 1 Introduction Of Algorithm Hour

Topic covered Notion of Algorithm

S.no Time Structure

1 2 Mins Attendance

2 2 Mins Technical Terms

3 7 Mins Revision: Importance of algorithm.

4 1 min Objective: Introduce the concept of algorithm

35 Content: Notion of algorithm

5
Mins

6 3 Mins Questions by Students :

Revision and Questions:

7 3 Mins
What is algorithm? And its uses?

Outcome: The student should be able to understand the concept of understand

8 1 Mins
the basics of algorithm.

9 1 Mins Next Class : Fundamentals of problem solving

Remarks:
 Faculty Incharge

PAAVAI COLLEGE OF ENGINEERING

DEPARTMENT OF ARTIFICIAL INTELLIGENCE & DATA SCIENCE

Design and analysis of Algorithm Lecture no 2

Subject Name
Date
Subject Code AD3351 Day

Unit 1 Introduction Of Algorithm Hour

Topic covered Fundamentals of Problem solving

S.no Time Structure

1 2 Mins Attendance

2 2 Mins Technical Terms

3 7 Mins Revision: Basics of algorithm

4 1 min Objective: Introduce the fundamentals of problem solving

Content:
35 Fundamental Problem solving.
5
Mins Design
Analysis the Algorithm

Questions by Students :
6 3 Mins

Revision and Questions:

7 3 Mins What is algorithm?
What is notion of algorithm?
Outcome: The student should be able to understand the concept of algorithm.
8 1 Mins

9 1 Mins Next Class : Importance of Problem types

Remarks:
 Faculty Incharge

PAAVAI COLLEGE OF ENGINEERING

DEPARTMENT OF ARTIFICIAL INTELLIGENCE & DATA SCIENCE
Design and analysis of Algorithm Lecture no 3
Subject Name
Date
Subject Code AD3351 Day
Unit 1 Introduction Of Algorithm Hour

Topic covered Notion of Algorithm

S.no Time Structure

1 2 Attendance
Mins
2 2 Technical Terms
Mins

3 7 Revision: Basics of algorithm

Mins
4 1 min Objective: Introduce the fundamentals of algorithm efficiency
Content:
35 Algorithm
5
Mins Fundamental of algorithm
Problem solving.
Questions by Students :
6 3
Mins
Revision and Questions:
7 3 Mins How to design the algorithm?
What are the fundamentals of algorithm?

Outcome: The student should be able to understand the concept of

8 1
fundamentals of algorithm and problem solving.
Mins
9 1 Next Class : Asymptotic Notations and their properties
Mins

Remarks:
 Faculty Incharge

PAAVAI COLLEGE OF ENGINEERING

DEPARTMENT OF ARTIFICIAL INTELLIGENCE & DATA SCIENCE

Design and analysis of Algorithm Lecture no 4

Subject Name
Date
Subject Code AD3351 Day

Unit 1 Introduction Of Algorithm Hour

Topic covered Asymptotic Notations and their Properties

S.no Time Structure

1 2 Attendance
Mins
2 2 Technical Terms
Mins

3 7 Revision: Basics of algorithm

Mins
4 1 min Objective: Introduce the fundamentals of algorithm efficiency
Content:
35 BIG OH
5
Mins OMEGA
THETA
Questions by Students :
6 3
Mins
Revision and Questions:
7 3 Mins What are the two kinds of efficiency?

Outcome: The student should be able to understand the concept of big oh,
8 1
omega, theta.
Mins
9 1 Next Class : Empirical Efficiency
Mins
Remarks:

Faculty Incharge

PAAVAI COLLEGE OF ENGINEERING

DEPARTMENT OF ARTIFICIAL INTELLIGENCE & DATA SCIENCE
Design and analysis of Algorithm Lecture no 5
Subject Name
Date
Subject Code AD3351 Day

Unit 1 Introduction Of Algorithm Hour

Topic covered Empirical Analysis

S.no Time Structure

1 2 Attendance
Mins
2 2 Technical Terms
Mins

3 7 Revision:
Mins
4 1 min Objective: Introduce the analysis of empirical algorithm
Content:
35 Design
5
Mins Analysis The Empirical Algorithm

Questions by Students :
6 3
Mins
Revision and Questions:
7 3 Mins Properties of Asymptotic notations?
What is basic efficiency classes?
Outcome: The student should be able to understand the concept of Empirical
8 1
algorithm.
Mins
9 1 Next Class : Recursive and Non-Recursive algorithm
Mins
Remarks:
 Faculty Incharge

PAAVAI COLLEGE OF ENGINEERING

DEPARTMENT OF ARTIFICIAL INTELLIGENCE & DATA SCIENCE
Design and analysis of Algorithm Lecture no 6
Subject Name
Date
Subject Code AD3351 Day

Unit 1 Introduction Of Algorithm Hour

Topic covered Empirical Analysis

S.no Time Structure

1 2 Attendance
Mins
2 2 Technical Terms
Mins

3 7 Revision:
Mins
4 1 min Objective: Introduce the recursive and Non-recursive
Content:
35 Binary(n)
5
Mins Factorial numbers
Arrays
Questions by Students :
6 3
Mins
Revision and Questions:
7 3 Mins How the algorithm’s efficiency is to be measured?
Define Profiling?

Outcome: The student should be able to understand the concept of Recursion

8 1
and Non-Recursion
Mins
 9 1 Next Class : Visualization
Mins

Remarks:

Faculty Incharge

PAAVAI COLLEGE OF ENGINEERING

DEPARTMENT OF ARTIFICIAL INTELLIGENCE & DATA SCIENCE
Design and analysis of Algorithm Lecture no 7
Subject Name
Date
Subject Code AD3351 Day

Unit 1 Introduction Of Algorithm Hour

Topic covered Visualization of Algorithm

S.no Time Structure

1 2 Attendance
Mins
2 2 Technical Terms
Mins

3 7 Revision:
Mins
4 1 min Objective: Introduce the visualization of algorithm
Content:
35 Static algoritm
5
Mins Dynamic Algorithm

Questions by Students :
6 3
Mins
Revision and Questions:
7 3 Mins Analyzing Efficiency Of Recursive Algorithms.
 Outcome: The student should be able to understand the concept of
8 1
visualization of algorithm.
Mins
9 1 Next Class : Unit 2
Mins

Remarks:

Faculty Incharge

TECHNICAL TERMS
UNIT 1

S.No Technical Literal Meaning Technical Meaning

Term
Derived from the name of the
A step-by-step procedure or formula
Persian mathematician Al-
for solving a problem or completing
Khwarizmi, it refers to a systematic
1 Algorithm a task, often used in computing and
method of solving a problem or a set
mathematics to process data and
of rules to be followed in
perform calculations.
calculations.
A belief, idea, or opinion about A general concept or idea, especially
something; an individual's one that is fundamental to
2 Notion
understanding or perception. understanding or discussing a
particular subject area.
Related to or characterized by Refers to a process or function that
repetition or looping, often in a self- calls itself within its own definition,
similar manner. commonly used in computer science
3 Recursive and mathematics to solve problems
by breaking them down into smaller,
more manageable sub-problems.
Relating to or based on practical Based on observation, experience,
experience rather than theoretical or experiment rather than theory or
knowledge. pure logic; often used in scientific
4 Empirical research to describe data or evidence
gathered through direct observation
or experimentation.
5 Non-Recursive Not involving or characterized by Describes a process or function that
recursion; straightforward and non- does not involve recursion; it does
repetitive.
 not call itself within its own
definition and typically uses
iterative methods instead.
Describes a line or curve that Referring to the behavior of
approaches another curve arbitrarily functions as they approach a
closely but never actually touches it. particular value or infinity, often
used in mathematics and computer
6 Asymptotic science to describe the efficiency or
complexity of algorithms as input
size grows.

Characteristics or attributes that

describe an object, system, or
7 Properties phenomenon. In computer science,
it can refer to attributes or data
members associated with an object.

2. Student Course Material

Unit I INTRODUCTION

1.0 Notion of algorithms

What is an algorithm?

An algorithm is a sequence of unambiguous instructions for solving a problem. i.e.., For obtaining
a required output for any legitimate input in a finite amount of time. This definition can be
illustrated by a simple diagram in Fig 1.

Fig 1 Notion of
Algorithms

 As examples illustrating the notation of algorithm, we consider three methods for

solving the same problem.

Ex: Computing the greatest common divisor of two integers.

 These examples will help us to illustrate several important points.

-The non-ambiguity requirement for each step of an algorithm cannot be compromised.

-The range of inputs for which an algorithm works has to be specified carefully.

-The same algorithm can be represented in several different ways.

-Several algorithms for solving the same problem may exist.

- Algorithms for the same problem can be based on very different ideas and can solve the
problem with different speeds.
Problem:
Greatest common divisor of two non-negative, non-both zero integers m and n, denoted
gcd(m,n).
gcd(m,n) is defined as the largest integer that divides both m and n evenly, i.e., with a remainder
zero.

I Method:

Euchid’s algorithm is based on gcd(m,n) = gcd(n,m mod n) until m mod n is equal to ø, the value of
m is also the gcd of the initial m and n.

Ex: gcd(60,24) can be computed as gcd(60,24) = gcd(24,12) = gcd(12,0) = 12

Structured description of the algorithm:

Step 1: If n = 0 return the value of n as the answer and stop; otherwise proceed to
step 2.

Step 2: Divide m by n and assign the value of the remainder to r.

Step 3: Assign the value of n to m and the value of r to n. Goto step 1.

Algorithm: Euclid(m,n)

//Input: Two non-negative, not both-zero integers m&n.

//Output: gcd of m and n.
While n ≠ 0 do

rm mod n
mn
nr

How this loop terminates:

From the observation, the second number of the pair gets smaller with each iteration and it
cannot be negative.
Hence the value of the second number in the pair eventually becomes 0, and the
algorithm stops.
II Method:
Common divisor cannot be greater than the smaller of these numbers. (i.e) t =
min{m,n}, we check whether t divides both m&n.
if it does, t is the answer.
If it does not, decreases t by 1 and try again.
 Ex: m = 60 n = 24
min(60,24) = 24
The algorithm will try first 24, then 23 and so in until it reaches 12, where it stops. Structured

algorithm:

Step 1: Assign the value of min{m,n} to t.tep 2: Divide m by t. If the remainder of this division O,
goto step 3; Otherwise goto step 4.

Step 3: Divide n by t. If the remainder of this division is 0, return the value of t as the answer and
stop; Otherwise proceed to step 4.

Step 4: Decrease the value of t by 1. Goto step 2.

This algorithm does not work correctly, when one of its input numbers is zero.

This example illustrates, the importance of specifying the range of an algorithm’s inputs explicitly and
carefully.

III Method:

Structured algorithm: Middle-school procedure

Step 1: Find the prime factors of m.

Step 2: Find the prime factors of n.

Step 3: Identify all the common factors in the two prime expansions found in step 1 and
step 2.

Step 4: Compute the product of the all the common factors and return it as the greatest
common divisor of the numbers given.

Ex: 60 and 24

60 = 2. 2. 3. 5

24 = 2. 2. 2. 3

gcd(60,24) = 2. 2. 3 = 12

 This algorithm is much more complex and slower than Euclid’s algorithm.

 It does not quality, in the form presented, as a legitimate algorithm. Because the prime
factorization steps are not defined unambiguously. They require list of prime numbers.

 Simple algorithm for generating consecutive primes not exceeding any given integer n.
 The algorithm

- Starts by initializing a list of prime candidates with consecutive integers from 2 to n.

- On the I iteration of the algorithm, it eliminates from the list all multiples of 2. i.e., 4,6 and so
on.

- Then it moves to the next item on the list, which is 3 and eliminates its multiples.

 The algorithm continues in this fashion until no more numbers can be eliminated from the list.

 The remaining integers of the list are the primes needed.

Ex: Finding the list of primes not exceeding n = 25.

2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25

2 3 5 7 9 11 13 15 17 19 21 23 25

2 3 5 7 11 13 17 19 23 25

23 5 7 11 13 17 19 23

Number of passes is needed because

 they would eliminate numbers already eliminated on previous iterations of the algorithm.

 The remaining numbers on the list are the consecutive primes less than or equal to 25.

 If p is a number whose multiples are being eliminated on the current pass, then the first
multiple we should consider is

p.p ≤ n => p cannot exceed �

�
p2 = n

p= �
� Sieve(n)
Algorithm:

for p  2 to [ 𝑛] do if
for p  2 to n do A[p]  p

A[p] ≠ 0
j  p*p
while j < n do
A[j]  0
j  j+p
i0
for p  2 to n do
if A[p] ≠ 0
 L[i]  A[p]
i  i+1
return L
So, we incorporate the sieve into middle school procedure to get a legitimate algorithm for computing
gcd.
1.1 Fundamentals of algorithm problem solving

Sequence of steps in designing & analyzing an algorithm.

Understanding the problem:

 Before designing an algorithm is to understand completely the problem.

 Read the problems description and ask questions or doubt about the problems.

 Solve the few small examples by hand.

 Think about special cases and ask questions again if needed.

 There are few types of problems that arise in computing applications quite often.

 There are known algorithms for solving it. It might help us to understand how such an
algorithm works and know its strengths and weakness.
 There will not be readily available algorithm for some problems and we have to design on our
own.

 An input to an algorithm specifies an instance of the problem the algorithm solves.

 It is very important to specify exactly the range of instances the algorithm needs to handle.

 If we fail to do, the algorithm may work for majority of inputs but crash on some
“boundary” value.
Ascertaining the capabilities of a computional device:

 The next step is to ascertain the capabilities of the computational device the algorithm is
intended for.

 The algorithms in use today are still designed to be programmed for a computer closely
resembles the current architecture.

 This architecture is captured by the so-called random-access machine (RAM).

 Its central assumption is that instructions are executed one after another, one
operation at a time.

 Accordingly, algorithms designed to be executed on such machines are called sequential

algorithms.

 The central assumptions does not hold for some newer computers that can execute operations
concurrently. i.e in parallel. This type of algorithms are called parallel algorithms.

 If it is a scientific problem, the algorithm can be designed independent of specification

parameters for a particular computer.

 If we were designing as a practical tool, the answer may depend on a problem you need to
solve.

 In many problems, we are not bothering about a computer speed and memory.

 There are important problems, which are very complex, have to process huge volumes of data,
in that case it is imperative to be aware of the speed and memory available on a particular
system.

Choosing between exact and approximate problems solving:

 The next principal decision is to choose between solving the problem exactly or solving it
approximately.
 In former case, it is solving the problem exactly called exact algorithm, latter approximation
algorithm.

 What is the need for approximation algorithm?

i) There are important problems that cannot be solved exactly such as

o Extracting square roots.
o Solving non-linear equations.
o Evaluation definite integrals.
 ii) The algorithms for solving a problem exactly can be unacceptably slow because of
the problems intrinsic complexity.
o Ex: Travelling salesman problem.

iii) Approximation algorithm can be part of a more sophisticated algorithm that solves a
problem exactly.

Deciding on approximate data structures:

 Some algorithms do not demand any ingenuity in representing their inputs.

 But others are, in fact predicated on ingenious data structures.

Algorithm design techniques:

 An algorithm design technique is a general approach to solving problems algorithmically that

is applicable to a variety of problems from different areas of computing.

 Need for learning algorithm design techniques:

i) They provide guidance for designing algorithms for new problems.

ii) Algorithm design techniques make it possible to classify algorithms according to an

underlying design idea, therefore, they can serve as natural way to both categorize and
study algorithm.

Methods of specifying an algorithm:

 There are two options that are most widely used nowadays for specifying algorithms:

i) Pseudo code

ii) Flowchart

 Pseudo code is a mixture of a natural language and programming language like

constructs.

 A pseudo code is usually more precise than a natural language and its usage of ten
yields more algorithm description.

 Flowchart a method of expressing algorithm by a collection of connected geometric shapes

containing descriptions of the algorithms steps.

 Pseudo code cannot be directly fed into a computer, instead it needs to be converted
into a computer program written in a particular computer language.

Proving an algorithms correctness:

 Once an algorithm has been specified, you have to prove its correctness.
 That it, you have to prove that the algorithm yields a required result for every
legitimate input in a finite amount of time.

For Ex: Euclid’s algorithm for computing the gcd

gcd(m,n) = gcd(n,m mod n)

 The simple observation that the second number gets smaller on every iteration of the
algorithm.

 Algorithm gets terminated when m mod n becomes 0.

 For some algorithms, a proof of correctness is quite easy; for others, it can be quite complex.

 A common technique for proving correctness is to use mathematical induction.

 In order to show that an algorithm is incorrect, we need just one instance of its input of which
the algorithm fails.

 If the algorithm is found to be incorrect, we need to redesign it.

Analyzing an algorithm:

 After correctness, the most important quality is efficiency.

 There are two kinds of algorithm efficiency

i) Time Efficiency.

ii) Space Efficiency.

- Time Efficiency indicates how fast the algorithm runs.

- Space Efficiency indicates how much extra memory the algorithm needs.

 Another desirable characteristic of an algorithm is simplicity.

 Simpler algorithms are easier to understand and easier to program. The resulting programs
usually contain fewer bugs.

 Another desirable characteristic of an algorithm is generality.

 There are two issues:

i) Generality of the problem the algorithm solves.

ii) The range of inputs it accepts.

First issue:
- Sometimes easier to design an algorithm for a problem posed in more general terms. Ex: gcd

- There are situations where designing a more general algorithm is unnecessary or

difficult or even impossible.

Ex: Quadratic equation cannot be generalized.

Second issue:

-Designing an algorithm that can handle a range of inputs that is natural for
the problem at hand.

 If we are not satisfied with the algorithms efficiency, simplicity, generality, we must redesign
the algorithm.

 In fact, even evaluation is positive, it is still worth searching for other

algorithms solutions.
Coding an algorithm:
 Most algorithms are destined to be ultimately implemented as computer programs.

 Correctness of the computer program is proven with mathematical rigor.

 As a practical matter, the validity of programs is still established by testing.

 Test and debug the program thoroughly whenever we implement an algorithm.

 Implementing an algorithm correctly is necessary but not sufficient. We should have efficient
algorithm.

 Better algorithm can make a difference in running time by orders of magnitude.

 Empirical analysis of the algorithm is based on timing the program on several inputs and
then analyzing the results obtained.

 An important issue of algorithms problem solving is whether or not every problem can
be solved by an algorithm.

 Ex: Finding real root of a quadratic equation with a negative discriminant there is no solution.

1.2 Important problem types

 There are many problems one may encounters in computing, but there are few areas that have
attracted particular attention from researchers.

 The most important problem types

 o i) Sorting

o ii) Searching

o iii) String processing

o iv) Graph problems

o v) Combinational problems

o vi) Geometric problems

o vii) Numerical problems

Sorting:

 The sorting problem is rearranging the items of a given list in ascending order.

 Usually need to sort the lists of numbers, characters from a alphabet, character strings and
most important, records similar to those maintained by schools, companies about their
employees.

 In case of records, we need to choose a piece of information to guide sorting.

Ex: Student records can be maintained in alphabetical order or by student number.

 Such specially chosen piece of information is called a key.

Need for a sorted list

 Sorting makes many questions about the list easier to answer.

 The most important is searching, that’s why dictionaries, telephone books and
so on are sorted.

 There are dozens of sorting algorithm.

 Although some algorithms are indeed better than others, there is no algorithms that would be
the best solution in all situations.

 Some of the algorithms are simple but relatively slow while others are faster but more complex,
some work better on randomly ordered pairs (inputs) while others do better on almost sorted
lists.

 Some are suitable only for lists residing in the fast memory while others can be
adapted for sorting large files stored on a disk and so on.

 Two properties of sorting algorithm

i) Stable

ii) In place
- Stable algorithm means it preserves the relative order of any two equal elements in its
input.

- If an input contain two equal elements in positions i & j, where i < j, then in the sorted list
they have to be in positions i1 and j1 respectively such that i1 < j1

Ex: Students list sorted alphabetically and we want to sort it according to GPA means

- Stable algorithm will yield a list in which students with the same GPA will still be sorted
alphabetically.

- An algorithm is said to be in place if it does not require extra memory, except, possibly, for a
few memory units.
Searching:

 The searching problem deals with finding a given value called a search key, in a given set.

 There are many searching algorithms range from the straight forward sequential search to a
limited binary search algorithms.

 There is no single algorithm that fits all situations best.

 Some algorithms work faster than other but require more memory. Some are very fast
but applicable only to sorted arrays and so on.

 There is no stability problem.

String processing:

 A string is a sequence of characters from an alphabet.

 Strings which comprise letters, numbers and special characters, bit strings which comprise
zeros and ones.

 String processing algorithms are important in computer science for a long time in
conjunction with computer languages and compiling issues.

 Searching for a given word in a text is called string matching.

Graph problems:

 Graph can be thought of as a collection of points called vertices, some of which are
connected by line segments call edges.

 Graphs can be used for modeling a wide variety of real-life applications including
transportation & communication networks, project scheduling.

 Basic graph algorithm includes

 i) Graph traversal algorithms

ii) Shortest path algorithms

iii) Topological sorting

 Some graph problems are computationally every hard that is only very small instances of such
problems can be solved in a realistic amount of time even with the fastest computers
imaginable.

Ex: Travelling sales man problem graph coloring problem.

Combinatorial problems:

 The above two said problems are examples of combinational problems.

 For these problems, need to find a combinatorial object – such as permutation, a combination,
or a subset – that satisfies certain constraints and has some desired property.

 Combinational problems are the most difficult problems in computing.

 There are two reasons for that:

o i) Number of combinational objects typically grows extremely fast with a

problem’s size.

o ii) There are no known algorithms for solving most such problems exactly in an
acceptable amount of time.

Geometric problems:

 Geometric problems deal with geometric objects such as points, lines and polygons.

 Algorithms are developed for constructing simple geometric shapes – triangles, circles and so
on.
 There are algorithms for two classic problems

o i) Closest pair problem

o ii) Convex hull problem

o Closest pair problem  Given n points in the plane, find the closest pair among them.

o Convex hull problem  Find the smallest convex polygon that would include all points
of a given set.

Numerical problems:

 Another special area of applications that involve mathematical objects of continuous nature:
Solving equations, systems of equations, computing definite integrals, evaluating functions and
 so on.

 These problems can be solved approximately.

 There are many algorithms that play a critical role in many scientific and engineering
applications.

1.3 Fundamentals of the analysis of algorithm efficiency

 The term “ analysis of algorithms” mean an investigation of an algorithms efficiency with

respect to two resources: running time and memory space.

1.3.1 - Analysis framework:

 Outlining a general framework for analyzing the efficiency of algorithms.

 There are two kinds of efficiency:

i) Time efficiency indicates how fast an algorithm runs.

ii) Space efficiency deals with extra space the algorithm requires.

1.3.2 Measuring an input’s size:

 The most obvious observation that almost all algorithms run longer on larger inputs.

Ex: It takes longer to sort larger arrays, multiply larger matrices and so on.

 Therefore, it is logical to investigate an algorithm’s efficiency as a function of some

parameter n indicating the algorithm’s input size.

 In some cases, selecting such a parameter is quite straight forward.

Ex: Sorting, searching  where the size of the list will be taken.

 For the problem of evaluating a polynomial

p(x) = an xn + …… + a0 of degree n.

Parameters may be

i) Polynomial’s degree

ii) Number of its co-efficient.

In some situation, where the input size matters.

 Ex: Computing the product of two n x n matrices.

There are two natural measures:

i) Matrix order n

ii) Total number of element N.

 So, the algorithm efficiency will be qualitatively different depending on which of the two
measures we use.

 The choice of an appropriate size metric can be influenced by operations of the

algorithm in question.

Ex: Measuring input’s size for spell checking algorithm.

i) If the algorithm examines individual characters of its input, then measure the size by
the number of characters.

ii) If it works by processing words, should count their number in the input.

 Should make a special note about measuring size of inputs for algorithms involving
properties of numbers.

Ex: Checking for prime.

Computer scientists prefer measuring size by the number of bits b in the n’s binary
representation.

b = [log2 n]+1

This gives the efficiency of algorithms.

1.3.3 - Units for measuring running time:

 Concerns about units of measuring an algorithm’s running time.

 We can simply use some standard unit of time measurement – a second, a millisecond and so
on to measure the running time of a program implementing the algorithm.

Drawbacks:

i) Dependence on the speed of a particular computer.

ii) Dependence on the quality of a program implementing the algorithm.

iii) The compiler used in generating the machine code.

iv) The difficulty of clocking the actual running time of the program.

 We should have a metric that does not depend on these extraneous factors.

 One possible approach is to identify the most important operation of the algorithm, called the
 basic operation. The operation contributing the most for the total running time and compute the
number of times the basic operation is executed.

 It is not difficult to identify the basic operation of an algorithm.

Ex-1: Sorting algorithms work by comparing elements of a list being sorted with each other. The
basic operation is a key comparison.

Ex-2 : Matrix multiplication and polynomial evaluation which requires two arithmetic operations:
Multiplication and addition.

 Thus, the established frame work for the analysis of an algorithm’s time efficiency suggests
measuring it by counting the number of time the algorithm’s basic operation is executed in
inputs of size n.

 The important application

Cop – The time of execution of an algorithm’s basic operation on a particular computer. C(n) – The

number of times this operation needs to be executed for this algorithm.

Estimating the running time T(n) of a program implementing this algorithm on that computer is

T(n) ≈ Cop C(n)

 In this formula, the count C(n) does not contain any information about operations that
are not basic. So, the count is computed approximately.

 Cop is also approximation.

 Unless n is extremely small or large, the formula can give a reasonable estimate of the
algorithm’s running time.

Assuming that C(n) = ½ n (n-1)

How much longer will the algorithm run if double its input size?

C (n) = ½ (n) (n-1) = ½ n2 – ½ n ≈ ½ n2 and

therefore

The answer is 4 times longer

o Cop value is not knowing. The value was neatly cancelled out in the ratio.

o ½ the multiplicative constant in the formula for the count C(n), was also cancelled out.

Therefore, the efficiency analysis framework ignores multiplicative constants and concentrates on the
count’s order of growth to within a constant multiple for larger-size inputs.
1.3.4- Orders of growth:

 A difference in running times on small inputs is not what really distinguishes efficient
algorithms from inefficient ones.

Ex: Computing gcd of two small numbers, the efficiency is not clear when Euclid’s
algorithm is compared with other two algorithms.

It is only clear when we find gcd of two large numbers.

n log2 n n n log2n n2 n3 2n n!

10 3.3 10
1
3.3.101 102 103 103 3.6.106

102 6.6 10
2 6.6.102 104 106 1.3.10 9.3.10157
30

103 10 10
3 1.0.104 106 109

Table 1 Values (some approximate) of several functions important for analysis of algorithms

 The magnitude of the numbers in Table 1 has a profound significance for the analysis of
algorithm.

 The function growing the slowest among these is the logarithmic function.

 On the other hand of the spectrum are the exponential function 2 n and n!. Both these functions
grow so fast that their values become astronomically large even for rather small values of n.

 Another way to appreciate the qualitative difference among the orders of growth of
the functions in table is if we increase two fold in the value of their argument n.

The function

log2 n => log2 2n = log2 2 + log2 n = 1+log2 n => increases in value by just 1. n log2 n

=> 2n log2 2n = 2n [1+log2 n] => increases slightly more than 2 fold. n2 => (2n)2 =

4n2 => 4 fold

n3 => (2n)3 = 8n3 => 8 fold

n! = increases much more than the above

1.3.5- Worst-case, Best-case and Average-case Efficiencies:

 To measure an algorithm’s efficiency as a function of a parameter indicating the size of

 the algorithm’s input.

 But there are many algorithms for which running time depends not only on an input size but
also on the specific of a particular input.

Ex: Sequential search.

This is a straight forward algorithm that searches for a given item in a list of n elements by
checking successive elements of the list until either a match with the search key is found or the
list is exhausted.

Algorithm:

 Sequential search (A[0… n-1], K)

// Input: An array A[0…n-1] and a search key K.

// Output: Returns the index of the first element of A that matches K or -1 if there are no matching
elements.

i0

while i < n and A[i] ≠ K do i

 i+ 1

if i < n return i

else return -1

 The running time of this algorithm can be quite different for the same list size n depends where the
K appears.

 In the worst case, when there are no matching elements or the first matching element happens to be
the last one on the list, the algorithm makes the largest number of key comparisons among all
possible inputs of size n:

Cworst (n) = n

 The Worst Case Efficiency (WCE) of an algorithm is its efficiency for the worst-case input of
size n, which is an input of size n for which the algorithm runs the longest among all possible inputs
of that size.

 Computing worst-case efficiency is straight forward, analyze what kind of inputs yield the largest
value of the basic operation’s count C(n) among all possible inputs of size n and then compute the
worst-case value

Cworst (n)
 WCE provides important information about an algorithm’s efficiency by bounding its running
time.

It guarantees the running time will not exceed Cworst (n)

 The Best Case Efficiency (BCE) of an algorithm is its efficiency for the best- case input of size
(n), which is an input of size n for which the algorithm runs fastest among all possible inputs of
that size.

Analyze the BCE:

i) Determine the kind of inputs for which the count C(n) will be the smallest among all possible
inputs of size n.

ii) Ascertain the value of C(n) on the most convenient inputs.

Ex: Sequential search

The first element is equal to a search key.

Cbest (n) = 1

Average Case Efficiency provide the necessary information about an algorithm’s

behavior on a “typical” or “random” input.

 We must take some assumptions about possible inputs of size n.

 The standard assumptions are

i) The probability of a successful search is equal to p(0 ≤ p ≤1)

ii) The probability of the first match occurring in the ith position of the list is the same for every i.

 Find the average number of key comparisons. Cavg (n)

as follows

Successful Search

- The probability of the first match occurring in the ith position of the list in p/n for every i.

- The number of comparisons in such a situation is obiviously i.

Unsuccessful search

- The number of comparison is n

- Probability is (1-p)

if p = 1

The average number of key comparisons = (n+1)/2 + n(1-1)

= (n+1)/2

(i.e) The algorithm will inspect, on average, about half of the list’s elements. if p = 0

The average number of key comparisons = 0(n+1)/2 + n(1-0)= n Therefore, the algorithm will
inspect all n elements on all such inputs.

 Investigation of the ACE is considerably more difficult than investigation of the worst- case &
best-case efficiencies.

 The direct approach for doing it involves dividing all instances of size n into several classes so
that for each instance of the class the number of times the algorithm’s basic operation is
executed is the same.

 The probability distribution of inputs need to be obtained or assumed so that the expected value
of the basic operation’s count then be derived.

1.3.6- Amortized Efficiency:

 It applies not to a single run of an algorithm but rather to a sequence of operations performed
on the same data structure.
 It turns out that in some situations a single operation can be expensive, but the total time for an
entire sequence of n such operations is always significantly better than the worst-case efficiency
of that single operation multiplied by n.
1.4 - Asymptotic Notations And Basic Efficiency Classes

 To compare orders of growth, there are three notations

i) Ο (Big Oh)

ii) Ω (Big Omega)

iii) θ (Big Theta)

t(n) and g(n) can be any non-negative functions defined on the set of natural numbers. t(n) =

algorithms running time.

g(n) = simple function to compare the count.

1.4.1 - Introduction

Ω(g(n)) stands for the set of all function with a larger or same order of growth as
g(n).

n3Ω(n2), ½ n(n-1)  Ω(n2), 100 +

5  Ω(n2)θ(g(n)) is a set of all

functions that have the same growth

as g(n). an2 + bn + c with a>0

θ(n2)

1.4.2 Asymptotic Notations and its Properties

Informal Introduction
O(g(n)) is the set of all functions with a smaller or same order of growth as g(n).
Examples:

n ε O(n2), 100n+5 ε O(n2), ½n(n-1) ε O(n2),

The first two function are linear and have a smaller order of growth than g(n)=n2.

But the last one quadratic and has the same order of growth as n2.
The second notation Ω(g(n)), set of all functions with a larger or same order of
growth as g(n).
Examples:
n3 ε Ω(n2), ½n(n-1) ε Ω(n2)
Finally Θ(g(n)), set of all functions have the same order of growth as g(n). Thus

every quadratic equation, an2+bn+c with a>0 is in Θ(n2).

1.4.3 O-notation

A function t(n) is said to be in O(g(n)), denoted t(n) ε O(g(n)), if t(n) is bounded above by some
constant multiple of g(n) for all large n, i.e., if there exist some positive constant c and non-
negative integer n0 such that t(n)≤cg(n) for all n≥n0

Example:
100n+5 ε O(n2)
100n+5≤100n+n(for all n≥5)=101n ≤ 101n2
As the values of constants c and n0 required by the definition, we take 101 and 5
respectively.
100n+5 ≤ 100n+5n (for all n ≥ 1) = 105n with c=105 and n0=1 1.4.4 Ω-

notation

A function t(n) is said to be in Ω(g(n)), denoted t(n) ε Ω(g(n)), if t(n) is bounded below
by some positive constant multiple of g(n) for all large n, i.e., if there exist some positive constant c
and some nonnegative integer n0 such that t(n)≥cg(n) for all n≥n0.
Example:
n3 ε Ω(n2)
n3≥n2 for all n≥0
i.e., we can select c=1 and n0=0.

1.4.5 Θ-notation

A function t(n) is said to be in Θ(g(n)), denoted t(n) ) ε Θ(g(n)), if t(n) is bounded both above
and below by some positive constant multiples of g(n) for all large n, i.e., if there exist some
positive constant c1and c2 and some nonnegative integer n0 such that c2g(n)≤t(n)≤c1g(n) for all
n≥n0

Prove ε
θ(n2) = ≤ for all n0≥1
Left inequality,

= ≥ = for all n ≥ 2
Right inequality,
Hence we select c2 = ¼ , c1 = ½ and n0 = 2

1.4.6 Properties of Asymptotic Notations

Theorem: If t1(n) ε O(g1(n)) and t2 ε O(g2(n)) then t1(n)+t2(n) ε

O(max{g1(n),g2(n)})

Proof: simple fact about four arbitrary real numbers a1, a2, b1, b2: if a1 ≤ b1 and a2 ≤ b2, then
a1+a2 ≤ 2 max{b1,b2}.
Since if t1(n) ε O(g1(n)), there exist some positive constant c1 and some nonnegative
integer n1 such that
t1(n) ≤ c1g1(n) for all n≥n1
Since if t2(n) ε O(g2(n)), t2(n) ≤ c2g2(n) for all n≥n2
Let us denote c3=max{c1,c2} and consider n≥max{n1,n2} we can use both inequalities. T1(n)
+t2(n)≤c1g1(n)+c2g2(n)
≤c3g1(n)+c3g2(n)=c3[g1(n)+g2(n)]
≤c32max{g1(n),g2(n)} So
t1(n)+t2(n) ε O(max{g1(n),g2(n)})
It implies that the algorithm’s overall efficiency is determined by the part with a larger
order of growth.

1.4.7Comparing Order’s of Growth using Limits.

 Though definitions of Ο, Ω and θ are indispensable for proving their abstract properties,
they are rarely used for comparisons of orders of growth of two specific functions.
 A much more convenient method for doing so is based on computing the limit of the
ratio of two functions.

Three principal cases may arise

0 implies that t(n) has a smaller order of growth than g(n) c > 0
implies that t(n) has a same order of growth as g(n)
=
∞ implies that t(n) has a larger order of growth than g(n)
First two cases mean that t(n) 
Ο(g(n)) Last two cases mean that t(n) 
Ω(g(n)) Second case mean that t(n) 
θ(g(n))
1.4.8 Basic Efficiency Classes

 The time efficiencies of a large number of algorithms fall into only a few classes.
 These classes are listed in increasing order of their growth along with their names and
a few comments.
1.5 Empirical Analysis

An alternative method to mathematical analysis is empirical analysis. The general plan for analysis of
algorithm time efficiency in Empirical analysis is given below.

1. Understand the experiment’s purpose.

2. Decide on the efficiency metric M to be measured and the measurement unit. ie.,
operation count and time unit.
3. Decide on characteristics of the input sample. ie., range and size.
4. Prepare a program implementing the algorithm for the experimentation.
5. Generate a sample of inputs
6. Run the algorithm on sample’s inputs and record the data observed.
7. Analyze the data obtained.
Different goals of analyzing algorithms empirically

1. Checking the accuracy of theoretical assertion about the algorithm’s efficiency.

2. Comparing efficiency of several algorithms for solving same problem or different

implementations of the same algorithm.
3. Developing hypothesis about the algorithm’s efficiency class.

4. Ascertaining the efficiency of the program implementing the algorithm on a particular

machine.
How the algorithm’s efficiency is to be measured?

1. Insert a counter in the program to count the number of times the algorithm’s basic operation
is executed.
2. Use UNIX system command time
3. Measure the running time of a code fragment before starting, tstart and tfinish and then compute
the difference (tstart and tfinish).

Important facts

1. System time is not very accurate.

2. Using high speed modern computers may fail to register the running time.
3. On a computer running under a time-sharing system, the reported running time may include
the time spent by the CPU on other programs.

The physical running time provides very specific information about an algorithm’s performance in a
particular computing environment and measuring time spent on different segments. Getting such data
called as profiling is an important resource in the empirical analysis of an algorithm’s running time.

Use a sample representing a typical input which is to be developed by the experimenter. Several sizes
of same sizes can be included. The instances can be generated randomly.

The empirical data obtained as the result of an experiment need to be recorded and then presented for
an analysis. Data can be presented numerically in a table or graphically in a scatterplot. One of the
possible applications of the empirical analysis is to predict the algorithm’s performance on an
instance not included in the sample.

1.6. Mathematical Analysis of Non-Recursive & Recursive Algorithms

1.6.1 Mathematical Analysis of Non-Recursive Algorithms

• Analyzing the efficiency of non-recursive algorithms.

 Before proceeding with examples, we review about list of summation formulas and two
basic rules of sum manipulation.

Problem: Finding the value of the largest element in a list of n numbers.

Assumption: List is implemented as array.
Algorithm MaxElement(A[0..n-1])
//To find the largest element
//Input: Array of n elements
//Output: Largest element
maxval  A[0]
for i  1 to n-1 do if
A[i] > maxval
maxval  A[i]
return maxval

Applying general framework outlined.

i) Measuring an input’s size:

- The input size is the number of elements in the array i.e.., n

i) Units for measuring running time:

- The operations most often executed are in the for loop.

- There are two operations

o Comparison A[i] > maxval

o Assignment maxval  A[i]

Which have to consider?

- Comparison is executed on each repetition of the loop

- Assignment is no. So, the basic operation is comparison

 The number of comparisons will be the same for all arrays of size n. There is no need to
distinguish among the worst, average and best cases.

 Count the number of times the basic operation performed and try to formulate as
function of size n.

 The algorithm makes one comparison on each execution of the loop, which is repeated for
each value of the loop’s variable I within the bounds between 1 and n-1.

𝑛−1
𝑛−1
𝐶𝑛 = ෍ 1 => ෍ = 𝑛 − 1 − 1 + 1 = 𝑛 − 1 𝜖 (𝑛)
𝑖=1 𝑖=1
General plan for analyzing efficiency of non-recursive algorithms:

i) Decide on a parameter indicating an input’s size.

ii) Identify the algorithm’s basic operation.

iii) Check whether the number of times the basic operation is executed depends only on the size
of an input.

iv) Set up a sum expressing the number of times the algorithm’s basic operation is
executed.

v) Establish its order of growth.

Example 2: Element uniqueness problem

Algorithm UniqueElements (A[0..n-1])

//To check the uniqueness of an array
//Input: An array of n elements
//Output: Returns true if the array does not have duplicate elements. Otherwise false for i  0
to n-2 do
for j  i+1 to n-1 do
if A[i] = A[j] return false
return true

i) Measuring input’s size:

- The number of elements in the array- n elements

i) Measuring running time:

- The innermost loops contains a single operation.

- This is the basic operation

- The number of elements comparison will depend not only on n but also on whether there are
equal elements in the array and if there are, which array positions they occupy.
Example 3: Given two nxn matrices. Find the Product of two matrices

ALGORITHM MatrixMul(A[0..n-1,0..n-1],B[0..n-1,0..n-1])
// To find the product of two matrices of order nxn
//Input: A and B are two matrices of nxn order
//Output: Resultant matrix C
for i0 to n-1 do
for j0 to n-1 do
C[i,j]=0
for k0 to n-1 do
C[i,j]=c[i,j]+A[i,k]*B[k,j]
return C
Algorithm analysis

 An input’s size is matrix order n.

 There are two arithmetical operations (multiplication and addition) in the innermost loop. But
we consider multiplication as the basic operation.
 Let us set up a sum for the total number of multiplications M(n) executed by the algorithm.
Since this count depends only on the size of the input matrices, we do not have to investigate
the worst-case, average-case, and best-case efficiencies separately.
 There is just one multiplication executed on each repetition of the algorithm’s innermost loop,
which is governed by the variable k ranging from the lower bound 0 to the upper bound n − 1.
 Therefore, the number of multiplications made for every pair of specific values of variables i
and j is

The total number of multiplications M(n) is expressed by the following triple sum:

The sum is computed by using the formula (S1) and Rule (R1)
Example 4: Finds the number of digits in the binary representation of a positive decimal integer

ALGORITHM Binary(n)
//To find the count of digits in a binary representation of a number
//Input: The number n
//Output: number of digits
count1
while n>1 do
countcount+1
n n/2

Algorithm analysis

 An input’s size is n.
 The loop variable takes on only a few values between its lower and upper limits.
 Since the value of n is about halved on each repetition of the loop, the answer should be about
log2 n.
 The exact formula for the number of times.
 The comparison 𝑛 > 1 will be executed is actually log2 n + 1.
- To determine a solution uniquely for recurrence relation, we need an initial condition that
tells us the value with which the sequence starts.

- We can obtain this value by inspecting the condition that makes the algorithm stop its
recursive calls.

if n = 0 return 1

M(0) = 0

The call stop when n = 0

- Thus, we succeed in setting up the recurrence relation and initial condition for the
algorithm’s number of multiplication M(n):

M(n) = M(n-1) + 1 for n > 0

M(0) = 0

- There are two recursively defined functions

o Factorial function F(n)

o Number of multiplications M(n)

- For solving recurrence relations, we use the method of backward substitutions M(n) =

M(n-1) + 1 substitute M(n-1) = M(n-2) + 1

=[M(n-2) + 1] + 1 = M(n-2) + 2 substitute M(n-2) = M(n-3) + 1

= [M(n-3) + 1] + 2 = M(n-3) + 3

Therefore, general formula = M(n) = M(n-i) + i

M(n) = M(n-1) + 1 . . . . = M(n-i) + i = . . . . .

Initial condition n=0 so substitute i=n

M(n-n) + n = n

General plan for analyzing efficiency of recursive algorithms:

1) Decide on a parameter.

2) Identify the algorithm's basic operation.

3) Measuring running time.

4) Setup a recurrence relation, with an appropriate initial condition for the number of times
the basic operation is executed.
5) Solve the recurrence.

Example 2: Towers of Hanoi problem

1. First move recursively (n-1) disks from peg 1 to peg2, peg 3 as auxiliary

2. Move the largest disk directly from peg 1 to peg3.

3. Move recursively (n-1) disks from peg 2 to peg3, peg 1 as auxiliary

i) Measuring input size:

- The number of disks n is the input size indicator.

i) Measuring running time:

- The algorithm’s basic operation is moving one disk.

- Clearly, the number of moves M(n) depends on n only.

- So, the following recurrence equation for it

M(n) = M(n-1) + 1 + M(n-1) for n > 1

= M(n-1) + 1 + M(n-1)

- Obvious initial condition M(1) = 1

M(n) = 2M(n-1) + 1 for n > 1

M(1) = 1

Solving recurrence relation equation by backward substitutions

M(n) = 2M(n-1) + 1 sub M(n-1) = 2M(n-2) + 1

= 2 [2M(n-2) + 1) + 1

= 22 M(n-2) + 2 + 1 sub M(n-2) = 2M(n-3) + 1

= 22 [2M (n-3) + 1] + 2 + 1

= 23 M(n-3) + 22 + 2 + 1

Therefore, after I substitutions, we get

M(n) = 2i M(n-i) + 2i-1 + 2i-2 + ……. + 2 + 1

= 2i M(n-i) + 2i -1
 M(n) = 2n-1 M(n-(n-1)) + 2n-1 -1 n-i= 1, i= n-1

= 2n-1 M(n-n+1) + 2n-1 -1

= 2n-1 M(1) + 2n-1 -1

= 2n-1 + 2n-1 -1

= 2 . 2n-1 -1 = 2n -1

Thus we have an exponential algorithm, which will run for an unimaginably long time even for
moderate values of n. This is not due to the fact that this algorithm is poor.

It is the problem’s intrinsic difficulty that makes it so computationally difficult.

Example 3: Counting the number of binary digits

Algorithm: Bin Rec(n)

if n = 1 return 1

else return Bin Rec([n/2]) + 1

- The number of additions made in computing Bin Rec([n/2]) is A([n/2]), plus one more addition
is made by the algorithm to increase the returned value by 1.

A[n] = A([n/2]) + 1 for n > 1

- if n = 1, the recursive calls end, there are no additions made, the initial condition is A(1) = 0

- The presence of [n/2] in the function’s argument makes the method of backward
substitutions stumble on values of n that are not powers of 2.

- Therefore the standard approach to solving such a recurrence is to solve it only for n

= 2k and then

o Apply the theorem called the smoothness rule.

o This theorem gives correct answer about the order of growth for all values of n.

- For n = 2k takes the form

 A(2k) = A(2k-1) + 1 for k > 0

A(20) = 0

Now backward substitutions encounter no problems:

A(2k) = A(2k-1) + 1 substitute A(2k-1) = A(2k-2)+1

= [A(2k-2) + 1] + 1

= A(2k-2) + 2 substitute A(2k-2) = A(2k-3) + 1

=[A(2k-3) + 1] + 2

= A[2k-3] + 3

= A[2k-i] + i

= A[2k-k] + k
Thus we end up with

A(2k) = A(1) + K

=0+k=k

After returning to the original variable n = 2k

Taking log on both sides.

Hence, log n = k log 2

K = log2 n

A(n)= log2 n  θ (log n)

Example 4 – Fibonacci Numbers

Fibonacci Series

0, 1, 1, 2, 3, 5 …..

• We can define by simple recurrence F(n)

= F(n-1) + F(n-2) for n > 1

And two initial conditions

 F(0) = 0, F(1) = 1

Explicit formula for the nth Fibonacci number

• Backward substitutions will fail to get an easily discernible pattern.

• So, we take a theorem that describes solutions to a homogeneous second-order

linear recurrence with constant co-efficients.

ax(n) + bx(n-1) + cx(n-2) = 0

a, b, c are coefficients of recurrence.

x(n)  unknown sequence to be found

1.7 Algorithm Visualization

Another approach to study algorithm is algorithm visualization and can be defined as the use of
images to convey some useful information about algorithms. That information can be a visual
illustration of an algorithm’s operation, performance on different kinds of inputs, its execution speed
for various algorithms for the same problem. Algorithm visualization uses graph – line segments or
bars to represent algorithm’s operation.
There are two principal variations:
1. Static algorithm visualization
2. Dynamic algorithm visualization, also called algorithm animation.

Static algorithm visualization shows an algorithm’s progress through a series of still images.
Algorithm animation, shows a continuous presentation of an algorithm’s operations.

In 1981, the appearance of the algorithm visualization classic, a 30 minute color sound film titled
sorting out sorting. Sorting-out-sorting algorithm can be visualized via vertical or horizontal bars or
sticks of different heights or lengths, which are rearranged according to their sizes. This will be
convenient for smaller amount of data.

Two principal applications of AV

i) Research: For researchers based on expectations that algorithm visualization may help uncover
some unknown features of algorithms

ii) Education: Application of education seeks to help students learning algorithms.

Features of an algorithm’s animation:
1. Be consistent
2. Be interactive
3. Be clear and concise
4. Be forgiving to the user
5. Adapt to the knowledge level of the user
6. Emphasize the visual component
7. Keep the user interested.
8. Incorporate both symbolic and iconic representations.
9. Include algorithm’s analysis and comparisons with other algorithms for the same
problem.
10. Include execution history.

Design an algorithm to find all the common elements in two sorted lists of numbers. For example, for
the lists 2, 5, 5, 5 and 2, 2, 3, 5, 5, 7, the output should be 2, 5, 5.What is the maximum number of
comparisons your algorithm makes if the lengths of the two given lists are m and n, respectively?

For each ai, the algorithm iterates over the entire b to check if ai ℇ b. So, it has
an O(mn) time complexity, which implies O(n2) if m and n are comparable.
Describe the standard algorithm for finding the binary representation of a positive decimal
integer a. in English. b. in pseudocode.

1. Divide the given number n by

2. The remainder rn (0 or 1) will be the next (from right to left) digit of the binary
representation
3. Replace n by the quotient of the last division
4. Repeat this operation until n becomes 0.