DSC13: Design and Analysis of Algorithm Page 1

Unit – 1: Introduction to Algorithm

Introduction, Fundamentals of the Analysis of Algorithm Efficiency

1. Notion or What is an Algorithm

2. Fundamentals of Algorithmic Problem Solving

3. Fundamentals of the Analysis of Algorithm Efficiency

- Time Efficiency and Space Efficiency

- Important Problem Types

- Fundamental data Structures

4. Analysis Framework,

- Measuring the input size

- Units for measuring Running time

- Orders of Growth

- Worst - case, Best - case and Average - case efficiencies

Prof. Nandan G P, BCA,SRS FGC - Chitradurga

 DSC13: Design and Analysis of Algorithm Page 2
Unit – 1: Introduction to Algorithm

1.1 Notion of Algorithm.

Q1. Define an algorithm. Explain the properties of an algorithm.

Algorithm: ―An algorithm is a sequence of unambiguous instructions for solving a problem.‖

Input

Problem Algorithm “Computer” Output

Program

Figure 1.1: Algorithm

Criteria/properties of Algorithm:
i) Input: Range of input should be clearly specified.
ii) Output: Algorithms should produce correct output.
iii) Definiteness: Each step must be clear and unambiguous.
iv) Finiteness: Algorithm must terminate after finite no. of steps.
v) Effectiveness: Each step must be solvable.

Q2. Explain the Different methods to find GCD of two numbers with example.

GCD: Greatest Common Divisor

“Largest integer [Common Divisor] that divide both numbers evenly; with remainder zero”
Different methods to find GCD of two numbers:
i) Euclid‘s algorithm: Uses Modulus method.
ii) Consecutive integer checking.
iii) Middle school procedure
iv) Repetitive subtraction method.

i) Euclid’s algorithm: Uses Modulus method

- Euclid is from Alexandria, Egypt.
- Algorithm involves successively dividing and calculating remainders as given below

gcd (m, n) = gcd (n, m mod n)

gcd (m, 0) = m

Example: Find gcd(60, 24)

gcd (60, 24) = gcd (24, 60 % 24) = gcd (24, 12)
gcd (24, 12) = gcd (12, 24 % 12) = gcd (12, 0)
gcd (12, 0) = 12

Find the GCD of i) 34 and 78 ii) 14 and 28 using Euclid method

Prof. Nandan G P, BCA,SRS FGC - Chitradurga

 DSC13: Design and Analysis of Algorithm Page 3
Unit – 1: Introduction to Algorithm

Algorithm:

Algorithm: gcdEuclid (m, n) //using Euclid’s algorithm

//Input: Two non-negative and non-zero integers m & n
//Output: GCD of m & n

Step 1: If n=0, return the value of m as the answer and stop; otherwise, proceed to Step 2.
Step 2: Divide m by n and assign the value of the remainder to r.
Step 3: Assign the value of n to m and the value of r to n. Go to Step 1.

Psedocode:

Algorithm: gcdEuclid (m, n) //using Euclid’s algorithm

//Input: Two non-negative and non-zero integers m & n
//Output: GCD of m & n
begin
while n ≠ 0
r = m mod n
m=n
n=r
end while
return m

ii) Consecutive Integer Checking

- This method is simply based on the definition of the greatest common divisor of ‗m‘
and ‗n‘ as the largest integer that divides both numbers evenly.
- Obviously, such a common divisor cannot be greater than the smaller of these
numbers, which we will denote by t = min{m, n}.
- So we can start by checking whether ‘t’ divides both ‘m’ and ‘n’: if it does, ‘t’ is the
answer; if it does not, we simply decrease ‘t’ by 1 and try again.

Algorithm: gcd (m, n) //Consecutive Integer checking

Step1 : Perform t = min { m, n }

Step2 : Perform t1 = m % t
Step3 : if t1 = 0 then Perform t2 = n % t otherwise goto step-5
Step4 : if t2 = 0 then return GCD = t and stop otherwise goto next-step
Step5 : Decrement t by 1 and Repeat from step-2

Prof. Nandan G P, BCA,SRS FGC - Chitradurga

 DSC13: Design and Analysis of Algorithm Page 4
Unit – 1: Introduction to Algorithm

 Example: Find the GCD of 6 and 10

In example: m = 6, n = 10 and
t = min {6, 10} = 6

t t1 = m % t t2 = n % t Comments
6 t1 = 6 % 6 = 0 t2 = 10 % 6 = 4 t1 =0 but t2 ≠ 0 hence t = t – 1
5 t1 = 6 % 5 = 1 t1 ≠ 0 hence t = t – 1
4 t1 = 6 % 4 = 2 t1 ≠ 0 hence t = t – 1
3 t1 = 6 % 3 = 0 t2 = 10 % 3 = 1 t1 =0 but t2 ≠ 0 hence t = t – 1
t1 = 0 and t2 = 0 ; hence
2 t1 = 6 % 2 = 0 t2 = 10 % 2 = 0 GCD = t
=2

iii) Middle School method (Prime Factorization method)

Example:
 Find the GCD of the numbers 60 and 24
Factors of 60 = 2.2.3.5
Factors of 24 = 2.2.2.3
gcd (60, 24) = Common factors = 2.2.3 = 12

Algorithm:
Algorithm : gcd (m, n) //using Middle school method
Step1: Find prime factors of m
Step2: Find prime factors of n
Step3: Identify all Common factors of step-1 and step-2.
Step4: Compute the product of all common factors and return it as GCD.

- The above algorithm is not legitimate. Reason: The algorithm has Ambiguous steps like
how to obtain prime factors or prime numbers is not explained. Also how would you find
common elements in two sorted lists?
- Algorithm for generating consecutive primes not exceeding any given integer n>1 is
invented in ancient Greece and is called as sieve of Eratosthenes.

Prof. Nandan G P, BCA,SRS FGC - Chitradurga

 DSC13: Design and Analysis of Algorithm Page 5
Unit – 1: Introduction to Algorithm

 Sieve of Eratosthene
- Algorithm to generate consecutive prime numbers not exceeding given integer ‗n>1‘ is
developed in Greece and is called as ―Sieve of Eratosthene.‖. The algorithm starts by
initializing list of prime candidates with consecutive integers from 2 to n.
- In the first iteration: Eliminate all multiples of 2, but not 2 itself. (i.e. 4, 6, .... )
- In the second iteration: Take next non-eliminated number-3. Eliminate all multiples of 3,
but not 3 itself. (i.e. 9, 15.... )
- In the Third iteration: Take next non-eliminated number-5. Eliminate all multiples of 5,
but not 3 itself. (i.e. 25, 35.... ). So on...
Example: Generate the prime numbers for n=25

2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25
2 3 5 7 9 11 13 15 17 19 21 23 25
2 3 5 7 11 13 17 19 23 25
2 3 5 7 11 13 17 19 23

Algorithm : Sieve(n)
//Input: positive integer n ≥ 2; Output: Array L with list of prime numbers of n
Begin
for p = 2 to n do
A[p] = p
end-for
for p = 2 to [ n ] do
if A[p] ≠ 0 then
j = p*p
while j ≤ n do
A[ j ] = 0
j= j+ p
end-for
end-if
end-for
i = 0
for p = 2 to n do
if A[p] ≠ 0 then
L[i] = A[p]
i= i+ 1
end-if
end-for
return L
end-Sieve

Prof. Nandan G P, BCA,SRS FGC - Chitradurga

 DSC13: Design and Analysis of Algorithm Page 6
Unit – 1: Introduction to Algorithm

iv) Repetitive Subtraction method

Example:
 Find the GCD of the numbers 10 and 6

if m > n; compute m = m – n If m = n then

.m .n
if n > m; compute n = n – m GCD = m
10 6 m > n ; m = 10 – 6 = 4
4 6 n>m; n=6–4=2
4 2 m > n; m = 4 – 2 = 2
2 2 m=n GCD = 2

Pseudo code:

Algorithm: gcd (m, n) //Repetitive subtraction method

Step1 : Input m and n; two positive integer numbers

Step2: if m ≠ n then goto next step otherwise goto step-5
Step3: if m > n then Perform m = m – n
Otherwise Perform n = n – m
Step4: Repeat from step -2.
Step5: return m as GCD

Pseudo code:
gcd (m, n) //Repetitive subtraction method
//Input: Two non-negative and non-zero integers m & n
//Output: Two non-negative and non-zero integers m & n
begin
while m ≠ n do
if ( m > n ) then
m= m – n
else
n= n – m
end-while
return m
end

Prof. Nandan G P, BCA,SRS FGC - Chitradurga

 DSC13: Design and Analysis of Algorithm Page 7
Unit – 1: Introduction to Algorithm

1.2 Fundamentals of Algorithmic Problem Solving

Q3. With the help of flowchart explain the various steps of algorithm design and analysis
process. (8 marks)

Understand the problem

Decide on:
- Computational means
- Exact Vs. Approximate technique
- Algorithm design technique

Design an algorithm

Prove correctness

Analyse the problem

Code the algorithm

Sequence of steps one typically goes through in designing and analyzing an algorithm are:
1. Understanding the Problem
 Study the problem completely. Read the problem‘s description carefully and ask
questions if you have any doubts about the problem, do a few examples by hand, think
about special cases. Think of known algorithm for solving it. If readily available
algorithm is not found; design your own.

2. Decide on Computational Means

 Algorithms designed considering computational means are classified as:
i) Sequential algorithms: Algorithms designed considering RAM, with assumption
that instructions are executed one after another, one operation at a time are called
sequential algorithm.
ii) Parallel algorithms: Algorithm which can be executed a piece at a time on many
different processing devices, and then combined together again at the end to get the
correct result are called parallel algorithm.

Prof. Nandan G P, BCA,SRS FGC - Chitradurga

 DSC13: Design and Analysis of Algorithm Page 8
Unit – 1: Introduction to Algorithm

3. Decide on Exact and Approximate Problem Solving

 Exact algorithms are the algorithms that can be solved exactly. Approximate algorithm
does not give exact output. Why should user go for approximate algorithm?
 Some algorithms simply cannot be solved exactly. Ex: extracting square roots, solving
nonlinear equations. Some algorithms for solving a problem exactly can be unacceptably
slow because of the problem‘s intrinsic complexity. Ex: Travelling salesman problem.

4. Decide on Algorithm Design Technique

 Algorithm Design Technique is a general approach to solving problems algorithmically
that is applicable to a variety of problems from different areas of computing.
 Learning these techniques is of utmost importance for the following reasons.
- they provide guidance for designing algorithms for new problems,
- make it possible to classify algorithms according to an underlying design idea;

5. Decide on Data Structures

 Data structures remain crucially important for both design and analysis of algorithms.
Algorithms + Data Structures = Programs

6. Methods of Specifying an Algorithm

 Algorithm can be specified in two ways.
Pseudocode is a mixture of a natural language and programming language-like
constructs. Pseudocode is usually more precise than natural language
Flowchart, a method of expressing an algorithm by a collection of connected geometric
shapes containing descriptions of the algorithm‘s steps.

7. Proving an Algorithm’s Correctness

 Once an algorithm has been specified, you have to prove its correctness. you have to
prove that the algorithm yields a required result for every legitimate input in a finite
amount of time.

8. Analyzing an Algorithm
 After correctness, the most important is efficiency.
 Two kinds of algorithm efficiency: time efficiency, indicating how fast the algorithm
runs, and space efficiency, indicating how much extra memory it uses.
 Other characteristics: Simplicity: Generality:

9. Coding an Algorithm
 Algorithms are ultimately implemented as computer programs.

Prof. Nandan G P, BCA,SRS FGC - Chitradurga

 DSC13: Design and Analysis of Algorithm Page 9
Unit – 1: Introduction to Algorithm

Q4. Explain algorithm correctness, efficiency of algorithm and generality of an algorithm.

(6 marks).

Proving an Algorithm’s Correctness

 Once an algorithm has been specified, you have to prove its correctness. you have to
prove that the algorithm yields a required result for every legitimate input in a finite
amount of time.
 Example: Euclid‘s algorithm gcd(m, n) = gcd(n, m mod n)
The simple observation that, the second integer gets smaller on each iteration, the
algorithm stops when the second integer becomes 0.
 For some algorithms, a proof of correctness is quite easy; for others, it can be quite
complex. A common technique for proving correctness is to use mathematical induction.
 The notion of correctness for approximation algorithms is less straightforward than it is
for exact algorithms. For an approximation algorithm, we usually would like to be able to
show that the error produced by the algorithm does not exceed a predefined limit.

Efficiency of an Algorithm
 Two kinds of algorithm efficiency:
i) Time efficiency, indicating how fast the algorithm runs, and
ii) Space efficiency, indicating how much extra memory it uses.

Generality of Algorithm:
Two issues here:
i) Generality of the problem the algorithm solves: it is sometimes easier to design an
algorithm for a problem posed in more general terms. Ex: computing GCD.
ii) The set of inputs it accepts: Your main concern should be designing an algorithm that can
handle a set of inputs that is natural for the problem.

Prof. Nandan G P, BCA,SRS FGC - Chitradurga

 DSC13: Design and Analysis of Algorithm Page 10
Unit – 1: Introduction to Algorithm

1.3 Important Problem Types

The most important problem types:

 Sorting
 Searching
 String processing
 Graph problems
 Combinatorial problems
 Geometric problems
 Numerical problems

 Sorting
Sorting problem is to rearrange the items of a given list in Ascending/Descending order. we
usually sort lists of numbers, characters, character strings, and, most important, records of
students, libraries, and companies. In the case of records, we choose a piece of information to
guide sorting. For example, To sort student records by names or by roll-no or by Grade
average. Such a specially chosen piece of information is called a key.
 Searching
The searching problem deals with finding a given value, called a search key, in a given set.
For searching, too, there is no single algorithm that fits all situations best. Some algorithms
work faster than others but require more memory; some are very fast but applicable only to
sorted arrays; and so on.

 String processing
A string is a sequence of characters from an alphabet. Searching for a given word in a text is
called string matching.

 Graph problems
A graph can be thought of as a collection of points called vertices, some of which are
connected by line segments called edges. Graph algorithms include shortest-path algorithms,
and topological sorting for graphs with directed edges

 Combinatorial problems
These are problems that ask, explicitly or implicitly, to find a combinatorial object—such as a
permutation, a combination, or a subset—that satisfies certain constraints.

 Geometric problems
Geometric algorithms deal with geometric objects such as points, lines, and polygons.

 Numerical problems
Numerical problems, another large special area of applications, are problems that involve
mathematical objects of continuous nature: solving equations and systems of equations,
computing definite integrals, evaluating functions, and so on. The majority of such
mathematical problems can be solved only approximately.

Prof. Nandan G P, BCA,SRS FGC - Chitradurga

 DSC13: Design and Analysis of Algorithm Page 11
Unit – 1: Introduction to Algorithm

1.4 Fundamental Data Structures.

Q5. Write short note on the fundamental data structures used in the design of
algorithms.

Fundamental Data Structures that are used by algorithms are:

1. Linear Data Structures

a. Array
b. Stack
c. Queue
d. Lists
2. Non-Linear Data structure
a. Graphs
b. Trees
c. Sets and Dictionaries

1. Liner Data structure

 Array
An Array is a sequence of n items of the same data type that are stored contiguously in
computer memory and made accessible by specifying a value of the array‘s index

Item[0] Item[0] ....... Item[n-1]

 Linked List
A linked list is a sequence of zero or more elements called nodes, each containing two
kinds of information: some data and one or more links called pointers to other nodes of
the linked list.

Item 0 Item 1 ...... Item n-1 null

.

 Stack
A stack is a list in which insertions and deletions can be done only at the end. This end is
called the top because a stack is usually visualized not horizontally but vertically.

 Queue
A queue, is a list from which elements are deleted from one end, called the front
(dequeue), and new elements are added to the other end, called the rear (enqueue).

Prof. Nandan G P, BCA,SRS FGC - Chitradurga

 DSC13: Design and Analysis of Algorithm Page 12
Unit – 1: Introduction to Algorithm

2. Non-Linear Data structure

 Graph
- Graph 𝑮 =< 𝑉, 𝐸 > is defined by a pair of two sets: a finite nonempty set V of items
called vertices and a set E of pairs of these items called edges.
- Undirected Graph: A graph G is called undirected if every edge in it is undirected.
- Digraph: A graph whose every edge is directed is called directed Graph. Directed
graphs are also called digraphs.

a c b a c b

d e f d e f

(a) (b)
FIGURE : (a) undirected graph. (b) Digraph

Graph Representations
- The Adjacency Matrix of a graph with n vertices is an n × n boolean matrix with one
row and one column for each of the graph‘s vertices, in which the element in the i th row
and the j th column is equal to 1 if there is an edge from the i th vertex to the j th vertex,
and equal to 0 if there is no such edge.
- The Adjacency Lists of a graph or a digraph is a collection of linked lists, one for each
vertex, that contain all the vertices adjacent to the list‘s vertex (i.e., all the vertices
connected to it by an edge).

a b c d e f c
a d
a 0 0 1 1 0 0
b c f
b 0 0 1 0 0 1
c a b c
c 0 0 0 0 1 0
d a e
d 0 0 0 0 1 0
e 0 0 1 1 0 1 e c d e
f 0 1 0 0 1 0 f b e

(a) Adjacency Matrix (b) Adjacency List

- Weighted Graphs A weighted graph (or weighted digraph) is a graph (or di-graph) with
numbers assigned to its edges. These numbers are called weights or costs.

Prof. Nandan G P, BCA,SRS FGC - Chitradurga

 DSC13: Design and Analysis of Algorithm Page 13
Unit – 1: Introduction to Algorithm

 Trees
A tree is a connected acyclic graph.
Forest: An acyclic graph that is not necessarily connected is called a forest. Each of its
connected components is a tree

 Sets and Dictionaries

A set can be described as an unordered collection (possibly empty) of distinct items called
elements of the set.
Set representation:
i) A set is defined either by an explicit listing of its elements
e.g., S = {2, 3, 5, 7} or
ii) By specifying a property that all the set‘s elements and only they must satisfy
e.g., S = {n: n is a prime number smaller than 10}.

Operations are:
i) Finding the union of two sets, which comprises all the elements in either or both of them;
ii) finding the intersection of two sets, which comprises all the common elements in the sets.

Question:

How many ways a graph can be represented in a computer? What is the advantage of
First-Child-Next sibling representation of a tree. Explain with example. –8 marks.

Prof. Nandan G P, BCA,SRS FGC - Chitradurga

 DSC13: Design and Analysis of Algorithm Page 14
Unit – 1: Introduction to Algorithm

1.5 The Analysis Framework.

Q2. Define time complexity? What are the factors on which time complexity depends on?

 There are two kinds of efficiency: time efficiency and space efficiency.
 Time efficiency (time complexity): Indicates how fast an algorithm in question runs.
 Space efficiency (space complexity): Refers to the amount of memory units required by
the algorithm in addition to the space needed for its input and output.
General framework for analyzing the efficiency of algorithms is:
1. Measuring an Input’s Size
2. Units for Measuring Running Time
3. Orders of Growth
4. Worst-Case, Best-Case, and Average-Case Efficiencies
5. Recapitulation of the Analysis Framework

1. Measuring an Input’s Size

 How can we measure an input size? For each input there is a different aspect to size.
 Input size is influenced by
1. The Data Representation: e.g. Matrix
- Input size is number of elements of the matrix.
2. Operations of the algorithm: e.g. Spell checker
- Input size is the number of characters in the string.
3. Properties of objects: e.g. Checking for prime or not
- Input size is measured by the number b of bits in the n’s binary representation:
𝑏 = 𝑙𝑜𝑔2 𝑛 + 1

2. Units for Measuring Running Time [Solution for Q-6]

 Different units for measuring an algorithm‘s running time are:
4. Measure using standard time unit: in seconds, milliseconds
- Does not work as different architectures have the different computing speeds.
5. Count number operation of the algorithm.
- This approach is both excessively difficult and unnecessary.
6. Count the number of basic operations in algorithm
- Algorithm‘s time efficiency can be measure by number of times its basic
operation is executed on inputs of size n.
 Basic equation of time efficiency:
Let, T(n) = running time
T(n) ≈ Cop C(n)
Where, Cop = Execution time for basic operation,
C(n) = Number of times basic operation is executed.

Prof. Nandan G P, BCA,SRS FGC - Chitradurga

 DSC13: Design and Analysis of Algorithm Page 15
Unit – 1: Introduction to Algorithm

3. Orders of Growth
 We cannot compare one algorithm with other algorithms with small inputs. For large
values of n, it is the function‘s order of growth that counts. The magnitude of the numbers
in the Table 2.1 has a profound significance for the analysis of algorithms.
 The function growing the lowest among these is the logarithmic function.

4. Worst-Case, Best-Case, and Average-Case Efficiencies

 There are many algorithms where running time depends on not only on input size ‗n‘ but
also on some particular input.
 The worst-case efficiency of an algorithm is the worst-case input of size ‘n’ for which the
algorithm runs the longest among all possible inputs of that size.
 The best-case efficiency of an algorithm is the best-case input of size ‘n’, for which the
algorithm runs the fastest among all possible inputs of that size.
 The Average-case efficiency of an algorithm is its efficiency for the ―random‖ or
―typical‖ input, where it is neither best-case nor worst-case.
 The Amortized-case efficiency of an algorithm is its efficiency that applies to a sequence
of operations on same data structure.

Example: Consider Sequential search

-------------------------------------------------------------------------------------------------------
ALGORITHM: SequentialSearch ( A*0..n−1+, K)
//Input: An array A[0..n−1]and a search key K
//Output: The index of the first element in A that matches K otherwise −1
Begin,
i = 0
while i < n and A[i] ≠ K do
i = i + 1
end-while
if i < n return i
else return −1
end-Algo
---------------------------------------------------------------------------------------------

Prof. Nandan G P, BCA,SRS FGC - Chitradurga

 DSC13: Design and Analysis of Algorithm Page 16
Unit – 1: Introduction to Algorithm

Analysis
 Worst-case: Element not found; or the first match happens to be last element in Array A.
Cworst(n)= n
 Best-case: First math happens to be the First element in Array A.
Cbest(n)= 1
 Average-case:
The standard assumptions for average case are
a) the probability of a successful search is equal to ‗p‘ (0 ≤ p ≤ 1)
𝒑
b) Probability that the first match happens at the ith position is: and is same for every i.
𝒏
c) For unsuccessful match, no of comparisons is n with probability is: n(1-p)
𝑝 𝑝 𝑝
Cavg(n) = (1. + 2. +...+ n. ) + n(1-p)
𝑛 𝑛 𝑛
𝑝
= .(1.+ 2 +...+ n ) + n(1-p)
𝑛
𝑝 𝑛(𝑛+2)
= . + n(1-p)
𝑛 2
𝑝(𝑛+2)
= + n(1-p)
2

5. Recapitulation of the Analysis Framework

Summary of the framework
 Both time and space efficiencies are measured as functions of the algorithm‘s input size.
 Time efficiency is measured by counting the number of times the algorithm‘s basic
operation is executed.
 Space efficiency is measured by counting the number of extra memory units consumed by
the algorithm.
 The efficiencies of some algorithms may differ significantly for inputs of the same size.
For such algorithms, we need to distinguish between the worst-case, average-case, and
best-case efficiencies.

Prof. Nandan G P, BCA,SRS FGC - Chitradurga