lOMoARcPSD|45233642

Fundamentals of Algorithmic Problem Solving

Analysis and Design of Algorithms (Jain (Deemed-to-be University))

Scan to open on Studocu

Studocu is not sponsored or endorsed by any college or university

Downloaded by ajay ([email protected])
 lOMoARcPSD|45233642

16BCA5D11 ANALYSIS AND DESIGN OF ALGORITHMS

Fundamentals of Algorithmic Problem Solving

A sequence of steps involved in designing and analyzing an algorithm is shown in

the figure below.

Figure: Flow diagram of designing and analyzing an algorithm

(i) Understanding the Problem

 This is the first step in designing of algorithm.

 Read the problem’s description carefully to understand the problem

statement completely.
 Ask questions for clarifying the doubts about the problem.

 Identify the problem types and use existing algorithm to find solution.

 Input (instance) to the problem and range of the input get fixed.

(ii) Decision making

The Decision making is done on the following:
(a) Ascertaining the Capabilities of the Computational Device
V Sem, Dept of BCA 9

Downloaded by ajay ([email protected])

 lOMoARcPSD|45233642

16BCA5D11 ANALYSIS AND DESIGN OF ALGORITHMS

 In random-access machine (RAM), instructions are executed one

after another (The central assumption is that one operation at a

time). Accordingly, algorithms designed to be executed on such
machines are called sequential algorithms.
 In some newer computers, operations are executed concurrently,

i.e., in parallel. Algorithms that take advantage of this capability

are called parallel algorithms.
 Choice of computational devices like Processor and memory is

mainly based on space and time efficiency

(b) Choosing between Exact and Approximate Problem Solving

 The next principal decision is to choose between solving the

problem exactly and solving it approximately.

An algorithm used to solve the problem exactly and produce

correct result is called an exact algorithm.

 If the problem is so complex and not able to get exact solution,

then we have to choose an algorithm called an approximation

algorithm. i.e., produces an approximate answer. E.g., extracting
square roots, solving nonlinear equations, and evaluating definite
integrals.

(c) Algorithm Design Techniques

 An algorithm design technique (or “strategy” or “paradigm”) is a

general approach to solving problems algorithmically that is

applicable to a variety of problems from different areas of
computing.

V Sem, Dept of BCA 10

Downloaded by ajay ([email protected])

 lOMoARcPSD|45233642

16BCA5D11 ANALYSIS AND DESIGN OF ALGORITHMS

 Algorithms+ Data Structures = Programs

 Though Algorithms and Data Structures are independent, but

they are combined together to develop program. Hence the choice

of proper data structure is required before designing the algorithm.
 Implementation of algorithm is possible only with the help of

Algorithms and Data Structures

 Algorithmic strategy / technique / paradigm are a general

approach by which many problems can be solved algorithmically.

E.g., Brute Force, Divide and Conquer, Dynamic Programming,
Greedy Technique and so on

(iii) Methods of Specifying an Algorithm

There are three ways to specify an algorithm. They are:
a. Natural language

b. Pseudo code

c. Flowchart

(iv)Proving an Algorithm’s Correctness

a. Once an algorithm has been specified then its correctness must be proved.

b. An algorithm must yields a required result for every legitimate

input in a finite amount of time. For example, the correctness of

Euclid’s algorithm for computing the greatest common divisor
stems from the correctness of the equality gcd(m, n) = gcd(n, m
mod n).
c. A common technique for proving correctness is to use
mathematical induction because an algorithm’s iterations provide a
natural sequence of steps needed for such proofs.
d. The notion of correctness for approximation algorithms is less

straightforward than it is for exact algorithms. The error produced

by the algorithm should not exceed a predefined limit
V Sem, Dept of BCA 11

Downloaded by ajay ([email protected])

 lOMoARcPSD|45233642

16BCA5D11 ANALYSIS AND DESIGN OF ALGORITHMS

(v)Analyzing an Algorithm
a. For an algorithm the most important is efficiency. In fact, there are

two kinds of algorithm efficiency. They are:

b. Time efficiency, indicating how fast the algorithm runs, and

c. Space efficiency, indicating how much extra memory it uses.

d. The efficiency of an algorithm is determined by measuring both

time efficiency and space efficiency.

e. So factors to analyze an algorithm are:

 Time efficiency of an algorithm

 Space efficiency of an algorithm
 Simplicity of an algorithm
 Generality of an algorithm

(vi)Coding an Algorithm
a. The coding / implementation of an algorithm is done by a suitable

programming language like C, C++, JAVA.

b. The transition from an algorithm to a program can be done either

incorrectly or very inefficiently. Implementing an algorithm

correctly is necessary. The Algorithm power should not reduce by
inefficient implementation.

c. Standard tricks like computing a loop’s invariant (an expression

that does not change its value) outside the loop, collecting common
sub-expressions, replacing expensive operations by cheap ones,
selection of programming language and so on should be known to
the programmer.
d. Typically, such improvements can speed up a program only by a

constant factor, whereas a better algorithm can make a difference in

V Sem, Dept of BCA 12

Downloaded by ajay ([email protected])

 lOMoARcPSD|45233642

16BCA5D11 ANALYSIS AND DESIGN OF ALGORITHMS

running time by orders of magnitude.

e. It is very essential to write an optimized code (efficient code) to

reduce the burden of compiler.

Role of algorithms in computing
The most important problem types related to computing are:
1) Sorting: The sorting problem is to rearrange the items of a given list in

non-decreasing (ascending) order.

2) Searching: The searching problem deals with finding a given value, called

a search key, in a given set.

3) String processing: A string is a sequence of characters from an alphabet.

Strings comprise letters, numbers, and special characters; bit strings,

which comprise zeros and ones; and gene sequences, which can be
modeled by strings of characters from the four character alphabet {A, C,
G, T}. It is very useful in bioinformatics. Searching for a given word in a
text is called string matching.

4) Graph problems: A graph is a collection of points called vertices, some of

which are connected by line segments called edges. Some of the graph
problems are graph traversal, shortest path algorithm, topological sort,
traveling salesman problem and the graph-coloring problem and so on.

5) Combinatorial problems: These are problems that ask, explicitly or

implicitly, to find a combinatorial object such as a permutation, a

combination, or a subset that satisfies certain constraints.

6) Geometric problems: Geometric algorithms deal with geometric objects

such as points, lines, and polygons. Geometric algorithms are used in

V Sem, Dept of BCA 13

Downloaded by ajay ([email protected])

 lOMoARcPSD|45233642

16BCA5D11 ANALYSIS AND DESIGN OF ALGORITHMS

computer graphics, robotics, and tomography.

7) Numerical problems are problems that involve mathematical equations,

systems of equations, computing definite integrals, evaluating functions,

and so on. The majority of such mathematical problems can be solved
only approximately.

Algorithms as a Technology.

There a huge number of real life applications where, algorithms play the major
and vital role. Few of such applications are discussed below.

 The Internet without which it is difficult to imagine a day is the result of

clever and efficient algorithms. With the aid of these algorithms, various
sites on the Internet are able to manage and manipulate this large volume of
data. Finding good routes on which the data will travel and using search
engine to find pages on which particular information is present.

 Another great milestone is the Human Genome Project which has great
progress towards the goal of identification of the 100000 genes in human
DNA, determining the sequences of the 3 billion chemical base pairs that
make up the human DNA, storing this huge amount of information in
databases, and developing tools for data analysis. Each of these steps
required sophisticated and efficient algorithms.

 The day-to-day electronic commerce activities is hugely dependent on our

personal information such as credit/debit card numbers, passwords, bank
statements, OTPs and so on. The core technologies used include public-key
crypto currency and digital signatures which are based on numerical

V Sem, Dept of BCA 14

Downloaded by ajay ([email protected])

 lOMoARcPSD|45233642

16BCA5D11 ANALYSIS AND DESIGN OF ALGORITHMS

algorithms and number theory.

 The approach of linear programming is also one such technique which is

widely used like
 In manufacturing and other commercial enterprises where resources
need to be allocated scarcely in the most beneficial way.
 Or a institution may want to determine where to spend money buying
advertising in order to maximize the chances of their institution to
grow.

 Shortest path algorithm also has an extensive use as

 In a transportation firm such as a trucking or railroad company, may
have financial interest in finding shortest path through a road or rail
network because taking shortest path result in lower labour or fuel
costs.
 Or a routing node on the Internet may need to find the shortest path
through the network in order to route a message quickly.

 Even an application that does not require algorithm content at the

application level relies heavily on algorithms as the application depends on
hardware, GUI, networking or object orientation and all of these make an
extensive use of algorithms.

Getting Started

Fundamentals of the Analysis of Algorithm Efficiency

The efficiency of an algorithm can be in terms of time and space. The

algorithm efficiency can be analyzed by the following ways.
a. Analysis Framework.
b. Asymptotic Notations and its properties.

V Sem, Dept of BCA 15

Downloaded by ajay ([email protected])

 lOMoARcPSD|45233642

16BCA5D11 ANALYSIS AND DESIGN OF ALGORITHMS

c. Mathematical analysis for Recursive algorithms.

d. Mathematical analysis for Non-recursive algorithms.

Algorithm analysis is an important part of computational complexity theory,

which provides theoretical estimation for the required resources of an
algorithm to solve a specific computational problem. Most algorithms are
designed to work with inputs of arbitrary length. Analysis of algorithms is the
determination of the amount of time and space resources required to execute it.

The complexity of an algorithm describes the efficiency of the algorithm in

terms of the amount of the memory required to process the data and the
processing time.

Complexity of an algorithm is analyzed in two perspectives: Time and Space.

(a) Time Complexity

It’s a function describing the amount of time required to run an

algorithm in terms of the size of the input. "Time" can mean the
number of memory accesses performed, the number of comparisons
between integers, the number of times some inner loop is executed,
or some other natural unit related to the amount of real time the
algorithm will take.

Components that affect Time Efficiency :

A. Speed of the Computer
B. Choice of the Programming Language
C. Compiler Used
D. Choice of the Algorithm
E. Size of Inputs / Outputs

(b) Space Complexity

V Sem, Dept of BCA 16

Downloaded by ajay ([email protected])

 lOMoARcPSD|45233642

16BCA5D11 ANALYSIS AND DESIGN OF ALGORITHMS

It’s a function describing the amount of memory an algorithm takes

in terms of the size of input to the algorithm. We often speak of
"extra" memory needed, not counting the memory needed to store
the input itself. Again, we use natural (but fixed-length) units to
measure this.

Space complexity is sometimes ignored because the space used is

minimal and/or obvious, however sometimes it becomes as
important an issue as time.

Components that Affects Space Efficiency :

A. Program Space
B. Data Space
C. Stack Space

The Need for Analysis

In this section, we will discuss the need for analysis of algorithms and how to
choose a better algorithm for a particular problem as one computational
problem can be solved by different algorithms.
Algorithms are often quite different from one another, though the objective of
these algorithms are the same. For example, we know that a set of numbers
can be sorted using different algorithms. Number of comparisons performed
by one algorithm may vary with others for the same input. Hence, time
complexity of those algorithms may differ. At the same time, we need to
calculate the memory space required by each algorithm.
The algorithm analysis framework consists of the following:
 Measuring an Input’s Size
 Units for Measuring Running Time
 Orders of Growth
 Worst-Case, Best-Case, and Average-Case Efficiencies

Measuring an Input’s Size:

V Sem, Dept of BCA 17

Downloaded by ajay ([email protected])

 lOMoARcPSD|45233642

16BCA5D11 ANALYSIS AND DESIGN OF ALGORITHMS

An algorithm’s efficiency is defined as a function of some parameter n

indicating the algorithm’s input size. In most cases, selecting such a parameter
is quite straightforward. For example, it will be the size of the list for problems
of sorting, searching.

Units for Measuring Running Time:

Some standard unit of time measurement such as a second, or millisecond, and so
on can be used to measure the running time of a program after implementing the
algorithm.
Drawbacks
 Dependence on the speed of a particular computer.
 Dependence on the quality of a program implementing the algorithm.
 The compiler used in generating the machine code.
 The difficulty of clocking the actual running time of the program.
So, we need metric to measure an algorithm’s efficiency that does not depend
on these extraneous factors. One possible approach is to count the number of
times each of the algorithm’s operations is executed. This approach is
excessively difficult.

Orders of Growth
A difference in running times on small inputs is not what really
distinguishes efficient algorithms from inefficient ones. Some common
orders of growth seen often in complexity analysis are:

O(1) constant
O(log n) logarithmic
O(n) linear
O(n log "n log n"
n)
O(n2) quadratic
O(n )
3
cubic
nO(1)
polynomial
2O(n) exponential

V Sem, Dept of BCA 18

Downloaded by ajay ([email protected])

 lOMoARcPSD|45233642

16BCA5D11 ANALYSIS AND DESIGN OF ALGORITHMS

Table: Values (approximate) of several functions important for analysis of algorithms

Worst-Case, Best-Case, and Average-Case Efficiencies

Analysis of algorithm is the process of analyzing the problem-solving
capability of the algorithm in terms of the time and size required (the size of
memory for storage while implementation). However, the main concern of
analysis of algorithms is the required time or performance. Generally, we
perform the following types of analysis −

 Worst-case − The maximum number of steps taken on any instance of size a.

 Best-case − The minimum number of steps taken on any instance of size a.
 Average case − An average number of steps taken on any instance of size a.

EXAMPLE:
Consider the Sequential Search algorithm
SequentialSearch(A[0..n - 1], K)

//Searches for a given value in a given array by sequential search

//Input: An array A[0..n - 1] and a search key K
//Output: The index of the first element in A that matches K or -1 if there
is no match i ←0

V Sem, Dept of BCA 19

Downloaded by ajay ([email protected])

 lOMoARcPSD|45233642

16BCA5D11 ANALYSIS AND DESIGN OF ALGORITHMS

while i < n and A[i] ≠

K do i ←i + 1
if i < n
return i else
return -1

Clearly, the running time of this algorithm can be quite different for the same
list size n. In the worst case, there is no matching of elements or the first
matching element can found at last on the list. In the best case, there is matching
of elements at first on the list.
The worst-case efficiency of an algorithm is its efficiency for the worst case
input of size n. The algorithm runs the longest among all possible inputs of that
size.
The best-case efficiency of an algorithm is its efficiency for the best case input
of size n. The algorithm runs the fastest among all possible inputs of that size n.
In sequential search, If we search a first element in list of size n. (i.e. first
element equal to a search key).
The Average case efficiency lies between best case and worst case. To analyze
the algorithm’s average case efficiency, we must make some assumptions about
possible inputs of size n.

V Sem, Dept of BCA 20

Downloaded by ajay ([email protected])