Algorithm

"algorithm" is referred to a method that can be used by a computer for the solution
of a problem

Defn-1: is a sequence of unambiguous(meaning clear) instructions for solving a problem,

i.e., for obtaining a required output for any legitimate(valid) input in a finite amount of time.
Defn-2: An algorithm is a finite set of instructions that, if followed, accomplishes a particular
task.
In addition, all algorithms must satisfy the following criteria:
1. Input: Zero or more quantities are externally supplied.
2. Output: At least one quantity is produced.
3. Definiteness : each instruction is clear and unambiguous
4. Finiteness: If we trace out the instructions of an algorithm, then for all cases, the
algorithm terminates after a finite number of steps.
5. Effectiveness : Every instruction must be very basic so that it can be carried out, in
principle, by a person using only pencil and paper. it also must be feasible
Properties of algorithm
• The non-ambiguity requirement for each step of an algorithm cannot be compromised.
• The range of inputs for which an algorithm works has to be specified carefully.
• The same algorithm can be represented in several different ways.(multiplicity)
• There may exist several algorithms for solving the same problem.
• Different ideas different speed
Example : GCD of two non-negative integer numbers(not both zeros)
GCD => greatest common divisor of two nonnegative, not-both-zero integers m and n,
denoted gcd(m, n), is defined as the largest integer that divides both m and n evenly,
i.e., with a remainder of zero.
Methods used

1. Euclid’s algorithm(2 marks)

2. Consecutive integer checking algorithm
3. Middle-school procedure
1. Euclid’s algorithm(2 marks) => applying repeatedly the equality
gcd(m, n) = gcd(n, m mod n), until m mod n is equal to 0.
Result : Since gcd(m, 0) = m, the last value of m is the greatest common
divisor of the initial m and n.
E.g : gcd(60, 24) can be computed as follows: gcd(60, 24) = gcd(24, 12) = gcd(12, 0) = 12.
2. Consecutive integer checking algorithm for computing gcd(m, n)
▪ common divisor cannot be greater than the smaller of these numbers
How it works
1. Find t = min{m, n}.
2. start by checking whether t divides both m and n: if it does, t is the answer; if it
does not, simply decrease t by 1 and try again.

• does not work correctly when one of its input numbers is zero.
• illustrates it is important to specify the set of an algorithm’s inputs explicitly and
carefully.
• Euclid's algorithm is faster than consecutive algorithm.
3. Middle-school procedure for computing gcd(m, n)
Step 1 : Find the prime factors of m.
Step 2 : Find the prime factors of n.
Step 3 : Identify all the common factors in the two prime expansions found in Step 1
and Step 2. Step 4: Compute the product of all the common factors and return it as
the greatest common divisor of the numbers given.

much more complex and slower than Euclid’s algorithm.

prime factorization steps are not defined unambiguously(not an legitimate algorithm)
sieve of Eratosthenes=> Generating consecutive primes not exceeding any
given integer n > 1
1. starts by initializing a list of prime candidates with consecutive integers from 2 to
n.
2. on its first iteration, the algorithm eliminates from the list all multiples of 2, i.e., 4,
6, and so on.
3. Then it moves to the next item on the list, which is 3, and eliminates its multiples
4. algorithm continues until no more numbers can be eliminated from the list.
5. The remaining integers of the list are the primes needed
Fundamentals of the Analysis of Algorithmic Efficiency/Analysis
Framework
• Analysis of algorithms means investigation of an algorithm’s efficiency
with respect to two resources: running time and memory space
1. Time efficiency/time complexity=> amount of computer time it takes
to run for completion/how fast an algorithm runs.
2.Space efficiency/space complexity=> amount of memory it needs to
run for completion.
• checking whether the algorithm is efficient or not means analyzing the
algorithm
• Systematic approach is modelled(showed) by a framework(structure) called
as ANALYSIS FRAMEWORK.
• The efficiency of an algorithm => decided by measuring the performance
Space complexity
⮚ To compute the space complexity two factors are consider : constant and
instance characteristic
⮚ Space requirement S(P) of any algorithm P
S(p) = C + Sp
C – Constant -> space taken by instruction, variable and identifiers
Sp – space dependent upon instance characteristic
✔ When analyzing the space complexity of an algorithm , concentrate solely
on estimating Sp(instance characteristics).
✔ For any problem first determine which instance characteristics use to
measure the space requirements
✔ This is problem specific
E.g.1
Algorithm abc (a, b, c)
// addition of 3 numbers
//input: 3 positive integers
//output: sum
return a + b + c;

S(P)= = C + Sp => 3+0= 3 units of memory

C=3(the size need by a,b,c)
Sp=0(no memory is needed based on the input a or b or c)

E.g.2
S(P)= = C + Sp => 3+n= 3+n units of
memory
C=3(the size need by s, n, i)
Sp=n(memory for array a based on the input
n)
Time Complexity
T(P) taken by a program P =compile time + the run (or execution)time.
• The compile time does not depend on the instance characteristics.
• a compiled program can be run several times without recompilation
• Therefore consider only run time of a program.
• run time is denoted by tp (instance characteristics)
Frequency count/step count : a count denoting number of times of execution
of statement
Measuring an Input’s Size
• Time complexity depends on the number of inputs => i.e, the running time of
an algorithm increases with the input size.
✔ E.g., : It takes longer time to sort larger arrays, to multiply larger matrices
and so on.
• So, an algorithm’s efficiency is calculated as a function of parameter n
which indicates the algorithm’s input size.
• In most cases, selecting parameter n is straight forward. Ex :
1. For problems of sorting, searching, finding largest element etc., the size
metric will be the size of the list.
2. For problem of evaluating a polynomial P(x) = an xn +... + a0 of degree n, the
input size metric will be the polynomial’s degree or the number of
coefficients, which is larger by one than its degree
• parameter indicating an input size does matter
✔ computing the product of two n × n matrices.=> two natural measures of
size
i. matrix order n.
ii. total number of elements N in the matrices being multiplied
• choice of an appropriate size metric can be influenced by operations of the
algorithm
i. algorithm examines individual characters => the size measure will be the
number of characters in the input.
ii. If the algorithm works by processing words=>the size measure will be
the number of words in the input
• Some algorithms require more than one parameter to indicate the size of their
inputs (e.g., the number of vertices and the number of edges for algorithms
Units for Measuring Running Time
• Some standard unit of time measurements – a second, a millisecond, and so
on can be used to measure the running time of a program implementing
the algorithm.
• Drawbacks : The running time of an algorithm depends on:
i. Speed of a particular computer
ii. Quality of a program implementing the algorithm
iii. compiler used in generating the machine code
iv. difficulty of clocking the actual running time of the program.
• Therefore to measure of an algorithm’s efficiency, use a metric that does not
depend on these extraneous(peripheral) factors(h/w and s/w)
To measure an algorithm’s efficiency:
• One approach is to count the number of times each of the algorithm’s
operations are executed. =>excessively difficult and usually unnecessary .
• another approach is to identify the basic operation(primitive/important
operation)
✔ the operation contributing the most to the total running time
✔ compute the number of times the basic operation is executed on inputs
of size n.
• Ex:
1. Sorting algorithms works by comparing elements of a list being sorted
with each other. For such algorithms, basic operation is a key comparison.
2. Matrix multiplication requires two arithmetic operations: multiplication
and addition.
Small Example
• Let Cop be the execution time of an algorithm’s basic operation on a
particular computer.
• C(n) be the number of times this basic operation needs to be executed
for this algorithm.
running time T(n) ≈ Cop. C(n)
✔ Cop is an approximation
✔ C(n) does not contain any information about operations that are not basic.
• the formula works for medium range of i/p’s , does not work form larger
/smaller i/p’s
E.g.,1: “How much faster would an algorithm run on a machine that is 10 times
faster than the one we have?” Ans: 10 times.
E.g.2: Assuming , , how much longer will the algorithm run if we
double its input size?
Solution:

Note: The Efficiency analysis framework ignores multiplicative constants and

concentrates on the basic operation count’s order of growth for large-size inputs.
.
Orders of Growth => Measuring the performance of an algorithm with respect
to the input sizes.

• orders of growth of the functions 2 n and n!, yet both are often referred to as
“exponential-growth functions”
• Algorithms that require an exponential number of operations are practical for
solving only problems of very small sizes
Worst-Case, Best-Case, and Average-Case Efficiencies
• Algorithm efficiency depends on the input size n
• For some algorithms efficiency depends on type of input.
1. Worst-case efficiency=> algorithm runs the longest(more) among all possible
inputs of size n.
2. Best-case Efficiency => algorithm runs the fastest(less) among all possible inputs
of size n.
3. Average-case Efficiency=> typical/random input of size n.
Example: Sequential Search.
Problem: Given a list of n elements and a search key K, find an element equal to K,
if any. Algorithm: Scan the list and compare its successive elements with K until
either a matching element is found (successful search) or the list is exhausted
(unsuccessful search)
Basic Operation: Comparison(sequentially)
1. Worst-case efficiency => this algorithm makes the largest number of key
comparisons among all possible inputs of size n
a. when there are no matching elements(A<>K) (or)
b. the first matching element happens to be the last one on the list(K=A[n-1])

Cworst(n) = n
Note: gives very important information about an algorithm’s efficiency by
bounding its running time, for any instance of size n, the running time will not
exceed Cworst(n)

2. Best-case efficiency

▪ first element equal to a search key(K=A[0])

3. Average Case Efficiency
Calculation assumptions:
⮚ Probability of successful search is equal to p
⮚ Probability of the first match occurring in the ith position of the list is the same
for every i.

To find the average number of key comparisons Cavg(n)

✔ For Successful search
• probability of the first match occurring in the ith position of the list is p/n for
every i (and)
• the number of comparisons is i.
✔ For unsuccessful search
• the number of comparisons will be n (and)
• the probability of such a search being (1− p).
From the general formula of Cavg
⮚ For successful search
✔ p = 1 , the average number of comparisons made = (n + 1)/2; on average,
about half of
⮚ For unsuccessful search
✔ p= 0 , the average number of comparisons =n, because the algorithm will
compare all n elements
Fundamentals of Algorithmic Problem Solving
• Algorithm=>a procedural solutions to problems
• Here solutions are not answers => specific instructions for getting answers/ existence of a
solution to a problem
• Therefore, requires steps in designing and analyzing an algorithm
sequence of steps in designing and analyzing an algorithm
Important problem types

1. sorting=> rearranging the elements in the list (ascending/descending order)

• Key is need to sort the elements in the list
• Why sorting?
✔ makes questions to answer easier. e.g., searching process easy
Sorting is used as auxiliary step in many algorithms. E.g., Binary Search
Sorting algorithms
• are simple but relatively slow, while others are faster but more complex
• some work better on randomly ordered inputs, some on almost-sorted lists
• some are suitable only for lists residing in the fast memory, some adapted
for sorting large files stored on a disk
Two properties of sorting algorithms
1. Stable
⮚ preserves the relative order of any two equal elements in its input
⮚ E.g., bubble sort, insertion sort, merge sort
⮚ not stable like Quick Sort, Heap Sort

2. In-place => if it does not require extra memory, except, possibly, for a few
memory units.
⮚ Merge sort is a stable algorithm but not an in-place algorithm. It requires extra
array storage.
Types of sorting
1. Internal Sorting
⮚ Input size are small
⮚ Entire sorting is done in main memory
2.External Sorting
⮚ Used to Sort huge amount of data that resides in secondary storage where the
speed is relatively slow(movement of data between secondary and main memory)
⮚ e.g., merge sort
2. Searching => finding a given value, called a search key, in a given set
• plenty of searching algorithms available
• PageRank (PR) is an algorithm used by Google Search to rank web pages in their
search engine results
• searching is combined with two other operations: an addition to and deletion from
the data set of an item
3. String Processing
• Deals with non numerical data
• string is a sequence of characters from an alphabet
1. text strings: which comprise letters, numbers, and special characters
2. bit strings: which comprise zeros and ones
3. gene sequences: which can be modeled by strings of characters from the four-
character alphabet {A,C, G, T}
• string matching algorithm => process of searching for a given word in a
text
4. Graph problems
Applications
1. transportation
2. communication
3. project scheduling
Basic graph algorithms
a. graph-traversal algorithms(BFS & DFS)
b. Shortest path algorithm
c. MST
d. TSP
e. Graph-coloring
5. Combinatorial problems
Find a combinatorial object: permutation, combination, subset that satisfies
constraints and objectives
6. Geometric problems : Closest-pair problem: given n points in the plane, find the
closest pair
7. Numerical problems : Solving equations
Empirical(based on experience) Analysis of Algorithms
alternative to the mathematical analysis
General Plan for the Empirical Analysis of Algorithm Time Efficiency
1. Understand the experiment’s purpose.
2. Decide on the efficiency metric M to be measured and the measurement unit
(an operation count vs. a time unit).
3. Decide on characteristics of the input sample (its range, size, and so on).
4. Prepare a program implementing the algorithm (or algorithms) for the
experimentation.
5. Generate a sample of inputs.
6. Run the algorithm (or algorithms) on the sample’s inputs and record the data
observed.
7. Analyze the data obtained.
Ex: pseudorandom(generated by linear congruential method.)
“Pseudo”, because generating numbers using a known method removes the potential for true
randomness.

• seed :chosen arbitrarily and is often set to the current date and time
• m :should be large
• a:should be selected as an integer between 0.01m and 0.99m
• b=1
Visualization
• uses images to convey some useful information about algorithms
• uses graphic elements—points, line segments, two- or three-dimensional bars
• are two principal variations of algorithm visualization
1. Static algorithm visualization : algorithm’s progress through a series of still
images
2. Dynamic algorithm visualization, also called algorithm animation : shows a
continuous,movie-like presentation of an algorithm’s operations