CA-5.2 & 7.

5 Design and Analysis of Algorithms

Unit 1:
INTRODUCTION:
Problem, Instance, Analysis of Algorithms, Principles of Algorithm Design, Phases
of Algorithm design and analysis, Asymptotic complexity, Recursion, Rules of
Removal of Recursion

In the context of algorithms, understanding the concepts of "Problem" and "Instance" is

fundamental as they form the basis for defining and solving computational tasks.
Problem
A problem in the context of algorithms refers to a general computational task or question that we
want to solve. It defines what needs to be accomplished without specifying how to achieve it.
Problems are typically described in a way that allows multiple solutions.
Characteristics of a Problem:
Abstract: Describes what needs to be done without specifying how to do it.
General: Can have multiple instances (specific inputs) that require solutions.
Can Have Different Solutions: A single problem can often be solved by different algorithms,
each with different efficiencies or trade-offs.
Example:
Problem: Sorting a list of numbers in ascending order.

Instance
An instance of a problem is a specific example or input that is given to an algorithm to solve. It
takes the abstract problem statement and provides concrete data or parameters that the algorithm
will work on.
Characteristics of an Instance:
Concrete: Provides specific values or data that the algorithm will process.
Defines Inputs: Specifies the exact inputs and parameters for a particular run of the algorithm.
Unique: Each instance can be different, even if they belong to the same problem class.
Example:
Problem: Sorting a list of numbers in ascending order.
Instance: [5, 2, 8, 1, 3]
This instance specifies the exact list of numbers that needs to be sorted.

Relationship Between Problem and Instance

Problem: Defines the abstract task or goal.
Instance: Provides the specific data or inputs for the algorithm to work on.
Understanding the distinction between a problem and its instance is crucial in algorithm design
and analysis. The problem sets the stage for what needs to be achieved, while instances provide
the actual data that algorithms operate on to produce solutions. This clear distinction helps in
formulating algorithms that are both effective and efficient in solving computational tasks.

Algorithms:
Precise method useable by a computer for the solution of a problem. An Algorithm is composed
of a finite set of steps, each of which may require one or more operations.
Algorithms Properties:
1. Definite: means it must be perfectly clear what should be done.
2. Effective: each step must be such that it can, be done by a person using pencil and
paper in a finite amount of time.
3. Terminate: algorithm terminate after a finite number of operations.
4. Having one or more outputs and may have zero or more inputs.

Analysis of Algorithms:
Analysis of Algorithms involves evaluating algorithms to understand their efficiency and
performance characteristics. Given an algorithm to be analyzed,
1. The first task is to determine which operations are employed and what their relative costs
are.
2. The second task is to determine a sufficient number of data sets which cause the
algorithm to exhibit all possible patterns of behavior.
In producing a complete analysis of the computing time of an algorithm, we distinguish between
two phases:
1. Priori Analysis:
We obtain a function which bounds the algorithms computing time.
2. Posterior Analysis:
We collect actual statistics about the algorithms consumption of time and space, while it
is executing.
This analysis helps in comparing different algorithms for solving the same problem and selecting
the most appropriate one based on various factors like time complexity, space complexity, and
practical considerations.
Key Aspects of Analysis:
Time Complexity: This measures the amount of time an algorithm takes to run as a function of
the size of its input. It provides an upper bound on the running time. Common notations include
Big O (O), Big Omega (Ω), and Big Theta (Θ).
Space Complexity: This measures the amount of memory or space an algorithm uses as a
function of the size of its input. It also uses notations like O, Ω, and Θ.
Best, Worst, and Average Case Analysis: Algorithms can perform differently depending on the
input data. Analysis considers these cases to understand algorithm behavior comprehensively.
Asymptotic Analysis: Focuses on how algorithms behave as the input size grows very large. It
helps in understanding the algorithm's scalability.
For example, consider the three program segments a, b, c:
a. X ← X + Y The frequency count of this statement is i.e. O(1)
b. for i ← 1 to n do The frequency of this statement is n i.e. O(n)
x←x+y
repeat
c. for i ← 1 to n do The frequency of this statement is n2 i.e. O(n2)
for j ← 1 to n do
x←x+y
repeat
repeat
Principles of Algorithm Design:
Principles of Algorithm Design guide the creation of efficient and effective algorithms that solve
problems correctly and with optimal resource usage.
Key Principles:
Correctness: An algorithm must produce the correct output for all valid inputs. This is ensured
through rigorous testing and logical reasoning.
Efficiency: Algorithms should use resources (time, memory) efficiently. This includes
minimizing time complexity (fast execution) and space complexity (minimal memory usage).
Scalability: Algorithms should handle large inputs gracefully without a significant increase in
execution time or memory usage.
Modularity: Breaking down algorithms into smaller, manageable components makes them
easier to understand, maintain, and debug.
Optimization: Continuously improving algorithms to achieve better performance while
maintaining correctness and efficiency.

Phases of Algorithm Design and Analysis:

Algorithm Design and Analysis involves several phases to systematically develop and evaluate
algorithms.
Problem Definition: Clearly define the problem statement, inputs, and expected outputs.
Understand the constraints and requirements.
Algorithm Design: Devise a step-by-step approach (algorithm) to solve the problem. Consider
different strategies (e.g., brute force, divide and conquer) and choose the most appropriate.
Algorithm Analysis: Evaluate the algorithm's performance in terms of time complexity, space
complexity, and other relevant factors. Compare it with alternative algorithms.
Implementation: Translate the algorithm into a programming language. Write code that follows
the design and analysis.
Testing and Debugging: Test the algorithm with various inputs to ensure correctness and
efficiency. Debug any errors or issues that arise.
Optimization: Improve the algorithm based on analysis and testing results. Enhance
performance without sacrificing correctness or clarity.
Asymptotic complexity:
Asymptotic complexity is a critical concept in the analysis of algorithms, providing a way to
estimate how the runtime or space requirements of an algorithm grow with respect to the size of
its input. It focuses on how algorithms behave as the input size approaches infinity.

Understanding Asymptotic Notations

Asymptotic notations, such as Big O (O), Big Omega (Ω), and Big Theta (Θ), are used to
describe the asymptotic behavior of algorithms:
Big O Notation (O): This notation provides an upper bound on the asymptotic growth rate of the
algorithm's running time or space complexity. It defines the worst-case scenario.
f(n) = O(g(n)) (read as “f of n equals big oh of g of n”) iff there exist two positive constants c
and n0 such that |f(n)| <= c|g(n)| for all n >= n0
Example: If an algorithm has a time complexity of O(n2), this means that its running time grows
no faster than n2 as the input size n increases.

Big Omega Notation (Ω): This notation provides a lower bound on the asymptotic growth rate
of the algorithm's running time or space complexity. It defines the best-case scenario.
f(n) = Ω(g(n)) (read as “f of n equals omega of g of n”) iff there exist two positive constants c
and n0 such that |f(n)| >= c|g(n)| for all n > n0
Example: If an algorithm has a time complexity of Ω(n), this means that its running time is at
least proportional to n as n becomes large.

Big Theta Notation (Θ): This notation provides a tight bound on the asymptotic growth rate of
the algorithm's running time or space complexity, combining both upper and lower bounds.
f(n) = Ө(g(n)) (read as “f of n equals theta of g of n”) iff there exist two positive constants c1, c2
and n0 such that c1 |g(n)| <= |f(n)| <= c2 |g(n)| for all n > n0
Example: If an algorithm has a time complexity of Θ(n), this means that its running time grows
linearly with n as n increases, with a constant factor.
Why Asymptotic Complexity Matters
Comparative Analysis: Asymptotic complexity allows us to compare algorithms and determine
which one is more efficient for large inputs.
Predictive: It predicts how an algorithm will perform as the size of the input grows, helping in
making informed decisions about algorithm selection.
Simplicity: It abstracts away constant factors and lower-order terms, focusing on the dominant
factors that determine the algorithm's efficiency.

Examples of Common Time Complexities

O(1): Constant time complexity. Operations take a constant time, independent of the input size.
O(log n): Logarithmic time complexity. Common in algorithms that divide the problem in half at
each step (e.g., binary search).
O(n): Linear time complexity. Time taken grows linearly with the size of the input.
O(n log n): Log-linear time complexity. Often seen in efficient sorting algorithms like merge
sort and quicksort.

Recursion
Recursion is a fundamental concept in computer science and programming where a function calls
itself directly or indirectly to solve a smaller instance of the same problem. It's a powerful
technique that allows for elegant and concise solutions to certain types of problems, particularly
those that exhibit repetitive or self-similar structures.

Example: Factorial Calculation

Problem: Compute the factorial of a non-negative integer 𝑛, denoted as 𝑛!, which is the product
of all positive integers less than or equal to 𝑛.

Base Case: This is the terminating condition that stops the recursion. It defines the simplest
problem or smallest input for which the function does not make further recursive calls. Without a
base case, the recursion would continue indefinitely, leading to stack overflow or infinite
recursion.
Recursive Case: This is where the function calls itself with a smaller or simpler input than the
current state. The recursive case breaks down the larger problem into smaller subproblems until
reaching the base case.

def factorial(n):
if n == 0: # Base case
return 1
else: # Recursive case
return n * factorial(n - 1)

Removal of Recursion:
Steps:
1. At the beginning of the procedure, code is inserted which declares a stack and initializes
it to be empty. In the most general case, the stack will be used to hold the values of
parameters, local variables, function value, and return address for each recursive call.
2. The label L1 is attached to the first executable statement.
3. Store the values of all parameters and local variables in the stack. The pointer to the top
of the stack can be treated as global.
4. Create the ith new label, Li, and store I in the stack. The value i of this label will be used
to compute the return address. This label is placed in the program as described in rule(7).
5. Evaluate the arguments of this call and assign these values to the appropriate formal
parameters.
6. Insert an unconditional branch to the beginning of the procedure.
7. If this procedure is a function, attach the label created in (4) to a statement which
retrieves the function value from the top of the stack. Add code to use this value in the
way described in the recursive procedure. If this procedure is not a function, then affix
the label created in (4) to the statement immediate following the branch of (6)
These steps are sufficient to remove all recursive calls in a procedure. We must now alter all
return statements in the following way. In place of each return do the following:
 8. If the stack is empty, then execute a normal return.
9. Otherwise take the current values of all output parameters and assign these values to the
corresponding variables which are in the top of the stack.
10. Now insert code which removes the index of the return address from the stack if one has
been placed there. Assign this address to some unused variable.
11. Remove from the stack the values of all local variables and parameters and assign them
to their corresponding variables.
12. If this is a function, insert instructions to evaluate the expression immediately following
return and store the result in the top of the stack.
13. Use the index of the label of the return address to execute a branch to that label.
Consider the following example,
Procedure MAX1(i)
// this is function which returns the largest integer k
// such that A(k) is the maximum element in A(i:n)
global Integer n, A(1:n), j, k
integer i
if i<n then j ← MAX(i+1)
if A(i) > A(j) then k ← i
else k ← j
endif
else k ← n
endif
return(k)
end MAX1
Algorithm after removal of recursion:
Procedure MAX2(i)
local integer j, k; global integer n, A(1:n)
integer I
integer STACK (1:2 * n) //rule (1)
top ← 0 //rule (1)
L1: if I < n //rule (2)
 then top ← top + 1; STACK(top) ← i; //rule (3)
top ← top + 1; STACK(top) ← 2; //rule (4)
i ← i +1 //rule (5)
go to L1 //rule (6)
L2: j ← STACK (top;
top ← top – 1 //rule (7)
if A(i) > A(j) then k ← i
else k ← j
endif
else k ← n
endif
if top = 0 then return (k) //rule (8)
else addr ← STACK(top); top ← top -1 //rule (10)
i ← STACK(top); top ← top – 1 //rule (11)
top ← top + 1; STACK(top) ← k //rule (12)
If addr = 2 then go to L2 endif //rule (13)
endif
End MAX2
Now we begin to simplify the program, we needn't stack the return address since there is only
one place to which the procedure returns. This leaves only the function values in the stack.
However, at any point in time there is only one value of the function, that is, the index of the
current maximum.
Thus we can store this value in a single variable and eliminate the stack entirely. Another
simplification is to remove the loop created by the statement go to L1.
Equivalently, we set i to n and use k to hold the index of the current maximum. The resulting
simplified program follows:
Procedure MAX3(A,n)
integer i, k, n
i←k←n
while i > 1 do
i←i–1
 if A(i) > A(k) then k ← i endif
repeat
return(k)
end MAX3