Thursday, January 16, 2025 12:53 AM

analysis of algorithms

topic: algorithms

consists of finite set of steps/statements to solve a given problem in finite amount of time.

a procedure that reduces the solution of some class of problems to a series of rote steps which, if followed to the
latter, and as far as may be necessary, is bound to:

• always give some answer rather than ever give no answer.

• always give the right answer and never give a wrong answer.

• always be completed in a finite number of steps, rather than in an infinite number

• work for all instances of problems of the class, an algorithm must work for all valid instances of a given
problem. it should not fail or refuse to work for a specific subset of inputs.
- the algorithm should always provide a correct and meaningful output.
- if the input is invalid, the algorithm should gracefully handle errors instead of failing unexpectedly.

concept: fundamental operations

(i) each statement/steps consists of one and more basic fundamental operations.

fundamental operation cannot be further sub-divided.

(i) arithmetic operations
(ii) logical operations
(iii) assignment operations
Chapter - 1 (Analysis of Algorithms) Page 1
(iii) assignment operations
(iv) comparison operations
(v) data transfer
(vi) i/o operation

- example: x=y+z;
normal variables: x, y and z
basic and fundamental operations (cannot be further divided):
(i) addition
(ii) assignment
trivial (not neccesary)
- load

- example: for (i=1; i<=n; ++i);

basic and fundamental operations (cannot be further divided):
(i) initialization (i=1)
(ii) comparison (i<=n)
(iii) increment (update) (++i)
every step consists of one and more statements.

(ii) fundamental properties of an operation in an algorithm

each fundamental operation must satisfy two property:

(a) definiteness (unambigious/clear)

- every step in an algorithm must be precise and clear.
- each operation should have only one meaning, avoiding any confusion.

example of an ambiguous operation:

a=b*c
here, the operation * is undefined (is it multiplication, exponentiation, or something else?).

to be definite, we must clearly specify:

a = b × c (multiplication)
a = b ** c (exponentiation)

(b) effectiveness (complete in finite time)/finiteness

- each operation should be doable in a finite amount of time.
- an algorithm must always terminate after a finite number of steps.

Chapter - 1 (Analysis of Algorithms) Page 2

 - an algorithm must always terminate after a finite number of steps.
- if an operation does not complete in finite time, the algorithm may never end (infinite loop).

example of an ineffective operation:

while true:
print("hello")
- this loop runs forever, violating the effectiveness requirement.
- an algorithm must not have infinite loops unless explicitly required (e.g., an operating system running
indefinitely).

(iii) input and output requirements

(a) input
- an algorithm may take zero or more inputs.

example:
- sorting algorithm takes a list as input.
- printing a message (print("hello")) is also considered input (internal to the algorithm).

(b) output
- an algorithm must produce at least one output.
- without output, we cannot verify if the algorithm is correct or useful.

- example of an invalid algorithm:

def useless_function():
x = 10 # assigns a value but does nothing
- no output → we cannot check correctness.

- example of a valid algorithm:

def add(a, b):
return a + b
- returns a result → meets the output requirement.

conclusion
- every fundamental operation in an algorithm must be definite and effective.
- an algorithm must always terminate in finite time.
- it may take zero or more inputs but must always produce at least one output to be meaningful.

concept: problems vs algorithms vs programs

for each problem or class of problems there may be many different algorithms, for each algorithms there
may be many different implementations (programs).

problem (a task that requires a solution)

sorting

Chapter - 1 (Analysis of Algorithms) Page 3

 quick sort algorithm ( a step-by-step method to solve the problem)
bubble sort

java an implementation of algorithm in a programming language

c++

concept: expressing algorithms

an algorithm may be expressed in a number of ways, including:

(a) natural language: usually verbose and ambiguous (english)

(b) flow chart: avoid most (if not all) issues of ambiguity; difficult to modify without specialized tools;
largerly standardized (graphical way)

(c) pseudo-code (spark in sahani): also avoid most issues of ambiguity vaguely (weakly) resembles
common elements of prorgramming languages; no particular agreement on syntax.

(d) programming language: tend to require expressing low-level details that are not necessary for a high-
level understanding.

concept: common elements/statements of algorithms

(i) acquire data (input)
some means of reading values from an external source; most algorithms require data values to define the specific
problem (e.g., coefficients of a polynomial)

(ii) computation
some means of performing arithmetic computations, comparisons, testing logical conditions, and so forth...

(iii) selection
some means of choosing among two or more possible courses of action, based upon initial data, user input and/or
computed results

(iv) iteration
some means of repeatedly executing a collection of instructions, for a fixed number of times or until some logical
condition holds

(v) report results (output)

some means of reporting computed results to the user, or requesting additional data from the user

concept: lifecyle steps

lifecycle steps in algorithm design and analysis
algorithm development follows a structured lifecycle to ensure correctness, efficiency, and feasibility.

(i) problem definition (statement)

-
Chapter - 1 (Analysis of Algorithms) Page 4
- clearly define the input, output, and objective of the problem.
example: 4-queens problem → place 4 queens on a 4×4 chessboard so that no two queens attack each other.

(ii) constraints (conditions)

- define the rules or limitations that must be satisfied.
for the 4-queens problem, the constraints are:
○ no two queens should be in the same row.
○ no two queens should be in the same column.
○ no two queens should be on the same diagonal.
- this step is equivalent to software requirement specifications (srs).

(iii) design/logic (approach/strategy)

- choose an approach to solve the problem.
common strategies for 4-queens problem:
○ backtracking (most efficient for small board sizes).
○ brute force (try all possible placements).
○ constraint satisfaction (using optimization techniques).

(iv) express the algorithm (development)

- express the logic in a structured way (pseudocode or flowchart).
example: backtracking algorithm for 4-queens problem
1. start with an empty chessboard.
2. place a queen in a row, one at a time.
3. check if the queen placement is valid (not in same column or diagonal).
4. if valid, move to the next row.
5. if not valid, backtrack (remove the last placed queen and try a different column).
6. repeat until all 4 queens are placed.
7. if solution is found, return success; otherwise, return failure.

Chapter - 1 (Analysis of Algorithms) Page 5

(v) validation (correctness check)
- check whether the algorithm correctly follows the constraints.
for 4-queens, validate:
○ row constraints → no queens in the same row.
○ column constraints → no queens in the same column.
○ diagonal constraints → no queens on the same diagonal.

- techniques for validation:

○ mathematical proof (proof by induction).
○ test cases (trying different board configurations).

(vi) analysis (complexity and efficiency)

- determine how efficient the algorithm is in terms of time complexity and space complexity.
- for 4-queens using backtracking:
○ worst-case complexity: o(n!), since all possible placements may be tried.
○ optimized approaches like bitwise optimization reduce the time complexity.

(vii) implementation (programs + os + hardware)

- convert the algorithm into an actual program in a programming language.
Chapter - 1 (Analysis of Algorithms) Page 6
 - convert the algorithm into an actual program in a programming language.

(viii) testing and debugging

- run multiple test cases to ensure correctness.
example test cases for 4-queens:
○ n = 4 → should return valid board configurations.
○ n = 1, 2, 3 → should return no valid solution.
○ edge cases → check for large values of n (e.g., n = 8).

focus on design and analysis of algorithms (daa)

daa focuses on:
(i) designing efficient algorithms (choosing the best approach).
(ii) analyzing time & space complexity (how fast it runs).
(iii) proving correctness (ensuring constraints are met).

concept: types of problems

problems

decidable undecidable
a problem is decidable if there exists an algorithm a problem is undecidable if there is
that can solve it in finite time for every valid no algorithm that can solve it in
input, runs on deterministic turing machine finite time for every valid input,
proven using turing machinens and
halting problem

tractable intractable
problems that can be solved problems that can be solved but
efficiently (in polynomial time). not efficiently (in exponential
these problems belong to p time or worse).
(polynomial time) complexity these problems belong to np-hard
class (n2) or exponential complexity classes
(2n)

doubt(i): when do i say a solution with regard to algorithm is efficient and optimal?
- if it is bounded by a polynomial.

doubt(ii): what are np-hard, np-completeness problems?

Chapter - 1 (Analysis of Algorithms) Page 7

topic: need, methodology and types of analysis.
concept: need of analysis/why we need to do analysis:

reason (i) resource consumption

when designing and analyzing algorithms, it's essential to evaluate their resource consumption in terms of
time and space. this allows us to compare different algorithms, optimize performance, and choose the best
solution for a given problem.

reason (ii) perfomance comparison

when a problem has multiple solutions or algorithms (possibly using different strategies), it’s crucial to
determine which one is the best, optimal, or most efficient. this process of comparing algorithms based on
certain parameters (metric: time and space) is known as performance comparison.

determine resource consmuption,

what are resources for all algorithms?
metric: time of cpu and space of memory.
so we should analyse time and space

why?
to determine the resource consumption so that we can make perfomance comparison.
Chapter - 1 (Analysis of Algorithms) Page 8
to determine the resource consumption so that we can make perfomance comparison.

so they are inter-related.

doubt (i) why we analyse time and space complexity?

(a) time complexity measures how much time is required to run the algorithm on cpu.
(b) space complexity measures how much memory is required to store variables, data structures, etc (memory
algorithm is going to use).

doubt (ii) time and space are the only two resources that need to be analysed?
no, there are many other resources
for example:
(a) time
(b) space
(c) registers
(d) energy (power)
(e) bandwidth/channel capacity

then why we choose time and space?

there can be other resources with respect to environment but the algorithms we are going to study here are
basically for personal computing domain. so for von-neunmann architecture, time and space are important
resources.

evolution of efficiency definitions in algorithms

the definition of efficiency in algorithms has evolved over time as researchers recognized the limitations of early
definitions.

(i) proposed definition of efficiency: vague definition of efficiency (early perspective)

'an algorithm is efficienct if, when implemented, it runs quickly on real input instances'
in the beginning, efficiency was thought to be implementation-specific.

why was this definition problematic?

consider these scenarios:

Chapter - 1 (Analysis of Algorithms) Page 9

consider these scenarios:
(a) a fast algorithm on a slow machine
- even if an algorithm is fast, running it on slow hardware may result in poor performance.
- this means the algorithm’s efficiency appears low due to the machine's speed, not the algorithm itself.

(b) a better algorithm on slow hardware

- an algorithm with a better strategy may still take longer if the machine is slow.
- this makes it hard to judge whether the algorithm is truly efficient.

conclusion:
- this definition mixed algorithmic efficiency with hardware performance, making it inconsistent across
different machines.
- researchers realized that efficiency should not depend on the machine but on the algorithm itself.

(ii) proposed definition of efficiency: second definition, exploring all possibilities

'an algorithm is efficient if it achieves qualitatively better worst-case perfomance, at an analytical level, than
brute-force search'

efficiency was defined based on exploring all possible solutions. this approach focused on brute-force methods
that checked every potential answer.

why was this definition not good?

- exhaustive search is impractical:
○ checking all possibilities works for small problems, but for large inputs, it becomes computationally
expensive.
○ for example, in the traveling salesperson problem (TSP), checking all routes (n! possibilities) is infeasible
for large n.

doubt(i) what is traveling salesperson problem?

Chapter - 1 (Analysis of Algorithms) Page 10

- ignores optimization techniques:
- many efficient algorithms avoid exploring all possibilities by using smart techniques like dynamic -
programming, greedy methods, or divide-and-conquer.
conclusion:
- efficiency should not be measured by how many possibilities an algorithm explores.
- a good algorithm should be able to find a solution without brute-force searching.

(iii) proposed definition of efficiency: final stable definition of efficiency

'an algorithm is efficient if it has a polynimial running time'
the modern definition of algorithm efficiency is based on asymptotic analysis (big-O notation).

efficiency is determined by how the algorithm’s time and space requirements grow with input size n.
it should be independent of hardware and not rely on exhaustive search.

why is this the best definition?

(i) consistent across machines:
big-O notation measures efficiency based on operations performed, not CPU speed.
(ii) avoids brute-force inefficiencies:
allows us to compare algorithms that use smarter strategies.
(iii) scales with input size:
makes it clear how an algorithm performs for large inputs, which is crucial in real-world applications.

doubt (i) difference between polynomial and exponential?

basics discussed in pre-requisites of algorithms.
here, consider their rate of growth:

(a) polynomial growth:

polynomial functions grow at slower rate compared to exponential functions as input size increases.

examples: n2, n3, n4

the growth rate is controlled and manageable, meaning it increases in a predictable way as the input
size (n) grows.
for example:
- if n=10, n2=100
- if n=100, n2=10000
this is a slow increase.

(a) exponential growth:

exponential functions grow at extremely fast rate

examples: 2n, 3n
the growth rate is very rapid and unmanageable, meaning it increases in a unpredictable way for even

Chapter - 1 (Analysis of Algorithms) Page 11

the growth rate is very rapid and unmanageable, meaning it increases in a unpredictable way for even
relatively small increases in input size.
for example:
- if n=10, 210=1024
- if n=20, 220=1,048,576
this is a massive increase even for small changes in n.

visualisation of rate of growth:

n2<2n

doubt (ii) why we choose polynomial time in algorithms instead of exponential?

(i) time has a polynomial rate of growth (4n 2+8n+6) (example explained below)
(ii) algorithm time is bounded by a polynomial
it is a quadratic equation of degree 2.

time of algorithms is represented by a mathematical function with respect to input size.

in apriori analysis, we are going to derive the time by means of mathematical function with respect to input
size, which can come in form of polynomial and exponential.

our goal as the algorithm developer is develop algorithm with polynomial rate of growth (they are efficient, they
grow slow, algo runs fast) (if function runs fast like exponential then algorithm runs slowly)

- algorithms with polynomial time grow slower as input size increases. the time complexity increases at a rate
that is more predictable and manageable.
Chapter - 1 (Analysis of Algorithms) Page 12
 that is more predictable and manageable.

- polynomial time algorithms can handle larger inputs without performance suffering drastically. in
contrast, exponential time algorithms quickly become unusable as inputs grow even slightly.

- polynomial time allows an algorithm to run efficiently even for moderate-sized inputs.

- exponential time algorithms are inefficient because the time grows so rapidly that they can't solve large
problems in any reasonable time.

999,999 60,000,000 3,599,999,999 86,399,999,999 2,592,000,000,0.. 31,535,999,999.. 3,153,999,999..

1000 7,745 60,000 293,938 1,609,968 5,615,692 56,156,922

1,000,000 60,000,000 3,600,000,000 86,400,000,000 2,592,000,000,.. 31,536,000,000.. 3,153,600,000..

50,171 2,322,119 113,402,436 2,378,179,394 62,855,863,648 703,268,238,422 61,251,696,237..

1000 7,745 60,000 293,938 1,609,968 5,615,922 56,156,922

100 391 1532 4420 13,736 31,593 164,645

19 25 31 36 41 44 51

9 11 12 13 15 16 17

concept: types of analysis

how to analyse for time complexity?
method (i) a posteriori analysis (platform dependent/experimental analysis)
platform-dependent a posteriori analysis (or experimental analysis) refers to evaluating an algorithm's
performance based on real-world experiments(relative analysis) that are influenced by the specific platform or
environment where the algorithm is executed(depends on the language of compiler and type of hardware).

- in this case, the performance is analyzed after the algorithm has been implemented and tested on a specific
hardware platform, which could include factors like CPU, memory, operating system, and other system-
specific characteristics. so it is difficult to compare the efficiency of two algorithms unless experiments on
their running times have been performed in the same hardware and software environments.

- the time complexity of an algorithm using a posteriori analysis differ from system to system, if the time
taken by the program is less then the credit will go to the compiler and hardware.

- maintenance phase is required to tune the algorithm

- experiments can be done only on a limited set of test inputs and care must be taken to make sure these are
Chapter - 1 (Analysis of Algorithms) Page 13
- experiments can be done only on a limited set of test inputs and care must be taken to make sure these are
representive.

analysis:
example: x←y+z; what is the time taken by statement?
to determine the time taken by this statement i need a platform/environemt

platform

hardware factors software factors

governed by governed by
(i) cpu(speed) : the number of cycles required to execute (i) compiler optimization: compilers may re-order
an addition operations depends on the clock speed of the insructions or use cpu registers for optimization
processor
(ii) operating system: os scheduling may affection
(ii) processor architecture: may have different execution time if multiple processes are running.
instructions sets for performing addition
(iii) programming language: in c the addition
(iii) caching and memory hierarchy: if y and z is in operation is extremely fast, in python it involves
cache then operation will be faster, if they are fetched additional overhead
from ram operation will be slower.

doubt (i) why we don't prefer a posteriori analysis for algorithm efficiency? what are the advantages and
drawbacks?

Chapter - 1 (Analysis of Algorithms) Page 14

 advantages drawbacks

(i) gives the exact execution time (i) not easy to analyse across platforms
- since we are executing the algorithm in a real time - execution time is influenced by hardware
environment, we get the actual time values in - makes it difficult to derive general conclusions
seconds, miliseconds or nano seconds
(ii) not applicable for all input classes.
(ii) helps in real-world perfomance tuning - a posteriori analysis requires us to run the
- useful for optimizing software for a particular algorithm on specific test cases.
machine or setup. - but algorithms may behave differently for different
- identifies hardware bottlenecks. inputs
- we cannot test all possible inputs.

(iii) non-uniformity (platform dependence)

- execution time varies from system to system, it
makes cross platform perfomance comparison
difficult.

(iv) requires actual implementation and testing

- we must first implement the algorithm in a
programming language
- this is time-consuming, cannot compare algorithms
quickly.

method (ii) a priori analysis/platform independent analysis

a priori analysis (also called theoretical analysis or platform-independent analysis) is a method of
evaluating an algorithm’s efficiency before actually implementing it.

instead of running the algorithm on a specific system, we analyze its performance mathematically,
independent of hardware or software constraints. so it allows us to evaluate the efficiency of two algorithms
in a way that is independent from the hardware and software environment.

- the time complexity of an algorithm using a priori analysis is same for every system, if the algorithm is
running faster the credit goes to programmer

- it is done before execution of an algorithm (a priori)

- maintenance phase is not required to tune the algorithm

this method helps us understand the growth rate of time and space complexity based on the input size n,
using asymptotic notation (e.g., big-o notation).

methodology for apriori analysis:

- based on the framework/platform (fictitious/not real platform)
- analyse the pseudo code.
Chapter - 1 (Analysis of Algorithms) Page 15
 - analyse the pseudo code.

what are the components of framework on which we are going to analyse?

(i) step-count method
(ii) order of magnitude

concept: methodology of analysis

components of a priori/asymptotic a priori analysis framework:
(a) a language for describing algorithms:
algorithms must be described in precise and unambiguous way.
we need formal notations (english is a vague language)
- pseudocode.

(b) a computational model that algorithms execute within (RAM model).

we need a standarized model to evaluate an algorithm's efficiency.

(c) a metric for measuring algorithm running time.

algorithm time depends on some metric
(i) searching: 'comparison' is the metric where the time depends
(ii) sorting: 'swap and comparison' is the metric where the time depends.

- actual execution time varies across platforms due to hardware differences (cpu speed, memory, compilers,
etc.)
- to ensure a platform-independent analysis, we measure fundamental operations instead of real-time
execution.
- a mathematical performance measure helps predict efficiency before implementation (a priori analysis).

common metrics used:

(i) input size (n): how the algorithm’s execution time scales with nn.
(ii) number of operations: counting steps based on loop iterations, function calls, and comparisons.
(iii) asymptotic analysis: big-O, big-θ, and big-Ω notations to describe growth rate.
- big-O (O): worst-case complexity.
- big-theta (Θ\theta): average-case complexity.
- big-omega (Ω\omega): best-case complexity.

(d) an approach for characterizing running times, including recursive algorithms.

algorithms behave differently for different inputs (best-case, worst-case, average-case), recursive
algorithms need a different approach using recurrence relations.

concept: RAM model

Chapter - 1 (Analysis of Algorithms) Page 16

additonal doubt (figuring out the cormen text)
(i) assumption about integer representation and why we restrict word size?

Chapter - 1 (Analysis of Algorithms) Page 17

(a) why we need ⌊log2n⌋+1 bits?
- an array of size n, we need to index elements from 0 to n-1.
- to represent a number upto n, we need atleast ⌊log2n⌋+1 bits.

(b) why do we assume a limit (upper bound) on word size?

- if the word size was arbitrarily larger, we could store huge amounts of data in one word and perform
operations unrealistically fast.
- for example, if a single word could store an entire array, we could sort it in O(1) time, breaking the
RAM model’s intent.
- if word size grew arbitrarily, we could store entire datasets in one word, making operations
artificially O(1) when they shouldn't be.

(c) why choose 'c log n' as the word size?

- to balance flexibility and realism, we assume a word size of 'c log 2n' bits, where c≥1
- this means we can index up to nc elements.
- keeping c as a constant prevents situations where a word can store huge data structures, making
operations unrealistically fast.

- example: if c=2, then the word size is '2 log2n' bits, allowing us to index up to n2 elements.

this keeps the RAM model realistic while still allowing efficient computation.

(ii) real computers vs the ram model

- the ram model simplifies computations by assuming basic operations take constant time.

- in real-world computers, some complex instructions exist (e.g., exponentiation), which take more than
O(1).

- this creates gray areas where an operation might be assumed to be constant time but actually isn't.

(iii) is exponential a constant-time operation?

Chapter - 1 (Analysis of Algorithms) Page 18

exponentiation (xn) is usually not O(1) because:

(i) it involves multiple multiplications.

- example: computing 210 requires about O(log n) multiplications.

(ii) the result may not fit in a word.

- 2100 is a huge number that won’t fit in a 32-bit or 64-bit word.

however, there are special cases where exponentiation can be O(1), such as when n is an exact power of 2.

(iv) shift-left operation and constant time

- bitwise left shift (<<) moves bits left, multiplying the number by 2n.

- since shift operations work on a fixed word size, they usually take constant time O(1).

- as long as n is within word size, this runs in O(1).

- but if n is too large, overflow occurs, and extra operations may be needed.

(v) avoiding gray areas in the RAM model

- in theoretical analysis, we assume operations like computing 2n using bit shifting are O(1).

- but in real-world hardware, if the result is too large to fit in one word, extra computations occur, making
the time complexity non-O(1).

definition of RAM model:

- the ram model is a theoretical computation model used in a priori analysis to analyze an algorithm’s
efficiency.
- it provides a standardized framework to measure running time independent of any hardware or
platform.
- it assumes a single processor system with a simple set of rules.

characteristics of the ram model

Chapter - 1 (Analysis of Algorithms) Page 19
characteristics of the ram model
(i) every fundamental operation takes constant time (O(1))
- arithmetic operations
in the RAMmodel, the following arithmetic operations are assumed to take constant time O(1):
• addition (+)
• subtraction (-)
• multiplication (×)
• division (÷)
• modulo (%)

- comparison/relational operations:
in the RAM model, comparison and relational operations are assumed to take constant timeO(1).
these operations include:
• greater than ( > )
• less than ( < )
• greater than or equal to ( ≥ )
• less than or equal to ( ≤ )
• equal to ( = )

- logical operations:
in the RAM(random access machine) model, the following logical operations are assumed to take
constant time O(1):
• logical AND (∧ or &&)
• logical OR (∨ or ||)
• logical NOT (¬ or !)
• bitwise AND (&)
• bitwise OR (|)
• bitwise XOR (^)
• bitwise NOT (~)
• bitwise shift left (<<) (when shifting within word size)
• bitwise shift right (>>) (when shifting within word size)

- transfer operations:
in the random access machine (RAM) model, the following data transfer operations are assumed to take
constant time O(1):
(a) loading and storing operations
• reading a value from memory:
example: x = A[i] (fetching an element from an array).
• storing a value into memory:
example: A[i] = x (writing an element into an array).

(b) register operations

• copying values between registers:
example: R1 = R2 (assigning a value from one register to another).
• loading a constant into a register:
example: R1 = 5.

Chapter - 1 (Analysis of Algorithms) Page 20

(c) pointer dereferencing
• accessing memory using a pointer:
example: x = *ptr (dereferencing a pointer to get a value).
• storing a value at a pointer location:
example: *ptr = x.

(d) simple stack operations

• pushing and popping values from a stack:
example: push(x), y = pop().

- i/o operations:
in the RAM model, the following input/output (i/o) operations are assumed to take constant time o(1):
(a) reading input
• reading a single integer, float, character, or word from standard input:
example: x = input() (reading a value from the user).

(b) writing output

• printing a single integer, float, character, or word to standard output:
example: print(x) (displaying a value on the screen).

(c) file read/write (simplified model)

• reading a single word or character from a file.
• writing a single word or character to a file.

assumption
- in reality, i/o operations depend on hardware and are usually much slower than memory operations.
- in the ram model, we assume these operations take 0(1) for theoretical simplicity in analyzing
algorithms.

executes one by one on hypothetical machine

memory cpu
(assume memory is large enough to
accomodate any algorithm)
find out what is
all algorithmic total time in units
steps are stored
here, all operations
Chapter - 1 (Analysis of Algorithms) Page 21
here, all operations
of algorithms are
stored in memory

requirements for a apriori analysis methodology:

this methodology aims at associating with each algorithm a function f(n) that characterizes the
running time of the algorithm in terms of the input size n.

typical functions that will be encountered include n and n2.

for example, we will write statements of the type "algorithm A runs in time proportional to n," meaning
that if we were to perform experiments, we would find that the actual running time of algorithm A on
any input of size n never exceeds cn, where c is a constant that depends on the
hardware and software environment used in the experiment.

given two algorithms A and B, where A runs in time proportional to n and B runs in time proportional to
n2, we will prefer A to B, since the function n grows at a smaller rate than the function n2.

compare rate of growth:

A: log2n and B: n

method(i)
- take values of n=32
log2n n
log232 32
5 < 5.6

method(ii) generic (better method)

log2n n
take log of both
log2 log2n log2 n
log2 log2n log2(n)1/2
log2 log2n < 1/2log2n

applying log twice

calculating time complexity:

concept: step-count method

algorithm test:
{
x←y+z;

for i←1 to n;
Chapter - 1 (Analysis of Algorithms) Page 22
 for i←1 to n;
a←b*c;

for i←1 to n;
for j←1 to n;
k=k*5;
}

executing on computational model:

- every fundamental operation be of whatever category will take one unit of time/constant time.

(i) x←y+z;
+(addition): takes one unit
← (storing into x): takes one unit
total: 2 units

(ii) for i←1 to n;

a←b*c;

i←1 to n;
- first we initliase i=1; takes one unit
- then we give it condition, comparing i with n (n times) + 1 (when the condition fails and loop terminates);
takes n+1 units
- loop increments the value of i until loop does is satisfying the condition so i++; takes n units
(1+(n+1)+n)

a←b*c;
- this statement is within the loop means it runs for n times not n+1 times because on the last
condition(when it fails) it will not enter but terminates the loop; takes n units
total: 1+n+(n+1)+n+n

(iii) for i←1 to n;

for j←1 to n;
k=k*5;

i←1 to n;
- first we initliase i=1; takes one unit
- then we give it condition, comparing i with n (n times) + 1 (when the condition fails and loop terminates);
takes n+1 units
- loop increments the value of i until loop does is satisfying the condition so i++; takes n units
(1+(n+1)+n)

for j←1 to n;
- first we initliase j=1; takes one unit
- then we give it condition, comparing i with n (n times) + 1 (when the condition fails and loop terminates);
takes n+1 units
- loop increments the value of j until loop does is satisfying the condition so j++; takes n units

j runs for n times

Chapter - 1 (Analysis of Algorithms) Page 23
j runs for n times
n(1+(n+1)+n)
n+n(n+1)+n.n

k=k*5;
- this statement is within the loop of j and j is within the loop of i that means
- k*5; runs for 2n times; takes n.n units
- k=k*5; runs for 2n times; takes n.n units

final:
1+(n+1)+n+n+n(n+1)+n.n+n.n+n.n
1+n+1+n+n+n2+n+n2+n2+n2
4n2+4n+2

total time:
2+(4n+2)+4n2+4n+2'
4n2+8n+6

n is input size.
as you increase the value, time increases.

in coding:
x = y + z;

for (i = 1; i <= n; i++) {

a = b * c; // executes n time
}

for (i = 1; i <= n; i++) {

for (j = 1; j <= n; j++) {
k = k * 5; // executes n * n times
}
}

concept: order of magnitude

- refers to the approximate size or scale of a function.
- focuses on the dominant term while ignoring constants and lower-order terms.
- often used in scientific notation to express values that differ by powers of 10.
example:
- f(n) = 3n² + 5n + 100 → order of magnitude is n²
because n² dominates for large n.

algorithm test:
{
x←y+z;

Chapter - 1 (Analysis of Algorithms) Page 24

 for i←1 to n;
a←b*c;

for i←1 to n;
for j←1 to n;
k=k*5;
}

executing on computational model:

- each statement/step/data structure operation will have order of magnitude 0(n)
order of magnitude refers to frequency (number of times)(highest number count) of the
fundamental operation/metric invloved in that statement/step/operation.

(a) what does "order of magnitude" mean?

- how an algorithm's running time or operation count grows as the input size n increases.
- in asymptotic analysis, we focus on the dominant term because it dictates the behavior of the
algorithm for large n.

(b) what does "highest number count" mean?

- "highest number count" means the most frequently executed operation in a given computational
model/the maximum number of times an operation is performed in a program.
- if multiple operations occur in a step, the one that happens the most times is the dominant
operation and determines the overall complexity.

• this is the dominant term in asymptotic analysis.

• it defines the order of magnitude (big-o complexity).
• we ignore lower-order terms and constants when expressing time complexity.

x←y+z;
+(addition)
← (storing into x)
both have same frequency '1'

for i←1 to n;
a←b*c;
loop runs for n+1 times but the operations will execute for n times
so,
*(multipication)
← (storing into a)
both have same frequency 'n'
highest frequency is 'n'

for i←1 to n;
for j←1 to n;
k=k*5;
loop runs for n(n+1) times but the operations will execute for n 2 times
so,
*(multipication)
Chapter - 1 (Analysis of Algorithms) Page 25
*(multipication)
← (storing into a)
both have same frequency 'n 2'
highest frequency is 'n2'

final: n2+n+1

step count order of magnitude

4n2+8n+6 n2+n+1
structure is same

topic: asymptotic notation

(i) algorithm withconcept 3 steps:

time: 4n2+8n+6

(ii) algorithm with 1000 steps:

time: .....1000?
no.

instead use asymptotic notation.

- why use asymptotic notation?

asymptotic notation helps us express the efficiency of an algorithm in a simple and generalized way,
ignoring constant factors and lower-order terms. instead of writing an exact formula, we focus on the
dominant term that dictates growth.

characterize the time in elegant way, by limiting the bounds of the function.

- ignores constants and lower-order terms

allows us to focus on the dominant term that affects performance as n increases.
- helps compare algorithms
tells us which algorithm is more efficient for large n.
- provides a general performance trend
rather than exact execution time, it shows how an algorithm scales.

(i) algorithm with 3 steps:

time complexity: 4n2+8n+6
- the highest order term is n².
- we ignore constants and lower-order terms.
Chapter - 1 (Analysis of Algorithms) Page 26
 - we ignore constants and lower-order terms.
- simplified complexity:
O(n²).

4n2+8n+6 =0(4n2) = 0(n2)

(a) 0(4n2)
order of functiom is that term in the function which is having highest degree or highest value.

(b) 0(n2)
what does it mean?
it means that the algorithm may have one step, 10 step, hundred steps but there is no step whose time is more
than n2.

all the step may have n2 or less than n2

why not just write 4?

- constants (like 4) do not define how an algorithm scales.
- we care about growth rate, not exact execution time.
- asymptotic notation allows us to compare algorithms fairly.

general rule
- drop constants → ignore coefficients (e.g., 4n² → n²).
- keep the dominant term → ignore lower-order terms (e.g., 8n and 6 are removed).
- result: big-0 notation → focus on worst-case growth.

doubt(i) : why we removed 8n+6?

we ignore constants and lower-order terms. here's why:
hypothetical example:

seller business grows seller business grows seller business grows seller business grows seller business grows seller business grows
and sells item by and sells item by bike and sells item by car and sells item by and sells item by and sells item by
cycle truck helicopter airplane
cost: 2lakh cost: 25lakh
cost: 5k cost: 50lakh cost: 50crore cost: 400crore

he maintained everything in villa, if somebody asked him how much you have invested 'roughly' in
mode of transport?
400crore, as rest of the money becomes insignificant as compared to 400crore.

our example:
- time complexity: n2+n+6
n2>n>6
- rate of growth: n and 6 is much lesser and insignificant than n2 so, the other terms become
insignificant.
n2 having better growth than n.

Chapter - 1 (Analysis of Algorithms) Page 27

the rate of growth of running time, or order of growth, is an approximation of how long a
computer program will take to run as the input size increases. it focuses on operations that
increase propotionally to the input size, while ignoring the lower order terms and the
constant factor needed for fixed operations.

the rate of growth of running time can be described using different notations, including
big-theta and big-0 notation (asn)

time is represented by a mathematical function like n 2+n+1

whereas, asymptotic notation is a mathematical tool to represent/determine the bounds of the
functions.

therefore we extend asymptotic notation in analysis of algorithms to characterize the time or to get
the bounds of the functions representing the time

maximum bound: max time a algorithm may take

minimum bound: min time a algorithm may take.

doubt (i) : n2+n2+n2+.....+n2 (n times) becomes n3?

no, it is still 0(n 2) because algorithm cannot be of 'n' steps, always 'finite' steps.

there are two aspects of apriori analysis:

apriori analysis

determine the rate of growth of time determine the bhevaiour of the

with respect to the input size algorithm function with respect to a
fixed input size (say n)
type of analysis

Chapter - 1 (Analysis of Algorithms) Page 28

 type of analysis

concept: understanding the best-case, worst-case and average-case

input: <a1, a2, a3,........,an> (fixed input size)

different arrangements of input are called input classes.

<a1, a2, a3,........,an>

instances: i1: inc i2: dec i3: random ik: ...

type of analysis: determining the behavious of algorithm for different input classes

point to remember: why best-case, worst-case and average-case?

there is a possibility that for each input classes the algorithm may take different time, we
analyze best-case, worst-case, and average-case complexities to cover all secnarios.

(i) best-case: the input class for which algorithm does least amount of work and takes minimum time.

- the best-case scenario depends on how the input is structured in a way that makes the algorithm work
the fastest.

- it is useful for analyzing optimal performance but does not give a complete picture of efficiency.

example: linear search

- if the target element is at the first position, the search completes in O(1) time.
- this is the best case because the search stops immediately.

(ii) worst-case: the input class for which the algorithm does the most amount of work and takes maximum
time.

- the worst-case scenario helps us understand the upper bound on running time, ensuring the algorithm
never takes longer than this.

- it is useful for performance guarantees and robustness.

example: linear search

- if the target element is at the last position or not in the array at all, the algorithm must check every
element.
- this results in O(n) time complexity.

Chapter - 1 (Analysis of Algorithms) Page 29

point to remember:
no average case in our syllabus

(iii) average-case: the expected running time of an algorithm over all possible inputs, assuming each input
of size n occurs with equal probability.

- the average-case complexity gives a realistic expectation of performance.

- it requires probability analysis to compute the expected number of steps.

example: linear search

• assuming the target is equally likely to be at any position, we expect to search about half the array
before finding the target.
• this results in an expected time complexity of Θ(n/2) = Θ(n)

example:
<a1, a2, a3,........,an>

instances: i1: inc i2: dec i3: random ik: ...

t1 t2 t3 tk
p1 p2 p3 pk

(i)list out all the input k classes

(ii) for each input class we have to determine the time time taken t 1, t2, t3, tk
(iii) associated with each input class we are going to have a funciton probability p 1 representing the
probability with which the algorithm take input from i 2 class.

what is the probabilty with which algorithm take input from i 2 class?

average time:
probability

number of classes

time

finding relationship between these cases

let's assume the following notations:

b(n): best case
Chapter - 1 (Analysis of Algorithms) Page 30
b(n): best case
w(n): worst case
a(n): average case

in general: for most algorithms, we generally have the following relationship

b(n) ≤ a(n) ≤ w(n)
- best-case (b(n)) is always the minimum possible time.
- worst-case (w(n)) is always the maximum possible time.
- average-case (a(n)) usually falls between the best and worst cases.

(will study them later in detail)

(i) merge sort:
- merge sort always splits the input into halves and merges them back.
- regardless of the input, the recursive structure is the same.
b(n)=a(n)=w(n)
0(nlogn)=0(nlogn)=0(nlogn)

(ii) quick sort:

- best-case: O(nlog n) (when pivot always divides array into equal halves).
- average-case: O(nlog n) (expected case with random pivots).
- worst-case: O(n2)(when pivot is always the smallest or largest element, creating unbalanced partitions).
[b(n)=a(n)]<w(n)
[0(nlogn)=0(nlogn)]<0(n2)

(iii) linear search :

- best-case: O(1) (when the element is found at the first position).
- average-case: O(n) (when the element is randomly placed in the list).
- worst-case: O(n) (when the element is at the last position or not found).
(since average and worst cases both require scanning most of the array).
b(n)<[a(n)=w(n)]
0(1)<[0(n)=0(n)]

Chapter - 1 (Analysis of Algorithms) Page 31

topic: types of asymptotic notations
asymptotic notation is used to describe the growth rate of an algorithm's time or space complexity.

the three main types are:

(i) big-0 notation ( O ): upper bound (worst-case complexity)

- represents the worst-case scenario (maximum time an algorithm takes).
- provides an upper bound on the growth rate.
- useful for ensuring an algorithm won't perform worse than a certain limit.

interpretation: "this algorithm will take at most this much time."

(ii) omega notation ( Ω ): lower bound (best-case complexity)

- represents the best-case scenario (minimum time an algorithm takes).
- provides a lower bound on the growth rate.
- useful for understanding the fastest an algorithm can possibly run.

interpretation: "this algorithm will take at least this much time."

(iii) theta notation ( Θ ): tight bound (average-case complexity)

- represents the average-case scenario.
- provides a tight bound, meaning the algorithm always runs within a specific range.
- useful for understanding an algorithm’s expected performance.

interpretation: "this algorithm will always take this much time, no more and no less."

apart from big-o, big-omega, and theta, there are two more notations:

(iv) small-o notation (o): strict upper bound

- describes an upper bound that is not tight.
- means the function grows strictly slower than the reference function.

interpretation: "this function grows slower than f(n), but it is not equal to it."

(v) small-omega notation(ω): strict lower bound

- describes a lower bound that is not tight.
- means the function grows strictly faster than the reference function.

interpretation: "this function grows faster than f(n), but it is not equal to it."

Chapter - 1 (Analysis of Algorithms) Page 32

why are we using it in algorithms?
in apriori analysis we are representing the time complexity by the means of a mathematical
function, a mathematical function may have 100 terms and it may looks clumsy

so we characterize it by means of
- upper bound: time of algorithm cannot be more than this/his algorithm will take at most this
much time.
- lower bound: time of algorithm cannot be lesser than this/this algorithm will take at least this
much time.

let's understand upper bound and lower bound hypothetically:

this is upper
13 bound for
12 person 'A'
11
10 person 'A' standing on this floor
9
this is lower
8
bound for
person 'A'

multifloor
building

floor values
(i) >10: upper bound, i can have many upper bounds
- closest upper bound: 11

(ii) <10: lower bound, i can have many lower bounds

- closest lower bound: 9

let's try to understand upper bound and lower bound in more depth through hypothetical scenario
scenario: estimating the construction cost
a person wants to build a house and needs to estimate the construction cost. however, the exact cost
can only be known after the construction is complete.
- the budget is around 50 lakhs.
- the person asks two builders to provide estimates:
○ upper bound estimate (maximum possible cost)
○ lower bound estimate (minimum possible cost)

(i) upper bound (big-o analogy)

- upper bound means the maximum possible cost.
- this tells us that the cost will never exceed this value.
Chapter - 1 (Analysis of Algorithms) Page 33
 - this tells us that the cost will never exceed this value.

(a) builder 'a' gives an upper bound estimate: "< 1 crore"

○ this means the maximum cost required to construct a house is at most 1 crore.
○ this is a valid upper bound but not useful because it is too large.

(b) builder 'b' gives a better upper bound estimate: "< 48 lakh"
○ this means the maximum cost required to construct a house is at most 48 lakh.
○ this is a tighter (better) upper bound because it is closer to our budget (50 lakh).

big-0 takeaway:
- big-0 notation represents the upper bound of a function's growth.
- we always try to find the closest possible valid upper bound.

(ii) lower bound (big-omega analogy)

- lower bound means the minimum possible cost.
- this tells us that the cost will never go below this value.

(a) builder 'a' gives a lower bound estimate: "> 10,000"

○ this means the cost will not be lower than 10,000.
○ this is a valid lower bound but not useful because it is too low.

(b) builder 'b' gives a better lower bound estimate: "> 48 lakh"
○ this means the cost will not be lower than 48 lakh.
○ this is a tighter (better) lower bound because it is closer to our budget (50 lakh).

big-omega takeaway:
- big-omega notation represents the lower bound of a function's growth.
- we always try to find the largest possible valid lower bound.

definition: let f(n) and g(n) be the functions from the set of integers/reals to the set of real numbers.
concept: big-0h(0), upper bound
(i) understand big-oh notation:
- it tells us how fast a function grows as the input n gets bigger.
- we ignore small details and focus only on big values of nn.

example:
- if someone says O(n²), it means the function doesn’t grow faster than n² for large n.
- if someone says O(n), it means the function grows at most as fast as n.

Chapter - 1 (Analysis of Algorithms) Page 34

definition of Oh:
a function f(n) is said to be O(g(n)) if there exist positive constants c and k(n 0)
such that:
f(n)≤c⋅ g(n) for all n ≥ k (c,k>0)

analysis of the text:

(i) existence of constants:
- we must find a constant c > 0 and a threshold k > 0 such that for all n≥k , the inequality
holds.
- c is a scaling factor that allows us to compare f(n) and g(n).
- k ensures that we only consider large values of n, ignoring small fluctuations.

(ii) f(n) grows at most as fast as g(n):

- the function f(n) should never exceed c⋅ g(n) for large n.
- this means f(n) is either smaller than or grows at the same rate as g(n), but never faster.

(iii) this must hold for large values of n (asymptotic behavior):

- the definition is valid only when n is sufficiently large (i.e., n≥k).
- for small values of n, the behavior of f(n) doesn’t matter.

breaking it down:
(i) f(n) is said to be 0(g(n))

then we have to determine

another function
if i want to find the
upper bound of the
function

(ii) if there exists a constant c>0 such that when you multiply g(n) by c, the function f(n) is always
less than or equal to c⋅ g(n) for all sufficiently large n (i.e., for all n≥k), then we can say that f(n) is
O(g(n)).

f(n)≤c⋅ g(n) for all n ≥ k (c,k>0)

Chapter - 1 (Analysis of Algorithms) Page 35

 f(n)≤c⋅ g(n) for all n ≥ k (c,k>0)

when you multiply g(n) by c

if there exists a constant c>0

f(n)≤c⋅ g(n) for all n ≥ k (c,k>0)

then the function f(n) is always

less than or equal to c.g(n)

f(n)≤c⋅ g(n) for all n ≥ k (c,k>0)

for all sufficiently large 'n' (n≥k)

then we can say that f(n) is O(g(n)).

doubt(i) what do we mean by set here?

in this context, the "set" refers to a collection or group of functions that satisfy a particular
condition.

specifically, the set O(g(n)) is defined as:

O(g(n))={f(n):there exist positive constants c and n0(k)

such that
0≤ f(n) ≤ c.g(n) for all n ≥ n0(k)}

this means that the set O(g(n)) contains all functions f(n) that, for sufficiently large (greater
than or equal to n0(k)), grow no faster than g(n) multiplied by a constant c.

why call it a "set"?

- in mathematics, a set is simply a well-defined collection of objects (in this case, functions).

- O(g(n)) groups all functions that satisfy the given inequality condition with respect to g(n).

- it helps formalize the concept of "asymptotic upper bounds" in computational complexity. by

saying f(n)∈O(g(n)), we are declaring that f(n) belongs to this collection of functions that

Chapter - 1 (Analysis of Algorithms) Page 36

 saying f(n)∈O(g(n)), we are declaring that f(n) belongs to this collection of functions that
grow at most as fast as g(n) when n becomes large.

set of functions f(n) that satisfies the inequality

f.. f f(n) ≤ c.g(n) for all n ≥ n0(k)}
5
f1
f3
f.. f2
f.. f..
f4
f.. f..

set 0(g(n))

set of functions f(n) that satisfies 0(n 2)

f(n) ≤ c.g(n) for all n ≥ n0(k)}
3lgn+8 4n2 ≤ n2
5n+7 6n2+9

2nlgn 5n2+2n

set 0(n2)

(additional doubts from cormen; some confusing terms for me)

doubt(i) what is asymptotically nonnegative?

- a function f(n) is asymptotically nonnegative if it is nonnegative (≥ 0) for sufficiently large
n.

in simpler terms:
○ for small values of n, f(n) might be negative.
○ for large values of n, f(n) must be ≥0.

example:
f(n)=−100+n2

- for small n, the function is negative (e.g., at n=5, f(5)=−100+25=−75)

for large n, the function becomes positive (e.g., at n=11, f(11)=−100+121=21).

Chapter - 1 (Analysis of Algorithms) Page 37

 - for large n, the function becomes positive (e.g., at n=11, f(11)=−100+121=21).

since f(n) eventually stays nonnegative for all large n, it is asymptotically nonnegative.

doubt (ii): why must f(n) in O(g(n)) be asymptotically nonnegative?

big-o notation describes an upper bound on function growth.

- we write f(n)=O(g(n)) when f(n) grows at most as fast as g(n),

meaning: f(n)≤c⋅ g(n)for large n

- if f(n) were asymptotically negative, this would not make sense, because:
○ big-o is used to compare growth rates (negative growth would not fit the concept).
○ in complexity analysis, we talk about time or space complexity, which are always nonnegative.

doubt(iii): why must g(n) also be asymptotically nonnegative?

- the function g(n) represents the upper bound, meaning it must grow nonnegatively.

- if g(n) were negative, then O(g(n)) would not make sense, because:
○ no function f(n) can be bounded above by a negative function for large n.
○ the set O(g(n)) would be empty (no valid functions would fit).

Chapter - 1 (Analysis of Algorithms) Page 38

graphical understanding of the function:

beyond k, the rate of

growth of c.g(n) is
like this (function are
converging)

rate of the growth of the

function f(n)

(less than c.g(n))

between 0 to k there is no well

defined relationship between f and
g, sometimes f is more and
sometimes g.

functions are not converging 0

between 0 to k (k)
value of n is increasing
some value k f(n)≤c⋅ g(n) for all n ≥ k (c,k>0)

let's take an example and compare the graphs to get a better understanding:

Chapter - 1 (Analysis of Algorithms) Page 39

let's take an example and compare the graphs to get a better understanding:
f(n)=1+n+n2
0(n2)

let's check
1<n2
1+n<n2+n2
1+n+n2<n2+n2+n2
1+n+n2<3n2, n>1

point to remember:
smaller functions are in the order of bigger ones in big-0 notation because big-0 represents an upper
bound

analysis:
we have the function: f(n) = 1 + n + n²

now, the goal is to prove that this function grows at most as fast as n², and we are doing this using
big-Oh notation. the idea is to show that for sufficiently large n, f(n) is bounded above by some
constant multiple of n².

(i) big-o definition :

a function f(n) is said to be O(g(n)) if there exist positive constants c and k(n 0)such that:
f(n)≤c⋅ g(n) for all n ≥ k (c,k>0)

(ii) the comparison step:

- start with the function:
f(n) = 1 + n + n²

(iii) compare each term with n²:

(a) the first term is 1. we know that for large n, 1 is much smaller than n², so we can compare it to
n².

(b) the second term is n. again, for large n, n is smaller than n², so we can compare it to n².

(iv) write the inequality:

to prove that f(n) ≤ 3n², '
we compare each term in f(n) to n²:
f(n) = 1 + n + n²

we can bound the first two terms (1 and n) by n².

specifically:
1 ≤ n² (for sufficiently large n)
n ≤ n² (for sufficiently large n)

now compare:
1 + n + n² ≤ n² + n² + n² = 3n²
Chapter - 1 (Analysis of Algorithms) Page 40
 1 + n + n² ≤ n² + n² + n² = 3n²

this shows that for sufficiently large n, f(n) is always less than or equal to 3n².

conclusion:
• we are not comparing each term directly with n², but rather, we're bounding the smaller terms by n²
and showing that the entire function f(n) is less than or equal to a constant multiple of n².

• we chose c = 3 to show that for sufficiently large n, f(n) is bounded above by 3n².
thus, we proved that f(n) = o(n²).

why use this comparison?

• the reasoning here is to simplify the function and show that, beyond a certain point (i.e., for large
n), all terms behave similarly to n² in terms of their growth.
• this approach is common in big-o analysis because we’re focusing on the asymptotic behavior for
large n, not the exact details of the function for small n.

1+n+n2 <3n2 ,n>1

c g
k
f(n)

- f(n) is g(n)
f(n) is 0(n2)

Chapter - 1 (Analysis of Algorithms) Page 41

 f(n)≤c⋅ g(n) for all n ≥ k (c,k>0)

1+n+n2<3n2 ,n>1

this region is where

1+n+n2<3n2

look closely, when n

becomes greater than 1,
1+n+n2<3n2

how many upper bounds possible?

1+n+n2<3n2

f(n)=1+n+n2 ≤ c⋅ n2, n>1 closest upper bound 0(n2)

f(n)=1+n+n2 ≤ c⋅ n3, n>1

f(n)=1+n+n2 ≤ c⋅ n4, n>1

.
.
f(n)=1+n+n2 ≤ c⋅ nk, n>1

doubt(i) why 0(n2) is upper bound?

- big-o notation does not require the function to be smaller at all times. it just needs to be bounded for
sufficiently large n.

- for small values of n, yes, 1+n+n2 might be larger than n2, but we ignore small n in asymptotic
analysis.

Chapter - 1 (Analysis of Algorithms) Page 42

 analysis.

- the key is that 1+n+n2 grows at the same rate as n2 for large n, meaning it can be bounded by some
constant multiple of n2.

- 0(n2) is the closest and tightest upper bound (asymptotically closest).

this is why we chose 0(n2) as upper bound.

doubt(ii) why 0(n3) is not chosen as upper bound?

- 0(n3) is upper bound but not closest/tightest, its too loose. f(n) does not grow as fast as n3.

- we always prefer tightest upper bound, which in our case is 0(n2)

let's take some examples:

example(i)
we are given:
f(n) = n
and we need to check whether it belongs to different big-o notations.

(a) f(n)= n ≤ c1⋅ n, n ≥ 1 = 0(n)

point to remember:
smaller functions are in the order of bigger ones in big-0 notation because big-0 represents an upper
bound

(b) f(n)= n ≤ c2⋅ n2, n ≥ 1 = 0(n2)

since, n2 grows faster than n, 0(n2) is an upper bound, but it is a loose upper bound.
- we prefer the tightest bound, which is O(n), but technically O(n2) is valid.

(c) f(n)= n ≤ c3⋅ logn, n ≥ 1 ≠ 0(logn)

- for large n, n grows much faster than logn.
- no matter what constant c3 we choose, there will always be a point where: n>c3.logn for all large n,
violating the definition of big-o notation.

example (ii)
we are given:
f(n)=n+logn

step (1): the definition of big-o notation

a function f(n) is in O(g(n)) if there exist constants c>0 and k>0

such that:
f(n)≤c⋅ g(n)for all n≥k

step (2): compare f(n)=n+logn with g(n)

f(n)≤c⋅ g(n)for all n≥k
n+logn ≤ n+n, n>1
Chapter - 1 (Analysis of Algorithms) Page 43
 n+logn ≤ n+n, n>1
n+logn ≤ 2n, n>1
n+logn ≤ 2n, n>1 = 0(n)

- in n+logn, the term n grows much faster than logn, especially for large n, so logn becomes negligible
as compared to n.
n ≤ 2n, n>1

- smaller functions are in the order of bigger ones in big-0 notation because big-0 represents an upper
bound
n ≤ 2n, n>1 = 0(n)

f(n): n+logn
c: 2
g(n): n
0(n)

- for large n, log n grows much slower than n.

- this means that n+logn behaves very similarly to n for large n.

doubt(i) why does O(n) work?

as n→∞, log n becomes insignificant compared to n.

for example:
○ if n=106, then logn≈20, which is tiny compared to 10 6.
○ this means that n+logn is almost the same as n for large n.
thus, n+logn is asymptotically bounded by O(n).

Chapter - 1 (Analysis of Algorithms) Page 44

example(iii)
we are given:
f(n)=2100(c)=0(1)

why is f(n)=O(1)?
- f(n)=2100 is a constant value (a fixed number).
Chapter - 1 (Analysis of Algorithms) Page 45
- f(n)=2100 is a constant value (a fixed number).
- it does not depend on n or grow as n increases.
- in big-o notation, a constant function is always considered O(1).

formal proof using big-o definition

a function f(n) is in O(g(n)) if there exist constants c>0 and k>0 such that:
f(n)≤c⋅ g(n)for all n≥k

applying big-Oh to f(n)=2100 and g(n)=1:

- here, we are comparing f(n)=2 100(a constant) with g(n)=1(another constant).

- we want to find constants c and k

such that:
f(n)≤c⋅ g(n)
which simplifies to:
2100 ≤ 3100⋅ 1
2100 ≤ c⋅ 1

intuitive explanation
- imagine a machine that always takes the same amount of time to complete a task, no matter the input
size.
- whether n=1 or n=106, the time taken is always the same: 2 100.
- this means the function has constant time complexity, i.e., O(1).

point to remember:
when i say f ≤ c.g then f is 0(g)

bigger functions
smaller functions
are in the order of

example (iv)
f(n)=n2(polynomial) and g(n)=2n(exponential)
so, prove f(n) is 0(g(n)), whenever n>k
- is f order of g (f 0(g)) or g order of f (g 0(f))?

(i) polynomial is a smaller function as compared to exponential so as we know smaller function are in
the order of bigger function.

so, f(n) is 0(g(n)), whenever n>k

but, not always.

n f(n2) g(2n)
1 1 2
Chapter - 1 (Analysis of Algorithms) Page 46
 n f(n2) g(2n)
1 1 2
2 4 4 relationship before 4 is
unpredictable
3 9 8
(non-converging)
4 16 16
5 25 32
6 36 64

f(n) ≤ c⋅ g(n), n ≥ k
1 ≤ 2, n ≥ 1
4 ≤ 4, n ≥ 2
9 ≤ 8, n ≥ 3
16 ≤ 16, n ≥ 4
25 ≤ 32, n ≥ 5 so, f(n) is 0(g(n)), whenever n>4
32 ≤ 64, n ≥ 6

graphical visualisation:

64

32
32

16 25
9 16
2 4
8
4
1

Chapter - 1 (Analysis of Algorithms) Page 47

concept: big-omega(Ω), lower bound
(i) understand big-omega(Ω) notation:
- it tells us the lower bound (minimum time) an algorithm will take in the best -case scenario.
- it guarantees that the algorithm at least takes this much time.
- unlike big-0h (which focuses on worst-case), big-omega looks at the fastest possible execution

definition of big-omega Ω :
a function f(n) is said to be Ω(g(n)) if there exist positive constants c and k(n 0)
such that:
f(n)≥c⋅ g(n) for all n > k (c,k>0)

analysis of the text:

(i) existence of constants:
- we must find a constant c > 0 and a threshold k > 0 such that for all n≥k, the inequality
holds.
- c is a scaling factor that ensures f(n) is always at least c⋅ g(n).
- k ensures that we only consider large values of n, ignoring small fluctuations.

(ii) f(n) grows at least as fast as g(n):

- the function f(n) should never be smaller than c⋅ g(n) for large n.
- this means f(n) is either greater than or grows at the same rate as g(n), but never slower.

(iii) this must hold for large values of n (asymptotic behavior):

- the definition is valid only when n is sufficiently large (i.e., n≥k).
- for small values of n, the behavior of f(n) doesn’t matter.

key difference from big-o notation:

• big-0h focuses on the upper bound (worst-case performance).
• big-omega focuses on the lower bound (best-case performance).
• instead of saying " f(n) grows at most as fast as g(n)," we now say " f(n) grows at least as fast
as g(n)."
doubt(i) what do we mean by set here?

Chapter - 1 (Analysis of Algorithms) Page 48

in this context, the "set" refers to a collection or group of functions that satisfy a particular
condition.

specifically, the set Ω(g(n)) is defined as:

Ω(g(n))={f(n):there exist positive constants c and n0(k)

such that
∃c>0 and n0(k)>0 such that 0 ≤ cg(n) ≤ f(n)for all n≥n0.

the set Ω(g(n)) contains all functions f(n) that grow at least as fast as g(n), up to a
constant factor, for sufficiently large n.

set of functions f(n) that satisfies the inequality

f.. f f(n) ≥ c.g(n) for all n ≥ n0(k)}
5
f1
f3
f.. f2
f.. f..
f4
f.. f..

set 0(g(n))

set of functions f(n) that satisfies Ω(n2)

f(n) ≥ c.g(n) for all n ≥ n0(k)}
4n2 4n3+3n2 ≥ n2
6n2+9 6n6+n4

5n2+2n 2n2+4n

set Ω(n2)

graphical representation:
rate of the growth of the
function f(n)

(more than c.g(n))

Chapter - 1 (Analysis of Algorithms) Page 49

 rate of the growth of the
function f(n)

(more than c.g(n))

beyond k, the rate of

growth of c.g(n) is
between 0 to k there is no well like this (function are
defined relationship between f and converging)
g, sometimes f is more and
sometimes g.

functions are not converging 0

between 0 to k (k)
value of n is increasing
some value k f(n) ≥ c⋅ g(n) for all n ≥ k (c,k>0)

Chapter - 1 (Analysis of Algorithms) Page 50

let's take an example and compare the graphs to get a better understanding:
f(n)=1+n+n2
Ω(n2)

let's check

(i) 1+n+n2 ≥ c1.n, n>1

- Ω(n)
satisfies but it is not closer

(ii) 1+n+n2 ≥ 1.n2, n>1

- Ω(n)
satisfies and more closer to the function

point to remember:

Chapter - 1 (Analysis of Algorithms) Page 51

 satisfies and more closer to the function

point to remember:
bigger functions are in the order of smaller ones in big-omega notation because big-omega represents
an lower bound
analysis:
we have the function:
f(n) = 1 + n + n²

now, the goal is to prove that this function grows at least as fast as n², and we are doing this
using big-omega notation. the idea is to show that for sufficiently large n, f(n) is bounded
below by some constant multiple of n².

(i) big-omega definition:

a function f(n) is said to be Ω(g(n)) if there exist positive constants c and k (where k is the
threshold) such that:
f(n) ≥ c ⋅ g(n) for all n ≥ k,

where c is a constant, and k is the threshold for large n.

(i) the comparison step:

- start with the function:
f(n) = 1 + n + n²

(a) compare each term with n²:

○ the first term is 1. we know that for large n, 1 is much smaller than n², but still, we want
to see how the smaller terms relate to n².

○ the second term is n. similarly, for large n, n is smaller than n², but we need to prove that
the whole function grows at least as fast as some constant multiple of n².

(ii) write the inequality: to prove that f(n) ≥ n²,

we need to show:
f(n) = 1 + n + n² ≥ n².

○ we know that 1 and n are much smaller than n² as n grows large, so we bound the terms
to ensure the inequality holds.

○ for sufficiently large n, we have:

1 ≤ n²
n ≤ n²

(iii) now compare:

1 + n + n² ≥ n²

○ when n becomes sufficiently large, both 1 and n are smaller than n², so the entire
expression is bounded below by n².

Chapter - 1 (Analysis of Algorithms) Page 52

 ○ this inequality holds for large values of n.

conclusion:
• we are not comparing each term directly with n², but rather, we're showing that the function
f(n) has a lower bound that is proportional to n² for large values of n.

• this shows that for sufficiently large n, f(n) is always greater than or equal to n², so f(n) is
Ω(n²).

why use this comparison?

• the reasoning here is to simplify the function and show that, beyond a certain point (i.e., for
large n), all terms behave similarly to n² in terms of their growth, but from below.

• this approach is common in big-omega analysis because we’re focusing on the asymptotic
lower bound behavior for large n, not the exact details of the function for small n.

1+n+n2 > 1.n2 ,n>1

c g
k
f(n)

- f(n) is g(n)
f(n) is Ω(n2)

1+n+n2 ≥ n3, n>1 (does not satisfy)

n3 grows much faster than n 2, so it does not hold.

takeways:
- big-omega (Ω) ignores lower-order terms for large n.
- the function must be at least as large as a constant multiple of g(n).
- since n2 dominates, 1+n+n2 is at least Ω(n2).

Chapter - 1 (Analysis of Algorithms) Page 53

doubt (i) so we found Ω(n2) for the function 1+n+n2, meaning it grows at least as fast as n2. similarly,
we found O(n2) for the function 1+n+n2, meaning it grows at most as fast as 3n2.

my question is: can we choose constants arbitrarily (without a specific or fixed reason or without being
bound by any rule or limitation)? for big-omega, we chose c=1, so the lower bound became n2. for big-oh,
we chose c = 3, making n2 the upper bound. what’s the catch here? am i missing something in my
understanding?

step 1: what we proved

we analyzed the function:
f(n)=1+n+n2
and found:

(a)big-omega Ω(n2)(lower bound)

○ we showed that for some constant c = 1 and sufficiently large n,
f(n) ≥ c⋅ n2
meaning f(n) grows at least as fast as n2.
Chapter - 1 (Analysis of Algorithms) Page 54
 meaning f(n) grows at least as fast as n2.

(b) big-oh O(n2)(upper bound)

○ we showed that for some constant c=3 and sufficiently large n,
f(n) ≤ c⋅ n2
meaning f(n) grows at most as fast as n2.

step 2: why do we choose different constants?

the choice of different constants c is not random but just a proof technique.

(a)big-oh O(n2) finds an upper bound

○ we need to find any constant c that makes f(n) ≤ c⋅ g(n) true.
○ we found that c = 3 works.
○ this means that f(n) never grows faster than n2 up to a constant multiple.
○ if we found another valid constant (e.g., c=5c = 5), that would also work.

(b) big-omega Ω(n2) finds a lower bound

○ we need to find any constant c that makes f(n)≥c⋅ g(n) true.
○ we found that c=1 works.
○ this means that f(n) always grows at least as fast as n2.
○ if we found another valid constant (e.g., c=0.5), that would also work.

point to remember:
the actual work in proving big-oh (O) or big-omega (Ω) is to find a constant 'c' (along with a threshold
'k') that makes the inequality hold for sufficiently large n.

example (ii):
we are given
f(n)=n+logn
Chapter - 1 (Analysis of Algorithms) Page 55
f(n)=n+logn

step (1): definition of big-omega notation

a function f(n) is in Ω(g(n)) if there exist constants c>0 and k > 0

such that:
f(n) ≥ c⋅ g(n) for all n ≥ k

step (2): compare f(n)=n+logn with g(n)

we need to show:
n+log n≥c⋅ g(n)for all n≥k

let’s choose g(n)=n and check if a valid c exists:

n+logn ≥ n,n>1
n+logn ≥ 1⋅ n, n>1
n+logn ≥ n, n>1=Ω(n)

why does Ω(n) hold?

- in n+logn, the term logn is always positive, meaning f(n) is always at least as large as n.
- for sufficiently large n, lon is small compared to nn, but it never makes f(n) smaller than n.

thus, we have:
f(n): n+logn
c: 1
g(n): n
Ω(n)

doubt (ii): why does Ω(n) work?

as n→∞, the term logn adds a small positive value to n, meaning n+logn is always greater than or equal to
n.

for example:
- if n=106, then logn≈20, which is tiny compared to 10 6.
- this means n+logn is practically the same as n for large n, confirming the lower bound Ω(n).
thus, n+logn is asymptotically lower-bounded by Ω(n).

Chapter - 1 (Analysis of Algorithms) Page 56

example(iii)
we are given:
f(n)=2100(c)=Ω(1)

why is f(n)=Ω(1)?
- f(n)=2100 is a constant value (a fixed number).
- it does not depend on n or grow as n increases.
- in big-omega notation, a constant function is always considered O(1).

formal proof using big-o definition

a function f(n) is in O(g(n)) if there exist constants c>0 and k>0 such that:
f(n) ≥ c⋅ g(n)for all n ≥ k

applying big-omega to f(n)=2100 and g(n)=1:

- here, we are comparing f(n)=2 100(a constant) with g(n)=1(another constant).

- we want to find constants c and k

such that:
f(n)≥c⋅ g(n)
which simplifies to:
2100 ≥ 100⋅ 1
2100 ≥ c⋅ 1

- this inequality is true for any c ≤ 2100, because 2100 is a constant, and it will always be greater than
or equal to c⋅ 1 as long as c is a constant value less than or equal to 2 100.

point to remember:
when i say f ≥ c.g then f is 0(g)

Chapter - 1 (Analysis of Algorithms) Page 57

point to remember:
when i say f ≥ c.g then f is 0(g)

smaller functions
bigger functions
are in the order of

some own doubts: summarising the concepts of why we use notations, understanding of analysis and
specially trying to gain the understanding of constants.

doubt(i) why do we need a constant c?

(a) asymptotic analysis focuses on large n:
○ asymptotic analysis is about studying how a function behaves as n grows very large.
○ at large values of n, small details (like constants or lower-order terms) have little impact on the
overall growth rate.
○ therefore, we ignore the exact values and focus on how the function grows relative to other functions.

(b) showing growth in terms of another function:

○ in big-omega (Ω) and big-oh (O) notations, we are interested in bounding the growth of one function
by a multiple of another.
○ this allows us to compare how quickly two functions grow without worrying about exact values.

(c) c as a scaling factor:

○ the constant c acts as a scaling factor. it scales the growth of the function to make the inequality
true.
○ c ensures that the growth of the function is appropriately bounded by the function we’re comparing it
to (like n2, n, or a constant).

(d) different constants indicate different "tightness" of the bound:

○ different values of c reflect how tight the bound is.
○ for example, a smaller constant means a looser bound, and a larger constant means a tighter bound.

(e) we prove bounds, not exact equations:

○ asymptotic analysis isn’t about finding an exact formula for a function’s growth. instead, it’s about
proving bounds—the function will grow at least as fast (for Ω) or at most as fast (for O) as some
other function, for large nn.

(f) proving the existence of a valid constant:

○ for big-omega and big-oh, the task is to prove the existence of some constant c such that the inequality
holds for sufficiently large n.
○ we don’t need to worry about finding the exact best constant; we just need to show that for some c (and
a threshold k), the inequality holds true.

doubt(ii) can we choose constants arbitrarily (without a specific or fixed reason or without being bound by
any rule or limitation)?
constants like c are not completely arbitrary, but they are flexible within certain limits.

Chapter - 1 (Analysis of Algorithms) Page 58

(a) in Ω(1):
- the constant cc must satisfy:
2100 ≥ c⋅ 1

- any c ≤ 2100 works.

for example:
c=1, c=0.5, are valid.

- but if c>2100, the inequality fails, so cc cannot be completely arbitrary.

(b) in O(1):
- the constant cc must satisfy:
2100 ≤ c⋅ 1

- any c ≥ 2100 works. for example:

c=300⋅ 2100 are valid.

- but if c<2100, the inequality fails, so c cannot be completely arbitrary.

doubt(iii): why do we choose different constants for Ω and O?

- in Ω :
○ we aim to prove a lower bound,

so c must be small enough to ensure:

f(n) ≥ c⋅ g(n)

- in O:
○ we aim to prove an upper bound,

so c must be large enough to ensure:

f(n)≤c⋅ g(n)

the difference in c reflects the nature of the bounds. c is chosen based on whether we are proving a lower
bound (Ω) or an upper bound (O).

summary:
what’s the "catch"?
the catch is that constants must satisfy the inequality being proved.
- constants are not completely arbitrary, as they depend on the relationship between f(n) and g(n).
- we only need one valid constant c to prove the inequality. other valid constants might exist, but once we
find one, the proof is complete.

final example:

Chapter - 1 (Analysis of Algorithms) Page 59

big oh: big omega:

f(n)≤c⋅ g(n) for all n ≥ k (c,k>0) f(n) ≥ c⋅ g(n) for all n ≥ k (c,k>0)
f(n)=1+n+n2 f(n)=1+n+n2

1<n2 1+n+n2 ≥ 1.n2, n>1

1+n<n2+n2
1+n+n2<n2+n2+n2 f(n): 1+n+n2
1+n+n2<3n2, n>1 c: 1
g(n): n2
f(n): 1+n+n2 Ω(n2)
c: 3
g(n): n2
0(n2)

concept: theta(θ), tight bound

- theta (θ) is used to represent a tight bound on the growth rate of a function.
- it means that a function grows at the same rate, both in the upper bound (big-o) and lower bound (big-
omega).

definition of theta (θ):

a function f(n)f(n) is said to be θ(g(n)) if there exist positive constants c1, c2 and n0(k)
such that:
c1⋅ g(n) ≤ f(n) ≤ c2⋅ g(n), for all n≥n0(k)

- c1: a constant for the lower bound (big-omega).

- c2: a constant for the upper bound (big-o).
- n0(k): the threshold where the inequality starts to hold.

tight bound meaning:

- the function f(n) grows at the same rate as g(n), with constants bounding it from both below and above.
Chapter - 1 (Analysis of Algorithms) Page 60
- the function f(n) grows at the same rate as g(n), with constants bounding it from both below and above.
- this means f(n) is sandwiched between c 1⋅ g(n) and c2⋅ g(n) as n→∞.

why do we need theta?

(a) precise classification: while big-o only provides an upper bound and big-omega gives a lower bound, theta
gives both, describing the exact growth rate.

(b) removes ambiguity: theta tells us that g(n) perfectly captures f(n)'s growth, not faster or slower.

(c) asymptotic tightness: it focuses on the exact growth for large n, ignoring smaller terms.

doubt(i) what do we mean by set here?

in this context, the "set" refers to a collection or group of functions that satisfy a particular
condition.

specifically, the set Θ(g(n)) is defined as:

Θ(g(n))={f(n):there exist positive constants c1, c2 and n0(k)

such that
c1⋅ g(n) ≤ f(n) ≤ c2⋅ g(n), for all n ≥ n0(k)

for Θ(g(n)), the set contains all functions f(n) that are asymptotically tightly bound by
g(n), meaning f(n) neither grows significantly faster nor slower than g(n) for large n.

f(n) is bounded both above and below by constant multiples of g(n) for sufficiently large n.

the set Θ(g(n)) includes all functions f(n) that satisfy the upper and lower bounds defined
by c1 and c2.

set of functions f(n) that satisfies the inequality

c1⋅ g(n) ≤ f(n) ≤ c2⋅ g(n), for all n≥n0(k)
f..
f1 f5 f3
f.. f.. f..
f4 f2
f..
f..

θ(f(n))= 0(g(n))∩ Ω(g(n))

Chapter - 1 (Analysis of Algorithms) Page 61

θ(f(n))= 0(g(n))∩ Ω(g(n))

set of functions f(n) that satisfies Ω(n2)

c1⋅ g(n) ≤ f(n) ≤ c2⋅ g(n), for all n≥n0(k)
3lgn+8 4n2 4n3+3n2
0(n2)∩ Ω(n2)

5n+7 6n2+9 6n6+n4

2nlgn 5n2+2n 2n2+4n

θ(n2)= 0(n2)∩ Ω(n2)

rate of the growth of the

function f(n)

(more than c1.g(n) but less

than c2g(n))

between 0 to k there is no well

defined relationship between f and
g, sometimes f is more and
sometimes g.

functions are not converging 0

between 0 to k (k)
value of n is increasing
some value k c1⋅ g(n) ≤ f(n) ≤ c2⋅ g(n), for all n≥n0(k)

Chapter - 1 (Analysis of Algorithms) Page 62