Advanced Algorithms

(PCCO6020T)
Unit 1

Prof. Swapnil H. Chaudhari

Computer Engineering Department,

R.C.Patel Institute of Technology, Shirpur

Syllabus Structure
Introduction
Definition of algorithms:

 A set of instructions that solve a specific problem or accomplish a specific

task.

Importance of algorithms in computer science:

 Algorithms form the backbone of computer science and are essential in

fields such as artificial intelligence, data science, cryptography, and more.

Types of algorithms:

 There are many different types of algorithms, including search algorithms,

sorting algorithms, graph algorithms, and dynamic programming
algorithms, among others.
Introduction
Time and space complexity:

 Two important factors in evaluating the efficiency of an algorithm are its

time complexity, which refers to the amount of time it takes to run, and its
space complexity, which refers to the amount of memory it requires.

 Big O notation: Big O notation is a mathematical notation used to describe

the upper bound of an algorithm's time complexity. It provides a way to
express the worst-case scenario for an algorithm's running time.
What is time complexity

 Time complexity is a function that describes the amount of time required

to run an algorithm, as input size of the algorithm

 Calculating time complexity by the growth of the algorithm is the most

reliable way of calculating the efficiency of an algorithm
Calculate the Time Complexity
 A common mistake with time complexity is to think of it as the running
time(clock time) of an algorithm.

 The running time of the algorithm is how long it takes the computer to
execute the lines of code to completion, usually measured in milliseconds or
seconds.

 Using this method is not the most efficient way of calculating the running
time of an algorithm, cause the running time of an algorithm depends on

 The speed of the computer (Hardware).

 The programming language used (Java, C++, Python).

 The compiler that translates our code into machine code (Clang, GCC, Min
GW).
 Consider a model machine
 Assigning values to variables.
 Making comparisons.
 Executing arithmetic operations.
 Accessing objects from memory.
 Time Complexity:
 In the above code “Hello World” is printed only once on the screen.
So, the time complexity is constant: O(1)
 1) Loop

for( i = 1 ; i < = n ; i++)

{
x= y + z;
}
 Amortized Analysis

16
 Amortized Analysis
17

 Amortized analysis is applied on data structures that

support many operations.
 The sequence of operations and the multiplicity of each
operation is application specific or the associated
algorithm specific.
 Classical asymptotic analysis gives worst case analysis of
each operation without taking the effect of one operation
on the other.
 Amortized analysis focuses on a sequence of operations,
an interplay between operations, and thus yielding an
analysis which is precise and depicts a micro-level
analysis.
18 Amortized Analysis

 Purpose is to accurately compute the total time spent in

executing a sequence of operations on a data structure
 Three different approaches:
Aggregate method
Accounting method
Potential method
19 Aggregate method

 We determine an upper bound T(n) on Total cost of a sequence of n-

operations.
 In the worst case :
𝑇(𝑛)
 The average cost or amortized cost per operation is =
𝑛
 Note that this amortized cost applies to each operation, even when there is
several types of operations in sequence.

Note that this amortized cost applies to each operation, even when there is
several types of operations in sequence.
 How large should a hash
table be?
Goal: Make the table as small as possible, but
large enough so that it won’t overflow (or
otherwise become inefficient).
Problem: What if we don’t know the proper size
in advance?
Solution: Dynamic tables.
IDEA: Whenever the table overflows, “grow” it
by allocating (via malloc or new) a new, larger
table. Move all items from the old table into the
new one, and free the storage for the old table.
 Example of a dynamic table

1. INSERT 1
2. INSERT overflow
 Example of a dynamic table

1. INSERT 1
2. INSERT overflow
 Example of a dynamic table

1. INSERT 1
2. INSERT 2
 Example of a dynamic table

1. INSERT 1
2. INSERT 2
3. INSERT overflow
 Example of a dynamic table

1. INSERT 1
2. INSERT 2
3. INSERT overflow
 Example of a dynamic table

1. INSERT 1
2. INSERT 2
3. INSERT
 Example of a dynamic table

1. INSERT 1
2. INSERT 2
3. INSERT 3
4. INSERT 4
 Example of a dynamic table

1. INSERT 1
2. INSERT 2
3. INSERT 3
4. INSERT 4
5. INSERT overflow
 Example of a dynamic table

1. INSERT 1
2. INSERT 2
3. INSERT 3
4. INSERT 4
5. INSERT overflow
 Example of a dynamic table

1. INSERT 1
2. INSERT 2
3. INSERT 3
4. INSERT 4
5. INSERT
 Example of a dynamic table

1. INSERT 1
2. INSERT 2
3. INSERT 3
4. INSERT 4
5. INSERT 5
6. INSERT 6
7. INSERT 7
 Worst-case analysis

Consider a sequence of n insertions. The worst-case time

to execute one insertion is (n).Therefore, the worst-case
time for n insertions is n · (n) = (n2).

WRONG! In fact, the worst-case cost for

n insertions is only (n) ≪ (n2).

Let’s see why.

 Tighter analysis

Let ci = the cost of the i th insertion

i if i – 1 is an exact power of 2,
=
1 otherwise.

i 1 2 3 4 5 6 7 8 9 10
sizei 1 2 4 4 8 8 8 8 16 16
ci 1 2 3 1 5 1 1 1 9 1
 Tighter analysis

Let ci = the cost of the i th insertion

i if i – 1 is an exact power of 2,
=
1 otherwise.

i 1 2 3 4 5 6 7 8 9 10
sizei 1 2 4 4 8 8 8 8 16 16
1 1 1 1 1 1 1 1 1 1
ci
1 2 4 8
 Tighter analysis (continued)
n
Cost of n insertions   c i
i1
lg(n1)
n  2j
j0
 3n
 (n).
Thus, the average cost of each dynamic-table
operation is (n)/n = (1).
 Example for amortized analysis
• Amortized analysis can be used to show that the average cost of an operation is
small , if one averages over a sequence of operations ,even though a single
operation within the sequence might be expensive
• Stack operations:
– PUSH(S,x), O(1)
– POP(S), O(1)
– MULTIPOP(S,k), min(s,k)
•while not STACK-EMPTY(S) and k>0
• do POP(S)
• k=k-1
• Let us consider a sequence of n PUSH, POP, MULTIPOP.
– The worst case cost for MULTIPOP in the sequence is O(n), since the stack
size is at most n.
– thus the cost of the sequence is O(n2). Correct, but not tight.
 Aggregate Analysis
• In fact, a sequence of n operations on an
initially empty stack cost at most O(n). Why?

Each object can be POP only once (including in MULTIPOP) for each time
it is PUSHed. #POPs is at most #PUSHs, which is at most n.

Thus the average cost of an operation is O(n)/n = O(1).

Amortized cost in aggregate analysis is defined to be average cost.

 Another example: increasing a binary counter

• Binary counter of length k, A[0..k-1] of bit array.

• INCREMENT(A)
1. i0
2. while i<k and A[i]=1
3. do A[i]0 (flip, reset)
4. ii+1
5. if i<k
6. then A[i]1 (flip, set)
Amortized (Aggregate) Analysis of INCREMENT(A)
Observation: The running time determined by #flips but not all bits flip each time INCREMENT is
called.

A[0] flips every time, total n times.

A[1] flips every other time, n/2 times.
A[2] flips every forth time, n/4 times.
….
for i=0,1,…,k-1, A[i] flips n/2i times.

log 𝑛 ∞
Thus total #flips is 𝑛
< 𝑛
2𝑖 2𝑖
𝑖=0 𝑖=0

=2n.
Accounting Method
 In accounting method, we assign different charges to
different operations. The amount we charge is called
amortized cost
 𝐶𝑖 = is the actual cost
 𝐶𝑖 =is the amortized cost
 Accounting method
• Charge i th operation a fictitious amortized cost
ĉi, where $1 pays for 1 unit of work (i.e., time).
• This fee is consumed to perform the operation.
• Any amount not immediately consumed is stored
in the bank for use by subsequent operations.
• The bank balance must not go negative! We
must ensure that n n
ci  cˆi
i1 i1
for all n.
• Thus, the total amortized costs provide an upper
bound on the total true costs.
 Accounting analysis of
dynamic tables
Charge an amortized cost of ĉi = $3 for the i th
insertion.
• $1 pays for the immediate insertion.
• $2 is stored for later table doubling.
When the table doubles, $1 pays to move a recent
item, and $1 pays to move an old item.
 Accounting analysis
(continued)
Key invariant: Bank balance never drops below 0.
Thus, the sum of the amortized costs provides an
upper bound on the sum of the true costs.
i 1 2 3 4 5 6 7 8 9 10
sizei 1 2 4 4 8 8 8 8 16 16
ci 1 2 3 1 5 1 1 1 9 1
ĉi 2* 3 3 3 3 3 3 3 3 3
banki 1 2 2 4 2 4 6 8 2 4
 Potential method

IDEA: View the bank account as the potential

energy (à la physics) of the dynamic set.
Framework:
• Start with an initial data structure D0.
• Operation i transforms Di–1 to Di.
• The cost of operation i is ci.
• Define a potential function  : {Di}  R,
such that (D0 ) = 0 and (Di )  0 for all i.
• The amortized cost ĉi with respect to  is
defined to be ĉi = ci + (Di) – (Di–1).
 Understanding potentials

ĉi = ci + (Di) – (Di–1)

potential difference i

• If i > 0, then ĉi > ci. Operation i stores

work in the data structure for later use.
• If i < 0, then ĉi < ci. The data structure
delivers up stored work to help pay for
operation i.
 The amortized costs bound
the true costs

The total amortized cost of n operations is

n n

 cˆi   ci  (Di )  (Di1 ) 

i1 i1
Summing both sides.
The amortized costs bound
the true costs
The amortized costs bound
the true costs
 Conclusions

• Amortized costs can provide a clean abstraction

of data-structure performance.
• Any of the analysis methods can be used when
an amortized analysis is called for, but each
method has some situations where it is arguably
the simplest.
• Different schemes may work for assigning
amortized costs in the accounting method, or
potentials in the potential method, sometimes
yielding radically different bounds.