Data

Structures and
Algorithms
ALGORITHMIC DESIGN AND
TECHNIQUES

Straight forward Programming Problems:

• Has straightforward implementation

• Natural solution is already efficient

WHAT IS
THE
POSSIBLE
SOLUTION
OF THESE
PROBLEMS?
Simple Programming Problems:

• Has linear scan

• Cannot do much better

• The obvious program works

ALGORITHMIC PROBLEMS

NOT SO CLEAR WHAT TO DO

Algorithms Problems:

• Not clear how to do

• Simple ideas too slow

• Room for optimization

Let us take algorithm problem for Fibonacci Numbers

Definition of Fibonacci Numbers:

What was the

purpose of
0, 1, 1, 2, 3, 5, 8, 13, 21, 34, . . . Fibonacci Numbers?
OR
How it came into
existence?
NAÏVE ALGORITHM
RUNNING TIME (T(n)):
• RUNNING TIME IS THE NUMBER OF LINES OF
CODE TO BE EXECUTED TO SOLVE THE
PROBLEM.
• THEN WHAT IS THE RUNNING TIME OF
FIBRECURS(N) ALGORITHM?
RUNNING TIME
TOO SLOW
EFFICIENT ALGORITHM

1 Statement Operations Iterations Subtotal

2 1 1
3
4 2
5 3
6
4

6
EFFICIENT ALGORITHM

1 Statement Operations Iterations Subtotal

2 1 1 1
3
4 2
5 3
6
4

6
EFFICIENT ALGORITHM

1 Statement Operations Iterations Subtotal

2 1 1 1 1
3
4 2
5 3
6
4

6
EFFICIENT ALGORITHM

1 Statement Operations Iterations Subtotal

2 1 1 1 1
3
4 2 1

5 3
6
4

6
EFFICIENT ALGORITHM

1 Statement Operations Iterations Subtotal

2 1 1 1 1
3
4 2 1 1

5 3
6
4

6
EFFICIENT ALGORITHM

1 Statement Operations Iterations Subtotal

2 1 1 1 1
3
4 2 1 1 1

5 3
6
4

6
EFFICIENT ALGORITHM

1 Statement Operations Iterations Subtotal

2 1 1 1 1
3
4 2 1 1 1

5 3 1 1 1
6
4

6
EFFICIENT ALGORITHM

1 Statement Operations Iterations Subtotal

2 1 1 1 1
3
4 2 1 1 1

5 3 1 1 1
6
4 1

6
EFFICIENT ALGORITHM

1 Statement Operations Iterations Subtotal

2 1 1 1 1
3
4 2 1 1 1

5 3 1 1 1
6
4 1 N-1

6
EFFICIENT ALGORITHM

1 Statement Operations Iterations Subtotal

2 1 1 1 1
3
4 2 1 1 1

5 3 1 1 1
6
4 1 N-1 N-1

6
EFFICIENT ALGORITHM

1 Statement Operations Iterations Subtotal

2 1 1 1 1
3
4 2 1 1 1

5 3 1 1 1
6
4 1 N-1 N-1

5 1 N-1 N-1

6
EFFICIENT ALGORITHM

1 Statement Operations Iterations Subtotal

2 1 1 1 1
3
4 2 1 1 1

5 3 1 1 1
6
4 1 N-1 N-1

5 1 N-1 N-1

6 1 1 1
EFFICIENT ALGORITHM

Statement Operations Iterations Subtotal

1 1 1 1 1

2
2 1 1 1
3
4 3 1 1 1

5 4 1 N-1 N-1
6
5 1 N-1 N-1

6 1 1 1

Total number of operations 2n+2

Therefore,

T(n) = 2n + 2. So T(100) = 202.

Easy to compute.
 RUNTIME
ANALYSIS
2n + 2 lines of code. Does this really describe the runtime of the
algorithm?
 Time complexity
What do we actually mean by Time Complexity of Algorithms?

It is not the actual running time. It is not the standard.

• It varies from machine to machine & compiler to compiler.

• It even varies on one machine due to availability of resources.

Otherwise figuring out actual time would be a huge mess.

All of these issues can multiply runtimes by (large) constants.

Time complexity
So, measure runtime in a way that

ignores constant multiples.

Solution:
Consider asymptotic runtimes. How

does runtime scale with input size.

As the input size ‘n’ gets larger, does

the output scale porpotional to ‘n’ or ‘n

squared’ or is it exponential to ‘n’.

Big-Oh(O) Notation:

We will use big-O notation to report algorithm runtimes.

It has bunch of advantages:
• It clarifies growth rate.
- (How does the runtime scale with input size)
• It cleans up notation
- O(n2) Vs 3n2 + 5n + 2
- O(n) Vs n + log2(n) + Sin(n)
• Big-O doesn’t care about constants.
- O(nlog(n)) Vs 4nlog2(n)+7
Common Rules:
Following are the common rules of comparing the order of growth of
functions arising frequently in algorithm analysis.
Rule 1: Multiplicative constants can be ommitted
Rule 2: Out of two polynomials, the one with larger degree grows faster
Rule 3: Any polynomial grows slower than any exponential
Rule 4: Any polylogarithm grows slower than any polynomial
Rule 5: Smaller terms can be omitted