Fundamental Concept

of Algorithms
 Objectives 1

• Learn about problem solving skills

• Explore the algorithmic approach for problem solving
• Learn about algorithm development
• Become aware of problem solving process
 Problem Solving 2

• A process that combines expertise from various disciplines to

create and assess plans that significantly enhance product
and process performance.
• Problem solving process
▪ Define the problem
▪ Analyze the problem
▪ Design steps (algorithm) to solve the problem
▪ Implement the solution (program)
▪ Evaluate the solution
 Define the Problem 3

• Differentiate fact from opinion

• Specify underlying causes
• Consult each faction involved for information
• State the problem specifically
• Identify what standard or expectation is violated
• Determine in which process the problem lies
• Avoid trying to solve the problem without data
 Analyze the Problem 4

• Thoroughly understand the problem

• Specify short- and long-term alternatives
• Brainstorm on others' ideas
• Seek alternatives that may solve the problem
• Understand problem requirements
▪ Does program require user interaction?
▪ Does program manipulate data?
▪ What is the output?

• If the problem is complex, divide it into subproblems

▪ Analyze each subproblem as above
 What is Algorithm? 5

“An algorithm is an ordered set of unambiguous executable

steps, defining a terminating process.”

Input Algorithm Output

• Design or Planning state

• Step-by-step problem-solving process
• Finite and unambiguous sequence of
instructions to solve a problem
 Properties of Algorithms 6

• Input : 0 or more well-defined inputs

• Output : at least 1 or more well-defined outputs
• Correctness : always returns the desired output for all legal instances of the problem.
• Finiteness : terminate after a finite number of steps
• Unambiguity : one statement should be clear and must lead to only one meaning
• Efficiency
▪ Can be measured in terms of Time and Space
▪ Time tends to be more important
 Algorithm vs Program 7

• Algorithm • Program
▪ Design Phase ▪ Implementation Phase
▪ Natural Language ▪ Any Programming Language
▪ Domain Knowledge ▪ Programmer
▪ OS and H/S Independent ▪ OS and H/S Dependent
▪ Analyze ▪ Testing

Program = Algorithm + Data Structure

• Programming is a subprocess of problem solving.

• Program is a representation of an algorithm designed for
computer applications.
 Representation of Algorithms 8

• A single algorithm can be represented in many ways:

▪ Formulas: written in mathematical formula
▪ Words: written in natural language
▪ Flowcharts : show a single program's sequence of instructions with symbols.
▪ Pseudo-code : like a programming language but its rules are less stringent.

• In each case, the algorithm stays the same; the implementation

differs!
 Expressing Algorithms 9

Example 1: Write an algorithm, a pseudo-code and draw a flowchart that will read the two sides of a
rectangle and calculate its area.

Formulas: Area = Width x Length

Algorithm Pseudo-code Flow Chart Implementation (C#)

Step 1: Begin Step 1: Start void area()
start
Step 2: Input W, L {
Step 2: Input the width (W) and
Step 3: AWxL //input
Length (L) of a rectangle Step 4: Print (A) W, L int W = 10;
Step 3: Calculate the area (A) Step 5: Stop int L = 5;
by multiplying L with W
AWxL
Step 4: Print area (A) //operation
area = W * L;
Step 5: End
Print (A)
//output
Console.Write(area);
end }
 Expressing Algorithms (Con’t) 10

Example 2: Write an algorithm and draw a flowchart that will read the two numbers and find its largest
number.

Algorithm Flow Chart Implementation (C#)

void Large(a, b)
Step 1: Begin start
{
Step 2: Read a, b if (a>b)
a, b
Step 3: If a>b then ‘a’ is largest {
Console.Write(a);
otherwise ‘b’ is largest Is No }
Print b
Step 4: End a>b? else
Yes {
Console.Write(b);
Print a }
}

end
 Algorithm Analysis 11

• Time complexity
▪ is the amount of CPU time it needs to run to completion.
• Space complexity
▪ is the amount of memory it needs to run to completion.
• Power consumption
• Network consumption
• Computational resource
 Asymptotic Notations 12

• Best case : Theta (θ) Notation (tighter bound)

• Average case: Omega (Ω) Notation (lower bound)
• Worst case: Big Oh (O) Notation (upper bound)

Best case efficiency: Worst case efficiency:

▪ List = [25, 31, 42, 41, 105], find 25 ▪ If we want to search the element which is present at
▪ 25 is present at the first position the last of the list or not present at all in the list then
▪ Θ(f(x)) = 1 such cases are called the worst case efficiency.

▪ List = [25, 31, 42, 41, 105]

Average case efficiency: ▪ Find 105
▪ List = [25, 31, 42, 41, 105], find 42 ▪ 105 is present at the last position
▪ Let the element is not present at the first ▪ O(f(x)) = n
or the last position
▪ Ω(f(x)) = 0 ≤ p ≤ n ▪ Find 110
▪ 110 is not in the list
▪ O(f(x)) = n
 Orders of Growth 13

Degree Name Notation

Best Constant Time O(1)
Better Logarithmic Time O(log n)
Square Root Time O(√𝑛)
Good Linear Time O(n)
Fair Log Linear Time O(n log n)
Poor Quadratic Time O(n2)
Cubic Time O(n3)
Worse Exponential Time O(2n)
Worst Factorial Time O(n!)
Polynomial Time O(nk)
Super-Exponential Time O(nn)
 Constant Time O(1) Complexity 14

Example : Write a program that takes two numbers as input, swaps them.

Statement Count Variable Count

Algorithm Swap(a, b)
temp = a; 1 a 1
{
temp = a; a = b; 1 b 1
a = b; b = temp; 1 temp 1
b = temp; return a, b; 1
return a, b;
T(n) 4 S(n) 3
}
Time O(1) Space O(1)
 Logarithmic Time 𝑂(log 𝑛) Complexity 15

Iteration i
Assume :
Algorithm() 1 1
i ≥ n : terminate point
{ 2 1*2 = 21 i = 2k
for(i=1; i < n; i = i * 2) 3 21*2 = 22
{ → 2k ≥ n
4 22*2 = 23
staments; 2k = n
} - -
- -
k = Log2n
}
n 2k Time Complexity = O(Log2n)
 Square Root Time O( 𝒏) Complexity 16

Assume :
Algorithm() Iteration i p p > n : terminate point
{ 1 1 0+1 = 1 𝑘(𝑘+1)
p=
2
p = 0; 2 2 1+2 = 3
for(i=1; p <= n; i++) 𝑘(𝑘+1)
3 3 1+2+3 → >n
{ 2

p = p + i; 4 4 1+2+3+4 𝑘 2 +𝑘
} >n
- - - 2
return p; - - - 𝑘2 > n
} n k 1+2+3+…+k
k= 𝑛
Time Complexity = O( 𝑛)
 Linear Time O(n) Complexity 17

Example : Write a program that finds the sum of all the numbers in an array.

Algorithm Sum(arr, n)
{ Statement Count Variable Count
sum = 0; sum = 0; 1 arr n
for( i = 0; i < n; i++) for( i = 0; i < n; i++) n+1 n 1
{ sum = sum + arr[i]; n sum 1
sum = sum + arr[i]; return sum; 1 i 1
} T(n) 2n + 3 S(n) n+3
return sum; Time O(n) Space O(n)
}
 Log-Linear Time 𝑂(𝑛 log 𝑛) Complexity 18

Algorithm()
{
for( i = 0; i < n; i++) Statement Count
{ for( i = 0; i < n; i++) n+1
for(j=1; j < n; j = j*2) for(j=1; j < n; j = j*2) n * log n
{ staments; n * log n
staments; T(n) 2n log n + n
} Time O(n log n)
}
}
 Quadratic Time O(n2) Complexity 19

Example : Write a program that finds the sum of two matrix.

Statement Count Variable Count

Algorithm Sum(A, B, n)
{ for( i = 0; i < n; i++) n+1 A n2
for( i = 0; i < n; i++) for( j = 0; j < n; j++) n * (n +1) B n2
{ C[i ,j] = A[i, j] + B[i, j]; n*n C n2
for( j = 0; j < n; j++) n 1
{
i 1
C[i, j] = A[i, j] + B[i, j];
} j 1
} T(n) 2n2 + 2n + 1 S(n) 3n2 + 3
} Time O(n2) Space O(n2)
 Cubic Time O(n3) Complexity 20

Example : Write a program that finds the multiplication of two matrix.

Algorithm Multiply(A, B, n) Statement Count Variable Count

{ for( i = 0; i < n; i++) n+1 A n2
for( i = 0; i < n; i++)
{ for( j = 0; j < n; j++) n (n +1) B n2
for( j = 0; j < n; j++) C[i, j] = 0; n*n C n2
{ for(k=0; k<n; k++) n*n (n+1) n 1
C[i, j] = 0;
for(k=0; k<n; k++) C[i, j] = C[i, j] + A[i, k] * B[k, j]; n*n*n i 1
{ j 1
C[i, j] = C[i, j] + A[i, k] * B[k, j];
k 1
}
} T(n) 2n3+3n2+2n+1 S(n) 3n2+4
} Time O(n3) Space O(n2)
}
 Exponential Time 𝑂(2n) Complexity 21

n=4
Algorithm Fib(n)
{ Fib(1)
if( n == 0) { Fib(2)
return 0; Fib(3) Fib(0)
}
Fib(1)
if( n == 1) {
return 1; Fib(4)
}
Fib(1)
return Fib(n-1) + Fib(n-2);
Fib(2)
}
Fib(0)

F(n) Invoked → 1 2 = 21 4 = 22 -------- 2n-1

Time : O(2n)
 Factorial Time 𝑂(𝑛!) Complexity 22

0
Fun(1) Fun(0) statements;
0
n=3 0
Algorithm Fun(n) Fun(2) Fun(1) Fun(0) statements;
1
{
0
if( n == 0) { 0
statements; 0 Fun(1) Fun(0) statements;
1
} Fun(3) Fun(2)
0
for( i = 0; i < n; i++) { 1 Fun(1) Fun(0) statements;
Fun(n – 1);
2
} 0 0
} Fun(2) Fun(1) Fun(0) statements;
1 0
Fun(1) Fun(0) statements;

Printed Statement → 6 = 3! = n!

Time : O(n!)
 Further Work 23

Optimizing Time Complexity

1. Choose the Right Data Structure
2. Use Efficient Algorithms and Principles
3. Divide and Conquer
4. Dynamic Programming
5. Greedy Algorithms
6. Pruning and Branching (Backtracking)
7. Parallelism and Concurrency
8. Caching or Memorization
9. Reduce Problem Size