ALGORITHM DESIGN AND ANALYSIS

474CSS-3
LECTURE 3
CHAPTER # 2 - FUNDAMENTALS OF THE ANALYSIS OF
ALGORITHM EFFICIENCY
 COURSE LEARNING OUTCOME

CLO_2
Measure the efficiency of algorithms by evaluating the time
complexity of an algorithm using the asymptotic notation (Big-O(
O ), Omega( Ω ), Theta( 𝚯 )

2
College of Computer Science & Information Systems, Najran University, KSA
 LECTURE OUTLINES
q Analysis Framework:
§ Measuring an Input's Size.
§ Orders Of Growth.
§ Units For Measuring Running Time.
§ Worst-Case, Best Case and Average-Case Efficiencies.

q Asymptotic Notations and Basic Efficiency Classes:

§ Informal Introduction
§ O- notation.
§ Ω-notation.
§ θ-notation.

3
College of Computer Science & Information Systems, Najran University, KSA
 Performance Analysis

q Program performance is the amount of time and computer

memory needed to run a program.

qHow is it determined?
1. Analytically
• Performance analysis
2. Experimentally
• Performance measurement

4
College of Computer Science & Information Systems, Najran University, KSA
 Criteria for Measurement

qSpace
• Amount of memory program occupy
• Usually measured in bytes, KB or MB

qTime
• Execution time
• Usually measured by the number of execution steps

5
College of Computer Science & Information Systems, Najran University, KSA
 Complexity
Complexity is the resources required by the algorithms, most known as:
• The time (how many steps it needs) and
• The space (how much memory it takes)
Running Time
Running time — the actual time spent in execution of an algorithm.
It depends on a few factors
• Inputs data
• The hardware and software environment.

6
College of Computer Science & Information Systems, Najran University, KSA
 Time Complexity

Time complexity is the amount of computer time a program needs to

run (how many steps) .
Why do we care about time complexity?
§ Some computers require upper limits for program execution times.
§ Some programs require a real-time response.
§ If there are many solutions to a problem, typically we would like
to choose the quickest.

7
College of Computer Science & Information Systems, Najran University, KSA
 Analysis of an Algorithm
There are often many different algorithms which can be used to solve the
same problem.
Thus, it makes sense to develop techniques that allow us to:
§ Compare different algorithms with respect to their “efficiency”
§ Choose the most efficient algorithm for the problem
The efficiency of any algorithmic solution to a problem is a measure of the:
§ Time efficiency: the time it takes to execute.
§ Space efficiency: the space (primary or secondary memory) it uses.
We will focus on an algorithm’s efficiency with respect to time (Time Complexity).

8
College of Computer Science & Information Systems, Najran University, KSA
 Analysis Framework
Analysis of Algorithms:
q Issues:
Ø Correctness: Is the algorithm correct?
Ø Time Efficiency: How much time does the algorithm use?
Ø Space Efficiency: How much extra space (in addition to the space needed
for the input and output) does the algorithm use?
q Approaches:
Ø Theoretical Analysis: Proof of correctness and big-O and related notations.
Ø Experimental Analysis: Testing and measurement over a range of instances.

College of Computer Science & Information Systems, Najran University, KSA

Analysis Framework (cont.)

College of Computer Science & Information Systems, Najran University, KSA

 Theoretical analysis of time efficiency

Time efficiency is analyzed by determining the number of

repetitions of the basic operation as a function of input size
• Basic operation: the operation that contributes the most towards
the running time of the algorithm
input size

T(n) ≈ Cop(n)
running time execution time
for basic operation
or cost
Note: Different basic operations may cost differently!

College of Computer Science & Information Systems, Najran University, KSA

 Input size and basic operation examples

Problem Input size measure Basic operation

Searching for key in a

Number of listʼs items, i.e. n Key comparison
list of n items
Multiplication of two Matrix dimensions or total Multiplication of
matrices number of elements two numbers
Comparing the GCD of
Two numbers Division
two numbers
Visiting a vertex or
Typical graph problem #vertices and/or edges
traversing an edge

College of Computer Science & Information Systems, Najran University, KSA

 Example of Basic Operations
q Arithmetic operations: *, /, %, +, -
q Assignment statements of simple data types
q Reading of primitive types
q Writing of primitive types
q Indexing into an array
q Following an object reference
q Returning from a method
q Simple conditional tests: if (x < 12) ...
q a method's return statement
q We consider an operation such as ++ , += , and *= as consisting
of two basic operations
13
College of Computer Science & Information Systems, Najran University, KSA
 Best-case, average-case, worst-case
Some algorithms perform differently on various input of similar size. It is
sometimes helpful to consider
§ The worst-case
§ The best-case
§ The average-case
Performance of the algorithm.
For example, say we want to search an array A of size n for a given value k
Worst-case: k A, then we must check every item. Cost= n comparisons
Best-case: k is the first item in the array. Cost=1 comparison
Average-case: probability analysis

14
College of Computer Science & Information Systems, Najran University, KSA
 Best-case, average-case, worst-case (cont.)
For some algorithms, efficiency depends on form of input:

Worst case: Cworst (n) – maximum over inputs of size n

Best case: Cbest(n) – minimum over inputs of size n

Average case: Cavg(n) – “average” over inputs of size n

§ Number of times the basic operation will be executed on typical input
§ NOT the average of worst and best case

15
College of Computer Science & Information Systems, Najran University, KSA
 Best-case, average-case, worst-case (cont.)

We are usually interested in the worst-case complexity

16
College of Computer Science & Information Systems, Najran University, KSA
Example: Sequential search

17
College of Computer Science & Information Systems, Najran University, KSA
 Example
Find the exact number of basic operations in the following program fragment:
double x, y;
x = 2.5 ; y = 3.0; 2
for(int i = 1; i <= n; i++) 1, (n+1), 2*n
{
a[i] = x * y; 2*n
x = 2.5 * x; 2*n
y = y + a[i]; 2*n
}
The total number of basic operations is 6 * n + 2 * n + (n + 1) + 3
= 9n + 4

College of Computer Science & Information Systems, Najran

18
University, KSA
 The Execution Time of Algorithms

Each operation in an algorithm (or a program) has a cost.

è Each operation takes a certain amount of time.

count = count + 1; è take a certain amount of time, but it is constant

A sequence of operations:

count = count + 1; Cost: c1

sum = sum + count; Cost: c2

è Total Cost = c1 + c2
19
 The Execution Time of Algorithms(cont.)

Example: Simple If-Statement

Operation Times
if (n < 0) c1 1
absval = -n c2 1
else
absval = n; c3 1

Total Cost <= c1 + max(c2,c3)

College of Computer Science & Information Systems, Najran University, KSA 20

 The Execution Time of Algorithms(cont.)

Example: Simple Loop Operations Times

i = 1; c1 1
sum = 0; c2 1
while (i <= n) c3 n+1
{
i = i + 1; c4 2*n
sum = sum + i; c5 2*n
}
Total Cost = 1 + 1 + (n+1) + 2*n + 2*n
è The time required for this algorithm is proportional to n
21
College of Computer Science & Information Systems, Najran University, KSA
 The Execution Time of Algorithms(cont.)
Example: Nested Loop Operations Times

c1 1
i=1;
sum = 0; c2 1
while (i <= n) { c3 1*(n+1)
j=1; c4 1*n
while (j <= n) { c5 n*(n+1)
sum = sum + i; c6 2(n*n)
j = j + 1; c7 2(n*n)
}
i = i +1; c8 2*n
}
Total Cost : T(n) = 1 + 1 + 1*(n+1) + 1*n + n*(n+1)+2(n*n)+2(n*n)+2*n

è The time required for this algorithm is proportional to n2 to n

22
College of Computer Science & Information Systems, Najran University, KSA
Questions & answer session

College of Computer Science & Information Systems, Najran

23
University, KSA