Analysis of algorithms

Data Structures & Algorithms
Lecturer: Dr. Muhammad M Iqbal
Ph.D (DCU), M.Phil (GCU)
1

Overview
– The concept of algorithm analysis
– Big-Oh notation
– Experimental analysis of algorithms in Java
– Comparing various growth functions
– Efficiency goals
AlgorithmInput Output
2

Computational and
Asymptotic Complexity
• Computational Complexity measures the degree of
difficulty of an algorithm
• Indicates how much effort is needed to apply an
algorithm or how costly it is
• To evaluate an algorithm’s efficiency, use logical units
that express a relationship such as:
– The size n of a file or an array
– The amount of time t required to process the data 3

Analysis of Algorithms
• Method 1:
• Transform the algorithm into some
programming language and measure the
execution time for a set of input values.
• Method 2:
• Use the Big O notation (mathematical
notation) to analyse the behaviour of the
algorithms. (Independent of software and
hardware).
4

• Let us consider two algorithms to solve a problem:
– First algorithm (X) spends 100 * x time units
– Second algorithm (Y) spends 5000 time units
• Which one is better?
– Algorithm X is better if our problem size is small, that is, if x < 50
– Algorithm Y is better for larger problems, with x > 50
• A challenge is to handle the large size of the problems. 5
Complexity of Algorithms
• To understand the performance of an algorithm, it is best to
observe how well or poorly it does for different size of inputs.
• This indicates that the selection of the algorithm for a particular
problem demands analysis based on the input size.

Algorithm Analysis
Consider the problem of summing
FIGURE: Three algorithms for computing the sum
1 + 2 + . . . + n for an integer n > 0 6
TimedAlgorithms.java

Running Time
• Most algorithms transform input
objects into output objects.
• The running time of an algorithm
typically grows with the input size.
• Average case time is often difficult
to determine.
• We focus on the worst case running
time.
– Easier to analyze
– Crucial to applications such as games,
finance and robotics
0
20
40
60
80
100
120
RunningTime 1000 2000 3000 4000
Input Size
best case
average case
worst case
7

Algorithm Efficiency
• The efficiency of an algorithm is usually expressed in
terms of its use of CPU time.
• The analysis of algorithms involves categorizing an
algorithm in terms of efficiency.
• An everyday example: washing dishes.
• Suppose washing a dish takes 30 seconds and drying a
dish takes an additional 30 seconds.
• Therefore, n dishes require n minutes to wash and dry.
8

Algorithm Efficiency
• Now consider a less efficient approach that
requires us to redry all previously washed
dishes after washing another one
seconds4515
2
)1(30
30dishes)(time
)30*()wash timeseconds30(*
2
n
1i
nn
nn
nn
in



 
9

Problem Size
• For every algorithm we want to analyze,
we need to define the size of the problem
• The dishwashing problem has a size n –
number of dishes to be washed/dried
• For a search algorithm, the size of the
problem is the size of the search pool
• For a sorting algorithm, the size of the
program is the number of elements to be
sorted
10

Growth Functions
• We must also decide what we are trying to
efficiently optimize
i. time complexity – CPU time
ii. space complexity – memory space
• CPU time is generally the focus
• A growth function shows the relationship
between the size of the problem (n) and the
value optimized (time)
11

• The growth function of the second dishwashing algorithm is
t(n) = 15n2 + 45n
• It is not typically necessary to know the exact growth
function for an algorithm
• We are mainly interested in the asymptotic complexity of an
algorithm – the general nature of the algorithm as n
increases
• Asymptotic complexity helps to explore the performance of
the algorithms 12

• Asymptotic complexity is
based on the dominant
term of the growth
function – the term that
increases most quickly as
n increases
• The dominant term for
the second dishwashing
algorithm is n2:
13
t(n) = 15n2 + 45n

14
Computational and
Asymptotic Complexity (continued)
Figure 2-1 The growth rate of all terms of function
f (n) = n2 + 100n + log10n + 1,000

Big-Oh Notation
• The coefficients and the lower order terms
become increasingly less relevant as n
increases
• So we say that the algorithm is order n2,
which is written O(n2)
• This is called Big-Oh notation
• There are various Big-Oh categories
• Two algorithms in the same category are
generally considered to have the same
efficiency, but that doesn't mean they have
equal growth functions or behave exactly
the same for all values of n 15

Big-Oh
Categories
• Some sample
growth functions
and their Big-Oh
categories:
• As n increases, the
various growth
functions diverge
dramatically:
16

Picturing Efficiency
FIGURE:
An O(n)
algorithm
FIGURE
An O(n2)
algorithm
17

Analyzing Loop Execution
• First determine the order of the body of the
loop, then multiply that by the number of times
the loop will execute
for (int count = 0; count < n; count++)
// some sequence of O(1) steps
• N loop executions times O(1) operations
results in a O(n) efficiency
18

Analyzing Loop Execution
• Consider the following loop:
count = 1;
while (count < n)
{
count *= 2;
}
• The loop is executed log2n times, so the loop is
O(log n)
19

Analyzing Nested Loops
• When the loops are nested, we multiply the complexity of the outer loop
by the complexity of the inner loop
for (int count = 0; count < n; count++)
for (int count2 = 0; count2 < n; count2++)
{
}
• Both the inner and outer loops have complexity of O(n)
• The overall efficiency is O(n2)
20
• The body of a loop may contain a call to a method
• To determine the order of the loop body, the order of the method must be
taken into account
• The overhead of the method call itself is generally ignored

Experimental Studies
• Write a program
implementing the algorithm
• Run the program with inputs
of varying size and
composition, noting the time
needed:
• Plot the results 0
1000
2000
3000
4000
5000
6000
7000
8000
9000
0 50 100
Input Size
Time(ms)
21

Limitations of Experiments
• It is necessary to implement the algorithm,
which may be difficult
• Results may not be indicative of the running
time on other inputs not included in the
experiment.
• In order to compare two algorithms, the same
hardware and software environments must be
used
22

Theoretical Analysis
• Uses a high-level description of the algorithm instead of an implementation
• Characterizes running time as a function of the input size, n
• Takes into account all possible inputs
• Allows us to evaluate the speed of an algorithm independent of the
hardware/ software environment
23

Comparison of Two Algorithms
Insertion sort is
n2 / 4
Merge sort is
2 n lg n
sort a million items?
insertion sort takes
roughly 70 hours
while
merge sort takes
roughly 40 seconds
This is a slow machine, but if
100 x as fast then it’s 40 minutes
versus less than 0.5 seconds24

More Big-Oh Examples
 7n - 2
7n-2 is O(n)
need c > 0 and n0  1 such that 7 n - 2  c n for n  n0
this is true for c = 7 and n0 = 1
 3 n3 + 20 n2 + 5
3 n3 + 20 n2 + 5 is O(n3)
need c > 0 and n0  1 such that 3 n3 + 20 n2 + 5  c n3 for n  n0
 3 log n + 5
3 log n + 5 is O(log n)
need c > 0 and n0  1 such that 3 log n + 5  c log n for n  n0
25

Prefix Averages 2 (Linear)
The following algorithm uses a running summation to improve
the efficiency
Algorithm prefixAverage2 runs in O(n) time! 26

Experimental Study
• A simple mechanism for collecting such running times in Java is based on use
of the currentTimeMillis method of the System class.
long startTime = System.currentTimeMillis( ); // record the starting time
/* (run the algorithm) */
long endTime = System.currentTimeMillis( ); // record the ending time
long elapsed = endTime − startTime; // compute the elapsed time
27
StringExperiment.java

Experimental Studies
• A simple mechanism for collecting such running times in Java is
based on use of the currentTimeMillis method of the System
class.
StringExperiment.zip
n
50000
repeat1
1607
repeat2
3
100000 5074 4
200000 11727 5
400000 37939 12
800000 152135 12
1600000 648259 16
28

• To analyze the algorithms effectively, a general methodology for analysis
of the running time of algorithms (theoretical analysis)
• In contrast to the "experimental approach", this methodology:
– Uses a high-level description of the algorithm instead of testing one
of its implementations.
– Characterizes running time as a function of the input size, n.
– Takes into account all possible inputs.
– Allows one to evaluate the efficiency of any algorithm in a way that is
independent from the hardware and software environment.
Theoretical Analysis
29

Performance Analysis of the
Algorithms
• In order to analyse the algorithms
i. We should calculate the time using some formula based on the input
size.
ii. We should find a method to determine the memory requirement based
on the input size.
• We can choose the “size of input” to be the parameter that most
influences the actual time/space required
– It is usually obvious what this parameter is
• e.g., number of elements in the array you want to sort
– Sometimes we need two or more parameters
• However, the time performs a major role in all evaluations. 30
Time and Space

Big-Oh Notation
BETTER
WORSE
order 1
order n
order n squared
31

Three-Way Set Disjointness
Suppose we are given three sets, A, B, and C, stored in
three different integer arrays. We will assume that no
individual set contains duplicate values, but that there
may be some numbers that are in two or three of the
sets. The three-way set disjointness problem is to
determine if the intersection of the three sets is empty,
namely, that there is no element x such that x ∈ A, x ∈ B,
and x ∈ C. 32

Three-Way Set Disjointness
Worst-case
running time of
disjoint1 is O(n3).
Worst-case
running time for
disjoint2 is O(n2).
33
Testing.java

Big-Oh Notation
Statements with method calls
• When a statement involves a method call, the complexity of the
statement includes the complexity of the method call.
f(k); // O(1)
g(k); // O(N)
• When a loop is involved, the same rule applies
for (j = 0; j < N; j++)
g(N);
– It has complexity (N2).
– The loop executes N times
– Each method call g(N) is of complexity O(N). 34

Question
• A program’s execution time depends in part on
– the speed of the computer it is running on (time)
– the memory capacity
– the language the algorithm is written in
– all of the above
35

Resources/ References
• Introduction to Algorithms, Thomas H. Cormen, Charles E. Leiserson,
Ronald L. Rivest and Clifford Stein 3rd, 2009, MIT Press and
McGraw-Hill.
• Data Structures: Abstraction and Design Using Java, Elliott B.
Koffmann, 2nd, 2010, Wiley.
• Data Structures and algorithm analysis in java, Mark A. Weiss, 3rd,
2011, Prentice Hall.
• Frank M. Carrano, Data Structures and Abstractions with Java™, 3rd
Edition, Prentice Hall, PEARSON, ISBN-10: 0-13-610091-0.
• This lecture used some images and videos from the Google search
repository to raise the understanding level of students.
https://www.google.ie/search?
Dr. Muhammad M Iqbal*

Analysis of algorithms

More Related Content

What's hot (20)

Similar to Analysis of algorithms (20)

Recently uploaded (20)

Analysis of algorithms