Efficiency of Algorithms Algorithms

Efficiency of algorithms Algorithms
• Algorithms • Algorithm is a well-defined sequence of steps

• Computational resources: time and space which leads to solving a certain problem.
• Best, worst and average case performance Steps should be:
• How to compare algorithms: machine-independent • concrete
measure of efficiency • unambiguous
• Growth rate • there should be finitely many of them
• Complexity measure O( )
Efficiency of algorithms Binary search and linear search

• How much time does it need • One seems faster than the other
• How much memory (space) does it use • Can we characterise the difference more
precisely?
Best, worst and average case Binary search

Linear search: • Best case - item in the middle, one check
• Best performance: the item we search for is in the • Worst case - item in the last possible division; the
first position; examines one position maximal number of times an array of length N can
• Worst performance: item not in the array or in the be divided is log2 N
last position; examines all positions • Average case: item is found after performing half
• Average performance (given that the item is in the of the possible number of divisions; ½ log2 N
array): examines half of the array
1
Which is more useful? How to compare
• For real time programming: the worst case • Suppose we settle on comparing the worst case
• For getting a general idea of running time: average performance of linear and binary search.
case; however, often difficult to establish • Where do we start?
• For choosing between several available • Timing
algorithms: helps to know what is the best case • ...
(maybe your data are in the best case format, for
example random).
Machine Independence Some clarifications

• The evaluation of efficiency should be as machine • "Basic operations"?
independent as possible. • "Size of input"?
• For the time complexity of an algorithm,
• we count the number of basic operations the algorithm
performs
• we calculate how this number depends on the size of
the input.
• Space complexity: how much extra space is
needed in terms of the space used to represent the
input.
Size of input Size of input contd.

• It is up to us to decide what is a useful parameter • Graph search: we may be interested in how time
to vary when we measure efficiency of an grows when the number of nodes increases, or in
algorithm. how time grows when the number of edges
• For algorithms searching a linear collection, the increases.
natural choice for size of input is the number of • Sometimes we measure efficiency as a function of
items in the collection which we are searching several parameters: e.g. number nodes and edges
(e.g. length of the array). in a graph.
2
Basic operations Basic operations contd.
• Basic operations are operations which take • If we are not sure how operations are
constant time (at most time C for some constant implemented, we have to exercise judgement: can
C). something be in principle implemented as a
• In other words, time required to perform the constant time operation?
operation does not grow with the size of the • For example, adding two 32-bit integers can be,
operands, or is bounded by a constant. but adding a list of 32-bit integers can not: it is
going to take longer for a longer list.
Example Binary search

binarySearch(int[] arr, int value){
linearSearch(int[] arr, int value){ int left = 0;
int right = arr.length - 1;
for(int i=0; i<arr.length; i++) int middle;
if(arr[i]==value) return true; while (right >= left) {
middle = (left+right)/2;
return false;} if (value == arr[middle]) return true;
if (value < arr[middle]) right=middle-1;
• Basic operations: comparing two integers; else left = middle+1;
incrementing i. Constant time (at most some C) }
spent at each iteration. return false;
}
• Size of input: length of the array N.
• Time usage in the worst case: t(N)=C * N.
Analysis of binary search Rate of Growth

• Size of input = size of the array, say N We don't know how long the steps actually take;
• Basic operations: assignments and comparisons we only know it is some constant time. We can
just lump all constants together and forget about
• Total number of steps: 3 assignments plus a block them.
of assignment, check and assignment repeated
log2 N times. Assume 3 assignments take at most
time C1 and at each iteration we spend at most What we are left with is the fact that the time in
time C2. sequential search grows linearly with the input,
while in binary search it grows logarithmically -
• Total time = C1 + C2 log2 N much slower.
3
O() complexity measure Upper bound example
f(N)=2N
Big O notation gives an asymptotic upper bound t(N)=3+N
on the actual function which describes
time/memory usage of the algorithm.
t(N) is in O(N)
The complexity of an algorithm is O(f(N)) if there because for all N>3,
exists a constant factor K and an input size N0 2N > 3+N
such that the actual usage of time/memory by the Here, N0 = 3 and
algorithm on inputs greater than N0 is always less K=2.
than K f(N).
N0 N
In other words Comments

An algorithm actually makes g(N) steps, Obviously lots of functions form an upper bound,
for example g(N) = C1 + C2log2N we try to find the closest.
there is an input size N' and We also want it to be a simple function, such as
there is a constant K, such that constant O(1)
for all N > N' , g(N) <= K f(N) logarithmic O(log N)
then the algorithm is in O(f(N). linear O(N)
Binary search is O(log N): quadratic, cubic, exponential...
C1 + C2log2N <= (C1 + C2 ) log2N for N > 2
Typical complexity classes Contd.

Algorithms which have the same O( ) complexity • O(N log N) : splitting into subtasks and combining
belong to the same complexity class. the results later
Common complexity classes: • O(N2): quadratic. Usually arises when all pairs of
• O(1) constant time: independent of input length input elements need to be processed
• O(log N) logarithmic: usually results from • O(2N): exponential. Usually emerges from a brute-
splitting the task into smaller tasks, where the size force solution to a problem.
of the task is reduced by a constant fraction
• O(N) linear: usually results when a given constant
amount of processing is carried out on each
element in the input.
4
Practical hints Warning about O-notation
• Find the actual function which shows how the • O-notation only gives sensible comparisons of
time/memory usage grows depending on the input algorithms when N is large
N. Consider two algorithms for same task:
• Omit all constant factors. Linear: g(N) = 1000 N is in O(N)
Quadratic: g'(N) = N2/1000 is in O(N2 )
• If the function contains different powers of N,
(e.g. N4 + N3 + N2), leave only the highest power • The quadratic one is faster for N < 1 000 000.
(N4). • Some constant factors are machine dependent, but
• Similarly, an exponential (2N) eventually others are a property of the algorithm itself.
outgrows any polynomial in N.
Summary Recommended reading

• Big O notation is a rough measure of how the • Shaffer, Chapter 3 (note that we are not going to
time/memory usage grows as the input size use Ω and Θ notation in this course, only the
increases. upper bound O()).
• Big O notation gives a machine-independent
measure of efficiency which allows comparison of
algorithms.
• It makes more sense for large input sizes. It
disregards all constant factors, even those intrinsic
to the algorithm.
Informal coursework
Which statements below are true?
• If an algorithm has time complexity O(N2), it
always makes precisely N2 steps, where N is the
size of the input.
• An algorithm with time complexity O(N) is
always runs slower than an algorithm with time
complexity O(log2(N)), for any input.
• An algorithm which makes C1 log2(N) steps and
an algorithm which makes C2 log4(N) steps
belong to the same complexity class (C1 and C2
are constants).

Efficiency of Algorithms Algorithms

Uploaded by

Copyright:

Available Formats

Efficiency of Algorithms Algorithms

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Efficiency of Algorithms Algorithms

Uploaded by

Copyright:

Available Formats

Efficiency of algorithms Algorithms

• Algorithms • Algorithm is a well-defined sequence of steps

Efficiency of algorithms Binary search and linear search

Best, worst and average case Binary search

Machine Independence Some clarifications

Size of input Size of input contd.

Example Binary search

Analysis of binary search Rate of Growth

In other words Comments

Typical complexity classes Contd.

Summary Recommended reading

You might also like