Scribd
Upload
41 views15 pages

Algorithmic Complexity: Module #7

This document discusses algorithmic complexity and how to analyze the complexity of algorithms. It defines algorithmic complexity as a measure of how difficult a computation is based on the resources required. Time and space complexity are common measures. Complexity is usually expressed as a function of input size and analyzed in the worst case. Common orders of growth are discussed like constant, logarithmic, linear, polynomial and exponential time. Problems can be classified as tractable or intractable based on having polynomial time algorithms. Examples are provided to illustrate complexity analysis.

Uploaded by

brm1shubha
Copyright
© Attribution Non-Commercial (BY-NC)
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PPT, PDF, TXT or read online on Scribd
41 views15 pages

Algorithmic Complexity: Module #7

This document discusses algorithmic complexity and how to analyze the complexity of algorithms. It defines algorithmic complexity as a measure of how difficult a computation is based on the resources required. Time and space complexity are common measures. Complexity is usually expressed as a function of input size and analyzed in the worst case. Common orders of growth are discussed like constant, logarithmic, linear, polynomial and exponential time. Problems can be classified as tractable or intractable based on having polynomial time algorithms. Examples are provided to illustrate complexity analysis.

Uploaded by

brm1shubha
Copyright
© Attribution Non-Commercial (BY-NC)
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PPT, PDF, TXT or read online on Scribd
You are on page 1/ 15

Module #7 - Complexity

4/26/2013 1
Module #7:
Algorithmic Complexity
Module #7 - Complexity
4/26/2013 2
What is complexity?
The word complexity has a variety of different
technical meanings in different research fields.
There is a field of complex systems, which studies
complicated, difficult-to-analyze non-linear and
chaotic natural & artificial systems.
Another concept: Informational or descriptional
complexity: The amount of information needed to
completely describe an object.
As studied by Kolmogorov, Chaitin, Bennett, others
In this course, we will study algorithmic or
computational complexity.
Module #7 - Complexity
4/26/2013 3
2.2: Algorithmic Complexity
The algorithmic complexity of a computation is,
most generally, a measure of how difficult it is to
perform the computation.
That is, it measures some aspect of the cost of
computation (in a general sense of cost).
Amount of resources required to do a computation.
Some of the most common complexity measures:
Time complexity: # of operations or steps required
Space complexity: # of memory bits reqd
Module #7 - Complexity
4/26/2013 4
Complexity Depends on Input
Most algorithms have different complexities
for inputs of different sizes.
E.g. searching a long list typically takes more
time than searching a short one.
Therefore, complexity is usually expressed
as a function of the input length.
This function usually gives the complexity for
the worst-case input of any given length.
Module #7 - Complexity
4/26/2013 5
Complexity & Orders of Growth
Suppose algorithm A has worst-case time
complexity (w.c.t.c., or just time) f(n) for
inputs of length n, while algorithm B (for
the same task) takes time g(n).
Suppose that fee(g), also written .
Which algorithm will be fastest on all
sufficiently-large, worst-case inputs?
g f
Module #7 - Complexity
4/26/2013 6
Complexity analysis of max
procedure max(a
1
, a
2
, , a
n
: integers)
v := a
1
t
1

for i := 2 to n t
2

if a
i
> v then v := a
i
t
3

return v t
4
First, whats an expression for the exact total
worst-case time? (Not its order of growth.)

Times for
each
execution
of each
line.
Module #7 - Complexity
4/26/2013 7
Review 2.2: Complexity
Algorithmic complexity = cost of computation.
Focus on time complexity for our course.
Although space & energy are also important.
Characterize complexity as a function of input
size: Worst-case, best-case, or average-case.
Use orders-of-growth notation to concisely
summarize the growth properties of complexity
functions.
Module #7 - Complexity
4/26/2013 8
Example 3: Binary Search
procedure binary search (x:integer, a
1
, a
2
, , a
n
:
distinct integers, sorted smallest to largest)
i := 1
j := n
while i<j begin
m := (i+j)/2
if x>a
m
then i := m+1 else j := m
end
if x = a
i
then location := i else location := 0
return location
O(1)
O(1)
O(1)
Key question:
How many loop iterations?
Module #7 - Complexity
4/26/2013 9
Binary search analysis
Suppose that n is a power of 2, i.e., -k: n=2
k
.
Original range from i=1 to j=n contains n items.
Each iteration: Size ji+1 of range is cut in ~half.
Loop terminates when size of range is 1=2
0
(i=j).
Therefore, the number of iterations is:
k = log
2
n = O(log
2
n)= O(log n)
Even for n=2
k
(not an integral power of 2),
time complexity is still O(log
2
n) = O(log n).
Module #7 - Complexity
4/26/2013 10
Names for some orders of growth
O(1) Constant
O(log
c
n) Logarithmic (same order c)
O(log
c
n) Polylogarithmic
O(n) Linear
O(n
c
) Polynomial (for any c)
O(c
n
) Exponential (for c>1)
O(n!) Factorial
(With c
a constant.)
Module #7 - Complexity
4/26/2013 11
Problem Complexity
The complexity of a computational problem
or task is (the order of growth of) the
complexity of the algorithm with the lowest
order of growth of complexity for solving
that problem or performing that task.
E.g. the problem of searching an ordered list
has at most logarithmic time complexity.
(Complexity is O(log n).)
Module #7 - Complexity
Upper and lower bounds
Upper bound: If algorithm A solves the
problem P, and has complexity O(f), then
we say P has an upper bound of O(f).
Lower bound: If we show that NO
algorithm can solve the problem P in O(f)
time (or that any algorithm to solve P has to
take at least O(f) time), then we say P has a
lower bound of (f).
4/26/2013 12
Module #7 - Complexity
4/26/2013 13
Tractable vs. intractable
A problem or algorithm with at most polynomial
time complexity is considered tractable.
P is the set of all tractable problems.
A problem or algorithm that has complexity
greater than polynomial is considered intractable
(or infeasible).
Note that n
1,000,000
is technically tractable, but
really very hard. n
log log log n
is technically
intractable, but easy. Such cases are rare though.
Module #7 - Complexity
4/26/2013 14
Computer Time Examples
Assume time
= 1 ns (10
9

second) per
op, problem
size = n bits,
and #ops is a
function of n,
as shown.
#ops(n) n=10 n=10
6
log
2
n 3.3 ns 19.9 ns
n 10 ns 1 ms
n log
2
n 33 ns 19.9 ms
n
2
100 ns 16 m 40 s
2
n

1.024 s
10
301,004.5

Gyr
n! 3.63 ms Ouch!

(125 kB)
(1.25 bytes)
Module #7 - Complexity
4/26/2013 15
Key Things to Know
Definitions of algorithmic complexity, time
complexity, worst-case time complexity.
Names of specific orders of growth of
complexity.
How to analyze the worst case, best case, or
average case order of growth of time
complexity for simple algorithms.

You might also like