Scribd
Upload
26 views26 pages

Week 5 Intro - Algo

Uploaded by

Farhana Zainol Abidin
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PPTX, PDF, TXT or read online on Scribd
26 views26 pages

Week 5 Intro - Algo

Uploaded by

Farhana Zainol Abidin
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PPTX, PDF, TXT or read online on Scribd
You are on page 1/ 26

Analysis of Algorithms

Analysis of Algorithms / Slide 2

Introduction
 What is Algorithm?
 a clearly specified set of simple instructions to be followed to
solve a problem
 Takes a set of values, as input and
 produces a value, or set of values, as output
 May be specified
 In English
 As a computer program
 As a pseudo-code

 Data structures
 Methods of organizing data
 Program = algorithms + data structures
Analysis of Algorithms / Slide 3

Introduction
 Why need algorithm analysis ?
 writing a working program is not good enough
 The program may be inefficient!
 If the program is run on a large data set, then the
running time becomes an issue
Analysis of Algorithms / Slide 4

Example: Selection Problem


 Given a list of N numbers, determine the kth
largest, where k  N.
 Algorithm 1:
(1) Read N numbers into an array
(2) Sort the array in decreasing order by some
simple algorithm
(3) Return the element in position k
Analysis of Algorithms / Slide 5

Example: Selection Problem…


 Algorithm 2:
(1) Read the first k elements into an array and sort
them in decreasing order
(2) Each remaining element is read one by one
 If smaller than the kth element, then it is ignored
 Otherwise, it is placed in its correct spot in the array,
bumping one element out of the array.
(3) The element in the kth position is returned as
the answer.
Analysis of Algorithms / Slide 6

Example: Selection Problem…


 Which algorithm is better when
 N =100 and k = 100?
 N =100 and k = 1?
 What happens when N = 1,000,000 and k =
500,000?
 There exist better algorithms
Analysis of Algorithms / Slide 7

Algorithm Analysis
 We only analyze correct algorithms
 An algorithm is correct
 If, for every input instance, it halts with the correct output
 Incorrect algorithms
 Might not halt at all on some input instances
 Might halt with other than the desired answer
 Analyzing an algorithm
 Predicting the resources that the algorithm requires
 Resources include
 Memory
 Communication bandwidth
 Computational time (usually most important)
Analysis of Algorithms / Slide 8

Algorithm Analysis…
 Factors affecting the running time
 computer
 compiler
 algorithm used
 input to the algorithm
 The content of the input affects the running time
 typically, the input size (number of items in the input) is the main
consideration
 E.g. sorting problem  the number of items to be sorted
 E.g. multiply two matrices together  the total number of

elements in the two matrices


 Machine model assumed
 Instructions are executed one after another, with no
concurrent operations  Not parallel computers
Analysis of Algorithms / Slide 9

Example
N
 Calculate i 3

i 1

1
1
2 2N+2
3 4N
4 1

 Lines 1 and 4 count for one unit each


 Line 3: executed N times, each time four units
 Line 2: (1 for initialization, N+1 for all the tests, N for
all the increments) total 2N + 2
 total cost: 6N + 4  O(N)
Analysis of Algorithms / Slide 10

Worst- / average- / best-case


 Worst-case running time of an algorithm
 The longest running time for any input of size n
 An upper bound on the running time for any input
 guarantee that the algorithm will never take longer
 Example: Sort a set of numbers in increasing order; and the
data is in decreasing order
 The worst case can occur fairly often
 E.g. in searching a database for a particular piece of information
 Best-case running time
 sort a set of numbers in increasing order; and the data is
already in increasing order
 Average-case running time
 May be difficult to define what “average” means
Analysis of Algorithms / Slide 11

Running-time of algorithms
 Bounds are for the algorithms, rather than
programs
 programs are just implementations of an algorithm,
and almost always the details of the program do
not affect the bounds

 Bounds are for algorithms, rather than


problems
 A problem can be solved with several algorithms,
some are more efficient than others
Analysis of Algorithms / Slide 12

Growth Rate

 The idea is to establish a relative order among functions


for large n
  c , n0 > 0 such that f(N)  c g(N) when N  n0
 f(N) grows no faster than g(N) for “large” N
Analysis of Algorithms / Slide 13

Asymptotic notation: Big-Oh


 f(N) = O(g(N))
 There are positive constants c and n 0 such
that
f(N)  c g(N) when N  n0

 The growth rate of f(N) is less than or equal to


the growth rate of g(N)
 g(N) is an upper bound on f(N)
Analysis of Algorithms / Slide 14

Big-Oh: example
 Let f(N) = 2N2. Then
 f(N) = O(N4)
 f(N) = O(N3)
 f(N) = O(N2) (best answer, asymptotically tight)

 O(N2): reads “order N-squared” or “Big-Oh N-squared”


Analysis of Algorithms / Slide 15

Big Oh: more examples


 N2 / 2 – 3N = O(N2)
 1 + 4N = O(N)
 7N2 + 10N + 3 = O(N2) = O(N3)
 log10 N = log2 N / log2 10 = O(log2 N) = O(log N)
 sin N = O(1); 10 = O(1), 1010 = O(1)


N
i 1
i  N N O( N 2 )

i 1
N 2 2 3
i  N N O ( N )
 log N + N = O(N)
 logk N = O(N) for any constant k
 N = O(2N), but 2N is not O(N)
 210N is not O(2N)
Analysis of Algorithms / Slide 16

Math Review: logarithmic functions


a
x b iff log x b a
log ab log a  log b
log m b
log a b 
log m a
b
log a b log a
a log n n log a
b b b
log a (log a ) log a
d log e x 1

dx x
Analysis of Algorithms / Slide 17

Some rules
When considering the growth rate of a function using
Big-Oh
 Ignore the lower order terms and the coefficients of
the highest-order term
 No need to specify the base of logarithm
 Changing the base from one constant to another changes the
value of the logarithm by only a constant factor

 If T1(N) = O(f(N) and T2(N) = O(g(N)), then


 T1(N) + T2(N) = max(O(f(N)), O(g(N))),
 T1(N) * T2(N) = O(f(N) * g(N))
Analysis of Algorithms / Slide 18

Big-Omega

  c , n0 > 0 such that f(N)  c g(N) when N  n0


 f(N) grows no slower than g(N) for “large” N
Analysis of Algorithms / Slide 19

Big-Omega

 f(N) = (g(N))
 There are positive constants c and n 0 such
that
f(N)  c g(N) when N  n0

 The growth rate of f(N) is greater than or


equal to the growth rate of g(N).
Analysis of Algorithms / Slide 20

Big-Omega: examples
 Let f(N) = 2N2. Then
 f(N) = (N)
 f(N) = (N2) (best answer)
Analysis of Algorithms / Slide 21

f(N) = (g(N))

 the growth rate of f(N) is the same as the growth rate


of g(N)
Analysis of Algorithms / Slide 22

Big-Theta

 f(N) = (g(N)) iff


f(N) = O(g(N)) and f(N) = (g(N))
 The growth rate of f(N) equals the growth rate
of g(N)
 Example: Let f(N)=N2 , g(N)=2N2
 Since f(N) = O(g(N)) and f(N) = (g(N)),
thus f(N) = (g(N)).
 Big-Theta means the bound is the tightest
possible.
Analysis of Algorithms / Slide 23

Typical Growth Rates


Analysis of Algorithms / Slide 24

Growth rates …
 Doubling the input size
 f(N) = c  f(2N) = f(N) = c
 f(N) = log N  f(2N) = f(N) + log 2
 f(N) = N  f(2N) = 2 f(N)
 f(N) = N2  f(2N) = 4 f(N)
 f(N) = N3  f(2N) = 8 f(N)
 f(N) = 2N  f(2N) = f2(N)
 Advantages of algorithm analysis
 To eliminate bad algorithms early
 pinpoints the bottlenecks, which are worth coding
carefully
Analysis of Algorithms / Slide 25

General Rules
 For loops
 at most the running time of the statements inside
the for-loop (including tests) times the number of
iterations.
 Nested for loops

 the running time of the statement multiplied by the


product of the sizes of all the for-loops.
 O(N2)
Analysis of Algorithms / Slide 26

General rules (cont’d)


 Consecutive statements

 These just add


 O(N) + O(N2) = O(N2)
 If S1
Else S2
 never more than the running time of the test plus the larger of
the running times of S1 and S2.

You might also like