Scribd
Upload
3 views

Chapter 3-Algorithms

Algorithms

Uploaded by

Ahmed Mohamed
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
3 views

Chapter 3-Algorithms

Algorithms

Uploaded by

Ahmed Mohamed
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 21

MATH301-

DISCRETE MATHEMATICS
Chapter 3: Algorithms

Dr. Nahid Sultana


Email: [email protected]

Copyright © Nahid Sultana 2020-2021


10/4/2023
Chapter Summary
2

3.1 Algorithms
➢ Example Algorithms
➢ Algorithmic Paradigms
3.2 Growth of Functions
➢ Big-O and other Notation
3.3 Complexity of Algorithms

Copyright © Nahid Sultana 2020-2021 10/4/2023


Algorithms
3

An algorithm is a finite set of precise instructions for performing a


computation or for solving a problem.
Example: Describe an algorithm for finding the maximum value in a
finite sequence of integers.
Solution: Perform the following steps:
1. Set the temporary maximum equal to the first integer in the sequence.
2. Compare the next integer in the sequence to the temporary maximum.
◼ If it is larger than the temporary maximum, set the temporary maximum
equal to this integer.
3. Repeat the previous step if there are more integers. If not, stop.
4. When the algorithm terminates, the temporary maximum is the largest
integer in the sequence.
Copyright © Nahid Sultana 2020-2021 10/4/2023
3.2 Growth of Functions
4

❖Big-O Notation
❖Big-O Estimates for Important Functions
❖Big-Omega and Big-Theta Notation

Copyright © Nahid Sultana 2020-2021 10/4/2023


The Growth of Functions
5

❖ In both computer science and in mathematics, there are many times when we
care about how fast a function grows.
❖ In computer science, we want to understand how quickly an algorithm can
solve a problem as the size of the input grows.
❖ We can compare the efficiency of two different algorithms for solving the
same problem.
❖ We can also determine whether it is practical to use a particular algorithm
as the input grows.
❖ Two of the areas of mathematics where questions about the growth of
functions are studied are:
❖ Number theory (covered in Chapter 4)

❖ Combinatorics (covered in Chapters 6 and 8)

Copyright © Nahid Sultana 2020-2021 10/4/2023


Big-O Notation
6

Definition: Let f and g be arbitrary functions defined on ℝ or a


subset of ℝ. We say that f(x) is order g(x) if there exists a real
number k and a positive constant c such that
|f(x)| ≤ c |g(x)| for all x ≥ k.

➢ This is read as “f(x) is big-O of g(x)” or “g asymptotically


dominates f.”
➢ The constants c and k are called witnesses to the relationship
f(x) is O(g(x)).

Copyright © Nahid Sultana 2020-2021 10/4/2023


Big-O Notation (Cont…)
7

Example: Show that f(x)= x2+2x+1 is O(x2)?

Solution: when x > 1, then x2 > x and x2 > 1.


Now x2+2x+1 < x2 + 2x2+ x2
⇒x2+2x+1 < 4 x2 , when x >1
Take c = 4 and k = 1 as witnesses to show that f(x) is O(x2).

Note: You can choose other values of witness.

❖ Alternatively, when x > 2, we have x2 > 2x and x2 >1.


Now x2+2x+1 < x2 +x2+ x2
⇒x2+2x+1 < 3 x2 , when x > 2.
Take c = 3 and k = 2 as witnesses instead to show that f(x) is O(x2).
Copyright © Nahid Sultana 2020-2021 10/4/2023
Illustration of Big-O Notation
8

is

Copyright © Nahid Sultana 2020-2021 10/4/2023


Big-O Notation
9

❖ If f(x) is O(g(x)) and h(x) is larger than g(x) for all positive real
numbers, then f(x) is O(h(x)).

❖ Note that if |f(x) ≤ c|g(x)| for x > k and if |h(x)|>|g(x)|


for all x, then |f(x) ≤ c |h(x)| if x > k. Hence, f(x) is O(h(x)).

Copyright © Nahid Sultana 2020-2021 10/4/2023


Using the Definition of Big-O Notation
10

Example: Show that 7x2 is O(x3).


Solution: When x > 7, then x2 >7x and x3 > 7x2 i.e. 7x2 < x3.
Take C =1 and k = 7 as witnesses to establish that 7x2 is O(x3).

Example: Show that n2 is not O(n).


Solution: Suppose there are constants C and k for which n2 ≤ Cn,
whenever n > k. Then dividing both sides of n2 ≤ Cn by n,
then n ≤ C must hold for all n > k. A contradiction!

Copyright © Nahid Sultana 2020-2021 10/4/2023


Big-O Estimates for Polynomials
11

Example: Let
where are real numbers with an ≠0.
Then f(x) is O(xn).
Uses triangle inequality, an
Proof: |f(x)| = |anxn + an-1 xn-1 + ∙∙∙ + a1x1 + a1exercise
| in Section 1.8.
≤ |an|xn + |an-1| xn-1 + ∙∙∙ + |a1|x1 + |a1|
Assuming x > 1
= xn (|an| + |an-1| /x + ∙∙∙ + |a1|/xn-1 + |a1|/ xn)
≤ xn (|an| + |an-1| + ∙∙∙ + |a1|+ |a1|)
➢ Take C = |an| + |an-1| + ∙∙∙ + |a1|+ |a1| and k = 1. Then f(x)
is O(xn).
➢ The leading term anxn of a polynomial dominates its growth.
Copyright © Nahid Sultana 2020-2021 10/4/2023
Big-O Estimates for some Important
12
Functions
Example: Use big-O notation to estimate the sum of the first n
positive integers.
Solution:

Example: Use big-O notation to estimate the factorial function


Solution:

Copyright © Nahid Sultana 2020-2021 10/4/2023 Continued →


Big-O Estimates for some Important
13
Functions
Example: Use big-O notation to estimate log n!

Solution: Given that (previous slide)


then .

Hence, log(n!) is O(n∙log(n)) taking C = 1 and k = 1.

Copyright © Nahid Sultana 2020-2021 10/4/2023


Display of Growth of Functions
14

Note the difference in behavior of functions as n gets larger


Copyright © Nahid Sultana 2020-2021 10/4/2023
Combinations of Functions
15

➢ If f1 (x) is O(g1(x)) and f2 (x) is O(g2(x)) then


( f1 + f2 )(x) is O(max(|g1(x) |,|g2(x) |)).

➢ If f1 (x) and f2 (x) are both O(g(x)) then


( f1 + f2 )(x) is O(g(x)).

➢ If f1 (x) is O(g1(x)) and f2 (x) is O(g2(x)) then


( f1 f2 )(x) is O(g1(x)g2(x)).

Copyright © Nahid Sultana 2020-2021 10/4/2023


Ω is the upper case
Big-Omega Notation version of the lower
case Greek letter ω.
16

Definition: Let f and g be functions from the set of integers or the


set of real numbers to the set of real numbers.
We say that if there are constants C and k such
that when x > k.
➢ We say that “f(x) is big-Omega of g(x).”
➢ Big-O gives an upper bound on the growth of a function, while
Big-Omega gives a lower bound. Big-Omega tells us that a
function grows at least as fast as another.
➢ f(x) is Ω(g(x)) if and only if g(x) is O(f(x)). This follows from the
definitions.
Copyright © Nahid Sultana 2020-2021 10/4/2023
Big-Omega Notation
17

Example: Show that is


where .

Solution: for all positive real


numbers x.

➢ Is it also the case that is ?

Copyright © Nahid Sultana 2020-2021 10/4/2023


Θ is the upper case
Big-Theta Notation version of the lower
case Greek letter θ.
18

Definition: Let f and g be functions from the set of integers or the set
of real numbers to the set of real numbers. The function
if and .

➢ We say that “f is big-Theta of g(x)” and also that “f(x) is of order


g(x)” and also that “f(x) and g(x) are of the same order.”

➢ if and only if there exists constants C1 , C2 and k


such that C1g(x) < f(x) < C2 g(x) if x > k. This follows from the
definitions of big-O and big-Omega.

Copyright © Nahid Sultana 2020-2021 10/4/2023


Big-Theta Notation
19

Example: Show that f(x) = 3x2 + 8x log x is Θ(x2).


Solution:
➢ 3x2 + 8x log x ≤ 11x2 for x > 1,
since 0 ≤ 8x log x ≤ 8x2 .
➢ Hence, 3x2 + 8x log x is O(x2).

➢ x2 is clearly O(3x2 + 8x log x).


➢ Hence, 3x2 + 8x log x is Θ(x2).

Copyright © Nahid Sultana 2020-2021 10/4/2023


Big-Theta Notation
20

➢ Note that if and only if it is the case that


and .

Copyright © Nahid Sultana 2020-2021 10/4/2023


Big-Theta Estimates for Polynomials
21

Theorem: Let
where are real numbers with an ≠0.
Then f(x) is of order xn (or Θ(xn)).

Example:
The polynomial is order of x5 (or Θ(x5)).

The polynomial is order


of x199 (or Θ(x199) ).

Copyright © Nahid Sultana 2020-2021 10/4/2023

You might also like