Discrete Structures

Chapter 3
The Fundamentals: Algorithms, the
Integers, and Matrices

Prof. Dr. Malik Sikander Hayat Khiyal

Faculty of Computer Science,
Preston University, Islamabad Campus,
85, Street 3, Sector H-8/1,
Islamabad, PAKISTAN.
[email protected]
01/31/24 Discrete Mathematics Chapter 3 1
 Discrete Structures Contents
Chapter 1. The Foundations: Logic and Proofs
Chapter 2. Basic Structures: Sets, Functions, Sequences, and Sums
Chapter 3. The Fundamentals: Algorithms, the Integers, and Matrices
Chapter 4. Number Theory and Cryptography
Chapter 5. Induction and Recursion
Chapter 6. Counting
Chapter 7. Discrete Probability
Chapter 8. Advanced Counting Technique
Chapter 9. Relations
Chapter 10. Graphs
Chapter 11. Trees
Chapter 12. Boolean Algebra

01/31/24 Discrete Mathematics Chapter 3 2

 Algorithms
What is an algorithm?

An algorithm is a finite set of precise instructions for

performing a computation or for solving a problem.

This is a rather vague definition. You will get to know a

more precise and mathematically useful definition in
design and analysis of Algorithm course.

But this one is good enough for now…

01/31/24 Discrete Mathematics Chapter 3 3

 Algorithms
Properties of algorithms:

Input from a specified set,

Output from a specified set (solution),
Definiteness of every step in the computation,
Correctness of output for every possible input,
Finiteness of the number of calculation steps,
Effectiveness of each calculation step and
Generality for a class of problems.

01/31/24 Discrete Mathematics Chapter 3 4

 Algorithm Examples
We will use a pseudocode to specify algorithms, which
slightly reminds us of Basic and Pascal.
Example: an algorithm that finds the maximum element in
a finite sequence

procedure max(a1, a2, …, an: integers)

max := a1
for i := 2 to n
if max < ai then max := ai
{max is the largest element}

01/31/24 Discrete Mathematics Chapter 3 5

 Algorithm Examples
Another example: a linear search algorithm, that is, an
algorithm that linearly searches a sequence for a particular
element.
procedure linear_search(x: integer; a1, a2, …, an:
integers)
i := 1
while (i  n and x  ai)
i := i + 1
if i  n then location := i
else location := 0
{location is the subscript of the term that equals x, or is
zero if x is not found}
01/31/24 Discrete Mathematics Chapter 3 6
 Algorithm Examples

If the terms in a sequence are ordered, a binary search

algorithm is more efficient than linear search.

The binary search algorithm iteratively restricts the

relevant search interval until it closes in on the position of
the element to be located.

01/31/24 Discrete Mathematics Chapter 3 7

 Algorithm Examples

binary search for the letter ‘j’

search interval

a c d f g h j l m o p r s u v x z

center element

01/31/24 Discrete Mathematics Chapter 3 8

 Algorithm Examples

binary search for the letter ‘j’

search interval

a c d f g h j l m o p r s u v x z

center element

01/31/24 Discrete Mathematics Chapter 3 9

 Algorithm Examples

binary search for the letter ‘j’

search interval

a c d f g h j l m o p r s u v x z

center element

01/31/24 Discrete Mathematics Chapter 3 10

 Algorithm Examples

binary search for the letter ‘j’

search interval

a c d f g h j l m o p r s u v x z

center element

01/31/24 Discrete Mathematics Chapter 3 11

 Algorithm Examples

binary search for the letter ‘j’

search interval

a c d f g h j l m o p r s u v x z

center element

found !
01/31/24 Discrete Mathematics Chapter 3 12
 Algorithm Examples
procedure binary_search(x: integer; a1, a2, …, an:
integers)
i := 1 {i is left endpoint of search interval}
j := n {j is right endpoint of search interval}
while (i < j)
begin
m := (i + j)/2
if x > am then i := m + 1
else j := m
end
if x = ai then location := i
else location := 0
{location is the subscript of the term that equals x, or is
zero if x is not found}
01/31/24 Discrete Mathematics Chapter 3 13
 Algorithm Examples
Sorting:
Use the bubble sort to put 3, 2, 4, 1, 5 into increasing
order.

01/31/24 Discrete Mathematics Chapter 3 14

 Algorithm Examples
Sorting Algorithm

01/31/24 Discrete Mathematics Chapter 3 15

 Algorithm Examples
Sorting Algorithm
THE INSERTION SORT
Use the insertion sort to put the elements of the list 3, 2, 4, 1, 5
in increasing order.
Solution: The insertion sort first compares 2 and 3. Because 3 > 2, it places
2 in the first position, producing the list 2, 3, 4, 1, 5 (the sorted part of
the list is shown in color). At this point, 2 and 3 are in the correct order.
Next, it inserts the third element, 4, into the already sorted part of the list
by making the comparisons 4 > 2 and 4 > 3. Because 4 > 3, 4 remains in
the third position. At this point, the list is 2, 3, 4, 1, 5 and we know that
the ordering of the first three elements is correct. Next, we find the
correct place for the fourth element, 1, among the already sorted
elements, 2, 3, 4. Because 1 < 2, we obtain the list 1, 2, 3, 4, 5. Finally,
we insert 5 into the correct position by successively comparing it to 1, 2,
3, and 4. Because 5 > 4, it stays at the end of the list, producing the
correct order for the entire list
01/31/24 Discrete Mathematics Chapter 3 16
 Algorithm Examples
Sorting Algorithm

01/31/24 Discrete Mathematics Chapter 3 17

 Complexity

In general, we are not so much interested in the time and

space complexity for small inputs.

For example, while the difference in time complexity

between linear and binary search is meaningless for a
sequence with n = 10, it is gigantic for n = 230.

01/31/24 Discrete Mathematics Chapter 3 18

 Complexity

For example, let us assume two algorithms A and B that

solve the same class of problems.
The time complexity of A is 5,000n, the one for B is
1.1n for an input with n elements.
For n = 10, A requires 50,000 steps, but B only 3, so B
seems to be superior to A.
For n = 1000, however, A requires 5,000,000 steps, while
B requires 2.51041 steps.

01/31/24 Discrete Mathematics Chapter 3 19

 Complexity

This means that algorithm B cannot be used for large

inputs, while algorithm A is still feasible.

So what is important is the growth of the complexity

functions.

The growth of time and space complexity with

increasing input size n is a suitable measure for the
comparison of algorithms.

01/31/24 Discrete Mathematics Chapter 3 20

 Complexity
Comparison: time complexity of algorithms A and B

Input Size Algorithm A Algorithm B

n 5,000n 1.1n
10 50,000 3
100 500,000 13,781
1,000 5,000,000 2.5x1041
1,000,000 5.109 4.8x1041392

01/31/24 Discrete Mathematics Chapter 3 21

 The Growth of Functions
The growth of functions is usually described using the
big-O notation.

Definition: Let f and g be functions from the integers or

the real numbers to the real numbers.
We say that f(x) is O(g(x)) if there are constants C and k
such that
|f(x)|  C|g(x)|
whenever x > k.

01/31/24 Discrete Mathematics Chapter 3 22

 The Growth of Functions

When we analyze the growth of complexity functions,

f(x) and g(x) are always positive.
Therefore, we can simplify the big-O requirement to
f(x)  Cg(x) whenever x > k.

If we want to show that f(x) is O(g(x)), we only need to

find one pair (C, k) (which is never unique).

01/31/24 Discrete Mathematics Chapter 3 23

 The Growth of Functions
The idea behind the big-O notation is to establish an
upper boundary for the growth of a function f(x) for
large x.
This boundary is specified by a function g(x) that is
usually much simpler than f(x).
We accept the constant C in the requirement
f(x)  Cg(x) whenever x > k,
because C does not grow with x.
We are only interested in large x, so it is OK if
f(x) > Cg(x) for x  k.

01/31/24 Discrete Mathematics Chapter 3 24

 The Growth of Functions
Example: 1
Show that f(x) = x2 + 2x + 1 is O(x2).

For x > 1 we have:

x2 + 2x + 1  x2 + 2x2 + x2
 x2 + 2x + 1  4x2
Therefore, for C = 4 and k = 1:
f(x)  Cx2 whenever x > k.

 f(x) is O(x2).

01/31/24 Discrete Mathematics Chapter 3 25

 The Growth of Functions
Alternatively, we can estimate the size of f (x) when x>2.
When x>2, we have 2x≤ x2 and 1 ≤ x2. Consequently, if x >
2, we have
0 ≤ x2 + 2x + 1 ≤ x2 + x2 + x2 = 3x2.
It follows that C = 3 and k = 2 are also witnesses to the
relation f (x) is O(x2).
Observe that in the relationship “f (x) is O(x2),” x2 can be
replaced by any function that has larger values than x2 for
all x ≥ k for some positive real number k. For example, f
(x) is O(x3), f (x) is O(x2+x+7), and so on.
It is also true that x is O(x2+2x+1), because x2< x2+2x+1
whenever x > 1. This means that C = 1 and k = 1 are
witnesses to the relationship x2 is O(x2 + 2x + 1).
01/31/24 Discrete Mathematics Chapter 3 26
 The Growth of Functions

01/31/24 Discrete Mathematics Chapter 3 27

 The Growth of Functions
When f (x) is O(g(x)), and h(x) is a function that has larger
absolute values than g(x) does for sufficiently large values
of x, it follows that f (x) is O(h(x)). In other words, the
function g(x) in the relationship f (x) is O(g(x)) can be
replaced by a function with larger absolute values. To see
this, note that if
|f (x)|≤ C |g(x)| if x > k,
and if |h(x) | > |g(x)| for all x > k, then
|f (x) | ≤ C |h(x) | if x > k. Hence, f (x) is O(h(x)).
When big-O notation is used, the function g in the
relationship f (x) is O(g(x)) is often chosen to have the
smallest growth rate of the functions belonging to a set of
reference functions, such as functions of the form xn,
where n is a positive real number.
01/31/24 Discrete Mathematics Chapter 3 28
 The Growth of Functions

Example 2 illustrates how big-O notation is used to

estimate the growth of functions.

01/31/24 Discrete Mathematics Chapter 3 29

 The Growth of Functions

Question: If f(x) is O(x2), is it also O(x3)?

Yes. x3 grows faster than x2, so x3 grows also faster than

f(x).

Therefore, we always have to find the smallest simple

function g(x) for which f(x) is O(g(x)).

01/31/24 Discrete Mathematics Chapter 3 30

 The Growth of Functions
THEOREM 1
Let f (x) = anxn + an−1xn−1 + ⋯ + a1x + a0, where a0, a1, … , an−1, an
are real numbers. Then f (x) is O(xn).

01/31/24 Discrete Mathematics Chapter 3 31

 The Growth of Functions
EXAMPLE . How can big-O notation be used
to estimate the sum of the first n positive integers?
Solution: Because each of the integers in the sum of
the first n positive integers does not exceed
n, it follows that
1 + 2 + ⋯ + n ≤ n + n + ⋯ + n = n2.
From this inequality it follows that 1 + 2 + 3 + ⋯ + n
is O(n2), taking C = 1 and k = 1 as wit- nesses. (In
this example the domains of the functions in the big-
O relationship are the set of positive integers.)

01/31/24 Discrete Mathematics Chapter 3 32

 The Growth of Functions
EXAMPLE 6 Give big-O estimates for the
factorial function and the logarithm of the factorial
function, where the factorial function f (n) = n! is
defined by
n! = 1 ⋅ 2 ⋅ 3 ⋅ ⋯ ⋅ n
whenever n is a positive integer, and 0! = 1. For
example,
1! = 1, 2! = 1 ⋅ 2 = 2, 3! = 1 ⋅ 2 ⋅ 3 = 6,
4! = 1 ⋅ 2 ⋅ 3 ⋅ 4 = 24.
Note that the function n! grows rapidly. For
instance, 20! = 2,432,902,008,176,640,000.
01/31/24 Discrete Mathematics Chapter 3 33
 The Growth of Functions
Solution: A big-O estimate for n! can be obtained by
noting that each term in the product does not exceed n.
Hence,
n! = 1 ⋅ 2 ⋅ 3 ⋅ ⋯ ⋅ n
≤n⋅n⋅n⋅⋯⋅n
= nn.
This inequality shows that n! is O(nn), taking C = 1 and k =
1 as witnesses. Taking logarithms of both sides of the
inequality established for n!, we obtain
log n! ≤ log nn = n log n.
This implies that log n! is O(n log n), again taking C = 1
and k = 1 as witnesses

01/31/24 Discrete Mathematics Chapter 3 34

 The Growth of Functions
“Popular” functions g(n) are
n log n, 1, 2n, n2, n!, n, n3, log n

Listed from slowest to fastest growth:

1
log n
n
n log n
n2
n3
2n
n!

01/31/24 Discrete Mathematics Chapter 3 35

 The Growth of Functions

01/31/24 Discrete Mathematics Chapter 3 36

 The Growth of Functions

A problem that can be solved with polynomial worst-case

complexity is called tractable.

Problems of higher complexity are called intractable.

Problems that no algorithm can solve are called

unsolvable.

01/31/24 Discrete Mathematics Chapter 3 37

 Useful Rules for Big-O
For any polynomial f(x) = anxn + an-1xn-1 + … + a0, where a0,
a1, …, an are real numbers,
f(x) is O(xn).

If f1(x) is O(g1(x)) and f2(x) is O(g2(x)), then

(f1 + f2)(x) is O(max(g1(x), g2(x)))

If f1(x) is O(g(x)) and f2(x) is O(g(x)), then

(f1 + f2)(x) is O(g(x)).

If f1(x) is O(g1(x)) and f2(x) is O(g2(x)), then

(f1f2)(x) is O(g1(x) g2(x)).
01/31/24 Discrete Mathematics Chapter 3 38
 The Growth of Functions

01/31/24 Discrete Mathematics Chapter 3 39

 The Growth of Functions

EXAMPLE. Give a big-O estimate for f (n) = 3nlog(n!) +

(n2 + 3) log n, where n is a positive integer.
Solution: First, the product 3n log(n!) will be estimated.
We know that log(n!) is O(nlogn). Using this estimate and
the fact that 3n is O(n), product gives the estimate that
3nlog(n!) is O(n2log n).
Next, the product (n2 + 3)log n will be estimated. Because
(n2+ 3) < 2n2 when n > 2, it follows that n2 + 3 is O(n2).
Thus, it follows that (n2 + 3) log n is O(n2 log n).
To combine the two big-O estimates for the products
shows that f (n) = 3nlog(n!) + (n2 + 3) log n is O(n2log n).

01/31/24 Discrete Mathematics Chapter 3 40

 The Growth of Functions
EXAMPLE. Give a big-O estimate for f (x) = (x + 1)
log(x2 + 1) + 3x2.
Solution: First, a big-O estimate for (x + 1) log(x2 + 1)
will be found. Note that (x + 1) is O(x).
Furthermore, x + 1 ≤ 2x when x > 1. Hence,
log(x2+1) ≤ log(2x2) = log2 + logx2 = log2 + 2log x ≤
3log x,
if x > 2. This shows that log(x2 + 1) is O(log x).
It follows that (x + 1)log(x + 1) is O(x log x). Because
3x is O(x), Now f (x) is O(max(xlog x, x )). Because
xlogx ≤ x , for x > 1, it follows that f (x) is O(x2).
01/31/24 Discrete Mathematics Chapter 3 41
 The Growth of Functions
Definition: Let f and g be functions from the set
of integers or the set of real numbers to the set
of real numbers. We say that f (x) is
Ω(g(x)) if there are constants C and k with C
positive such that
|f (x)| ≥ |g(x) |
Whenever x > k. [This is read as “f (x) is big-
Omega of g(x).”]
There is a strong connection between big-O
and big-Omega notation. In particular, f (x) is
Ω(g(x)) if and only if g(x) is O(f (x)).
01/31/24 Discrete Mathematics Chapter 3 42
 The Growth of Functions

EXAMPLE. The function f (x) = 8x3 + 5x2 + 7 is

Ω(g(x)), where g(x) is the function g(x) = x3. This
is easy to see because f (x) = 8x3 + 5x2 + 7 ≥ 8x3
for all positive real numbers x. This is equivalent
to saying that g(x) = x is O(8x3 + 5x2 + 7), which
can be established directly by turning the
inequality around

01/31/24 Discrete Mathematics Chapter 3 43

 The Growth of Functions
Definition. Let f and g be functions from the set of
integers or the set of real numbers to the set of real
numbers. We say that f (x) is Θ(g(x)) if f (x) is O(g(x))
and f (x) is Ω(g(x)). When f (x) is Θ(g(x)), we say that f
is big-Theta of g(x), that f (x) is of order g(x), and that f
(x) and g(x) are of the same order.
Note that f (x) is Θ(g(x)) if and only if there are
positive real numbers C1 and C2 and a positive real
number k such that
C1|g(x)| ≤ |f(x)| ≤ C2|g(x)|
whenever x > k. The existence of the constants C1, C2,
and k tells us that f(x) is Ω(g(x)) and that f (x) is
O(g(x)), respectively.
01/31/24 Discrete Mathematics Chapter 3 44
 The Growth of Functions
EXAMPLE 13 Show that 3x2 + 8xlogx is
Θ(x2).
Solution:
Because 0 ≤ 8x log x ≤ 8x2, it follows that
3x2 + 8x log x ≤ 11x2 for x > 1.
Consequently, 3x2 + 8x log x is O(x2).
Clearly, x2 is O(3x2 + 8x log x).
Consequently, 3x2 + 8x log x is Θ(x2).

01/31/24 Discrete Mathematics Chapter 3 45

 Complexity
One measure of efficiency is the time used by a
computer to solve a problem using the algorithm,
when input values are of a specified size. A second
measure is the amount of computer memory
required to implement the algorithm when input
values are of a specified size.
Questions such as these involve the computational
complexity of the algorithm. An analysis of the
time required to solve a problem of a particular size
involves the time complexity of the algorithm. An
analysis of the computer memory required involves
the space complexity of the algorithm.
01/31/24 Discrete Mathematics Chapter 3 46
 Complexity
Time Complexity
The time complexity of an algorithm can be
expressed in terms of the number of operations
used by the algorithm when the input has a
particular size. The operations used to measure
time complexity can be the comparison of
integers, the addition of integers, the
multiplication of integers, the division of integers,
or any other basic operation

01/31/24 Discrete Mathematics Chapter 3 47

 Complexity
EXAMPLE. Describe the time complexity of Algorithm 1
for finding the maximum element in a finite set of integers.
Solution: The number of comparisons will be used as the
measure of the time complexity of the algorithm, because
comparisons are the basic operations used.
Because two comparisons are used for each of the second
through the nth elements and one more comparison is used
to exit the loop when i = n + 1, exactly 2(n−1)+1 = 2n − 1
comparisons are used whenever this algorithm is applied.
Hence, the algorithm for finding the maximum of a set of n
elements has time complexity Θ(n), measured in terms of
the number of comparisons used. Note that for this
algorithm the number of comparisons is independent of
particular input of n numbers.
01/31/24 Discrete Mathematics Chapter 3 48
 Complexity Examples
What does the following algorithm compute?
procedure -(a1, a2, …, an: integers)
m := 0
for i := 1 to n-1
for j := i + 1 to n
if |ai – aj| > m then m := |ai – aj|
{m is the maximum difference between any two numbers in
the input sequence}
Comparisons: n-1 + n-2 + n-3 + … + 1
= (n – 1)n/2 = 0.5n2 – 0.5n

Time complexity is O(n2).

01/31/24 Discrete Mathematics Chapter 3 49
 Complexity Examples
Another algorithm solving the same problem:
procedure max_diff(a1, a2, …, an: integers)
min := a1
max := a1
for i := 2 to n
if ai < min then min := ai
else if ai > max then max := ai
m := max - min
Comparisons: 2n - 2
Time complexity is O(n).
01/31/24 Discrete Mathematics Chapter 3 50