Discrete Structures

Davood, Pour Yousefian Barfeh

davoodpyb@yahoo.com

Lyceum of the Philippines

University – Laguna 1

College of Computer Studies

Enough Mathematical Appetizers!

•Algorithms

2
 Algorithms
•What is an algorithm?

•An algorithm is a finite set of precise instructions for performing a computation

or for solving a problem.

3
 Algorithms
• Properties of algorithms:

• Input from a specified set,

• Output from a specified set (solution),

• Definiteness of every step in the computation,

• Correctness of output for every possible input,

• Finiteness of the number of calculation steps,

• Effectiveness of each calculation step and

• Generality for a class of problems.

4
 Algorithm Examples
•We will use a pseudo code to specify algorithms, which slightly reminds us of
Basic and Pascal.

•Example: an algorithm that finds the maximum element in a finite sequence

•procedure max(a1, a2, …, an: integers)

•max := a1
•for i := 2 to n
• if max < ai then max := ai
•{max is the largest element}

5
 Algorithm Examples
•Another example: a linear search algorithm, that is, an algorithm that linearly
searches a sequence for a particular element.

procedure linear_search(x: integer; a1, a2, …, an: integers)

i := 1
while (i  n and x  ai)
i := i + 1
if i  n then location := i
else location := 0
{location is the subscript of the term that equals x, or is zero if x is not found}

6
 Algorithm Examples

•If the terms in a sequence are ordered, a binary search algorithm is more
efficient than linear search.

•The binary search algorithm iteratively restricts the relevant search interval
until it closes in on the position of the element to be located.

7
 Algorithm Examples

binary search for the letter ‘j’

search interval

a c d f g h j l m o p r s u v x z

center element

8
 Algorithm Examples

binary search for the letter ‘j’

search interval

a c d f g h j l m o p r s u v x z

center element

9
 Algorithm Examples

binary search for the letter ‘j’

search interval

a c d f g h j l m o p r s u v x z

center element

10
 Algorithm Examples

binary search for the letter ‘j’

search interval

a c d f g h j l m o p r s u v x z

center element

11
 Algorithm Examples

binary search for the letter ‘j’

search interval

a c d f g h j l m o p r s u v x z

center element
found !
12
 Algorithm Examples
procedure binary_search(x: integer; a1, a2, …, an: integers)

i := 1 {i is left endpoint of search interval}

j := n {j is right endpoint of search interval}
while (i < j)

begin
m := (i + j)/2
if x > am then i := m + 1
else j := m

end
if x = ai then location := i
else location := 0

{location is the subscript of the term that equals x, or is zero if x is not found}

13
 Complexity

•In general, we are not so much interested in the time and space
complexity for small inputs.

•For example, while the difference in time complexity between linear

and binary search is meaningless for a sequence with n = 10, it is
gigantic for n = 230.

14
 Complexity
•For example, let us assume two algorithms A and B that solve the
same class of problems.

•The time complexity of A is 5,000n, the one for B is 1.1n for an

input with n elements.

•For n = 10, A requires 50,000 steps, but B only 3, so B seems to be

superior to A.

•For n = 1000, however, A requires 5,000,000 steps, while B requires

41
2.510 steps.

15
 Complexity
•This means that algorithm B cannot be used for large inputs, while
algorithm A is still feasible.

•So what is important is the growth of the complexity functions.

•The growth of time and space complexity with increasing input size
n is a suitable measure for the comparison of algorithms.

16
 Complexity
• Comparison: time complexity of algorithms A and B

Input Size Algorithm A Algorithm B

n 5,000n n
1.1 
10 50,000 3
100 500,000 13,781
1,000 5,000,000 2.51041
1,000,000 5109 4.81041392

17
 The Growth of Functions
The growth of functions is usually described using the big-O notation.

Definition: Let f and g be functions from the integers or the real

numbers to the real numbers.
We say that f(x) is O(g(x)) if there are constants C and k such that

|f(x)|  C|g(x)|

whenever x > k.

18
 The Growth of Functions
When we analyze the growth of complexity functions, f(x) and g(x) are
always positive.

Therefore, we can simplify the big-O requirement to

f(x)  Cg(x) whenever x > k.

If we want to show that f(x) is O(g(x)), we only need to find one pair
(C, k) (which is never unique).

19
 The Growth of Functions
The idea behind the big-O notation is to establish an upper boundary
for the growth of a function f(x) for large x.

This boundary is specified by a function g(x) that is usually much

simpler than f(x).

We accept the constant C in the requirement

f(x)  Cg(x) whenever x > k,

because C does not grow with x.

We are only interested in large x, so it is OK if

f(x) > Cg(x) for x  k.

20
 The Growth of Functions
•Example:
Show that f(x) = x2 + 2x + 1 is O(x2).

For x > 1 we have:

x2 + 2x + 1  x2 + 2x2 + x2
 x2 + 2x + 1  4x2

Therefore, for C = 4 and k = 1:

f(x)  Cx2 whenever x > k.

 f(x) is O(x2).

21
 The Growth of Functions

•Question: If f(x) is O(x2), is it also O(x3)?

Yes. x3 grows faster than x2, so x3 grows also faster than f(x).

Therefore, we always have to find the smallest simple function g(x) for which
f(x) is O(g(x)).

22
 The Growth of Functions
“Popular” functions g(n) are
n log n, 1, 2n, n2, n!, n, n3, log n

Listed from slowest to fastest growth:

• 1
• log n
• n
• n log n
• n2
• n3
• n4
.
.
• 2n
• n!

23
 The Growth of Functions

•A problem that can be solved with polynomial worst-case complexity is called

tractable.

•Problems of higher complexity are called intractable.

•Problems that no algorithm can solve are called unsolvable.

•You will find out more about this in CMSC441.

24
 Useful Rules for Big-O
•For any polynomial f(x) = anxn + an-1xn-1 + … + a0, where a0, a1, …, an are
n n-1 0 0 1 n
real numbers,
f(x) is O(xn).

•If f1(x) is O(g1(x)) and f2(x) is O(g2(x)), then

(f1 + f2)(x) is O(max(g1(x), g2(x)))

•If f1(x) is O(g(x)) and f2(x) is O(g(x)), then

(f1 + f2)(x) is O(g(x)).

•If f1(x) is O(g1(x)) and f2(x) is O(g2(x)), then

(f1f2)(x) is O(g1(x) g2(x)).

25
 Complexity Examples
•What does the following algorithm compute?
procedure who_knows(a1, a2, …, an: integers)
m := 0
for i := 1 to n-1
for j := i + 1 to n
if |ai – aj| > m then m := |ai – aj|
{m is the maximum difference between any two numbers in the input sequence}
Comparisons: n-1 + n-2 + n-3 + … + 1
= (n – 1)n/2 = 0.5n2 – 0.5n

Time complexity is O(n2).

26
 Complexity Examples
•Another algorithm solving the same problem:

procedure max_diff(a1, a2, …, an: integers)

min := a1
max := a1
for i := 2 to n
if ai < min then min := ai
else if ai > max then max := ai
m := max - min
Comparisons: 2n - 2

Time complexity is O(n).

27
 Let us get into…

•Number Theory

28
 Introduction to Number Theory

•Number theory is about integers and their properties.

•We will start with the basic principles of

• divisibility,
• greatest common divisors,
• least common multiples, and
• modular arithmetic

•and look at some relevant algorithms.

29
 Division
•If a and b are integers with a  0, we say that
a divides b if there is an integer c so that b = ac.

•When a divides b we say that a is a factor of b and that b is a multiple of a.

•The notation
a |a |b b means that a divides b.

•We write when a does not divide b

30
 Divisibility Theorems
•For integers a, b, and c it is true that
• if a | b and a | c, then a | (b + c)
• Example: 3 | 6 and 3 | 9, so 3 | 15.

• if a | b, then a | bc for all integers c

• Example: 5 | 10, so 5 | 20, 5 | 30, 5 | 40, …

• if a | b and b | c, then a | c
• Example: 4 | 8 and 8 | 24, so 4 | 24.

31
 Primes
•A positive integer p greater than 1 is called prime if the only positive factors of
p are 1 and p.
•Note: 1 is not a prime

•A positive integer that is greater than 1 and is not prime is called composite.

•The fundamental theorem of arithmetic:

•Every positive integer can be written uniquely as the product of primes, where
the prime factors are written in order of increasing size.

32
 Primes
•Examples:

15 = 35
48 = 22223 = 2 3
4


17 = 17 
100 = 2
2255 = 2  5
2

512 = 222222222 = 2
9

515 = 5103
28 = 227

33
 Primes
•If n is a composite integer, then n has a prime divisor less than or equal √n .

•This is easy to see: if n is a composite integer, it must have at least two prime
divisors. Let the largest two be p1 and p2. Then p1p2 <= n.

•p1 and p2 cannot both be greater than √n , because then p1p2 > n.

34
 The Division Algorithm
•Let a be an integer and d a positive integer.
•Then there are unique integers q and r, with
0  r < d, such that a = dq + r.

•In the above equation,

• d is called the divisor,
• a is called the dividend,
• q is called the quotient, and
• r is called the remainder.

35
 The Division Algorithm
•Example:

•When we divide 17 by 5, we have

17 = 53 + 2

• 17 is the dividend,
• 5 is the divisor,
• 3 is called the quotient, and
• 2 is called the remainder.

36
 The Division Algorithm
•Another example:

•What happens when we divide -11 by 3 ?

•Note that the remainder cannot be negative.

-11 = 3(-4) + 1

• -11 is the dividend,

• 3 is the divisor,
• -4 is called the quotient, and
• 1 is called the remainder.

37
 Greatest Common Divisors
•Let a and b be integers, not both zero.
•The largest integer d such that d | a and d | b is called the greatest common
divisor of a and b.

•The greatest common divisor of a and b is denoted by gcd(a, b).

•Example 1: What is gcd(48, 72) ?
•The positive common divisors of 48 and 72 are
1, 2, 3, 4, 6, 8, 12, 16, and 24, so gcd(48, 72) = 24.

•Example 2: What is gcd(19, 72) ?

•The only positive common divisor of 19 and 72 is
1, so gcd(19, 72) = 1.

38
 Greatest Common Divisors
•Using prime factorizations:

a = p1a1  p2a2 … pnan , b = p1b1  p2b2 … pnbn ,

•where p1 < p2 < … < pn and ai, bi  N for 1  i  n

gcd(a, b) = p1min(a1, b1 ) p2min(a2, b2 ) … pnmin(an, bn )

•Example:

a = 60 = 22 31 51
b = 54 = 21  33 50
gcd(a, b) = 21 31 50 = 6
39
 Relatively Prime Integers
•Definition:

•Two integers a and b are relatively prime if

gcd(a, b) = 1.

•Examples:

•Are 15 and 28 relatively prime?

•Yes, gcd(15, 28) = 1.
•Are 55 and 28 relatively prime?
•Yes, gcd(55, 28) = 1.
•Are 35 and 28 relatively prime?
•No, gcd(35, 28) = 7.

40
 Relatively Prime Integers
•Definition:

•The integers a1, a2, …, an are pairwise relatively prime if gcd(ai, aj) = 1
whenever 1  i < j  n.

•Examples:

•Are 15, 17, and 27 pairwise relatively prime?

•No, because gcd(15, 27) = 3.

•Are 15, 17, and 28 pairwise relatively prime?

•Yes, because gcd(15, 17) = 1, gcd(15, 28) = 1 and gcd(17, 28) = 1.

41
 Least Common Multiples
•Definition:

•The least common multiple of the positive integers a and b is the smallest
positive integer that is divisible by both a and b.

•We denote the least common multiple of a and b by lcm(a, b).

•Examples:

lcm(3, 7) = 21
lcm(4, 6) = 12
lcm(5, 10) = 10
42
 Least Common Multiples
•Using prime factorizations:

a = p1a1 p2a2 … pnan , b = p1b1 p2b2 … pnbn ,

•where p1 < p2 < … < pn and ai, bi  N for 1  i  n

lcm(a, b) = p1max(a1, b1 ) p2max(a2, b2 ) … pnmax(an, bn )

•Example:

a = 60 = 22 31 51
b = 54 = 21 33 50
lcm(a, b) = 22 33 51 = 4275 = 540
43
 GCD and LCM

a = 60 = 22  31  51

b = 54 = 21  33  50

gcd(a, b) = 21 31 50 =6

lcm(a, b) = 22 33 51 = 540

Theorem: ab = gcd(a,b)lcm(a,b)

44
 Modular Arithmetic
•Let a be an integer and m be a positive integer.
We denote by a mod m the remainder when a is divided by m.

•Examples:

9 mod 4 = 1
9 mod 3 = 0
9 mod 10 = 9
-13 mod 4 = 3

45
 Congruences
•Let a and b be integers and m be a positive integer. We say that a is
congruent to b modulo m if
m divides a – b.

•We use the notation a  b (mod m) to indicate that a is congruent to b

modulo m.

•In other words:

a  b (mod m) if and only if a mod m = b mod m.

46
 Congruences
•Examples:
•Is it true that 46  68 (mod 11) ?
•Yes, because 11 | (46 – 68).
•Is it true that 46  68 (mod 22)?
•Yes, because 22 | (46 – 68).
•For which integers z is it true that z  12 (mod 10)?
•It is true for any z{…,-28, -18, -8, 2, 12, 22, 32, …}
•Theorem: Let m be a positive integer. The integers a and b are
congruent modulo m if and only if there is an integer k such that
a = b + km.

47
 Congruences
•Theorem: Let m be a positive integer.
If a  b (mod m) and c  d (mod m), then
a + c  b + d (mod m) and ac  bd (mod m).

•Proof:
•We know that a  b (mod m) and c  d (mod m) implies that there are
integers s and t with
b = a + sm and d = c + tm.

•Therefore,
•b + d = (a + sm) + (c + tm) = (a + c) + m(s + t) and
•bd = (a + sm)(c + tm) = ac + m(at + cs + stm).
•Hence, a + c  b + d (mod m) and ac  bd (mod m).

48
 Congruences
•Theorem: Let m be a positive integer. a  b (mod m) iff a mod m = b mod
m.

•Proof:
•Let a = mq1 + r1, and b = mq2 + r2.
Only if part: a mod m = b mod m  r1 = r2, therefore
a – b = m(q1 – q2), and a  b (mod m).
If part: a  b (mod m) implies
a – b = mq
mq1 + r1 – (mq2 + r2) = mq
r1 – r2 = m(q – q1 + q2).
Since 0  r1, r2  m, 0  |r1 - r2|  m. The only multiple in that range is 0.
Therefore r1 = r2, and a mod m = b mod m.

49
 The Euclidean Algorithm
•The Euclidean Algorithm finds the greatest common divisor of two
integers a and b.

•For example, if we want to find gcd(287, 91), we divide 287 by 91:

287 = 913 + 14

•We know that for integers a, b and c,

if a | b and a | c, then a | (b + c).

Therefore, any divisor (including their gcd) of 287 and 91 must also be
a divisor of 287 - 913 = 14.

•Consequently, gcd(287, 91) = gcd(14, 91).

50
 The Euclidean Algorithm
•In the next step, we divide 91 by 14:
91 = 146 + 7

This means that gcd(14, 91) = gcd(14, 7).

So we divide 14 by 7:

14 = 72 + 0

We find that 7 | 14, and thus gcd(14, 7) = 7.

Therefore, gcd(287, 91) = 7.

51
 The Euclidean Algorithm
•In pseudocode, the algorithm can be implemented as follows:
procedure gcd(a, b: positive integers)
x := a
y := b
while y  0
begin
r := x mod y
x := y
y := r
end {x is gcd(a, b)}

52
 Representations of Integers
•Let b be a positive integer greater than 1.
Then if n is a positive integer, it can be expressed uniquely in the form:

n = a bk
k
+ ak-1bk-1 + … + a1b1 + a0b0,
•where k is a nonnegative integer,
•a0, a1, …, ak are nonnegative integers less than b,
•and ak  0.

•Example for b=10:

859 = 8102 + 5101 + 9100

53
 Representations of Integers

•Example for b=2 (binary expansion):

(10110) = 124 + 122 + 121 = (22)

2 10

•Example for b=16 (hexadecimal expansion):

•(we use letters A to F to indicate numbers 10 to 15)

(3A0F) = 3  16 3 + 10162 + 15160 = (14863)

• 16 10

54
 Representations of Integers
•How can we construct the base b expansion of an integer n?
•First, divide n by b to obtain a quotient q0 and remainder a0, that is,
•n = bq0 + a0, where 0  a0 < b.
•The remainder a0 is the rightmost digit in the base b expansion of n.
•Next, divide q0 by b to obtain:
•q0 = bq1 + a1, where 0  a1 < b.
•a1 is the second digit from the right in the base b expansion of n.
Continue this process until you obtain a quotient equal to zero.

55
 Representations of Integers
•Example:
What is the base 8 expansion of (12345) ?
10

•First, divide 12345 by 8:

12345 = 81543 + 1

1543 = 8192 + 7
192 = 824 + 0
24 = 83 + 0
3 = 8 0 + 3

•The result is: (12345)10 = (30071)8.

56
 Representations of Integers
procedure base_b_expansion(n, b: positive integers)
q := n
k := 0
while q  0
begin
ak := q mod b
q := q/b
k := k + 1
end
{the base b expansion of n is (ak-1 … a1a0) }
b

57
 Addition of Integers
•How do we (humans) add two integers?
1 11 carry
•Example: 7583
+ 4932

1 2 51 5
1 1 carry
Binary expansions: (1011)2
+ (1010)2
(101 01 )2

58
 Addition of Integers
Let a = (a a …a a ) , b = (b b …b b ) .
n-1 n-2 1 0 2 n-1 n-2 1 0 2

How can we algorithmically add these two binary numbers?

•First, add their rightmost bits:

a + b = c 2 + s ,
0 0 0 0

•where s0 is the rightmost bit in the binary expansion of a + b, and c0 is the

carry.

•Then, add the next pair of bits and the carry:

a + b + c = c 2 + s ,
1 1 0 1 1

•where s1 is the next bit in the binary expansion of a + b, and c1 is the carry.
59
 Addition of Integers (cont.)

•Continue this process until you obtain cn-1.

•The leading bit of the sum is sn = cn-1.

•The result is:

a + b = (s s …s s )
n n-1 1 0 2

60
 Addition of Integers
•Example:
Add a = (1110) and b = (1011) .
2 2

a + b = 0 + 1 = 02 + 1, so that c = 0 and s = 1.

0 0 0 0

a + b + c = 1 + 1 + 0 = 12 + 0, so c = 1 and s = 0.
1 1 0 1 1

a + b + c = 1 + 0 + 1 = 12 + 0, so c = 1 and s = 0.
2 2 1 2 2

a + b + c = 1 + 1 + 1 = 12 + 1, so c = 1 and s = 1.
3 3 2 3 3

•s4 = c3 = 1.

•Therefore, s = a + b = (11001)2.
61
 Addition of Integers
procedure add(a, b: positive integers)
c := 0
for j := 0 to n-1
begin
d := (a + b + c)/2
j j
s := a + b + c – 2d
j j j
c := d
end
s := c
n
{the binary expansion of the sum is (s s …s s ) }
n n-1 1 0 2

62