0% found this document useful (0 votes)
2 views59 pages

CH 3

The document provides an overview of algorithms, defining them as finite sets of instructions for computations or problem-solving. It includes examples such as finding the maximum element and searching algorithms, discussing their time complexity and efficiency. Additionally, it introduces number theory concepts, including divisibility, primes, and greatest common divisors, along with relevant algorithms and their complexities.

Uploaded by

sekaniesempireee
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PPT, PDF, TXT or read online on Scribd
0% found this document useful (0 votes)
2 views59 pages

CH 3

The document provides an overview of algorithms, defining them as finite sets of instructions for computations or problem-solving. It includes examples such as finding the maximum element and searching algorithms, discussing their time complexity and efficiency. Additionally, it introduces number theory concepts, including divisibility, primes, and greatest common divisors, along with relevant algorithms and their complexities.

Uploaded by

sekaniesempireee
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PPT, PDF, TXT or read online on Scribd
You are on page 1/ 59

Enough Mathematical

Appetizers!

Let us look at something more interesting:

Algorithms

Fall 2011 CSC 213 - Discrete Structures 1


Algorithms
What is an algorithm?

An algorithm is a finite set of precise


instructions for performing a computation or
for solving a problem.

Fall 2011 CSC 213 - Discrete Structures 2


Algorithm Examples
We will use a pseudocode to specify algorithms,
which slightly reminds us of Basic and Pascal.
Example: an algorithm that finds the maximum
element in a finite sequence

procedure max(a1, a2, …, an: integers)


max := a1
for i := 2 to n
if max < ai then max := ai
{max is the largest element}

Fall 2011 CSC 213 - Discrete Structures 3


Algorithm Examples
Another example: a linear search algorithm,
that is, an algorithm that linearly searches a
sequence for a particular element.
procedure linear_search(x: integer; a1, a2, …,
an : integers)
i := 1
while (i  n and x  ai)
i := i + 1
if i  n then location := i
else location := 0
{location is the subscript of the term that
equals x, or is zero if x is not found}
Fall 2011 CSC 213 - Discrete Structures 4
Algorithm Examples

If the terms in a sequence are ordered, a


binary search algorithm is more efficient than
linear search.

The binary search algorithm iteratively


restricts the relevant search interval until it
closes in on the position of the element to be
located.

Fall 2011 CSC 213 - Discrete Structures 5


Algorithm Examples

binary search for the letter ‘j’

search
interval

a c d f g h j l m o p r s u v x z

center
element

Fall 2011 CSC 213 - Discrete Structures 6


Algorithm Examples

binary search for the letter ‘j’

search
interval

a c d f g h j l m o p r s u v x z

center
element

Fall 2011 CSC 213 - Discrete Structures 7


Algorithm Examples

binary search for the letter ‘j’

search
interval

a c d f g h j l m o p r s u v x z

center
element

Fall 2011 CSC 213 - Discrete Structures 8


Algorithm Examples

binary search for the letter ‘j’

search
interval

a c d f g h j l m o p r s u v x z

center
element

Fall 2011 CSC 213 - Discrete Structures 9


Algorithm Examples

binary search for the letter ‘j’

search
interval

a c d f g h j l m o p r s u v x z

center
element
found !
Fall 2011 CSC 213 - Discrete Structures 10
Algorithm Examples
procedure binary_search(x: integer; a1, a2, …,
an : integers)
i := 1 {i is left endpoint of search interval}
j := n {j is right endpoint of search interval}
while (i < j)
begin
m := (i + j)/2
if x > am then i := m + 1
else j := m
end
if x = ai then location := i
else location := 0
{location is the subscript of the term that
equals
Fall 2011
x, or is zero if x is not found}
CSC 213 - Discrete Structures 11
Complexity

In general, we are not so much interested in


the time and space complexity for small
inputs.

For example, while the difference in time


complexity between linear and binary search
is meaningless for a sequence with n = 10, it
is gigantic for n = 230.

Fall 2011 CSC 213 - Discrete Structures 12


Complexity

For example, let us assume two algorithms A


and B that solve the same class of problems.
The time complexity of A is 5,000n, the one
for B is 1.1n for an input with n elements.
For n = 10, A requires 50,000 steps, but B
only 3, so B seems to be superior to A.
For n = 1000, however, A requires 5,000,000
steps, while B requires 2.51041 steps.

Fall 2011 CSC 213 - Discrete Structures 13


Complexity

This means that algorithm B cannot be used


for large inputs, while algorithm A is still
feasible.

So what is important is the growth of the


complexity functions.

The growth of time and space complexity with


increasing input size n is a suitable measure
for the comparison of algorithms.

Fall 2011 CSC 213 - Discrete Structures 14


Complexity
Comparison: time complexity of algorithms A and B

Input Size Algorithm A Algorithm B


n 5,000n 1.1n
10 50,000 3
100 500,000 13,781
1,000 5,000,000 2.51041
1,000,000 5109 4.81041392

Fall 2011 CSC 213 - Discrete Structures 15


The Growth of Functions
The growth of functions is usually described
using the big-O notation.

Definition: Let f and g be functions from the


integers or the real numbers to the real
numbers.
We say that f(x) is O(g(x)) if there are
constants C and k such that
|f(x)|  C|g(x)|
whenever x > k.
Fall 2011 CSC 213 - Discrete Structures 16
The Growth of Functions

When we analyze the growth of complexity


functions, f(x) and g(x) are always positive.
Therefore, we can simplify the big-O
requirement to
f(x)  Cg(x) whenever x > k.

If we want to show that f(x) is O(g(x)), we only


need to find one pair (C, k) (which is never
unique).

Fall 2011 CSC 213 - Discrete Structures 17


The Growth of Functions
The idea behind the big-O notation is to
establish an upper boundary for the growth
of a function f(x) for large x.
This boundary is specified by a function g(x)
that is usually much simpler than f(x).
We accept the constant C in the requirement
f(x)  Cg(x) whenever x > k,
because C does not grow with x.
We are only interested in large x, so it is OK if
f(x) > Cg(x) for x  k.

Fall 2011 CSC 213 - Discrete Structures 18


The Growth of Functions
Example:
Show that f(x) = x2 + 2x + 1 is O(x2).

For x > 1 we have:


x2 + 2x + 1  x2 + 2x2 + x2
 x2 + 2x + 1  4x2
Therefore, for C = 4 and k = 1:
f(x)  Cx2 whenever x > k.

 f(x) is O(x2).

Fall 2011 CSC 213 - Discrete Structures 19


The Growth of Functions

Question: If f(x) is O(x2), is it also O(x3)?

Yes. x3 grows faster than x2, so x3 grows also


faster than f(x).

Therefore, we always have to find the


smallest simple function g(x) for which f(x) is
O(g(x)).

Fall 2011 CSC 213 - Discrete Structures 20


The Growth of Functions
“Popular” functions g(n) are
n log n, 1, 2n, n2, n!, n, n3, log n

Listed from slowest to fastest growth:


• 1
• log n
• n
• n log n
• n2
• n3
• 2n
• n!

Fall 2011 CSC 213 - Discrete Structures 21


The Growth of Functions

A problem that can be solved with polynomial


worst-case complexity is called tractable.

Problems of higher complexity are called


intractable.

Problems that no algorithm can solve are


called unsolvable.

Fall 2011 CSC 213 - Discrete Structures 22


Useful Rules for Big-O
For any polynomial f(x) = anxn + an-1xn-1 + … +
a0, where a0, a1, …, an are real numbers,
f(x) is O(xn).

If f1(x) is O(g1(x)) and f2(x) is O(g2(x)), then


(f1 + f2)(x) is O(max(g1(x), g2(x)))

If f1(x) is O(g(x)) and f2(x) is O(g(x)), then


(f1 + f2)(x) is O(g(x)).

If f1(x) is O(g1(x)) and f2(x) is O(g2(x)), then


(f1f2)(x) is O(g1(x) g2(x)).
Fall 2011 CSC 213 - Discrete Structures 23
Complexity Examples
What does the following algorithm compute?
procedure who_knows(a1, a2, …, an: integers)
m := 0
for i := 1 to n-1
for j := i + 1 to n
if |ai – aj| > m then m := |ai – aj|
{m is the maximum difference between any two
numbers in the input sequence}
Comparisons: n-1 + n-2 + n-3 + … + 1
= (n – 1)n/2 = 0.5n2 – 0.5n

Time complexity is O(n2).


Fall 2011 CSC 213 - Discrete Structures 24
Complexity Examples
Another algorithm solving the same problem:
procedure max_diff(a1, a2, …, an: integers)
min := a1
max := a1
for i := 2 to n
if ai < min then min := ai
else if ai > max then max := ai
m := max - min
Comparisons: n - 1
Time complexity is O(n).
Fall 2011 CSC 213 - Discrete Structures 25
Let us get into…

Number Theory

Fall 2011 CSC 213 - Discrete Structures 26


Introduction to Number Theory

Number theory is about integers and their


properties.

We will start with the basic principles of


• divisibility,
• greatest common divisors,
• least common multiples, and
• modular arithmetic

and look at some relevant algorithms.


Fall 2011 CSC 213 - Discrete Structures 27
Division
If a and b are integers with a  0, we say that
a divides b if there is an integer c so that b =
ac.

When a divides b we say that a is a factor of b


and that b is a multiple of a.

The notation a | b means that a divides b.

We write a χ b when a does not divide b


(see book for correct symbol).
Fall 2011 CSC 213 - Discrete Structures 28
Divisibility Theorems
For integers a, b, and c it is true that

• if a | b and a | c, then a | (b + c)
Example: 3 | 6 and 3 | 9, so 3 | 15.

• if a | b, then a | bc for all integers c


Example: 5 | 10, so 5 | 20, 5 | 30, 5 | 40, …

• if a | b and b | c, then a | c
Example: 4 | 8 and 8 | 24, so 4 | 24.

Fall 2011 CSC 213 - Discrete Structures 29


Primes
A positive integer p greater than 1 is called
prime if the only positive factors of p are 1 and
p.
Note: 1 is not a prime
A positive integer that is greater than 1 and is
not prime is called composite.
The fundamental theorem of arithmetic:
Every positive integer can be written uniquely
as the product of primes, where the prime
factors are written in order of increasing size.
Fall 2011 CSC 213 - Discrete Structures 30
Primes
Examples:
15 = 3·5
48 = 2·2·2·2·3 = 24·3
17 = 17
100 2·2·5·5 = 22·52
=
512 2·2·2·2·2·2·2·2·2 = 29
=
515 5·103
=
28 = 2·2·7
Fall 2011 CSC 213 - Discrete Structures 31
Primes

If n is a composite integer, then n has a prime


divisor less than or equal n .

This is easy to see: if n is a composite integer,


it must have at least two prime divisors. Let
the largest two be p1 and p2. Then p1p2 <= n.

p1 and p2 cannot both be greater than


n , because then p p > n.
1 2

Fall 2011 CSC 213 - Discrete Structures 32


The Division Algorithm
Let a be an integer and d a positive integer.
Then there are unique integers q and r, with
0  r < d, such that a = dq + r.

In the above equation,


• d is called the divisor,
• a is called the dividend,
• q is called the quotient, and
• r is called the remainder.

Fall 2011 CSC 213 - Discrete Structures 33


The Division Algorithm
Example:

When we divide 17 by 5, we have

17 = 53 + 2.

• 17 is the dividend,
• 5 is the divisor,
• 3 is called the quotient, and
• 2 is called the remainder.

Fall 2011 CSC 213 - Discrete Structures 34


The Division Algorithm
Another example:
What happens when we divide -11 by 3 ?
Note that the remainder cannot be negative.
-11 = 3(-4) + 1.

• -11 is the dividend,


• 3 is the divisor,
• -4 is called the quotient, and
• 1 is called the remainder.

Fall 2011 CSC 213 - Discrete Structures 35


Greatest Common Divisors
Let a and b be integers, not both zero.
The largest integer d such that d | a and d | b is
called the greatest common divisor of a and b.
The greatest common divisor of a and b is denoted
by gcd(a, b).
Example 1: What is gcd(48, 72) ?
The positive common divisors of 48 and 72 are
1, 2, 3, 4, 6, 8, 12, 16, and 24, so gcd(48, 72) = 24.
Example 2: What is gcd(19, 72) ?
The only positive common divisor of 19 and 72 is
1, so gcd(19, 72) = 1.

Fall 2011 CSC 213 - Discrete Structures 36


Greatest Common Divisors
Using prime factorizations:

a = p1a1 p2a2 … pnan , b = p1b1 p2b2 … pnbn ,


where p1 < p2 < … < pn and ai, bi  N for 1  i  n

gcd(a, b) = p1min(a1, b1 ) p2min(a2, b2 ) … pnmin(an, bn )

Example:
a = 60 22 31 51
=
b = 54 21 33 50
=
gcd(a, b) 21 31 50 = 6
=Fall 2011 CSC 213 - Discrete Structures 37
Relatively Prime Integers
Definition:
Two integers a and b are relatively prime if
gcd(a, b) = 1.

Examples:
Are 15 and 28 relatively prime?
Yes, gcd(15, 28) = 1.
Are 55 and 28 relatively prime?
Yes, gcd(55, 28) = 1.
Are 35 and 28 relatively prime?
No, gcd(35, 28) = 7.
Fall 2011 CSC 213 - Discrete Structures 38
Relatively Prime Integers
Definition:
The integers a1, a2, …, an are pairwise
relatively prime if gcd(ai, aj) = 1 whenever 1
 i < j  n.

Examples:
Are 15, 17, and 27 pairwise relatively prime?
No, because gcd(15, 27) = 3.
Are 15, 17, and 28 pairwise relatively prime?
Yes, because gcd(15, 17) = 1, gcd(15, 28) = 1
and gcd(17, 28) =CSC1.
Fall 2011 213 - Discrete Structures 39
Least Common Multiples
Definition:
The least common multiple of the positive
integers a and b is the smallest positive
integer that is divisible by both a and b.
We denote the least common multiple of a and
b by lcm(a, b).
Examples:
lcm(3, 7) 21
=
lcm(4, 6) 12
=
lcm(5, 10) 10
= Fall 2011 CSC 213 - Discrete Structures 40
Least Common Multiples
Using prime factorizations:

a = p1a1 p2a2 … pnan , b = p1b1 p2b2 … pnbn ,


where p1 < p2 < … < pn and ai, bi  N for 1  i  n

lcm(a, b) = p1max(a1, b1 ) p2max(a2, b2 ) … pnmax(an, bn )

Example:
a = 60 22 31 51
=
b = 54 21 33 50
=
lcm(a, b) 22 33 51 = 4275 = 540
=Fall 2011 CSC 213 - Discrete Structures 41
GCD and LCM

a = 60 22 31 51
=
b = 54 21 33 50
=
gcd(a, b) 21 31 50 =6
=
lcm(a, b) 22 33 51 = 540
=
Theorem: ab gcd(a,b)lcm(a,
= b)
Fall 2011 CSC 213 - Discrete Structures 42
Modular Arithmetic
Let a be an integer and m be a positive integer.
We denote by a mod m the remainder when a
is divided by m.

Examples:

9 mod 4 1
=
9 mod 3 0
=
9 mod 10 9
=
-13 mod 4 3
=
Fall 2011 CSC 213 - Discrete Structures 43
Congruences
Let a and b be integers and m be a positive
integer. We say that a is congruent to b
modulo m if
m divides a – b.

We use the notation a  b (mod m) to indicate


that a is congruent to b modulo m.

In other words:
a  b (mod m) if and only if a mod m = b mod
m.
Fall 2011 CSC 213 - Discrete Structures 44
Congruences
Examples:
Is it true that 46  68 (mod 11) ?
Yes, because 11 | (46 – 68).
Is it true that 46  68 (mod 22)?
Yes, because 22 | (46 – 68).
For which integers z is it true that z  12 (mod
10)?
It is true for any z{…,-28, -18, -8, 2, 12, 22, 32,
…}

Theorem: Let m be a positive integer. The


integers a and b are congruent modulo m if and
only if there is an integer k such that a = b + km.
Fall 2011 CSC 213 - Discrete Structures 45
Congruences
Theorem: Let m be a positive integer.
If a  b (mod m) and c  d (mod m), then
a + c  b + d (mod m) and ac  bd (mod m).
Proof:
We know that a  b (mod m) and c  d (mod m)
implies that there are integers s and t with
b = a + sm and d = c + tm.
Therefore,
b + d = (a + sm) + (c + tm) = (a + c) + m(s + t)
and
bd = (a + sm)(c + tm) = ac + m(at + cs + stm).
Hence, a + c  b + d (mod m) and ac  bd (mod
m).
Fall 2011 CSC 213 - Discrete Structures 46
Congruences
Theorem: Let m be a positive integer. a  b (mod
m) iff a mod m = b mod m.
Proof:
Let a = mq1 + r1, and b = mq2 + r2.
Only if part: a mod m = b mod m  r1 = r2,
therefore
a – b = m(q1 – q2), and a  b (mod m).
If part: a  b (mod m) implies
a – b = mq
mq1 + r1 – (mq2 + r2) = mq
r1 – r2 = m(q – q1 + q2).
Since 0  r1, r2  m, 0  |r1 - r2|  m. The only
multiple in that range is 0.
Therefore r1 = r2, and a mod m = b mod m.
Fall 2011 CSC 213 - Discrete Structures 47
The Euclidean Algorithm
The Euclidean Algorithm finds the greatest
common divisor of two integers a and b.
For example, if we want to find gcd(287, 91),
we divide 287 by 91:
287 = 913 + 14
We know that for integers a, b and c,
if a | b and a | c, then a | (b + c).
Therefore, any divisor (including their gcd) of
287 and 91 must also be a divisor of 287 -
913 = 14.
Consequently, gcd(287, 91) = gcd(14, 91).
Fall 2011 CSC 213 - Discrete Structures 48
The Euclidean Algorithm
In the next step, we divide 91 by 14:
91 = 146 + 7
This means that gcd(14, 91) = gcd(14, 7).

So we divide 14 by 7:
14 = 72 + 0
We find that 7 | 14, and thus gcd(14, 7) = 7.

Therefore, gcd(287, 91) = 7.


Fall 2011 CSC 213 - Discrete Structures 49
The Euclidean Algorithm
In pseudocode, the algorithm can be
implemented as follows:

procedure gcd(a, b: positive integers)


x := a
y := b
while y  0
begin
r := x mod y
x := y
y := r
end {x is gcd(a, b)}

Fall 2011 CSC 213 - Discrete Structures 50


Representations of Integers
Let b be a positive integer greater than 1.
Then if n is a positive integer, it can be
expressed uniquely in the form:

n = akbk + ak-1bk-1 + … + a1b + a0,

where k is a nonnegative integer,


a0, a1, …, ak are nonnegative integers less than b,
and ak  0.

Example for b=10:


859 = 8102 + 5101 + 9100
Fall 2011 CSC 213 - Discrete Structures 51
Representations of Integers

Example for b=2 (binary expansion):


(10110)2 = 124 + 122 + 121 = (22)10

Example for b=16 (hexadecimal


expansion):
(we use letters A to F to indicate numbers 10 to
15)
(3A0F)16 = 3163 + 10162 + 15160 = (14863)10

Fall 2011 CSC 213 - Discrete Structures 52


Representations of Integers
How can we construct the base b expansion of an
integer n?
First, divide n by b to obtain a quotient q0 and
remainder a0, that is,
n = bq0 + a0, where 0  a0 < b.
The remainder a0 is the rightmost digit in the
base b expansion of n.
Next, divide q0 by b to obtain:
q0 = bq1 + a1, where 0  a1 < b.
a1 is the second digit from the right in the base b
expansion of n. Continue this process until you
obtain
Fall 2011
a quotient equal to zero.
CSC 213 - Discrete Structures 53
Representations of Integers
Example:
What is the base 8 expansion of (12345)10 ?

First, divide 12345 by 8:


12345 = 81543 + 1
1543 = 8192 + 7
192 = 824 + 0
24 = 83 + 0
3 = 80 + 3
The result is: (12345)10 = (30071)8.
Fall 2011 CSC 213 - Discrete Structures 54
Representations of Integers
procedure base_b_expansion(n, b: positive
integers)
q := n
k := 0
while q  0
begin
ak := q mod b
q := q/b
k := k + 1
end
{the base b expansion of n is (ak-1 … a1a0)b }
Fall 2011 CSC 213 - Discrete Structures 55
Addition of Integers
How do we (humans) add two integers?

1 11 carry
Example: 7583
+ 4932
12515
1 1 carry
Binary expansions: (1011)2
+ (1010)2
(10101 )2

Fall 2011 CSC 213 - Discrete Structures 56


Addition of Integers
Let a = (an-1an-2…a1a0)2, b = (bn-1bn-2…b1b0)2.
How can we algorithmically add these two
binary numbers?
First, add their rightmost bits:
a0 + b0 = c02 + s0,
where s0 is the rightmost bit in the binary
expansion of a + b, and c0 is the carry.
Then, add the next pair of bits and the carry:
a1 + b1 + c0 = c12 + s1,
where s1 is the next bit in the binary expansion
of a + b, and c1 is the
Fall 2011
carry.
CSC 213 - Discrete Structures 57
Addition of Integers

Continue this process until you obtain cn-1.

The leading bit of the sum is sn = cn-1.

The result is:


a + b = (snsn-1…s1s0)2

Fall 2011 CSC 213 - Discrete Structures 58


Addition of Integers
Example:
Add a = (1110)2 and b = (1011)2.

a0 + b0 = 0 + 1 = 02 + 1, so that c0 = 0 and s0 = 1.


a1 + b1 + c0 = 1 + 1 + 0 = 12 + 0, so c1 = 1 and s1 = 0.
a2 + b2 + c1 = 1 + 0 + 1 = 12 + 0, so c2 = 1 and s2 = 0.
a3 + b3 + c2 = 1 + 1 + 1 = 12 + 1, so c3 = 1 and s3 = 1.
s4 = c3 = 1.

Therefore, s = a + b = (11001)2.

Fall 2011 CSC 213 - Discrete Structures 59

You might also like