Automata Theory

CS411 & CS675 2015F-01

Set Theory & Proof Techniques
David Galles

Department of Computer Science

University of San Francisco
01-0: Syllabus
Office Hours
Course Text
Prerequisites
Test Dates & Testing Policies
Check dates now!
Grading Policies
01-1: How to Succeed
Come to class. Pay attention. Ask questions.
01-2: How to Succeed
Come to class. Pay attention. Ask questions.
A question as vague as “I don’t get it” is
perfectly acceptable.
If you’re confused, at least 2 other people are,
too.
01-3: How to Succeed
Come to class. Pay attention. Ask questions.
A question as vague as “I don’t get it” is
perfectly acceptable.
If you’re confused, at least 2 other people are,
too.
Come by my office
I am very available to students.
01-4: How to Succeed
Come to class. Pay attention. Ask questions.
A question as vague as “I don’t get it” is
perfectly acceptable.
If you’re confused, at least 2 other people are,
too.
Come by my office
I am very available to students.
Start the homework assignments early
Homework in this class requires “thinking time”
01-5: How to Succeed
Come to class. Pay attention. Ask questions.
A question as vague as “I don’t get it” is
perfectly acceptable.
If you’re confused, at least 2 other people are,
too.
Come by my office
I am very available to students.
Start the homework assignments early
Homework in this class requires “thinking time”
Read the textbook.
Ask Questions! The textbook can be hard to
follow –reading a dense, technical work is a
“learning outcome” for this class
01-6: Class Goals
Prove that there are some problems that cannot be
solved
Show that there are some problems that (are
believed to) require an exponential amount of time
to solve (NP-Complete)
Examine some strategies for dealing with these
problems
Along the way, learn how to model computation
mathematically, and pick up some useful
formalisms & techniques
DFA, regular expressions, CFGs, etc.
01-7: Review of the Basics
Most (but perhaps not all) of the following material
is review from discrete mathematics
I will go fairly fast, assuming it is review
Ask me to slow down if you have any questions!
01-8: Sets – Definition
A set is an unordered collection of objects
S = {a, b, c}
a, b, c are elements or members of the set S
01-9: Sets – Definition
A set is an unordered collection of objects
S = {a, b, c}
a, b, c are elements or members of the set S
Elements in a set need have no relation to each
other
S1 = {1, 2, 3}
S2 = { red, farmhouse, π , -32 }
01-10: Sets – Definition
Sets can contain other sets as elements
S1 = {3, {3, 4}, {4, {5, 6}}}
S2 = {{1, 2}, {{4}}}
Sets do not contain duplicates
NotASet = {4, 2, 4, 5}
01-11: Sets – Cardinality
Cardinality of a set is the number of elements in
the set
|{a, b, c}| = 3
|{{a, b}, c}| =?
01-12: Sets – Cardinality
Cardinality of a set is the number of elements in
the set
|{a, b, c}| = 3
|{{a, b}, c}| = 2 ({a, b} and c)
01-13: Sets – Empty, Singleton
Empty Set: {} or ∅, |{}| = |∅| = 0
Singleton set – set with one element
{1}
{4}
{} ?
{{}} ?
{{3, 1, 2}} ?
01-14: Sets – Empty, Singleton
Empty Set: {} or ∅, |{}| = |∅| = 0
Singleton set – set with one element
{1} Singleton
{4} Singleton
{} Not a Singleton (empty)
{{}} Singleton
{{3, 1, 2}} Singleton
01-15: Sets – Membership
Set membership: x ∈ S
3 ∈ {1, 3, 5}
a 6∈ {b, c, d}
3 ∈ {1, {2, 3}} ?
{} ∈ {1, 2, 3} ?
{} ∈ {1, {}, 4} ?
01-16: Sets – Membership
Set membership: x ∈ S
3 ∈ {1, 3, 5}
a 6∈ {b, c, d}
3 6∈ {1, {2, 3}}
{} 6∈ {1, 2, 3}
{} ∈ {1, {}, 4}
01-17: Sets – Describing
Referring to sets
List all members
{3, 4, 5}, {0, 1, 2, 3, . . . }
S = {x : x has a certain property}
S = {x | x has a certain property}
S = {x : x ∈ N ∧ x < 10}
N is the set of natural numbers {0, 1, 2, . . . }
S = {x : x is prime }
A ∪ B = {x : x ∈ A ∨ x ∈ B}
A ∩ B = {x : x ∈ A ∧ x ∈ B}
A − B = {x : x ∈ A ∧ x 6∈ B}
01-18: Sets – ∪, ∩
More Union & Intersection
A and B are disjoint if A ∩ B = {}
S is a collection of sets (set of sets)
S = {x : x ∈ A for some A ∈ S }
S
S
{{1, 2}, {2, 3}} = {1, 2, 3}
T = {x : x ∈ A for all A ∈ S }
T
S
{{1, 2}, {2, 3}} = {2}
01-19: Sets – Subset
Subsets & Supersets
A is a subset of B , A ⊆ B if:
∀x, x ∈ A =⇒ x ∈ B
∀(x ∈ A), x ∈ B
A is a proper subset of B , A ⊂ B if:
A ⊆ B ∧ (∃x, x ∈ B ∧ x 6∈ A)
{} is a subset of any set (including itself)
{} is the only set that does not have a proper
subset
01-20: Sets – Power Set
Power set: Set of all subsets
2S = {x : x ⊆ S}
2{a,b} = ?
2{} = ?
|2S | = ?
01-21: Sets – Power Set
Power set: Set of all subsets
2S = {x : x ⊆ S}
2{a,b} = {{}, {a}, {b}, {a, b}}
2{} = {{}}
|2S | = 2|S|
01-22: Sets – Partition
Π is a partition of S if:

Π ⊂ 2S
{} 6∈ Π
∀(X, Y ∈ Π), X 6= Y =⇒ X ∩ Y = {}
Π=S
S

{{a, c}, {b, d, e}, {f}} is a partition of {a,b,c,d,e,f}

{{a, b, c, d, e, f}} is a partition of {a,b,c,d,e,f}
{{a, b, c}, {d, e, f}} is a partition of {a,b,c,d,e,f}
01-23: Sets – Partition
In other words, a partition of a set S is just a division
of the elements of S into 1 or more groups.

All the partitions of the set {a, b, c}?

01-24: Sets – Partition
In other words, a partition of a set S is just a division
of the elements of S into 1 or more groups.

All the partitions of the set {a, b, c}?

{{a, b, c}}, {{a, b}, {c}}, {{a, c}, {b}}, {{a},{b, c}},
{{a}, {b}, {c}}
01-25: Ordered Pair
(x, y) is an ordered pair
Order matters – (x, y) 6= (y, x) if x 6= y
hence ordered
x and y are the components of the ordered pair
(x, y)
01-26: Cartesian Product
A × B = {(x, y) : x ∈ A ∧ y ∈ B}

{1, 2} × {3, 4} = {(1, 3), (1, 4), (2, 3), (2, 4)}
{1, 2} × {1, 2} = ?
2{a}×{b} = ?
2{a} × 2{b} = ?
01-27: Cartesian Product
A × B = {(x, y) : x ∈ A ∧ y ∈ B}

{1, 2} × {3, 4} = {(1, 3), (1, 4), (2, 3), (2, 4)}
{1, 2} × {1, 2} = {(1, 1), (1, 2), (2, 1), (2, 2)}
2{a}×{b} = {{(a, b)}, {}}
2{a} × 2{b} =
{({a}, {b}), ({a}, {}), ({}, {b}), ({}, {})}
01-28: Cartesian Product
Which of the following is true:

∀(A, B) A×B =B×A

∀(A, B) A × B 6= B × A
None of the above
01-29: Cartesian Product
Which of the following is true:

∀(A, B) A×B =B×A

If and only if A = B
∀(A, B) A × B 6= B × A
If and only if A 6= B
01-30: Cartesian Product
Why “Cartesian”?
Think “Cartesian Coordinates” (standard
coordinate system)
R × R is the real plane
Set of all points (x, y) where x, y ∈ R
R is the set of real numbers (think “floats” if
you’re CS)
01-31: Cartesian Product
Can take the Cartesian product of > 2 sets.
A × B × C = {(x, y, z) : x ∈ A, y ∈ B, z ∈ C}
{a} × {b, c} × {d} = {(a, b, d), (a, c, d)}
(Techinally, A × B × C = (A × B) × C )
{a} × {b, c} × {d} = {((a, b), d), ((a, c), d)}
Often drop the extra parentheses for readability
01-32: Relations
A relation R is a set of ordered pairs
For example the relation < over the Natural
Numbers is the set:

{ (0,1), (0,2), (0,3), ...

(1,2), (1,3), (1,4), ...
(2,3), (2,4), (2,5), ...
... }
01-33: Relations
Often, relations are over the same set
that is, a subset of A × A for some set A
Not all relations are over the same set, however
Relation describing prices of computer
components
{(Hard dive, $55), (WAP, $49), (2G DDR, $44),
. . .}
01-34: Functions
A function is a special kind of relation (all
functions are relations, but not all relations are
functions)
A relation R ⊆ A × B is a function if:
For each a ∈ A, there is exactly one ordered
pair in R with the first component a
01-35: Functions
A function f that is a subset of A × B is written:
f : A 7→ B
(a, b) ∈ f is written f (a) = b
A is the domain of the function
if A′ ⊆ A, f (A′ ) = {b : a ∈ A′ ∧ f (a) = b} is
the image of A′
The range of a function is the image of its
domain
01-36: Functions
A function f : A 7→ B is:

one-to-one if no two elements in A match to the

same element in B
onto Each element in B is mapped to by at least
one element in A
a bijection if it is both one-to-one and onto

The inverse of a binary relation R ⊂ A × B is denoted

R−1 , and defined to be {(b, a) : (a, b) ∈ R}
A function only has an inverse if ...
01-37: Functions
A function f : A 7→ B is:

one-to-one if no two elements in A match to the

same element in B
onto Each element in B is mapped to by at least
one element in A
a bijection if it is both one-to-one and onto

The inverse of a binary relation R ⊂ A × B is denoted

R−1 , and defined to be {(b, a) : (a, b) ∈ R}
A function only has an inverse if it is a bijection
01-38: Functions
What if we want to take the inverse of a function
that is not a bijection – what can we do?
Want to preserve full information about the
original function
Resulting inverse must be an actual function
01-39: Functions
What if we want to take the inverse of a function
that is not a bijection – what can we do?
Want to preserve full information about the
original function
Resulting inverse must be an actual function
How can we have an element map to 0, 1, or more
elements, and still have a function? HINT: If we
modified the range ...
01-40: Functions
What if we want to take the inverse of a function
that is not a bijection – what can we do?
Want to preserve full information about the
original function
Resulting inverse must be an actual function
f : A 7→ B f −1 : B 7→ 2A

(example on chalkboard)
01-41: Relations
Q and R are two relations
The composition of Q and R, Q ◦ R is:
{(a, b) : (a, c) ∈ Q, (c, b) ∈ R for some c}

Q = {(a, c), (b, d), (c, a)}

R = {(a, c), (b, c), (c, a)}
Q ◦ R = {(a, a), (c, c)}
Q◦Q? (Q ◦ R) ◦ Q?
01-42: Relation Graph
Each element is a node in the graph
if (a, b) ∈ R, then there is an edge from a to b in
the graph

R = {(a, b), (a, c), (c, a), (b, b), (b, d)}
a c

b d
01-43: Relation Types
A relation R ⊆ A × A is reflexive if
(a, a) ∈ R for each a ∈ A
∀(a ∈ A), (a, a) ∈ R
Each node has a self loop

a b a b

c d c d
Reflexive Not Reflexive
01-44: Relation Types
A relation R ⊆ A × A is symmetric if
(a, b) ∈ R whenever (b, a) ∈ R
(a, b) ∈ R =⇒ (b, a) ∈ R
Every edge goes “both ways”

a b a b

c d c d
Symmetric Not Symmeteric
01-45: Relation Types
A relation R ⊆ A × A is antisymmetric if
whenever (a, b) ∈ R, a, b are distinct (b, a) 6∈ R
(a, b) ∈ R ∧ a 6= b =⇒ (b, a) 6∈ R
No edge goes “both ways”

a b a b

c d c d
Antisymmetric Not Antisymmeteric

Can a relation be neither symmetric nor

antisymmetric?
01-46: Relation Types
A relation R ⊆ A × A is antisymmetric if
whenever (a, b) ∈ R, a, b are distinct (b, a) 6∈ R
(a, b) ∈ R ∧ a 6= b =⇒ (b, a) 6∈ R
No edge goes “both ways”

a b a b

c d c d
Antisymmetric Not Antisymmeteric

Can a relation be both symmetric and

antisymmetric?
01-47: Relation Types
A relation R ⊆ A × A is transitive if
whenever (a, b) ∈ R, and (b, c) ∈ R, (a, c) ∈ R
(a, b) ∈ R ∧ (b, c) ∈ R =⇒ (a, c) ∈ R
Every path of length 2 has a direct edge

a b a b

c d c d

Transitive Not Transitive

01-48: Closure
A set A ⊆ B is closed under a relation
R ⊆ ((B × B) × B) if:
a1 , a2 ∈ A ∧ ((a1 , a2 ), c) ∈ R =⇒ c ∈ A
That is, if a1 and a2 are both in A, and
((a1 , a2 ), c) is in the relation, then c is also in A
N is closed under addtion
N is not closed under subtraction or division
01-49: Closure
Relations are also sets (of ordered pairs)
We can talk about a relation R being closed over
another relation R′
Each element of R′ is an ordered triple of
ordered pairs!
01-50: Closure
Relations are also sets (of ordered pairs)
We can talk about a relation R being closed over
another relation R′
Each element of R′ is an ordered triple of
ordered pairs!
Example:
R ⊆A×A
R′ = {(((a, b), (b, c)), (a, c)) : a, b, c ∈ A}
If R is closed under R′ , then . . .
01-51: Closure
Relations are also sets (of ordered pairs)
We can talk about a relation R being closed over
another relation R′
Each element of R′ is an ordered triple of
ordered pairs!
Example:
R ⊆A×A
R′ = {(((a, b), (b, c)), (a, c)) : a, b, c ∈ A}
If R is closed under R′ , then R is transitive!
01-52: Closure
Reflexive closure of a relation R ⊆ A × A is the
smallest possible superset of R which is reflexive
Add self-loop to every node in relation
Add (a,a) to R for every a ∈ A
Transitive Closure of a relation R ⊆ A × A is
the smallest possible superset of R which is
transitive
Add direct link for every path of length 2.
∀(a, b, c ∈ A) if (a, b) ∈ R ∧ (b, c) ∈ R add
(a, c) to R.

(examples on board)
01-53: Relation Types
Equivalence Relation
Symmetric, Transitive, Reflexive
Examples:
Equality (=)
A is the set of English words, (w1 , w2 ) ∈ R if w1
and w2 start with the same letter

(example graphs)
01-54: Relation Types
Equivalence Relation
Symmetric, Transitive, Reflexive
Separates set into equivalence classes (all words
that start with a, for example
If a ∈ A, then [a] represents equivalence class that
contains a.
01-55: Relation Types
Partial Order
Antisymmetric, Transitive, Reflexive
Examples:
≤ for integers
A is the set of integers, (a, b) ∈ R if a ≤ b
Ancestor
R ⊆ A × A = {(x, y) : x is an ancestor of y ,
or x = y }

(example graphs)
01-56: Relation Types
Total Order
R ⊆ A × A is a total order if:
R is a partial order
For all a, b ∈ A, either (a, b) ∈ R or (b, a) ∈ R
Is ≤ a total order?
Is Ancestor a total order?

(example graphs)
01-57: Cardinality
How can we tell if two sets A and B have the same
cardinality?
01-58: Cardinality
How can we tell if two sets A and B have the same
cardinality?
Calculate |A| and |B|, make sure numbers are
the same
Match each element in A to an element in B
Create a bijection f : A 7→ B
(or f : B 7→ A)
01-59: Cardinality
What about infinite sets? Are they all
equinumerous (that is, have the same cardinality)?
A set is countable infinite (or just countable) if
it is equinumerous with N.
01-60: Countable Sets
A set is countable infinite (or just countable) if
it is equinumerous with N.
Even elements of N?
01-61: Countable Sets
A set is countable infinite (or just countable) if
it is equinumerous with N.
Even elements of N?
f (x) = 2x
01-62: Countable Sets
A set is countable infinite (or just countable) if
it is equinumerous with N.
Integers (Z)?
01-63: Countable Sets
A set is countable infinite (or just countable) if
it is equinumerous with N.
Integers (Z)?
f (x) = ⌈ x2 ⌉ ∗ (−1)x

-4 -3 -2 -1 0 1 2 3 4

...
01-64: Countable Sets
A set is countable infinite (or just countable) if
it is equinumerous with N.
Union of 3 (disjoint) countable sets A, B, C?
01-65: Countable Sets
A set is countable infinite (or just countable) if
it is equinumerous with N.
Union of 3 (disjoint) countable sets A, B, C?
a0 a1 a2 a3 a4 ...

b0 b1 b2 b3 b4 ...

c0 c1 c2 c3 c4 ...

 a x3
 if x mod 3 = 0
f (x) = b x−1
3
if x mod 3 = 1
 c x−2 if x mod 3 = 2

3
01-66: Countable Sets
A set is countable infinite (or just countable) if
it is equinumerous with N.
N × N?
(0,0) (0,1) (0,2) (0,3) (0,4) ...

(1,0) (1,1) (1,2) (1,3) (1,4) ...

(2,0) (2,1) (2,2) (2,3) (2,4) ...

(3,0) (3,1) (3,2) (3,3) (3,4) ...

(4,0) (4,1) (4,2) (4,3) (4,4) ...

...

...

...

...

...
...
01-67: Countable Sets
A set is countable infinite (or just countable) if
it is equinumerous with N.
N × N?
(0,0) (0,1) (0,2) (0,3) (0,4) ...

(1,0) (1,1) (1,2) (1,3) (1,4) ...

(2,0) (2,1) (2,2) (2,3) (2,4) ...

(3,0) (3,1) (3,2) (3,3) (3,4) ...

(4,0) (4,1) (4,2) (4,3) (4,4) ...

...

...

...

...

...
...

(x+y)∗(x+y+1)
f ((x, y)) = 2
+x
01-68: Countable Sets
A set is countable infinite (or just countable) if
it is equinumerous with N.
Real numbers between 0 and 1 (exclusive)?
01-69: Uncountable R
Proof by contradiction
Assume that R between 0 and 1 (exclusive) is
countable
(that is, assume that there is some bijection
from N to R between 0 and 1)
Show that this leads to a contradiction
Find some element of R between 0 and 1
that is not mapped to by any element in N
01-70: Uncountable R
Assume that there is some bijection from N to R
between 0 and 1

0 0.3412315569...
1 0.0123506541...
2 0.1143216751...
3 0.2839143215...
4 0.2311459412...
5 0.8381441234...
6 0.7415296413...
...

...
01-71: Uncountable R
Assume that there is some bijection from N to R
between 0 and 1

0 0.3412315569...
1 0.0123506541...
2 0.1143216751...
3 0.2839143215...
4 0.2311459412...
5 0.8381441234...
6 0.7415296413...
...
...

...

Consider: 0.425055...
01-72: Proof Techniques
Three basic proof techniques used in this class
Induction
Diagonalization
Pigeonhole Principle
01-73: Induction
Can create exact postage for any amount ≥ $0.08
using only 3 cent and 5 cent stamps
01-74: Induction
Can create exact postage for any amount ≥ $0.08
using only 3 cent and 5 cent stamps

Base case

Can create postage for 0.08 using one 5-cent and one
3-cent stamp
01-75: Induction
Can create exact postage for any amount ≥ $0.08
using only 3 cent and 5 cent stamps

Inductive case
To show: if we can create exact postage for $x
using only 3-cent and 5-cent stamps, we can
create exact postage for $x + $0.01 using
3-cent and 5-cent stamps
Two cases:
Exact postage for $x uses at least one 5-cent
stamp
Exact postage for $x uses no 5-cent stamps
01-76: Induction
To show: if we can create exact postage for $x
using only 3-cent and 5-cent stamps, we can
create exact postage for $x + $0.01 using 3-cent
and 5-cent stamps
Exact postage for $x uses at least one 5-cent
stamp
Replace a 5-cent stamp with two 3-cent
stamps to get $x + $0.01
Exact postage for $x uses no 5-cent stamps
Replace three 3-cent stamps with two 5-cent
stamps to get $ + $0.01
01-77: Pigeonhole Principle
A, B are finite sets, with |A| > |B|, then there is
no one-to-one function from A to B
If you have n pigeonholes, and > n pigeons, and
every pigeon is in a pigeonhole, there must be at
least one hole with > 1 pigeon.
01-78: Pigeonhole Principle
Show that in a relation R over a set A, if there is a
path from ai to aj in R, then there is a path from a
to b whose length is at most |A|.
01-79: Pigeonhole Principle
Proof by Contradiction

Assume that there exists some shortest path from

ai to aj of length > |A|.
By pigeonhole principle, some element must
repeat:
{ai , . . . , ak , . . . , ak . . . aJ }
We can create a shorter path by removing
elements between ak s.
We’ve just found a shorter path from ai to aj – a
contradiction