‫ر‬

‫الجامعة السعودية االلكتونية‬

‫ر‬
‫االلكتونية‬ ‫الجامعة السعودية‬

‫‪26/12/2021‬‬
Theory of Computing

Headline separator Module 1

Mathematical Preliminaries
Contents
1. Sets, relations and functions
2. Graphs and trees
3. Strings and Their Properties
4. Principle of Induction
5. Proof by Contradiction
Weekly Learning Outcomes
1. Explain the concepts of sets relations and functions.
2. Describe graphs and trees.
3. Explain the concept of strings and languages.
4. Able to prove some well-known identities using induction
technique.
Required Reading
1. Introduction to Formal Proof
2. Additional Forms of Proof
3. Inductive Proofs
4. The Central Concepts of Automata Theory
Chapter 2 (KLP Mishra, N. Chandrasekaran (2008). Theory of
Computer Science: Automata, Languages and Computation. (3rd
Edition). PHI.)
1. Sets, Relations and Functions
2. Graphs and Trees
3. Boolean Logic
Recommended Reading
https://www.inf.ed.ac.uk/teaching/courses/dmmr/slides/13-14/Ch2.pdf
https://ncert.nic.in/ncerts/l/kemh104.pdf

This Presentation is mainly dependent on the textbook: Introduction to Automata Theory, Languages, and Computation: Global Edition, 3rd edition (2013) PHI
by John E. Hopcroft, Rajeev Motwani and Jeffrey D. Ullman
• Sets, relations and functions
What is a set?
• A set is a group of “objects”
• Students in a class: { Ali, Idris, Fazal, Ravi}
• Courses offered by a department: { CS101, CS211, … }
• Colors: { red, orange, yellow, green, blue, purple }
• Although a set can contain (almost) anything, we will
most often use sets of numbers
• All positive numbers less than or equal to 7: {1, 2, 3, 4, 5, 6, 7}
• The set of all natural numbers: {1, 2, 3, 4, …}

7
Set properties

• Order does not matter

• {1, 2, 3, 4, 5, 6} is equivalent to {6, 3, 5, 2, 4, 1}

• Sets are notated with curly brackets

• Sets do not have duplicate elements
• Consider the set of vowels in the alphabet.
• Not correct! {a, e, i, o, o, u}
• Correct! {a, e, i, o, u}
• Note that a list is like a set, but order does matter and
duplicate elements are allowed

8
 Specifying a set

• Can use an ellipsis (…): B = {0, 1, 2, 3, …}

• Can cause confusion. Consider the set C = {3, 5, 7, …}. What
comes next?
• Can use set-builder notation
• D = {x | x is prime and x > 2}
• E = {x | x is odd and x > 2}
• A set is said to “contain” the various “members” or “elements” that
make up the set
• If an element p is a member of (or an element of) a set T, we use then
notation p  T
• 2  {1, 2, 3, 9}
• If an element is not a member of (or an element of) a set S, we use the
notation q  T
• 7  {1, 2, 3, 9}

9
Special sets
• U is the universal set – the set of all of elements (or the “universe”)
from which given any set is drawn
• For the set {-2, 0.4, 2}, U would be the real numbers
• For the set {0, 1, 2}, U could be the natural numbers (zero and up), the
integers, the rational numbers, or the real numbers, depending on the
context
• If a set has zero elements, it is called the empty (or null) set
• Written using the symbol 
• Thus,  = { }
• As the empty set is a set, it can be a element of other sets
• { , 1, 2, 3, x } is a valid set

10
Set equality
• Two sets are equal if they have the same elements
• {1, 2, 3, 4, 5} = {5, 4, 3, 2, 1}
• Remember that order does not matter!
• {1, 2, 3, 2, 4, 3, 2, 1} = {4, 3, 2, 1}
• Remember that duplicate elements do not matter!
• Two sets are not equal if they do not have the same
elements
• {1, 2, 3, 4, 5} ≠ {1, 2, 3, 4}

11
Subsets
• If all the elements of a set S are also elements of a set T, then S
is a subset of T
• For example, if S = {2, 4, 6} and T = {1, 2, 3, 4, 5, 6, 7}, then
S is a subset of T
• This is specified by S  T
• Or by {2, 4, 6}  {1, 2, 3, 4, 5, 6, 7}
• If S is not a subset of T, it is written as such:
• ST
• For example, {1, 2, 8}  {1, 2, 3, 4, 5, 6, 7}
• Note that any set is a subset of itself!
• Given set S = {2, 4, 6}, since all the elements of S are
elements of S, S is a subset of itself
• This is kind of like saying 5 is less than or equal to 5
• Thus, for any set S, S  S

12
Proper Subsets
• If S is a subset of T, and S is not equal to T, then S is a
proper subset of T
• Let T = {0, 1, 2, 3, 4, 5}
• If S = {1, 2, 3}, S is not equal to T, and S is a subset of T
• A proper subset is written as S  T
• Let R = {0, 1, 2, 3, 4, 5}. R is equal to T, and thus is a subset
(but not a proper subset) or T
• Can be written as: R  T and R  T (or just R = T)
• Let Q = {4, 5, 6}. Q is neither a subset or T nor a proper
subset of T

13
Set cardinality
• The cardinality of a set is the number of elements in a
set
• Written as |A|
• Examples
• Let R = {1, 2, 3, 4, 5}. Then |R| = 5
• || = 0
• Let S = {, {a}, {b}, {a, b}}. Then |S| = 4
• This is the same notation used for vector length in
geometry
• A set with one element is sometimes called a singleton
set

14
Power sets
• Given the set S = {0, 1}. What are all the possible subsets of S?
• They are:  (as it is a subset of all sets), {0}, {1}, and {0, 1}
• The power set of S (written as P(S)) is the set of all the subsets of S
• P(S) = { , {0}, {1}, {0,1} }
• Note that |S| = 2 and |P(S)| = 4

• Let T = {0, 1, 2}. The P(T) = { , {0}, {1}, {2}, {0,1}, {0,2}, {1,2},
{0,1,2} }
• Note that |T| = 3 and |P(T)| = 8

• P() = {  }
• Note that || = 0 and |P()| = 1

• If a set has n elements, then the power set will have 2n

elements

15
Cartesian products
• A Cartesian product is a set of all ordered 2-tuples where
each “part” is from a given set
• Denoted by A x B, and uses parenthesis (not curly brackets)
• For example, 2-D Cartesian coordinates are the set of all ordered
pairs Z x Z
• Recall Z is the set of all integers
• This is all the possible coordinates in 2-D space
• Example: Given A = { a, b } and B = { 0, 1 }, what is their Cartesian
product?
• C = A x B = { (a,0), (a,1), (b,0), (b,1) }
• Note that Cartesian products have only 2 parts in these examples
(later examples have more parts)
• Formal definition of a Cartesian product:
• A x B = { (a,b) | a  A and b  B }

16
Set operations: Union
• Formal definition for the union of two sets:
A U B = { x | x  A or x  B }
• Further examples
• {1, 2, 3} U {3, 4, 5} = {1, 2, 3, 4, 5}
• {New York, Washington} U {3, 4} = {New York, Washington, 3, 4}
• {1, 2} U  = {1, 2}
• Properties of the union operation
• AU=A Identity law
• AUU=U Domination law
• AUA=A Idempotent law
• AUB=BUA Commutative law
• A U (B U C) = (A U B) U C Associative law

17
Set operations: Intersection
• Formal definition for the intersection of two sets: A ∩ B = { x | x  A
and x  B }
• Further examples
• {1, 2, 3} ∩ {3, 4, 5} = {3}
• {New York, Washington} ∩ {3, 4} = 
• No elements in common
• {1, 2} ∩  = 
• Any set intersection with the empty set yields the empty set
• Properties of the intersection operation
• A∩U=A Identity law
• A∩= Domination law
• A∩A=A Idempotent law
• A∩B=B∩A Commutative law
• A ∩ (B ∩ C) = (A ∩ B) ∩ C Associative law

18
Disjoint sets
• Formal definition for disjoint sets: two sets are disjoint if their
intersection is the empty set
• Further examples
• {1, 2, 3} and {3, 4, 5} are not disjoint
• {New York, Washington} and {3, 4} are disjoint
• {1, 2} and  are disjoint
• Their intersection is the empty set
•  and  are disjoint!
• Their intersection is the empty set

19
Set operations: Difference

• Formal definition for the difference of two sets:

A - B = { x | x  A and x  B }
A-B=A∩B
• Further examples
• {1, 2, 3} - {3, 10, 15} = {1, 2}
• {Crow, Cat} - {2, 5} = {Crow, Cat}
• {11, 22} -  = {11, 22}
• The difference of any set S with the empty set will be the set S
• The difference of any set S with itself will be the empty set .

20
Complement sets
• Formal definition for the complement of a set: A = { x | x  A }
• Or U – A, where U is the universal set
• Further examples (assuming U = Z)
• {1, 2, 3} = { …, -2, -1, 0, 4, 5, 6, … }
• Properties of complement sets

• A=A Complementation law

• AUA=U Complement law
• A∩A= Complement law

21
Set identities: DeMorgan’s Law

• Very common law:

A B = A B
A B = A B

22
Relations and Functions

23
Definitions
• A relation between two variables x and y is a set of
ordered pairs.
• An ordered pair consists of an x and y-coordinate
◼ A relation may be viewed as ordered pairs, mapping
design, table, equation, or written in sentences.
• x-values are called input, independent variable, or
domain.
• y-values are called output, dependent variable, or
range.

24
Example 1
Is this a relation?

Input 4 –5 0 9 –1

Output –2 7

• Domain
{4, -5, 0, 9, -1}
• Range
{-2, 7}

25
Is relation a function?

• What is a function?
• A function is a relation in which every input has
exactly one output”.
• Focus on the x-coordinates, when given a relation
• If the set of ordered pairs has different x-coordinates,
it is a function.

• If the set of ordered pairs has same x-coordinates,

it is not a function.
• Y-coordinates have no bearing in determining
functions.

26
RELATIONS

R = {(x1, y1), (x2, y2), (x3, y3), …}

xi R yi

e. g. if R = ‘>’: 2 > 1, 3 > 2, 3 > 1

In relations xi can be repeated

Equivalence Relations
• Reflexive: xRx
• Symmetric: x R y yRx
• Transitive: x R Y and y R z xRz

Example: R = ‘=‘
•x=x
•x=y y=x
• x = y and y = z x=z
Equivalence Classes
For equivalence relation R equivalence class of x = {y : x R y}
Example:
R = { (1, 1), (2, 2), (1, 2), (2, 1), (3, 3), (4, 4), (3, 4), (4, 3) }
Equivalence class of 1 = {1, 2}
Equivalence class of 3 = {3, 4}
Example 2

{(0, −5),(1, −4),(2, −3),(3, −2),(4, −1),(5, 0)}

• Is this a relation?
◼ Yes

• Is this a function?
◼ Yes

30
Example 3
{(–1, −7), (1, 0), (2, −3), (0, −8), (0, 5), (–2, −1)}

• Is this a function?
◼ No

• Is this still a relation?

◼ Yes

31
Example 4
Which relation mapping represents a function?

One Two

–1 2 2
3
1 2 –1 3
0 3 3 –2
0

Ans. One

32
Example 5
Which relation mapping represents a function?

A. B. B.

Ans. B

33
Vertical Line Test
• Vertical Line Test: A relation is a function if a
vertical line drawn through its graph, passes
through only one point.
• The Pencil Test: Take a pencil and move it from
left to right (–x to x); if it crosses more than one
point, it is not a function

34
Vertical Line Test

Is this graph a function?

Ans. Yes

35
Vertical Line Test

Is this graph a function?

Ans. No

36
Discrete or Continuous Functions (1)

A Discrete function
▪ Domain: {-7, 1, 5, 7, 8, 10}
▪ Range: {1, 0, -7, 5, 2, 8}

37
Discrete or Continuous Functions (2)

A Continuous function
▪ Domain: [-8, 8]
▪ Range: [-6, 6]

38
Domain and Range in Real Life
• Johnny bought at most 10 tickets to a concert for him
and his friends. The cost of each ticket was $12.50.
• Complete the table below to list the possible domain
and range.
1 2 3 4 5 6 7 8 9 10
12.50 25.00 37.50 50 62.50 75 87.50 100 112.50 125

• What is the independent variable?

◼ The number of tickets bought.
• What is the dependent variable?
◼ The total cost of the tickets.

39
• Graphs and trees
What is a graph?
• Graphs represent the relationships among data items
• A graph G consists of
• a set V of nodes (vertices)
• a set E of edges: each edge connects two nodes
• Each node represents an item
• Each edge represents the relationship between two items

node

edge

41
Formal Definition of graph
• The set of nodes is denoted as V
• For any nodes u and v, if u and v are connected by an edge,
such edge is denoted as (u, v)
• The set of edges is denoted as E
• A graph G is defined as a pair (V, E)

v
(u, v)

42
Path and simple path
• A path from v1 to vk is a sequence of nodes v1, v2, …, vk that are
connected by edges (v1, v2), (v2, v3), …, (vk-1, vk)
• A path is called a simple path if every node appears at most once.

- v2, v3, v4, v2, v1 is a path

- v2, v3, v4, v5 is a path, also it is a simple path

v2 v3
v1

v4 v5

43
 Cycle and simple cycle

• A cycle is a path that begins and ends at the same node

• A simple cycle is a cycle if every node appears at most once, except for
the first and the last nodes
- v2, v3, v4, v5 , v3, v2 is a cycle
- v2, v3, v4, v2 is a cycle, it is also a simple cycle

v2
v1 v3

v4 v5

44
Connected graph

• A graph G is connected if there exists path between every pair of

distinct nodes; otherwise, it is disconnected

v2 v3
v1

v4 v5

This is a connected graph because there exists path

between every pair of nodes

45
Directed graph (digraph)
• All previous graphs are undirected graph
• If each edge in E has a direction, it is called a directed edge
• A directed graph is a graph where every edges is a directed
edge

Madinah Makkah
500

Directed edge
1400
1000

Riyadh
46
 Property of graph
• A undirected graph that is connected and has no cycle is a tree.
• A tree with n nodes have exactly n-1 edges.
• A connected undirected graph with n nodes must have at least n-1 edges.

47
 What is a tree?

• Trees are structures used to represent hierarchical

relationship
• Each tree consists of nodes and edges
• Each node represents an object
• Each edge represents the relationship between two nodes.

node
edge

48
Terminology I
• For any two nodes u and v, if there is an edge pointing
from u to v, u is called the parent of v while v is called
the child of u. Such edge is denoted as (u, v).
• In a tree, there is exactly one node without parent,
which is called the root. The nodes without children
are called leaves.

root

u
u: parent of v
v: child of u
v
leaves 49
Terminology II

• In a tree, the nodes without children are called leaves. Otherwise,

they are called internal nodes.

internal nodes

leaves
50
 Terminology III
• If two nodes have the same parent, they are siblings.
• A node u is an ancestor of v if u is parent of v or
parent of parent of v or …
• A node v is a descendent of u if v is child of v or child
of child of v or …

v and w are siblings

u and v are ancestors of x v w
v and x are descendents of u
x
51
Terminology IV

• A subtree is any node together with all its descendants.

T
A subtree of T
v v

52
 Terminology V
• Level of a node n: number of nodes on the path from root to
node n
• Height of a tree: maximum level among all of its node

Level 1

Level 2
height=4
n Level 3

Level 4

53
Binary Tree
• Binary Tree: Tree in which every node has at most 2 children
• Left child of u: the child on the left of u
• Right child of u: the child on the right of u

x: left child of u
u v y: right child of u
w: right child of v
z: left child of w
x y w
z
54
Property of binary tree (I)
• A full binary tree of height h has 2h-1 nodes
No. of nodes = 20 + 21 + … + 2(h-1)
= 2h – 1
Level 1: 20 nodes

Level 2: 21 nodes

Level 3: 22 nodes

Consider a binary tree T of height h. The number of nodes of T  2h-1

Reason: you cannot have more nodes than a full binary tree of height h.
The minimum height of a binary tree with n nodes is log(n+1)
By property (II), n  2h-1
Thus, 2h  n+1
That is, h  log2 (n+1)
55
• Strings and Their Properties
Alphabet

An alphabet is a finite, non-empty set of

symbols.
• The symbol ∑ (sigma) is used to denote an
alphabet.
• Examples:
• Binary: ∑ = {0, 1}
• All lower case letters: ∑ = {a, b, c, …, z}
• Alphanumeric: ∑ = {a-z, A-Z, 0-9}

57
Strings
A string or word is a finite sequence of symbols
taken from ∑.
• The empty string is  (epsilon).

• Length of a string w, denoted by |w|, is equal

to the number of (non-) characters in the
string.
• E.g., x = 010100 |x| = 6
• x = 01  0  1  00  |x| = 6

• xy = concatenation of two strings x and y

58
Powers of an alphabet
Let ∑ be an alphabet.

• ∑k = the set of all strings of length k

• ∑* = ∑0 U ∑1 U ∑2 U …

• ∑+ = ∑1 U ∑2 U ∑3 U …

59
Languages
L is a said to be a language over alphabet ∑, only if L  ∑*
▪ This is because ∑* is the set of all strings (of all possible length
including 0) over the given alphabet ∑.
Examples:
1. Let L be the language of all strings consisting of n 0’s followed
by n 1’s:
L = {, 01, 0011, 000111,…}
1. Let L be the language of all strings of equal number of 0’s and
1’s:
L = {, 01, 10, 0011, 1100, 0101, 1010, 1001,…}

Definition: Ø denotes the Empty language

• Let L = {}; Is L = Ø?
• NO

60
The Membership Problem
Given a string w ∑*and a language L over ∑, decide whether
or not w  L.

Example:
Let w = 100011
Q) Is w  the language of strings with equal number of 0s
and 1s?
A) Yes

61
Finite Automata
• Some Applications
• Software for designing and checking the behavior of
digital circuits
• Lexical analyzer of a typical compiler
• Software for scanning large bodies of text (e.g., web
pages) for pattern finding
• Software for verifying systems of all types that have a
finite number of states (e.g., stock market transaction,
communication/network protocol)

62
Finite Automata: Examples
• On/Off switch action

state

• Modeling recognition of the

word “then”

Start state Transition Intermediate Final state

state

63
Structural expressions
• Grammars
• Regular expressions
• E.g., Unix style to capture city names such as “Palo Alto CA”:
• [A-Z][a-z]*([ ][A-Z][a-z]*)*[ ][A-Z][A-Z]

Other space delimited words

Start with a letter (part of city name)
Should end w/ 2-
A string of other letter state code
letters (possibly empty)

64
• Principle of Induction
Induction
This method consists of three basic steps:
Step 1: Prove P(n) for n = 0 or 1. This is called the proof for the basis.
Step 2: Assume the result/properties for P(n). This is called the
induction hypothesis.
Step 3: Prove P(n + 1) using the induction hypothesis.
If we have statements P1, P2, P3, …
If we know:
for some k that P1, P2, …, Pk are true, for any n >= k tha P1, P2, …, Pn
imply Pn+1, then every Pi is true.
Proof by Induction (modified)
1. Inductive basis:
▪ Find P1, P2, …, Pk which are true
2. Inductive hypothesis:
▪ Let us assume P1, P2, …, Pn are true, for any n >= k
3. Inductive step:
▪ Show that Pn+1 is true
Example
Theorem: A binary tree of height n has at most 2n leaves.
Proof:
let l(i) be the number of leaves at level i

l(0) = 0
l(1) = 2
l(2) = 4

l(3) = 8
Induction Step

Level
n hypothesis: l(n) <= 2n

n+1
We want to show: l(i) <= 2i
• Inductive basis
l(0) = 1 (the root node)
• Inductive hypothesis
Let’s assume l(i) <= 2i for all i = 0, 1, …, n
• Induction step
•we need to show that l(n + 1) <= 2n+1
 Induction Step

Level
n hypothesis: l(n) <= 2n

n+1

l(n+1) <= 2 * l(n) <= 2 * 2n = 2n+1

• Proof by Contradiction
Proof by Contradiction

We want to prove that a statement P is true.

▪ We assume that P is false
▪ Then we arrive at an incorrect conclusion
▪ Therefore, statement P must be true
Example (1)
Theorem: 2 is not rational
Proof:
Assume by contradiction that it is rational

2 = n/m
n and m have no common factors
We will show that this is impossible
Example (2)

2 = n/m 2 m2 = n2
n is even
Therefore, n2 is even
n=2k
m is even
2 m2 = 4k2 m2 = 2k2
m=2p
Therefore, m and n have common factor 2
Contradiction!
Deductive Proofs

From the given statement(s) to a conclusion

statement (what we want to prove)
• Logical progression by direct implications

Example for parsing a statement:

• “If y ≥ 4, then 2y ≥ y2.”
given conclusion

(there are other ways of writing this).

76
Example: Deductive proof
Let Claim 1: If y ≥ 4, then 2y ≥ y2.

Let x be any number which is obtained by adding the squares of 4

positive integers.
Claim 2:
Given x and assuming that Claim 1 is true, prove that 2x ≥ x2
Proof:
1) Given: x = a2 + b2 + c2 + d2
2) Given: a ≥ 1, b ≥ 1, c ≥ 1, d ≥ 1
3) ➔ a2 ≥ 1, b2 ≥ 1, c2 ≥ 1, d2 ≥ 1 (by 2)
4) ➔x≥4 (by 1 & 3)
5) ➔ 2x ≥ x2 (by 4 and Claim 1)

“implies” or “follows”

77
Main Reference
1. Sets, set operations, and terminologies
2. Relations, functions and their kinds
3. Graphs and Trees
4. Strings and Languages
5. Boolean Logic
Fundamental Proof Techniques (KLP Mishra, N. Chandrasekaran
(2008). Theory of Computer Science: Automata, Languages and
Computation. (3rd Edition). PHI.)
Additional References
https://www.inf.ed.ac.uk/teaching/courses/dmmr/slides/13-14/Ch2.pdf
https://ncert.nic.in/ncerts/l/kemh104.pdf
https://www.cs.virginia.edu/~asb/teaching/cs202-spring07/slides/07-sets.ppt
https://www.paulding.k12.ga.us/cms/lib/GA01903603/Centricity/Domain/3809/Relations%20and
%20Functions%20Power%20Point.ppt

This Presentation is mainly dependent on the textbook: Introduction to Automata Theory, Languages, and Computation: Global Edition, 3rd edition (2013) PHI
by John E. Hopcroft, Rajeev Motwani and Jeffrey D. Ullman
Thank You