Unit I: Basic Foundations

1.1. Review of Set Theory, Logic, Functions, Proofs

1.2. Automata, Computability and Complexity:
• Complexity Theory,
• Computability Theory,
• Automata Theory
1.3. Basic concepts of Automata Theory:
• Alphabets,
• Power of Alphabet,
• Kleen Closure Alphabet,
• Positive Closure of Alphabet, Strings,
• Empty String,
• Suffix, Prefix and Substring of a string,
• Concatenation of strings,
• Languages,
• Empty Language,
• Membership in Language
Definition:
The Theory of Computation is a branch of theoretical computer science that deals with the study of
algorithms, computability, and computational complexity.
Importance:
It forms the fundamental of our understanding of what computers can and cannot do, influencing the
design and analysis of algorithms and programming languages.

Automata Theory:
• Study of abstract machine and their properties, providing a mathematical notion of “computer”.
• Automata are abstract mathematical models of machines that perform computations on an input
by moving through a series of states or configurations. If the computation of an automaton reaches
an accepting configuration it accepts that input.
• Examples of Abstract models of computation: Finite automata, pushdown automata, and Turing
machines.
Computability Theory:
• Examining the limits of what can be computed algorithmically.
Complexity Theory:
• Analyzing the efficiency of algorithms and classifying problems by their computational complexity.
Abstract Model
• An abstract model is a model of computer system (considered either as
hardware or software) constructed to allow a detailed and precise analysis of
how the computer system works.
• Such a model usually consists of input, output and operations that can be
performed and so can be thought of as a processor.
• E.g. an abstract machine that models a banking system can have operations
like “deposit”, “withdraw”, “transfer”, etc.

Fig: Finite automaton modeling an on/off switch Fig: Finite automaton modeling of language accepting even
number of 1’s or 0’s
 Applications of TOC
Web Form Validations Algorithms: TOC concepts are employed in validation of web forms.
Application: Allow regex rule to validate web form fields with complex rules.

Compiler Design: TOC principles are fundamental in lexical analysis and parsing stages of compiler design.
Application: Regular expressions and context-free grammars (CFGs) derived from TOC help define the syntax of
programming languages. Validates the rules and structure of programs.

Formal Language Specification: TOC provides the theoretical foundation for formal languages and their grammars.
Application: Finite-state machines are basis for programs for spell checking, indexing, grammar checking, searching large
bodies of text.

Algorithm Design and Analysis: TOC, particularly complexity theory, helps analyze and compare the efficiency of
algorithms.
Application: Engineers use TOC to choose the most suitable algorithm based on the problem's computational complexity.

Automated Verification and Model Checking: Automata theory and formal languages assist in formal methods for system
verification.
Application: Ensures correctness and reliability in critical systems, such as hardware designs or software protocols.

Artificial Intelligence (AI) and Machine Learning (ML): TOC contributes to the understanding of the computational limits
of AI and ML algorithms.
Application: Helps in assessing the feasibility and complexity of solving specific problems using AI and ML techniques.
BASIC TERMS::
Symbol (basic building block)
Alphabets (Σ) : some collection of symbol (we can form alphabet from symbol )
Strings (w): finite sequence of symbols selected from some alphabets
Languages (L): A set of strings all of which are chosen from Σ*, where Σ is particular
alphabet, is called a language
Mathematical Preliminaries
• Finite and Infinite Sets
• Cardinality of set:
• Subset
• Set operations (Union,Intersection,Differences)
• Sequences and Tuples
• Relations And Functions
• Closures
• Method of proofs:
Set: The term set is the collection of well-defined objects, which are called
the elements of the sets.
Sets are used throughout the theory of computation.
• Some examples of sets are given below:
• • The set of vowels in English alphabet
• • The set of natural numbers less than 12.
• • The set of odd numbers less than 20.
There are three methods to describe a set:
• a. Enumeration method: When the elements of a set are enumerated or
listed, we enclose them in braces.
e.g. A={1, 2, 3, ……, 100}
• b. Standard method or description method : Frequently used sets are
usually given symbols that are reserved for them only. e.g. N (Natural
Numbers)= {1, 2, 3, ……., n}
• c. Set builder method: Another way of representing set is to use set builder
method.
e.g. we could define the rational numbers as Q={x/y: x, yϵZ; y≠0}
Types of sets
• a. Empty set: A set, which has no element, is called as empty set or null set or
void set. It is denoted by Φ
• b. Singleton set: A set, which has single element, is called as singleton set.
• c. Disjoint sets: Two or more sets are said to be disjoints, if there are no common
elements among.
• d. Overlapping sets: Two or more sets are said to be disjoints, if there are at least
one common element among them.
• e. Finite sets: A set having specified number of elements is called as a finite set.
• f. Infinite sets: A set is called infinite set, if it is not finite set.
• g. Universal set: The set of all objects or things under consideration in discussion
is called the universal set.
Relation
• A relation is a correspondence between two sets (called the domain and the
range) such that to each element of the domain, there is assigned one or
more elements of the range.

Q. State the domain and range of the following relation. Is the relation a
function? {(2, –3), (4, 6), (3,–1), (6, 6), (2, 3)}
Solution:
• The above list of points, being a relationship between certain x's and certain
y's, is a relation. The domain is all the x-values, and the range is all the y-
values.
• To give the domain and the range, we just list the values without duplication:
• domain: {2, 3, 4, 6}
• range: {–3, –1, 3, 6}
Types of relation
• a. Identity relation:
Consider a set A={1,2,3} Then, its identity relation is: R={(1,1), (2,2),(3,3)}
• b. Reflexive relation:
Consider a set A={1,2,3} Then, one of the possible reflexive relations can be:
R={(1,1), (2,2), (3,3), (1,2), (1,3)}
• c. Symmetric relation:
Consider some of the examples of the symmetric relation,
R1={(1,2),(2,1),(1,3),(3,1)}
R2={(1,2),(1,3),(2,1),(3,1),(3,3)}
• d. Transitive relation:
Iff(x,y)∈R and (y,z)∈R⇒(x,z)∈R for all a,b,c∈A
• e. Equivalence relation: A relation is equivalence relation if it is reflexive,
symmetric and transitive at the same time
a. Identity relation:
• In an identity relation "R", every element of the set “A” is related to itself
only. Note the conditions conveyed through words “every” and “only”.
The word “every” conveys that identity relation consists of ordered pairs
of element with itself - all of them. The word “only” conveys that this
relation does not consist of any other combination.
• Consider a set A={1,2,3} Then, its identity relation is: R={(1,1), (2,2),(3,3)}
b. Reflexive relation
• In reflexive relation, "R", every element of the set “A” is related to itself.
The definition of reflexive relation is exactly same as that of identity
relation except that it misses the word “only” in the end of the sentence.
The implication is that this relation includes identity relation and permits
other combination of paired elements as well.
• Consider a set A={1,2,3} Then, one of the possible reflexive relations can
be:
R={(1,1), (2,2), (3,3), (1,2), (1,3)}
• However, following is not a reflexive relation: R1={(1,1), (2,2), (1,2), (1,3)}
c. Symmetric relation
• In symmetric relation, the instance of relation has a mirror image. It means that if
(1,3) is an instance, then (3,1) is also an instance in the relation. Clearly, an ordered
pair of element with itself like (1,1) or (2,2) is themselves their mirror images.
Consider some of the examples of the symmetric relation,
• R1={(1,2),(2,1),(1,3),(3,1)}
• R2={(1,2),(1,3),(2,1),(3,1),(3,3)}
• We have purposely jumbled up ordered pairs to emphasize that order of elements
in relation is not important. In order to decide symmetry of a relation, we need to
identify mirror pairs. We state the condition of symmetric relation as:
Iff(x,y)∈R⇒(y,x)∈R for all x,y∈A
• The symbol “Iff” means “If and only if”. Here one directional arrow means
“implies”.
• Alternatively, the condition of symmetric relation can be stated as: xRy⇒yRx for all
x,y∈A
d. Transitive relation
• If “R” be the relation on set A, then we state the condition of transitive
relation as:
• Iff(x,y)∈R and (y,z)∈R⇒(x,z)∈R for all a,b,c∈A
• Alternatively, xRy and yRz⇒xRz for all x,y,z∈A

e. Equivalence relation
• A relation is equivalence relation if it is reflexive, symmetric and
transitive at the same time.
• In order to check whether a relation is equivalent or not, we need to
check all three characterizations.
Function
• A function is a correspondence between two sets (called the domain and the
range) such that to each element of the domain, there is assigned exactly one
element of the range.

Q. Determine the domain and range of the given function:

• The domain is all values that x can take on. The only problem I have with this
function is that I cannot have a negative inside the square root. So we'll set the
insides greater than or equal to zero, and solve. The result will be domain are:
• –2x + 3 > 0 =>–2x > –3 =>2x < 3 =>x < 3/2
• Then the domain is "all x < 3/2".
• Alphabet, Powers of Alphabet
• Closure of an alphabet
• String, Empty String, Length of string
• Concatenating of string
 Alphabets (Represented by ‘Σ’)
• Alphabet is a finite non-empty set of symbols.
• The symbols can be the letters such as {a, b, c}, bits {0, 1}, digits {0, 1, 2, 3…
9}.
• It is denoted by ‘Σ’ symbol.
• E.g.: An alphabet of set of decimal numbers is given by Σ= {0, 1,…….., 9).
• The alphabet for binary number is Σ={0, 1}.

Power of alphabet
The set of all strings of certain length k from an alphabet is the kth power
of that alphabet.
i.e. Σk = {w / |w| = k}
If Σ = {0, 1} then,
Σ0 = {ε}
Σ1 = {0, 1}
Σ2 = {00, 01, 10, 11}
Σ3 = {000, 001, 010, 011, 100, 101, 110, 111}
Strings (Strings are denoted by lower case letters)
• A string or word is a finite sequence of symbols selected from
some alphabets.
• E.g. if Σ={a, b} then ‘abab’ is a string over Σ.

• A string is generally denoted by ‘w’. The empty string is the string

with 0 (zero) occurrence of symbols. This string is represented by ε
or e or ^

Concatenating of string
• Let w1 and w2 be two strings, then w1w2 denotes the concatenation of w1
and w2.
• e.g.
if w1 = abc, w2 = xyz, then
w1w2 = abcxyz
Empty String: -
• It is a string with zero occurrences of symbols. It is denoted by ‘ε’ (epsilon).

Length of String
• The length of a string w, denoted by | w |, is the number of positions for symbols in
w. we have for every string s, length (s) ≥ 0.
• | ε | = 0 as empty string have no symbols.
• | 0110 | = 4

Substring
• A string s is called substring of a string w if it is obtained by removing 0 or more
leading or trailing symbols in w. It is proper substring of w if s ≠ w.
• If s is a string then Substr (s, i, j) is substring of s beginning at ith position & ending at
jth position both inclusive.
Suffix of a string
• A string s is called a suffix of a string w if it is obtained by removing 0 or
more leading symbols in w. For example;
• w = abcd
• s = bcd is suffix of w.
• here s is proper suffix if s ≠ w.

Prefix of a string
• A string s is called a prefix of a string w if it is obtained by removing 0 or
more trailing symbols of w. For example;
• w = abcd
• s = abc is prefix of w,
• Here, s is proper suffix i.e. s is proper suffix if s ≠ w.
Kleen Closure/ Closure of an alphabet
• Kleen Closure (Closure of an alphabet ) is set of all all strings over
alphabet Σ with length 0 or more.
• ∴Σ* = Σ0 ∪ Σ1 ∪ Σ2 ∪ Σ3 ∪ ……………
• E.g. If albhabet , A = {X} ; Then Kleen Closure of A is:
• A* = {Xn | n = 0, 1, 2, …}

Positive Closure: -
• The set of all the strings over an alphabet Σ, except the empty
string is called positive closure and is denoted by Σ+.
• ∴Σ+ = Σ1 ∪ Σ2 ∪ Σ3 ∪……………
• E.g. If albhabet , A = {X} ; Then Positive Closure of A is:
• A+ = {Xn | n = 1, 2, …}
NOTE:
Σ* = Σ + U {ε}
• Let Σ = {0, 1} then
• Σ* = {ε, 0, 1, 00, 10, 01, 11,….}
• Σ1 = {0, 1}
• Σ2 = {00, 01, 10, 11}
• Σ3 = {000, 001, 010, 011, 100, 101, 110, 111}
• Σ+ = Σ1 U Σ2 U Σ3 U….
• Therefore, Σ* = Σ + U {ε}
Languages
• A set of strings all of which are chosen from Σ*, where Σ is particular
alphabet, is called a language.

• Let Σ = {0, 1} then,

• L = {all strings over Σ with equal number of 0’s and 1’s}
= {01, 0011, ……}
• α ⊆ Σ*
Problem

• A problem is the question of deciding whether a given

string is a member of some particular language.

• In other words, if Σ is an alphabet & L is a language

over Σ, then problem is;
• - Given a string w in Σ*, decide whether or not w is in L.
• Exercises:
1. Let A be a set with n distinct elements. How many different
binary relations on A are there?
2. If ∑= {a,b,c} then find the followings
• a. ∑1, ∑ 2, ∑3.
3. If ∑={0,1}. Then find the following languages
a. The language of string of length zero.
b. The language of strings of 0’s and 1’s with equal number of
each.
c. The language {0n1n | n≥1}
d. The language {0i0j | 0≤i≤j}.
e. The language of strings with odd number of 0’s and even
number of 1’s.
4. Define the Kleen closure and power of alphabets.
Exercises
1. What is set? What are its types?
2. How can we represent set?
3. What is relation? Give examples.
4. Explain the types of relations.
5. What do you mean function?
6. Define the following terms with examples:
a. String
b. Alphabet
c. Symbol
d. Keene closure
e. Function
f. Union
g. Concatenation
• Methods of proof
Theorem,
• Proof

• Theorem
• A theorem is a mathematical proposition that is true.
• Many theorems are conditional propositions.

• For example, if f(x) and g(x) are continuous then f(x)± g(x)
are also continuous.

• If theorem is of the form “if p then q”, the p is called

hypothesis and q is called conclusion.
• Proof
• A proof of a theorem is a logical argument that
establishes the theorem to be true.
• There are different types of proofs of a theorem. Some
of them are given below:
• • Trivial proof
• • Vacuous proof
• • Direct proof
• • Indirect proof
• • Proof by contradiction
• • Proof by cases
• • Proof by mathematical induction
• • Proof by counter examples
• Trivial proof

• We say is trivially true if q is true, and this kind of

proof (i.e. showing q is true for without referring to p)
is called a trivial proof.

• Consider an implication: p→q

• If it can be shown that q is true, then the implication is
always true by definition of an implication.
• Vacuous proofs

• Consider an implication: p→q

• If it can be shown that p is false, then the implication is
always true by definition of an implication.
• Note that you are showing that the antecedent is false

NOTE: a→B
• Direct Proofs
• To prove p→q, we start assuming hypothesis p is true and
we use information already available to prove q is true, and
if q is true then the argument is valid. This is called direct
proof.

• E.g. If a and b are odd integers, then a+b is an odd integers.

• Here a and b are odd integers. Since every odd numbers can
be written by 2l+1 where l is any integer.
• So, a = 2m+1, b = 2n+1 for some integers m and n
• Now, a+b = 2m+1+2n+1
• = 2m+2n+2 = 2(m+n+1) = 2*k where k
• = m+n+1 is any integer.
• This shows a+b is even (since not in form of 2l+1)
• Indirect proof (proof by contraposition)
• Since, p→q is equivalent to ¬q→¬p.
• To prove p→q, we assume the conclusion is false; using the fact
if p becomes false, original implication is true.

• e.g. If the product of two integers a and b is even, then either a

is even or b is even.
• Suppose, if possible both a and b are odd integers. So, a = 2m+1
and b = 2n+1.
• And axb = (2m+1)(2n+1)
• = 4mn+2m+2n+1 = 2(2mn+m+n)+1
• = 2l+1
• where l = 2mn+m+n, which is not true.
• So, our original implication is true.
• Proof by contradiction
• The following proof proceeds by contradiction. That is, we will assume that
the claim we are trying to prove is wrong and reach a contradiction. If all the
derivations along the way are correct, then the only thing that can be wrong
is the assumption, which was that the claim we are trying to prove does not
hold. This proves that the claim does hold.

• Eg: For all integers n, if n2 is odd, then n is odd.

• Suppose not. [We take the negation of the given statement and suppose it to
be true.]
• Assume, to the contrary, that ∃ an integer n such that n2 is odd and n is even.
[We must deduce the contradiction.]

• By definition of even, we have

• n = 2k for some integer k.
• So, by substitution we have
• n . n = (2k) . (2k)
• Now (2.k.k) is an integer because products of integers are integer; and 2 and
k are integers.
• Hence,
• n . n = 2 . (some integer)
• or n2 = 2. (some integer)
• and so by definition of n2 even, is even.

• So the conclusion is since n is even, n2, which is the

product of n with itself, is also even.
• This contradicts the supposition that n2 is odd. [Hence,
the supposition is false and the proposition is true.]
• Proof by cases
• You can sometimes prove a statement by:
• 1. Dividing the situation into cases which exhaust all the
possibilities; and
• 2. Showing that the statement follows in all cases.
• It's important to cover all the possibilities. And don't confuse
this with trying examples; an example
• is not a proof.

• Eg:
• Theorem: If n is a positive integer then n7 - n is divisible by 7.
• Proof:
• First we factor n7 - n = n(n6 - 1) = n(n3 - 1)(n3 + 1)
• = n(n-1)(n2 + n + 1)(n+1)(n2 - n + 1).
• Now there are 7 cases to consider, depending on n = 7 q + r
where r = 0, 1, 2, 3, 4, 5, 6, 7.
• Case 1: n = 7q. Then n7 - n has the factor n, which is divisible by
7.
• Case 2: n = 7q + 1. Then n7 - n has the factor n-1 = 7q which is
divisible by 7.
• Case 3: n = 7q + 2. Then the factor n2 + n + 1 = (7q + 2)2 + (7q+2)
+ 1 = 49 q2 + 35 q + 7 is clearly divisible by 7.
• Case 4: n = 7q + 3. Then the factor n2 - n + 1 = (7q + 3)2 - (7q+3)
+ 1 = 49 q2 + 35 q + 7 is clearly divisible by 7.
• Case 5: n = 7q + 4. Then the factor n2 + n + 1 = (7q + 4)2 + (7q+4)
+ 1 = 49 q2 + 63 q + 21 is clearly divisible by 7.
• Case 6: n = 7q + 5. Then the factor n2 - n + 1 = (7q + 5)2 - (7q+5)
+ 1 = 49 q2 + 63 q + 21 is clearly divisible by 7.
• Case 7: n = 7q + 6. Then the factor n + 1 = 7q +7 is clearly
divisible by 7.
• Poof by mathematical induction
• Mathematical induction is a powerful, yet straightforward method of
proving statements whose "domain" is a subset of the set of integers.
• Usually, a statement that is proven by induction is based on the set of
natural numbers.
• This statement can often be thought of as a function of a number n,
• where n = 1,2,3...

• Proof by induction involves three main steps:

• 1. proving the base of induction,
• 2. forming the induction hypothesis,
• 3. and finally proving that the induction hypothesis holds true for all
numbers in the domain.

• Proving the base of induction involves showing that the claim holds true for
some base value (usually 0, 1, or 2). There are sometimes many ways to do
this, and it can require some ingenuity.
• We will outline this with a simple example.
• Proof. (By Mathematical Induction.)
• 1. Initial Step.
• When n = 0, the formula gives us
• (1 - 1/22n)/2 = (1 - 1/2)/2 = 1/4 = a0.
• So the closed form formula gives us the correct answer when n =
0.
• Inductive Step.
• 2. Our inductive assumption is: Assume there is a k, greater than
or equal to zero, such that ak = (1 - 1/22k)/2. We must prove the
formula is true for n = k+1.
• First we appeal to the recurrsive definition of
• ak+1 = 2 ak(1-ak).
• 3. Next, we invoke the inductive assumption, for this k, to get
• ak+1 = 2 (1 - 1/22k)/2 (1 - (1 - 1/22k)/2)
• = (1 - 1/22k)(1 + 1/22k)/2
• = (1 - 1/22k+1)/2.
• Proof by Counterexample
• Consider a statement of the form ∀x∈ M, if P(x) then Q(x).
• Suppose that we wish to prove that this statement is false. In
order to disprove this statement, we have to find a value of x in
M for which P(x) is true and Q(x) is false. Such an x is called a
counterexample.

• Furthermore, proving that this statement is false is equivalent to

showing that its negation is true.
• The negation of the above statement is
• ∃x in M such that P(x) and not Q(x).
• ∃x M | P(x)^ ~Q(x).

• Finding an x that makes the above statement true will disprove

the original statement.