Lenguajes Formales y Compiladores

Chomsky Classification and Turing Machines

Sergio Ramı́rez Rico

Escuela de Ciencias Aplicadas e Ingenierı́a

Área de Ciencias Fundamentales

1 / 87
Kozen 1997, Lectures 28, 29, 32, 34.

2 / 87
Table of Contents

1. Chomsky Classification

2. Turing Machines

3. Decidability

4. Examples: Grammar Types

3 / 87
Overview

Chomsky defined four rule types:

4 / 87
Overview

Chomsky defined four rule types:

• A rule of type 3 coincides with the unilinear form (grammars for regular languages)

5 / 87
Overview

Chomsky defined four rule types:

• A rule of type 3 coincides with the unilinear form (grammars for regular languages)
• A rule of type 2 is context-free: the left part is one nonterminal

6 / 87
Overview

Chomsky defined four rule types:

• A rule of type 3 coincides with the unilinear form (grammars for regular languages)
• A rule of type 2 is context-free: the left part is one nonterminal
• A rule of type 1: the right part of a rule must be at least as long as the left part

7 / 87
Overview

Chomsky defined four rule types:

• A rule of type 3 coincides with the unilinear form (grammars for regular languages)
• A rule of type 2 is context-free: the left part is one nonterminal
• A rule of type 1: the right part of a rule must be at least as long as the left part
• A rule of type 0 can replace an arbitrary nonempty string over terminals and
nonterminals, with another arbitrary string

8 / 87
Chomsky Classification

Grammar Production Language Recognizer

type form family model

9 / 87
Chomsky Classification

Grammar Production Language Recognizer

type form family model

A → uB or A → Bu, with A ∈ N ,
Type 3 Regular Finite automaton
u ∈ Σ∗ and B ∈ (N ∪ {ε})

10 / 87
Chomsky Classification

Grammar Production Language Recognizer

type form family model

A → uB or A → Bu, with A ∈ N ,
Type 3 Regular Finite automaton
u ∈ Σ∗ and B ∈ (N ∪ {ε})
Type 2 A → α with A ∈ N and α ∈ (Σ ∪ N )∗ Context-free Pushdown automaton

11 / 87
Chomsky Classification

Grammar Production Language Recognizer

type form family model

A → uB or A → Bu, with A ∈ N ,
Type 3 Regular Finite automaton
u ∈ Σ∗ and B ∈ (N ∪ {ε})
Type 2 A → α with A ∈ N and α ∈ (Σ ∪ N )∗ Context-free Pushdown automaton
Turing machine with
β → α with α, β ∈ (Σ ∪ N )+
Type 1 Context-depend space complexity limited
and |β| ≤ |α|
by input length

12 / 87
Chomsky Classification

Grammar Production Language Recognizer

type form family model

A → uB or A → Bu, with A ∈ N ,
Type 3 Regular Finite automaton
u ∈ Σ∗ and B ∈ (N ∪ {ε})
Type 2 A → α with A ∈ N and α ∈ (Σ ∪ N )∗ Context-free Pushdown automaton
Turing machine with
β → α with α, β ∈ (Σ ∪ N )+
Type 1 Context-depend space complexity limited
and |β| ≤ |α|
by input length
β → α with β ∈ (Σ ∪ N )+ and Recursively
Type 0 Turing machine
α ∈ (Σ ∪ N )∗ enumerable

13 / 87
Table of Contents

1. Chomsky Classification

2. Turing Machines

3. Decidability

4. Examples: Grammar Types

14 / 87
Motivation

• Alan Turing invented them in 1936.

15 / 87
Motivation

• Alan Turing invented them in 1936.

• Turing machines (TM) are the most powerful automata we study in this course.

16 / 87
Motivation

• Alan Turing invented them in 1936.

• Turing machines (TM) are the most powerful automata we study in this course.

• Turing machines can compute any function considered computable

17 / 87
Motivation

• Alan Turing invented them in 1936.

• Turing machines (TM) are the most powerful automata we study in this course.

• Turing machines can compute any function considered computable

▶ It is reasonable to define computable to mean computable by a TM

18 / 87
Motivation

• Alan Turing invented them in 1936.

• Turing machines (TM) are the most powerful automata we study in this course.

• Turing machines can compute any function considered computable

▶ It is reasonable to define computable to mean computable by a TM

• They can be thought of as the predecessor of CPU’s

19 / 87
Relation with Other Automata

Available at https://commons.wikimedia.org/wiki/File:Teor%C3%ADa_de_aut%C3%B3matas.svg
20 / 87
Effective Computation

• Turing machines

21 / 87
Effective Computation

• Turing machines
• Post systems

22 / 87
Effective Computation

• Turing machines
• Post systems
• µ-recursive functions

23 / 87
Effective Computation

• Turing machines
• Post systems
• µ-recursive functions
• λ-calculus

24 / 87
Effective Computation

• Turing machines
• Post systems
• µ-recursive functions
• λ-calculus
• Combinatory logic

25 / 87
Church’s Thesis

Church’s Thesis
Any effectively computable function can be computed by a Turing machine.

26 / 87
Church’s Thesis

Church’s Thesis
Any effectively computable function can be computed by a Turing machine.

No computational procedure will be considered as an algorithm unless it can be represented as

a Turing Machine.

27 / 87
Turing Machine (Informal)

• The input string is finite

28 / 87
Turing Machine (Informal)

• The input string is finite

• It can read or write on the tape and moves either to the right or to the left

29 / 87
Turing Machine (Informal)

• The input string is finite

• It can read or write on the tape and moves either to the right or to the left
• It has a finite set of states, the transition between states is determined by a transition
function δ

30 / 87
Turing Machine (Informal)

• The input string is finite

• It can read or write on the tape and moves either to the right or to the left
• It has a finite set of states, the transition between states is determined by a transition
function δ
• It accepts an input by entering a special accept state t

31 / 87
Turing Machine (Informal)

• The input string is finite

• It can read or write on the tape and moves either to the right or to the left
• It has a finite set of states, the transition between states is determined by a transition
function δ
• It accepts an input by entering a special accept state t
• It rejects by entering a special reject state r

32 / 87
Turing Machine (Informal)

• The input string is finite

• It can read or write on the tape and moves either to the right or to the left
• It has a finite set of states, the transition between states is determined by a transition
function δ
• It accepts an input by entering a special accept state t
• It rejects by entering a special reject state r
• On some inputs it may run without ever accepting or rejecting: Loop on input
33 / 87
Turing Machine (Formal Specification)

A deterministic one-tape Turing machine is a 9-tuple M = (Q, Σ, Γ, ⊢, ⊔, δ, s, t, r) where

34 / 87
Turing Machine (Formal Specification)

A deterministic one-tape Turing machine is a 9-tuple M = (Q, Σ, Γ, ⊢, ⊔, δ, s, t, r) where

• Q is a finite set (the states)

35 / 87
Turing Machine (Formal Specification)

A deterministic one-tape Turing machine is a 9-tuple M = (Q, Σ, Γ, ⊢, ⊔, δ, s, t, r) where

• Q is a finite set (the states)

• Σ is a finite set (the input alphabet)

36 / 87
Turing Machine (Formal Specification)

A deterministic one-tape Turing machine is a 9-tuple M = (Q, Σ, Γ, ⊢, ⊔, δ, s, t, r) where

• Q is a finite set (the states)

• Σ is a finite set (the input alphabet)

• Γ is a finite set (the tape alphabet)

(Σ ⊆ Γ)

37 / 87
Turing Machine (Formal Specification)

A deterministic one-tape Turing machine is a 9-tuple M = (Q, Σ, Γ, ⊢, ⊔, δ, s, t, r) where

• Q is a finite set (the states)

• Σ is a finite set (the input alphabet)

• Γ is a finite set (the tape alphabet)

(Σ ⊆ Γ)

• ⊔ ∈ Γ − Σ, the blank symbol

38 / 87
Turing Machine (Formal Specification)

A deterministic one-tape Turing machine is a 9-tuple M = (Q, Σ, Γ, ⊢, ⊔, δ, s, t, r) where

• Q is a finite set (the states)

• Σ is a finite set (the input alphabet)

• Γ is a finite set (the tape alphabet)

(Σ ⊆ Γ)

• ⊔ ∈ Γ − Σ, the blank symbol

• ⊢∈ Γ − Σ, the left endmarker

39 / 87
Turing Machine (Formal Specification)

A deterministic one-tape Turing machine is a 9-tuple M = (Q, Σ, Γ, ⊢, ⊔, δ, s, t, r) where

• Q is a finite set (the states) • δ : Q × Γ → Q × Γ × {L, R}, the

transition function
• Σ is a finite set (the input alphabet)

• Γ is a finite set (the tape alphabet)

(Σ ⊆ Γ)

• ⊔ ∈ Γ − Σ, the blank symbol

• ⊢∈ Γ − Σ, the left endmarker

40 / 87
Turing Machine (Formal Specification)

A deterministic one-tape Turing machine is a 9-tuple M = (Q, Σ, Γ, ⊢, ⊔, δ, s, t, r) where

• Q is a finite set (the states) • δ : Q × Γ → Q × Γ × {L, R}, the

transition function
• Σ is a finite set (the input alphabet)
• s ∈ Q, the start state
• Γ is a finite set (the tape alphabet)
(Σ ⊆ Γ)

• ⊔ ∈ Γ − Σ, the blank symbol

• ⊢∈ Γ − Σ, the left endmarker

41 / 87
Turing Machine (Formal Specification)

A deterministic one-tape Turing machine is a 9-tuple M = (Q, Σ, Γ, ⊢, ⊔, δ, s, t, r) where

• Q is a finite set (the states) • δ : Q × Γ → Q × Γ × {L, R}, the

transition function
• Σ is a finite set (the input alphabet)
• s ∈ Q, the start state
• Γ is a finite set (the tape alphabet)
(Σ ⊆ Γ) • t ∈ Q, the accept state

• ⊔ ∈ Γ − Σ, the blank symbol

• ⊢∈ Γ − Σ, the left endmarker

42 / 87
Turing Machine (Formal Specification)

A deterministic one-tape Turing machine is a 9-tuple M = (Q, Σ, Γ, ⊢, ⊔, δ, s, t, r) where

• Q is a finite set (the states) • δ : Q × Γ → Q × Γ × {L, R}, the

transition function
• Σ is a finite set (the input alphabet)
• s ∈ Q, the start state
• Γ is a finite set (the tape alphabet)
(Σ ⊆ Γ) • t ∈ Q, the accept state

• ⊔ ∈ Γ − Σ, the blank symbol • r ∈ Q, the reject state, r ̸= t

• ⊢∈ Γ − Σ, the left endmarker

43 / 87
Meaining of δ

Notice that δ(p, a) = (q, b, d) means:

44 / 87
Meaining of δ

Notice that δ(p, a) = (q, b, d) means:

When in state p scanning symbol a, write b on that tape cell, move the head in direction d,
and enter state q.

45 / 87
Example TM

Turing machine that accepts the non-context free set {an bn cn | n ≥ 0}.

Formalization
• Q = {s, q1 , . . . , q10 , t, r}
• Σ = {a, b, c}
• Γ = Σ ∪ {⊢, ⊔, ⊣} where ⊣ denotes the right endmarker.

46 / 87
Example TM (ongoing)
Turing machine that accepts the non-context free set {an bn cn | n ≥ 0}

Available in Kozen 1997 47 / 87

TM Restrictions

• The left endmarker ⊢ is never overwritten with other symbol and the machine never
moves off the tape to the left:

(∀p ∈ Q)(∃q ∈ Q)(δ(p, ⊢) = (q, ⊢, R))

48 / 87
TM Restrictions

• The left endmarker ⊢ is never overwritten with other symbol and the machine never
moves off the tape to the left:

(∀p ∈ Q)(∃q ∈ Q)(δ(p, ⊢) = (q, ⊢, R))

• Once the machine enters its accept/reject state, it never leaves it:

49 / 87
TM Restrictions

• The left endmarker ⊢ is never overwritten with other symbol and the machine never
moves off the tape to the left:

(∀p ∈ Q)(∃q ∈ Q)(δ(p, ⊢) = (q, ⊢, R))

• Once the machine enters its accept/reject state, it never leaves it:

(∀b ∈ Γ)(∃c ∈ Γ)(∃d ∈ {R, L})(δ(t, b) = (t, c, d))

50 / 87
TM Restrictions

• The left endmarker ⊢ is never overwritten with other symbol and the machine never
moves off the tape to the left:

(∀p ∈ Q)(∃q ∈ Q)(δ(p, ⊢) = (q, ⊢, R))

• Once the machine enters its accept/reject state, it never leaves it:

(∀b ∈ Γ)(∃c ∈ Γ)(∃d ∈ {R, L})(δ(t, b) = (t, c, d))

(∀b ∈ Γ)(∃c ∈ Γ)(∃d ∈ {R, L})(δ(r, b) = (r, c, d))

51 / 87
Configurations

Definition (Configurations)
A configuration of a Turing machine is an element of Q × {y ⊔ω | y ∈ Σ∗ } × N, where ω is
the smallest infinite ordinal and ⊔ω denotes the semi-infinite string ⊔⊔⊔ · · · .

52 / 87
Configurations

Definition (Configurations)
A configuration of a Turing machine is an element of Q × {y ⊔ω | y ∈ Σ∗ } × N, where ω is
the smallest infinite ordinal and ⊔ω denotes the semi-infinite string ⊔⊔⊔ · · · .

A configuration (p, z, n) specifies a current state p of the finite control, current tape contents
z and current position of the read/write head n ≥ 0.

53 / 87
Configurations

Definition (Configurations)
A configuration of a Turing machine is an element of Q × {y ⊔ω | y ∈ Σ∗ } × N, where ω is
the smallest infinite ordinal and ⊔ω denotes the semi-infinite string ⊔⊔⊔ · · · .

A configuration (p, z, n) specifies a current state p of the finite control, current tape contents
z and current position of the read/write head n ≥ 0.

Start Configuration
The start configuration on input x ∈ Σ∗ is the configuration (s, ⊢x⊔ω , 0).

54 / 87
Next Configuration Relation

Definition
For a string z ∈ Γω :
• Let zn be the nth symbol of z (the leftmost symbol is z0 )

55 / 87
Next Configuration Relation

Definition
For a string z ∈ Γω :
• Let zn be the nth symbol of z (the leftmost symbol is z0 )
• Let snb (z) denote the string obtained from z by substituting b for zn at position n.

56 / 87
Next Configuration Relation

Definition
For a string z ∈ Γω :
• Let zn be the nth symbol of z (the leftmost symbol is z0 )
• Let snb (z) denote the string obtained from z by substituting b for zn at position n.

For example:
s4b (⊢baaacabca · · · ) = ⊢baabcabca · · ·

57 / 87
Next Configuration Relation

Definition
1
Let M be a Turing machine. The relation −→ is defined by:
M
(
1 (q, snb (z), n − 1) if δ(p, zn ) = (q, b, L)
(p, z, n) −→
M (q, snb (z), n + 1) if δ(p, zn ) = (q, b, R)

58 / 87
Next Configuration Relation

Definition
∗ 1
Let M be a Turing machine. We define the reflexive transitive closure −→ of −→ inductively:
M M
0
• α −→ α,
M

59 / 87
Next Configuration Relation

Definition
∗ 1
Let M be a Turing machine. We define the reflexive transitive closure −→ of −→ inductively:
M M
0
• α −→ α,
M

• α −−→ β if α −n→ γ −→
n+1 1
β for some γ, and
M M M

60 / 87
Next Configuration Relation

Definition
∗ 1
Let M be a Turing machine. We define the reflexive transitive closure −→ of −→ inductively:
M M
0
• α −→ α,
M

• α −−→ β if α −n→ γ −→
n+1 1
β for some γ, and
M M M
∗ n
• α −→ β if α −→ β for some n ≥ 0.
M M

61 / 87
Language Accepted by a TM

Definition
Let M = (Q, Σ, Γ, ⊢, ⊔, δ, s, t, r) be a Turing machine and x ∈ Σ∗ :

62 / 87
Language Accepted by a TM

Definition
Let M = (Q, Σ, Γ, ⊢, ⊔, δ, s, t, r) be a Turing machine and x ∈ Σ∗ :
• M is said to accept x iff
∗
(s, ⊢x⊔ω , 0) −→ (t, y, n)
M

for some y and n,

63 / 87
Language Accepted by a TM

Definition
Let M = (Q, Σ, Γ, ⊢, ⊔, δ, s, t, r) be a Turing machine and x ∈ Σ∗ :
• M is said to accept x iff
∗
(s, ⊢x⊔ω , 0) −→ (t, y, n)
M

for some y and n,

• M is said to reject x iff
∗
(s, ⊢x⊔ω , 0) −→ (r, y, n)
M

for some y and n.

64 / 87
Language Accepted by a TM

Definition
Let M = (Q, Σ, Γ, ⊢, ⊔, δ, s, t, r) be a Turing machine. The language accepted by M , denoted
L(M ), is defined as
∗
L(M ) = {x ∈ Σ∗ | (s, ⊢x⊔ω , 0) −→ (t, y, n) for some y and n}
M

65 / 87
TM Characteristics

• It is said that a TM halts on input x if it either accept or rejects x

66 / 87
TM Characteristics

• It is said that a TM halts on input x if it either accept or rejects x

▶ It is possible that it neither accepts nor rejects: loop on input

67 / 87
TM Characteristics

• It is said that a TM halts on input x if it either accept or rejects x

▶ It is possible that it neither accepts nor rejects: loop on input
• A TM is said to be total if it halts on all inputs

68 / 87
TM Characteristics

• It is said that a TM halts on input x if it either accept or rejects x

▶ It is possible that it neither accepts nor rejects: loop on input
• A TM is said to be total if it halts on all inputs
• A programming language is said to be Turing Complete if it can simulate any TM

69 / 87
Languages Accepted by TM

Acceptance
A language L is

70 / 87
Languages Accepted by TM

Acceptance
A language L is
• recursively enumerable (r.e.) iff there exists a Turing machine M that accepts L,

71 / 87
Languages Accepted by TM

Acceptance
A language L is
• recursively enumerable (r.e.) iff there exists a Turing machine M that accepts L,
• co-recursively enumerable (co-r.e.) iff its complement is r.e.,

72 / 87
Languages Accepted by TM

Acceptance
A language L is
• recursively enumerable (r.e.) iff there exists a Turing machine M that accepts L,
• co-recursively enumerable (co-r.e.) iff its complement is r.e.,
• recursive iff there exists a total Turing machine M that accepts L,

73 / 87
Exercise

Exercise
Consider the non-CFL {ww | w ∈ {a, b}∗ }.
Propose a strategy to design a TM for this language.

74 / 87
Table of Contents

1. Chomsky Classification

2. Turing Machines

3. Decidability

4. Examples: Grammar Types

75 / 87
Decidability and Semidecibality: Equivalences

P is decidable ⇐⇒ {x|P (x)} is recursive ⇐⇒ {x | P (x)} = L(M ) for some total TM M

P is semidecidable ⇐⇒ {x | P (x)} is r.e. ⇐⇒ {x | P (x)} = L(M ) for some TM M

76 / 87
Decidability and Semidecibality: Equivalences

P is decidable ⇐⇒ {x|P (x)} is recursive ⇐⇒ {x | P (x)} = L(M ) for some total TM M

• A is recursive ⇐⇒ “x ∈ A” is decidable

P is semidecidable ⇐⇒ {x | P (x)} is r.e. ⇐⇒ {x | P (x)} = L(M ) for some TM M

77 / 87
Decidability and Semidecibality: Equivalences

P is decidable ⇐⇒ {x|P (x)} is recursive ⇐⇒ {x | P (x)} = L(M ) for some total TM M

• A is recursive ⇐⇒ “x ∈ A” is decidable

P is semidecidable ⇐⇒ {x | P (x)} is r.e. ⇐⇒ {x | P (x)} = L(M ) for some TM M

• A is r.e. ⇐⇒ “x ∈ A” is semidecidable

78 / 87
Decidability and Semidecibality

Some decidable problems

• Whether a given string x is of the form ww.
• Whether a given TM has at least 481 states.
• Whether a given TM takes more than 481 steps on input ε.

79 / 87
Decidability and Semidecibality

Some decidable problems

• Whether a given string x is of the form ww.
• Whether a given TM has at least 481 states.
• Whether a given TM takes more than 481 steps on input ε.

Some semidecidable problems

• Whether a polynomial equation with integer coefficients have a solution in the integers
• Whether a program written in some programming language will crash on some input.

80 / 87
Undecidability

Some undecidable problems

• Whether a given TM accepts ε.
• Whether a given TM accepts a regular set.
• Whether a given TM accepts a CFL.
• Halting problem.

81 / 87
Undecidability is the rule, not the exception

Theorem (Rice’s Theorem)

Every nontrivial property of recursively enumerable sets is undecidable.

82 / 87
Table of Contents

1. Chomsky Classification

2. Turing Machines

3. Decidability

4. Examples: Grammar Types

83 / 87
Type 1 Grammar

Example
The language {an bn cn | n ≥ 1} is generated by the context-sensitive grammar:

1. S → aSBC 4. bB → bb
2. S → abC 5. bC → bc
3. CB → BC 6. cC → cc

84 / 87
Type 1 Grammar

The language {ycy | y ∈ {a, b}+ } is generated by the context-sensitive grammar including ⊣
at the end of each sentence:

S→X⊣ XA → XA′ A′ A → AA′ A′ ⊣ → a B ′ a → ba

X → aXA XB → XB ′ A′ B → BA′ B′ ⊣ → b B ′ b → bb
X → bXB B ′ A → AB ′ A′ a → aa Xa → ca
′ ′ ′
B B → BB A b → ab Xb → cb

The ⊣ symbol is assumed to appear at the right end of the strings in order to simplify the
grammar.

85 / 87
Type 0 Grammar

The language {an | n is a power of 2} is generated by the following grammar

S → ⊢Ca⊣ | a Ca → aaC ⊢D → ⊢C
C⊣ → D⊣ | E aD → Da aE → Ea
⊢E → ε

86 / 87
References

Aho, Alfred V. et al. (2006). Compilers: Principles, Techniques, and Tools (2nd Edition).
USA: Addison-Wesley Longman Publishing Co., Inc. isbn: 0321486811.
Kozen, Dexter C. (1997). Automata and Computability. en. New York, NY: Springer New
York. isbn: 978-1-4612-7309-7 978-1-4612-1844-9. doi: 10.1007/978-1-4612-1844-9.
url: http://link.springer.com/10.1007/978-1-4612-1844-9 (visited on
01/23/2024).
Milner, R. (1999). Communicating and Mobile Systems: The Pi-Calculus. Cambridge
University Press, Cambridge, UK.
Reghizzi, Stefano Crespi, Luca Breveglieri, and Angelo Morzenti (2019). Formal Languages
and Compilation. 3rd. Springer Publishing Company, Incorporated. isbn: 3030048780.

87 / 87