COSC/MATH 4P61 - Theory of Computation

COSC/MATH 4P61
Theory of Computation
• Michael Winter
– office: J323
– office hours: Mon & Fri, 10:00am-noon
– email: [email protected]
• course web page:
http://www.cosc.brocku.ca/~mwinter/Courses/4P61/
• Main Text
– John E. Hopcroft, Rajeev Motwani, and Jeffrey Ullman:
Introduction to Automata Theory, Languages, and
Computation (3rd ed.), Addison-Wesley (2007).

© M. Winter 1.1
 COSC/MATH 4P61 - Theory of Computation

Course Work

• Marking Scheme
– Term Tests (3x20%) 60%
– Final Exam 40%

• Term Test 1 (30 min): October 01, 2015 (in class)

• Term Test 2 (30 min): November 05, 2015 (in class)

• Term Test 3 (30 min): November 26, 2015 (in class)

© M. Winter 1.2
 COSC/MATH 4P61 - Theory of Computation

Course Outline
Week Date Topic
1 Sep 10 Introduction, Formal Languages and Computability
2 Sep 17 Finite Automata & Regular Languages: Definition, Kleene's Theorem
Finite Automata & Regular Languages: Minimalization, Closure Properties, Pumping
3 Sep 24
Lemma for Regular Languages
4 Oct 01 Context-Free Languages, CFG's, Parse-Trees, Normal Forms of CFG's (Test 1)
Pushdown Automata, Equivalence for PDA's and CFG's, Pumping Lemma for
5 Oct 08
Context-Free Languages
6* Oct 22 Turing Machines, Universal Turing Machines
7 Oct 29 Context-Sensitive Languages
8 Nov 05 Recursively Enumerable Languages (Test 2)
9 Nov 12 Undecidable Problems, Halting Problem
10 Nov 19 Partial Recursive Functions, Primitive Recursive Functions
11 Nov 26 Complexity (Test 3)
12 Dec 03 Complexity, P vs. NP

*
October 12-16 is Fall Reading Week, no classes

© M. Winter 1.3
 COSC/MATH 4P61 - Theory of Computation

Computability – What’s the deal?

Imaging the following situation:

You are working in a software company and your boss has the idea to
develop a new antivirus system. This program should analyse
automatically any incoming program file and will do the following:
• If the program will harm your computer, it will be deleted.
• If the program will not harm your computer, it will be started.

Do you want to be the team leader for this project?

© M. Winter 1.4
 COSC/MATH 4P61 - Theory of Computation

Computability – What’s the deal?

Imaging the following situation:

You are working in a software company and your boss has the idea to
develop a new antivirus system. This program should analyse
automatically any incoming program file and will do the following:
• If the program will harm your computer, it will be deleted.
• If the program will not harm your computer, it will be started.

Do you want to be the team leader for this project?

Probably not, because this is a so-called undecidable problem, i.e., such a

program does not exist.

© M. Winter 1.5
 COSC/MATH 4P61 - Theory of Computation

Non-computable Functions
Let us consider functions of the form f : ℕ → ℕ and programs (in any
language of your choice) that take a natural number as input and return a
natural number.

How many functions/program do we have?

© M. Winter 1.6
 COSC/MATH 4P61 - Theory of Computation

Diagonalization

f1 f2 f3 f4 f5 
1 f1(1)1 f2 (1) f3(1) f4 (1) f5(1) 
2 f1(2) f2 (2) 1 f3(2) f4 (2) f5(2) 
3 f1(3) f2(3) f3(3) 1 f4(3) f5(3) 
4 f1(4) f2(4) f3(4) f4 (4) 1 f5(4) 
5 f1(5) f2(5) f3(5) f4(5) f5(5) 1 
      g (n)

© M. Winter 1.7
 COSC/MATH 4P61 - Theory of Computation

Non-computable Functions
Let us consider functions of the form f : ℕ → ℕ and programs (in any
language of your choice) that take a natural number as input and return a
natural number.

How many functions/program do we have?

1. There are uncountable many functions of this form.

© M. Winter 1.8
 COSC/MATH 4P61 - Theory of Computation

Coding Strings by Numbers

Suppose we have the alphabet A  {' a' , ' b' ,  , ' z' ., ' '}
Then we can code every string over this alphabet as a number natural
number to the base A .

Example:

hello world 
7  27 0  4  271  11  27 2  11  273  14  27 4
 26  27 5  22  27 6  14  27 7  17  278  11  27 9  3  2710
 706511619670984

© M. Winter 1.9
 COSC/MATH 4P61 - Theory of Computation

Non-computable Functions
Let us consider functions of the form f : ℕ → ℕ and programs (in any
language of your choice) that take a natural number as input and return a
natural number.

How many functions/program do we have?

1. There are uncountable many functions of this form.

2. There are countable many programs of this kind.

There are more non-computable functions than there are computable ones.

Can we provide reasonable examples for non-computable functions?

© M. Winter 1.10
 COSC/MATH 4P61 - Theory of Computation

Potential Questions
• What are typical examples of non-computable functions?
• How long does it take to compute a certain result?
• Which of the algorithms at hand is better?
• Can I solve my problem using language xyz?
• Is it possible to make my program faster?

In order answer those questions we need mathematical models of

computation. During this course we will consider:
• Functions on natural numbers • Finite Automatons
• Languages (sets of strings) • Grammars
• Pushdown Automatons
• Turing Machines
• Recursive Functions

© M. Winter 1.11
 COSC/MATH 4P61 - Theory of Computation

Finite Automaton

© M. Winter 1.12
 COSC/MATH 4P61 - Theory of Computation

Finite Automaton
M  (Q, ,  , q0 , F )

where
Q  {q0 , q1 , q2 , q3 }  0 1
  {0,1}
q0 q2 q1
F  {q0 }
q1 q3 q0
q2 q0 q3
q3 q1 q2

© M. Winter 1.13
 COSC/MATH 4P61 - Theory of Computation

Finite Automaton

© M. Winter 1.14
 COSC/MATH 4P61 - Theory of Computation

Nondeterministic Finite Automaton

M  (Q, ,  , q0 , F )

where
 0 1
Q  {q0 , q1 , q2 , q3 , q4 }
  {0,1} q0 {q0 , q3 } {q0 , q1}
F  {q2 , q4 } q1  {q2 }
q2 {q2 } {q2 }
q3 {q4 } 
q4 {q4 } {q4 }

© M. Winter 1.15
 COSC/MATH 4P61 - Theory of Computation

Nondeterministic Finite Automaton

© M. Winter 1.16
 COSC/MATH 4P61 - Theory of Computation

NFA with ε-Moves

M  (Q, ,  , q0 , F )

where
Q  {q0 , q1 , q2 }
  {0,1,2}  0 1 2 
F  {q2 } q0 {q0 }   {q1}
q1  {q1}  {q2 }
q2   {q2 } 

© M. Winter 1.17
 COSC/MATH 4P61 - Theory of Computation

NFA with ε-Moves

© M. Winter 1.18