CSCI 5444: Introduction to

Theory of Computation
Lecture 01: Introduction

Alexandra Kolla([email protected])
Department of Computer Science, University of Colorado Boulder
Logistics
• Web-page: http://home.cs.colorado.edu/~alko5368/index5444.html
• Instructor and Grading Assistant
o Alexandra Kolla ([email protected])
o Charles Carlson ([email protected] )
• Lectures
o Tuesday (11:30am – 12:15am)
o Thursday (11:30am – 12:15am)
• Office hours
o TBD
o By appointment
• Venue
o Class: HUMN 1B90
o Office hours: ECCS 122
Logistics (Contd.)
• Requisite
o Discrete Structures (CSCI 2824) / Discrete Mathematics
o UG Algorithms (CSCI 3104)
• Textbook
o Michael Sipser. Introduction to the Theory of Computation, PWS Publishing Company.
• Other supplemental materials
o Automata and Computability , Dexter C. Kozen
o Automata Theory, Languages, and Computation , Hopcroft, Motwani, and Ullman (3rd
edition).
o Elements of the theory of computation , Lewis and Papadimitriou (2nd edition).
o Descriptive Complexity, Neil Immerman
o Elements of Finite Model Theory, Leonid Libkin
o Computational Complexity, Sanjeev Arora and Boaz Barak
o Online notes and readings distributed by instructor
Logistics (Contd.)
• Zoom
• Moodle
o All assignments will be posted on moodle.
o Your identikey is needed for signing in.
Logistics: Grading

n The
e nt o tly)
si g nm shor
k’s as me
wee to co
d this oddle
Fin sit (m
web
Theory of Computation
What are the fundamental capabilities and limitations of computers?
 Mechanical Electro-mechanical First Electronic
computer Computer general-purpose computer
“Antikythera” (2 BC) “Bombe” (1938) “ENIAC” (1946) Datacenters
Google (Now)
What are the fundamental capabilities and limitations of computation?
• What do we mean by computation?
• What is a problem?
• Are all problems computable?
• What is an “efficient” computation?
• Are some problems inherently more difficult than others?
What are the fundamental capabilities and limitations of computers?
• How do we model “computational machines”?
• Are all computational machines equally powerful?
• Why should we study computationally weaker models?
• Why a practically-oriented computer-programmer should learn theory of
computation?
Theory of Computation
Automata Theory
o Formalization of the notion of problems via formal
languages
o Formalization of the notion of computation using "abstract
computing devices" called automata
o Understanding a hierarchy of classes of problems or formal
languages (regular, context-free, context-sensitive, decidable,
and undecidable)
o Understanding a hierarchy of classes of automata (finite
automata, pushdown automata, and Turing machines)
o Understanding applications to pattern matching, parsing,
and programming languages

Computability Theory

Complexity Theory
Theory of Computation
Automata Theory

Computability Theory
o Understanding Church-Turing thesis (Turing machines as a
notion of "general-purpose computers")
o Understanding the concept of reduction , i.e., solving a
problem using a solution (abstract device) for a different
problem
o Understanding the concept of undecidability , i.e., when a
problem can not be solved using computers

Complexity Theory
Theory of Computation
Automata Theory

Computability Theory

Complexity Theory
o Complexity classes : how to classify decidable problems
based on their time and space requirements
o Complexity classes P and NP
o When a problem is called intractable (NP-completeness)
o Using reductions to prove problems intractable
o Space-complexity classes L and NL, PSPACE, and so on
Theory of Computation: (Rough) Schedule
• Week 1 – Week 5 : Automata Theory (In-Class Quiz I)
• Week 6 – Week 10: Computability Theory (In-Class Quiz II)
• Week 11 – Week 15: Complexity Theory (In-Class Quiz III)
• Week 15 – Week 16: Special Topics
Special Topics
• Randomized Computation and Complexity
• Quantum Computation and Complexity
• Approximate Computation and Complexity
• Historical paper review
• Other? (suggestions welcome)
Discrete Mathematics: Review
Discrete Mathematics: Review
• A set is a collection of objects, e.g.
• ! = #, %, &, ' and ( = %, '
• Empty set ∅ = {} (why it is not same as {∅})
• ℕ = 0, 1, 2, 3, … and ℤ = {… , −2, −1, 0, 1, 2, … }
• ℚ is the set of rational numbers.
• ℝ is the set of real numbers.
Georg Cantor
• # ∈ ! : element of a set, belongs to, or contains March 3, 1845 – January 6, 1918
• Subset of ! ⊆ ℕ, or proper subset of ! ⊂ ℕ
• Notions of set union, intersection, difference, and disjoint
• Power set 29 of a set ! (example)
• Partition of a set
Discrete Mathematics: Review (Contd.)
• A ordered pair is a pair !, # of elements with natural order
• Similarly we define triplet, quadruplet, $-tuples, and so on
• Cartesian product A×' of two sets is the set of orderd pairs
A×' = !, # ∶ ! ∈ + !$, # ∈ '
• Binary relation - on two sets + and ' is a subset of A×'
• Recall definitions of
• Reflexive, Symmetric, and Transitive relations,
• and Equivalence relation.
Discrete Mathematics: Review (Contd.)
• A function (or mapping) ! from set " to # is a binary relation s.t. sfor
all $ ∈ " we have that ($, () ∈ ! and ($, (′) ∈ ! implies that ( = (ʹ.
• We often write !($) for the unique element b such that $, ( ∈ !.
• Function !: " → # is one-to-one if for any two distinct elements
$, ( ∈ " we have that ! $ ≠ !(().
• Function !: " → # is onto if for every element ( ∈ # there is an
element $ ∈ " such that !($) = (.
• Function !: " → # is called bijection if it is both one-to-one and onto.
Cardinality of a Set
• Cardinality |"| of a set ", e.g. |#| = 4 and ℕ is an infinite
number.
• Two sets have same cardinality if there is a bijection between them.
• A set is countably infinite (or denumerable) if it has same
cardinality as ℕ.
• A set is countable if it is either finite or countably infinite.
• A transfinite number is a cardinality of some infinite set.
Theorem: Cardinality
Theorem
1. The set of integers is countably infinite. (idea: interlacing)
2. The union of a finite number of countably infinite sets is countably infinite
as well. (idea: dove-tailing)
3. The union of a countably infinite number of countably infinite sets is
countably infinite.
4. The set of rational numbers is countably infinite.
5. The power set of the set of natural numbers has a greater cardinality than
itself. (idea: contradiction, diagonalization)
Cantor’s Theorem

Theorem. If a set ! is of any infinite cardinality then its power set

Corollary. There is an infinite series of infinite cardinals.

Theorem. There is an infinite series of infinite cardinals.

“ a "grave disease" infecting the discipline of mathematics” −Henri Poincaré