Chapter 5

Introduction to Complexity Theory

Bayisa G.(MSc)

11/26/2022 Complexity Theory 1

Introduction
Theory of Computation tries to answer the following questions:
 What are the mathematical properties of computer hardware
and software?
What is a computation, and what is an algorithm?
What are the limitations of computers? Can “everything” be
computed?(As we proceed, the answer to this question is “no”.)
Central Question in the Theory of Computation: What are the
fundamental capabilities and limitations of computers
Theory of computation can divided in to Complexity Theory,
Computability Theory, and Automata Theory.

11/26/2022 Complexity Theory 2

Automata, Computability and Complexity Theory
They are linked on question of
“What are the fundamental capabilities and limitation of a
computer”.
Complexity Theory
 What makes some problem computationally hard and others
easy?
 Computer problems comes in different varieties, some are easy
and some are hard.
 For example: the sorting problem is an easy one. Scheduling
problem is much hard than sorting.

11/26/2022 Complexity Theory 3

Cont..
Complexity theory classify problem according to computational resource
required to
Running time
Storage space
Parallelism
Randomness
Round of interaction, communication others.
Attempts to answer what is computationally feasible with limited
resource?
Central Question in Complexity Theory: Classify problems according to
their degree of “difficulty”. Give a rigorous proof that problems that seem
to be “hard” are really “hard”.
It works on the objective to classify problems as easy ones and hard ones;

11/26/2022 Complexity Theory 4

Complexity Class
Complexity have two class P(Polynomial)and NP(Non
polynomial).
P:the set of problems decidable in polynomial time.
NP: The set of problem undecidable in given polynomial
time.
Need to model of computation to define classes that
capture important aspects of computation.
Our model of computation :Turing machine

11/26/2022 Complexity Theory 5

Complexity Class NP
Solving a problem is harder than verifying a given solution to
the problem.
Class NP captures the set of decision problem whose solutions
are efficiently verifiable.

11/26/2022 Complexity Theory 6

Computability Theory

11/26/2022 Complexity Theory 7

Automata Theory
Automata theory deals with definition and properties of
mathematical models of computations.
Study  of  abstract  computing  devices,  or  “machines”  
•  Automaton  =  an  abstract  computing  device  
These models play a role in several area of computer science.
One model is called ,Finite automata ,is used in text processing,
compiler design and hardware design.
An other called context free grammar is used in programming
language and artificial intelligence.
Central Question in Automata Theory: Do these models have the
same power, or can one model solve more problems than the other?

11/26/2022 Complexity Theory 8

Model of Computation

11/26/2022 Complexity Theory 9

Example of Computation
3
f ( x)  x

11/26/2022 Complexity Theory 10

Cont..

11/26/2022 Complexity Theory 11

Cont..
Temporary memory

11/26/2022 Complexity Theory 12

Model of Automaton

11/26/2022 Complexity Theory 13

Automaton

11/26/2022 Complexity Theory 14

Kinds of Automata
Different Kinds of Automata
Automata are distinguished by the temporary memory
 Finite Automata: no temporary memory
 Pushdown Automata: stack
 Turing Machines: random access memory

11/26/2022 Complexity Theory 15

Finite Automaton

Example: Elevators, Vending Machines (small computing

power)
11/26/2022 Complexity Theory 16
Pushdown Automaton

Example: Compilers for Programming Languages (medium

computing power)
11/26/2022 Complexity Theory 17
Turing Machine

Examples: Any Algorithm (highest computing power)

11/26/2022 Complexity Theory 18

Power of Automata

11/26/2022 Complexity Theory 19

Goal of Complexity Theory
obtaining an understanding of the relations between various computational
phenomena (e.g., relating one fact regarding computational complexity to
another).
That is, it provides tools to measure the difficulty of computational
problems both absolutely and in comparison with other problems.
Generally:
Absolute Questions:
How much time is needed to perform the task?
How much resources will be needed?
Relative Questions:
More difficult than other tasks?
Are there “most difficult" tasks?

11/26/2022 Complexity Theory 20

Cont..
It is concerned with the study of the intrinsic complexity
of computational tasks.
That is, a typical complexity theoretic study refers to the
computational resources required to solve a computational
task (or a class of such tasks), rather than referring to a
specific algorithm or an algorithmic schema.
Its “final” goals include the determination of the
complexity of well defined computational task.

11/26/2022 Complexity Theory 21

Hard Computational Problems
Some computational problems are hard
Despite a numerous attempts we do not know any efficient
algorithms for these problems
We are also far away from the proof that these problems are
indeed hard to solve, in other words
NP=P or NPP, this is a question

11/26/2022 Complexity Theory 22

Decision/Optimisation Problems
A decision problem (DP) is a computational problem for which
the intended output is either yes or no
In an optimisation problem (OP) we rather try to maximise or
minimise some value
An OP can be turned into a DP if we add a parameter k, and
then ask whether the optimal value in OP is at most or at least k
Note that if a DP is hard, then its related its optimisation
version must be hard too.

11/26/2022 Complexity Theory 23

Example
 Optimisation problem - Given graph G with integer weights on
its edges. What is the weight of a minimum spanning tree
(MST) in G?
 Decision problem – Given graph G with integer weights on its
edges, and an integer k. Does G have a minimum spanning tree
of weight at most k?

11/26/2022 Complexity Theory 24

Problems and Languages
We say that the algorithm A accepts an input string x if A
outputs yes on input x
A decision problem can be viewed as a set L of (binary) strings
– the strings that should be accepted by an algorithm that
correctly solves the problem
We often refer to L as a language
We say that an algorithm A accepts a language L if A outputs
yes for each x in L and outputs no otherwise

11/26/2022 Complexity Theory 25

Cont…
One possibility: function from strings to strings
F : ∑*  ∑*
Function problem:
given x, compute f(x)
Decision problem: f:∑*  {yes, no}
given x, accept or reject

11/26/2022 Complexity Theory 26

Cont..
Decision problems: f:∑* {yes, no}
equivalent notion: language L  ∑*
L = set of “yes” instances
Examples:
set of strings encoding satisfiable formulas
set of strings that encode pairs (n,k) for which n has factor
<k
decision problem associated with L:
Given x, is x in L?

11/26/2022 Complexity Theory 27