Computational complexity theory

It is an investigation of the time, memory,

or other resources required for solving
computational problems.

The fundamentals of time complexity

theory has been explored in the next slides.

1
 Tasks with time complexity theory

• Introduce a way of measuring the time

used to solve a problem
• Show how to classify problems according
to the amount of time required.
• Show the possibility that certain
decidable problems require enormous
amounts of time.

2
 Measuring complexity

The running time of an algorithm can be

computed as a function of the length of the
string representing the input only.

3
 Worst-case analysis

In worst-case analysis, consider the longest

running time of all inputs of a particular
length.

4
 Running Time or Time Complexity
of Turing machine
Let M be a deterministic Turing machine
that halts on all inputs.

The running time or time complexity of M

is the function t: N → N, where t(n) is the
maximum number of steps that M uses on
any input of length n.

5
 t(n) time Turing machine

M is called a t(n) time Turing machine, if M

runs in time t (n), that is, its running time is
t(n) .
Here M is a deterministic Turing machine
that halts on all inputs.

6
Asymptotic analysis for estimation of running
time of algorithm for large inputs
• The method is based upon considering only the
highest order term of the expression for the running
time of the algorithm, disregarding both the
coefficient of that term and any lower order terms,
because the highest order term dominates the other
terms on large inputs.
• Then the running time t is said to be asymptotically at
most the highest order term .

7
 Big-O Notation

Big-O notation says that one function is

asymptotically no more than another.

• The asymptotic notation or big-O notation for

describing this relationship is
f (n) = O(highest order term ).

8
 Formal Definition Big-O Notation

Say that f( n) = O(g( n)) if positive integers

c and n0 exist such that for every
integer n  n0 , f( n)  c g(n).
• g(n) is an asymptotic upper bound for
f( n).
• f and g be functions f, g: N → R+, where
R+ be the set of nonnegative real numbers

9
 Example
Let f1( n) be the function 5n3 + 2n2 + 22n + 6.
Then, selecting the highest order term 5n3 and
disregarding its coefficient 5 gives f1( n) = O(n3).
• Verify that this result satisfies the formal
definition. Let c be 6 and n0 be 10.
• Then, 5n3 + 2n2 + 22n + 6  6 n3 for every
n  10.

10
Example (cont.)

11
 Type of bounds
1. Polynomial bounds : Bounds are derived of
the form nc for c greater than 0, are called
polynomial bounds.

2. Exponential bounds : Bounds of the form

2(n^ ) are called exponential bounds when  is
a real number greater than 0.

12
 Small-o notation

Small-o notation is used to say that one

function is asymptotically less than .

13
Definition

Definition: Small-o notation

Say that f( n) = o(g( n)) if

Such that for any real number c > 0, a number n0

exists, where f( n) < c g(n) for all n  n0
.

• f and g be functions f, g: N → R+, where R+

be the set of nonnegative real numbers.

14
 The difference between the big-O and
small-o notations
The difference between the big-O and
small-o notations is analogous to the
difference between  and <.

15
 Example : Analyzing algorithms

Let's analyze the TM algorithm for the language

A = {0k1k | k  0}. Where M1 = "On input string
w:
1.Scan across the tape and reject if a 0 is found
to the right of a 1.
2.Repeat if both 0s and 1s remain on the tape:
3.Scan across the tape, crossing off a single 0
and a single 1.

16
 Example : (cont.)
4. If 0s still remain after all the 1s have been
crossed off, or if 1s still remain after all the 0s
have been crossed off, reject. Otherwise, if
neither 0s nor 1s remain on the tape, accept."

17
 Steps for simulation of string 0*1*
In stage 1, the machine scans across the tape to verify
that the input is of the form 0*1* .
• Performing this scan uses n steps. ( n represents the
length of the input).
• Repositioning the head at the left-hand end of the tape
uses another n steps.
• So the total used in this stage is 2n steps.
• This stage uses O(n) steps.

18
 Steps (cont.)
In stages 2 and 3, the machine repeatedly scans the tape
and crosses off a 0 and 1 on each scan.
• Each scan uses O(n) steps.
• Because each scan crosses off two symbols, at most
n/2 scans can occur.
• So the total time taken by stages 2 and 3 is (n/2)O(n)
= O(n2 ) steps.

19
 Steps (cont.)
In stage 4 the machine makes a single scan to decide
whether to accept or reject.
• The time taken in this stage is at most O(n).
• Thus the total time of M1 on an input of length n is
O(n) + O(n2) + O(n), or O(n2).
• Thus its running time is O(n2), which completes the
time analysis of this machine.

20
Time Complexity Class - TIME(t(n))
It is the collection of all languages that are decidable by
an O(t(n)) time deterministic Turing machine.
Where t: N → R+ be a function and n – length of input.
Formally
TIME(t(n)) = {L| L is a language decided by a O(t(n))
time deterministic Turing machine}.

Example:
For the language A = {0k 1 k | k  0}, analysis shows
that A  TIME(n2) because M1 decides A in time O(n2)
and TIME(n2) contains all languages that can be decided
in O(n2) time.
21
 Nondeterministic Time Complexity
Class - NTIME(t(n))
It is the collection of all languages that are
decidable by an O(t(n)) time non-deterministic
Turing machine.
Where t: N → R+ be a function and n – length of
input.
Formally
NTIME(t(n)) = {L| L is a language decided by a
O(t(n)) time nondeterministic Turing machine}.

22
 Theorem
Let t(n) be a function, where t(n)  n, n = length
of input. Then every t(n) time multi-tape Turing
machine has an equivalent O(t2(n)) time single-
tape Turing machine.

23
 Proof (Idea)
• Convert any multi-tape TM into a single-tape
TM that simulates it.
• Show that simulating each step of the multi-
tape machine uses at most O(t(n)) steps on the
single-tape machine.
• Hence the total time used is O(t2(n)) steps.

24
 The running time of a nondeterministic
Turing machine.
Let N be a nondeterministic Turing machine that
is a decider, all its computation branches halt on
all inputs.
• The running time of N is the function t: N →
N, where t (n) is the maximum number of steps
that N uses on any branch of its computation on
any input of length n.

25
 Theorem
Let t(n) be a function, where t(n)  n. Then every
t(n) time nondeterministic single-tape Turing
machine has an equivalent 2O(t(n)) time
deterministic single-tape Turing machine.

26
 Proof:

Let N be a nondeterministic TM running in t(n)

time.
Construct a deterministic TM D that simulates N
by searching N's nondeterministic computation
tree.
• On an input of length n, every branch of N's
nondeterministic computation tree has a length
of at most t(n).

27
 Proof (cont.)

• Every node in the tree can have at most b

children, where b is the maximum number of
legal choices given by N's transition function.
• Thus the total number of leaves in the tree is at
most bt(n)
• The simulation proceeds by exploring this tree
breadth first.
• It visits all nodes at depth d before going on
to any of the nodes at depth d + 1.
28
 Proof (cont.)

• The total number of nodes in the tree is less

than twice the maximum number of leaves, so
bound it by O(bt(n)).
• The time for starting from the root and
traveling down to a node is O(t(n)). Therefore
the running time of D is O(t(n) bt(n)) = 2O(t(n)).

29
 Proof (cont.)

• The TM D has three tapes. Converting to a

single-tape TM at most squares the running
time.
• Thus the running time of the single-tape
simulator is (2O(t(n)))2 = 2O(2t(n)) = 2O(t(n)),
and the theorem is proved.

30