Recursion

• Sometimes, the best way to solve a problem is by

solving a smaller version of the exact same problem
first
• Recursion is a technique that solves a problem by
solving a smaller problem of the same type.
Requirements for Recursive Solution
• At least one “small” case that you can solve directly
• A way of breaking a larger problem down into:
One or more smaller subproblems
Each of the same kind as the original
• A way of combining sub-problem results into an
overall solution to the larger problem
 Recursion…..cont..
Recursive procedures are functions that invoke themselves either
directly (call themselves from within themselves) or
indirectly (calls another method that calls original method.)
Recursion:
. An alternative to iteration
. Recursion can be very elegant at times,
. Not inexpensive to implement...
Classic examples of recursion
. Calculation of a string length, factorials, divide and conquer,
towers of Hanoi, binary searches, and more
Recursive functions are used in many applied areas.
. In artificial intelligence.
. In searching data structures that are themselves "recursive" in nature,
such as
trees. Concepts and implementation of recursive functions is an
important topic.
 Problems defined recursively
There are many problems whose solution can be defined
recursively.
Example: n factorial

1 if n = 0
n!= (recursive solution)
(n-1)!*n if n > 0

1 if n = 0
n!= (closed form solution)
1*2*3*…*(n-1)*n if n > 0
 Coding the factorial function

Recursive implementation Iterative implementation

int Factorial(int n) int Factorial(int n)
{ {
int fact = 1;
if (n==0) // base case for(int count = 2; count <= n;
return 1; count++)
else fact = fact * count;
return n * Factorial(n-1); return fact;
//Recursive case }
}
 Example
n choose k (combinations)
Given n things, how many different sets of size k can
be chosen?
n = n-1 + n-1 , 1 < k < n (recursive solution)
k k k-1
n = n! , 1 < k < n (closed-form solution)
k k!(n-k)!
with base cases:
n = n (k = 1), n = 1 (k = n)
1 n
 n choose k (combinations)

int Combinations(int n, int k)

{
if(k == 1) // base case 1
return n;
else if (n == k) // base case 2
return 1;
else
return(Combinations(n-1, k) +
Combinations(n-1, k-1));
}
 Recursion vs. iteration

• Iteration can be used in place of recursion

• An iterative algorithm uses a looping construct
• A recursive algorithm uses a branching structure

• Recursive solutions are often less efficient, in terms of

both time and space, than iterative solutions
• Recursion can simplify the solution of a problem, often
resulting in shorter, more easily understood source code
 Towers of Hanoi

• Move stack of disks from one peg to another

• Move one disk at a time
• Larger disk may never be on top of smaller disk
 The Expense of Recursive Functions
. Recursion => has a significant overhead…
. "Deeper" we go => need additional copies of the data
.. not always small number of data items.
. These ‘copies’ are stored in "stack frames“ which contain
Current values and outstanding function calls, and more
. Each "call" results in a stack frame.
Each requires space, allocation of space, and processing
time.
Note:.. Recursive functions – not efficient from a system
resource perspective.
.. Can pay dividends in:
... ease of writing and maintaining as compared to the writing
of iterative procedures.
.. Termed ‘elegant’ by some
 General Recursive Design Strategy
• Identify the base case(s) (for direct solution)
• Devise a problem splitting strategy
• Subproblems must be smaller
• Subproblems must work towards a base case
• Devise a solution combining strategy
Procedure:
1.Find a "Base Case " (That is what we shoot for.)
2. Develop the Recursive Case:
Base Case was sum(1).
We know the answer to this one!
we will always have a simple *normally’ assignment statement here,
such as assigning a value of null or 0 or 1. You will see.
Recursive Case was sum (n-1) which will (must) eventually lead to
sum(1)
Your general recursive case must progress to the base case.
An absolute MUST to conclude the method calls.
 Base Case
"Base case"
a very important notion!
 A base case is one that is simple;
 one that we are working ‘down’ toward
working "down" toward, since, as it turns out, we normally go
"down" or “decrease” some value or some quantity (such as
length of a string) toward the base case. (e.g. We know the
length of an empty string is 0; we know that 0! And 1! By
definition = 1…)

 Determining the base case ensures the recursive function

will terminate someday!!
 Writing Recursive Programs.

Have spoken about the "base case" and "recursive case."

Difficulty in writing recursive routines is identifying

.. base case, and (Some books calls this: "trivial case“)
.. Recursive case. ( Some books calls this: "complex case")

Base case
A case that is not recursive and directly obtainable.
Will have a value of 1 or 0 or null or something
having some kind of discrete value…

Recursive case
A case that is ultimately defined in terms of base case.
 Stated equivalently:

Complex case
. Must be defined in terms of a "simpler" case
. Plus the base case must be:
. a directly solvable, non-recursive trivial
case
We develop solutions using the assumption that the simpler
case has already been solved.
 SIMULATING RECURSION
• Some languages do not support recursion.
• It is often very common to be able to come up with a recursive
solutions to a problem.
• Since some compilers do not support recursion, we need to be able to
come up with non-recursive solutions. (Iterative Routines)

• Recursive solutions are often more expensive than non-recursive

solutions in Time and space.
• Heavy use of recursive solutions for production systems may dictate
non-recursive solutions to problems.
• Let's look at iterative solutions first.
 In Practice: Conflicts

There are conflicts between

• machine efficiency and
• programmer efficiency
• It is more efficient for the machine to lose some efficiency than
a high-cost programmer.
• It may not be worth the effort to construct a non-recursive
solution for a problem solved naturally recursively.
Recursively Enumerable
and
Recursive
Languages
Definition:

A language is recursively enumerable: if some Turing

machine accepts it.
Let L be a recursively enumerable language and M the
Turing Machine t h a t accepts it
For string w :
If w ∈ L then M: halts in a final state.
If w ∉ L then M : halts in a non-final state or loops
forever
Definition:
• A language is recursive: if some Turing machine
accepts it and halts on any input string
• In other words: A language is recursive if there is a
membership algorithm for it.

• Let L be a recursive language and M the Turing

Machine t h a t accepts it.
For string w :
• If w ∈L then M halts in a final state
• If w ∉ L then M halts in a non-final state
We will prove:

1. There is a specific language which is not

recursively enumerable (not accepted by any
Turing Machine)

2.There is a specific language which is

recursively enumerable but not recursive
 TM Block Diagrams:
• If L is a recursive language, then a TM M that accepts L and
always halts can be pictorially represented by a “chip” or “box”
that has one input and two outputs.
yes
w M
no

• If L is a recursively enumerable language, then a TM M that

accepts L can be pictorially represented by a “box” that has one
output.
yes
w M

Conceivably, M could be provided with an output for “no,” but this

output cannot be counted on. Consequently, we simply ignore it.
Theorem 1: The recursive languages are closed with respect to
complementation, i.e., if L is a recursive language, then so is
L  *  L
Proof: Let M be a TM such that L = L(M) and M always halts.
Construct TM M’ as follows:
M’
yes
yes
w M no
no

Note That:
M’ accepts iff M does not, M’ always halts since M always halts

From this it follows that the complement of L is recursive.

Question: How is the construction achieved? Do we simply complement the final
states in the TM? No! A string in L could end up in the complement of L.
Suppose q5 is an accepting state in M, but q0 is not.
If we simply complemented the final and non-final states, then q 0 would be an
accepting state in M’ but q5 would not.
Since q0 is an accepting state, by definition all strings are accepted by M’
Theorem 2: The recursive languages are closed with respect to union,
i.e., if L1 and L2 are recursive languages, then so is
L3 L1  L2
Proof: Let M1 and M2 be TMs such that L1 = L(M1) and L2 = L(M2) and
M1 and M2 always halts. Construct TM M’ as follows:
M’ yes
yes start
w M1 M2 no
no

Note That:
L(M’) = L(M1) L(M2)
L(M’) is a subset of L(M1) U L(M2)
L(M1) U L(M2) is a subset of L(M’)
M’ always halts since M1 and M2 always halt

L  L1  L2
It follows from this that3 is recursive.
• Theorem 3: The recursive enumerable languages are closed with
respect to union, i.e., if L1 and L2 are recursively enumerable
languages, then so is L3  L1  L2

• Proof: Let M1 and M2 be TMs such that L1 = L(M1) and L2 = L(M2).

Construct M’ as follows:
M’ yes yes
M1

w
yes
M2

Note That:
L(M’) = L(M1) U L(M2)
L(M’) is a subset of L(M1) U L(M2)
L(M1) U L(M2) is a subset of L(M’)
M’ halts and accepts iff M1 or M2 halts and accepts
L3  L1  L2
It follows from this that is recursively enumerable.
Question: How do you run two TMs in parallel?
• Suppose, M1 and M2 had outputs for “no” in the previous
construction, and these were transferred to the “no” output for M’

M’ yes yes
M1
no
w
yes
M2 no
no

Question: What would happen if w is in L(M1) but not in L(M2)?

Answer: You could get two outputs – one “yes” and one “no.”
At least M1 will halt and answer accept, M2 may or may not halt.
As before, for the sake of convenience the “no” output will be
ignored.
Theorem 4: If L and L are both recursively enumerable then L (and
therefore L ) is recursive.

Proof: Let M1 and M2 be TMs such that L = L(M1) and L = L(M2).

Construct M’ as follows:
M’ yes yes
M1

w
yes
M2 no

Note That:
L(M’) = L
L(M’) is a subset of L
L is a subset of L(M’)
M’ is TM for L
M’ always halts since either M1 or M2 halts for any given string
M’ shows that L is recursive
ItLfollows from this that L (and therefore its’
complement) is recursive.
So, is also recursive (we proved it before).
In terms of the hierarchy: (possibility #1)

Non-Recursively Enumerable Languages

Recursively Enumerable Languages

L L

Recursive Languages
In terms of the hierarchy: (possibility #2)

Non-Recursively Enumerable Languages

L L

Recursively Enumerable Languages

Recursive Languages
In terms of the hierarchy: (possibility #3)

Non-Recursively Enumerable Languages

L L

Recursively Enumerable Languages

Recursive Languages
In terms of the hierarchy: (Impossibility #1)

Non-Recursively Enumerable Languages

L L

Recursively Enumerable Languages

Recursive Languages
In terms of the hierarchy: (Impossibility #2)

Non-Recursively Enumerable Languages

Recursively Enumerable Languages

Recursive Languages
In terms of the hierarchy: (Impossibility #3)

Non-Recursively Enumerable Languages

Recursively Enumerable Languages

Recursive Languages
• Note: This gives/identifies three approaches to show
that a language is not recursive.
• Show that the language’s complement is not recursive, in
one of the two ways:
• Show that the language’s complement is recursively enumerable but not
recursive
• Show that the language’s complement is not even recursively enumerable
 The Halting Problem - Background
Definition: A decision problem is a problem having a yes/no answer (that
one presumably wants to solve with a computer). Typically, there is a list
of parameters on which the problem is based.
Given a list of numbers, is that list sorted?
Given a number x, is x even?
Given a C program, does that C program contain any syntax errors?
Given a TM (or C program), does that TM contain an infinite loop?
From a practical perspective, many decision problems do not seem all
that interesting. However, from a theoretical perspective they are for the
following two reasons:
Decision problems are more convenient/easier to work with when proving complexity results.
Non-decision counter-parts can always be created & are typically at least as difficult to
solve.
Notes:
The following terms and phrases are analogous:
Algorithm - A halting TM program
Decision Problem - A language (will show shortly)
(un)Decidable - (non)Recursive
• We want to f i n d a language t h a t is not Recursively
Enumerable

• This language is not accepted by any Turing Machine

Consider alphabet {a}

Strings: a, aa, aaa, aaaa, ...

••
•
Consider Turing Machines
that accept languages over alphabet {a}

They are countable:

1
Example language accepted by Mi

L ( M i ) ==
{aa,aaaa,aaaaaa}

2 4 6
L(Mi)=={a ,a ,a}

Alternative representation
4 as a6 a7
a ...
0 1 0 1 0 ···
 a 2 •••

L(M1) 1 0 1 •••

0
0 0 1 •••

L(M 2 ) 1
1 1 1 •
••

L(M3) 0
L(M4) 0 0 0 1 ••
•
Consider the
language
.
.
E L(Mi)}
L =={al:al

L consists from the l'sin the diagonal

 • •
•

L(M1) 0 1 0 1 ...

L(M2 ) 1 0 0 1 ...

I 1 1 ...
L(M 3 ) 0

0 0 1 ...

3 4
L = = { a , a , . .}
1 5
 Consider the language L

.
E L(Mi)}
L =={al:
a 1

L == : al
{a L(Mi)}
-
L consists of the O'sin the diagonal
 • •
•

L(M1) 0 1 0 1 ...

L(M 2 ) 1 0 0 1 ...

1 1 1 ...
L(M3) 0

0 1 ...
- 1
L={a 2
a, ...}
,
Theorem:
Language L is not recursively enumerable
Proof: Assume for contradiction that
i.t recursively enumerable

There must exist some machine Mk

that accepts L
 •
••

L(M1) 0 1 0 1 ...

0 1 ...
L(M 2 ) 1 0

1 1 1 ...
L(M3) 0

L(M4) 0 0 0 1 ...
 •
••

L(M1) 0 1 0 1 ...

0 1 ...
L(M 2 ) 1 0

1 1 1 ...
L(M3) 0

0 1 ...
 •
••

L(M1) 0 1 0 1 ...

0 1 ...
L(M 2 ) 1 0

1 1 1 ...
L(M3) 0

L(M4) 0 0 0 1 ...
 •
••

L(M1) 0 1 0 1 ...

0 1 ...
L(M 2 ) 1 0

1 1 1 ...
L(M3) 0

0 1

...

a 22 L(M2 )
 •
••

L(M1) 0 1 0 1 ...

0 1 ...
L(M 2 ) 1 0

1 1 1 ...
L(M3) 0

L(M4) 0 0 0 1 ...

Question: Mic=M 3 ?
 •
••

L(M1) 0 1 0 1 ...

0 1 ...
L(M2 ) 1 0

L(M3) 0 1 1 1 ...

0 1

...
3
Similarly:

Because either:

or
Therefore, the machine Mk cannot exist
Therefore, the language L
is not recursively enumerable
Observation:

There is no algorithm that describes L

(otherwise L would be accepted

by some Turing Machine)
Non Recursively Enumerable

L
Recursively Enumerable

Recursive
 A Language which is
Recursively Enumerable
and not Recursive
 We want to find a language which

Is But not
recursively recursi ve
•

enumerable

There is a The machine

Turing Machine doesn't halt
that accepts on some input
the language
We will prove that t he
language
.
.
E L( Mi )}
L =={al: al
Is recursively enumerable
but not recursive
 • •
•

L(M1) 0 1 0 1 ...

L(M2 ) 1 0 0 1 ...

I 1 1 ...
L(M 3 ) 0

0 0 1 ...

3 4
L = = { a , a , . .}
3 3
Theorem:

. .

The language L == { a : a
1 1
E L(Mi)}

is recursively enumerable
Proof:
We will give a Turing Machine that
accepts L
 Turing Machine that accepts L
For any input string w
.
·Compute i, which w == al
for
·Find Turing machine Mi
(using an enumeration procedure
for Turing Machines)
.

·Simulate Mi on input al
·If Mi accepts, then accept w
End of
Observation:

Recursively enumerable
. .

L =={al: al E L(Mi)}

Not recursively enumerable

. .

L =={al: al L(Mi)}

(Thus, also not recursive)

Theorem:

. .

The language L == { a : a
1 1
E L(Mi)}
is not recursive
Proof
:Assume for contradiction that L is recursive

Then L 1s •

Take the recursi

Turingve:
Machine M that accepts L
•

hittfts on any input:

If M accepts
then reject If M
rejects then accept
Therefore:

L is recursive

But we know:

L is not recursively enumerable

thus, not recursive

CONTRADICTION!!!!
Therefore, L is not recursive

End of
Proof
Non Recursively Enumerable

L
Recursively Enumerable

L
Recursive
Turing acceptable languages
and
Enumeration Procedures
We will prove:
(weak result)
·If a language is recursive then
there is an enumeration
procedure for

it
(strong result)
• A language is recursively enumerable
if and only if
there is an enumeration procedure for it
Theorem:
if a language L is recursive then
there is an enumeration procedure for it
Proof:

Enumeration Machine

..
-
M
, - ...,,;

M -.
p
p

. .
.

t
'
Enumerates all
Accepts L
strings of input alphabet
If,...._ the alphabet is {a,b}
Mn enumerate strings as follows:
then
,

a
b
aa
ab
ba
b
aaa
b
aa
b
••••••
Enumeration procedure

Repeat:
M
,-....;

generates a string w
M checks if w E L
YES:print vt,to output
NO:ignore w

End of
Proof
Example:L
=={b,ab,bb,aaa.,...}
,... . Enumeration
M
,
L(M Output
a )
b b b
aa
ab a a
b b b
a
b b b
aa aa aa
ab b
a
b
a
aa •••••• ••••••
Theorem:

if L is recursively enumerable then

language
there is an enumeration procedure for it
Proof:

Enumeration Machine

-
'" .
r
•

M .
M r

I .
+
-

Enumerates all
Accepts L
strings of input alphabet
If,...._ the alphabet is {a,b}
Mn enumerate strings as follows:
then
,

a
b
aa
ab
ba
b
aaa
b
aa
b
 NAIVE APPROACH
Enumeration procedure

Repeat: M generates a string

w
checks if
M w
YES:print
E L
l,1,ito
output
NO:ignore
w

Problem: If w L
machine M may loop forever
 BETTER
APPROACH
M Generates first string w1
,-..

.._;

M executes first step on w1

M
,-..
Generates second string
.._;
W2
M executes first step on w2

second step on w1
M Generates third string w3

M executes f i r s t step on w3

second step on w2

third step on w1

And so on............
 1 - ------- 1 1
1--.
Step
,n
• 2 2
string
3 3 3 3

•••
If for any string wi
machine M halts in a final s t a t e
then it prints wi on the output

End of
Theorem:

If for language L
there is an enumeration procedure
then L is recursively enumerable
Proof
Input Tape
:
w

Machine that
accepts L

Enumerator Compar
for L ..
e
.
Turing machine that accepts L
For input string w
Repeat:
·Using the enumerator,
generate the next string of
L
·Compare generated string with

w
We have proven:

A language is recursively enumerable

if and only if there is an
enumeration procedure for it