DYNAMIC PROGRAMMING
INTERVIEWS
A BOTTOM-UP APPROACH TO PROBLEM SOLVING
yAAY
vv
MEENAKSHI & KAMAL RAWAT
FOUNDER, RITAMBHARA TECHNOLOGIES @DYNAMIC PROGRAMMING
CODING
INTERVIEWS
ABOTTOM-UPAPPROACH TOPROBLEMSOLVING
MEENAKSHI & KAMAL RAWAT
FOUNDER, RITAMBHARA TECHNOLOGIES
© notionpress©
Notion Press
Old No. 38, New No. 6
McNichols Road, Chetpet
Chennai - 600 031
First Published by Notion Press 2017
Copyright © Meenakshi & Kamal Rawat 2017
All Rights Reserved.
ISBN 978-1-946556-70-7
This book has been published with all efforts taken to make the material error-
free after the consent of the author. However, the author and the publisher do
not assume and hereby disclaim any liability to any party for any loss, damage,
or disruption caused by errors or omissions, whether such errors or omissions
result from negligence, accident, or any other cause.
No part of this book may be used, reproduced in any manner whatsoever without
written permission from the author, except in the case of brief quotations
embodied in critical articles and reviews.
The contents of this book including but not limited to the views, representations,
descriptions, statements, informations, opinions and references [“Contents”]
are the sole expression and opinion of its Author(s) and do not reflect the
opinion of the Publisher or its Editors. Neither Publisher nor its Editors endorse
the contents of this book or guarantees the accuracy or completeness of the
content published herein and shall not be liable whatsoever for any errors,
misrepresentation, omissions, or claims for damages, including exemplary
damages, arising out of use, inability to use, or with regard to the accuracy or
sufficiency of the information contained in this book.DEDICATION
This book is dedicated to a yosi
MASTER MAHA SINGH
who mended relations and excelled in each role he lived
SON | BROTHER | FRIEND | HUSBAND | FATHER |
FATHER-IN-LAW | TEACHER | FARMER | CITIZEN |
DEVOTEE
the most upright, conscientious, and able man.
We miss you in every possible way Papa.Preface
We have been teaching students ‘How to prepare for Coding Interviews’
for many years now.
‘A Data Structure problem like reverse a linked list or add operation
getMinimum() to a Stack or implement a thread safe Queue are relatively
easy to solve, because there are theoretical reference to start in
such problems. The most difficult category of problems asked in
coding competitions or interview of companies like Google, Amazon,
Facebook, Microsoft etc. fall under the umbrella of Dynamic Programming
@P).
Most DP problems do not require understanding of complex
data structure or programming design, they just need right strategy
and methodical approach.
A solution is first visualized in the mind before it takes formal
shape of an Algorithm that finally translates into a computer
program. DP problems are not easy to visualize and hence not easy to
solve.
‘The best way to visualize a DP problem is using recursion because DP
problems demonsttates optimal substructure behavior. Recursion gives
the right solution but usually takes exponential time. This unreasonably
high time is taken because it solves subproblems multiple times.
DP is a bottom-up approach to problem solving where
one subproblem is solved only once. In most cases this approach is
counter-intuitive. It may require a change in the way we approach a
problem. Even experienced coders struggle to solve DP problems.
This book is written with an intention that if a teacher reads it, he will
make dynamic programming very interesting to his students. If this book is
in the hands of a developer, he will feel super confident in
answering algorithm questions in interviews, and anyone who read it will
get a robust tool to approach problems asked in coding competitions.
Being a good coder is not about learning programming languages, it
is about mastering the art of problem solving.
Code in this book is written in C language, If you have never
written any program in C language, then we suggest you to tead first few
chapters of C from any good book and try writing some basic
programs in C language.Acknowledgments
To write a book, one need to be in a certain state of mind where one is
secure inside himself so that he can work with single minded focus.
A guru helps you be in that state, We want to thank our ideological
guru, Shri Rajiv Dixit Ji and Swamy Ramdev Ji.
The time spent on this book was stolen from the time of family and
friends. Wish to thank them, for they never logged an FIR of their stolen
time.
We also wish to thank each other, but that would be tantamount to
cone half of the body thanking the other half for supporting it. The body
does not function that way.How to Read this Book
We have discussed only simple problems in first six chapters. The idea is
to explain concepts and not let the reader lost in complexity of
questions. More questions are discussed in last three chapters with
last chapter completely dedicated to practice questions.
Tf you have the luxury of time, then we strongly recommend you
to read this book from cover to cover.
If you do not have time, then, how you read this book depends
on how good you are in coding and how comfortable you are with
recursion.
If you are good at forming logic and can write reasonably complex
programs of Binary Tree and Linked List comfortably, then you may
choose to skip the first chapter. I think the below logic may help you
find a starting point
IF (one day left for Interview)
IF (champion of DP)
READ Chapter 8,9
ELSE IF (Good @ Programming)
READ Chapter 3, 4, 5, 6, 7.
ELSE
Pray to God, Start from Chapter 1.
ELSE IF (more than one week left for interview)
IF (champion of DP)
READ Chapter 2, 7, 8, 9
ELSE IF (Student OR Researcher)
READ complete book
ELSE IF (Reading to learn DP)
READ complete BookRecursion
int main(){
printf("Cannot stop till I die");
main();
)
Most computer concepts have their origin in Mathematics, Recursion is no
exception. If you have studied mathematics at high school level, then you
must have come across equations like.
Creo Plt ts 9 ree
OEE
SG eo
Pret ol
& is a symbol for summation. It is read as “Same of first nm numbers is equal
to n plus Sum of first (n-1) numbers. When m becomes 1, Sum(1) has a fied value,
1” n, in this case is positive integer.
In plain words, Sum is defined in terms of Sum itself. This is called
Recursion,
In computer programming, “when a function calls itself either directly or
indirectly it is called a Recursive Function and the process is called Recursion.”
Typically the function performs some part of the task and rest is
delegated to the recursive call of the same function, hence, there are
multiple instances of same function each performing some part of the
overall task. Function stops calling itself when a terminating condition is
reached.Recursion is a problem solving technique, where solution of
a larger problem is defined in terms of smaller instances of
itself.
Points to note in recursion are the following:
1. Recursion always have a terminating condition (lsc it will be
infinite recursion). In Sum function, the condition, if n=1, then,
stop recutsion and return 1, is the terminating condition.
2, Recursive function performs some part of task and delegate
rest of it to the recursive call. In above example, the function
performs addition of n with the return value of the Sum(n-1)
but delegate the computation of Sum(n-1) to recursive call.
Writing recursive code is not as difficult as it may appear, in fact, in
most cases it is relatively simpler because we ate not solving the complete
problem. Recursive function code is a two-step process
1. Visualize the recursion by defining larger solution in terms of
smaller solutions of the exact same type (with natrow
parameters), and,
2. Add a terminating condition.
Example 1.1 translate the logic of calculating sum of first n positive
integers to C language program.
Example 1.1: C language function to compute sum of first n
positive numbers:
int sum(unsigned int n){
Lf (n = 4)
return 1;
else
return n + sum(n-1);
Code: 1.1
‘This code has an issue, If someone call it for n=0, then it will behave
in an undefined manner.As a good coding practice, we must check our program against
boundary values of input parameters. If we call the function as
sum(0);
then, it will skip our terminating condition (n == 1) and enter into the
recursion calling itself forn = -1, and this will make the result undefined’.
We should be able to catch this issue while self-reviewing our code
and should be able to correct it:
int sum(unsigned int n) {
// First terminating condition
if(n == 0)
return 0;
// Second terminating condition
if(n == 1)
return 1;
return n + sum(n-1);
Code: 1.2
We have just skipped the else part, because we ate returning from the
terminating conditions.
Code 1.2 may be written in a more compact form as shown below
int sum(int n){
return (n==0)? 0: ((n==1)? 1: (n+sum(n-1)));
Code: 1.3
Which of the two codes should we prefer to write?
"In C language, conversion from signed to unsigned is not defined for negative numbers.
For example, the value of y in the below code is not defined:
int x = -1;
unsigned int y = x;
When Sum is called for n=0. Then, it will skip the terminating condition and call the
function recursively forn = -1.Some coders, have an undue bias in favour of writing compact codes,
this is especially true during the interview. Either they think that such code
impress the interviewer, or they may just be in the habit of writing such
code. Thumb rule for good code is,
“when there is a choice between simple and obfuscated code, go for the simpler one,
unless the other has performance or memory advantage.”
This rule is not just for interview, its genetic. The code that we write is
read by many people in the team, the simpler it is the better it is. One extra
advantage of writing simple code duting the intetview is that, it provide us
some space to correct mistakes there itself, because it leaves more white
space on the paper.
The rule is just for spacing of code and choosing between two
statements doing exactly same work. In no way a check should be omitted
in favor of simplicity or clarity or extra space on the paper.
Never miss the terminating condition, else the function may fall into infinite
recursion.
It is not mandatory to write a recursive function to compute sum of n
numbers. It can be done using a loop without making any recursive call, as
demonstrated in Code 1.4
Non-recursive code to compute sum of fitst n_ numbers
int sum(int n){
int sum = 0;
for(int i=l; i<=n; i++)
sum += i;
return sum;
Code: 1.4
Question 1.1; Factorial function is defined recursively for all non-negative
integers as:
Fact (n) = n * Fact (n-1) if n>1
=1 if n=1
Write both recursive and non-recursive function that accepts an
unsigned int n and return factorial of n?.
? Example7.1 in Chapter 7 gives the solution to this question.Question 1.2: Given an array, arr, of integers, write a recursive function
that add sum of all the previous numbers to each index of the array. For
example, if input array is
Then, your function should update the artay to
{a ]s3 | 6 | a0] as | 2a
Example 1.2: Recursion to compute n> power of a number x" is defined
as below:
Function for above recursion is:
int power(int x, int n){
if(0 == n)
return 1;
else if (1
return x;
else
return x * power(x, n-1);
Code: 1.5
Recursive function in Code 1.5 accept two parameters. One of them
remains fixed, and other changes and terminates the recursion, Terminating
condition for this recursion is defined as
IF (n EQUALS 0) THEN return 1
But we have used two terminating conditions,
IF (n EQUALS 0) THEN return 1
IF (x EQUALS 1) THEN return x
‘This is to avoid unnecessary function calls when x is 1. In next chapter
we will learn that every function call is an overhead, both in terms of time
and memory.The four things that we should focus on while writing a function (in
this ordet) are:
1. It should serve the purpose. For every possible parameter the function
must always return expected results. It should not be ambiguous for
any input.
‘The time taken by function to execute should be minimized.
‘The extra memory this function consumes should be minimized.
4. Function should be easy to understand. Ideally the code should be
self-explanatory to an extent that it does not even require any
documentation (comments).
At no point during coding or while in the interview do we really care
about how many lines of codes does patticular function rans into as long as
length of code is justified (we are not writing duplicate piece of code).
In next chapter, we will learn that a recursive solution takes mote time
and more memory than the corresponding iterative (non-recursive)
solution, In Example 1.1, both iterative and recursive solutions are equally
easy to code, In such situations we should always go for non-recursive
solution.
Ye INTERVIEW TIP
If both recursive and non-recursive (iterative) solutions are equally easy and take almost
equal time to code, then always write the iterative solution. It takes less time and less
memory to execute,
‘The advantage of recursion is that, sometimes, a solution that
otherwise is very complex to comprehend, can be very easily visualized
recursively. We just need to solve the problem for base case and leave rest
of the problem to be solved by recursion. Consider Example 1.3 below:
Example 1.3: Tower of Hanoi
Tower of Hanoi is a Mathematical Game. There are 3 pegs (Rods),
Source, Destination and Extra marked as S, D and = respectively,
and there ate n discs, each of different size, which can be inserted into any
of these three pegs.
3 Recursion is just replacing a loop in the iterative solution.All discs are initially inserted into Source peg in decreasing order
(smallest at the top) as shown in Picture 1.1 (for n=4).
L
a
Ss D E
Picture: 1.1
We have to move all the discs from Source peg (S) to Destination
peg (D). The final state should be as shown in Picture 1.2
‘There are 2 restrictions:
1. Only one disc can be moved at a time,
2. Atany point in the process we should never place a larger disc on
top of a smaller disc.
Write a function that accept chatacters representing three rods (S, D
& E) and the number of discs (n), and print the movement of discs
between pegs such that all discs are moved from the initial state (inside 8)
to the final state (inside D). Signature of the function is
/* s, ad, e represents three pegs
* (source, destination and extra).
* n is number of discs (All initially in s)*/
void towerOfHanoi(char s, char d, char e, int n)
This may appear to be a complex problem to solve otherwise, but if
we think recursively, then the problem can be solved in three simple steps
Step-1: Move n-1 discs from S to E using D
Let us assume that somehow n-1 discs are moved from S$ to E, and we
have used D as the third peg (extra). This problem is similar to the original
problem (of moving n discs from 5 to D using E).After this step, the state of pegs and discs is as shown in the picture 1.3
s D E
Picture: 1.3
Step-2: Move the n’th disc from S to D
Move Disc No. n from S to D. This is a single step of execution.
Step-3: Move n-1 discs from E to D using S
This problem is again of the same nature as step-1. Here we are
moving n-1 discs from E to D using peg-S as extra.
If we carefully notice, Step-1 and Step-3, being problems of the same
type, are recursive calls to the same function. Code 1.6 is the recursive
function for Tower of Hanoi:
void towerOfHanoi(char s, char d, char e, int n){
// TERMINATING CONDITION
if(n <= 0)
return;
towerOfHanoi(s, e, d, n-1);
printf ("Move Disk-%d FROM %d TO 8d”, n, s, 4);
towerOflanoi(e, d, s, n-1);
Code: 1.6
‘The terminating condition here is when there is no disk (n==0) .
Notice that we have put the condition as less-than ot equal to, to handle
cases when n is negative, alternatively, we can change signature of function
to receive unsigned int in place of int.
If we call this function for 3 discs (n=3) like below
towerOfHanoi('s', 'd', 'e', 3)7The output is
Move Disk-1 FROM s TO d
Move Disk-2 FROM s TO e
Move Disk-1 FROM d TO e
Move Disk-3 FROM s TO d
Move Disk-1 FROM e TO s
Move Disk-2 FROM e TO d
Move Disk-1 FROM s TO d
Now we can appreciate, how helpful recursion can be even if it takes
more time and more memory to execute.
Head Recursion and Tail Recursion
‘A Recursive function typically perform some task and call itself. If the call is
made before the function performs its own task, then it is called Head-
Recursion (Recursion is performed at the head of the function body). If recursive call
is made at the end, then itis ‘Tail-Recursion.
In Code 1.1, function sum (3), call function sum(2) first and then
perform the add operation’ (return value from sum (2) added with 3). This
makes function sum, a head-recursive function.
To see the difference, consider Code 1.7 having two recursive
functions to traverse a link list:
A recursive function is head-recursive if recursive call is
made before it performs its own task. tail-recursion is when
recursive call is made at end of the function (after it
performs its own tasks).
/* Head Recursion,
* First traverse rest of the list, then
* print value at current Node. */
4 Tip: In C language the order of e valuation of operands for plus operator (+) is
not defined. It means that in the below statement:
x = funl() + fun2()7
x will be sum of return values of two functions but, whether fun1 is called first or
funz is not defined in the language. This has to be defined by the compiler.void traversel (Node* head) {
if (head != NULL) {
traversel (head->next) ;
printf (“td”, head->data);
}
/* Tail Recursion.
* First traverse rest of the list, then
* print value at current Node. */
void traverse2 (Node* head) {
if(head != NULL) {
print£("%d", head->data) ;
traverse2 (head->next) ;
Code: 1.7
Tf below linked list is passed as input to the two functions in Code 1.7:
Te? [+> 14> NN
Then, traverse function prints the list in backward order and
traverse2 prints it in forward order.
Output of traversel: 4 3 21
Output of traverse2: 1 2 3 4
A tail recursion is very easy to re-write in form of a loop. So there
should ideally be no teason to write a tail recursion in code unless we are
writing it to demonstrate recursion itself.
Head and tail recursion ate just given here as concepts. Most of the
times (esp. in case of dynamic programming problems) recursion is more
complex and may not be simple head ot tail recursion. Consider one of the
most rampant examples of recursion, in-order traversals of a Binary Tree.
In in-order traversal, we first traverse left subtree in in-order, then
traverse (eg, print) the root and finally traverse the right subtree in in-order
as shown in Picture 1.4.Clearly the in-order function is defined in terms of in-order of left and
right subtrees, hence recursion.
Algorithm: (A)
1. Traverse Root
2. Traverse Left Subiree in PreOrder © ©
3. Traverse Right SubTree in PreOrder
op ECra Q® ®®
Picture: 1.4
If we take structure of the Node as below:
Struct Node{
Node *left; // Pointer to Left subtree
int data;
Node *right; // Pointer to Right subtree
Le
Then code of in-order traversal is as shown in Code 1.8 below:
/* Print In-Order traversal of the tree */
void inOrder(node* r) {
if(x == NULL)
return;
inOrder (r->left) +
printf£(“d ”, r->data);
inOrder (r->right) ;
Code: 1.8
In the above code, recursion cannot be termed as either head or tail
recursion,
‘The terminating condition that we have taken in Code 1.8 is when root
is NULL, It will have extra function calls for leaf nodes, because function is
called for left and right subtree even when both are NULL. A better solution
is to check that a subtree is not NULL before calling the function for that
subtree.
/* Print In-Order traversal of the tree */
void inOrder(node* r) {if(x == NULL)
return;
if(r->left != NULL)
inOrder (r->left);
printf("td “, r->data);
if(r->right != NULL)
inOrder (r->right);
Code: 1.9
Tt may look like small improvement, but it will reduce our number of
function calls to almost half, because in a binary tree the number of null
pointers is always greater than the number of valid pointers. In Code 1.8,
we are making one function call for each pointer (null or non-null). But in
code 1.9, the function calls are against non-null pointers only. Hence the
total number of function calls in Code 1.9 is almost half as compared to
function calls made in code 1.8.
Consider the below binary tree
Thete are 8 null pointers, 6 as children of leaf
() nodes and tight child of £ and left child of c.
@ © If we consider root also as one pointer
(pointing to the root node, A), then total number of
©) ©) © swanall pointers are 7 (one pointing to each ade
in the trec).
© Putting such small checks not just optimize
our code but also shows our commitment toward writing better code.
Next chapter discuss how does memory actually looks when a
recursive function is called in contrast to an iterative function.
How to Solve a Problem Using Recursion
Our focus usually is to solve a problem, with recursion, we can actually
code without solving a problem, if we can somehow define the large
problem in terms of smaller problems of same type.Our focus is on solving the problem fot top case and leave the rest for
recursive calls to do. Consider the below example:
Example 1.4: We all know about Bubble Sort, where an array is sorted in n
passes as shown in the below code:
void bubbleSort (int *arr, int n)
{
for(int ¢ i
arr[j+1])
swap(6arr[j], &arr[j+1])¢
# j arr[j+1])
swap(Sarr[j], sSarr[j+1]);
bubbleSortRec (arr, n-1);
Code: 1.12
We cannot be good coders without mastering the art of recursion.
‘My suggestion to everyone reading this book is that when you write simple
programs like linear search, binary search, sorting, etc. try to also implement
them recursively. It will be a good net practice before the actual match.
Question 1.1: Below code print the mathematical table of n.
void printTable(int n){
for(int i=l; i<=10; i++) {
printf (“$d * %d = %d\n”, n, i, (n*i));
Code: 1.13
Write a recursive function that prints the mathematical table of n.
Hint: You may have to pass/ accept i as parameter.How it Looks in Memory
‘Application’
a
|
oo T
Before discussing how a recursive calls looks inside memory, we should
understand how memory is divided internally and what part of the program
goes in which section of the memory. This chapter is specific to C language,
but the concepts are similar in other popular languages also.
ant g = 5; — Goes in Data area.
aint main()
4 > 20 (default value)
static int a; .
int b; Josie, ConA of main)
ant *p; cefautt — guvtage
(int*) malloc (sizeof(int));
10;
} act AR of
Picture 2.1 shows lifecycle of a C language program:
Source Files
User-defined__
Header Files
°
Standard a
header files £
Picture: 2.1
Read any good C language book to learn about compilation and
linking of a C language program, this book assume that you have workingknowledge of C language. After compiling and linking, * the binary
executable of program gets generated (exe on windows). When this
executable binary is actually executing (running) it is called a process.
When a process is executed, first it is loaded into memory (RAM).
Area of memory where process is loaded is called process address space.
Picture 2.2 shows broad layout of process address space (Pidure 2.2 is
independent of platform, actual layout may differ for operating system and for program).
» Executable Code
Global / Static Data
HEAP —}-+ Dynamic memory (Allocated using
| malloc, ealloc, new, et.)
STACK —}-+ Activation Records of functions
Command line arg. & environment variables
Picture: 2.2
This memoty is allocated to our program by the operating system. The
process address space has following segments
1. Code segment (or Text segment)
2. Data segment
3. Stack segment
4. Heap segment
In next section, we are discussing these segments one by one.
Code segment
v This segment contains machine code (in form of executable
instructions) of the compiled program.
Y — Itis read-only and cannot be changed when the program is executing,
VY May be shareable so that only a single copy is in memory for different
executing programs®.
Y Size of code segment is fixed at load time.
5 Most IDEs compile, link and execute the program using a single button click. But
internally all these steps are performed.
§ Shareable code is outside the scope of this book.Data Segment
v
v
v
All global and static data variables are allocated memory in this
segment.
Memory is allocated in this area when the progtam is loading (before
main function is called). That’s why global and static variables are also
called load-time variables.
All load-time variables (global and static), are initialized at the load-
time. If no initial value is given for a load-time variable then it is
initialized with the zero of its type’.
Internally, this segment is divided in two areas, initialized and
uninitialized. If initial value of a variable is given it goes in the
initialized data area else it goes in the uninitialized data area, All the
uninitialized variables are then initialized with zeros. The main reason
why they are stored separately within data segment, is, bacause the
uninitialized data area can be memset to zero in a single operation.
Size of data segment is fixed at load time and does not change when
program is executing.
Stack Segment
v
Stack segment contains Activation Records (also called Stack Frames)
of all the active functions. An active function is a function that is
curtently under the call. Consider Code 2.1 below
int main(){
funl();
}
void funl(){
fun2();
}
void fun2(){
}
void fun3(){
// NEVER CALLED
Code: 2.1
7 Zero of int data type is 0. Zero of pointer data type is NULL.When main is called, it is the only active function. Then main calls
fun1. At this point fun1 is executing but both main and fun1 are
active. When funl calls fun2, then the execution is in fun2, but
main, fun1 and fun2 are all active and has their activation records
in Stack.
When function fun2 returns, then activation record of fun2 is poped
from the Stack and execution is back in fun1. At this point main and
fun1 are active and has their activation records in the Stack.
fun3, is never active, because it is never called and hence its activation
record never gets created on the Stack.
¥ When a function is called, its Activation Record is created and pushed
on the top of stack.
Y When a function returns then the corresponding Activation Record is
poped from the Stack.
Y Size of Stack keeps changing while the program is executing because
the number of active functions keep changing.
¥ Non-static local variables of a function are allocated memory inside
Activation Record of that function when it is active.
v Variables that are allocated memory on Stack are not initialized by
default. If initial value of a variable is not given then it is not initialized
and its value will be garbage (this is different from load-time variables
allocated memory in Data Segment).
Y Activation record also contains other information required in function
execution.
Y Stack Pointer (SP register) keeps track of the top of the Stack.
Y Point of execution is always inside the function whose activation
record is on the top of Stack. Function whose activation record is
inside Stack, but not at the top is active but not executing.
¥ If a function is recursive then multiple activation records of the
function may be present on the Stack (one activation record for each
instance of the function call),
Heap Segment
v
When we allocate memory at run time using malloc (), calloc(),
and realloc() in C language (new and new{] in C++), then that
memory is allocated on the Heap. It is called dynamic memory or
run-time memory,Y InC language we cannot initialize the memory allocated on Heap. In
C++, if we use new operator to allocate memory, then we can initialize
it using constructors.
Y Memory allocated in heap does not have a name (unlike memory
allocated in and Stack segments). The only way to access this memory
is via pointers pointing to it. If we lose address of this memory, there is
no way to access it and such a memory will become memory leak. It
is one of the largest sources of ertor in C/C++ programming.
Y Both Heap and Stack segment shares 2 common area and grows
toward each other.
After compilation and linking, the executable code (in machine
language) gets generated. The first thing that happens when this executable
code is executed is that it is loaded in the memory. Loading has following
steps:
Y Code goes in code area. Code is in the form of binary machine
language instructions and Instruction Pointer (IP register) holds the
address of current instruction being executed.
¥ Global and static variables are allocated memory in the data area.
Data area has two sub-parts, Initialized and Un Initialized data area, if
initial value of a variable is given by us, it gets allocated in the initialized
data area, else memory to the vatiable is allocated in the un-initialized
data area and it is initialized with zero.
¥ Global and static variables are initialized. If we have given the
initial value explicitly, then variables are initialized with that value
otherwise they are initialized with zeros of their data types.
int x = 5; // initialized with 5
int y; // initialized with 0
After these steps, we say that *he program is loaded. After loading is
complete, the main function is called and actual execution of the program
begins. Read the entire program given in Code 2.2 carefully:
// Go in data area at load time. Initialized with 0.
int total;
/** Code (machine instructions) of function goes in
* code area. When this function is called, then
® Question: Who calls the main function ?* Activation Record of the function is created on
* Stack.
+f
int square(int x){
// * goes in AR’ when this function is called.
return x*x;
/** Code of function goes in the code area. When this
* function is called (at run-time), its AR gets
* created on Stack and memory to non-static local
* variables (x and y) is allocated in that AR.
* count, being a static variable, is allocated in
* data area at load time.
int squareOfSum(int x, int y) {
static int count = 0; // Load-time var
printf ("Fun called %d times", ++count) ;
return square (xty);
/** Code goes in code area. When main is called, its
* activation record gets created on Stack and memory
* to non-static local variables (a and b) is
* allocated in that Activation Record.
any
int main() {
int a=4, b=2;
total = squareOfSum(a, b);
printf ("Square of Sum = %d",total);
Code: 2,2
This program computes (a+b)? and print the result. To keep it
simple, we are using the hard coded values 4 and 2 for a and b respectively.
The function squareOfSum also keeps a count of how many times it is
called in a static variable count, and print the count every time it is called.
Code 2.2 may not be the best implementation, but it serves our
purpose. Read the code again, especially the comments before each
° AR = Activation Recordfunction and make sure that we understand everything.
After compilation and linking, the executable of the program is created
and when this executable is run, the first thing that happens is that this
executable is loaded in the memory (RAM). At this point the main
function is not yet called and the memory looks like Picture 2.3:
CODE
0 |totjal
Q__|count
STACK@) @@dV4H DATA
Picture: 2.3
After loading is complete, main function is called. When a function is
called, its Activation Record is cteated and pushed in the Stack. The AR has
Y Local (non-static) variables of a function (a andb for main).
v¥ Other things stored in the Activation Record.
In the diagrams, we are only showing non-static local variables in
Activation Records, Aftet main function is called, the memory looks as
shown in Picture 2.4
CODE
O_ |total
O__|count
STACK™ @@dVaH DATA
4 AR of
2 |b main()At any time, the point of execution (Instruction Pointer) is in the
function whose AR is at the top of Stack. Let us understand what all
happens internally duting a function call.
When a function is called:
1. State (register values, Instruction Pointer value, etc.) of calling function
is saved!” in the memory.
2. Activation record of called function is created and pushed on the top
of Stack. Local variables of called function are allocated memory inside
the AR.
3, Instruction pointer (IP register) moves to the first executable
instruction of called function.
4, Execution of the called function begins.
Similarly when the called function returns back (to the calling function),
following work is done:
1, Return value of the function is stored in some register.
2. AR of called function is popped from the memory (Stack size is
reduced and freed memory gets added to the free pool, which can be
used by either the stack or heap).
3, — State of the calling function is restored back to what it was before the
fanction call (Point-1 in fanction call process above).
4, Instruction pointer moves back to the instruction where it was before
calling the function and execution of calling function begins from the
point at which it was paused'!,
5, Value returned from called function is replaced at the point of call in
calling function.
1° Value of local variables of a function under execution are stored in the AR of function
which is preserved in the stack. But Registers will also have some values, these values
also need to be saved (because Registers are needed by the called function). This state
is saved in the memory.
" ‘This is conceptually similar to Context Switch of process contexts in a
muti-processing operating system when one process is preempted to execute another
process and after some time control returns back to the first process and it starts
executing from the same point where it was preempted.Clearly, a function call is a lot of overhead both in terms of
time and memory.
One of the reasons behind the popularity of macros in C language
(even after all the evil that they bring along) is this overhead in function call.
Another was the type independence that macros bring!2,
Some compilers optimize the performance by replacing function call
with entire code of the function during compilation, hence avoiding the
actual function call overheads. This is called inline expansion. For example,
in Code 2.2, the compiler may just put entire code of function square
inside squareOfSum and remove the function call all together as shown
below.
int squareOfSum(int x, int y){
static int count = 0; // Load-time var
printf (“Fun called %d times”, ++count) ;
return (x+y) * (x+y);
Code: 2.3
Recursive functions are very difficult to expand inline because
compiler may not know the depth of function call at compile time.
Example 2.1: Let us also sce how memory looks like if we miss the
terminating condition in recursion. Code 2.4 is example of infinite recursion.
int main() {
int x = 0;
xtt;
Lf (x<5) {
printf (“Hello”);
main();
Code: 2.4
When the program is executed after compilation, it is first loaded in
the memory and then the main function is called. At this point (after
2? In C++, both the benefits are given in the form of inline functions and templates and
they are not error prone like macros.calling main) the memory looks as shown in Picture 2.5. Code atea has the
code, The Data area is empty because there is no load-time (global or static)
variable, Stack has only one activation record of function main.
‘CODE,
DATA
STACK
[0 _] Je AR of main()
Picture: 2.5
Initial value of x is 0, after increment x become 1, since x<5, the
condition is true and main is called again. A new AR for this newly called
main is created on the Stack and this AR also has local variable x that is
different from variable x in AR of previous call (see Picture 2.6). Value of
this new x is again 0, and main is called again. Every time main is called,
the value of x in the new activation record is 0.
CODE
DATA
el o JAR of main()
STACK
x 1 JAR of main ()
Picture: 2.6
Every instance of main is actually using a different x (from their own
instance of AR).
Code 2.4 will continue to print “He11o”, until a point when no space
is left in the Stack to create new AR. At this point main cannot be called
farther and the program will crash.
An important thing to note is that the program will not print “Hello”
infinitely. It is printed, till the memory stack overflows.Recursive v/s Non-Recursive Inside Memory
Let us consider Example 1.1 from chapter-1 again.
int sum(int n){
if (n==1)
return 1;
else
return n + sum(n-1);
Code: 2.5
When we call this function for n=3, as sum(3); It will call sum(2) ;
which will in-turn call sum (1) 7
At this point (when execution control is in sum(1)), the memory
stack will have three instances of activation records of function sum, each
having a local variable n, as shown in Picture 2.7.
a{_1_| JAR of sum(1y
al 2] JeAR of sumc2)
n [3] [AR of sum(3)
Picture: 2.7
In the iterative version (Code 1.4) there is only one function call to
sum(3) and three local variables n, i and sum on the Activation Record
(AR) of the function as shown in Picture 2.8.
[a of sum (3)
Picture: 2.8
In recursive version, one activation record is created for each value of
n. If n=1000 then 1000 ARs are created. Therefore the extra memory
taken is O(n). Table 2.1 gives a comparison of asymptotic running time
and extra memory taken for recursive and non-tecutsive sum functions.The asymptotic time may be same, 0 (n) for both the cases, but actual
time taken for recursive version is much more than the iterative version
because of the constant multiplier.
Recursive Non-Recursive
Time O(n) O(n)
Memory O(n) (1)
Table: 24
Example 2.2: Let us consider one more example. Code 2.6 is the recursive
function to computes factorial of n:
int factorial(int n){
if(1==n || O==n)
return 1;
else
return n * factorial (n-1);
Code: 2.6
If function is called for n=4,
fact (4);
from some other function to compute factorial of 4. During successive
function calls, the memoty looks like Picture 2.9.
=e oe
Se ree
¥ sreactian sreacteay [im 3)?" facta)
eteenm nen re ee
emma wesc are a
Picture: 2.9.
When the functions return value to their caller functions, the AR will
be poped from the stack and the stack will look like Picture 2.10 (return
values shown on right side),
Picture: 2.10There will also be other function’s AR in the Stack (eg, main function).
They are not shown to save the space.
Code 2.7 shows the non-recursive code to compute factorial of n.
int factorial (int n){
int £ = 1;
for(int i=2; i<=n; i++)
f-£* i;
return f;
Code: 2.7
‘The memory image of function in Code 2.7 is shown in Picture 2.11.
Compare it with the memory taken by the recursive code.
mw
Ql a
Flnl_4
Picture: 2.11
Code 2.7 may have mote local variables, but there is just one AR in the
memory irrespective of the value of n.
Recursion is a huge overhead. Both in terms of memory and
execution time.
The examples of recursion seen till now ate simple linear recursions.
One of the major problem with recursive function comes when recursive
calls starts overlapping at the level of subproblems. Overlapping
subproblems is discussed in detail in chapter 4.
Memory Layout as a Problem-Solving Tool
A clear understanding of lifecycle of program execution and how a program
is loaded in memory comes handy in solving many mote questions other
than recursion. Consider the below examples:Example 2.3: What is value of x in the below code?
static int x = strlen("Hello");
The above code is compile-time error. To put it simply, “static and
global variables cannot be initialized with the return value of a function.”
We are trying to initialize a static variable with the return value of
function strlen.
We know that static variables ate initialized at load time.
But wait, functions cannot be called at load time. A function can only
be called when loading is complete and the program is executing (first
function that gets called is main).
How can we initialize, at load-time, with something that is not
available until execution time. Hence, Error!
What if we break up the statement in two parts?
static int x; // initialized with zero
x = strlen("Hello");
Now thete is no problem. At load time variable x is initialized with
zero. During execution the function strlen ("Hello") is called and x is
assigned the value 5.
Example 2.4: What value will get printed if we call function fun?
void fun() {
int a = 5;
static int b = a;
printf ("value: 34", b);
Code: 2.8
No, the answer is not 5 ot 0. The above code is also a compile time
ERROR.
We know, static variables are initialized at load time. In code 2.8 we
are initializing b with a, but variable a, is not available while loading. It will
be allocated memory in the activation record when function £un is calledand fun is called at execution-time, It is called only after the loading is
complete and when the code starts executing.
Also, if there are more than one instances of any function in the Stack
(in case of recursive functions). Then each AR have a separate copy of local
variable a, but there is only one copy of static variable (allocated in the
data area). By that logic also static variable (single copy) cannot be initialized
with a local variable (possible zero or multiple copies).
Load-time variables cannot be initialized with local variables.
Conclusion
1. A function will have multiple ARs inside stack if and only if it is
recursive.
2. Global and static variables can only be initialized with constants.
3. The memory to load-time variables is allocated before any function is
called.
4. "The memory to load-time variables is released only after the execution
is complete.
We have not discussed the Heap area because the purpose was to explain
recursion and not pointers or dynamic memory allocation or deallocation.
To learn how heap area is used, read some good book on pointers in C
language.Optimal Substructure
Am I just recursion?
Optimal substructure means, that optimal solution to a problem of size n
(having n elements) is based on an optimal solution to the same problem of
smaller size (less than n elements). ic while building the solution for a
problem of size n, define it in terms of similar problems of smaller size, say,
k (k 2
} Fibonacci sequence appears in Indian mathematics, in connection with Sanskrit
prosody dated back to 450 B.C. Like most other things, the serics went from
east to west and was named after the guy who introduced it to west, rather
inal founders, that guy happens to be Fibonacci.The simplest algorithm to compute n* term of Fibonacci is direct
translation of mathematical definition:
int fib(int n){
if (m==1 || n==2)
return 1;
else
return fib(n-1) + fib(n-2);
Code: 4.1
Code 4.1 is a recursive code. We may want to put an extra check and
throw an exception if n is negative or zero, so that our code does not run
into infinite recursion if called with zero or negative value as argument. We
skipped this check to keep the code simple.
It may not be obvious, but Example 4.1 also has optimal substructure
property. To find optimal solution (the only solution in this case) for n™
term we need to find the optimal solution for n-1* tetm and n-2% term.
‘The equation of time taken by the function in Code 4.1 is
T(m) = T(n-1) + T(n-2) + O(1)
‘This is an equation for exponential time. The reason why it is taking
exponential time for such a simple algorithm is because it is solving
the subproblems (computing Kk term, k