SYLLABUS

Unit I

Introduction: Overview-Definition- Concept of data structures-Overview –

Implementation-Arrays-Definition-One Dimensional Array-Multidimensional Array-
Pointer Array-Linked list: Definition-Singly linked list-Double linked list-Circular
double linked list-Application-Memory representation.

Unit II

Stack: Definition-Representation-Operations-Applications-Queues:Definition-
Representation-Various queue structures-Applications- Trees-Definition-Representation
–Operations-Types.

Unit III

Divide and conquer: General method-Binary search-Finding the maximum minimum-

Merge sort-Quick sort-Selection- Strassen‘s matrix multiplication.

Unit IV

Greedy method-General method-Optimal storage on tapes- Knapsack problem-Job

sequencing with deadlines-Optimal merge pattern-Minimum spanning trees-Single
shortest paths.

DATA STRUCTURES AND COMPUTER ALGORITHMS

Unit V

Dynamic programming-General method-Multistage graphs-all pair shortest path-optimal

binary search trees-0/1 Knapsack reliability Design-Traveling sales person problem-flow
shop scheduling.

Text book:
Classic data structures by D.Samantha
Fundamentals of computer algorithm by Ellis Horowitz, Sartaj sahini, Sanguthevar
Rajasekaran

DATA STRUCTURES AND COMPUTER ALGORITHMS

 UNIT - I
Introduction and overview
Definitions
Before going to study the actual subject, basic terminologies and concepts are described
below.

Data:
Data means value or set of values. Following are the some examples of data
(i) 34
(ii) 12/01/1965
(iii)ISBN 81-203-0000-0
(iv) Pascal
Values are represented in different ways. Each value or collection of all such values is
termed as data.

Entity:
An entity Data Entity is one the components defined in a logical data model. A logical
data model is a representation of all of the organizations data which are being organized
in terms of data management technology.

A relationship captures how two or more entities are related to one another. Relationships
can be thought of as verbs, linking two or more nouns.

Examples: an owns relationship between a company and a computer,

a supervises relationship between an employee and a department, a performs relationship
between an artist and a song, a proved relationship between a mathematician and a
theorem.

Abstract data types

DATA STRUCTURES AND COMPUTER ALGORITHMS

When designing Data Structures and Algorithms it is desirable to avoid making
decisions based on the accident of how you first sketch out a piece of code. All
Design should be motivated by the explicit needs of the application. The idea of
an Abstract Data Type (ADT) is to support this (the idea is generally considered
good for program maintainability as well, but that is no great concern for this
particular course).
The specification of an ADT is a list of the operations that may be performed on it,
together with the identities that they satisfy. The user of an ADT is expected to view this
specification as the complete description of how the data type and its associated functions
will behave — no other way of interrogating or modifying data is available, and the
response to any circumstances not covered explicitly in the specification are deemed
undefined.

To help make this clearer, here is a specification for an Abstract Data Type
called STACK:
Make empty stack (): manufactures an empty stack.
Is empty stack(s): s is a stack. Returns TRUE if and only if it is empty.
Push(x,s): x is an integer, sis a stack. Returns a non-empty stack which can
Be used with top and pop. is empty stack(push(x,s))=FALSE.
Top: s is a non-empty stack; returns an integer. Top(x,s)) (push= x.
Pop: s is a non-empty stack; returns a stack. pop(push(x,s)) =s.3
ADT circle
Data
r³0 (radius);
Operations
Costructor:
Initial values:radius of circle;
Process: Assign initial value of r;
Area:
Input: none;

DATA STRUCTURES AND COMPUTER ALGORITHMS

 Preconditions: none;
Process: A = p*r*r
Output: A
Postcondition: none
Circumference:
Input: None;
Preconditions: none;
Process: C = 2*p*r
Output: C
Postconditions: none;
End ADT circle;

CONCEPT OF DATA STRUCTURES

Digital computer can manipulate only primitive data that is data in terms of 0‘s and 1‘
s.real life applications, various kinds of data other than the primitive data are involved.
1. Storage: representation of user data: user data should be stored in such a way
computer can understand it.
2. Retrieval of stored data: data stored in a computer should be retrieved in such a way
that user can under stand it.
3. Transformation of user data: various operations which require to be performed on
user data so that it can be transformed from one form to another.
In user data, its structure implies the following:

1. Domain (d):this is the range of values that the data may have. This domain also
termed as data object.
2. Fnction (f): this is the set of operations which mazy legally applied to elements of
data object. This implies that for data structure we must specify the set of operations.
3. Aioms (a); this is the set of rules with which the different operation belongs to
function actually can be implemented.

DATA STRUCTURES AND COMPUTER ALGORITHMS

 A data structure D is a triplet that is D= (d, f ,a) where d is set of data object and f is a
set of functions, and a is a set of axioms.
d = (0, +1or -1, +2or-2,)
f = (+,-,*, /, %)
a= (a set of binary arithmetic‘s to perform addition, subtraction, division, multiplication,
and also modulo operations.)

Overview of data structures:

In computer science several data structures are known depending on area of

applications. Of them, few data structures are there which are frequently used almost in
all application areas.
These data structures known as fundamental data structures or classic data structures.

Classification of Classic data structures:

1. Linear data structures- arrays, linked lists, stacks, queues
2. Non-linear data structures-trees, graphs, tables, set.

In the linear structure all the elements are form a sequence or maintain a linear ordering.
But in non linear data structure no such sequence and all the elements are distributed.

Implementation of data structures:

The heap sort algorithm works with a heap tree and a result list. They can be stored
efficiently in one array.

Storing Heap tree and Result List in an Array

Heap tree and result list are stored together in one array. The array has as many elements
as there are keys to sort. No additional storage space for pointers or intermediate results is
needed.
DATA STRUCTURES AND COMPUTER ALGORITHMS
Providing Input Data

A program that wants to sort data with the heap sort algorithm provides it as an array.
The array appears to the algorithm as a heap tree containing all keys. In the notation of
figure 1 the green heap tree occupies the whole array. That is why the heap tree needs not
be built but exists instantly, complete and completely filled.

Running the algorithm

The function heap sort performs the complete algorithm. Within the algorithm the
function move max moves keys from the heap tree to the result list. This removes one
key from the heap tree which makes room in the array at the border between heap tree
and result list. The result list grows using this free node and stores the new element there.

Providing Output Data

After the algorithm run the result list occupies the complete array. There is no need to
copy this list, because the array can be used directly by the program.

Mapping Heap tree Nodes to Array Nodes

The heap tree root is stored at index 1 in the array.

For a node stored at index k in the array, its left child is stored at index 2*k and its right
child at index 2*k+1 (figure 2). It follows that the parent node is at index (k div 2).

Figure: Mapping child nodes to indices

DATA STRUCTURES AND COMPUTER ALGORITHMS

Figure 3 shows the array indices of the heap tree nodes for the heap tree used in the
courseware.

Figure: Array indices of heap tree nodes

Arrays

Definition

i) An array is simply a number of memory locations, each of which can store an item of
data of the same data type and which are all referenced through the same variable
name. Ivor Horton.

ii) Array may be defined abstractly as finite order set of homogeneous elements. So we
can say that there are finite numbers of elements in an array and all the elements are of
same data type. Also array elements are ordered i.e. we can access a specific array
element by an index.

How to declare an array?

The general form of declaring a simple (one dimensional) array is

DATA STRUCTURES AND COMPUTER ALGORITHMS

in your C/C++ program you can declare an array like

array type variable_name[array_size];

int age[10];

Here array_type declares base type of array which is the type of each element in array. In
our example array_type is int and its name is Age. Size of the array is defined by
array_size i.e. 10. We can access array elements by index, and first item in array is at
index 0. First element of array is called lower bound and its always 0. Highest element in
array is called upper bound.

In C programming language upper and lower bounds cannot be changed during the
execution of the program, so array length can be set only when the program in written.

Age 0 Age 1 Age 2 Age 3 Age 4 Age 5 Age 6 Age 7 Age 8 Age 9

30 32 54 32 26 29 23 43 34 5

Array has 10 elements

Note: One good practice is to declare array length as a constant identifier. This will
minimize the required work to change the array size during program development.

Considering the array we declared above we can declare it like

#define NUM_EMPLOEE 10

int Age[NUM_EMPLOYEE];

How to initialize an array?

Initialization of array is very simple in c programming. There are two ways you can
initialise arrays.

Declare and initialize array in one statement.

DATA STRUCTURES AND COMPUTER ALGORITHMS

 Declare and initialize array separately.

Look at the following C code which demonstrates the declaration and initialisation of an
array.

int Age [5] = {30, 22, 33, 44, 25};

int Age [5];

Age [0] =30;

Age [1] =22;

Age [2] =33;

Age [3] =44;

Age [4] =25;

Array can also be initialized in a ways that array size is omitted, in such case compiler
automatically allocates memory to array.

int Age[] = {30, 22, 33, 44, 25};

One dimensional array:

The array which is used to represent and store data in a linear form is called as 'single or
one dimensional array.'

Syntax:

<data-type> <array_name> [size];

Example:

int a[3] = {2,3,5};

char ch[20] = ―tech exam;

DATA STRUCTURES AND COMPUTER ALGORITHMS

float stax[3] = {5003.23,1940.32,123.20};

Total size (in Bytes);

total size = length of array * size of data type

In above example, a is an array of type integer which has storage size of 3 elements. The
total size would be 3 * 2 = 6 bytes.
Memory allocation for an array:

#include <stdio.h>
#include <conio.h>
void main()
{
int a[3], i;;
clrscr();
printf("\n\t Enter three numbers : ");
for(i=0; i<3; i++)
{
scanf("%d", &a[i]); // read array
}
printf("\n\n\t Numbers are : ");
for(i=0; i<3; i++)
{
printf("\t %d", a[i]); // print array
}

DATA STRUCTURES AND COMPUTER ALGORITHMS

 getch();
}

Output :

Enter three numbers: 9 4 6

Numbers are: 9 4 6_

Operations on array:

Various operations that can be performed on an array are traversing, sorting, searching,
insertion, deletion, merging.

Traversing

This operation is used visiting all elements in an array. A simplified algorithm is below:

Description: Here A is a linear array with lower bound LB and upper bound UB. This
algorithm traverses

Array A and applies the operation PROCESS to each element of the array.

1. Repeat For I = LB to UB

2. Apply PROCESS to A[I]

[End of For Loop]

3. Exit

Explanation: Here A is a linear array stored in memory with lower bound LB and upper
bound UB.

DATA STRUCTURES AND COMPUTER ALGORITHMS

For loop iterates from LB to UB and visits each element of the array. During this visit, it
applies the operation PROCESS to the elements of the array A.

Note: The operation PROCESS in the traversal algorithm may use certain variables
which must be initialized before PROCESS is applied to any of the elements in the array.
Therefore, the algorithm may need to be preceded by such an initialization step.

Sorting:

One of the simplest methods to sort an array is an insertion sort. An example of an

insertion sort occurs in everyday life while playing cards. To sort the cards in your hand
you extract a card, shift the remaining cards, and then insert the extracted card in the
correct place. This process is repeated until all the cards are in the correct sequence. Both
average and worst-case time is O(n2)..

Theory

Starting near the top of the array in Figure 2-1(a), we extract the 3. Then the above
elements are shifted down until we find the correct place to insert the 3. This process
repeats in Figure 2-1(b) with the next number. Finally, in Figure 2-1(c), we complete the
sort by inserting 2 in the correct place.

DATA STRUCTURES AND COMPUTER ALGORITHMS

 Figure: Insertion Sort

Assuming there are n elements in the array, we must index through n - 1 entries. For each
entry, we may need to examine and shift up to n - 1 other entries, resulting in a O(n2)
algorithm. The insertion sort is an in-place sort. That is, we sort the array in-place. No
extra memory is required. The insertion sort is also a stable sort. Stable sorts retain the
original ordering of keys when identical keys are present in the input data.

Algorithm:

PRE: array of N elements (from 0 to N-1)

POST: array sorted

1. An array of one element only is sorted

2. Assume that the first p elements are sorted.

For j = p to N-1
Take the j-th element and find a place for it among the
first j sorted elements

DATA STRUCTURES AND COMPUTER ALGORITHMS

 int j, p;
comparable tmp;

for ( p = 1; p < N ; p++)

{
tmp = a[p];
for (j = p; j > 0 && tmp < a[j-1]; j--)

a[j] = a[j-1];
a[j] = tmp;
}

Insertion:

Insertion sort is a simple sorting algorithm: a comparison sort in which the sorted array
(or list) is built one entry at a time. It is much less efficient on large lists than more
advanced algorithms such as quicksort, heapsort, or merge sort. However, insertion sort
provides several advantages:

Simple implementation
Efficient for (quite) small data sets
Adaptive (i.e., efficient) for data sets that are already substantially sorted:
thetime complexity is O(n + d), where d is the number of inversions
More efficient in practice than most other simple quadratic (i.e., O(n2)) algorithms
such as selection sort or bubble sort; the best case (nearly sorted input) is O(n)
Stable; i.e., does not change the relative order of elements with equal keys
In-place; i.e., only requires a constant amount O(1) of additional memory space
Online; i.e., can sort a list as it receives it
insertionSort(array A)

{ This procedure sorts in ascending order. }

DATA STRUCTURES AND COMPUTER ALGORITHMS

begin
for i := 1 to length[A]-1 do
begin
value := A[i];
j := i - 1;
done := false;
repeat
{ To sort in descending order simply reverse
the operator i.e. A[j] < value }
if A[j] > value then
begin
A[j + 1] := A[j];
j := j - 1;
if j < 0 then
done := true;
end
else
done := true;
until done;
A[j + 1] := value;
end;
end
Applications of arrays

Arrays are used to implement mathematical vectors and matrices, as well as other
kinds of rectangular tables. Many databases, small and large, consist of (or
include) one-dimensional arrays whose elements are records.

Arrays are used to implement other data structures, such as heaps, hash
tables, deques, queues, stacks, strings, and VLists.

DATA STRUCTURES AND COMPUTER ALGORITHMS

 One or more large arrays are sometimes used to emulate in-program dynamic
memory allocation, particularly memory pool allocation. Historically, this has
sometimes been the only way to allocate "dynamic memory" portably.

Arrays can be used to determine partial or complete control flow in programs, as a

compact alternative to (otherwise repetitive), multiple IF statements. They are
known in this context as control tables and are used in conjunction with a purpose
built interpreter whose control flow is altered according to values contained in the
array. The array may contain subroutine pointers (or relative subroutine numbers
that can be acted upon by SWITCH statements) - that direct the path of the
execution.

Multi dimensional arrays

Two Dimensional Arrays:
The array which is used to represent and store data in a tabular form is called as
'two dimensional array.' Such type of array specially used to represent data in a
matrix form.

The following syntax is used to represent two dimensional array.

Syntax:
<data-type> <array_nm> [row_subscript][column-subscript];
Example:
Int a[3] [3];

In above example, a is an array of type integer which has storage size of 3 * 3 matrix.
The total size would be 3 * 3 * 2 = 18 bytes.
It is also called as 'multidimensional array.'
MEMORY ALLOCATION:

DATA STRUCTURES AND COMPUTER ALGORITHMS

Figure: Memory allocation for memory
Program:

/* Program to demonstrate two dimensional array.

#include <stdio.h>
#include <conio.h>
void main()
{
int a[3][3], i, j;
clrscr();
printf("\n\t Enter matrix of 3*3 : ");
for(i=0; i<3; i++)
{
for(j=0; j<3; j++)
{
scanf("%d",&a[i][j]); //read 3*3 array
}
}
printf("\n\t Matrix is : \n");
for(i=0; i<3; i++)
{
for(j=0; j<3; j++)
{
scanf("\t %d",a[i][j]); //print 3*3 array
}

DATA STRUCTURES AND COMPUTER ALGORITHMS

 printf("\n");
}
getch();
}

Output :
Enter matrix of 3*3: 3 4 5 6 7 2 1 2 3
Matrix is :
3 4 5
6 7 2
1 2 3_

Limitations of two dimensional arrays:

o We cannot delete any element from an array.
o If we don‘t know that how many elements have to be stored in a memory in
advance, then there will be memory wastage if large array size is specified.

Sparse matrices:

DATA STRUCTURES AND COMPUTER ALGORITHMS

A sparse matrix obtained when solving a finite element problem in two dimensions. The
nonzero elements are shown in black.

In the subfield of numerical analysis, a sparse matrix is a matrix populated primarily

with zeros. Conceptually, sparsely corresponds to systems which are loosely coupled.
Consider a line of balls connected by springs from one to the next; this is a sparse system.
By contrast, if the same line of balls had springs connecting each ball to all other balls,
the system would be represented by a dense matrix. The concept of sparsity is useful
in combinatorics and application areas such as network theory, which have a low density
of significant data or connections.

When storing and manipulating sparse matrices on a computer, it is beneficial and often
necessary to use specialized algorithms and data structures that take advantage of the
sparse structure of the matrix. Operations using standard dense matrix structures and
algorithms are slow and consume large amounts of memory when applied to large sparse
matrices. Sparse data is by nature easily compressed, and this compression almost always
results in significantly less computer data storage usage. Indeed, some very large sparse
matrices are infeasible to manipulate with the standard dense algorithms.

Storing a sparse matrix

The naive data structure for a matrix is a two-dimensional array. Each entry in the array
represents an element ai,j of the matrix and can be accessed by the two indices i and j. For
an m×n matrix, enough memory to store at least (m×n) entries to represent the matrix is
needed.

Substantial memory requirement reductions can be realised by storing only the non-zero
entries. Depending on the number and distribution of the non-zero entries, different data
structures can be used and yield huge savings in memory when compared to a naïve
approach.

Formats can be divided into two groups: (1) those that support efficient modification, and
(2) those that support efficient matrix operations. The efficient modification group
includes DOK, LIL, and COO and is typically used to construct the matrix. Once the
matrix is constructed, it is typically converted to a format, such as CSR or CSC, which is
more efficient for matrix operations.

Dictionary of keys (DOK)

DATA STRUCTURES AND COMPUTER ALGORITHMS

DOK represents non-zero values as a dictionary mapping (row, column)-tuples to
values. This format is good for incrementally constructing a sparse array, but poor for
iterating over non-zero values in sorted order. One typically creates the matrix with this
format, then converts to another format for processing. See scipy.sparse.dok_matrix.

List of lists (LIL)

LIL stores one list per row, where each entry stores a column index and value. Typically,
these entries are kept sorted by column index for faster lookup. This is another format
which is good for incremental matrix construction. See scipy.sparse.lil_matrix.

Coordinate list (COO)

COO stores a list of (row, column, value) tuples. Ideally, the entries are sorted (by
row index, then column index) to improve random access times. This is the traditional
format for specifying a sparse matrix in Matlab (via the sparse function), except as three
separate arrays instead of a single array of triples. This is another format which is good
for incremental matrix construction. Seescipy.sparse.coo_matrix and Matlab sparse.

Yale format

The Yale Sparse Matrix Format stores an initial sparse m×n matrix, M, in row form using
three one-dimensional arrays. Let NNZ denote the number of nonzero entries of M. The
first array is A, which is of length NNZ, and holds all nonzero entries of M in left-to-right
top-to-bottom (row-major) order. The second array is IA, which is of length m + 1 (i.e.,
one entry per row, plus one). IA(i) contains the index in A of the first nonzero element of
row i. Row i of the original matrix extends from A(IA(i)) to A(IA(i+1)-1), i.e. from
the start of one row to the last index before the start of the next. The third array, JA,
contains the column index of each element of A, so it also is of length NNZ.

For example, the matrix

[1 2 0 0]
[0 3 9 0]
[0 1 4 0]

is a three-by-four matrix with six nonzero elements, so

A= [1 2 3 9 1 4]

DATA STRUCTURES AND COMPUTER ALGORITHMS

IA= [0 2 4 6]

JA= [0 1 1 2 1 2]

In this case the Yale representation contains 16 entries, compared to only 12 in the
original matrix. The Yale format saves on memory only when NNZ < (m(n − 1) − 1) / 2.

Compressed sparse row (CSR or CRS)

Is effectively identical to the Yale Sparse Matrix format, except that the column array is
normally stored ahead of the row index array. I.e. CSR is (val, col_ind, row_ptr),
where val is an array of the (left-to-right, then top-to-bottom) non-zero values of the
matrix; col_ind is the column indices corresponding to the values; and, row_ptr is the
list of value indexes where each row starts. The name is based on the fact that row index
information is compressed relative to the COO format. One typically uses another format
(LIL, DOK, and COO) for construction. This format is efficient for arithmetic operations,
row slicing, and matrix-vector products..

Compressed sparse column (CSC or CCS)

It is similar to CSR except that values are read first by column, a row index is stored for
each value, and column pointers are stored. I.e. CSC is (val, row_ind, col_ptr),
where val is an array of the (top-to-bottom, then left-to-right-bottom) non-zero values of
the matrix; row_ind is the row indices corresponding to the values; and, col_ptr is the
list of val indexes where each column starts. The name is based on the fact that column
index information is compressed relative to the COO format. One typically uses another
format (LIL, DOK, and COO) for construction. This format is efficient for arithmetic
operations, column slicing, and matrix-vector products. See scipy.sparse.csc_matrix.

Example

A bitmap image having only 2 colors, with one of them dominant (say a file that stores a
handwritten signature) can be encoded as a sparse matrix that contains only row and
column numbers for pixels with the non-dominant color.

Band matrix

DATA STRUCTURES AND COMPUTER ALGORITHMS

An important special type of sparse matrices is band matrix, defined as follows.
The lower bandwidth of a matrix A is the smallest number p such that the
entry aij vanishes whenever i > j + p. Similarly, the upper bandwidth is the
smallest p such that aij = 0 whenever i < j − p (Golub & Van Loan 1996, §1.2.1). For
example, a tridiagonal matrix has lower bandwidth 1 and upper bandwidth 1. As another
example, the following sparse matrix has lower and upper bandwidth both equal to 3.
Notice that zeros are represented with dots.

Matrices with reasonably small upper and lower bandwidth are known as band matrices
and often lend themselves to simpler algorithms than general sparse matrices; or one can
sometimes apply dense matrix algorithms and gain efficiency simply by looping over a
reduced number of indices.

By rearranging the rows and columns of a matrix A it may be possible to obtain a

matrix A’ with a lower bandwidth. A number of algorithms are designed for bandwidth
minimization.

Diagonal matrix

A very efficient structure for an extreme case of band matrices, the diagonal matrix, is to
store just the entries in the main diagonal as a one-dimensional array, so a
diagonal n×n matrix requires only n entries.
"Fill-in" redirects here. For the puzzle, see Fill-In (puzzle).
The fill-in of a matrix are those entries which change from an initial zero to a non-zero
value during the execution of an algorithm. To reduce the memory requirements and the
number of arithmetic operations used during an algorithm it is useful to minimize the fill-
in by switching rows and columns in the matrix. The symbolic Cholesky
decomposition can be used to calculate the worst possible fill-in before doing the
actual Cholesky decomposition.

There are other methods than the Cholesky decomposition in use. Orthogonalization
methods (such as QR factorization) are common, for example, when solving problems by
least squares methods. While the theoretical fill-in is still the same, in practical terms the

DATA STRUCTURES AND COMPUTER ALGORITHMS

"false non-zeros" can be different for different methods. And symbolic versions of those
algorithms can be used in the same manner as the symbolic Cholesky to compute worst
case fill-in.

Three dimensional array.

An array having more than two dimensions is called a multidimensional array.
Multidimensional arrays are an extension of the normal two-dimensional matrix.
Matrices have two dimensions: the row dimension and the column dimension.

(1,1) (1,2) (1,3) (1,4)

(2,1) (2,.2) (2,3) (2,4)
(3,1) (3,2) (3,3) (3,4)
(4,1) (4,2) (4,3) (4,4)

You can access a two-dimensional matrix element with two subscripts: the first
representing the row index, and the second representing the column index.

Multidimensional arrays use additional subscripts for indexing. A three-dimensional

array, for example, uses three subscripts:

The first references array dimension 1, the row.

The second references dimension 2, the column.
The third references dimension 3. This illustration uses the concept of a page to
represent dimensions 3 and higher.

DATA STRUCTURES AND COMPUTER ALGORITHMS

To access the element in the second row, third column of page 2, for example, you use
the subscripts (2,3,2).

As you add dimensions to an array, you also add subscripts. A four-dimensional array,
for example, has four subscripts. The first two reference a row-column pair; the second
two access the third and fourth dimensions of data.

Most of the operations that you can perform on matrices (i.e., two-dimensional arrays)
can also be done on multidimensional arrays.

DATA STRUCTURES AND COMPUTER ALGORITHMS

To access an entire plane of the image, use

redPlane = RGB(:,:,1);
To access a subimage, use

subimage = RGB(20:40,50:85,:);
The RGB image is a good example of data that needs to be accessed in planes for
operations like display or filtering. In other instances, however, the data itself might be
multidimensional. For example, consider a set of temperature measurements taken at
equally spaced points in a room. Here the location of each value is an integral part of the
data set—the physical placement in three-space of each element is an aspect of the
information. Such data also lends itself to representation as a multidimensional array.

DATA STRUCTURES AND COMPUTER ALGORITHMS

Now to find the average of all the measurements, use

mean(mean(mean(TEMP)));
To obtain a vector of the "middle" values (element (2,2)) in the room on each page, use

B = TEMP (2, 2, :);

Pointer arrays:
Pointer array is use to store address of some memory variables or the address of another
arrays. Address of memory variables or array is known as pointer and an containing
pointers as its elements is known as pointer array.

When setting up data structures like lists, queues and trees, it is necessary to have
pointers to help manage how the structure is implemented and controlled. Typical
examples of pointers are start pointers, end pointers, and stack pointers. These pointers
can either be absolute (the actual physical address or a virtual address in virtual memory)
or relative (an offset from an absolute start address ("base") that typically uses fewer bits
than a full address, but will usually require one additional arithmetic operation to
resolve).

A two-byte offset, containing a 16-bit, unsigned integer, can be used to provide relative
addressing for up to 64 kilobytes of a data structure. This can easily be extended to 128K,
256K or 512K if the address pointed to is forced to be on a half-word, word or double-
word boundary (but, requiring an additional "shift left" bitwise operation—by 1,2 or 3
bits—in order to adjust the offset by a factor of 2,3 or 4, before its addition to the base

DATA STRUCTURES AND COMPUTER ALGORITHMS

address). Generally, though, such schemes are a lot of trouble, and for convenience to the
programmer a flat address space is preferred.

A one byte offset, such as the hexadecimal ASCII value of a character (e.g. X'29') can be
used to point to an alternative integer value (or index) in an array (e.g. X'01'). In this way,
characters can be very efficiently translated from 'raw data' to a usable
sequential index and then to an absolute address without a lookup table.

Name of storage content

Variable address

Fig:
Pointer array

Linked list

In computer science, a linked list is a data structure that consists of a sequence of data
records such that in each record there is a field that contains a reference (i.e., a link) to
the next record in the sequence.

DATA STRUCTURES AND COMPUTER ALGORITHMS

Singly linked list:
Our node data structure will have two fields. We also keep a variable first Node which
always points to the first node in the list, or is null for an empty list.

Algorithm:
record Node {
data; // The data being stored in the node
Node next // A reference to the next node, null for last node
}
record List {
Node firstNode // points to first node of list; null for empty
list
}

Traversal of a singly linked list is simple, beginning at the first node and
following each next link
until we come to the end:

node := list.firstNode

while node not null

(do something with node.data)
node = node.next

The following code inserts a node after an existing node in a singly linked list.
The diagram shows how it works. Inserting a node before an existing one cannot
be done directly; instead, you have to keep track of the previous node and insert a
node after it.

function insertAfter(Node node, Node newNode) // insert newNode after node

newNode.next := node.next

DATA STRUCTURES AND COMPUTER ALGORITHMS

node.next:= newNode

Inserting at the beginning of the list requires a separate function. This requires
updating firstNode.

function insertBeginning(List list, Node newNode) // insert node

before current first node
newNode.next
:= list.firstNode
list.firstNode := newNode

Similarly, we have functions for removing the node after a given node, and for
removing a node from the beginning of the list. The diagram demonstrates the
former. To find and remove a particular node, one must again keep track of the
previous element.

function removeAfter(node node) // remove node past this one

obsoleteNode := node.next
node.next := node.next.next
destroy obsoleteNode
function removeBeginning(List list) // remove first node
obsoleteNode := list.firstNode
list.firstNode := list.firstNode.next // point past deleted node
destroy obsoleteNode

Notice that removeBeginning() sets list.firstNode to null when removing the last
node in the list.Since we can't iterate backwards, efficient "insertBefore" or
"removeBefore" operations are not possible.Appending one linked list to another
can be inefficient unless a reference to the tail is kept as part of the List structure,
because we must traverse the entire first list in order to find the tail, and then

DATA STRUCTURES AND COMPUTER ALGORITHMS

 append the second list to this. Thus, if two linearly linked lists are each of length
n, list appending has asymptotic time complexity of O(n). In the Lisp family of
languages, list appending is provided by the append procedure. Many of the
special cases of linked list operations can be eliminated by including a dummy
element at the front of the list. This ensures that there are no special cases for the
beginning of the list and renders both insert Beginning() and remove Beginning()
unnecessary. In this case, the first useful data in the list will be found at
list.firstNode.next.

A linked list whose nodes contain two fields: an integer value and a link to the next node.

Linked lists are among the simplest and most common data structures; they
provide an easy implementation for several important abstract data structures,
including stacks, queues, associative arrays, and symbolic expressions.
The principal benefit of a linked list over a conventional array is that the order of
the linked items may be different from the order that the data items are stored in
memory or on disk. For that reason, linked lists allow insertion and removal of
nodes at any point in the list, with a constant number of operations.
On the other hand, linked lists by themselves do not allow random access to the
data, or any form of efficient indexing. Thus, many basic operations — such as
obtaining the last node of the list, or finding a node that contains a given datum,
or locating the place where a new node should be inserted — may require
scanning most of the list elements.

Circular linked lists

In the last node of a list, the link field often contains a null reference, a special
value that is interpreted by programs as meaning "there is no such node". A less
common convention is to make it point to the first node of the list; in that case the

DATA STRUCTURES AND COMPUTER ALGORITHMS

 list is said to be circular or circularly linked; otherwise it is said to be open or
linear.
Both types of circularly linked lists benefit from the ability to traverse the full list
beginning at any given node. This often allows us to avoid storing first Node and
last Node, although if the list may be empty we need a special representation for
the empty list, such as a last Node variable which points to some node in the list
or is null if it's empty; we use such a last Node here. This representation
significantly simplifies adding and removing nodes with a non-empty list, but
empty lists are then a special case.
Assuming that some Node is some node in a non-empty circular singly linked list,
this code iterates through that list starting with some Node:

function iterate(someNode)
if someNode ≠ null
node := someNode
do
do something with node.value
node := node.next
while node ≠ someNode

Notice that the test "while node ≠ some Node" must be at the end of the loop. If it
were replaced by the test "" at the beginning of the loop, the procedure would fail
whenever the list had only one node.

This function inserts a node "new Node" into a circular linked list after a given
node "node". If "node" is null, it assumes that the list is empty.

function insertAfter(Node node, Node newNode)

if node = null
newNode.next := newNode
else
DATA STRUCTURES AND COMPUTER ALGORITHMS
newNode.next := node.next
node.next := newNode

Suppose that "L" is a variable pointing to the last node of a circular linked list (or
null if the list is empty). To append "new Node" to the end of the list, one may do

insertAfter(L, newNode)
L := newNode

To insert "newNode" at the beginning of the list, one may do

insertAfter(L, newNode)
if L = null
L := newNode

A circular linked list

Doubly linked list

In computer science, a doubly linked list is a linked data structure that consists of
a set of sequentially linked records called nodes. Each node contains two fields,
called links that are references to the previous and to the next node in the
sequence of nodes. The beginning and ending nodes' previous and next links,
respectively, point to some kind of terminator, typically a sentinel node or null, to
facilitate traversal of the list. If there is only one sentinel node, then the list is
circularly linked via the sentinel node. It can be conceptualized as two singly
linked lists formed from the same data items, but in opposite sequential orders.

DATA STRUCTURES AND COMPUTER ALGORITHMS

 A doubly linked list whose nodes contain three fields: an integer value, the link to
the next node, and the link to the previous node. The two node links allow
traversal of the list in either direction. While adding or removing a node in a
doubly linked list requires changing more links than the same operations on a
singly linked list, the operations are simpler and potentially more efficient (for
nodes other than first nodes) because there is no need to keep track of the
previous node during traversal or no need to traverse the list to find the previous
node, so that its link can be modified.
In a doubly-linked list, each node contains, besides the next-node link, a second
link field pointing to the previous node in the sequence. The two links may be
called forward(s) and backwards, or next and prev(ious).
The first and last nodes of a doubly linked list are immediately accessible (i.e.,
accessible without traversal) and therefore allow traversal of the list from the
beginning or end of the list, respectively. e.g., Traversing the list from beginning
to end, or from end to beginning, in a search of the list for a node with specific
data value. Any node of a doubly linked list, once gotten, can be used to begin a
new traversal of the list, in either direction (towards beginning or end), from the
given node.
The link fields of a doubly linked list node are often called next and previous or
forward and backward. The references stored in the link fields are usually
implemented by pointers; But (as in any linked data structure) they may also be
address offsets or indices into an array where the nodes live.

record DoublyLinkedNode {
prev // A reference to the previous node
next // A reference to the next node
data // Data or reference to data
}
record DoublyLinkedList {
Node firstNode
DATA STRUCTURES AND COMPUTER ALGORITHMS
// points to first node of list
Node lastNode
// points to last node of list
}

Traversing the list

Traversal of a doubly linked list can be in either direction. In fact, the direction of
traversal can change many times, if desired. Traversal is often called iteration, but
that choice of terminology is unfortunate, for iteration has well-defined semantics
(e.g., in mathematics) which are not analogous to traversal.
Forwards
node := list.firstNode
while node ≠ null
<do something with node.data>
node := node.next
Backwards
node := list.lastNode
while node ≠ null
<do something with node.data>
node := node.prev
Inserting a node

These symmetric functions insert a node either after or before a given node, with the
diagram
demonstrating after:

function insertAfter(List list, Node node, Node newNode)

newNode.prev := node
newNode.next := node.next
if node.next == null
list.lastNode := newNode
DATA STRUCTURES AND COMPUTER ALGORITHMS
else
node.next.prev := newNode
node.next := newNode
function insertBefore(List list, Node node, Node newNode)
newNode.prev := node.prev
newNode.next := node
if node.prev == null
list.firstNode := newNode
else
node.prev.next := newNode
node.prev
:= newNode

We also need a function to insert a node at the beginning of a possibly empty list:

function insertBeginning(List list, Node newNode)

if list.firstNode == null
list.firstNode := newNode
list.lastNode := newNode
newNode.prev := null
newNode.next := null
else
insertBefore(list, list.firstNode, newNode)

A symmetric function inserts at the end:

function insertEnd(List list, Node newNode)

if list.lastNode == null
insertBeginning(list, newNode)
else
insertAfter(list, list.lastNode, newNode)

DATA STRUCTURES AND COMPUTER ALGORITHMS

 Removing a node

Removal of a node is easier than insertion, but requires special handling if the node to be
removed is the firstNode or lastNode:

function remove(List list, Node node)

if node.prev == null
list.firstNode := node.next
else
node.prev.next := node.next
if node.next == null
list.lastNode := node.prev
else
node.next.prev := node.prev
destroy node

One subtle consequence of the above procedure is that deleting the last node of a
list sets both firstNode and lastNode to null, and so it handles removing the last
node from a one-element list correctly. Notice that we also don't need separate
"removeBefore" or "removeAfter" methods, because in a doubly linked list we
can just use "remove(node.prev)" or "remove(node.next)" where these are valid.

A doubly-linked list whose nodes contain three fields: an integer value, the link
forward to the next node, and the link backward to the previous node.The technique
known as XOR-linking allows a doubly-linked list to be implemented using a single
link field in each node. However, this technique requires the ability to do bit

DATA STRUCTURES AND COMPUTER ALGORITHMS

 operations on addresses, and therefore may not be available in some high-level
languages.

Circular double linked list:

START Node A Node B

1000 2000 3000 4000 5000

START Node A Node B

1000 2000 3000 4000 5000

3500
Node N
PT
R

that list starting with some Node (any node will do):

Forwards

node := someNode
do
do something with node.value
node := node.next
while node ≠ someNode

Backwards

DATA STRUCTURES AND COMPUTER ALGORITHMS

node := someNode
do
do something with node.value
node := node.prev
while node ≠ someNode

Notice the postponing of the test to the end of the loop. This is important for the case
where the list contains only the single node someNode.

Inserting a node

This simple function inserts a node into a doubly linked circularly linked list after a given
element:
function insertAfter(Node node, Node newNode)
newNode.next := node.next
newNode.prev := node
node.next.prev := newNode
node.next
:= newNode
To do an "insertBefore", we can simply "insertAfter(node.prev, newNode)".
Inserting an element in a possibly empty list requires a special function:
function insertEnd(List list, Node node)
if list.lastNode == null
node.prev := node
node.next := node
else
insertAfter(list.lastNode, node)
list.lastNode := node
To insert at the beginning we simply "insertAfter(list.lastNode, node)". Finally,
removing a node must deal with the case where the list empties:

DATA STRUCTURES AND COMPUTER ALGORITHMS

function remove(List list, Node node)
if node.next == node
list.lastNode := null
else
node.next.prev := node.prev
node.prev.next := node.next
if node == list.lastNode
list.lastNode := node.prev;
destroy node

Application of linked lists:

Inserting an element in a sorted linked list.Let the data be sorted and put in a
singly linked linear list which is being pointed by address ―head ―. Let the new
data to be entered be ―d‖.
Use malloc get a node with address ―pNew‖. Suppose we want to write a code to
enter data ―d‖ into the node and insert it into its proper place in the list.
typedef struct node
{
int data;
struct node *next;
;{
struct node* pNew = (struct node*)
(malloc(sizeof(struct node)));
pNew -> data = d;
pNew ->next = NULL;
pCur = head ;
/* check if data is smaller than smallest item on the list*/
if (pNew -> data < pCur -> data )
}
pNew ->next = pCur ;
head = pNew;

DATA STRUCTURES AND COMPUTER ALGORITHMS

{
/* now examine the remaining list */p = pCur -> next ;
while(p!=NULL ||p->data < pNew->data )
}
pCur = pCur -> next ;
p = p -> next ;
{
pNew -> next = pCur -> next;
pCur -> next = pNew ;Implementation of stacks and queues
using linked lists.

Stacks and Queues are easier to implement using linked

lists. There is no need to set a limit on the size of the stack.
Let us develop push, pop and enqueue and dequeue
functions for nodes containing integer data.
typedef struct node {
int data;
struct node *next;
;{
The Push function places the data in a new node, attaches
that node to the front of stacktop.
void push(struct node**stacktop,int d )
}
struct node* pNew = (struct node*)
(malloc(sizeof(struct node)));
pNew-> data = d ;
pNew->next = *stacktop;
*stacktop = pNew ;
{
Pop Function returns the data from top element, and
frees that element from the stack.

DATA STRUCTURES AND COMPUTER ALGORITHMS

int pop(struct node* *stacktop)
}
struct node* temp;
if(*stacktop== NULL)
}
printf(―\nstack empty\n‖);
{
else
}
temp = *stacktop;
d = temp->data;
*stacktop = temp->next ;
free (temp);
{
return d;
{
For Queue operations, we can maintain two pointers –qfront and qback as we had done
for the case of array implementation of queues.For the Enqueue operation, the data is first
loaded on a new node. If the queue is empty, then after insertion of the first node, both
qfront and qback are made to point to this node, otherwise, the new node is simply
appended and qback
updated.void enquire
(struct node**qfront,int d
, struct node**qback)
}
struct node* pNew = (struct node*)
(malloc(sizeof(struct node)));
pNew-> data = d ;
pNew->next = NULL;
if (*qfront ==NULL && *qback == NULL)
}

DATA STRUCTURES AND COMPUTER ALGORITHMS

 *qfront = pNew;
*qback = pNew;
{
else
}
*qback->next = pNew;
*qbac.

Memory representation:

MEMORY REPRESENTATION OF 2D ARRAY

Generally there are two ways to represent 2D arrays in memory
ROW- MAJOR ORDER :- row by row . COLUMN - MAJOR ORDER :- column
by column . It‘s the work of programming language that in which way array will be
stored in memory .
(0.0) (0,1) (0,2) (0,3) (1,0) (1,1) (1,2) (1,3) (2,0) (2,1) (2,2) (2,3)

ROW 1 ROW 2 ROW 3 ROW-MAJOR ORDER

(0,0) (1,0) (2,0) (0,1) (1,1) (2,1) (0,2) (1,2) (2,2) (0,3) (1,3)
(2,3)

COL 1 COL 2 COL 3 COL 4 COL-MAJOR ORDER

A Simple memory representation of a 3 × 4 array. How to calculate the address of a
particular element where the base address is only known ?? Suppose there is a M × N
array (ARR) is given whose base address is given by BASE . Find out the address of
ARR [i] [j] ?
Follow the following formula
• If the array is in row major order then
&ARR[i] [j] = BASE + size * ( N * i + j)
• If the array is in column major order then

DATA STRUCTURES AND COMPUTER ALGORITHMS

&ARR[i] [j] = BASE + size * ( M * j + i)
Similarly you can declare more than two dimension array such as 3D, 4D , 5D
etc……How to know the address of a given element in nth dimension array ?? Suppose
an array is given ARR[r1] [r2] [r3] ……[rn] and you have to find the address of ARR[i1]
[i2] [i3] …[in]
Where the base address is given by BASE BASE + size ( ( i1 * r2 * r3…rn ) + ( i2 * r3
* r4…rn ) + ( i(n-1) * rn) + in ) Two dimension array as matrix .
As 2D array is used as matrix we can do some operations on matrices like
• Matrix addition
• Matrix subtraction
• Matrix multiplication
• Transpose of a matrix
• To know whether a matrix is symmetry or not
• Check whether matrix is identity(unit) matrix or not
Etc…
Matrix Addition:
It is the operation that adds two matrices and stores the result in the 3rd
matrix. This operation is only possible when the order of the matrix is same and the
order of the resultant matrix will be same as the given matrix. Now suppose A and B be
two matrices of order R1 X C1 and R2 X C2 respectivily. So for addition The order of
the matices must be equal. That means R1 = R2 and C1 = C2. The sum of A and B,
written as A+B is the R1 X C1 matrix C whose ijth element is given by Cij = Aij + Bij
for each value of i and j.

Algorithm for addition of two matrices:

ADDITION(A, R1, C1, B, R2, C2, C, R3, C3)
Let A and B be two matrices of order R1 X C1 and R2 X C2. This algorithm adds
two matrices A and B and stores the result in C.
Step1: (initialize loop counter)
Set i = 1 and j = 1
Step2: (Check for the validity)

DATA STRUCTURES AND COMPUTER ALGORITHMS

if R1!= R2 and C1 != C2 then,
Print ―addition is not possible‖ and exit

Step2: Repeat steps 3 to 6 while I <= R1

Step3: repeat steps 4 and 5 while j <= C1
Step4 set C[i][j] = A[i][j] + B[i][j]
Step5 (increment inner loop counter)
Set j = j + 1.
End of step3 inner loop
Step6: (increment the outer loop counter )
Set i = i + 1.
End of step2 loop. Step7 assign order to matrix C
Set R3 = R1 and C3 = C1.
Matrix Multiplication:
Suppose A and B are two matrices having order R1XC1 and R2XC2 respectively. Then
multiplication between two matrices is possible only if C1 = R2 and the product of A and
B, written AB, is the R1XC2 matrix C whose ijth element Cij is given by
K=C1
Cij
= ∑ AikBkj
K =1

Algorithm for matrix multiplication:

MATRIX MULTIPLICATION( A, R1, C1, B, R2, C2, C, R3, C3)
Let A and B are two matrices of order R1 X C1 and R2 X C2 respectively. This
algorithm
stores the product of A and B in C.
Step1: (initialize loop counter)
I = j = 1.
Step2: if C1 != R2
then,

DATA STRUCTURES AND COMPUTER ALGORITHMS

 Print ―multiplication is not possible ‖ and exit
Step3: Repeat steps 4 to 9 while i <= R1
Step4: Repeat steps 5to 9 while j <= C2
Step5: set C[i][j] = 1 (initialize C[i][j])
Step6: set k = 1 (initialize loop counter k)
Step7: Repeat steps 8 and 9 while k <= C1
Step8: C[i][j] = C[i][j] + A[i][k]*B[k][j]
Step9: set k = k + 1 (incremen k)
End of step5 loop
End of step3 loop
End of step2 loop
Step10: Set R3 = R1 and C3 = C2
Step11: Exit.

DATA STRUCTURES AND COMPUTER ALGORITHMS

DATA STRUCTURES AND COMPUTER ALGORITHMS
 UNIT - II
Stack
Definition:

In computer science, a stack is a last in, first out (LIFO) abstract data type and data
structure. A stack can have any abstract data type as an element, but is characterized by
only three fundamental operations: push, pop and stack top.

The push operation adds a new item to the top of the stack, or initializing the stack if it is
empty, but if the stack is full and does not contain more space to accept the given item it
is considered as an Overflow state (It means that the stack is overloaded or no more space
for new item).

The pop operation removes an item from the top of the stack, A pop either reveals
previously concealed items, or results in an empty stack, but if the stack is empty then it
goes under underflow state (It means no items are present in stack to be removed).

The stack top operation removes the data from top most position without deleting it and
returns it to user, the same underflow state can also occur in stack top operation if stack is
empty.

A stack is a restricted data structure, because only a small number of operations are
performed on it. The nature of the pop and push operations also mean that stack elements
have a natural order. Elements are removed from the stack in the reverse order to the
order of their addition: therefore, the lower elements are those that have been on the stack
the longest.

Stack representation

DATA STRUCTURES AND COMPUTER ALGORITHMS

Stack representation in data structure implementation the array and linked list in various
methods. The implementation of array is

Array implementation.

Representing stacks with arrays is a natural idea. The first problem that you might
encounter is implementing the constructor ArrayStackOfStrings(). An instance
variable a[] with an array of strings to hold the stack items is clearly needed, but how big
should it be? For the moment, we will finesse this problem by having the client provide
an argument for the constructor that gives the maximum stack size. We keep the items
in reverse order of their arrival. This policy allows us to add and remove items at the end
without moving any of the other items in the stack.

We could hardly hope for a simpler implementation of Array Stack Of Strings ava: all of
the methods are one-liners! The instance variables are an array a[] that hold the items in
the stack and an integer N that counts the number of items in the stack. To remove an
item, we decrement N and then return a[N]; to insert a new item, we set a[N] equal to the
new item and then increment N. These operations preserve the following properties: the
items in the array are in their insertion order the stack is empty when the value of N is 0
the top of the stack (if it is nonempty) is at a[N-1] Infix to Postfix Conversion

DATA STRUCTURES AND COMPUTER ALGORITHMS

The primary characteristic of this implementation is that the push and pop operations take
constant time. The drawback of this implementation is that it requires the client to
estimate the maximum size of the stack ahead of time and always uses space proportional
to that maximum, which may be unreasonable in some situations.

Linked lists.

For classes such as stacks that implement collections of objects, an important objective is
to ensure that the amount of space used is always proportional to the number of items in
the collection. Now we consider the use of a fundamental data structure known as
a linked list that can provide implementations of collections (and, in particular, stacks)
that achieves this important objective.

A linked list is a recursive data structure defined as follows: a linked list is either empty
(null) or a reference to a node having a reference to a linked list. The node in this
definition is an abstract entity that might hold any kind of data in addition to the node
reference that characterizes its role in building linked lists. With object-oriented
programming, implementing linked lists is not difficult. We start with a simple example
of a class for the node abstraction:

DATA STRUCTURES AND COMPUTER ALGORITHMS

Class Node

String item;

Node next;

A Node has two instance variables: a String and a Node. The String is a placeholder in
this example for any data that we might want to structure with a linked list (we can use
any set of instance variables); the instance variable of type Node characterizes the linked
nature of the data structure. Now, from the recursive definition, we can represent a linked
list by a variable of type Node just by ensuring that its value is either null or a reference
to a Node whose next field is a reference to a linked list.
We create an object of
type Node by invoking its
(no-argument) constructor.
This creates a reference to
a Node object whose instance
variables are both
initialized to the
value null. For example, to
build a linked list that
contains the
items "to", "be", and "or", we
create a Node for each
item:Node first = new Node();
Node second = new Node();
Node third = new Node();

and set the item field in each of the nodes to the desired item value:
first.item = "to";

DATA STRUCTURES AND COMPUTER ALGORITHMS

second.item = "be";
third.item = "or";
and set the next fields to build the linked list:
first.next = second;
second.next = third;
third.next = null;
When tracing code that uses linked lists and other linked structures, we use a visual
representation of the changes where we draw a rectangle to represent each object we put
the values of instance variables within the rectangle we depict references as arrows that
point to the referenced object This visual representation captures the essential
characteristic of linked lists and allows us to focus on the links.

Insert. Suppose that you want to insert a new node into a linked list. The easiest
place to do so is at the beginning of the list. For example, to insert the
string "not" at the beginning of a given linked list whose first node is first, we
save first in oldfirst, assign to first a new Node, assign itsitem field
to "not" and its next field to oldfirst.

DATA STRUCTURES AND COMPUTER ALGORITHMS

 Remove. Suppose that you want to remove the first node from a list. This
operation is even easier: simply assign to first the value first.next. Normally,
you would retrieve the value of the item(by assigning it to some String variable)
before doing this assignment, because once you change the value of first, you
may not have any access to the node to which it was referring. Typically, the node
object becomes an orphan, and the memory it occupies is eventually reclaimed by
the Java memory management system.

These two operations take constant time (independent of the length of the list).

Implementing stacks with linked lists.

Program LinkedStackOfStrings.java uses a linked list to implement a stack of strings.

The implementation is based on a nested class Node like the one we have been using.
Java allows us to define and use other classes within class implementations in this natural
way. The class is private because clients do not need to know any of the details of the
linked lists.

DATA STRUCTURES AND COMPUTER ALGORITHMS

 Stack operations:

There are two operations which we use in stack to add or delete an element for the
stack. The operations are listed below.

1. PUSH ():- This function is used to add an element into the Stack.
2. POP ():- This function is used to delete an element from the Stack.

DATA STRUCTURES AND COMPUTER ALGORITHMS

Other than this, in stack we use a pointer called TOP which gets incremented or
decremented when we add/remove and element from the stack
PUSH ():
This function will add elements/items into the stack. Whenever we add an
element into the stack the pointer TOP gets incremented as
TOP+1 or TOP++
This is used to show the position of the element in the stack
This is an example of Stack using Arrays.
Example:
Push (5)
In Stack
Array position
Element
Pointer
1
5
Top

Push (6)

In Stack
Array position
Element
Pointer
2
6
Top
1
5
Now the Pointer TOP gets incremented as (TOP+1) and moved to 2nd position in the
element which points the element 6 (NEWLY ADDED ELEMENT).

DATA STRUCTURES AND COMPUTER ALGORITHMS

POP ():
This function will delete elements/items from the stack. Whenever we add an
element into the stack the pointer TOP gets decremented as
TOP-1 or TOP--
This is used to show the position of the element in the stack

Example:
Assume the stack is,
Array position
Element
Pointer
2
6
Top
1
5
Now the pointer is pointed in the element 6 that is, it shows the 2nd position of the array
were the element is available.
POP (6)
Array position
Element
Pointer
2
1
5
Top
Now the pointer TOP gets decremented (TOP-1or TOP--) and move to the position 1.

E.g. Initialize(S) creates a new empty stack named S Push

e.g. Push(X, S) adds the value X to the TOP of stack S
Pop

DATA STRUCTURES AND COMPUTER ALGORITHMS

e.g. Pop(S) removes the TOP node and returns its value

Example

B B B

A A A A

s.push(‗A‘); s.push(‗B‘); s.push(‗C‘);

s.pop();

returns C

B B

A A A

s.push(‗F‘); s.pop();

returns F

s.pop();

returns B

s.pop();

returns A
DATA STRUCTURES AND COMPUTER ALGORITHMS
 Example program in c

#include<stdio.h>

int main()
{
int a[100], i;
printf("To pop enter -1\n");
for(i = 0;;)
{
printf("Push ");
scanf("%d", &a[i]);
if(a[i] == -1)
{
if(i == 0)
{
printf("Underflow\n");
}
else
{
printf("pop = %d\n", a[--i]);
}
}
else
{
i++;
}
}
}

Applications of stack

DATA STRUCTURES AND COMPUTER ALGORITHMS

• Stacks are linear lists.All deletions and insertions occur at one end of the stack
known as the TOP.Data going into the stack first, leaves out last.Stacks are also
known as LIFO data structures (Last-In, First-Out).
• Stacks structures are usually implemented using arrays or linked lists.For both
implementations, the running time is O(n).We will be examining common Stack
Applications.
• Reversing Data: We can use stacks to reverse data. (example: files, strings)

Very useful for finding palindromes. Consider the following pseudocode:

1) read (data)

2) loop (data not EOF and stack not full)

1) push (data)

2) read (data)

3) Loop (while stack notEmpty)

1) pop (data)

2) print (data)

• Converting Decimal to Binary: Consider the following pseudo code

1) Read (number)
2) Loop (number > 0)

1) digit = number modulo 2

2) print (digit)

3) number = number / 2

DATA STRUCTURES AND COMPUTER ALGORITHMS

 The problem with this code is that it will print the binary number backwards. (ex: 19
becomes 11001000 instead of 00010011. )To remedy this problem, instead of printing
the digit right away, we can push it onto the stack. Then after the number is done
being converted, we pop the digit out of the stack and print it.

• Postponement: Evaluating arithmetic expressions.

• Prefix: + a b
• Infix: a + b (what we use in grammar school)Postfix: a b +

• In high level languages, infix notation cannot be used to evaluate expressions. We

must analyze the expression to determine the order in which we evaluate it. A
common technique is to convert a infix notation into postfix notation, then
evaluating it.

Rules:

• Operands immediately go directly to output Operators are pushed into the stack
(including parenthesis) Check to see if stack top operator is less than current
operator
• If the top operator is less than, push the current operator onto stack If the top
operator is greater than the current, pop top operator and push onto stack, push
current perator onto stack
• Priority 2: * /
• Priority 1: + -
• Priority 0: (
• If we encounter a right parenthesis, pop from stack until we get matching left
parenthesis. Do not output parenthesis.Example of infix and postfix
• A+B*C-D/E

Infix Stack(bot->top) Postfix

DATA STRUCTURES AND COMPUTER ALGORITHMS

 a) A + B * C - D / E

b) + B * C - D / E A

c) B*C-D/E + A

d) *C-D/E + AB

e) C-D/E +* AB

f) -D/E +* ABC

g) D/E +- ABC*

h) /E +- ABC*D

i) E +-/ ABC*D

j) +-/ ABC*DE

k) ABC*DE/-+

Operand: push

Operator: pop 2 operands, do the math, pop result back onto stack

123+*

Postfix Stack( bot -> top )

123+*

23+* 1

3+* 12

+* 123
DATA STRUCTURES AND COMPUTER ALGORITHMS
 * 1 5 // 5 from 2 + 3

5 // 5 from 1 * 5

Back tracking

• Stacks can be used to backtrack to achieve certain goals.

• Usually, we set up backtrack tokens to indicate a backtrack opportunity.

Queues

Definition

A queue supports the insert and removes operations using a FIFO discipline. By
convention, we name the queue insert operation enqueue and the remove
operation dequeues.

public class Queue<Item> {

public boolean isEmpty();
public void enqueue(Item
item);
public Item dequeue();
}

Representation of queue

Linked list implementation. Program Queue.java implements a FIFO queue of

strings using a linked list. Like Stack, we maintain a reference first to the least-
recently added Node on the queue. For efficiency, we also maintain a
reference last to the least-recently added Node on the queue.

DATA STRUCTURES AND COMPUTER ALGORITHMS

 Array implementation. Similar to array implementation of stack, but a little
trickier since need to wrap-around. Program DoublingQueue.java implements the
queue interface. The array is dynamically resized using repeated doubling.

Iteration.

Sometimes the client needs to access all of the items of a collection, one at a time,
without deleting them. To maintain encapsulation, we do not want to reveal the internal
representation of the queue (array or linked list) to the client. "Decouple the thing that
needs to traverse the list from the details of getting each element from it." We solve this
design challenge by using Java'sjava.util.Iterator interface:
public interface Iterator<Item> {
boolean hasNext();
Item next();
void remove(); // optional
}

DATA STRUCTURES AND COMPUTER ALGORITHMS

That is, any data type that implements the Iterator interface promises to implement two
methods:hasNext() and next(). The client uses these methods to access the list
elements one a time using the following idiom.
Queue<String> queue = new Queue<String>();
...
Iterator<String> i = queue.iterator();
while (i.hasNext()) {
String s = i.next();
StdOut.println(s);
}

Queue iterator in Java. Queue.java illustrates how to implement

an Iterator when the items are stored in a linked list.

public Iterator iterator() { return new QueueIterator(); }

private class QueueIterator implements Iterator<Item> {

Node current = first;

public boolean hasNext() { return current != null; }

public Item next() {

Item item = current.item;
current = current.next;
return item;
}
}
It relies on a private nested subclass QueueIterator that implements
the Iterator interface. The method iterator() creates an instance of
type QueueIterator and returns it as an Iterator. This enforces the iteration
abstraction since the client will only the items through
the hasNext()and next() methods. The client has no access to the internals of
DATA STRUCTURES AND COMPUTER ALGORITHMS
the Queue or even theQueueIterator. It is the client's responsibility to only add
elements to the list when no iterator is in action.

Enhanced for loop. Iteration is such a useful abstraction that Java provides
compact syntax (known as the enhanced for loop) to iterate over the elements of a
collection (or array).

Iterator<String> i = queue.iterator();
while (i.hasNext()) {
String s = i.next();
StdOut.println(s);
}

for (String s : queue)

StdOut.println(s);

To take advantage of Java's enhanced foreach syntax, the data type must
implement Java'sIterable interface.
public interface Iterable<Item> {
Iterator<Item> iterator();
}
That is, the data type must implement a method named iterator() that returns
an Iterator to the underlying collection. Since our Queue ADT now includes
such a method, we simply need to declare it as implementing
the Iterable interface and we are ready to use the foreach notation.

public class Queue<Item> implements Iterable<Item>

DATA STRUCTURES AND COMPUTER ALGORITHMS

Various queue data structure

Storing a queue in a static data structure

This implementation stores the queue in an array. The array indices at which the head

and tail of the queue are currently stored must be maintained. The head of the queue is

not necessarily at index 0. The array can be a ―circular array‖ – the queue ―wraps round‖

if the last index of the array is reached.

Example – storing a queue in an array of length 5

Add(A,Q) A Head: 0 Tail: 0

Add(D,Q) A D Head: 0 Tail: 1
Add(Z,Q) A D Z Head: 0 Tail: 2
RemoveQ) D Z Head: 1 Tail: 2
Add(X,Q) D Z X Head: 1 Tail: 3
Add(C,Q) D Z X Head: 1 Tail: 4
C
Remove(Q) Z X C Head: 2 Tail: 4
Add(F,Q) F Z X C Head: 2 Tail: 0
Remove(Q) F X C Head: 3 Tail: 0

EXERCISE: Queue stored in a static data structure

Continue the above example to show the state of the queue after the following

operations:

Add(E,Q)

Remove(Q)

DATA STRUCTURES AND COMPUTER ALGORITHMS

Add(W,Q)

Add(J,Q)

Add(K,Q)

What happens at the last of these steps?

Storing a queue in a dynamic data structure

As in the case of the stack, each node in a dynamic data structure contains data AND a

reference to the next node.

A queue also needs a reference to the head node AND a reference to the tail node.

The following diagram describes the storage of a queue called Queue. Each node

consists of data (DataItem) and a reference (NextNode).

• The first node is accessed using the name Queue.Head.

• Its data is accessed using Queue.Head.DataItem

• The second node is accessed using Queue.Head.NextNode

• The last node is accessed using Queue.Tail

Queue.Head DataItem NextNode DataItem NextNode Queue.Tail

Adding a node (Add)

The new node is to be added at the tail of the queue. The reference Queue.Tail should

point to the new node, and the NextNode reference of the node previously at the tail of

the queue should point to the DataItem of the new node.

DATA STRUCTURES AND COMPUTER ALGORITHMS
DataItem NextNode DataItem NextNode

DataItem NextNode

DataItem NextNode DataItem NextNode

DataItem NextNode

Queue.Head Queue.Tail

Queue.Head Queue.Tail

NewNode

NewNode Software Development 2 Bell College

Removing a node (Remove)

The value of Queue.Head.DataItem is returned. A temporary reference Temp is declared

and set to point to head node in the queue (Temp = Queue.Head). Queue.Head is then

set to point to the second node instead of the top node.The only reference to the

original head node is now Temp and the memory used by this node can then be freed.

DataItem NextNode DataItem NextNode

DataItem NextNode DataItem NextNode

Queue.Head Queue.Tail

Queue.Head Queue.Tail

Temp

DATA STRUCTURES AND COMPUTER ALGORITHMS

Queue applications.

queues have numerous useful applications.

Queue applications: Computing applications: serving requests of a single shared

resource (printer, disk, CPU), transferring data asynchronously (data not
necessarily received at same rate as sent) between two processes (IO buffers),
e.g., pipes, file IO, sockets. Buffers on MP3 players and portable CD players,
iPod playlist. Playlist for jukebox - add songs to the end, play from the front of
the list. Interrupt handling: When programming a real-time system that can be
interrupted (e.g., by a mouse click or wireless connection), it is necessary to
attend to the interrupts immediately, before proceeding with the current activity.
If the interrupts should be handles in the same order they arrive, then a FIFO
queue is the appropriate data structure.

Arithmetic expression evaluation. Program Evaluate.java evaluates a fully

parenthesized arithmetic expression.

An important application of stacks is in parsing. For example, a compiler must

parse arithmetic expressions written using infix notation. For example the
following infix expression evaluates to 212.
(2+((3+4)*(5*6)))
We break the problem of parsing infix expressions into two stages. First, we
convert from infix to a different representation called postfix. Then we parse the
postfix expression, which is a somewhat easier problem than directly parsing
infix.

o Evaluating a postfix expression. A postfix expression is....

234+56**+

First, we describe how to parse and evaluate a postfix expression. We read

the tokens in one at a time. If it is an integer, push it on the stack; if it is a
binary operator, pop the top two elements from the stack, apply the

DATA STRUCTURES AND COMPUTER ALGORITHMS

 operator to the two elements, and push the result back on the stack.
Program Postfix.java reads in and evaluates postfix expressions using this
algorithm.

o Converting from infix to postfix. Now, we describe how to convert from

infix to postfix. We read in the tokens one at a time. If it is an operator, we
push it on the stack; if it is an integer, we print it out; if it is a right
parentheses, we pop the topmost element from the stack and print it out; if
it is a left parentheses, we ignore it. Program Infix.java reads in an infix
expression, and uses a stack to output an equivalent postfix expression
using the algorithm described above. Relate back to the parse tree example
in Section 4.3.

Function calls. Perhaps the most important application of stacks is to implement

function calls. Most compilers implement function calls by using a stack. This
also provides a technique for eliminating recursion from a program: instead of
calling a function recursively, the programmer uses a stack to simulate the
function calls in the same way that the compiler would have done so. Conversely,
we can often use recursion instead of using an explicit stack. Some programming
languages provide a mechanism for recursion, but not for calling functions.

Programming languages have built in support for stacks (recursion), but no

analogous mechanism for dealing with queues.

Postscript and FORTH programming languages are stack based. Java bytecode is
interpreted on (virtual) stack based processor. Microsoft Intermediate Language
(MSIL) that .NET applications are compiled to.

M/M/1 queue. The Markov/Markov/Single-Server model is a fundamental

queueing model in operations research and probability theory. Tasks arrive
according to a Poisson process at a certain rate λ. This means that λ customers
arrive per hour. More specifically, the arrivals follow an exponential distribution
with mean 1 / λ: the probability of k arrivals between time 0 and t is (λ t)^k e^(-λ
t) / k!. Tasks are serviced in FIFO order according to a Poisson process with rate

DATA STRUCTURES AND COMPUTER ALGORITHMS

μ. The two M's standard for Markov: it means that the system is memoryless: the
time between arrivals is independent, and the time between departures is
independent.

Analysis of M/M/1 model. We are interested in understanding the queueing

system. If &lambda > μ the queue size increases without limit. For simple models
like M/M/1 we can analyze these quantities analytically using probability theory.
Assuming μ > λ, the probability of exactly n customers in the system is (λ / μ)^n
(1 - λ / &mu).

o L = average number of customers in the system = λ / (μ - λ).

o LQ = average number of customers in the queue = λ2 / (μ (μ - λ)).

o W = average time a customer spends in the system = 1 / (μ - λ).

o WQ = average time a customer spends in the queue = W - 1 / μ.

Program MM1Queue.java For more complex models we need to resort to

simulation like this. Variants: multiple queues, multiple servers, sequential multi-
stage servers, using a finite queue and measuring number of customers that are
turned away. Applications: customers in McDonalds, packets in an internet router,

Little's law asserts that the average number of customers in a (stable) queueing
system equals the average arrival rate times their average time in the system. But
the variance of customer waiting times satisfies: Var(FIFO) < Var(SIRO) <
Var(LIFO).

The distribution of the number of customers in the system does not depend on the
queueing discipline (so long as it is independent of their service times). Same for
expected waiting time.

M/D/1 queue. Program MD1Queue.java is similar but the service occurs at a fixed
rate (rather than random).

Load balancing. Write a program LoadBalance.java that performs a load-

balancing simulation.\

DATA STRUCTURES AND COMPUTER ALGORITHMS

Quick sort

Quicksort is a fast sorting algorithm, which is used not only for educational purposes,
but widely applied in practice. On the average, it has O(n log n) complexity, making
quicksort suitable for sorting big data volumes. The idea of the algorithm is quite
simple and once you realize it, you can write quicksort as fast as bubble sort.

Algorithm

The divide-and-conquer strategy is used in quicksort. Below the recursion step is

described:

1. Choose a pivot value. We take the value of the middle element as pivot value,
but it can be any value, which is in range of sorted values, even if it doesn't
present in the array.
2. Partition. Rearrange elements in such a way, that all elements which are lesser
than the pivot go to the left part of the array and all elements greater than the
pivot, go to the right part of the array. Values equal to the pivot can stay in any
part of the array. Notice, that array may be divided in non-equal parts.
3. Sort both parts. Apply quicksort algorithm recursively to the left and the right
parts.

Partition algorithm in detail

There are two indices i and j and at the very beginning of

the partition algorithm i points to the first element in the array andj points to the last
one. Then algorithm moves i forward, until an element with value greater or equal to
the pivot is found. Index j is moved backward, until an element with value lesser or
equal to the pivot is found. If i ≤ j then they are swapped and i steps to the next
position (i + 1), j steps to the previous one (j - 1). Algorithm stops, when i becomes
greater than j.

DATA STRUCTURES AND COMPUTER ALGORITHMS

After partition, all values before i-th element are less or equal than the pivot and all
values after j-th element are greater or equal to the pivot.

Example. Sort {1, 12, 5, 26, 7, 14, 3, 7, 2} using quicksort.

Notice, that we show here only the first recursion step, in order not to make example
too long. But, in fact, {1, 2, 5, 7, 3} and {14, 7, 26, 12} are sorted then recursively.

Why does it work?

On the partition step algorithm divides the array into two parts and every element a from
the left part is less or equal than every element b from the right part.
Also a and b satisfy a ≤ pivot ≤ b inequality. After completion of the recursion calls both
of the parts become sorted and, taking into account arguments stated above, the whole
array is sorted.

Complexity analysis

On the average quicksort has O(n log n) complexity, but strong proof of this fact is not
trivial and not presented here. Still, you can find the proof in [1]. In worst case,
quicksort runs O(n2) time, but on the most "practical" data it works just fine and
outperforms other O(n log n) sorting algorithms.

Code snippets

Partition algorithm is important per se, therefore it may be carried out as a separate
function. The code for C++ contains solid function for quicksort, but Java code
contains two separate functions for partition and sort, accordingly.

Java

int partition(int arr[], int left, int right)

{
int i = left, j = right;

DATA STRUCTURES AND COMPUTER ALGORITHMS

 int tmp;
int pivot = arr[(left + right) / 2];

while (i <= j) {
while (arr[i] < pivot)
i++;
while (arr[j] > pivot)
j--;
if (i <= j) {
tmp = arr[i];
arr[i] = arr[j];
arr[j] = tmp;
i++;
j--;
}
};

return i;
}

void quickSort(int arr[], int left, int right) {

int index = partition(arr, left, right);
if (left < index - 1)
quickSort(arr, left, index - 1);
if (index < right)
quickSort(arr, index, right);
}

C++

void quickSort(int arr[], int left, int right) {

int i = left, j = right;
DATA STRUCTURES AND COMPUTER ALGORITHMS
 int tmp;
int pivot = arr[(left + right) / 2];

/* partition */
while (i <= j) {
while (arr[i] < pivot)
i++;
while (arr[j] > pivot)
j--;
if (i <= j) {
tmp = arr[i];
arr[i] = arr[j];
arr[j] = tmp;
i++;
j--;
}
};

/* recursion */
if (left < j)
quickSort(arr, left, j);
if (i < right)
quickSort(arr, i, right);
}
Trees:
Definition

In computer science, a tree is a widely-used data structure that emulates a

hierarchical tree structure with a set of linked nodes.Mathematically, it is an ordered
directed tree, more specifically an arborescence: an acyclic connected graph where each
node has zero or more children nodes and at most one parentnode. Furthermore, the
children of each node have a specific order.
DATA STRUCTURES AND COMPUTER ALGORITHMS
• A node is a structure which may contain a value, a condition, or represent a
separate data structure (which could be a tree of its own). Each node in a tree has
zero or more child nodes, which are below it in the tree (by convention, trees are
drawn growing downwards). A node that has a child is called the child's parent
node (or ancestor node, or superior). A node has at most one parent.

• Nodes that do not have any children are called leaf nodes. They are also referred
to as terminal nodes.A free tree is a tree that is not rooted.The height of a node is
the length of the longest downward path to a leaf from that node. The height of
the root is the height of the tree. The depth of a node is the length of the path to its
root (i.e., its root path). This is commonly needed in the manipulation of the
various self balancing trees, AVL Trees in particular. Conventionally, the value -1
corresponds to a subtree with no nodes, whereas zero corresponds to a subtree
with one node.

• The topmost node in a tree is called the root node. Being the topmost node, the
root node will not have parents. It is the node at which operations on the tree
commonly begin (although some algorithms begin with the leaf nodes and work
up ending at the root). All other nodes can be reached from it by
following edges or links. (In the formal definition, each such path is also unique).
In diagrams, it is typically drawn at the top. In some trees, such as heaps, the root
node has special properties. Every node in a tree can be seen as the root node of
the subtree rooted at that node.

DATA STRUCTURES AND COMPUTER ALGORITHMS

 • An internal node or inner node is any node of a tree that has child nodes and is
thus not a leaf node. Similarly, an external node or outer node is any node that
does not have child nodes and is thus a leaf.

• A subtree of a tree T is a tree consisting of a node in T and all of its descendants

in T. (This is different from the formal definition of subtree used in graph
theory.[1]) The subtree corresponding to the root node is the entire tree; the subtree
corresponding to any other node is called a proper subtree (in analogy to the
term proper subset).

Representation of trees

A node is a structure which may contain a value, a condition, or represent a separate data
structure (which could be a tree of its own). Each node in a tree has zero or more child
nodes, which are below it in the tree (by convention, trees are drawn growing
downwards). A node that has a child is called the child's parent node (or ancestor node,
or superior). A node has at most one parent.

Nodes that do not have any children are called leaf nodes. They are also referred to as
terminal nodes.A free tree is a tree that is not rooted. The height of a node is the length of
the longest downward path to a leaf from that node. The height of the root is the height of
the tree. The depth of a node is the length of the path to its root (i.e., its root path). This is
commonly needed in the manipulation of the various self balancing trees, AVL Trees in
particular. Conventionally, the value -1 corresponds to a sub tree with no nodes, whereas
zero corresponds to a sub tree with one node.

The topmost node in a tree is called the root node. Being the topmost node, the root node
will not have parents. It is the node at which operations on the tree commonly begin
(although some algorithms begin with the leaf nodes and work up ending at the root). All
other nodes can be reached from it by following edges or links. (In the formal definition,
each such path is also unique). In diagrams, it is typically drawn at the top. In some trees,
such as heaps, the root node has special properties. Every node in a tree can be seen as
the root node of the sub tree rooted at that node.

DATA STRUCTURES AND COMPUTER ALGORITHMS

An internal node or inner node is any node of a tree that has child nodes and is thus not
a leaf node. Similarly, an external node or outer node is any node that does not have child
nodes and is thus a leaf.

A sub tree of a tree T is a tree consisting of a node in T and all of its descendants in T.
(This is different from the formal definition of sub tree used in graph theory. The sub tree
corresponding to the root node is the entire tree; the sub tree corresponding to any other
node is called a proper sub tree (in analogy to the term proper subset).

A binary tree is a tree data structure in which each node has at most two child nodes,
usually distinguished as "left" and "right". Nodes with children are parent nodes, and
child nodes may contain references to their parents. Outside the tree, there is often a
reference to the "root" node (the ancestor of all nodes), if it exists. Any node in the data
structure can be reached by starting at root node and repeatedly following references to
either the left or right child. Binary trees are used to implement binary search
trees and binary heaps. Directed edge refers to the link from the parent to the child (the
arrows in the picture of the tree).

There are many different ways to represent trees; common representations represent the
nodes as records allocated on the heap (not to be confused with the heap data structure)
with pointers to their children, their parents, or both, or as items in an array, with
relationships between them determined by their positions in the array (e.g., binary heap)

Heap is a specialized tree-based data structure that satisfies the heap property: if B is
a child node of A, then key (A) ≥ key (B). This implies that an element with the greatest
key is always in the root node, and so such a heap is sometimes called a max-heap.
(Alternatively, if the comparison is reversed, the smallest element is always in the root
node, which results in a min-heap.) There is no restriction as to how many children each
node has in a heap. The heap is one maximally-efficient implementation of an abstract
data type called a priority queue. Heaps are crucial in several
efficient graph algorithms such as Dijkstra's algorithm.

A heap data structure should not be confused with the heap which is a common name for
dynamic allocated memory. The term was originally used only for the data structure.

DATA STRUCTURES AND COMPUTER ALGORITHMS

Some early popular languages such as LISP provided dynamic memory allocation using a
heap data structures, which gave the memory area its name.

Heaps are usually implemented in an array, and do not require pointers between
elements.

The operations commonly performed with a heap are:

 find-max or find-min: find the maximum item of a max-heap or a minimum item

of a min-heap, respectively
 delete-max or delete-min: removing the root node of a max- or min-heap,
respectively
 increase-key or decrease-key: updating a key within a max- or min-heap,
respectively
 insert: adding a new key to the heap
 Merge: joining two heaps to form a valid new heap containing all the elements of
both.

 The root node of a tree is the node with no parents. There is at most one root node
in a rooted tree.
 A leaf node has no children.

DATA STRUCTURES AND COMPUTER ALGORITHMS

 The depth of a node n is the length of the path from the root to the node. The set
of all nodes at a given depth is sometimes called a level of the tree. The root node is
at depth zero.
 The height of a tree is the length of the path from the root to the deepest node in
the tree. A (rooted) tree with only one node (the root) has a height of zero.
 Siblings are nodes that share the same parent node.
 A node p is an ancestor of a node q if it exists on the path from q to the root. The
node q is then termed a descendant of p.
 The size of a node is the number of descendants it has including itself.
 In-degree of a node is the number of edges arriving at that node.
 Out-degree of a node is the number of edges leaving that node.
 The root is the only node in the tree with In-degree = 0.

e.g.: depth of tree with level =3,then, size of tree is, level+1=4

 A rooted binary tree is a tree with a root node in which every node has at most
two children.
 A full binary tree (sometimes proper binary tree or 2-tree or strictly binary tree) is
a tree in which every node other than the leaves has two children.
 A perfect binary tree is a full binary tree in which all leaves are at the
same depth or same level.[1] (This is ambiguously also called a complete binary tree.)
 A complete binary tree is a binary tree in which every level, except possibly the
last, is completely filled, and all nodes are as far left as possible.[2]
 An infinite complete binary tree is a tree with levels, where for each level d
the number of existing nodes at level d is equal to 2d. The cardinal number of the set
of all nodes is . The cardinal number of the set of all paths is . The infinite
complete binary tree essentially describes the structure of the Cantor set; the unit
interval on the real line (of cardinality ) is the continuous image of the Cantor set;
this tree is sometimes called the Cantor space.
 A balanced binary tree is commonly defined as a binary tree in which the height
of the two sub trees of every node never differ by more than 1.,[3] although in general

DATA STRUCTURES AND COMPUTER ALGORITHMS

 it is a binary tree where no leaf is much farther away from the root than any other
leaf. (Different balancing schemes allow different definitions of "much farther" [4]).
Binary trees that are balanced according to this definition have a predictable depth
(how many nodes are traversed from the root to a leaf, root counting as node 0 and
subsequent as 1, 2, ..., depth). This depth is equal to the integer part
of log2(n) where n is the number of nodes on the balanced tree. Example 1: balanced
tree with 1 node, log2(1) = 0 (depth = 0). Example 2: balanced tree with 3
nodes, log2(3) = 1.59 (depth=1). Example 3: balanced tree with 5 nodes, log2(5) =
2.32 (depth of tree is 2 nodes).
 A rooted complete binary tree can be identified with a free magma.
 A degenerate tree is a tree where for each parent node, there is only one
associated child node. This means that in a performance measurement, the tree will
behave like a linked list data structure.
 A Tango tree is a tree optimized for fast searches.

Note that this terminology often varies in the literature, especially with respect to the
meaning "complete" and "full".

 A Strictly Binary Tree:- Its when the tree is fully expanded i.e., with 2 degree
expansion.

 The number of nodes n in a perfect binary tree can be found using this
formula: n = 2h + 1 − 1 where h is the height of the tree.
 The number of nodes n in a complete binary tree is minimum: n = 2h and
maximum: n = 2h + 1 − 1 where h is the height of the tree.
 The number of leaf nodes L in a perfect binary tree can be found using this
formula: L = 2h where h is the height of the tree.
 The number of nodes n in a perfect binary tree can also be found using this
formula: n = 2L − 1 where L is the number of leaf nodes in the tree.
 The number of NULL links in a Complete Binary Tree of n-node is (n+1).

 The number of leaf nodes in a Complete Binary Tree of n-node is .

DATA STRUCTURES AND COMPUTER ALGORITHMS

 For any non-empty binary tree with n0 leaf nodes and n2 nodes of degree
2, n0 = n2 +

Operations of trees

Here are a variety of different operations that can be performed on trees. Some
are mutator operations, while others simply return useful information about the tree.
• Insertion

Nodes can be inserted into binary trees in between two other nodes or added after
an external node. In binary trees, a node that is inserted is specified as to which child it is.

• External nodes

Say that the external node being added on to is node A. To add a new node after node A,
A assigns the new node as one of its children and the new node assigns node A as its
parent.

The process of inserting a node into a binary tree

Insertion on internal nodes is slightly more complex than on external nodes. Say that the
internal node is node A and that node B is the child of A. (If the insertion is to insert a
right child, then B is the right child of A, and similarly with a left child insertion.) A
assigns its child to the new node and the new node assigns its parent to A. Then the new
node assigns its child to B and B assigns its parent as the new node.

• Deletion

Deletion is the process whereby a node is removed from the tree. Only certain nodes in a
binary tree can be removed unambiguously.

Say that the node to delete is node A. If a node has no children (external node), deletion
is accomplished by setting the child of A's parent to null and A's parent to null. If it has
one child, set the parent of A's child to A's parent and set the child of A's parent to A's
child.

DATA STRUCTURES AND COMPUTER ALGORITHMS

Node with two childrenIn a binary tree, a node with two children cannot be deleted
unambiguously. However, in certain binary trees these nodes canbe deleted,
including binary search trees.

• Iteration

Often, one wishes to visit each of the nodes in a tree and examine the value there, a
process called iteration or enumeration. There are several common orders in which the
nodes can be visited, and each has useful properties that are exploited in algorithms based
on binary trees:

 Pre-Order: Root first, children after

 Post-Order: Children first, root after
 In-Order: Left child, root, right child.
]Pre-order, in-order, and post-order traversal

Main article: Tree traversal

Pre-order, in-order, and post-order traversal visit each node in a tree by recursively
visiting each node in the left and right subtrees of the root. If the root node is visited
before its subtrees, this is pre-order; if after, post-order; if between, in-order. In-order
traversal is useful inbinary search trees, where this traversal visits the nodes in increasing
order.

Depth-first order

In depth-first order, we always attempt to visit the node farthest from the root that we
can, but with the caveat that it must be a child of a node we have already visited. Unlike a
depth-first search on graphs, there is no need to remember all the nodes we have visited,
because a tree cannot contain cycles. Pre-order is a special case of this. See depth-first
search for more information.

Breadth-first order

Contrasting with depth-first order is breadth-first order, which always attempts to visit
the node closest to the root that it has not already visited. See breadth-first search for
more information. Also called a level-order traversal.

DATA STRUCTURES AND COMPUTER ALGORITHMS

In type theory, a binary tree with nodes of type A is defined inductively as TA = μα. 1
+ A × α × α.For each binary tree data structure, there is equivalent rooted binary tree in
graph theory.

Graph theorists use the following definition: A binary tree is a connected acyclic
graph such that the degree of each vertex is no more than three. It can be shown that in
any binary tree of two or more nodes, there are exactly two more nodes of degree one
than there are of degree three, but there can be any number of nodes of degree two.
A rooted binary tree is such a graph that has one of its vertices of degree no more than
two singled out as the root.

With the root thus chosen, each vertex will have a uniquely defined parent, and up to two
children; however, so far there is insufficient information to distinguish a left or right
child. If we drop the connectedness requirement, allowing multiple connected
components in the graph, we call such a structure a forest.

Another way of defining binary trees is a recursive definition on directed graphs. A

binary tree is either:

 A single vertex.
 A graph formed by taking two binary trees, adding a vertex, and adding an edge
directed from the new vertex to the root of each binary tree.

This also does not establish the order of children, but does fix a specific root node.

• In combinatorics one considers the problem of counting the number of binary

trees of a given size. Here the trees have no values attached to their nodes (this
would just multiply the number of possible trees by an easily determined factor),
and trees are distinguished only by their structure; however the left and right child
of any node are distinguished (if they are different trees, then interchanging them
will produce a tree distinct from the original one). The size of the tree is taken to
be the number n of internal nodes (those with two children); the other nodes are
leaf nodes and there are n + 1 of them. The number of such binary trees of
size n is equal to the number of ways of fully parenthesizing a string of n +
1 symbols (representing leaves) separated by n binary operators (representing

DATA STRUCTURES AND COMPUTER ALGORITHMS

 internal nodes), so as to determine the argument subexpressions of each operator.
For instance for n = 3 one has to parenthesize a string like X * X * X * X, which
is possible in five ways:

• The correspondence to binary trees should be obvious, and the addition of

redundant parentheses (around an already parenthesized expression or around the
full expression) is disallowed (or at least not counted as producing a new
possibility).

• There is a unique binary tree of size 0 (consisting of a single leaf), and any other
binary tree is characterized by the pair of its left and right children; if these have
sizes i and j respectively, the full tree has size i + j + 1. Therefore the
number Cn of binary trees of size nhas the following recursive description C0 = 1,

and for any positive integer n. It follows that Cn is

theCatalan number of index n.

• The above parenthesized strings should not be confused with the set of words of
length 2n in the Dyck language, which consist only of parentheses in such a way
that they are properly balanced. The number of such strings satisfies the same
recursive description (each Dyck word of length 2n is determined by the Dyck
subword enclosed by the initial '(' and its matching ')' together with the Dyck
subword remaining after that closing parenthesis, whose lengths 2i and
2j satisfy i + j + 1 = n); this number is therefore also the Catalan number Cn. So
there are also five Dyck words of length 10:

• These Dyck words do not correspond in an obvious way to binary trees. A

bijective correspondence can nevertheless be defined as follows: enclose the Dyck
word in a extra pair of parentheses, so that the result can be interpreted as
a Lisp list expression (with the empty list () as only occurring atom); then
the dotted-pair expression for that proper list is a fully parenthesized expression

DATA STRUCTURES AND COMPUTER ALGORITHMS

 (with NIL as symbol and '.' as operator) describing the corresponding binary tree
(which is in fact the internal representation of the proper list).

• The ability to represent binary trees as strings of symbols and parentheses implies
that binary trees can represent the elements of afree magma on a singleton
set.Binary trees can be constructed from programming language primitives in
several ways.

Nodes and references

• In a language with records and references, binary trees are typically

constructed by having a tree node structure which contains some data
and references to its left child and its right child. Sometimes it also
contains a reference to its unique parent. If a node has fewer than two
children, some of the child pointers may be set to a special null value, or
to a special sentinel node.

• In languages with tagged unions such as ML, a tree node is often a

tagged union of two types of nodes, one of which is a 3-tuple of data,
left child, and right child, and the other of which is a "leaf" node, which
contains no data and functions much like the null value in a language
with pointers.

Arrays

• Binary trees can also be stored as an implicit data structure in arrays,

and if the tree is a complete binary tree, this method wastes no space. In
this compact arrangement, if a node has an index i, its children are
found at indices 2i + 1(for the left child) and 2i + 2(for the right), while

its parent (if any) is found at index (assuming the root has
index zero). This method benefits from more compact storage and
better locality of reference, particularly during a preorder traversal.

DATA STRUCTURES AND COMPUTER ALGORITHMS

 However, it is expensive to grow and wastes space proportional to 2h -
n for a tree of height h with n nodes.

Succinct encodings

• A succinct data structure is one which takes the absolute minimum

possible space, as established by information theoretical lower bounds.
The number of different binary trees on n nodes is Cn, the nth Catalan
number (assuming we view trees with identicalstructure as identical).
For large n, this is about 4n; thus we need at least about log24n = 2n bits
to encode it. A succinct binary tree therefore would occupy only 2 bits
per node.

• One simple representation which meets this bound is to visit the nodes
of the tree in preorder, outputting "1" for an internal node and "0" for a
leaf. [1] If the tree contains data, we can simply simultaneously store it
in a consecutive array in preorder. This function accomplishes this:

function EncodeSuccinct(node n, bitstring structure, array data) {

if n = nil then
append 0 to structure;
else
append 1 to structure;
append n.data to data;
EncodeSuccinct(n.left, structure, data);
EncodeSuccinct(n.right, structure, data);
}

DATA STRUCTURES AND COMPUTER ALGORITHMS

 The string structure has only 2n + 1 bits in the end, where n is the number of
(internal) nodes; we don't even have to store its length. To show that no
information is lost, we can convert the output back to the original tree like this:

function DecodeSuccinct(bitstring structure, array data) {

remove first bit of structure and put it in b
if b = 1 then
create a new node n
remove first element of data and put it in n.data
n.left = DecodeSuccinct(structure, data)
n.right = DecodeSuccinct(structure, data)
return n
else
return nil
}

More sophisticated succinct representations allow not only compact storage of

trees but even useful operations on those trees directly while they're still in their
succinct form.

Encoding general trees as binary trees

• There is a one-to-one mapping between general ordered trees and binary

trees, which in particular is used by Lisp to represent general ordered
trees as binary trees. To convert a general ordered tree to binary tree, we
only need to represent the general tree in left child-sibling way. The
result of this representation will be automatically binary tree, if viewed
from a different perspective. Each node N in the ordered tree
corresponds to a node N' in the binary tree; the left child of N' is the
node corresponding to the first child of N, and the rightchild of N' is the
node corresponding to N 's next sibling --- that is, the next node in order
among the children of the parent of N. This binary tree representation of

DATA STRUCTURES AND COMPUTER ALGORITHMS

 a general order tree is sometimes also referred to as a left child-right
sibling binary tree (LCRS tree), or adoubly chained tree, or a Filial-Heir
chain.

• One way of thinking about this is that each node's children are in
a linked list, chained together with their right fields, and the node only
has a pointer to the beginning or head of this list, through its left field.
For example, in the tree on the left, A has the 6 children
{B,C,D,E,F,G}. It can be converted into the binary tree on the right

• The binary tree can be thought of as the original tree tilted sideways,
with the black left edges representing first child and the blue right edges
representing next sibling. The leaves of the tree on the left would be
written in Lisp as:(((N O) I J) C D ((P) (Q)) F (M))which would be
implemented in memory as the binary tree on the right, without any
letters on those nodes that have a left child.

Types of trees

Tree is a non-empty set, one element of which is designated the root of the tree while the
remaining elements are partitioned into non-empty sets each of which is a subtree of the
root.

Tree nodes have many useful properties. The depth of a node is the length of the path (or
the number of edges) from the root to that node. The height of a node is the longest path
from that node to its leaves. The height of a tree is the height of the root. A leaf nodehas
no children -- its only path is up to its parent.

See the axiomatic development of trees and its consequences for more information.

Types of trees:

Binary: Each node has zero, one, or two children. This assertion makes many tree
operations simple and efficient.

DATA STRUCTURES AND COMPUTER ALGORITHMS

Binary Search: A binary tree where any left child node has a value less than its parent
node and any right child node has a value greater than or equal to that of its parent node.

AVL: A balanced binary search tree according to the following specification: the heights
of the two child subtrees of any node differ by at most one.

Red-Black Tree: A balanced binary search tree using a balancing algorithm based on
colors assigned to a node, and the colors of nearby nodes.

AA Tree: A balanced tree, in fact a more restrictive variation of a red-black tree.

Traversal

Many problems require we visit* the nodes of a tree in a systematic way: tasks such as
counting how many nodes exist or finding the maximum element. Three different
methods are possible for binary trees: preorder, postorder, and in-order, which all do the
same three things: recursively traverse both the left and right subtrees and visit the
current node. The difference is when the algorithm visits the current node:

preorder: Current node, left subtree, right subtree(DLR)

postorder: Left subtree, right subtree, current node(LRD)

in-order: Left subtree, current node, right subtree.(LDR)

levelorder: Level by level, from left to right, starting from the root node.

 Visit means performing some operation involving the current node of a tree, like
incrementing a counter or checking if the value of the current node is greater than any
other recorded.

Sample implementations for Tree Traversal

preorder(node)
visit(node)
if node.left ≠ null then preorder(node.left)

DATA STRUCTURES AND COMPUTER ALGORITHMS

 if node.right ≠ null then preorder(node.right)
inorder(node)
if node.left ≠ null then inorder(node.left)
visit(node)
if node.right ≠ null then inorder(node.right)
postorder(node)
if node.left ≠ null then postorder(node.left)
if node.right ≠ null then postorder(node.right)
visit(node)
levelorder(root)
queue<node> q
q.push(root)
while not q.empty do
node = q.pop
visit(node)
if node.left ≠ null then q.push(node.left)
if node.right ≠ null then q.push(node.right)

For an algorithm that is less taxing on the stack, see Threaded Trees.

Examples of Tree Traversals

preorder: 50, 30, 20, 40, 90, 100

inorder: 20, 30, 40, 50, 90, 100

DATA STRUCTURES AND COMPUTER ALGORITHMS

postorder: 20, 40, 30, 100, 90, 50
levelorder: 50, 30, 90, 20, 40, 100

Balancing

When entries that are already sorted are stored in a tree, all new records will go the same
route, and the tree will look more like a list (such a tree is called a degenerate tree).
Therefore the tree needs balancing routines, making sure that under all branches are an
equal number of records. This will keep searching in the tree at optimal speed.
Specifically, if a tree with n nodes is a degenerate tree, the longest path through the tree
will be n nodes; if it is a balanced tree, the longest path will be log n nodes.

Algorithms/Left_rotation This shows how balancing is applied to establish a priority heap

invariant in a Treap , a data structure which has the queueing performance of a heap, and
the key lookup performance of a tree. A balancing operation can change the tree structure
while maintaining another order, which is binary tree sort order. The binary tree order is
left to right, with left nodes' keys less than right nodes' keys, whereas the priority order is
up and down, with higher nodes' priorities greater than lower nodes' priorities.
Alternatively, the priority can be viewed as another ordering key, except that finding a
specific key is more involved.

The balancing operation can move nodes up and down a tree without affecting the left
right ordering. Media:left_rotation.svg

Binary Search Trees

A typical binary search tree looks like this:

DATA STRUCTURES AND COMPUTER ALGORITHMS

Terms

Node Any item that is stored in the tree. Root The top item in the tree. (50 in the tree
above) Child Node(s) under the current node. (20 and 40 are children of 30 in the tree
above) Parent The node directly above the current node. (90 is the parent of 100 in the
tree above) Leaf A node which has no children. (20 is a leaf in the tree above)

Searching through a binary search tree

To search for an item in a binary tree:

1. Start at the root node

2. If the item that you are searching for is less than the root node, move to the left
child of the root node, if the item that you are searching for is more than the root
node, move to the right child of the root node and if it is equal to the root node,
then you have found the item that you are looking for.
3. Now check to see if the item that you are searching for is equal to, less than or
more than the new node that you are on. Again if the item that you are searching
for is less than the current node, move to the left child, and if the item that you
are searching for is greater than the current node, move to the right child.
4. Repeat this process until you find the item that you are looking for or until the
node doesn't have a child on the correct branch, in which case the tree doesn't
contain the item which you are looking for.
Example

DATA STRUCTURES AND COMPUTER ALGORITHMS

For example, to find the node 40...

1. The root node is 50, which is greater than 40, so you go to 50's left child.
2. 50's left child is 30, which is less than 40, so you next go to 30's right child.
3. 30's right child is 40, so you have found the item that you are looking for :)
Adding an item to a binary search tree

1. To add an item, you first must search through the tree to find the position that you
should put it in. You do this following the steps above.
2. When you reach a node which doesn't contain a child on the correct branch, add
the new node there.

For example, to add the node 25...

1. The root node is 50, which is greater than 25, so you go to 50's left child.
2. 50's left child is 30, which is greater than 25, so you go to 30's left child.
3. 30's left child is 20, which is less than 25, so you go to 20's right child.
4. 20's right child doesn't exist, so you add 25 there :)
Deleting an item from a binary search tree

It is assumed that you have already found the node that you want to delete, using the
search technique described above.

Case 1: The node you want to delete is a leaf

DATA STRUCTURES AND COMPUTER ALGORITHMS

For example, do delete 40...

 Simply delete the node!

Case 2: The node you want to delete has one child

1. Directly connect the child of the node that you want to delete, to the parent of the
node that you want to delete.

For example, to delete 90...

 Delete 90, then make 100 the child node of 50.

Case 3: The node you want to delete has two children

1. Find the left-most node in the right sub tree of the node being deleted. (After you
have found the node you want to delete, go to its right node, then for every node
under that, go to its left node until the node has no left node) From now on, this
node will be known as the successor.

DATA STRUCTURES AND COMPUTER ALGORITHMS

For example, to delete 30

1. The right node of the node which is being deleted is 40.

2. (From now on, we continually go to the left node until there isn't another one...)
The first left node of 40, is 35.
3. 35 has no left node, therefore 35 is the successor!
Case 1: The successor is the right child of the node being deleted

1. Directly move the child to the right of the node being deleted into the position of
the node being deleted.
2. As the new node has no left children, you can connect the deleted node's left
subtree's root as it's left child.

DATA STRUCTURES AND COMPUTER ALGORITHMS

For example, to delete 30

1. Move 40 up to where 30 was.

2. 20 now becomes 40's left child.
Case 2: The successor isn't the right child of the node being deleted

This is best shown with an example

To delete 30...

1. Move the successor into the place where the deleted node was and make it inherit
both of its children. So 35 moves to where 30 was and 20 and 40 become its
children.
2. Move the successor's (35s) right subtree to where the successor was. So 37
becomes a child of 40.
Node deletion

In general, remember that a node's left subtree's rightmost node is the closest node on the
left , and the right subtree's leftmost node is the closest node on the right, and either one
of these can be chosen to replace the deleted node, and the only complication is when the
replacing node has a child subtree on the same side to that node as the replacing node's
side of the deleting node, and the easiest thing to do is to always return the same side

DATA STRUCTURES AND COMPUTER ALGORITHMS

subtree of the replacing node as the child of the parent of the replacing node, even if the
subtree is empty.

B Trees

o B Trees were described originally as generalizations of binary trees , where a

binary tree is a 2-node B-Tree, the 2 standing for two children, with 2-1 = 1 key
separating the 2 children. Hence a 3-node has 2 values separating 3 children, and
a N node has N children separated by N-1 keys.

o A classical B-Tree can have N-node internal nodes, and empty 2-nodes as leaf
nodes, or more conveniently, the children can either be a value or a pointer to the
next N-node, so it is a union.

o The main idea with B-trees is that one starts with a root N-node , which is able to
hold N-1 entries, but on the Nth entry, the number of keys for the node is
exhausted, and the node can be split into two half sized N/2 sized N nodes,
separated by a single key K, which is equal to the right node's leftmost key, so
any entry with key K2 equal or greater than K goes in the right node, and
anything less than K goes in the left. When the root node is split, a new root node
is created with one key, and a left child and a right child. Since there are N
children but only N-1 entries, the leftmost child is stored as a separate pointer. If
the leftmost pointer splits, then the left half becomes the new leftmost pointer, and
the right half and separating key is inserted into the front of the entries.

o An alternative is the B+ tree which is the most commonly used in database

systems, because only values are stored in leaf nodes, whereas internal nodes only
store keys and pointers to other nodes, putting a limit on the size of the datum
value as the size of a pointer. This often allows internal nodes with more entries
able to fit a certain block size, e.g. 4K is a common physical disc block size.
Hence , if a B+ tree internal node is aligned to a physical disc block, then the
main rate limiting factor of reading a block of a large index from disc because it
isn't cached in a memory list of blocks is reduced to one block read.

DATA STRUCTURES AND COMPUTER ALGORITHMS

o A B+ tree has bigger internal nodes, so is more wider and shorter in theory than
an equivalent B tree which must fit all nodes within a given physical block size,
hence overall it is a faster index due to greater fan out and less height to reach
keys on average.

o Apparently, this fan out is so important, compression can also be applied to the
blocks to increase the number of entries fitting within a given underlying layer's
block size ( the underlying layer is often a file system block ).

o Most database systems use the B+ tree algorithm, including postgresql, mysql,
derbydb, firebird, many Xbase index types, etc.Many file systems also use a B+
tree to manage their block layout ( e.g. xfs, NTFS, etc. . don't know much detail
though ).

o Transwiki has a java implementation of a B+ Tree which uses traditional arrays as

key list and value list.Below is an example of a B Tree with test driver, and a B+
tree with a test driver. The memory / disc management is not included, but a
usable hacked example can be found at Hashing Memory Checking Example.This
B+ tree implementation was written out of the B Tree, and the difference from the
transwiki B+ tree is that it tries to use the semantics of Sorted Map and Sorted Set
already present in the standard java collections library.

o Hence, the flat leaf block list of this B+ implementation can't contain blocks that
don't contain any data, because the ordering depends on the first key of the
entries, so a leaf block needs to be created with its first entry.

DATA STRUCTURES AND COMPUTER ALGORITHMS

 Unit –III

Divide and conquer:

General method:

Splitting the n inputs to k distinct subset 1< k <=n, resulting k sub problems.
The sub problems are solved and sub solutions are combined into solutions of
the whole. If sub problems are large then divide and conquer be applied.

Control Abstraction:

A procedure whose flow of control is clear but the primary operations are
specified by other procedures

i. D and C is initially invoked as D and C (P) where P is the problem to be

solved.
ii. Small (p) – Boolean value function determines the input size small and answer
can be computed without splitting.
iii. If it is true, function f is invoked.
iv. Else problem P is divided into smaller sub problems(P 1,P2,P3….Pk)
v. These sub problems are solved by recursive apply of D and C
vi. Combine () – it is a function that determines solution to P using the solution to
the K sub problems.

DA and C Recurrence relation:

Example:
If problem P size is n. Size (k) sub problems are n1, n2….nk

G(n)
T(n) = T(n1)+T(n2)+….. T(nk)+ f(n)
DATA STRUCTURES AND COMPUTER ALGORITHMS
Otherwise T(n) for DA and C.
g(n) – time to compute the answers directly for small inputs.
f(n) – time for dividing and combing the solution of sub problems.

Complexity:
T(1) n=1
T(n) = aT(n/b) + f(n) n>1

Binary search:

i. Ai 1<= i <=n is a list of elements in increasing order.

ii. The problem is to determine whether the element x is present in the list or not.
iii. If x is present aj =x else j=0
Let P=(n, ai ….al, ax)
N is the number of elements in the list. A i…… al is the list of elements.
X is the elements to be searched.
iv. Divide the conquer is used to solve this problems.
v. Small(p) be true if n=1; s(p) will return the value i. if x=a i or return 0
vi. P has more than one elements it can be divided into a new sub problems.

Example:

If(5==1) then
{
Else
mid:= [(1+5)/2]; mid:=3
if(4=3)
else if(4<3)
Binsrch(5,4,5,4);
If(5==1)

DATA STRUCTURES AND COMPUTER ALGORITHMS

}
Finding the maximum and minimum:
i. The problem is to find the maximum and minimum items in a set
of n elements. The time is determined mainly by the total cost of
the element comparison.
ii. Best case occurs in increasing order; the number of elements
comparison is n-1
iii. Worst case occurs in decreasing order; the number of elements
comparison is less than 2(n-1)
iv. Average number of comparison is 3n/2-1.

Divide and conquer algorithm for this problems:

i. Let P = (n, a[i]…..a[j])

Where n is the number of elements in the list a[i]……a[j]
ii. Small(p) is true when n<=2
iii. If n=1 the maximum and minimum are a[i]
iv. If n=2 the problem can be solved by 1 comparison.
v. If n>2 p has to be divided into smaller instances.
Example:
P has two instances P 1 and P2
P1 = ([n/2],a[i]……a[n/2])
P2 = (n-[n/2], a[(n/2)+1]….a[n])
n=10
P1 = (5, a[i]…….a[5])
P2 = (5, a[6]……a[10])

DATA STRUCTURES AND COMPUTER ALGORITHMS

 Time complexity:
The time complexity of an algorithm is the amount of computed time it
needs time to completion.

Space complexity:

The space complexity of an algorithm is the amount of memory it

needs to run to completion.
 Constant
Algorithm that perform fixed number of steps irrespective of the input
size.
 Logarithmic:(logn)
Algorithms that breaks the problem into smaller ones.
 Linear (n)
Algorithms that breaks the problem into smaller one‘s.
 Linear(n logn)
Algorithm that break the problem into smaller ones and then combine
their result together.
 Quadratic:
This class of algorithms works with every pair of elements in inputs.
2kg elements
10gm gold
2kg paper
100g silver
4kg stone

Merge sort:

I. It is a simple short procedure which sorts two halves of the input and
procedures the sorted list by merging the sorted sub list.

DATA STRUCTURES AND COMPUTER ALGORITHMS

 II. It uses divide and conquer techniques.
III. Given a sequence of n elements a[1]…….a[n]. it is divided into two sets.
a[[n/2]+1]………..a[n]
IV. Each set is individually sorted and merged to single sorted sequence of n
elements.

Initial Unsorted input

Unsorted unsorted
Divide Sub list sub list

sorted sorted
Sub list sub list
Solve

Combine Sorted list

Example:

Sorted list 1 sorted list 2

(10, 25, 32, 35) (15, 21, 30, 42, 60)

output
Compare 10 and 15 10
Compare 25 and 15 15
Compare 25 and 21 21
Compare 25 and 30 25
Compare 32 and 30 30
Compare 32 and 42 32
Compare 35 and 42 35

DATA STRUCTURES AND COMPUTER ALGORITHMS

No comparison 42 and 60

Complexity : o(n logn)

Computing time:

T(n)= a n=1 a, a constant

2T(n/2)+n n>1 c, c constant

Example:

Sorted list1 sorted list2

(10, 25, 32, 35) (15, 21, 30, 42, 60)
Low=1 high=9 mid=10

Second sorting algorithm:

Insertion sort:
It works exceeding fast on arrays of small sized set.
It computing time 0(n2)
Example:
Input : 10 15 3 27 9 20 12

10 15 3 27 9 20 12
10 15 3 27 9 20 12
3 10 15 27 9 20 12
3 10 15 27 9 20 12
3 9 10 15 27 20 12
3 9 10 15 20 27 12
3 9 10 12 15 20 27

DATA STRUCTURES AND COMPUTER ALGORITHMS

Quick sort:

1. It is also referred to as partition exchange sort.

2. It use divide and conquer techniques.
3. The division into 2 sub arrays it‘s made. so that the sorted sub array do not need
to be merged later.
4. This can be done by rearranging the elements in a[1:n] such that a[i]<=a[j]. for all
I between 1 and m.
5. And all j between m+1 and n
6. The elements in a[1:n] and a[m+1:n] can be independently sorted. No merge is
needed.

Formal procedure for partition:

1. Position a pointer low to the second element and a pointer high to the last
element.
2. Move low forward till it reaches an element greater than k1
3. Move high backward till it reaches an element lass than k1
4. Interchange klow and khigh if low<high
5. Continued step 2 to 4 till low < high
6. Swap the element at first and high indices.

35 26 10 13 45 92 30 60

DATA STRUCTURES AND COMPUTER ALGORITHMS

 1. Best case for quick sort is when input can be divided into equal size subsets.
2. Complexity of quick sort in the best case is 0(nlogn)
3. Worst case is when the partitions are uneven its complexity is 0(n2)

Performance and measurements:

1. Quick sort and merge sort are evaluated.

2. Both sorts used recursive version.
3. Quick sort is faster than merge sort for all values.

Randomized sorting algorithm:

1. Average time of quick sort is 0(n logn) on n elements

2. Worst case is 0(n2)
3. Quick sort displays worst case behavior when the elements are already in sorted
order.
4. In this case the partition will results one elements in one part and the rest of the
elements in the other part.
5. The performance of the divide and conquer is good only when the sub problems
are in equal size.
6. Quick sort can be modified so that it can perform well on every input by using
randomized.
7. Instead of picking a(m) from an array a[p:q] the random element is picked as the
partitioned element.
8. The resultant randomized algorithm works on any input and runs in 0(n logn)

Selection sort:

1. Selection sort selects the largest or smallest element from the list to be sorted and
places in its correct position.
2. If the largest element is taken, then it is placed in the last position.

DATA STRUCTURES AND COMPUTER ALGORITHMS

 3. If the smallest element is taken, then it is placed in the first position.
4. With every pass, the sorted position of the list grows by 1.and the unsorted
potion shrinks by 1
eg:
10 15 3 27 9 20 12
10 15 3 12 9 20 27
10 15 3 12 9 20 27
10 9 3 12 15 20 27
10 9 3 12 15 20 27
10 9 3 12 15 20 27
3 9 10 12 15 20 27
In the first pass (n-1) comparisons are
In the second pass (n-2) comparisons are required
In the last pass (n-1), 1 comparison is done.
Total no of comparisons = (n-1)+(n-2)+…..+2+1
= n(n-1)/2
= 0(n2)

Strassen's Matrix Multiplication Algorithm

Problem Description
Write a threaded code to multiply two random matrices using Strassen's
Algorithm. The
application will generate two matrices A(M,P) and B(P,N), multiply them together using
(1) a sequential method and then (2) via Strassen's Algorithm resulting in C(M,N). The
applicationshould then compare the results of the two multiplications to ensure that the
Strassen'sresults match the sequential computations.The input to the application comes
from the command line. The input will be 3 integers describing the sizes of the matrices
to be used: M, N, and P.
Code restrictions: A very simple, serial version of the application, written in C,
will be available

DATA STRUCTURES AND COMPUTER ALGORITHMS

here. This source file should be used as a starting point. Your entry should keep the body
of the main function, the matrix generation function, the sequential multiplication code,
and the function to compare the two matrix product results. Changes needed for
implementation in a different language are permitted. You are also allowed to change the
memory allocation and other code to thread the Strassen's computations. (It would be a
good idea to document the changes and the reason for such changes in your solution
write-up.) After the needed changes, your submitted solution must use some form of
Strassen's Algorithm to compute the second matrix multiplication result.
Timing: The time for execution of the Strassen's Algorithm will be used for
scoring. Each
submission should include timing code to measure and print this time to stdout. If not, the
total execution time will be used.
Matrix multiplication
Given two matrices AM*P and BP*N, the product of the two is a matrix CM*N which is
computed as
follows:
void seqMatMult(int m, int n, int p, double** A, double** B, double** C) {
for (int i = 0; i < m; i++)
for (int j = 0; j < n; j++) {
C[i][j] = 0.0;
for (int k = 0; k < p; k++)
C[i][j] += A[i][k] * B[k][j];
}
}

Strassen's algorithm
To calculate the matrix product C = AB, Strassen's algorithm partitions the data to reduce
the
number of multiplications performed. This algorithm requires M, N and P to be powers of
2.

DATA STRUCTURES AND COMPUTER ALGORITHMS

The algorithm is described below.
1. Partition A, B and and C into 4 equal parts:
A=
A11 A12
A21 A22
B=
B11 B12
B21 B22
C=
C11 C12
C21 C22
2. Evaluate the intermediate matrices:
M1 = (A11 + A22) (B11 + B22)
M2 = (A21 + A22) B11
M3 = A11 (B12 – B22)
M4 = A22 (B21 – B11)
M5 = (A11 + A12) B22
M6 = (A21 – A11) (B11 + B12)
M7 = (A12 – A22) (B21 + B22)
3. Construct C using the intermediate matrices:
C11 = M1 + M4 – M5 + M7
C12 = M3 + M5
C21 = M2 + M4
C22 = M1 – M2 + M3 + M6
Serial Algorithm
1. Partition A and B into quarter matrices as described above.
2. Compute the intermediate matrices:
1. If the sizes of the matrices are greater than a threshold value, multiply them
recursively using Strassen's algorithm.
2. Else use the traditional matrix multiplication algorithm.
3. Construct C using the intermediate matrices.

DATA STRUCTURES AND COMPUTER ALGORITHMS

Parallelization
The evaluations of intermediate matrices M1, M2 ... M7 are independent and hence, can
be computed in parallel. On a machine with Q processors, Q jobs can be run at a time.
Initial Approach
The initial approach to a parallel solution used a task pool model to compute M1, M2 ...
M7. As shown in the diagram below, the second level of the algorithm creates 49 (7 * 7)
independent multiplication tasks which can all be executed in parallel, Q jobs at a time.
A Realization
Since the number of jobs is 49, ideally, on a machine with Q cores (where Q = 2q, Q <=
16), 48 of these would run concurrently while 1 would end up being executed later, thus,
giving poor processor utilization. It would be a better idea to split the last task further.
AB
M1
M2 M3 M4 M5 M6 M7
M1 M7
M1, M2... M7 M1, M2... M7 M1, M2... M7 M1, M2... M7 M1, M2... M7 M1, M2... M7
M1, M2... M7
Final Parallel Algorithm
The final solution uses thread pooling with a pool of Q threads (where Q is the number
of processors), in conjunction with the Strategy pattern to implement Strassen's
algorithm. The algorithm is described below:
1 If the sizes of A and B are less than the threshold
1.1 Compute C = AB using the traditional matrix multiplication algorithm.
2 Else use Strassen's algorithm
2.1 Split matrices A and B
2.2 For each of Mi i = 1 to 7
2.2.1 Create a new thread to compute Mi = A'i B'i
2.2.2 If the sizes of the matrices are less than the threshold
2.2.2.1 Compute C using the traditional matrix multiplication algorithm.
2.2.3 Else use Strassen's algorithm
2.2.3.1 Split matrices A'i and B'i

DATA STRUCTURES AND COMPUTER ALGORITHMS

2.2.3.2 For each of Mij j = 1 to 7
2.2.3.2.1 If i=7 and j=7 go to step 1 with A = A'77 and B = B'77
2.2.3.2.2 Get a thread from the thread pool to compute Mij = A'ij B'ij
2.2.3.2.3 Execute the recursive version of Strassen's algorithm in this thread
2.2.3.3 Wait for the Mij threads to complete execution
2.2.3.4 Compute Mi
2.3 Wait for the Mi threads to complete execution
2.4 Compute C
The Strategy pattern
The above algorithm defines 3 distinct strategies to be used with Strassen's algorithm:
1. Execute each child multiplication operation in a new thread (M1, M2, ..., M7)
2. Execute each child multiplication operation in a thread (Mij) from the thread pool
3. Execute each child multiplication operation using recursion
Hence, the Strategy pattern was used in the implementation.
AB
M1
M2 M3 M4 M5 M6 M7
M1 M7
M1, M2... M7 M1, M2... M7 M1, M2... M7 M1, M2... M7 M1, M2... M7 M1, M2... M7
M1, M2... M6 M7
Problems
Initially, an attempt was made to use the originally allocated storage in all computations.
However, due to cache thrashing, extremely poor performance was observed in this case.
Top showed less than 10% processor utilization on a quad core processor. This problem
was
resolved by creating copies of the required matrices for each thread. This speeds up the
solution at the expense of memory requirement.
Performance
The machine used for these tests had the following configuration:
Core 2 Quad Q8300 2.33GHz, 4GB DDR3 1333MHz RAM

DATA STRUCTURES AND COMPUTER ALGORITHMS

Following are the results for M = N = P = 2048. The traditional algorithm takes
approximately
154.54 seconds to perform the multiplication.
#Threads Time (seconds)
1 20.24
2 10.81
4 6.52
For 4 threads the traditional algorithm uses about 1.7% of the available memory whereas
the
parallel version uses about 14%.
Conclusion
Strassen's algorithm definitely performs better than the traditional matrix multiplication
algorithm due to the reduced number of multiplications and better memory separation.
However, it requires a large amount of memory to run. The performance gain is sub-
linear which could be due to the fact that there are threads waiting for other threads to
complete execution.

DATA STRUCTURES AND COMPUTER ALGORITHMS

 IV UNIT

Greedy method:

Most of the problem will have n input and require as obtaining a subset
that satisfies some constraints. Any subset satisfies these constraints are
called feasible solutions.
We have to find a feasible solution that either maximizes or a minimizes
given objective function.
A feasible solution that does this is called optimal solutions.The greedy
method suggest consider 1 input at a time. At each stage a decision is
made regarding whether a particular I/P is in an optimal solution.
If the inclusion of the next I/P into the partially constructed optimal
solution will result in an infeasible solutions, then this I/P is not added to
the partial solution, otherwise it is added.The selection procedure is based
on some optimization. This measure may be the objective function.
This version of the greedy techniques is called the subset paradigm.For
problem that do not call for the selection of an optimal subset, in the
greedy method we make decision by considering the inputs in some
order.This version of the greedy method is called the ordering
paradigm.Greedy method control abstraction for the subset paradigm.
General Method
Procedure GREEDY(A,n)
// A(1:n) contains the n inputs//
solution ¬ f //initialize the solution to empty//
for i ¬ 1 to n do
x ¬ SELECT(A)
if FEASIBLE(solution,x)
DATA STRUCTURES AND COMPUTER ALGORITHMS
 then solution ¬ UNION(solution,x)
endif
repeat
return(solution)
end GREEDY

Optimal storage on tapes

Problems, which deal with searching a set of solutions, or which ask for an
optimal solution satisfying some constraints can be solved using the
backtracking formulation. The backtracking algorithm yields the propersolution in fewer
trials.
The basic idea of backtracking is to build up a vector one component at a time and to test
whether the vector being formed has any chance of success. The major advantage of
this algorithm is that if it is realized that the partial vector generated does not lead to an
optimal solution then that vector may be ignored.
Backtracking algorithm determine the solution by systematically searching
the solution space for the given problem. This search is accomplished by using a free
organization. Backtracking is a depth first search with some bounding function. All
solutions using backtracking are required to satisfy a complex set of constraints. The
constraints may be explicit or implicit.
Explicit constraints are rules, which restrict each vector element to be chosen from the
given set. Implicit constraints are rules, which determine which of the tuples in
the solution space, actually satisfy the criterion function.

Cassette filling problem:

There are n programs that are to be stored on a tape of length L. Every program ‗i‘ is of
length li. All programs can be stored on the tape if and only if the sum of the lengths of
the programs is at most L. In this problem, it is assumed that whenever a program is to be
retrieved, the tape is positioned at the start end. Hence, the time tjneeded to retrieve
program ij from a tape having the programs in the order i1,i2, …,in is called mean retrieval
time(MRT) and is given by

DATA STRUCTURES AND COMPUTER ALGORITHMS

tj =S lik k=1,2,…,j
In the optimal storage on tape problem, we are required to find a permutation for the n
programs so that when they are stored on the tape, the MRT is minimized.
Let n=3 and (l1,l2,l3)=(5,10,3),there are n!=6 possible orderings. These orderings and their
respective MRT is given in the fig 6.1. Hence, the best order of recording is 3,1,2.

Subset problem:
There are n positive numbers given in a set. The desire is to find all possible subsets of
this set, the contents of which add onto a predefined value M.
Let there be n elements in the main set. W=w[1..n] represent the elements of the set. i.e.,
w = (w1,w2,w3,…,wn) vector x = x[1..n] assumes either 0 or 1 value. If element w(i) is
included in the subset then x(i) =1.
Consider n=6 m=30 and w[1..6]={5,10,12,13,15,18}. The partial backtracking tree is
shown in fig 6.2. The label to the left of a node represents the item number chosen for
insertion and the label to the right represents the space occupied in M. S represents
a solution to the given problem and B represents a bounding criteria if no solution can
be reached. For the above problem the solution could be (1,1,0,0,1,0), (1,0,1,1,0,0) and
(0,0,1,0,0,1). Completion of the tree structure is left as an assignment for the reader.

DATA STRUCTURES AND COMPUTER ALGORITHMS

queen problem:

The 8 queen problem can be stated as follows. Consider a chessboard of order 8X8. The
problem is to place 8 queens on this board such that no two queens are attack can attack
each other.
Illustration.

Consider the problem of 4 queens, backtracking solution for this is as shown in the fig
6.3. The figure shows a partial backtracking tree. Completion of the tree is left as an
assignment for the reader.

KNAPSACK PROBLEM:

DATA STRUCTURES AND COMPUTER ALGORITHMS

We are giving n objects and knapsack or a bag objects i has a weight wi and the
knapsack has a capacity m.
If a fraction Xi , 0<=Xi<=1 of object I is placed into the knapsack, then a profit of
PiXi is earned.
The objective is to obtain the filling of a knapsack that maximize the total profit
earned.
The above problem can be stated as
n
Maximize ∑ PiXi
i=1
n
subject to ∑ WiXi <=m and 0<=Xi<=1
i=1 1<=i<=n

consider the example n=3, m=20.

(P1,P2,P3) = (25,24,15)

(W1,W2,W3) = (18,15,10)

(X1,X2,X3) ∑ WiXi ∑ PiXi

∑ WiXi = W1X1 + W2X2 + W3X3

= 18 * ½ + 15 * 1/3 + 10 * ¼

= 9 + 5 + 5/2

= 14 + 2.5 = 16.5

∑ PiXi = P1X1 + P2X2 + P3X3

DATA STRUCTURES AND COMPUTER ALGORITHMS

= 25 * ½ + 24 *1/3 +15* ¼

= 25/2 + 8 + 15/4

= 12.5 +8+3.75

= 24.25

This is one of the feasible solution for the given problem.

(X1,X2,X3) = (1, 2/15, 0)

= 18 * 1 + 15 * 2/15 + 0

WiXi = 20

∑ PiXi = 25 * 1 + 24 *2/15

= 25 + 16/5

= 25 + 3.2

= 28.2

E can add from lower capacity to higher capacity (0, 2/3, 1)

WiXi = 18 * 0 + 15 * 2/3 + 10 * 1

= 10 + 10

DATA STRUCTURES AND COMPUTER ALGORITHMS

= 20

PiXi =25 * 0 + 24 * 2/3 + 15 * 1

= 0 + 16 + 15

= 31

At each step we select object which has the maximum profit per unit of capacity
used. This means the objects are consider in the ratio = Pi / Wi

Ratio = Pi / Wi = ( 25/18, 24/15, 15/10)

= (1.3, 1.6, 1.5)

(0, 1 , ½) = 25 * 0 +24*1+15 * ½

= 24 + 7.5

PiXi = 31.5

WiXi =20

JOB SEQUENCING WITH DEADLINES ALGORITHM

SIMULATION
Ordering the jobs be nonincreasing order of profit:
Jobs = (99, 67, 45, 34, 23, 10)
Job No. =(6, 3, 4, 2, 5, 1)
Deadlines =(2, 3, 1, 4, 5, 3)
New job no. =(I, II, III. IV, V, VI)

DATA STRUCTURES AND COMPUTER ALGORITHMS

Job I is allotted slot [0,1]
New Job no. I
Ordered by deadlines 2
Job no. 6
Job II is allotted slot [1,2], the deadlines are in increasing length.
New Job no. I II
Ordered by deadlines
23
Job no. 6 3
Job III is being considered. Deadline is 1, so we have to shift jobs I and II upward
New Job no. I II Ordered by
deadlines
23
Job no. 6 3New Job no. III I II
Ordered by deadlines
1 2 3
Job no. 4 6 3
The deadlines are in increasing order.
New Job no. III I II Ordered by
deadlines
1 2 3
Job no. 4 6 3

Job IV has a deadline of 4, so it can be allotted slot [3,4]

New Job no. III I II IV
Ordered by deadlines
1 2 3 4
Job no. 4 6 3 2
Job V has a deadline of 5, so it can be allotted slot [4,5]
New Job no. III I II IV V
Ordered by deadlines

DATA STRUCTURES AND COMPUTER ALGORITHMS

1 2 3 4 5
Job no. 4 6 3 2 5
Job VI has a deadline of 3 but we cannot shift the array to the left, so we reject
job VI.
The above is a the schedule. FAST Job Sequencing with deadlines.
Consider a scheduling problem where the 6 jobs have a profit of
(10,34,67,45,23,99) and corresponding deadlines (2,3,1,4,5,3). Obtain the
optimum schedule. What is the time complexity of your algorithm? Can you
improve it?
Sort jobs in nondecreasing order of profit:
Profits = (99, 67, 45, 34, 23, 10)
Job no. = ( 6, 3, 4, 2, 5, 1)
New no = ( I, II, III, IV, V, VI)
Deadline = (2, 3, 1, 4, 5, 3)
Start with all slots free.
Job I considered, do it in [1,2] slot which is free.
-1
[-1,0],free
-1
[0,1],free
-1
[1,2],free
-1
[2,3],free
-1
[3,4],free
-1
[4,5],free
Job II is considered, do it in [2,3] slot.
-1
[4,5],free

DATA STRUCTURES AND COMPUTER ALGORITHMS

-1
[3,4],free
-1
[2,3],free
-1
[1,2],job I
-1
[0,1],free
-1
[-1,0],free -1
[4,5],free
-1
[3,4],free
-1
[2,3],jobII
-1
[1,2],job I
-1
[0,1],free
-1
[-1,0],free
Take Union of [1,2] and new [2,3] slot.
Job III, considered.Deadline is 1. Allot [0,1] slot.
-1
[-1,0],free
-1
[0,1],free [1,2],job I
[2,3],jobII
-1
[3,4],free
-2 -1

DATA STRUCTURES AND COMPUTER ALGORITHMS

[4,5],free
[1,2],job I
-1
[4,5],free
-1
[3,4],free
[2,3],jobII
-1 -2
[0,1],III
-1
[-1,0],free Take union of [1,2] and new [0,1] slot.
-1
[4,5],free
-1
[3,4],free
[2,3],jobII
-3
[1,2],job I
[0,1],III
-1
[-1,0],free
Consider job no IV. Deadline is 4, allot to [3,4] slot which is free.
Take union of [1,2] and [3,4] slot.
-1
[4,5],free
-1
[3,4], IV
[2,3],jobII
-3
[1,2],job I
[0,1],III

DATA STRUCTURES AND COMPUTER ALGORITHMS

-1
[-1,0],free Job V has a deadline of 5, allot to [4,5] slot which is free.
-1
[-1,0],free
[0,1],III
[1,2],job I
-4
[2,3],jobII
[3,4],IV
-1
[4,5],free
Take union of [1,2] and [4,5] slot that has been created.
-1
[4,5],V
[3,4],IV
[2,3],jobII
-4
[1,2],job I
[0,1],III
-1
[-1,0],free Job VI considered now. It has a deadline of 3. No slot except [-1,0] is
free, counting down from [2,3]. So allot it to [-1,0] which is always free and
means reject the job.
-1
[-1,0],free
[0,1],III
[1,2],job I
-5
[2,3],jobII
[3,4],IV
[4,5],V

DATA STRUCTURES AND COMPUTER ALGORITHMS

[4,5],V
[3,4],IV
[2,3],jobII
-5
[1,2],job I
[0,1],III
-1
[-1,0],VI ABOVE IS FINAL SCHEDULE.
[-1,0],VI
[0,1],III
[1,2],job I
-6
[2,3],jobII

Optimal merge patterns

Input: N sorted arrays of length L[1], L[2],...,L[n]

Problem: Ultimateley, to merge the arrays pairwise as fast as possible. The
problem is to determine which pair to merge everytime.
Method (the Greedy method): The selection policy (of which best pair of arrays to
merge next) is to choose the two shortest remaining arrays.
Implementation:
o Need a data structure to store the lengths of the arrays, to find the shortest
2 arrays at any time, to delete those lengths, and insert in a new length (for
the newly merged array).
o In essence, the data structure has to support delete-min and insert. Clearly,
a min-heap is ideal.
o Time complexity of the algorithm: The algorithm iterates (n-1) times. At
every iteration two delete-mins and one insert is performed. The 3
operations take O(log n) in each iteration.

DATA STRUCTURES AND COMPUTER ALGORITHMS

 o Thus the total time is O(nlog n) for the while loop + O(n) for initial heap
construction.
o That is, the total time is O(nlog n).

Minimum spanning trees:

Graphs:
Graphs are used to represent network structures.
Definition:A graph is defined as G=(V,E) where V is the set of vertices(nodes)
and E is the set of edges(arcs) connecting the vertices.An edge is represented as a
pair of vertices (u,v)

2 3

4 5

V = { 1,2,3,4,5}

E = { (1,2), (1,3), (1,4), (2,3), (2,4),(2,5), (3,5)}

Spanning trees:

A spanning tree of a graph is a tree (V,E‘) where V is the vertices of G and E‘ is

the subset of edges. E(graph) that connects all vertices as a tree.

1 2 1 2

DATA STRUCTURES AND COMPUTER ALGORITHMS

4 4
 3 3

1 2 1 2

3
4
3 4

V = {1, 2, 3, 4}

E = {(1, 2), (1, 3), (1, 4), (2, 3), (2, 4), (3, 4)}

V= {1, 2, 3, 4}

E = {(1, 2), (1, 3), (2,4)}

Minimum spanning tree:

A minimum spanning tree of a graph G is its spanning tree in which the sum of
the weight of the edges is minimum.

1
DATA STRUCTURES AND COMPUTER ALGORITHMS
 28
1
10
10 2 2
6
14
14 16 7
3 6 7
16
3
25
5 4
24 25
18 1212
22 5 4
22

Prims Algorithm:

The tree started with an arbitrary root.

It is then connected to the other vertex with minimum weighted graph.
The iteration continue till all vertices are covered in the tree without any cycles.

Stages of Prims algorithm:

Stage : 1 Stage : 2

1
1
1
2
10
6
1
7
3 1
1
5
1 ALGORITHMS
DATA 4STRUCTURES AND COMPUTER
1
 25

Stage : 3 Stage : 4

1 2
2
1
7

7
6 3
6
3

12
5 4

5 4

Stage : 5 stage : 6

2
1 1
2
14
16
7 3
6 6
3

DATA STRUCTURES AND COMPUTER ALGORITHMS

 5 4 5 4

COMPLEXITY:

The time required by prims algorithm is 0(n2) where n is the number of vertices
in a graph G.
KRUSKAL‘S ALGORITHM:
Each vertex of a graph is taken as a one node tree.
Then a minimum weighted edge is selected to connect two trees together.
The iteration continue till all trees of the graph form a single tree.

Stages : 1 stage : 2

2 1
2
6
10 6
7
3 7
5
3
4 5

Stage : 3 stage: 4

1 1

10 10
2
2
6 7
6 ALGORITHMS
DATA STRUCTURES AND COMPUTER
7
3
5
4
 14
12 5 3

12
4
Stage: 5 stage:6

1 1
2
10 10 6 2
6
14 7
7
14
3
5 16 5 3
16 4
4

12 22 12

Job sequencing with deadlines:

We are given a set of n jobs associated with job i is an integer deadlines Vi >=0
and a profit Pi>0
For any job i the profit Pi is earn iff the job is completed by its deadlines.
To complete a job one has to process the job on a machine for one unit of time.
If only one machine is available for processing job.
A feasible solution for this problem is a subset J of jobs. Such that each job in this
subset can be completed by its deadlines.
The values of the feasible solution J is the sum of the profit of the job in J, are ∑i
and optimal solution is a feasible solution with maximum value.Example:
N=4
(P1,P2,P3,P4) = (100, 10, 15, 27)

DATA STRUCTURES AND COMPUTER ALGORITHMS

 (d1,d2,d3,d4) = (2, 1, 2, 1)
Feasible solution and their values:
Feasible solution:
(1,2)
(1,3)
(1,4)
(2,3)
(2,4)
(3,4)
(1)
(2)
(3)
(4)
There are 9 feasible solutions. Optimal solution is maximum profit. So it is 127
for (1,4)
The above problem 3 is the optimal solution and the jobs must be processed in the
order job 4 followed by job1To formulated the greedy algorithm we must obtain
an optimal solution for this we must formulated an optimization method.
The next job2 include one is the ∑i increase the most.
This requires that we have to consider jobs in the decreasing order of Pi.

Single shortest path

Graph theory, the shortest path problem is the problem of finding a path between two
vertices (or nodes) such that the sum of the weights of its constituent edges is minimized.
An example is finding the quickest way to get from one location to another on a road
map; in this case, the vertices represent locations and the edges represent segments of
road and are weighted by the time needed to travel that segment.

DATA STRUCTURES AND COMPUTER ALGORITHMS

Formally, given a weighted graph (that is, a set V of vertices, a set E of edges, and a real-
valued weight function f : E → R), and one element v of V, find a path P from v to
a v' of Vso that

is minimal among all paths connecting v to v' .The problem is also sometimes called
the single-pair shortest path problem, to distinguish it from the following
generalizations:

 The single-source shortest path problem, in which we have to find shortest paths
from a source vertex v to all other vertices in the graph.
 The single-destination shortest path problem, in which we have to find shortest
paths from all vertices in the graph to a single destination vertex v. This can be
reduced to the single-source shortest path problem by reversing the edges in the
graph.
 The all-pairs shortest path problem, in which we have to find shortest paths
between every pair of vertices v, v' in the graph.

The most important algorithms for solving this problem are:

 Dijkstra's algorithm solves the single-pair, single-source, and single-destination

shortest path problems.
 Bellman-Ford algorithm solves the single source problem if edge weights may be
negative.
 A* search algorithm solves for single pair shortest path using heuristics to try to
speed up the search.
 Floyd-Warshall algorithm solves all pairs shortest paths.
 Johnson's algorithm solves all pairs shortest paths, and may be faster than Floyd-
Warshall on sparse graphs.
 Perturbation theory finds (at worst) the locally shortest path.

Additional algorithms and associated evaluations may be found in Cherkassky et

al.[1]
DATA STRUCTURES AND COMPUTER ALGORITHMS
 Shortest path algorithms are applied to automatically find directions between
physical locations, such as driving directions on web mapping websites
like Mapquest or Google Maps. For this application fast specialized algorithms
are available.

If one represents a nondeterministic abstract machine as a graph where vertices

describe states and edges describe possible transitions, shortest path algorithms
can be used to find an optimal sequence of choices to reach a certain goal state,
or to establish lower bounds on the time needed to reach a given state. For
example, if vertices represents the states of a puzzle like a Rubik's Cube and
each directed edge corresponds to a single move or turn, shortest path
algorithms can be used to find a solution that uses the minimum possible
number of moves.

In a networking or telecommunications mindset, this shortest path problem is

sometimes called the min-delay path problem and usually tied with a widest
path problem. For example, the algorithm may seek the shortest (min-delay)
widest path, or widest shortest (min-delay) path.

A more lighthearted application is the games of "six degrees of separation" that

try to find the shortest path in graphs like movie stars appearing in the same
film.

Other applications include "operations research, plant and facility

layout, robotics, transportation, and VLSI design".For shortest path problems
in computational geometry, see Euclidean shortest path.

The travelling salesman problem is the problem of finding the shortest path that
goes through every vertex exactly once, and returns to the start. Unlike the
shortest path problem, which can be solved in polynomial time in graphs
without negative cycles (edges with negative weights), the travelling salesman
problem is NP-complete and, as such, is believed not to be efficiently solvable
(see P = NP problem). The problem of finding the longest path in a graph is
also NP-complete.

DATA STRUCTURES AND COMPUTER ALGORITHMS

 The Canadian traveller problem and the stochastic shortest path problem are
generalizations where either the graph isn't completely known to the mover,
changes over time, or where actions (traversals) are probabilistic.

The shortest multiple disconnected path [4] is a representation of the primitive

path network within the framework of Reptation theory.

The problems of recalculation of shortest paths arises if some graph

transformations (e.g., shrinkage of nodes) are made with a graph.[5]

The widest path problem seeks a path so that the minimum label of any edge is
as large as possible.

There is a natural linear programming formulation for the shortest path

problem, given below. It is very trivial compared to most other uses of linear
programs in discrete optimization, however it illustrates connections to other
concepts.

Given a directed graph (V, A) with source node s, target node t, and cost wij for
each arc (i, j) in A, consider the program with variables xij

minimize subject to and for

all i,

This LP, which is common fodder for operations research courses, has the
special property that it is integral; more specifically, everybasic optimal
solution (when one exists) has all variables equal to 0 or 1, and the set of
edges whose variables equal 1 form an s-tdipath. See Ahuja et al.[6] for one
proof, although the origin of this approach dates back to mid-20th century.

The dual for this linear program is

maximize yt - ys subject to for all ij, yj - yi ≤ wij

DATA STRUCTURES AND COMPUTER ALGORITHMS

 and feasible duals correspond to the concept of a consistent
heuristic for the A* algorithm for shortest paths. For any feasible
dual ythe reduced costs w'ij = wij − yj + yi are nonnegative
and A* essentially runs Dijkstra's algorithm on these reduced costs.

DATA STRUCTURES AND COMPUTER ALGORITHMS

 Unit V

Dynamic programming
For the programming paradigm, see Dynamic programming
language.In mathematics and computer science, dynamic programming is a
method for solving complex problems by breaking them down into simpler
subproblems. It is applicable to problems exhibiting the properties of overlapping
subproblems which are only slightly smaller and optimal substructure (described
below). When applicable, the method takes far less time than naïve methods.
The key idea behind dynamic programming is quite simple. In general, to solve a
given problem, we need to solve different parts of the problem (subproblems),
then combine the solutions of the subproblems to reach an overall solution. Often,
many of these subproblems are really the same. The dynamic programming
approach seeks to solve each subproblem only once, thus reducing the number of
computations. This is especially useful when the number of repeating
subproblems is exponentially large.
Top-down dynamic programming simply means storing the results of certain
calculations, which are later used again since the completed calculation is a sub-
problem of a larger calculation. Bottom-up dynamic programming involves
formulating a complex calculation as a recursive series of simpler calculations.
The term dynamic programming was originally used in the 1940s by Richard Bellman to
describe the process of solving problems where one needs to find the best decisions one
after another. By 1953, he refined this to the modern meaning, referring specifically to
nesting smaller decision problems inside larger decisions,[2] and the field was thereafter
recognized by the IEEE as a systems analysis and engineering topic. Bellman's
contribution is remembered in the name of the Bellman equation, a central result of
dynamic programming which restates an optimization problem in recursive form.
The word dynamic was chosen by Bellman to capture the time-varying aspect of the
problems, and also because it sounded impressive.[3] The word programming referred to
the use of the method to find an optimal program, in the sense of a military schedule for
DATA STRUCTURES AND COMPUTER ALGORITHMS
training or logistics. This usage is the same as that in the phrases linear
programming and mathematical programming, a synonym for mathematical optimization.

Finding the shortest path in a graph using optimal substructure; a straight line
indicates a single edge; a wavy line indicates a shortest path between the two
vertices it connects (other nodes on these paths are not shown); the bold line is
the overall shortest path from start to goal.
Dynamic programming is both a mathematical optimization method and a
computer programming method. In both contexts it refers to simplifying a
complicated problem by breaking it down into simpler subproblems in
a recursive manner. While some decision problems cannot be taken apart this
way, decisions that span several points in time do often break apart recursively;
Bellman called this the "Principle of Optimality". Likewise, in computer science,
a problem which can be broken down recursively is said to have optimal
substructure.
If subproblems can be nested recursively inside larger problems, so that dynamic
programming methods are applicable, then there is a relation between the value
of the larger problem and the values of the subproblems.[5] In the optimization
literature this relationship is called the Bellman equation.

Dynamic programming in mathematical optimization In terms of mathematical

optimization, dynamic programming usually refers to simplifying a decision by
breaking it down into a sequence of decision steps over time. This is done by
defining a sequence of value functions V1, V2 , ... Vn , with an
argument y representing the state of the system at times i from 1 to n. The
definition of Vn(y) is the value obtained in state y at the last time n. The values
Vi at earlier times i=n-1,n-2,...,2,1 can be found by working backwards, using
arecursive relationship called the Bellman equation. For i=2,...n, Vi -1 at any state
DATA STRUCTURES AND COMPUTER ALGORITHMS
y is calculated from Vi by maximizing a simple function (usually the sum) of the
gain from decision i-1 and the function Vi at the new state of the system if this
decision is made. Since Vi has already been calculated for the needed states, the
above operation yields Vi -1 for those states. Finally, V1 at the initial state of the
system is the value of the optimal solution. The optimal values of the decision
variables can be recovered, one by one, by tracking back the calculations already
performed.
Dynamic programming in computer programmingThere are two key attributes
that a problem must have in order for dynamic programming to be
applicable: optimal substructure andoverlapping subproblems. However, when
the overlapping problems are much smaller than the original problem, the
strategy is called "divide and conquer" rather than "dynamic programming". This
is why mergesort, quicksort, and finding all matches of a regular expression are
not classified as dynamic programming problems.
Optimal substructure means that the solution to a given optimization problem can
be obtained by the combination of optimal solutions to its subproblems.
Consequently, the first step towards devising a dynamic programming solution is
to check whether the problem exhibits such optimal substructure. Such optimal
substructures are usually described by means of recursion. For example, given a
graph G=(V,E), the shortest path p from a vertex u to a vertex v exhibits optimal
substructure: take any intermediate vertex won this shortest path p. If p is truly
the shortest path, then the path p1 from u to w and p2 from w to v are indeed the
shortest paths between the corresponding vertices (by the simple cut-and-paste
argument described in CLRS). Hence, one can easily formulate the solution for
finding shortest paths in a recursive manner, which is what the Bellman-Ford
algorithm does.
Overlapping subproblems means that the space of subproblems must be small,
that is, any recursive algorithm solving the problem should solve the same
subproblems over and over, rather than generating new subproblems. For
example, consider the recursive formulation for generating the Fibonacci series:
Fi = Fi-1 + Fi-2, with base case F1=F2=1. Then F43 = F42 + F41, and F42 = F41 + F40.

DATA STRUCTURES AND COMPUTER ALGORITHMS

Now F41 is being solved in the recursive subtrees of both F43 as well as F42. Even
though the total number of subproblems is actually small (only 43 of them), we
end up solving the same problems over and over if we adopt a naive recursive
solution such as this. Dynamic programming takes account of this fact and solves
each subproblem only once. Note that the subproblems must be
onlyslightly smaller (typically taken to mean a constant additive factor[citation
needed]
) than the larger problem; when they are a multiplicative factor smaller the
problem is no longer classified as dynamic programming.

The subproblem graph for the Fibonacci sequence. The fact that it is not
a treeindicates overlapping subproblems.This can be achieved in either of two
waysTop-down approach: This is the direct fall-out of the recursive formulation
of any problem. If the solution to any problem can be formulated recursively
using the solution to its subproblems, and if its subproblems are overlapping, then
one can easily memoize or store the solutions to the subproblems in a table.
Whenever we attempt to solve a new subproblem, we first check the table to see if
it is already solved. If a solution has been recorded, we can use it directly,
otherwise we solve the subproblem and add its solution to the table.
Bottom-up approach: This is the more interesting case. Once we formulate the
solution to a problem recursively as in terms of its subproblems, we can try
reformulating the problem in a bottom-up fashion: try solving the subproblems
first and use their solutions to build-on and arrive at solutions to bigger
subproblems. This is also usually done in a tabular form by iteratively generating
solutions to bigger and bigger subproblems by using the solutions to small

DATA STRUCTURES AND COMPUTER ALGORITHMS

 subproblems. For example, if we already know the values of F41 and F40, we can
directly calculate the value of F42.
Some programming languages can automatically memoize the result of a function
call with a particular set of arguments, in order to speed up call-by-
name evaluation (this mechanism is referred to as call-by-need). Some languages
make it possible portably (e.g. Scheme, Common Lisp or Perl), some need special
extensions (e.g. C++, see [6]). Some languages have automatic memoization built
in, such as tabled Prolog. In any case, this is only possible for a referentially
transparent function.
Example: Mathematical optimization
Optimal consumption and savingA mathematical optimization problem that is often used
in teaching dynamic programming to economists (because it can be solved by hand[7])
concerns a consumer who lives over the periods t = 0,1,2,...,T and must decide how much
to consume and how much to save in each period.
Let ct be consumption in period t, and assume consumption yields utility u(ct) = ln(ct) as
long as the consumer lives. Assume the consumer is impatient, so that he discounts future
utility by a factor b each period, where 0 < b < 1. Let kt be capital in period t. Assume
initial capital is a given amount k0 > 0, and suppose that this period's capital and

consumption determine next period's capital as , where A is a

positive constant and 0 < a < 1. Assume capital cannot be negative. Then the consumer's
decision problem can be written as follows:

subject to for all t = 0,1,2,...,T

Written this way, the problem looks complicated, because it involves solving for all the
choice variables c0,c1,c2,...,cT andk1,k2,k3,...,kT + 1 simultaneously. (Note that k0 is not a
choice variable—the consumer's initial capital is taken as given.)
The dynamic programming approach to solving this problem involves breaking it apart
into a sequence of smaller decisions. To do so, we define a sequence of value
functions Vt(k), for t = 0,1,2,...,T,T + 1 which represent the value of having any amount
of capital k at each time t. Note that VT + 1(k) = 0, that is, there is (by assumption) no
utility from having capital after death.
DATA STRUCTURES AND COMPUTER ALGORITHMS
The value of any quantity of capital at any previous time can be calculated by backward
induction using the Bellman equation. In this problem, for each t = 0,1,2,...,T, the
Bellman equation is

subject to
This problem is much simpler than the one we wrote down before, because it involves
only two decision variables, ct and kt + 1. Intuitively, instead of choosing his whole
lifetime plan at birth, the consumer can take things one step at a time. At time t, his
current capital kt is given, and he only needs to choose current consumption ct and
saving kt + 1.
To actually solve this problem, we work backwards. For simplicity, the current level of
capital is denoted as k. VT + 1(k) is already known, so using the Bellman equation once we
can calculate VT(k), and so on until we get to V0(k), which is the value of the initial
decision problem for the whole lifetime. In other words, once we know VT − j + 1(k), we
can calculate VT − j(k), which is the maximum of ln(cT − j) + bVT − j + 1(Aka − cT − j),

where cT − j is the choice variable and .

Working backwards, it can be shown that the value function at time t = T − j is

where each vT − j is a constant, and the optimal amount to consume at time t = T − j is

which can be simplified to

, and , and , etc.

We see that it is optimal to consume a larger fraction of current wealth as one gets older,
finally consuming all remaining wealth in period T, the last period of life.
Examples: Computer algorithms
Dijkstra's algorithm for the shortest path problem

DATA STRUCTURES AND COMPUTER ALGORITHMS

From a dynamic programming point of view, Dijkstra's algorithm for the shortest path
problem is a successive approximation scheme that solves the dynamic programming
functional equation for the shortest path problem by the Reaching method.[8][9] [10]
In fact, Dijkstra's explanation of the logic behind the algorithm,[11] namely
Problem 2. Find the path of minimum total length between two given nodes P and Q.
We use the fact that, if R is a node on the minimal path from P to Q, knowledge of the
latter implies the knowledge of the minimal path from P to R.
is a paraphrasing of Bellman's famous Principle of Optimality in the context of
the shortest path problem.
General method

Fibonacci sequence
Here is a naïve implementation of a function finding the nth member of the Fibonacci
sequence, based directly on the mathematical definition:
function fib(n)
if n = 0 return 0
if n = 1 return 1
return fib(n − 1) + fib(n − 2)
Notice that if we call, say, fib(5), we produce a call tree that calls the function on the
same value many different times:
fib(5)
fib(4) + fib(3)
(fib(3) + fib(2)) + (fib(2) + fib(1))
((fib(2) + fib(1)) + (fib(1) + fib(0))) + ((fib(1) + fib(0)) + fib(1))
(((fib(1) + fib(0)) + fib(1)) + (fib(1) + fib(0))) + ((fib(1) + fib(0))
+ fib(1))
In particular, fib(2) was calculated three times from scratch. In larger examples, many
more values of fib, or subproblems, are recalculated, leading to an exponential time
algorithm.
Now, suppose we have a simple map object, m, which maps each value of fib that has
already been calculated to its result, and we modify our function to use it and update it.
The resulting function requires only O(n) time instead of exponential time:

DATA STRUCTURES AND COMPUTER ALGORITHMS

 var m := map(0 → 0, 1 → 1)
function fib(n)
if map m does not contain key n
m[n] := fib(n − 1) + fib(n − 2)
return m[n]
This technique of saving values that have already been calculated is called memoization;
this is the top-down approach, since we first break the problem into subproblems and
then calculate and store values.
In the bottom-up approach we calculate the smaller values of fib first, then build larger
values from them. This method also uses O(n) time since it contains a loop that repeats n
− 1 times, however it only takes constant (O(1)) space, in contrast to the top-down
approach which requires O(n) space to store the map.
function fib(n)
var previousFib := 0, currentFib := 1
if n = 0
return 0
else if n = 1
return 1
repeat n − 1 times
var newFib := previousFib + currentFib
previousFib := currentFib
currentFib := newFib
return currentFib
In both these examples, we only calculate fib(2) one time, and then use it to calculate
both fib(4) and fib(3), instead of computing it every time either of them is evaluated.
(Note the calculation of the Fibonacci sequence is used to demonstrate dynamic
programming. An O(1) formula exists from which an arbitrary term can be calculated,
which is more efficient than any dynamic programming technique.)
A type of balanced 0-1 matrix
Consider the problem of assigning values, either zero or one, to the positions of
an n × n matrix, n even, so that each row and each column contains exactly n / 2 zeros

DATA STRUCTURES AND COMPUTER ALGORITHMS

and n / 2 ones. We ask how many different assignments there are for a given n. For
example, when n = 4, four possible solutions are

There are at least three possible approaches: brute force, backtracking, and dynamic
programming.
Brute force consists of checking all assignments of zeros and ones and counting those

that have balanced rows and columns (n / 2zeros and n / 2 ones). As there are
possible assignments, this strategy is not practical except maybe up to n = 6.
Backtracking for this problem consists of choosing some order of the matrix elements
and recursively placing ones or zeros, while checking that in every row and column the
number of elements that have not been assigned plus the number of ones or zeros are both
at least n / 2. While more sophisticated than brute force, this approach will visit every
solution once, making it impractical for nlarger than six, since the number of solutions is
already 116963796250 for n = 8, as we shall see.
Dynamic programming makes it possible to count the number of solutions without
visiting them all. Imagine backtracking values for the first row - what information would
we require about the remaining rows, in order to be able to accurately count the solutions
obtained for each first row values? We consider k × n boards, where 1 ≤ k ≤ n,
whose k rows contain n / 2 zeros and n / 2 ones. The function f to which memoization is
applied maps vectors of n pairs of integers to the number of admissible boards
(solutions). There is one pair for each column and its two components indicate
respectively the number of ones and zeros that have yet to be placed in that column. We

seek the value of (n arguments or one

vector ofn elements). The process of subproblem creation involves iterating over every

one of possible assignments for the top row of the board, and going through every
column, subtracting one from the appropriate element of the pair for that column,
depending on whether the assignment for the top row contained a zero or a one at that

DATA STRUCTURES AND COMPUTER ALGORITHMS

position. If any one of the results is negative, then the assignment is invalid and does not
contribute to the set of solutions (recursion stops). Otherwise, we have an assignment for
the top row of the k × n board and recursively compute the number of solutions to the
remaining (k - 1) × n board, adding the numbers of solutions for every admissible
assignment of the top row and returning the sum, which is being memoized. The base
case is the trivial subproblem, which occurs for a 1 × n board. The number of solutions
for this board is either zero or one, depending on whether the vector is a permutation
of n / 2 (0,1) and n / 2 (1,0) pairs or not.
For example, in the two boards shown above the sequences of vectors would be
((2, 2) (2, 2) (2, 2) (2, 2)) ((2, 2) (2, 2) (2, 2) (2, 2)) k=4
0 1 0 1 0 0 1 1

((1, 2) (2, 1) (1, 2) (2, 1)) ((1, 2) (1, 2) (2, 1) (2, 1)) k=3
1 0 1 0 0 0 1 1

((1, 1) (1, 1) (1, 1) (1, 1)) ((0, 2) (0, 2) (2, 0) (2, 0)) k=2
0 1 0 1 1 1 0 0

((0, 1) (1, 0) (0, 1) (1, 0)) ((0, 1) (0, 1) (1, 0) (1, 0)) k=1
1 0 1 0 1 1 0 0

((0, 0) (0, 0) (0, 0) (0, 0)) ((0, 0) (0, 0), (0, 0) (0, 0))
The number of solutions (sequence A058527 in OEIS) is

Links to the Perl source of the backtracking approach, as well as a MAPLE and a C
implementation of the dynamic programming approach may be found among the external
links.
Checkerboard
Consider a checkerboard with n × n squares and a cost-function c(i, j) which returns a
cost associated with square i, j (i being the row, j being the column). For instance (on a 5
× 5 checkerboard),

DATA STRUCTURES AND COMPUTER ALGORITHMS

 5 6 7 4 7 8

4 7 6 1 1 4

3 3 5 7 8 2

2 - 6 7 0 -

1 - - *5* - -

1 2 3 4 5
Thus c(1, 3) = 5
Let us say you had a checker that could start at any square on the first rank (i.e., row) and
you wanted to know the shortest path (sum of the costs of the visited squares are at a
minimum) to get to the last rank, assuming the checker could move only diagonally left
forward, diagonally right forward, or straight forward. That is, a checker on (1,3) can
move to (2,2), (2,3) or (2,4).
5

2 x x x

1 o

1 2 3 4 5
This problem exhibits optimal substructure. That is, the solution to the entire problem
relies on solutions to subproblems. Let us define a function q(i, j) as
q(i, j) = the minimum cost to reach square (i, j)
If we can find the values of this function for all the squares at rank n, we pick the
minimum and follow that path backwards to get the shortest path.
Note that q(i, j) is equal to the minimum cost to get to any of the three squares below it
(since those are the only squares that can reach it) plus c(i, j). For instance:
5

4 A

DATA STRUCTURES AND COMPUTER ALGORITHMS

 3 B C D

1 2 3 4 5
q(A) = min(q(B),q(C),q(D)) + c(A)
Now, let us define q(i, j) in somewhat more general terms:

The first line of this equation is there to make the recursive property simpler (when
dealing with the edges, so we need only one recursion). The second line says what
happens in the last rank, to provide a base case. The third line, the recursion, is the
important part. It is similar to the A,B,C,D example. From this definition we can make a
straightforward recursive code for q(i, j). In the following pseudocode, n is the size of the
board, c(i, j) is the cost-function, and min() returns the minimum of a number of
values:
function minCost(i, j)
if j < 1 or j > n
return infinity
else if i = 1
return c(i, j)
else
return min( minCost(i+1, j-1), minCost(i+1, j), minCost(i+1, j+1) ) + c(i, j)
It should be noted that this function only computes the path-cost, not the actual path. We
will get to the path soon. This, like the Fibonacci-numbers example, is horribly slow
since it spends mountains of time recomputing the same shortest paths over and over.
However, we can compute it much faster in a bottom-up fashion if we store path-costs in
a two-dimensional array q[i, j] rather than using a function. This avoids

DATA STRUCTURES AND COMPUTER ALGORITHMS

recomputation; before computing the cost of a path, we check the array q[i, j] to see if
the path cost is already there.
We also need to know what the actual shortest path is. To do this, we use another
array p[i, j], a predecessor array. This array implicitly stores the path to any
square s by storing the previous node on the shortest path to s, i.e. the predecessor. To
reconstruct the path, we lookup the predecessor of s, then the predecessor of that square,
then the predecessor of that square, and so on, until we reach the starting square.
Consider the following code:
function computeShortestPathArrays()
for x from 1 to n
q[1, x] := c(1, x)
for y from 1 to n
q[y, 0] := infinity
q[y, n + 1] := infinity
for y from 2 to n
for x from 1 to n
m := min(q[y-1, x-1], q[y-1, x], q[y-1, x+1])
q[y, x] := m + c(y, x)
if m = q[y-1, x-1]
p[y, x] := -1
else if m = q[y-1, x]
p[y, x] := 0
else
p[y, x] := 1
Now the rest is a simple matter of finding the minimum and printing it.
function computeShortestPath()
computeShortestPathArrays()
minIndex := 1
min := q[n, 1]
for i from 2 to n
if q[n, i] < min

DATA STRUCTURES AND COMPUTER ALGORITHMS

 minIndex := i
min := q[n, i]
printPath(n, minIndex)
function printPath(y, x)
print(x)
print("<-")
if y = 2
print(x + p[y, x])
else
printPath(y-1, x + p[y, x])
Sequence alignment
In genetics, sequence alignment is an important application where dynamic programming
is essential.[3] Typically, the problem consists of transforming one sequence into another
using edit operations that replace, insert, or remove an element. Each operation has an
associated cost, and the goal is to find the sequence of edits with the lowest total cost.
The problem can be stated naturally as a recursion, a sequence A is optimally edited into
a sequence B by either:
inserting the first character of B, and performing an optimal alignment of A and the tail
of B
deleting the first character of A, and performing the optimal alignment of the tail of A
and B
replacing the first character of A with the first character of B, and performing optimal
alignments of the tails of A and B.
The partial alignments can be tabulated in a matrix, where cell (i,j) contains the cost of
the optimal alignment of A[1..i] to B[1..j]. The cost in cell (i,j) can be calculated by
adding the cost of the relevant operations to the cost of its neighboring cells, and
selecting the optimum.
Different variants exist, see Smith-Waterman and Needleman-Wunsch.

DATA STRUCTURES AND COMPUTER ALGORITHMS

Multistage Graphs

A multistage graph is a graph

o G=(V,E) with V partitioned into K >= 2 disjoint subsets such that if (a,b) is in E,
then a is in Vi , and b is in Vi+1 for some subsets in the partition;
o and | V1 | = | VK | = 1.

The vertex s in V1 is called the source; the vertex t in VK is called the sink.

G is usually assumed to be a weighted graph.

The cost of a path from node v to node w is sum of the costs of edges in the path.

The "multistage graph problem" is to find the minimum cost path from s to t.

[Cf. the "network flow problem".]

Each set Vi is called a stage in the graph.

Consider the resource allocation problem:

Given n units of resources to be allocated to k projects.

For 1 <= i <= k, 0 <= j <= n,

P(i,j) = profit obtained by allocating "j" units of

the resource to project i.

Transform this to instance of "multistage graph problem".

Create a multistage graph:

V = {s} and denote s = V(1,0) -- read, we are at node 1 having

0 allocated 0 units of resource

DATA STRUCTURES AND COMPUTER ALGORITHMS

Stages 1 to k are such that stage i consists of a set:

{ V(i+1,j) } j=0 .. n

[we could denote the vertices in this set as: vi+1j

[or could instead call them vj of set Vi]

The edges are weighted with C(i,j) = -P(i,j) [the negative of the profit] to make it a
minimization problem.

Dynamic Programming solution:

Let path(i,j) be some specification of the minimal path from vertex j in set i to
vertex t; C(i,j) is the cost of this path;c(j,t) is the weight of the edge from j to t.

C(i,j) = min { c(j,l) + C(i+1,l) }

l in Vi+1

(j,l) in E

To write a simple algorithm, assign numbers to the vertices so those in stage Vi have
lower number those in stage Vi+1.

int[] MStageForward(Graph G)
{
// returns vector of vertices to follow through the graph
// let c[i][j] be the cost matrix of G

int n = G.n (number of nodes);

int k = G.k (number of stages);
float[] C = new float[n];

DATA STRUCTURES AND COMPUTER ALGORITHMS

 int[] D = new int[n];
int[] P = new int[k];
for (i = 1 to n) C[i] = 0.0;
for j = n-1 to 1 by -1 {
r = vertex such that (j,r) in G.E and c(j,r)+C(r) is minimum
C[j] = c(j,r)+C(r);
D[j] = r;
}
P[1] = 1; P[k] = n;
for j = 2 to k-1 {
P[j] = D[P[j-1]];
}
return P;
}
All pair shortest path

All pairs Shortest Paths Problem: Floyd's Algorithm

Floyd's Algorithm

This is an O(n3) algorithm, where n is the number of vertices in the digraph.

Uses the principle of Dynamic Programming
Let V = {1, 2,..., n}. The algorithm uses a matrix A[1..n][1..n] to compute the
lengths of the shortest paths.
Initially,

A[i, j] = C[i, j] if i j
=0 if i = j

Note that C[i, j] is taken as if there is no directed arc from ito j.

DATA STRUCTURES AND COMPUTER ALGORITHMS

The algorithm makes n passes over A. Let A0, A1,..., Anbe the values of A on
the n passes, with A0 being the initial value. Just after the k-1th iteration,

The kth pass explores whether the vertex k lies on an optimal path from i to j,
i, j

Principle of Floyd's algorithm

We use

Ak[i, j] = min

The algorithm:

void Floyd (floatC[n - 1][n - 1], A[n - 1][n - 1])

{ int i, j, k;

{ for(i = 0, i n - 1;i + +)

for(j = 0;j n - 1, j + +)

DATA STRUCTURES AND COMPUTER ALGORITHMS

 A[i, j] = C[i, j];

for(i = 0;i n - 1;i + +)

A[i, i] = 0;

for(k = 0;k n - 1;k + +);

for(i = 0;i n - 1;i + +)

for(j = 0;j n - 1, j + +)

if(A[i, k] + A[k, j] < A[i, j])

A[i, j] = A[i, k] + A[k, j]

DATA STRUCTURES AND COMPUTER ALGORITHMS

C=

A0 = ; A1 = A2

= ; A3 =

With adjacency matrix representation, Floyd's algorithm has a worst case

complexity of O(n3) where n is the number of vertices
If Dijkstra's algorithm is used for the same purpose, then with an adjacency list
representation, the worst case complexity will be O(nelog n). Thus if eis O(n2),
then the complexity will be O(n3log n) while if e is O(n), then the complexity
is O(n2log n).

Optimal binary search trees

If we don't plan on modifying a search tree, and we know exactly how often each item
will be accessed, we can construct an optimal binary search tree, which is a search tree
where the average cost of looking up an item (the expected search cost) is minimized.

Even if we only have estimates of the search costs, such a system can considerably speed
up lookups on average. For example, if you have a BST of English words used in a spell
checker, you might balance the tree based on word frequency in text corpora, placing
words like "the" near the root and words like "agerasia" near the leaves. Such a tree might
be compared with Huffman trees, which similarly seek to place frequently-used items

DATA STRUCTURES AND COMPUTER ALGORITHMS

near the root in order to produce a dense information encoding; however, Huffman trees
only store data elements in leaves and these elements need not be ordered.

If we do not know the sequence in which the elements in the tree will be accessed in
advance, we can use splay trees which are asymptotically as good as any static search tree
we can construct for any particular sequence of lookup operations.

Alphabetic trees are Huffman trees with the additional constraint on order, or,
equivalently, search trees with the modification that all elements are stored in the leaves.
Faster algorithms exist for optimal alphabetic binary trees (OABTs).

Example:

procedure Optimum Search Tree(f, f´, c):

for j = 0 to n do
c[j, j] = 0, F[j, j] = f´j
for d = 1 to n do
for i = 0 to (n − d) do
j=i+d
F[i, j] = F[i, j − 1] + f´ + f´j
c[i, j] = MIN(i<k<=j){c[i, k − 1] + c[k, j]} + F[i, j]

0/1 Knapsack reliability Design

Let i be the highest-numbered item in an optimal solution S for W pounds. Then S`= S -
{i} is an optimal solution for W-wipounds and the value to the solution S is Vi plus the
value of the subproblem.

We can express this fact in the following formula: define c[i, w] to be the solution for
items 1,2, . . . , i and maximum weight w. Then

0 if i = 0 or w = 0

DATA STRUCTURES AND COMPUTER ALGORITHMS

c[i,w] = c[i-1, w] if wi ≥ 0
max [vi + c[i-1, w-wi],c[i-1, w]} if i>0 and w ≥wi

This says that the value of the solution to i items either include ith item, in which case it
is vi plus a subproblem solution for (i-1) items and the weight excluding wi, or does not
include ith item, in which case it is a subproblem's solution for (i-1) items and the same
weight. That is, if the thief picks item i, thief takes vi value, and thief can choose from
items w-wi, and get c[i-1, w-wi] additional value. On other hand, if thief decides not to
take item i, thief can choose from item 1,2, . . . , i-1 upto the weight limitw, and get c[i-
1, w] value. The better of these two choices should be made.Although the 0-1 knapsack
problem, the above formula for c is similar to LCS formula: boundary values are 0, and
other values are computed from the input and "earlier" values of c. So the 0-1 knapsack
algorithm is like the LCS-length algorithm given in CLR-book for finding a longest
common subsequence of two sequences.

The algorithm takes as input the maximum weight W, the number of items n, and the two
sequences v = <v1, v2, . . . , vn> and w = <w1, w2, . . . , wn>. It stores the c[i, j] values in
the table, that is, a two dimensional array, c[0 . . n, 0 . . w] whose entries are computed in
a row-major order. That is, the first row of c is filled in from left to right, then the second
row, and so on. At the end of the computation, c[n, w] contains the maximum value that
can be picked into the knapsack.

Dynamic-0-1-knapsack (v, w, n, W)

for w = 0 to W
do c[0, w] = 0
for i = 1 to n
do c[i, 0] = 0
for w = 1 to W
do if wi ≤ w
then if vi + c[i-1, w-wi]
then c[i, w] = vi + c[i-1, w-wi]

DATA STRUCTURES AND COMPUTER ALGORITHMS

 else c[i, w] = c[i-1, w]
else
c[i, w] = c[i-1, w]

The set of items to take can be deduced from the table, starting at c[n. w] and tracing
backwards where the optimal values came from. If c[i, w] = c[i-1, w] item i is not part of
the solution, and we are continue tracing with c[i-1, w]. Otherwise item i is part of the
solution, and we continue tracing with c[i-1, w-W].

Analysis

This dynamic-0-1-kanpsack algorithm takes θ(nw) times, broken up as follows:

θ(nw) times to fill the c-table, which has (n+1).(w+1) entries, each requiring θ(1) time to
compute. O(n) time to trace the solution, because the tracing process starts in row n of the
table and moves up 1 row at each step.

Traveling salesman problem

The travelling salesman problem (TSP) is an NP-hard problem in combinatorial

optimization studied in operations research andtheoretical computer science. Given a list
of cities and their pairwise distances, the task is to find a shortest possible tour that visits
each city exactly once.

The problem was first formulated as a mathematical problem in 1930 and is one of the
most intensively studied problems in optimization. It is used as a benchmark for many
optimization methods. Even though the problem is computationally difficult, a large
number of heuristics and exact methods are known, so that some instances with tens of
thousands of cities can be solved.

The TSP has several applications even in its purest formulation, such
as planning, logistics, and the manufacture of microchips. Slightly modified, it appears as
a sub-problem in many areas, such as DNA sequencing. In these applications, the
concept cityrepresents, for example, customers, soldering points, or DNA fragments, and

DATA STRUCTURES AND COMPUTER ALGORITHMS

the concept distance represents travelling times or cost, or a similarity measure between
DNA fragments. In many applications, additional constraints such as limited resources or
time windows make the problem considerably harder.

In the theory of computational complexity, the decision version of the TSP (where, given
a length L, the task is to decide whether any tour is shorter than L) belongs to the class
of NP-complete problems. Thus, it is likely that the worst case running time for any
algorithm for the TSP increases exponentially with the number of cities.

The origins of the travelling salesman problem are unclear. A handbook for travelling
salesmen from 1832 mentions the problem and includes example tours through Germany
and Switzerland, but contains no mathematical treatment.

the travelling salesman problem was defined in the 1800s by the Irish mathematician W.
R. Hamilton and by the British mathematician Thomas Kirkman. Hamilton‘s Icosian
Gamewas a recreational puzzle based on finding a Hamiltonian cycle.[2] The general form
of the TSP appears to have been first studied by mathematicians during the 1930s in
Vienna and at Harvard, notably by Karl Menger, who defines the problem, considers the
obvious brute-force algorithm, and observes the non-optimality of the nearest neighbour
heuristic:

Hassler Whitney at Princeton University introduced the name travelling salesman

problemsoon after In the 1950s and 1960s, the problem became increasingly popular in
scientific circles in Europe and the USA. Notable contributions were made by George
Dantzig, Delbert Ray Fulkerson and Selmer M. Johnson at the RAND
Corporation in Santa Monica, who expressed the problem as an integer linear
program and developed the cutting plane method for its solution. With these new
methods they solved an instance with 49 cities to optimality by constructing a tour and
proving that no other tour could be shorter. In the following decades, the problem was
studied by many researchers from mathematics, computer science, chemistry, physics,
and other sciences.

Richard M. Karp showed in 1972 that the Hamiltonian cycle problem was NP-complete,
which implies the NP-hardness of TSP. This supplied a mathematical explanation for the
apparent computational difficulty of finding optimal tours.Great progress was made in the
DATA STRUCTURES AND COMPUTER ALGORITHMS
late 1970s and 1980, when Grötschel, Padberg, Rinaldi and others managed to exactly
solve instances with up to 2392 cities, using cutting planes and branch-and-bound.

In the 1990s, Applegate, Bixby, Chvátal, and Cook developed the program Concorde that
has been used in many recent record solutions. Gerhard Reinelt published the TSPLIB in
1991, a collection of benchmark instances of varying difficulty, which has been used by
many research groups for comparing results. In 2005, Cook and others computed an
optimal tour through a 33,810-city instance given by a microchip layout problem,
currently the largest solved TSPLIB instance. For many other instances with millions of
cities, solutions can be found that are guaranteed to be within 1% of an optimal tour.

TSP can be modeled as an undirected weighted graph, such that cities are the
graph'svertices, paths are the graph's edges, and a path's distance is the edge's length. A
TSP tour becomes a Hamiltonian cycle if and only if every edge has the same distance.
Often, the model is a complete graph (i.e., an edge connects each pair of vertices). If no
path exists between two cities, adding an arbitrarily long edge will complete the graph
without affecting the optimal tour.

Asymmetric and symmetric

In the symmetric TSP, the distance between two cities is the same in each opposite
direction, forming an undirected graph. This symmetry halves the number of possible
solutions. In the asymmetric TSP, paths may not exist in both directions or the distances
might be different, forming a directed graph. Traffic collisions, one-way streets, and
airfares for cities with different departure and arrival fees are examples of how this
symmetry could break down.

DATA STRUCTURES AND COMPUTER ALGORITHMS

Related problems

 An equivalent formulation in terms of graph theory is: Given a complete weighted

graph (where the vertices would represent the cities, the edges would represent the
roads, and the weights would be the cost or distance of that road), find a Hamiltonian
cyclewith the least weight.

 The requirement of returning to the starting city does not change

the computational complexity of the problem, see Hamiltonian path problem.

 Another related problem is the bottleneck travelling salesman problem (bottleneck

TSP): Find a Hamiltonian cycle in a weighted graph with the minimal weight of the
weightiest edge. The problem is of considerable practical importance, apart from
evident transportation and logistics areas. A classic example is in printed
circuit manufacturing: scheduling of a route of the drill machine to drill holes in a
PCB. In robotic machining or drilling applications, the "cities" are parts to machine or
holes (of different sizes) to drill, and the "cost of travel" includes time for retooling
the robot (single machine job sequencing problem).

 The generalized travelling salesman problem deals with "states" that have (one or
more) "cities" and the salesman has to visit exactly one "city" from each "state". Also
known as the "travelling politician problem". One application is encountered in
ordering a solution to the cutting stock problem in order to minimise knife changes.
Another is concerned with drilling in semiconductor manufacturing, see e.g. U.S.
Patent 7,054,798. Surprisingly, Behzad and Modarres[5] demonstrated that the
generalised travelling salesman problem can be transformed into a standard travelling
salesman problem with the same number of cities, but a modified distance matrix.

 The sequential ordering problem deals with the problem of visiting a set of cities
where precedence relations between the cities exist.

 The travelling purchaser problem deals with a purchaser who is charged with
purchasing a set of products. He can purchase these products in several cities, but at

DATA STRUCTURES AND COMPUTER ALGORITHMS

 different prices and not all cities offer the same products. The objective is to find a
route between a subset of the cities, which minimizes total cost (travel cost +
purchasing cost) and which enables the purchase of all required products.

The traditional lines of attack for the NP-hard problems are the following:

 Devising algorithms for finding exact solutions (they will work reasonably fast
only for small problem sizes).
 Devising "suboptimal" or heuristic algorithms, i.e., algorithms that deliver either
seemingly or probably good solutions, but which could not be proved to be optimal.
 Finding special cases for the problem ("subproblems") for which either better or
exact heuristics are possible.
Computational complexity

The problem has been shown to be NP-hard (more precisely, it is complete for
the complexity class FPNP; see function problem), and the decision problem version
("given the costs and a number x, decide whether there is a round-trip route cheaper
than x") is NP-complete. The bottleneck travelling salesman problem is also NP-hard.
The problem remains NP-hard even for the case when the cities are in the plane
with Euclidean distances, as well as in a number of other restrictive cases. Removing the
condition of visiting each city "only once" does not remove the NP-hardness, since it is
easily seen that in the planar case there is an optimal tour that visits each city only once
(otherwise, by the triangle inequality, a shortcut that skips a repeated visit would not
increase the tour length).

Complexity of approximation

In the general case, finding a shortest travelling salesman tour is NPO-complete. If the
distance measure is a metric and symmetric, the problem becomes APX-
complete and Christofides‘s algorithm approximates it within 1.5. If the distances are
restricted to 1 and 2 (but still are a metric) the approximation ratio becomes 7/6. In the
asymmetric, metric case, only logarithmic performance guarantees are known, the best
current algorithm achieves performance ratio 0.814 log n; it is an open question if a
constant factor approximation exists.

DATA STRUCTURES AND COMPUTER ALGORITHMS

The corresponding maximization problem of finding the longest travelling salesman tour
is approximable within 63/38. If the distance function is symmetric, the longest tour can
be approximated within 4/3 by a deterministic algorithm and within(33 + ε) / 25 by a
randomised algorithm.

Exact algorithms

The most direct solution would be to try all permutations (ordered combinations) and see
which one is cheapest (using brute force search). The running time for this approach lies
within a polynomial factor of O(n!), the factorial of the number of cities, so this solution
becomes impractical even for only 20 cities. One of the earliest applications of dynamic
programming is the Held–Karp algorithm that solves the problem in time O(n22n).

The dynamic programming solution requires exponential space. Using inclusion–

exclusion, the problem can be solved in time within a polynomial factor of 2n and
polynomial space.

Improving these time bounds seems to be difficult. For example, it is an open problem if
there exists an exact algorithm for TSP that runs in time O(1.9999n)

Other approaches include:

 Various branch-and-bound algorithms, which can be used to process TSPs

containing 40–60 cities.
 Progressive improvement algorithms which use techniques reminiscent of linear
programming. Works well for up to 200 cities.
 Implementations of branch-and-bound and problem-specific cut generation; this is
the method of choice for solving large instances. This approach holds the current
record, solving an instance with 85,900 cities, see Applegate et al. (2006).

An exact solution for 15,112 German towns from TSPLIB was found in 2001 using
the cutting-plane method proposed by George Dantzig, Ray Fulkerson, and Selmer
Johnson in 1954, based on linear programming. The computations were performed on a
network of 110 processors located at Rice University and Princeton University (see the

DATA STRUCTURES AND COMPUTER ALGORITHMS

Princeton external link). The total computation time was equivalent to 22.6 years on a
single 500 MHz Alpha processor. In May 2004, the travelling salesman problem of
visiting all 24,978 towns in Sweden was solved: a tour of length approximately 72,500
kilometers was found and it was proven that no shorter tour exists. In March 2005, the
travelling salesman problem of visiting all 33,810 points in a circuit board was solved
using Concorde TSP Solver: a tour of length 66,048,945 units was found and it was
proven that no shorter tour exists. The computation took approximately 15.7 CPU-years
(Cook et al. 2006). In April 2006 an instance with 85,900 points was solved
using Concorde TSP Solver, taking over 136 CPU-years, see Applegate et al. (2006).

Heuristic and approximation algorithms

Various heuristics and approximation algorithms, which quickly yield good solutions
have been devised. Modern methods can find solutions for extremely large problems
(millions of cities) within a reasonable time which are with a high probability just 2–3%
away from the optimal solution.

Several categories of heuristics are recognized.

Constructive heuristics

The nearest neighbour (NN) algorithm (or so-called greedy algorithm) lets the salesman
choose the nearest unvisited city as his next move. This algorithm quickly yields an
effectively short route. For N cities randomly distributed on a plane, the algorithm on
average yields a path 25% longer than the shortest possible path.[17] However, there exist
many specially arranged city distributions which make the NN algorithm give the worst
route (Gutin, Yeo, and Zverovich, 2002). This is true for both asymmetric and symmetric
TSPs (Gutin and Yeo, 2007). Rosenkrantz et al. [1977] showed that the NN algorithm has
the approximation factor Θ(log | V | ) for instances satisfying the triangle inequality.

Constructions based on a minimum spanning tree have an approximation ratio of 2.

The Christofides algorithm achieves a ratio of 1.5.

The bitonic tour of a set of points is the minimum-perimeter monotone polygon that has
the points as its vertices; it can be computed efficiently by dynamic programming.

DATA STRUCTURES AND COMPUTER ALGORITHMS

Another constructive heuristic, Match Twice and Stitch (MTS) (Kahng, Reda 2004 [18]),
performs two sequential matchings, where the second matching is executed after deleting
all the edges of the first matching, to yield a set of cycles. The cycles are then stitched to
produce the final tour.

Iterative improvement

Pairwise exchange, or Lin–Kernighan heuristics

The pairwise exchange or 2-opt technique involves iteratively removing two
edges and replacing these with two different edges that reconnect the fragments
created by edge removal into a new and shorter tour. This is a special case of
the k-opt method. Note that the label Lin–Kernighan is an often heard misnomer
for 2-opt. Lin–Kernighan is actually a more general method.
k-opt heuristic
Take a given tour and delete k mutually disjoint edges. Reassemble the remaining
fragments into a tour, leaving no disjoint subtours (that is, don't connect a
fragment's endpoints together). This in effect simplifies the TSP under
consideration into a much simpler problem. Each fragment endpoint can be
connected to 2k − 2 other possibilities: of 2k total fragment endpoints available,
the two endpoints of the fragment under consideration are disallowed. Such a
constrained 2k-city TSP can then be solved with brute force methods to find the
least-cost recombination of the original fragments. The k-opt technique is a
special case of the V-opt or variable-opt technique. The most popular of the k-opt
methods are 3-opt, and these were introduced by Shen Lin of Bell Labs in 1965.
There is a special case of 3-opt where the edges are not disjoint (two of the edges
are adjacent to one another). In practice, it is often possible to achieve substantial
improvement over 2-opt without the combinatorial cost of the general 3-opt by
restricting the 3-changes to this special subset where two of the removed edges
are adjacent. This so-called two-and-a-half-opt typically falls roughly midway
between 2-opt and 3-opt, both in terms of the quality of tours achieved and the
time required to achieve those tours.
V-opt heuristic

DATA STRUCTURES AND COMPUTER ALGORITHMS

The variable-opt method is related to, and a generalization of the k-opt method.
Whereas the k-opt methods remove a fixed number (k) of edges from the original
tour, the variable-opt methods do not fix the size of the edge set to remove.
Instead they grow the set as the search process continues. The best known method
in this family is the Lin–Kernighan method (mentioned above as a misnomer for
2-opt). Shen Lin and Brian Kernighan first published their method in 1972, and it
was the most reliable heuristic for solving travelling salesman problems for nearly
two decades. More advanced variable-opt methods were developed at Bell Labs in
the late 1980s by David Johnson and his research team. These methods
(sometimes called Lin–Kernighan–Johnson) build on the Lin–Kernighan method,
adding ideas from tabu search and evolutionary computing. The basic Lin–
Kernighan technique gives results that are guaranteed to be at least 3-opt. The
Lin–Kernighan–Johnson methods compute a Lin–Kernighan tour, and then
perturb the tour by what has been described as a mutation that removes at least
four edges and reconnecting the tour in a different way, then v-opting the new
tour. The mutation is often enough to move the tour from the local
minimum identified by Lin–Kernighan. V-opt methods are widely considered the
most powerful heuristics for the problem, and are able to address special cases,
such as the Hamilton Cycle Problem and other non-metric TSPs that other
heuristics fail on. For many years Lin–Kernighan–Johnson had identified optimal
solutions for all TSPs where an optimal solution was known and had identified
the best known solutions for all other TSPs on which the method had been tried.
Randomised improvement

Optimized Markov chain algorithms which use local searching heuristic

sub-algorithms can find a route extremely close to the optimal route for
700 to 800 cities.

TSP is a touchstone for many general heuristics devised for combinatorial

optimization such as genetic algorithms, simulated annealing, Tabu
search, ant colony optimization, river formation dynamics (see swarm
intelligence) and the cross entropy method.

DATA STRUCTURES AND COMPUTER ALGORITHMS

 Ant colony optimization

Artificial intelligence researcher Marco Dorigo described in 1997 a

method of heuristically generating "good solutions" to the TSP using
a simulation of an ant colony called ACS (Ant Colony System).[19] It
models behavior observed in real ants to find short paths between food
sources and their nest, an emergent behaviour resulting from each ant's
preference to follow trail pheromones deposited by other ants.

ACS sends out a large number of virtual ant agents to explore many
possible routes on the map. Each ant probabilistically chooses the next city
to visit based on a heuristic combining the distance to the city and the
amount of virtual pheromone deposited on the edge to the city. The ants
explore, depositing pheromone on each edge that they cross, until they
have all completed a tour. At this point the ant which completed the
shortest tour deposits virtual pheromone along its complete tour route
(global trail updating). The amount of pheromone deposited is inversely
proportional to the tour length: the shorter the tour, the more it deposits.

Special cases

Metric TSP

In the metric TSP, also known as delta-TSP or Δ-TSP, the intercity

distances satisfy the triangle inequality.

DATA STRUCTURES AND COMPUTER ALGORITHMS

 A very natural restriction of the TSP is to require that the distances
between cities form a metric, i.e., they satisfy the triangle inequality. This
can be understood as the absence of "shortcuts", in the sense that the direct
connection from A to B is never longer than the route via intermediate C:

The edge lengths then form a metric on the set of vertices. When the
cities are viewed as points in the plane, many natural distance
functions are metrics, and so many natural instances of TSP satisfy
this constraint.

The following are some examples of metric TSPs for various metrics.

 In the Euclidean TSP (see below) the distance between two cities
is the Euclidean distance between the corresponding points.
 In the rectilinear TSP the distance between two cities is the sum of
the differences of their x- and y-coordinates. This metric is often
called the Manhattan distance or city-block metric.
 In the maximum metric, the distance between two points is the
maximum of the absolute values of differences of their x- and y-
coordinates.
Flow shop scheduling problems

Flow Shop Scheduling Problems, or FSPs, are a class of scheduling problems with a
work shop or group shop in which the flow control shall enable an appropriate
sequencing for each job and for processing on a set of machines or with other resources
1,2,...,m in compliance with given processing orders. Especially the maintaining of a
continuous flow of processing tasks is desired with a minimum of idle time and a
minimum of waiting time. FSP may apply as well to production facilities as to computing
designs.

DATA STRUCTURES AND COMPUTER ALGORITHMS

A special type of FSPs is the Permutation Flow Shop Scheduling Problem, or PFSP, in
which the processing order of the jobs on the resources is the same for each subsequent
step of processing.
MINIMUM FLOW-SHOP SCHEDULING

INSTANCE: Number of processors, set J of jobs, each consisting of m operations with

(with to be executed by processor i), and for each operation a length .

SOLUTION: An open-shop schedule for

MEASURE: The completion time of the schedule, i.e., .

Good News: Approximable within m/2 if m is even and within m/2 + 1/6 if m is odd
[104].
Bad News: Not approximable within 5/4 for any [466].
Comment: Approximable within 5/3 if m=3 [104]. Variation in which m=2, but the two
processors are replaced by respectively and identical parallel processors, is approximable
within [103].
Garey and Johnson: SS15see MINIMUM OPEN-SHOP SCHEDULING) such that, for
each and , .MEASURE: The completion time of the schedule.

DATA STRUCTURES AND COMPUTER ALGORITHMS