Course Code – CSE : 371

LONGEST COMMON SUBSEQUENCE

Using LTDP and Rank Convergence

B.TECH PART – III

Submitted By:

Harpreet Singh 14075029

Saurav Kumar 14075047
Vivek Sourabh 14075059
Isha Agarwal 14075063
Under the guidance of

Dr. Ravindranath Chowdary

Department of Computer Science & Engineering

Indian Institute of Technology (Banaras Hindu University)
Varanasi – 221005 Uttar Pradesh
1|Page
 Index

1. Abstract 3

2. Introduction 4

3. Hirschberg’s Algorithm 5

4. Linear Tropical Dynamic Programming 6

5. Parallel Implementation

6. Enhancement

11

7. Evaluation 12

8. Theoretical Analysis 13
2|Page
 9. References 14

Abstract

We implemented a parallel algorithm to find the Longest Common

Subsequence using the concepts of Linear Tropical Dynamic
Programming(LTDP) and Rank Convergence. We implemented the
program in four ways:

1. Hirschberg’s Algorithm(Serial Algorithm)

2. Multi-Threaded program using OpenMP
3. Single Node Program using OpenMPI
4. Multi Node Program using OpenMPI

These approaches were implemented and compared. To improve the

performance of the algorithm we paralleized the computation of each
element in a given wavefront.

3|Page
 1. Introduction

The longest common subsequence (LCS) problem is the problem of

finding the longest subsequence common to all sequences in a set of
sequences (often just two sequences). It differs from problems of
finding common substrings: unlike substrings, subsequences are not
required to occupy consecutive positions within the original sequences.
The longest common subsequence problem is a classic computer
science problem, the basis of data comparison programs such as the
diff utility, and has applications in bioinformatics. It is also widely used
by revision control systems such as Git for reconciling multiple changes
made to a revision-controlled collection of files.
The LCS problem has an optimal substructure: the problem can be
broken down into smaller, simple "subproblems", which can be broken
down into yet simpler subproblems, and so on, until, finally, the
solution becomes trivial. The LCS problem also has overlapping
subproblems: the solution to high-level subproblems often reuse lower
level subproblems. Problems with these two properties—optimal
substructure and overlapping subproblems—can be approached by a
problem-solving technique called dynamic programming, in which
subproblem solutions are memoized rather than computed over and
over. The procedure requires memoization—saving the solutions to one
level of subproblem in a table so that the solutions are available to the
next level of subproblems.

4|Page
The LCS is not necessarily unique; for example the LCS of "ABC" and
"ACB" is both "AB" and "AC". Indeed, the LCS problem is often defined
to be finding all common subsequences of a maximum length. This
problem inherently has higher complexity, as the number of such
subsequences is exponential in the worst case, even for only two input
strings.

2. Hirschberg’s Algorithm

It is a linear space recursive algorithm that uses the divide and conquer
approach to solve the LCS problem.
The main highlights of this algorithm are –
1. The key idea is to split one of the input sentences into two.
2. We then find the LCS of Xpart1, Y and Xpart2rev , Yrev.
3. Then we fins a position y-position k that maximizes the sums of
these forward and backward LCS lengths.
4. Then we have a suitable split position for the second string as
well.
5. Thus our problem reduces to finding the answer for the two
smaller subsequences.

5|Page
 3. Linear Tropical Dynamic Programming

Dynamic programming is a method for solving problems that have

optimal substructure — the solution to a problem can be obtained from
the solutions to a set of its overlapping sub problems. This dependence
between sub problems is captured by a recurrence equation. Classic
dynamic programming implementations solve the sub problems
iteratively applying the recurrence equation in an order that respects
the dependence between sub problems. LTDP Definition A dynamic
programming problem is linear-tropical dynamic programming (LTDP),
if (a) the sub problems can be grouped into a sequence of stages such
that the solution to a sub problem in a stage only depends on the
solutions in the previous stage and (b) this dependence is linear in the

6|Page
tropical semiring. In other words, the solution to sub problem j in stage
of LTDP, is given by the recurrence equation

for appropriate constants . This linear dependence allows us to view

LTDP as computing a sequence of vectors, where

for an appropriate matrix of constants derived from the recurrence

equation. In this equation, we will call as the solution vector at stage
and call as the transformation matrix at stage. Also, is the initial
solution vector obtained from the base case of the recurrence
equation.
Once all the sub problems are solved, finding the solution to the
underlying optimization problem of LTDP usually involves tracing the
predecessors of sub problems.
A predecessor of a sub problem is the sub problem for which the
maximum in Equation 1 is reached. For ease of exposition, we define
the
predecessor product of a matrix A and a vector as the vector such that

Rank Convergence: Rank of the product of two matrices is not greater

than the rank of individual matrices. So the rank will converge when the
rank of either matrix is 1.

7|Page
 4. Parallel Implementation

In the parallel implementation the following insight is used to compute

the longest common subsequence. The figure uses three processors as
an example. Figure a represents the forward phase of the sequential
algorithm. Each stage is represented as a vertical column of cells and an
arrow between stages represents a multiplication with an appropriate
transformation matrix. Processor P0 starts from the initial solution
vector s0 and computes all its stages. Processor Pa waits for sa, the
solution vector in the final stage of Pb, to start its computation.
Similarly, processor Pb waits for sb the solution vector at the final stage
of Pa.

8|Page
In the parallel algorithm as shown in figure (b) processors P a and Pb
assume arbitrary solution vector and compute their part of the matrix
based on these solution vectors. Hence, there is no interdependency
between the processors as in the serial version so these processors can
work in parallel. Of course, the solutions for the stages computed by Pa
and Pb will start out as completely wrong (shaded dark in the figure).
However, if rank convergence occurs then these erroneous solution
vectors will eventually become parallel to the actual solution vectors
(shaded gray in the figure). Thus, Pa will generate some solution vector
śb parallel to sb and Pb will generate some solution vector śb parallel to sb .
In a subsequent fix up phase, shown in Figure 2(c), Pa uses computed by
P0 and Pa uses sa computed by P1 to fix stages that are not parallel to the
actual solution vector at that stage. After the fixup, the solution vectors
9|Page
at each stage are either the same as or parallel to the actual solution
vector at those respective stages.
1. Forward Phase (Parallel)
The goal of the parallel forward phase is to compute a solution
vector s[i] at stage i that is parallel to the actual solution vector ⃗si,
as shown in the figure. During the execution of the algorithm, we
say that a stage i has converged if s[i] computed by the algorithm
is parallel to its actual solution vector ⃗si.
When compared to the sequential algorithm, the parallel
algorithm should additionally store s[i] per stage required to test
for convergence in the fix up loop. If space is a constraint, then
the fix up loop can be modified to recompute s[i] in each iteration,
trading compute for space.

2. Backward Phase (Parallel)

Once the parallel forward phase is done, performing the
sequential backward phase from the figure will generate the right
result, even though s[i] is not the same as the correct solution ⃗si.
In many applications, the forward phase overwhelmingly Once the
parallel forward phase is done, performing the sequential
backward phase from Figure 2 will generate the right result, even
though s[i] is not the same as the correct solution ⃗si. In many
applications, the forward phase overwhelmingly dominates the
execution time and parallelizing the backward phase is not
necessary.
dominates the execution time and parallelizing the backward
phase is not necessary.

10 | P a g e
The LCS matrix is computed in an anti-diagonal manner using the
following recurrence relation -

The value of i,j is 1 if the characters at the respective positions are the
same else it is equal to 0.

5. Enhancement

Each diagonal of the matrix is referred to as a wavefront. Each

wavefront has a specific no. of elements. The computation of each
element of a given wavefront is independent of the other values of that
wavefront. So, there exists no dependency among the elements of a
wavefront. Their value is dependent on the values of the previous
wavefronts only. Hence, all the elements of a wavefront can be
calculated simultaneously. The value of the elements can be calculated
parallely. This is done using a multithreaded program in OpenMP.

11 | P a g e
Further, the program is also implemented in OpenMPI using both :
Single Node and Multi Node Cluster. The results of Multi Node Cluster
were better as compared to Single Node.

6. Evaluation

12 | P a g e
The above graph(time vs input size) shows the comparison between the
three different implementations.
It depicts the running time of the same algorithm when run on a single
machine, when run in a single machine multi-threaded environment
and a multi-node environment. We can clearly see that the single node
implementation is the worst whereas the other two have similar run-
times for input size up to 9000.
However, the multi node implementation is expected to perform better
as input size increases because with increase in input size the number
of threads required would also increase in a multi threaded
environment thus giving leading to poor performance because of
limitations on the maximum number of threads and the overhead
incurred to create them.

7. Theoretical Analysis

13 | P a g e
Serial Time : Ts = O(n2)

Parallel Time : Tp = n2/p + (ts + n√ 2 tp) + k n2/p + n2/p

[Computation Time + Communication Cost + FixupLoop +
BackwardPhase]

Cost = pTp = O(n2)

Ts/pTp = O(1) Therefore, Cost Optimal !

Isoefficiency = O(p2)

8. References

1. Maleki, Saeed, Madanlal Musuvathi, and Todd Mytkowicz.

"Parallelizing dynamic programming through rank
convergence." ACM SIGPLAN Notices 49.8 (2014): 219-232.
14 | P a g e
 2. http://wordaligned.org/articles/longest-common-subsequence

3. Kumar, V., Grama, A., Gupta, A., & Karypis, G. (1994). Introduction

to parallel computing: design and analysis of algorithms (Vol.
400). Redwood City, CA: Benjamin/Cummings.

4. D. S. Hirschberg. A linear space algorithm for computing maximal

common subsequences. Communications of the ACM, 18(6):341–
343, June 1975.

15 | P a g e