Maximum Subarray Problem in 1D and 2D Via Weighted Paths in Directed Acyclic Graphs
Maximum Subarray Problem in 1D and 2D Via Weighted Paths in Directed Acyclic Graphs
Yann Barsamian1
1 Introduction
The Maximum Subarray Problem (MSP) has been studied since 1977 (see for
example [2, Chapter 1] for a review). Grenander devised an O(n3) algorithm to
solve the MSP in one dimension (1D). In [4], Jon Bentley presented two O(n2)
algorithms, an O(n log n) one and an optimal O(n) algorithm due to Jay Kadane.
Here, we present an alternative O(n) algorithm for the MSP 1D. This al-
gorithm is a reduction to a single-source maximum weighted path problem in
a Directed Acyclic Graph (DAG). In [5, Footnote 1], the authors cite a per-
sonal communication which seems to imply this reduction, but to the best of
our knowledge, this – simple but useful – reduction has not been published
yet. We believe that this technique should be available in a written context.
Following Jon Bentley’s orders (Readers who feel that the linear-time algo-
rithm for the one-dimensional problem is ”obvious” are therefore urged to find
an ”obvious” algorithm for [the two-dimensional problem] ! ), we have designed
a similar technique for the MSP in two dimensions (2D). To scan all the con-
tiguous sub-matrices of a given matrix, we introduce a new type of graphs :
the red-blue graphs. With this data structure, we could nevertheless only come
up with a cubic algorithm, as is the case when we extend Kadane’s algorithm
in two dimensions. Algorithms exist with better complexity (Takaoka, in [10],
reduced this problem to (min, +) Matrix Multiplication for which we have
a
sub-cubic-time algorithm – there is an O(n3 log3 log n/ log2 n) algorithm for
All Pairs Shortest Path by Chan in [6] which shares the same asymptotical
time complexity, cf. [1, pp. 211–212] – leading to an O(m2n log3 log m/ log2
m) algo- rithm for the MSP 2D), but we hope that this new data structure
will lead to
an algorithm for finding maximum weighted paths with less complexity in the
future.
The main contributions of this paper are the presentations of :
1. a reduction of the MSP 1D to a graph problem with optimal complexity
2. a new data structure to scan all the contiguous sub-matrices of a given matrix
The remainder of this paper is organized as follows : Section 2 presents the
problem in 1D and 2D. Section 3 presents a reduction from the MSP 1D to a
single-source maximum weighted path problem in a DAG. Section 4 presents a
reduction from the MSP 2D to a single-source maximum weighted path
problem in a new type of graphs.
31 −41 59 26 −53 58 97
v0 v1 v2 v3 v4 v5 v6 v7
Fig. 1.
0 0 0 0
0 0 0
31 −41 59 26 −53 58 97
0v0 v1 v2 v3 v4 v5 v6 v7
0 0 0 0 0 0 0
p
Fig. 2.
In this graph, (s, v2, v3, v4, v5, v6, v7,(p) is the path of maximum
) weight, and
corresponds to the sub-vector W [2..6] = 59 26 −53 58 97 .
This example highlighted the reason why looking at the path of maximum
weight between s and p gives the contiguous sub-vector of maximum sum. We
will show in Sect. 3.2 that it is the case, and why doing so takes linear time.
0≤Li≤n−1 L L
i+1
0≤i≤n−1 1≤i≤n
Fig. 3.
We can improve this algorithm to have the same comparisons than Kadane’s
ones. At each vertex vi we make two comparisons because there are two ongoing
arcs : (s, vi) and (vi−1, vi). But (s, vi) weights 0, so we can initialize d with 0
and avoid looking at that arc in the loop. We will end up with one comparison,
as in Kadane’s algorithm to update MaxEndingHere. In Kadane’s algorithm, we
update MaxSoFar at each step. Here, we compute this at the end (when we scan
the predecessors of p), doing the same comparisons.
Theorem. The two algorithms return the same value on every input.
L
Proof. We will show that to every sub-vector V [i..j]i≤j of sum σ = V [k]
i≤k≤j
corresponds a path in G of weight σ. Then we will show that there is no other
path of non-zero weight by a counting argument.
To the empty sub-vector corresponds the path (s, v) of weight 0. To each
con- tiguous sub-vector V [i..j]i≤j corresponds the path (s, vi , vi+1 , . . . , vj ,
vj+1 , p). The weight of this path is w((s, vi)) + w((vi, vi+1)) + · · · + w((vj ,
vj+1)) +
L
w((vj+1 , p)) = 0 + V [i] + · · · + V [j] + 0 = V [k]. Thus :
i≤k≤j
4.2 An Example
Let us take as example the matrix N 1 2 −1 −4
−8 −3 4 −2 . Its associated red-blue
= graph is shown in Fig. 4. 3 8 10 1
0− 2−1−0−
1− + + + +
n0,0 2+ 1 0 n0,1 0− n0,2 2
+
1 0 n0,3
2− 1 2 −1 −4
0 −8 −3 4 −2
3 8 10 1
− −
n3,0 2+ n3,1 0 1 2− n3,2 2+ n3,3 −
0− 1 2−
2+0+ 1+ 2+ 0+ 1+
Fig. 4.
Let us take the path (s, n1,2, n2,2, n3,2, n3,3, n2,3, n1,3, p) shown in Fig. 5,
( 4 −2 )
which corresponds to the sub-matrix N [1..2][2..3] = 10 1 .
⋆ This path leaves the vertex s with a blue arc labelled 1−. It means that
the 0-th row of the contiguous sub-matrix is the 1-st row of N .
⋆ Then it takes the black arcs weighted with 10 and 1. It means that this
sub-matrix will contain these values, located in N [1][2] and N [2][2].
⋆ Then it takes the red arc labelled 2+. It means that the last row of this
sub-matrix is the 2-nd row of N .
⋆ In the next column, the black arcs mean that the sub-matrix contains both
N [2][3] and N [1][3]. The blue arc still indicates that the 0-th row is 1.
The black part of the graph (thick plain line) shows the values inside
the matrix, as in the 1D problem. We will explain in Sect. 4.3 why we put
the arcs alternatively down and up.
The blue part of the graph (normal plain line) shows the 0-th row of the
sub-matrix considered. To be valid, all the blue arcs in a path have to be
labelled with the same number. It means that every column starts at the same
row.
For example, Fig. 6 shows the path (s, n1,0, n2,0, n2,1, n1,1, n0,1, p) which
is not valid because the blue arc (s, n1,0) is labelled 1− but the blue arc (n0,1,
p) ( )
2
is labelled 0−. Of course, is not a sub-matrix !
−8 −3
The red part of the graph (dotted line) shows the last row of the sub-
matrix considered. To be valid, all the red arcs in a path have to be labelled
with the same number. It means that every column ends at the same row.
For example, Fig. 7 shows the path (s, n0,0, n1,0, n1,1, n0,1, n0,2, n1,2, n2,2, p)
which is not valid because the red arc (n1,0, n1,1) is labelled 0+ but the red arc
1 2 −1
(n2,2 , p) is labelled 1+ . Of course, ( ) is not a sub-matrix !
4 why looking at the valid path of
This example highlighted the reason
maxi- mum weight between s and p gives the contiguous sub-matrix of
maximum sum. We will show in Sect. 4.3 that it is the case.
Remark : Because blue and red arcs bear additional information (they have
additional labels to ensure row contiguity), we cannot use standard graph
algo- rithms to compute the maximum weighted path. If we forget these
constraints and just take the maximum weighted path in the underlying
weighted graph
(recall that the weight of those arcs is 0), the resulting maximum weight is an
upper bound for the maximum sum of contiguous sub-matrices.
In the example, the maximum weighted path (if we abstract from the row
constraints) is depicted on Fig. 8. But
2 −1
−3 4 is not a sub-matrix.
3 8 10 1
Bellman’s algorithm, which would take quadratic time, only gives us the
upper bound 24, the maximum among contiguous sub-matrices being 22 in
( )
the sub-matrix 3 8 10 1 .
In the general case, we have no more than an upper bound. It can even
happen that the maximum weighted path (if we abstract from the row
constraints) is disjoint from the sub-matrix of maximum sum. In Sect. 4.3 we
will then only focus on valid paths, and show that finding the sub-matrix of
maximum sum and finding the valid path of maximum weight is the same
problem.
s s
−
1
−
1
n0,0 n0,1 n0,2 n0,3 n0,0 n0,1 n0,2 n0,3
2
n1,0 n1,1 n1,2 n1,3
n1,0 n1,1 n1,2 n1,3
4 −2
−8 −3
n2,0 n2,1 n2,2 n2,3
n2,0 1+ n2,1 n2,2 n2,3
10 1
+
n3,0 n3,1 n n
,
3 3 −
1
n3 ,0 n3, 10− n3 ,2 n3 ,3
2
p p
Fig. 5. Fig. 6.
s s
−
0
−
n0,0 n0,1 0 n0,2 n0,3 n0,0 n0,1 0 n0,2 n0,3
1 2 −1 0 2 −1
4 −3 4
3 8 10 1
p p
Fig. 7. Fig. 8.
0≤i≤m−1 0≤i≤m−1
0≤2k≤n−1
1≤2k+1≤n−1
L
L
{(vi,2k+1, p)} ∪ L {(vi,2k+1, vi,2k+2)}
{(s, vi,2k)}∪ 0≤i≤m−1
0≤i≤m−1 0≤i≤m−1
0≤2k≤n−1 1≤2k+1≤n−1 2≤2k+2≤n−1
L L
{(s, vi,2k+1)} ∪ {(vi,2k, p)} ∪ L (vi,2k i,2k+1 )}
{ , v
∪
1≤i≤m 1≤i≤m 1≤i≤m
1≤2k+1≤n−1 0≤2k≤n−1 1≤2k+1≤n−1
With the weights : ∀ 0 ≤ i ≤ m − 1, ∀ 0 ≤ 2k ≤ n − 1, w((vi,2k,
vi+1,2k)) = M [i][2k] and w((vi+1,2k+1, vi,2k+1)) = M [i][2k + 1], and for every
other arc a, w(a) = 0.
And with additional labels for blue and red arcs, that denote the 0-th
and last row of the sub-matrix associated with a path containing these arcs :
blue arcs that enter or leave a vi,j vertex are labelled i− and red arcs that enter
or leave a vi,j vertex are labelled (i − 1)+.
Property. This graph has a quadratic (O(m × n)) size.
Proof. It has (m + 1)n + 2 vertices and 3mn + m(n − 1) + 1 arcs. □
Remark : The black arcs in the graph go down in the first column, then up
in the second one, then down, etc. There are of course other ways of
constructing a similar graph that would allow paths corresponding to sub-
matrices : having all the arcs going down for example. But in that case, we
would need an arc
from each vertex of one column to each vertex which is upper than it in the
next column. It would create (m − 1) + (m − 2) + · · · + 1 arcs per column
hence a
cubic (O(m2n)) number of arcs. If we want a less-than-cubic-time algorithm, we
cannot afford this.
Property. This graph is acyclic.
Proof. There is no cycle containing s because there is no arc going to s.
There is no cycle containing p because there is no arc going from p. There
is no cycle containing only vi,js because there is no arc going from vi,2k to
vj,2k if j ≥ i, there is no arc going from vi,2k+1 to vj,2k+1 if j ≤ i and there is no
arc from vi,k to vj,l if l < k. □
We can derive the topological ordering of that graph from this proof :
⋆ s < v0,0 < · · · < vm,0 < vm,1 < · · · < v0,1 < · · · < v0,n−1 < · · · < vm,n−1 < p
if n is odd and
⋆ s < v0,0 < · · · < vm,0 < vm,1 < · · · < v0,1 < · · · < vm,n−1 < · · · < v0,n−1 < p
if n is even.
Nevertheless, as noted in Sect. 4.2, we cannot apply standard graph algo-
rithms to find the path of maximum weight between s and p, because we need
to check that the paths are valid.
Theorem. The sum of elements in the contiguous sub-matrix of maximum
sum is the same as the weight of the valid path of maximum weight in the
associated red-blue graph.
Proof. We will show that to every sub-matrix N [i..j] of sum σ =
[k..l]i≤j
k≤l
L
L
N [a][b] corresponds a valid path in H of weight σ. Then we will show
i≤a≤j
k≤b≤l
that there is no other valid path of non-zero weight.
To a contiguous sub-matrix N [i..j][k..l]i≤j , if both k and l are even, corre-
k≤l
sponds the path (s, vi,k , . . . , vj+1,k , vj+1,k+1 , . . . , vi,k+1 , . . . , vi,l , . . . vj+1,l , p).
The weight of this path is :
]
w((s, vi,k )) + w((vi,k , vi+1,k )) + · · · + w((vj,k , vj+1,k )) +
]
w((vj+1,k , vj+1,k+1 )) + w((vj+1,k+1 , vj,k+1 )) + · · · + w((vi+1,k+1 , vi,k+1 )) +
]
· · · + w((vi,l , vi+1,l )) + · · · + w((vj,l , vj+1,l )) + w((vj+1,l , p))
( ) ( )
= 0 + N [i][k] + · · · + N [j][k] + 0 + N [j][k + 1] + · · · + N [i][k + 1] + · · · +
( )
N [i][l] + · · · + N [j][l] + 0
L L L L
= N [a][k] + N [a][k + 1] + · · · + N [a][l] =L N [a][b].
i≤a≤j i≤a≤ i≤a≤j i≤a≤j
j k≤b≤l
The computation is similar if k and/or l are odd, and to the empty sub-
vector corresponds the path (s, v) of weight 0. Thus :
Each contiguous sub-matrix sums up to the weight of a valid path. (i)
Let us now take a valid path in the red-blue graph, different from (s, p)
which weights 0. This path thus has at least one blue arc and one red arc (if it
starts
with a blue arc, entering an even column, the only arcs that leave this column
are red ; if it starts with a red arc, entering an odd column, the only arcs that
leave this column are blue). Let us call x− the label of all blue arcs in the path
and y+ the label of all red arcs in the path. Blue arcs enter only even columns (or
p) and red arcs enter only odd columns (or p). Blue arcs leave only odd
columns (or s) and red arcs leave only even columns (or s). There are only two
cases :
⋆ First case : y = x − 1. By construction, the x−-labelled arcs can only
enter or leave vertices in {vx,j | 0 ≤ j ≤ n − 1} and the y+-labelled arcs
can only enter of leave vertices in {vy+1,j | 0 ≤ j ≤ n − 1} (apart from s
and p). Because
y = x − 1, the only vertices apart from s and p are thus vx,j vertices. Thus the
path cannot contain black arcs. Its weight is 0 (an example of that kind of
paths is (s, n1,0, n1,1, n1,2, p)).
⋆ Second case : y ≥ x. For the same reason as in the first case, the path
enters the k-th even column via the vertex in vx,2k and leaves this column
via the vertex vy+1,2k. Thus the black arcs in that column are the arcs in
{(vi,2k, vi+1,2k) | x ≤ i ≤ y} which have the weights {N [i][2k] | x ≤ i ≤ y}.
And the same goes in the k-th odd column : the black arcs are {(vi+1,2k+1,
vi,2k+1) | y ≥ i ≥ x} which have the weights {N [i][2k + 1] | y ≥ i ≥ x}. All
in all, the total weight of the arcs is N [a][b] for some z ≤ t. This
corresponds
L to the sum of elements in
L
x≤a≤y
z≤b≤t
5 Conclusion
In this paper, we have presented a reduction from the maximum subarray
prob- lem in one dimension to a well known graph problem. We think that this
reduc- tion highlights the reason why we have a linear-time algorithm for this
problem.
To extend this idea to two dimensions, we have designed a new data
structure, red-blue graphs, that allows us to scan all the contiguous sub-
matrices of a given matrix. By looking at the valid paths in this data structure,
we obtain a reduction from the maximum subarray problem in two
dimensions. Unfortunately, this reduction does not ”obviously” lead to an
optimal algorithm. Nevertheless, we think that this data structure may help in
finding a better algorithm for the MSP 2D, and maybe for other matrix-related
problems that need to scan contiguous sub-matrices.
A natural extension for the MSP is searching for the k maximum sums,
instead of only the maximum sum. We can allow these sums to overlap (we
have an optimal O(n + k) algorithm in one dimension : [5]) or we can instead
want to
have disjoint sums (we also have an optimal O(n) algorithm in one dimension
: [9]). It’s interesting to note, like Brodal and Jørgensen, that an alternative
optimal algorithm when the overlap is allowed use the weighted graph presented.
If the 1D problem is essentially solved, there remains a lot of work in
higher dimensions. We hope that in the future, part of the ideas presented in
this paper will lead to better algorithms.
Last but not least, we would like to thank our advisor, E´ ric Violard, for his
help on the writing of this article.
References
[1] Alfred V. Aho, John E. Hopcroft, and Jeffrey D. Ullman. The Design and
Analysis of Computer Algorithms. Reading, MA: Addison-Wesley, 1974.
[2] Sung Eun Bae. “Sequential and parallel algorithms for the generalized
maximum subarray problem”. PhD thesis. University of Canterbury,
2007. url: http://hdl.handle.net/10092/1202.
[3] Richard Bellman. On a routing problem. Tech. rep. 1956. url: http://ww
w.dtic.mil/dtic/tr/fulltext/u2/606258.pdf.
[4] Jon Bentley. “Programming Pearls: Algorithm Design Techniques”. In:
Commun. ACM 27.9 (Sept. 1984), pp. 865–873. url: http://www.akira.
ruc.dk/~keld/teaching/algoritmedesign_f07/Artikler/05/Bentley8
4.pdf.
[5] Gerth Stølting Brodal and Allan Grønlund Jørgensen. “A Linear Time Al-
gorithm for the k Maximal Sums Problem”. In: Mathematical
Foundations of Computer Science 2007. Ed. by Ludˇek Kuˇcera and
Anton´ın Kuˇcera. Vol. 4708. Springer Berlin Heidelberg, 2007, pp. 442–
453. url: https:// users-cs.au.dk/gerth/papers/mfcs07sum.pdf.
[6] Timothy M. Chan. “More Algorithms for All-pairs Shortest Paths in
Weighted Graphs”. In: Proceedings of the Thirty-ninth Annual ACM
Symposium on Theory of Computing. STOC’07. 2007, pp. 590–598. url:
https://cs.uw aterloo.ca/~tmchan/moreapsp.pdf.
[7] Thomas H. Cormen, Charles E. Leiserson, Ronald L. Rivest, and Clifford
Stein. Introduction to Algorithms. 3rd ed. MIT Press and McGraw-Hill,
2009.
[8] Ulf Grenander. Pattern analysis: Lectures in Pattern Theory 2. New York:
Springer, 1978.
[9] Walter L. Ruzzo and Martin Tompa. “A Linear Time Algorithm for
Find- ing All Maximal Scoring Subsequences”. In: Proceedings of the
Seventh In- ternational Conference on Intelligent Systems for Molecular
Biology. AAAI Press, 1999, pp. 234–241. url:
http://homes.cs.washington.edu/~ruz
zo/papers/maxseq.pdf.
[10] Tadao Takaoka. “Efficient Algorithms for the Maximum Subarray
Prob- lem by Distance Matrix Multiplication”. In: Electronic Notes in
Theoretical Computer Science. CATS’02, Computing: the Australasian
Theory Sym- posium 61 (2002), pp. 191–200. url:
http://www.cosc.canterbury.ac. nz/tad.takaoka/cats02.pdf.