A SPECTRAL ALGORITHM FOR SERIATION

AND THE CONSECUTIVE ONES PROBLEM

JONATHAN E. ATKINSy , ERIK G. BOMANz , AND BRUCE HENDRICKSONx
Abstract.
In applications ranging from DNA sequencing through archaeological dating to sparse matrix
reordering, a recurrent problem is the sequencing of elements in such a way that highly correlated pairs
of elements are near each other. That is, given a correlation function f re ecting the desire for each pair
of elements to be near each other, nd all permutations  with the property that if (i) < (j ) < (k)
then f (i; j )  f (i; k) and f (j; k)  f (i; k). This seriation problem is a generalization of the well studied
consecutive ones problem. We present a spectral algorithm for this problem that has a number of
interesting features. Whereas most previous applications of spectral techniques provide only bounds
or heuristics, our result is an algorithm that correctly solves a nontrivial combinatorial problem.
In addition, spectral methods are being successfully applied as heuristics to a variety of sequencing
problems, and our result helps explain and justify these applications.
Key words. seriation, consecutive ones property, eigenvector, Fiedler vector, analysis of algo-
rithms

1. Introduction. Many applied computational problems involve ordering a set

so that closely coupled elements are placed near each other. This is the underlying
problem in such diverse applications as genomic sequencing, sparse matrix envelope
reduction and graph linear arrangement as well as less familiar settings such as ar-
chaeological dating. In this paper we present a spectral algorithm for this class of
problems. Unlike traditional combinatorial methods, our approach uses an eigenvec-
tor of a matrix to order the elements. Our main result is that this approach correctly
solves an important ordering problem we call the seriation problem which includes the
well known consecutive ones problem [5] as a special case.
More formally, we are given a set of n elements to sequence; that is, we wish to
bijectively map the elements to the integers 1; : : :; n. We also have a symmetric, real
valued correlation function (sometimes called a similarity function) that re ects the
desire for elements i and j to be near each other in the sequence. We now wish to
nd all ways to sequence the elements so that the correlations are consistent; that is,
if  is our permutation of elements and  (i) <  (j ) <  (k) then f (i; j )  f (i; k) and
f (j; k)  f (i; k). Although there may be an exponential number of such orderings,
they can all be described in a compact data structure known as a PQ-tree [5] which
we review in the next section. Not all correlation functions allow for a consistent
sequencing. If a consistent ordering is possible we will say the problem is well posed.
Determining an ordering from a correlation function is what we will call the seriation
problem, re ecting its origins in archaeology [29, 33].
 This work was supported by the Mathematical, Information and Computational Sciences Division
of the U.S. DOE, Oce of Energy Research, and work at Sandia National Laboratories, operated for
the U.S. DOE under contract No. DE-AC04-94AL85000. Sandia is multiprogram laboratory operated
by Sandia Corporation, a Lockheed Martin Company, for the U.S. DOE.
y
Dept. Mathematics, University of Michigan, 2072 East Hall, Ann Arbor, MI 48109.
[email protected].
z
Scientic Computing & Computational Mathematics, Gates Bldg. 2B, Stanford Univ., Stanford,
CA 94305{9025. [email protected].
x
Applied & Numerical Mathematics Dept., Sandia National Labs, Albuquerque, NM 87185{1110.
[email protected].
1
 The consecutive ones problem (C1P) is a closely related ordering problem. A
(0; 1)-matrix C has the consecutive ones property if there exists a permutation matrix
 such that for each column in C , all the ones form a consecutive sequence. If a
matrix has the consecutive ones property, then the consecutive ones problem is to nd
all such permutations. As shown by Kendall [19] and reviewed in x6, C1P is a special
case of the seriation problem.
Our algorithm orders elements using their value in an eigenvector of a Laplacian
matrix which we formally dene in x2. Eigenvectors related to graphs have been stud-
ied since the 1950's (see, for example, the survey books by Cvetkovic et al. [8, 7]. Most
of the early work involved eigenvectors of adjacency matrices. Laplacian eigenvectors
were rst studied by Fiedler [10, 11] and independently by Donath and Homan [9].
More recently, there have been a number of attempts to apply spectral graph theory
to problems in combinatorial optimization. For example, spectral algorithms have
been developed for graph coloring [3], graph partitioning [9, 28] and envelope reduc-
tion [4], and more examples can be found in the survey papers of Mohar [23, 24].
However, in most previous applications, these techniques have been used to provide
bounds, heuristics, or in a few cases, approximation algorithms [2, 6, 14] for NP-hard
problems. There are only a small number of previous results in which eigenvector
techniques have been used to exactly solve combinatorial problems including nding
the number of connected components of a graph [10], coloring k-partite graphs [3],
and nding stable sets (independent sets) in perfect graphs [16]. This paper describes
another such application.
Spectral methods are closely related to the more general method of semidenite
programming, which has been applied successfully to many combinatorial problems
(e.g. MAX-CUT and MAX-2SAT[14] and graph coloring[18]). See Alizadeh[1] for a
survey of semidenite programming with applications to combinatorial optimization.
Our result is important for several reasons. First, it provides new insight into the
well-studied consecutive ones problem. Second, some important practical problems
like envelope reduction for matrices and genomic reconstruction can be thought of
as variations on seriation. For example, if biological experiments were error-free, the
genomic reconstruction problem would be precisely C1P. Unfortunately, real experi-
mental data always contain errors, and attempts to generalize the consecutive ones
concept to data with errors seems to invariably lead to NP-complete problems [31, 15].
A spectral heuristic based upon our approach has recently been applied to such prob-
lems and found to be highly successful in practice [15]. Our result helps explain this
empirical success by revealing that in the error-free case the technique will correctly
solve the problem. This places the spectral method on a stronger theoretical footing
as a cross between a heuristic and an exact algorithm. Similar comments apply to
envelope reduction. Matrices with dense envelopes are closely related to matrices with
the consecutive ones property. Recent work has shown spectral techniques to be better
in practice than any existing combinatorial approaches at reducing envelopes [4]. Our
result sheds some light on this success.
Another way to interpret our result is that we provide an algorithm for C1P that
generalizes to become an attractive heuristic in the presence of errors. Designed as de-
cision algorithms for the consecutive ones property, existing combinatorial approaches
for C1P break down if there are errors and fail to provide useful approximate orderings.
However, our goal here is not to analyze the approach as an approximation algorithm,
but rather to prove that it correctly solves error-free problem instances.
2
 This paper is organized in the following way. In the next section we introduce the
mathematical notation and the results from matrix theory that we will need later. We
also describe a spectral heuristic for ordering problems which motivates the remainder
of the paper. The theorem that underpins our algorithm is proved in x3, the proof of
which requires the use of a classic theorem from matrix analysis. Several additional
results in x4 lead us to an algorithm and its analysis in x5. We review the connection
to C1P in x6.
2. Mathematical background.
2.1. Notation and Denitions. Matrix concepts are useful because the cor-
relation function dened above can be considered as a real, symmetric matrix. A
permutation of the elements corresponds to a symmetric permutation of this matrix,
a permutation of the matrix elements formed by permuting the rows and the columns
in the same fashion. The question of whether or not the ordering problem is well
posed can also be asked as a property of this matrix. Specically, suppose the ma-
trix has been permuted to re ect a consistent solution to the ordering problem. The
o-diagonal matrix entries must now be non-increasing as we move away from the
diagonal. More formally, we will say a matrix A is an R-matrix 1 if and only if A is
symmetric and
ai;j  ai;k for j < k < i;
ai;j  ai;k for i < j < k:
The diagonal entries of an R-matrix are unspecied. If A can be symmetrically per-
muted to become an R-matrix, then we say that A is pre-R. Note that pre-R matrices
correspond precisely to well-posed ordering problems. Also, the R-matrix property is
preserved if we add a constant to all o-diagonal entries, so we can assume without
loss of generality that all o-diagonal values are non-negative.
When  is a permutation of the natural numbers f1; : : :; ng and x is a column
vector, i.e. x = [x1; : : :; xn ]T , we will denote by x the permutation of x by  , i.e.
xi = x(i) . Similarly, A is the symmetric permutation of A by , i.e. ai;j = a(i);(j) .
We denote by e the vector whose entries are all 1, by ei the vector consisting of zeros
except for a 1 in position i, and by I the identity matrix. A symmetric matrix A is
reducible if there exists a permutation  such that
" #
A = B0 C0 ;

where B and C are non-empty square matrices. If no such permutation exists then
A is irreducible. If B and C are themselves irreducible, then we refer to them as the
irreducible blocks of A.
We say that  is an eigenvalue of A if Ax = x for some vector x 6= 0. A
corresponding vector x is an eigenvector. An n  n real, symmetric matrix has n
eigenvectors that can be constructed to be pairwise orthogonal, and its eigenvalues
are all real. We will assume that the eigenvalues are sorted by increasing value, and
refer to them as i, i = 1; : : :; n. The (algebraic) multiplicity of an eigenvalue  is
1
This class of matrices is named after W. S. Robinson who rst dened this property in his work
on seriation methods in archaeology [29].
3
dened as the number of times  occurs as a root in the characteristic polynomial
p(z) = det(A ? zI ). A value that occurs only once is called simple; the eigenvector
of a simple eigenvalue is unique (up to normalization). We write A  0 and say A is
non-negative if all its elements ai;j are non-negative. A real vector x is monotone if
xi  xi+1 for all 1  i < n or if xi  xi+1 for all 1  i < n.
We dene the Laplacian ofPa symmetric matrix A to be LA = DA ? A, where DA is
a diagonal matrix with di;i = nj=1 ai;j . The minimum eigenvalue with an eigenvector
orthogonal to e (the vector of all ones) is called the Fiedler value and a corresponding
eigenvector is called a Fiedler vector2 . Alternatively, the Fiedler value is given by
min
xT e=0;xT x=1
xT LAx;
and a Fiedler vector is any vector x that achieves this minimum while satisfying these
constraints. When A  0 and irreducible, it is not hard to show that the Fiedler
value is the smallest non-zero eigenvalue and a Fiedler vector is any corresponding
eigenvector. We will be notationally cavalier and refer to the Fiedler value and vector
of A when we really mean those of LA .
2.2. PQ-trees. A PQ-tree is a data structure introduced by Booth and Lueker
to eciently encode a set of related permutations [5]. A PQ-tree over a set U =
fu1; u2; : : :; ung is a rooted, ordered tree whose leaves are elements of U and whose
internal nodes are distinguished as either P-nodes or Q-nodes. A PQ-tree is proper
when the following three conditions hold:
1. Every element ui 2 U appears precisely once as a leaf.
2. Every P-node has at least two children.
3. Every Q-node has at least three children.
Two PQ-trees are said to be equivalent if one can be transformed into the other by
applying a sequence of the following two equivalence transformations:
1. Arbitrarily permute the children of a P-node.
2. Reverse the children of a Q-node.
Conveniently, the equivalence class represented by a PQ-tree corresponds precisely to
the set of permutations consistent with an instance of a seriation problem. In x5 we
describe an algorithm which uses Laplacian eigenvectors to construct a PQ-tree for an
instance of the seriation problem.
2.3. Motivation for Spectral Methods. With the above denitions we can
describe a simple heuristic for the seriation problem that will motivate the remainder
of the paper. This heuristic is at the heart of the more complex algorithms we will
devise, and underlies many previous applications of spectral algorithms [17]. We begin
by constructing a simple penalty function g whose value will P be small when closely
correlated elements are close to each other. We dene g ( ) = (i;j) f (i; j )(i ? j )2.
Unfortunately, minimizing g is NP-hard due to the discrete nature of the permu-
tation [13]. Instead we approximate it by a function h of continuous variables xi
that wePcan minimize and that maintains much of the structure of g . We dene
h(x) = (i;j) f (i; j )(xi ? xj )2 . Note that h does not have a unique minimizer, since
its value does not change if we add a constant P
to each x component. To avoid this
ambiguity, we need to add a constraint like i xi = 0. We still have a trivial solution
2
This is in recognition of the work of Miroslav Fiedler [10, 11].
4
 P
when all the xi 's are zero, so we need a second constraint like i x2i = 1. The resulting
minimization problem is now well dened.
X
(1) Minimize h(x) = f (i; j )(xi ? xj )2
(i;j)
X X 2
subject to: xi = 0; and x = 1:
i
i i
The solution to this continuous problem can be used as a heuristic for sequencing.
Merely construct the solution vector x, sort the elements xi and sequence based upon
their sorted order. One reason this heuristic is attractive is that the minimization
problem has an elegant solution. We can rewrite h(x) as xT LF x where F = ffij g is
the correlation matrix. The constraints require that x be a unit vector orthogonal to e,
and since LA is symmetric, all other eigenvectors satisfy the constraints. Consequently,
a solution to the constrained minimization problem is just a Fiedler vector.
Even if the problem is not well posed, sorting the entries of the Fiedler vector
generates an ordering that tries to keep highly correlated elements near each other. As
mentioned above, this technique is being used for a variety of sequencing problems [4,
15, 17]. The algorithm we describe in the remainder of the paper is based upon this
idea. However, when we encounter ties in entries of the Fiedler vector, we need to
recurse on the subproblem encompassing the tied values. In this way, we are able to
nd all permutations which make a pre-R matrix into an R matrix.
3. The key theorem. Our main result is that a modication of the simple
heuristic presented in x2.3 is actually an algorithm for well-posed instances of the
seriation problem. Completely proving this will require us to deal with the special
cases of multiple Fiedler vectors and ties within the Fiedler vector. The cornerstone
of our analysis is a classical result in matrix theory due to Perron and Frobenius [27].
The particular formulation below can be found on page 46 of [30].
Theorem 3.1 (Perron-Frobenius). Let M be a real, non-negative matrix. If
we dene (M ) = maxi ji(M )j, then
1. (M ) is an eigenvalue of M , and
2. there is a vector x  0 such that Mx = (M )x.
We are now ready to state and prove our main theorem.
Theorem 3.2. If A is an R-matrix then it has a monotone Fiedler vector.
Proof. Our proof uses the Perron-Frobenius Theorem 3.1. The non-negative vector
in that theorem will consist of dierences between neighboring entries in the Fiedler
vector of the Laplacian of A.
First dene the matrix S 2 IR(n?1)n as
2 3
? 1 1 0


0
6 0 ?1 1 . . . ... 7
6 7
6
S = 66 . . . . 7:
7
4 .. . . . . . . 0 5
7
0


0 ?1 1

5
Note that for any vector x, Sx = (x2 ? x1 ; : : :; xn ? xn?1 )T . Dene T 2 IRn(n?1) by
2
0 0


0 3
6 1 0 0 77
6
6 ... .. 77
T = 666 1 1 . 77 :
6 .. .. ...
4 . . 0 75
1 1


1
It is easy to verify that ST = In?1 , and that TS = In ? eeT1 . We dene MA =
SLAT = fmi;j g and let LA = fli;j g. We now show that Sx is an eigenvector of MA if
and only if x is an eigenvector of LA and x 6= e.
LA x = x; x 6= e ()
SLAx = Sx; x 6= e ()
SLA(I ? eeT1 )x = Sx; x =6 e ()
SLATSx = 6 e ()
Sx; x =
MA y = y; where y = Sx 6= 0:
The transformation from the second to the third line follows from LA e = 0. Equiv-
alence holds between all the above equations, so  is an eigenvalue for both LA and
MA for eigenvectors of LA other than e. Hence the eigenvalues of MA are the same as
the eigenvalues of LA with the zero eigenvalue removed, and the eigenvectors of MA
are dierences between neighboring entries of the corresponding eigenvectors of LA .
It is easily seen that (SLA )i;k = ?li;k + li+1;k for all i; k, so
n
X n
X n
X
mi;j = (SLA)i;k Tk;j = (?li;k + li+1;k ) = (ai;k ? ai+1;k ):
k=1 k=j+1 k=j+1
Since, by assumption, A is an R-matrix, ai;k  ai+1;k for i < k + 1, and therefore
mi;j  0 for i < j . For i > j we can use the fact that Pnk=1 li;k = 0 to obtain
n
X j
X j
X
mi;j = (?li;k + li+1;k ) = (li;k ? li+1;k ) = (?ai;k + ai+1;k ):
k=j+1 k=1 k=1
Again, from the R-matrix property we conclude that mij  0 for i > j . Consequently,
all the o-diagonal elements in MA are non-positive.
Now let  be a value greater than maxi fi ; miig, where i are the eigenvalues of
MA . Then M~A = I ? MA is non-negative with eigenvalues ~i =  ? i . Also, M~A and
MA share the same set of eigenvectors. By Theorem 3.1, there exists a non-negative
eigenvector y of M~A corresponding to the largest eigenvalue of M~A . But y is also an
eigenvector of MA corresponding to MA 's smallest eigenvalue. And this is just Sx,
where x is a Fiedler vector of LA . Since y = Sx is non-negative, the corresponding
Fiedler vector of LA is non-decreasing and the theorem follows. (Note that since the
sign of an eigenvector is unspecied, the Fiedler vector could also be non-increasing.)
Theorem 3.3. Let A be a pre-R matrix with a simple Fiedler value and a Fiedler
vector with no repeated values. Let 1 (respectively 2) be the permutation induced by
6
sorting the values in the Fiedler vector in increasing (decreasing) order. Then A1
and A2 are R-matrices, and no other permutations of A produce R-matrices.
Proof. First note that since the Fiedler value is simple, the Fiedler vector is unique
up to a multiplicative constant. Next observe that if x is the Fiedler vector of A, then
x is the Fiedler vector of A . So applying a permutation to A merely changes the
order of the entries in the Fiedler vector. Now let  be a permutation such that A
is an R-matrix. By Theorem 3.2 x is monotone since x is the only Fiedler vector.
Since x has no repeated values,  must be either 1 or 2 .
Theorem 3.3 provides the essence of our algorithm for the seriation problem, but
it is too restrictive as the Fiedler value must be simple and contain no repeated values.
We will show how to remove these limitations in the next section.
4. Removing the restrictions. Several observations about the seriation prob-
lem will simplify our analysis. First note that if we add a constant to all the correlation
values the set of solutions is unchanged. Consequently, we can assume without loss
of generality that the smallest value of the correlation function is zero. Note that
subtracting the smallest value from all correlation values does not change whether or
not the matrix is pre-R. In our algebraic formulation this translates into the following.
Lemma 4.1. Let A be a symmetric matrix and let A = A ? eeT for some real
. A vector x is a Fiedler vector of A i x is a Fiedler vector of A. So without loss
of generality we can assume that the smallest o-diagonal entry of A is zero.
Proof. By the denition of a Laplacian it follows that LA = LA + eeT ? nI ,
where n is the dimension of A. Then LAe = 0 but for any other eigenvector x of LA ,
LAx = LAx + 0 ? nx. That is, the eigenvalues are simply shifted down by n while
the eigenvectors are preserved.
This will justify the rst step of our algorithm, which subtracts the value of the
smallest correlation from every correlation. Accordingly, we now make the assumption
that our pre-R matrix has smallest o diagonal entry of zero. Next observe that if A is
reducible then the seriation problem can be decoupled. The irreducible blocks of the
matrix correspond to connected components in the graph of the nonzero values of the
correlation function. We can solve the subproblems induced by each of these connected
components, and link the pieces together in an arbitrary order. More formally, we have
the following lemma.
Lemma 4.2. Let Ai , i = 1; : : :; k be the irreducible blocks of a pre-R matrix A, and
let i be a permutation of block Ai such that the submatrix Ai i is an R-matrix. Then
any permutation formed by concatenating the i 's will make A become an R-matrix.
In terms of a PQ-tree, the i permutations are children of a single P-node.
Proof. By Lemma 4.1, we can assume all entries in the irreducible blocks are non-
negative. Consequently, the correlation between elements within a block will always
be at least as strong as the correlation between elements in dierent blocks. Also, by
the denition of irreducibility, each element within a block must have some positive
correlation with another element in that block. Hence, any ordering that makes Ai an
R-matrix must not interleave elements between dierent irreducible blocks. As long as
the blocks themselves are ordered to be R-matrices, any ordering of blocks will make
A an R-matrix since correlations across blocks are all identical.
With these preliminaries, we will now assume that the smallest o-diagonal value
is zero and that the matrix is irreducible. As the following three lemmas and theorem
show, this is sucient to ensure that the Fiedler vector is unique up to a multiplicative
7
constant.
Lemma 4.3. Let A be an n  n R-matrix with a monotone Fiedler vector x. If
J = [r; s] is a maximal interval such that xr = xs , then for any k 2= J ; ar;k = ar+1;k =
: : : = as;k .
Proof. We can without loss of generality assume x is non-decreasing since ?x is
also a Fiedler vector. We will show that ar;k = as;k for all k 2= J , and since A is an
R-matrix then all elements between ar;k and as;k must also be equal. Consider rows r
and s in the equation LA x = x.
n
X
(ls;k ? lr;k )xk = (xs ? xr ) = 0
k=1
P
Since LA is a Laplacian, we know that nk=1 li;k = 0 for all i. We get
n
X
0 = (lsk ? lrk )(xr ? xk )
k=1
?1
rX n
X
= ? lr;k)} (|xr ?{z xk )} +
(|ls;k {z ? lr;k )} |(xr ?{z xk )}
(ls;k {z
k=1 0 >0 k=s+1 | 0 <0

where we have used the fact that x is non-decreasing. Because all terms in the sum
are non-negative, all terms must be exactly zero. By assumption, xk 6= xr for k 2= J
and consequently lr;k = ls;k for k 2= J and the result follows.
The following lemma is essentially a converse of this. Its proof requires detailed
algebra, but it is not fundamental to what follows. Consequently, the proof is relegated
to the end of this section.
Lemma 4.4. Let A be an irreducible n  n R-matrix with an;1 = 0. If J = [r; s] 6=
[1; n] is an interval such that ar;k = as;k for all k 2= J , then xr = xr+1 = : : : = xs for
any Fiedler vector x.
Lemma 4.5. Let A be an irreducible R-matrix with an;1 = 0, and x a monotone
Fiedler vector of A. If J = [r; s] is an interval such that xr = xr+1 = : : : = xs , then
for any Fiedler vector y , yr = yr+1 = : : : = ys .
Proof. First apply Lemma 4.3 to conclude that for any k 2= J ; ar;k = ar+1;k =
: : : = as;k . Since xT e = 0, it follows that J 6= [1; n]. Now use this in conjunction with
Lemma 4.4 to obtain the result.
Theorem 4.6. If A is an irreducible R-matrix with an;1 = 0, then the Fiedler
value 2 is a simple eigenvalue.
Proof. We will assume that 2 is a repeated eigenvalue and produce a contradic-
tion. Let x and y be two linearly independent Fiedler vectors with x non-decreasing.
Dene z () = cos()x + sin()y , with 0     . Let  be the smallest value of 
that makes zk = zk+1 for some k where xk 6= xk+1 . Such a  must exist since x and
y are linearly independent.
By Lemma 4.5 the indices of any repeated values in x are indices of repeated
values in y and z (). Coupled with the monotonicity of x, this implies that z ( ) is
monotone. By Lemma 4.5 the indices of any repeated values in z ( ) must be repeated
in x which gives the desired contradiction.
All that remains is to handle the situation where the Fiedler vector has repeated
values. As the following theorem shows, repeated values decouple the problem into
pieces that can be solved recursively.
8
 Theorem 4.7. Let A be a pre-R matrix with a simple Fiedler value and Fiedler
vector x. Suppose there is some repeated value  in x and dene I , J and K to be
the indices for which
1. xi <  for all i 2 I ,
2. xi =  for all i 2 J ,
3. xi >  for all i 2 K.
Then  is an R-matrix ordering for A i  or its reversal can be expressed as (i ; j ; k ),
where j is an R-matrix ordering for the submatrix A(J ; J ) of A induced by J , and i
and k are the restrictions of some R-matrix ordering for A to I and K, respectively.
Proof. From Theorem 3.2 we know that for any R-matrix ordering A , x is
monotone, so elements in I must appear before (after) elements from J and elements
from K must appear after (before) elements from J . By Lemma 4.3, we have aik = ajk
for all i; j 2 J and k 2= J . Hence the orderings of elements inside J must be indierent
to the ordering outside of J and vice versa. Consequently, the R-matrix ordering of
elements in J depends only of A(J ; J ).
Algorithmically, this theorem means that we can break ties in the Fiedler vector
by recursing on the submatrix A(J ; J ) where J corresponds to the set of repeated
values. The distinct values in the Fiedler vector of A constrain R-matrix orderings,
but repeated values need to be handled recursively. In the language of PQ-trees,
the distinct values are combined via a Q-node, and the components (subtrees) of the
Q-node must then be expanded recursively.
Proof of Lemma 4.4. First we recall that the Fiedler value is the value obtained
by
X
(2) minT xT LA x = minT ai;j (xi ? xj )2;
xT e=0;x x=1 xT e=0;x x=1 i>j

and a Fiedler vector is a vector that achieves this minimum. We note that if we
replace A by a matrix that is at least as large on an elementwise comparison then
xT LA x cannot decrease for any vector x.
We consider A(J ; J ), the diagonal block of A indexed by J . By the denition
of an R-matrix, all values in A(J ; J ) must be at least as large as ar;s . However, ar;s
must be greater than zero. Otherwise, by the R-matrix property ai;j = 0 for all i  r
and j < s and for all j  r and i < s. But then by the statement of the theorem
ai;j = 0 for all i  s and j < s and all j  r and j < s which would make the matrix
reducible.
The remainder of the proof will proceed in two stages. First we will force all the
o-diagonal values in A(J ; J ) to be ar;s and show the result for this modied matrix.
We will then extend the result to our original matrix.
Stage 1:
We dene the matrix B to be identical to A outside of B (J ; J ), but all o-diagonal
values of B within B (J ; J ) are set to  = ar;s . It follows from the hypotheses that B
is an R-matrix. We dene  = li;i for i 2 J and note that, by the R-matrix property,
  (n ? 1).
We now dene L~B = LB ? ( + )I and consider the eigenvalue equation L~B x =
~2x. This matrix has the same eigenvectors as LB with eigenvalues shifted by  + .
Since ~lii =  ? ( + ) =  for i 2 J , all rows of L~B in J are identical. Consequently,
either all elements of x in J are equal, or ~ 2 = 0 (which is equivalent to 2 =  + ).
9
We will show that irreducibility and an1 = 0 implies 2 6=  + , which will complete
the proof of Stage 1.
We assume 2 =  +  and look for a contradiction. We introduce a new matrix
^
B as follows
8
< bi;j if i < r and j < r;
>
^bi;j = bi;j if i > s and j > s;
>
:  otherwise.
Since B is an R-matrix, B^ is at least as large as B elementwise, so 2(B^ )  2(B ).
We dene the vector y^ by
8
< ?(n ? s); if i < r;
>
y^i = > 0; if r  i  s;
: r ? 1; if i > s;
and x^ to be the unit vector in the direction of y^. We note that x^T e = 0, and that
x^T LB^ x^ = n. We have the following chain of inequalities.
(3) minT xT LB x  x^T LB x^ < x^T LB^ x^ = n:
2 = xT e=0;x x=1
The last inequality is strict since ^bn;1 =  while bn;1 = 0 and (^xn ? x^1)2 > 0.
If 2 =  +  then we can combine an inequality due to Fiedler [10],
2  n ?n 1 min l;
i ii
with the observation that mini li;i   to obtain 2  n?n 1    +  = 2. This can
only be true if equality holds throughout, implying that  = (n ? 1) and 2 = n.
But this contradicts (3), so 2 6=  +  and the proof of Stage 1 is complete.
Stage 2:
We will now show that A and B have the same Fiedler vectors. Since A is elementwise
at least as large as B , for any vector z , z T LA z  z T LB z . From Stage 1 we know that
any Fiedler vector of B satises xr = xr+1 = : : : = xs . In this vector, (xi ? xj ) = 0
for i; j 2 J so the contribution to the sum in (2) from B (J ; J ) is zero. But this
contribution will also be zero when applied to A(J ; J ). Since A and B are identical
outside of A(J ; J ) and B (J ; J ), we now have that a Fiedler vector of B gives an
upper bound for the Fiedler value of A; that is, 2(A)  2 (B ). It follows that the
Fiedler vectors of B are also Fiedler vectors of A and vice versa.
5. A spectral algorithm for the seriation problem. We can now bring all
the preceding results together to produce an algorithm for well-posed instances of the
seriation problem. Specically, given a well-posed correlation function we will generate
all consistent orderings. Given a pre-R matrix, our algorithm constructs a PQ-tree
for the set of permutations that produce an R-matrix.
Our Spectral-Sort algorithm is presented in Fig. 1. It begins by translating all
the correlations so that the smallest is 0. It then separates the irreducible blocks (if
there are more than one) into the children of a P-node and recurses. If there is only
one such block, it sorts the elements into the children of a Q-node based on their
values in a Fielder vector. If there are ties in the entries of the Fiedler vector, the
algorithm is invoked recursively.
10
 Input: A, an n  n pre-R matrix
U , a set of indices for the rows/columns of A
Output: T , a PQ-tree that encodes the set of all permutations 
such that A is an R-matrix
begin
(1)  := mini6=j ai;j
(1) A := A ? eeT
(2) fA1; : : :; Akg := the irreducible blocks of A
(2) fU1; : : :; Ukg := the corresponding index sets
(2) if k > 1
(2) for j := 1 : k
(2) Tj := Spectral-Sort(Aj ; Uj )
(2) end
(2) T := P-node(T1 ; T2; : : :; Tk )
else
(3) if (n = 1)
(3) T := u1
(3) else if (n = 2)
(3) T := P-node(u1 ; u2)
else
(4) x := Fiedler vector for LA
(4) Sort x
(5) t := number of distinct values in x
(5) for j := 1 : t
(5) Vj := indices of elements in x with j th value
(5) Tj := Spectral-Sort(A(Vj ; Vj ); Vj )
(5) end
T := Q-node(T1; : : :; TT )
end
end
end
Fig. 1 . Algorithm Spectral-Sort.

We now prove that the algorithm is correct. Step (1) is justied by Lemma 4.1,
and requires time proportional to the number of nonzeros in the matrix. The iden-
tication of irreducible blocks in step (2) can be performed with a breadth-rst or
depth-rst search algorithm, also requiring time proportional to the number of nonze-
ros. Combining the permutations of the resulting blocks with a P-node is correct by
Lemma 4.2.
Step (3) handles the boundary conditions of the recursion, while in step (4) the
Fiedler vector is computed and sorted. If there are no repeated elements in the Fiedler
vector then the Q-node for the permutation is correct by Theorem 3.3. Steps (3) and
(4) are the dominant computational steps and we will discuss their run time below.
The recursion in step (5) is justied by Theorem 4.7.
Note that this algorithm produces a tree whether A is pre-R or not. To determine
whether A is pre-R, simply apply one of the generated permutations. If the result
11
is an R-matrix then all permutations in the PQ-tree will solve the seriation problem,
otherwise the problem is not well posed.
The most expensive steps in algorithm Spectral-Sort are the generation and
sorting of the eigenvector. Since the algorithm can invoke itself recursively, these
operations can occur on problems of size n, n ? 1, : : :,1. So if the time for an eigen-
calculation on a matrix of size n is T (n), the runtime of algorithm Spectral-Sort is
O(n(T (n) + n log n)).
A formal analysis of the complexity of the eigenvector calculation can be simplied
by noting that for a Pre-R matrix, all that matters is the dominance relationships be-
tween matrix entries. So, without loss of generality, we can assume that all entries are
integers less than n2 . With this observation, it is possible to compute the components
of the Fiedler vector to a sucient precision that the components can be correctly
sorted in polynomial time. We now sketch one way this can be done, although we
don't recommend this procedure in a real-world implementation.
Let  denote a specic eigenvalue of L, in our case the Fiedler value. This can
be computed in polynomial time as discussed in [25]. Then we can compute the
corresponding eigenvector x symbolically by solving
(L ? zI )x = 0 mod p(z )
where p(z ) is the characteristic polynomial of L. Gaussian elimination over a eld is in
P [21], so if p(z ) is irreducible we obtain a solution x where each component xi is given
by a polynomial in z with bounded integer coecients. We note that letting z be any
eigenvalue will force x to be a true eigenvector. If p(z ) is reducible, we try the above.
If we fail to solve the equation, we will instead nd a factorization of p(z ) and proceed
by replacing p(z ) with the factor containing  as a root. This yields a polynomial
formula for each xi and we can identify equal elements by e.g. the method in [22]. To
decide the order of the remaining components, we evaluate the root  to a sucient
precision and then compute the xi 's numerically and sort. Since  is algebraic, the
xi's cannot be arbitrarily close [22] and polynomial precision is sucient.
In practice, eigencalculations are a mainstay of the numerical analysis community.
To calculate eigenvectors corresponding to the few highest or lowest eigenvalues (like
the Fiedler vector), the method of choice is known as the Lanczos algorithm. This is
an iterative algorithm in which the dominant cost in each iteration is a matrix-vector
multiplication which requires O(m) time. p The algorithm generally converges in many
fewer than n iterations, often only O( n) [26]. However, a careful analysis reveals a
dependence on the dierence between the distinct eigenvalues.
6. The consecutive ones problem. Ordering an R-matrix is closely related
to the consecutive ones problem. As mentioned in x1, a (0; 1)-matrix C has the
consecutive ones property if there exists a permutation matrix  such that for each
column in C , all the ones form a consecutive sequence.3 A matrix that has this
property without any rearrangement (i.e.  = I ) is in Petrie form4 and is called a P-
matrix. Analogous to R-matrices, we say a matrix with the consecutive ones property
is pre-P. The consecutive ones problem can be restated as: Given a pre-P matrix C ,
nd a permutation matrix  such that C is a P-matrix.
3
Some authors dene this property in terms of rows instead of columns.
4
Sir William M. F. Petrie was an archaeologist who studied mathematical methods for seriation
in the 1890's.
12
 There is a close relationship between P-matrices and R-matrices. The following
results are due to D.G. Kendall and are proved in [19] and [33].
Lemma 6.1. If C is a P-matrix, then A = CC T is an R-matrix.
Lemma 6.2. If C is pre-P and A = CC T is an R-matrix, then C is a P-matrix.
Theorem 6.3. Let C be a pre-P matrix, let A = CC T , and let  be a permutation
matrix. Then C is a P-matrix if and only if AT is an R-matrix.
This theorem allows us to use algorithm Spectral-Sort to solve the consecutive
ones problem. First construct A = CC T , and then apply our algorithm to A (note that
the elements of A are small non-negative integers). Now apply one of the permutations
generated by the algorithm to C . If the result is a P-matrix then all the permutations
produce C1P orderings. If not, then C has no C1P orderings.
The run time for this technique is not competitive with the linear time algorithm
for this problem due to Booth and Lueker [5]. However, unlike their approach, our
Spectral-Sort algorithm does not break down in the presence of errors and can
instead serve as a heuristic.
Several other combinatorial problems have been shown to be equivalent to the
consecutive ones problem. Among these are recognizing interval graphs [5, 12] and
nding dense envelope orderings of matrices [5].
One generalization of P-matrices is to matrices with unimodal columns (a uni-
modal sequence is a sequence that is non-decreasing until it reaches its maximum,
then non-increasing). These matrices are called unimodal matrices [32]. Kendall [20]
showed that the results 6.1 { 6.3 are also valid for unimodal matrices if the regular
matrix product is replaced by the matrix circle product dened by
X
(A  B )ij = min(aik ; bkj ):
k
Note that P-matrices are just a special case of unimodal matrices, and that the circle
product is equivalent to matrix product for (0; 1)-matrices. Kendall's result implies
that our spectral algorithm will correctly identify and order unimodal matrices.
Acknowledgements. We are indebted to Robert Leland for innumerable discus-
sions about spectral techniques and to Sorin Istrail for his insights into the consecutive
ones problem and his constructive feedback on an earlier version of this paper. We
are further indebted to David Greenberg for his experimental testing of our approach
on simulated genomic data, and to Nabil Kahale for showing us how to simplify the
proof of Theorem 3.2. We also appreciate the highly constructive feedback provided
by an anonymous referee.
REFERENCES
[1] F. Alizadeh. Interior point methods in semidenite programming with applications to combina-
torial optimization. SIAM J. Optim., 5:13{51, 1995.
[2] N. Alon and N. Kahale. A spectral technique for coloring random 3{colorable graphs. In Proc.
26th Annual Symposium on Theory of Computing, pages 346{355. ACM, 1994.
[3] B. Aspvall and J. R. Gilbert. Graph coloring using eigenvalue decomposition. SIAM J. Alg.
Disc. Meth., 5(4):526{538, 1984.
[4] S. T. Barnard, A. Pothen, and H. D. Simon. A spectral algorithm for envelope reduction of
sparse matrices. In Proc. Supercomputing '93, pages 493{502. IEEE, 1993.
[5] K. S. Booth and G. S. Lueker. Testing for the consecutive ones property, interval graphs, and
graph planarity using PQ{tree algorithms. JCSS, pages 333{379, 1976.
13
 [6] F. R. K. Chung and S.-T. Yau. A near optimal algorithm for edge seperators (preliminary
version). In Proc. 26th Annual Symposium on Theory of Computing, pages 1{8. ACM, 1994.
[7] D. M. Cvetkovic, M. Doob, I. Gutman, and A. Torgasev. Recent Results in the Theory of Graph
Spectra. North-Holland, 1988. (Annals of Discrete Mathematics 36).
[8] D. M. Cvetkovic, M. Doob, and H. Sachs. Spectra of Graphs: Theory and Application. Academic
Press, 1980.
[9] W. E. Donath and A. J. Homan. Lower bounds for the partitioning of graphs. IBM J. Res.
Develop., 17:420{425, 1973.
[10] M. Fiedler. Algebraic connectivity of graphs. Czech. Math. Journal, 23:298{305, 1973.
[11] M. Fiedler. A property of eigenvectors of nonnegative symmetric matrices and its application
to graph theory. Czech. Math. Journal, 25:619{633, 1975.
[12] D. R. Fulkerson and O. A. Gross. Incidence matrices and interval graphs. Pac. J. Math.,
3:835{855, 1965.
[13] A. George and A. Pothen. An analysis of spectral envelope{reduction via quadratic assignment
problems. Technical Report ICASE Technical Report 94{81, NASA LaRC, Hampton, VA,
September 1994. To appear in SIAM J. Matrix Anal.
[14] M. X. Goemans and D. P. Williamson. .878{approximation algorithms for MAX CUT and MAX
2SAT. In Proc. 26th Annual Symposium on Theory of Computing, pages 422{431. ACM,
1994.
[15] D. S. Greenberg and S. C. Istrail. Physical mapping by STS hybridization: Algorithmic strategies
and the challenge of software evaluation. J. Computational Biology, 2(2):219{274, 1995.
[16] M. Grotschel, L. Lovasz, and A. Schrijver. Geometric Algorithms and Combinatorial Optimiza-
tion. Springer-Verlag, 1988.
[17] M. Juvan and B. Mohar. Optimal linear labelings and eigenvalues of graphs. Disc. Appl. Math.,
36:153{168, 1992.
[18] D. Karger, R. Motwani, and M. Sudan. Approximate graph coloring by semidenite program-
ming. J. of the ACM, 1996.
[19] D. G. Kendall. Incidence matrices, interval graphs and seriation in archaeology. Pac. J. Math.,
28(3):565{570, 1969.
[20] D. G. Kendall. Abundance matrices and seriation in archaeology. Zeitschrift fur Wahrschein-
lichkeitstheorie, 17:104{112, 1971.
[21] D. C. Kozen. The Design and Analysis of Algorithms. Springer{Verlag, 1992.
[22] M. Mignotte. Identication of algebraic numbers. J. Algorithms, 3:197{204, 1982.
[23] B. Mohar. The Laplacian spectrum of graphs. In Y. Alavi et al., editor, Graph Theory, Combi-
natorics and Applications, pages 871{898. J. Wiley, New York, 1991.
[24] B. Mohar. Laplace eigenvalues of graphs { a survey. Disc. Math., 109:171{183, 1992.
[25] V. Pan. Algebraic complexity of computing polynomial zeros. Comput. Math. Applic., 14(4):285{
304, 1987.
[26] B. Parlett and D. Scott. The Lanczos algorithm with selective orthogonalization. Math. Comp.,
33:217{238, 1979.
[27] O. Perron. Zur Theorie der Matrizen. Math. Ann., pages 248{263, 1907.
[28] A. Pothen, H. D. Simon, and K.-P. Liou. Partitioning sparse matrices with eigenvectors of
graphs. SIAM J. Matrix Anal., 11(3):430{452, July 1990.
[29] W. S. Robinson. A method for chronologically ordering archaeological deposits. American
Antiquity, 16(4):293{301, 1951.
[30] R. S. Varga. Matrix Iterative Analysis. Prentice{Hall, 1962.
[31] M. Veldhorst. Approximation of the consecutive ones matrix augmentation problem. SIAM J.
Comput., 14(3):709{729, 1985.
[32] E. M. Wilkinson. Mathematics in the Archaeological and Historical Sciences, chapter Archae-
ological Seriation and the Travelling Salesman Problem, pages 276{284. University Press,
Edinburgh, 1971.
[33] E. M. Wilkinson. Techniques of data analysis and seriation theory. In Technische und Natur-
wissenschaftliche Beitrage Zur Feldarchaologie, pages 1{142. Rheinland{Verlag, 1974.

14