Integer Programming and Cryptography

H. W. Lenstra, Jr.
Not long ago it was reported m the press that Adi
Shamir, from the Weizmann Institute of Science m Is-
rael, had broken one of the first public key cryptosys-
tems, the Merkle-Hellman knapsack System Saentific
American (August 1982, p 79) reported maccurately
that
He did so by provmg that a mathematical problem called the knap
sack problem, which had been considered e^ceedmgly dtfficult, am
be solved rapidly by a simple Computer algonthm
Saentific American summanzed the purported method
of solution s follows
Shamir was able to solve the knapsack problem after he noted that
it could be transformed into a mathematically eqmvalent problem
in integer programming Shamir showed that the mteger-program-
mmg problem to which the knapsack problem is equwalent can be
solved by an algonthm recently mvented by Hendrik W Lenstra of
the Unwersity of Amsterdam Hence Lenstra s algonthm can also
solve the knapsack problem
It is the purpose of this article to explam exactly what
has and has not been proved and to outline the
methods that were used Shamir did not find a way to
solve the general knapsack problem, which appears to
be computationally intractable What he did do was to
find a way to use a recent integer programming algo-
nthm to solve a special type of knapsack problem that
occurs m cryptography
Useful background Information about cryptology
and linear programming can be found in the Mathe-
matical Intelhgencer articles by Simmons [1] and Lovsz
[2], respectively
Integer Programming
For our purposes, tbe integer programming problem is
most convemently formulated s follows Let n and m
be positive integers and let real n-vectors a
t
and real
numbers b
t
be given, for = l, 2, , m The problem
is to decide whether or not there exists an n vector
with integral coordmates x, satisfymg the inequahties
a
t
x b
t
for = l, 2, , m (1)
We shall assume throughout that the b, and the coor
dmates of the a, are integers This is not a substantial
restnction if they are rational, and allowmg more gen
eral real numbers leads to the question of how to
specify these exactly, which I do not want to discuss
Notice that we formulated the problem s a deasion
problem, which has the answer "yes" or "no " There
exist other versions of the integer programming
problem For example, if the answer is "yes," we may
ask for an actual integer vector satisfymg (1) to be
exhibited, or we may ask for such an maximizmg ex,
where c is a given n vector with integer coordmates
But all these versions are equivalent in the sense that
an efficient method for solvmg one of them easily leads
to an efficient method for solvmg the others
Efficient Algorithms
We are mterested m an algonthm for solvmg the in
teger programming problem that not only gives the
correct answer but also does so withm a reasonable
time Here "reasonable" can be exactly defmed The
time should be bounded by a polynomial functwn of the
length ( of the problem This length should be thought
of s the time it takes to wnte down the (n + l)m
coordmates of the n-veciors a, and the m-vector b Let
A denote the maximum absolute value of these coor-
dmates Then each coordmate has at most a constant
times log(A + 2) binary digits, so for our purposes we
can take i = (n + l)m log(A + 2)
If the integrality constramt on the coordmates of the
solution vector is dropped, then such a polynomial
algonthm indeed exists For a discussion of this algo
nthm, discovered by L G Khachiyan, we refer the
reader to the article by Lovsz [2]
For the integer programming problem, no poly-
nomial algonthm is likely to exist, since the problem
is NP-complete This means, roughly speaking, that it
is at least s difficult s many other problems that are
notonous for their computational intractability, such
s the travelmg salesman problem and the problem of
decomposing an integer into prime factorssee [3] for
a fuller discussion
The new result on integer programming that Scien-
tific American refers to is the following For every fixed
value of n, the number of variables, there does exist a
polynomial algonthm for solvmg the integer program-
14 THE MATHEMATICAL INTELLIGENCER VOL 6 NO 3 O 1984 Springer Verlag New York
ming problemsee [4]. This does not contradict the
previous paragraph: A running time 2" , for example,
would be polynomially bounded for fixed n, but not
in generaJ. (My algorithm is actually much slower.)
We shall now first describe the basic ideas behind
this new algorithm and discuss its cryptographic sig-
nificance later.
Integral Points in a Triangle
It is trivial to design an algorithm for the integer pro-
gramming pioblem with one variable: It suffices to per-
form a series of divisions and comparisons. These can
be done in polynormal time, just like the other arith-
metic operations such s addition, subtraction, and
multiplication.
The two-variable case is already nontrivial. Let K be
the plane region described by (1):
K = {x e R
2
: a,x b, for i =
m}
This is a convex set bounded by at most m straight line
segments. We shall consider the special case that K is
a triangle. Then the question becomes: How does one
decide, in polynomial time, whether a given triangle in the
plane contains a point with integral coordinates? It makes
no difference whether the triangle is given by the
equations a,x = b, (i = l, 2, 3) defining its edges, or
by the rational coordinates of the three vertices, since
it is easy to compute, in polynomial time, the vertices
from the edges and the edges from the vertices.
The reader who thinks very briefly about the above
problem will probably react: Draw the triangle and
look. He will argue that either the triangle is "large,"
in which case it must obviously contain an integral
point, or the triangle is "small," and then it is con-
tained in a small rectangle, all integral points in which
can be checked one by one.
If one tries lo make this argument precise, one dis-
covers that it works all right for the decent-looking
triangles one finds in geometry textbooks, but that a
problem is presented by triangles that are very lang
and very thin. They are too thin to obviously contain
an integral point, and so long that there are more in-
tegral points near the triangle than can possibly be
enumerated in polynomial time.
There are several ways to deal with such triangles.
It can be done with the help of continued fractions,
but I will avoid them in my discussion and describe a
method that generalizes better to higher dimensions.
The solution essentially consists of denying that
"special" triangles exist. If the triangle K looks a bit
weird, why not apply a nonsingular linear transfor-
mation such that the triangle [ ] looks better? To be
specific, choose so that the latter triangle is eqmlateml.
Clearly, K contains an element of Z
2
if and only if
Figure l
the new triangle [ ] contains an element of the lattice
L = [
2
] . If BI , e
2
are the Standard basis vectors of R
2
,
then ( {), (
2
) form a basis for L in the sense that L
The problem has now been shifted from the triangle
to the lattice. To describe the latter, it is notationally
convenient to identify R
2
in the usual way with the
complex plane C. It is a classical result that L, s every
lattice in C, has a basis y
lf
y
2
with the property that z
= y
2
lyi belongs to the well-known fundamental do-
main for the modular group:
Im z > 0 |Re z =
l
lz \z\ ^ l
Moreover, there exists a fairly straightforward algo-
rithm that transforms a given basis for L into the ba-
sis y
if
y
2
.
We need to know one more thing about L. Let h
denote the distance of y
2
to the line Ri/j (see Figure 1).
It is an elementary exercise to prove that the covenng
radius of L is at most /z/V2; i.e., closed discs of radius
/ / /2 centered at the points in L cover the whole com
plex plane:
For every M G C there exists y
such that u y\ h/ V2
L
(2)
The remaining part of the solution is very much like
the naive argument we started with. There are again
two cases. Denote by e the edge length of the equi
lateral triangle [ ]. In the first case the triangle is
large, i.e.,
///2
Here eV3/ 6 is the radius of the inscribed circle of the
triangle, so applying (2) with u equal to the center of
THEMA1HEMATICALINTELLIGENCERVOL 6, NO 3,1984 15
Figure 2
this circle we see that in this case there is indeed a
lattice point y in the triangle (see Figure 2).
In the second case the triangle is small:
so e/h < \ / 6. Observing that the parallel lines
fa/ 2 + k e z
have successive distances h from each other (see Figure
3) one easily proves that in this case no more than [ / 6]
+ l = 3 of these lines intersect the triangle. Since
every lattice point is on one of these lines, it now suf
fices to check these lines one by one, and this can be
done without difficulty.
If all details in this decision procedure are made ex
plicit, it turns out that the resulting algorithm runs
indeed in polynomial time.
Higher Dimensions
The above algorithm can be extended to the general
integer programming problem. We give only a brief
sketch. Let again K be the closed convex set described
by (1):
K = {x e R: a, x = b
t
for i = l, 2, . . . , m}
It can be shown that there is no loss of generality in
assuming that K is bounded and has positive volume.
One begins by constructing a nonsingular linear
transformation such that [ ] has a "round" appear
ance in the sense that the ratio
Outer radius of [ ]
Inner radius of [ \
is bounded above by a constant depending only on n.
Here the outer radius of [ ] is the radius of the smallest
sphere containing [ ], and the inner radius is the ra
dius of the largest sphere contained in it. Using Kha
chiyan's linear programming algorithm, Lovsz has
shown that such a transformation can be found in
polynomial time, even for varying n.
It is now to be decided whether [ ] intersects the
lattice L = [ "]. this end one constructs a basis y
lt
y
2
, , y for L that is reduced in the sense that
Volume{E?
=1
r
t
y
t
:
r, e R
0
1}
is bounded below by a constant depending only on n.
Notice that this ratio is always =sl, with equality if and
only if the y, are pairwise orthogonal. Thus a reduced
basis is "nearly orthogonal." There exists a polynomial
algorithm for finding such a basis, even for varying n.
This observation is again due to Lovsz, and his basis
reduction algorithm has several other applications, no-
tably to the factorization of polynomials [5].
As with the triangle, there are now two cases. Let it
be supposed that y is the longest of yi, y
2
, , y
n
and denote by h the distance of y to the hyperplane
?!} Ry,. In the first case, the inner radius of [ ] is
so much larger than h that the required lattice point in
[ ] exists by an analog of (2). In the other case one
proves that the number of integers k for which the
hyperplane
k y + Hl \ Ry,
meets the convex set [ ] is bounded by a constant
depending only on n. Since every lattice point is on
one of these hyperplanes, it suffices to investigate
these values of k one by one. For a fixed value of k one
obtains an integer programming problem with only n
l variables, and this problem can be solved by re
cursion.
This finishes the sketch of the algorithm. It can be
shown that for fixed n the algorithm runs in poly
nomial time.
Lovsz' two auxiliary algorithms mentioned above
were in fact invented later. They replace, and were
partly motivated by, earlier algorithms that were only
polynomial for fixed n.
Applications
So far, I have not heard of an actual Implementation
of the algorithm just described. This seems to indicate
that its practical value is rather limited. It is my un-
derstanding that the theoretical requirement that n be
fixed implies the practical requirement that n be small,
but that for small n older algorithms are adequate.
On the other hand, there is the application to cryp-
tography explained below. But even here it may be
16 THEMAIHEMATICAL1NTELL1GENCERVOI 6, NO 3, 1984
Figure 3
argued that this is an application not of the whole
integer programming algorithm but of the basis re-
duction algorithm that was used s a subroutine.
The Knapsack Problem
The knapsack problem is formulated s follows. Given
positive integers

, a
2
, , a
n
, b it is to be decided
whether there exists a subset I C. {l, 2, . . . , n} such
that
That is, given a knapsack of capacity b, and n items of
sizes , , . . . , a
n
, it is to be decided whether the
knapsack can be filled to capacity with a subset of
these items.
lf a denotes the n vector with coordinates a\ ,
0.2, , a
n
, it is clearly equivalent to ask whethei there
exists an n vector with integral coordinates x
}
such
that
ax =
0 s ;
for ] = 1, 2, .
This is an instance of the integer programming
problem, with m = I n + 2. However, the new integer
programming algorithm is of no use in solving the
knapsack problem. It would, in fact, be faster to apply
complete enumemtion, i.e., to try the 2" vectors e {0,
1}" one by one.
Below we shall encounter the knapsack problem in
a slightly different formulation: If a set I s above ex-
ists, we also want to find it. But it is easy to see that
both versions are equivalent, in the same sense s this
was the case for the integer programming problem.
No polynomial algorithm for solving the knapsack
problem is known, and since the knapsack problem is
NP-complete no such algorithm is expected to exist;
see [3].
Shamir has not found a way to solve the general
knapsack problem. What he has solved is a special
type of knapsack problem that occurs in cryptography,
which we shall now describe.
Cryptographic Knapsacks
The knapsack problems that occur in cryptography are
of a very special type. They have a hidden structure,
knowledge of which enables one to solve them in a
trivial manner. Before I describe how such knapsacks
are constructed, let me briefly indicate their use in
cryptography. For background, see Simmons' article [1].
Someone, to be called the sender, wishes to send a
certain message to someone eise, the receiver, It is sup-
posed that the message is represented s a sequence
x
= (x.)f
=l
E {0, 1}" of n "bits," for a suitable number
n. The message is to be sent over a public channel in
such a way that someone who listens inthe eaves-
dropperis not able to reconstruct the message x.
To this end the sender proceeds s follows. He looks
up the receiver's name in a public file, such s a tele-
phone directory, and there he finds n numbers e
l7
a
2
, . . . , fl. Next he sends to the receiver, instead of
the message x, the number b defined by
b = ' ;
= 1
,
;
After reception of b, the receiver uses the hidden struc
ture of flj,
2
/ /
a
n
to
solve the knapsack problem
and to recover the original message (x
lr
x
2
, . . . , x
n
).
MATHEMAT1CAL INTELLIGENCER VOL 6, NO 3, 1984 1 7
The eavesdropper knows a\, a
2
, . . . , a
n
trom the
public file, and he knows b by listening in to the public
channel, but he does not know the hidden structure.
Consequently, he is apparently faced with the task of
solving a general knapsack problem, for which no
good algorithm is known, and he will presumably be
unable to reconstruct the message.
How did the receiver construct the numbers a\,
a
2
, . , . , a
n
that were put into the public file? Several
methods to do this have been proposed by R. C.
Merkle and M. E. Hellman [6], to whom the above
idea is due, and it is only the simplest of these
methods that has been proved insecure by A. Shamir.
It is s follows.
A very easy knapsack problem to solve, not only for
the receiver but also for the eavesdropper, is one in
which the sequence a\, a
2
, . . . , is superincreasing.
This means that each a, is greater than the sum of its
predecessors:
a, > 2jr} a
}
for l = i ===
n
For such a knapsack problem, one must clearly have
x = l if b & a, and x = 0 if b < a
n
; in a similar way
x
n
-i, x
n
-2, . , x\ are successively determined.
When constructing his knapsack, the receiver Starts
from such a superincreasing sequence a{, a
2
, . . . , a'.
To hide its obvious structure, he chooses two secret
numbers u (the multiplier) and m (the modulus) satis-
fying
m>^
n
] = l
a] gcd(u, m) = l
He now defines
;
by
= ua', mod m 0 < a
m
and
l7
a-i, . . . , a
n
are the numbers he makes publicly
available; but a{, a
2
, . . . , a'
n
, u, m he keeps to himself.
To decode a received message b = ^
= :
, , the re
ceiver proceeds s follows. Using the Euclidean algo-
rithm he determines an integer w satisfying wu = l
mod m; this is the inverse multiplier. Next he calculates
the number b' defined by
b' = wb mod m 0 =s b' < m
Using the fact that a'
;
= wa, mod m and the inequality
m > "
= 1
a'j one now easily proves that
V =
= 1

;
'
;
Since the a'
}
are superincreasing, the x, can be solved
from this. The eavesdropper does not know w or m,
nor any of the a'
;
, and is therefore supposedly unable
to carry out the required transformation.
Shamir's Attack
Shamir devised an algorithm for solving knapsack
Problems k nown to have a hidden structure s described
above, but without the numbers u and m being known.
His algorithm solves most such knapsack problems but
is not guaranteed to solve them all; this is, however,
s he writes, "not a severe handicap in the context of
cryptography, since a cryptosystem becomes useless
when most of its keys can be efficiently cryptana-
lyzed" [7].
The performance of Shamir's algorithm may be for-
mulated s follows. Let a real number d > l be fixed,
to be thought of s the ratio
Number of bits of the encoded message
Number of bits of the original message
which is about (log b)/(n log 2). Let further an integer
m < 2
dn
be fixed. By S we denote the set of crypto-
graphic knapsacks with modulus m; so the elements
of S correspond one to one with the sequences (a{, a'
2
,
. . . , a'
n
, u) of positive integers satisfying
a{, a'i, . . . , a'n is superincreasing
^
=
j a'j < m
(u, m) = l M =s m
With this notation, it can be proved that a suitable
version of Shamir's algorithm solves almost all prob
lems in S, in the sense that the fraction it cannot solve
tends rapidly to zero s n tends to infinity. Further, for
fixed d the running time of Shamir's algorithm is
bounded by a polynomial function of n.
Shamir claims a proof of this only for d < 2; the
general case was proved by J. C. Lagarias (Bell Labo-
ratories).
It is the purpose of Shamir's algorithm to calculate,
given flj, iz
2
, . . . , a, new numbers w' and m' that can
be used for exactly the same purpose s w and m; that
is, there should exist numbers a'{, a'
2l
. . . , a satisfying
a" =
j mod m' ior l =S: j ^ n
a'{, a'
2
, . . . , a'n is a superincreasing sequence
"
= 1
It turns out that all pairs (w' , m'} for which w'/m' is
sufficiently close to w/m have this property. The object
of Shamir's algorithm is thus to find a good enough
Diophantine approximation w'/m' to w/m.
The main idea of Shamir's method and its relation
to integer programming are s follows. We have a', =
wa
}
mod m, so
- \fjrn
-i
(3)
for certain integers y
lf
y
2
, . . . , y. Here the cryptan-
alyst only knows the
;
, all the others are unknowns.
But he also knows that the a'
}
form a superincreasing
sequence, and from this it can be deduced that for
small j the numbers d
}
are quite small with respect to
m. Dividing (3) by a
;
m we therefore see that, say, the
numbers yj/flj, y
2
/a
2
, 1/3/03, y
4
/
4
are close to w/m and
therefore also close to each other. This leads to two-
18 THE MATHbMAriCAL INfELUChNCER VOL 6, NO 3, 1984
sided inequalities for the three numbers
a
, Vi ~
a
\ y, = 2, 3, 4
These inequalities, taken together with 0 < y
;
< a
)t
give
rise to a four dimensional integer programming
problems from which y
v
J/2' i/ 3' J/4
c a n
be solved. At
this point it must be shown that this four dimensional
integer programming problem is not likely to have
many extraneous Solutions for which y [/ a\ is not close
w/ m. This can be done under the assumption
that d < 2.
Once i/ ], 1/2, 1/3, 1/4 have been found, one knows a
nearly good enough approximation y\ la\ to w/ m. Using
Diophantine approximation techniques Shamir is then
usually able to find the desired numbers w' and m'.
This concludes my sketch of Shamir's method.
For higher values of d one must solve integer pro
gramming problems with more variables. According
to }. C. Lagarias, [d] + 2 variables suffice for d 5= 3.
Current research is directed toward the problem of
solving other, more complicated cryptographic knap
sacks proposed by M erkle and Hellman and by others.
Known attacks on these Systems use special properties
of cryptographic knapsacks which enable cryptana
lysts to apply Diophantine approximation tools, espe
cially Lovsz' basis reduction algorithm, to solve them.
None of these attacks, however, apply to general
knapsack problems.
Acknowledgments. I am indebted to }. C. Lagarias
and A. M. Odlyzko for commenting on earlier ver-
sions of this article, and to F. }. van der Linden for
preparing the figures.
References
1. G.]. Simmons (1979) Cryptology: The mathematics of
secure communication. Math. Intelligencer l(4):233-246
2. L. Lovsz (1980) A new linear programming algorithm
Better or worse than the simplex method? Math. Intelli-
gencer 2(3):141-146
3. M. R. Garey, D. S, Johnson (1979) Computers and Intract-
ability: A Guide to the Theory of NP-completeness. San Fran-
cisco: Freeman
4. H. W. Lenstra, Jr. (1983) Integer programming with a
fixed number of variables. Math. Oper. Res 8 (4) (in
press)
5. A. K. Lenstra, H. W. Lenstra, Jr., L. Lovsz. (1982) Fac-
toring polynomials with rational coefficients. Math. Ann.
261:515-534
6. R. C. Merkle, M. E. Hellman (1978) Hiding Information
and signatures in trap-door knapsacks. IEEE Trans. Inf.
Theory, IT-24-5, pp. 525-530
7. A. Shamir (1982) A polynomial time algorithm for
breaking the basic Merkle-Hellman cryptosystem, Proc.
23rd IEEE Symp. Found. Computer Sei. pp. 145-152
Mathematisch Instituut
Universiteit van Amsterdam
Roetersstraat 15
1018 WB Amsterdam
The Netherlands