PHYSICAL REVIEW E 71, 066707 共2005兲

Optimization by quantum annealing: Lessons from hard satisfiability problems

Demian A. Battaglia,1,* Giuseppe E. Santoro,1,2,† and Erio Tosatti1,2,‡

1
International School for Advanced Studies (SISSA), and INFM Democritos National Simulation Center,
Via Beirut 2-4, I-34014 Trieste, Italy
2
International Center for Theoretical Physics (ICTP), Trieste, Italy
共Received 18 February 2005; published 29 June 2005兲

The path integral Monte Carlo simulated quantum annealing algorithm is applied to the optimization of a
large hard instance of the random satisfiability problem 共N = 10 000兲. The dynamical behavior of the quantum
and the classical annealing are compared, showing important qualitative differences in the way of exploring the
complex energy landscape of the combinatorial optimization problem. At variance with the results obtained for
the Ising spin glass and for the traveling salesman problem, in the present case the linear-schedule quantum
annealing performance is definitely worse than classical annealing. Nevertheless, a quantum cooling protocol
based on field-cycling and able to outperform standard classical simulated annealing over short time scales is
introduced.

DOI: 10.1103/PhysRevE.71.066707 PACS number共s兲: 02.70.Uu, 07.05.Tp, 02.70.Ss, 03.67.Lx

I. INTRODUCTION man problem 关9兴. Here the quantum algorithm outperformed

classical simulated annealing 关11兴 and the convergence time
The aim of theoretical quantum computation is to under- was significantly shorter, even after taking into account the
stand how the peculiar properties of quantum states can be extra cost of simulating a quantum computation on a classi-
exploited in order to encode and manipulate information in a cal computer 关7,9兴.
way that is quantitatively superior to classical computers 关1兴. Nevertheless, very few results of universal validity are
The elementary unit of quantum information, the so-called known to date, and theoretical indications are poor and
qubit, is in general assumed to be a normalized vector in a sometimes contradictory 关3,12–14兴. In particular, it has not
Hilbert space H spanned by two orthogonal eigenstates, con- been proved that a good quantum optimization pathway must
ventionally labeled 兩0典 and 兩1典, and it is then completely de- always exist in general. In the cases in which a quantum
termined by giving two real numbers 共a component and a procedure proved to be more efficient than the best classical
relative phase兲. In principle, a quantum computation can be one, the superior quantum performance cannot be explained
seen as a sequence of appropriate unitary transformations quantitatively.
operating on sets of qubits, which requires having the quan- In this paper, we will present a careful analysis of the
tum state of the system under control for a sufficiently long PIMC-QA applied to the optimization of a hard instance of
time. the random boolean satisfiability problem 共see Sec. II for a
A completely different point of view is shared by a group definition and also 关15兴兲. The main emphasis will not be on
performance, but rather on understanding the properties that
of quantum computation paradigms that could be collectively
make simulated quantum annealing so different from its clas-
referred to as quantum ground state search techniques: two
sical counterpart. Because we are here mainly interested in
examples are given by the quantum adiabatic evolution tech- the comparison between algorithmic aspects, potentially rel-
nique and its variants 关2–6兴 and by the semi-classical simu- evant physical issues such as the study of aging and memory/
lated quantum annealing algorithms 关7–9兴. The basic idea rejuvenation phenomena 关16,17兴 will not be addressed di-
common to all these approaches is to simulate the externally rectly.
driven evolution of an artificial quantum system, whose The hardness of the chosen instance deserves some com-
ground states encode in some way the solutions of specific ments. It is common, in the quantum computation literature,
computational problems. The underlying hypothesis is that a to analyze the performance of newly introduced algorithms
quantum system can dynamically explore the state space by testing them on simplified toy problems of reduced size;
more efficiently than its classical counterpart. The positive this is essentially due to the demanding memory and time
role played by quantum fluctuations has been verified in a required by the classical simulation of a quantum computer.
number of different cases, both in real experiments 关10兴 and We decided instead to attempt the optimization of a true
in numerical simulations 关7,9兴. The path-integral Monte “competition problem,” of a kind known to threaten seri-
Carlo quantum annealing 共PIMC-QA兲 algorithm has been ously even the best heuristics known for the boolean satisfi-
applied with success, for instance, to the optimization of in- ability problem, namely WalkSAT 关18兴. Here smaller size
stances of the Ising spin glass 关7兴 and of the traveling sales- problems might indeed still be difficult, but the dynamics
would be dominated by large finite-size-fluctuation effects
关19兴 and misleading indications could be obtained about the
*Electronic address: [email protected] general properties of the quantum optimization of a glassy
†
Electronic address: [email protected] energy landscape, which becomes really “hard” only in the
‡
Electronic address: [email protected] thermodynamical limit.

1539-3755/2005/71共6兲/066707共10兲/$23.00 066707-1 ©2005 The American Physical Society

BATTAGLIA, SANTORO, AND TOSATTI PHYSICAL REVIEW E 71, 066707 共2005兲

For a hard satisfiability problem, one may wonder now if C1 ∧ C2 ¯ ∧ C M —each clause being formed by three vari-
the quantum system is actually able to tunnel across the ex- ables extracted at random among the N available, and ap-
tensive walls encircling the deep and scattered ground state pearing negated or directed with uniform probability—can
valleys. Here we expect that both classical and quantum an- be simultaneously satisfied by a truth value assignment
nealing will be trapped by local minima lying well above the 兵zi其. The optimization version of 3-SAT 共the so called
glassy threshold levels 关19,20兴, preventing access to the very MAX-3-SAT problem兲 is obtained if one is interested only in
low-energy region where clustering of states takes place. The finding a truth value assignment which minimizes the num-
comparison between the simulated classical and quantum re- ber of violated clauses. If we associate an Ising spin variable
laxations will nevertheless show that the two algorithms ex- Si = 共−1兲zi to each boolean variable zi, we can assign to any
plore sectors of the phase space characterized by different clause Ca involving three variables zi , z j , zk an energy Ea
geometrical properties. As we shall see, the quantum path- given by
ways tend to visit basins of attraction with a considerably
larger number of flat directions and this feature will turn out 共1 + Ja,iSi兲共1 + Ja,jS j兲共1 + Ja,kSk兲
to be highly counterproductive for an efficient optimization Ea = , 共1兲
in the specific case of the landscape under consideration. The 8
poor performance of quantum search techniques applied to
the 3-SAT problem, envisaged already in Ref. 关13兴, will then where the coupling Ja,i assumes the value −1 if the variable
find here a striking confirmation. zi appears negated in clause a, +1 otherwise. Evidently Ea
The paper is organized as follows. Sections II and III will = 0 if the corresponding clause is satisfied, Ea = 1 otherwise.
give respectively a brief description of the combinatorial op- Therefore, each given realization of the M random clauses is
timization problem under study and of the employed algo- associated to the Hamiltonian of a spin system with
rithms. In Sec. IV a first set of numerical experiments will be quenched disorder:
presented, while the analogy between PIMC-QA and
genetic-like algorithms will be pointed out in Sec. V. In Sec. M
VI the differences between quantum and classical Monte H共兵Si其兲 = 兺 Ea , 共2兲
Carlo dynamics will be highlighted by means of an autocor- a=1
relation analysis and of a characterization of the landscape
geometry. In Sec. VII a more sophisticated cooling schedule which simply counts the number of violated clauses. Notice
will be introduced, allowing the quantum algorithm to out- that the same variable appears typically in more than one
perform classical simulated annealing, at least over short clause, sometimes negated, sometimes not, giving thus origin
simulation times. In Sec. VIII inally, concluding comments potentially to frustration.
about open problems and possible future lines of develop- Statistical mechanics techniques can be used to determine
ment will be presented. the phase diagram of the Random 3-SAT problem 关26–28兴.
The main parameter in this phase diagram 共determining the
II. THE RANDOM BOOLEAN SATISFIABILITY hardness of a formula兲 is the ratio ␣ = M / N between the
PROBLEM number, M, of clauses and the number, N, of variables. The
probability that a satisfying assignment exists 共SAT formu-
Under many aspects, the boolean 3-satisfiability problem las兲 becomes one in the thermodynamical limit for ␣ ⬍ ␣c
共denoted as 3-SAT in the following, the meaning of the pre- ⯝ 4.267, but vanishes for larger ␣’s 共UNSAT formulas兲. Fur-
fix integer will be clarified very soon兲 can be considered as thermore, for ␣c ⬎ ␣ ⬎ ␣G ⯝ 4.15, the ground-state configura-
the prototype of most of the hard combinatorial optimization tions group in many clusters well separated among each
problems: it was indeed the first problem for which a direct other 共one-step replica symmetry breaking 关29兴兲, hidden to
proof of NP-completeness—the celebrated Cook’s theorem any local search heuristic by an exponentially larger number
关21兴—was ever obtained. Since then, the easiest way of of metastable states 共threshold states兲. A lower bound to the
proving the NP-completeness of some new problem has been energy of the threshold states acting as entropic traps is pro-
to show its equivalence with some 3-SAT instance 关15兴. Fur- vided by the so-called Gardner energy 关20兴.
thermore, 3-SAT is also of considerable practical relevance Numerical experiments 关30,31兴 are in agreement with the
in many engineering applications, notably in the field of ar- analytical predictions 关26–28兴, and confirm that instances of
tificial intelligence and automated planning 关22–25兴. random 3-SAT sampled in the vicinity of the SAT/UNSAT
In order to state the problem, consider a set of N boolean boundary ␣c are typically hard. The trapping effect induced
variables z1 , . . . , zN, where zi = 1 or 0 共“True” or “False”兲. by the threshold states cannot be neglected when the instance
Denoting by ␨i the variable zi or its negation z̄i, one then size is large 共N 艌 10 000兲 and large statistical fluctuations
considers the disjunction 共logical OR兲 of three variables C become sufficiently rare 关19兴. Smaller random formulas are,
= 共␨i ∨ ␨ j ∨ ␨k兲, which is called a 3-clause. In the more general on the other hand, often easily solvable by classical simu-
case of the K-SAT problem, one deals with disjunctions of K lated annealing. As already pointed out in the Introduction,
variable. It is possible to prove that the K-SAT problem is the efficiency of any new heuristics can then be significantly
NP-complete as soon as K3 共see, for instance, 关15兴兲. tested only over 3-SAT samples of a rather large size, which
The random 3-SAT problem consists in deciding if the is considerably demanding for long quantum simulation
conjunction 共logical AND兲 of M different clauses times.

066707-2
OPTIMIZATION BY QUANTUM ANNEALING: LESSONS… PHYSICAL REVIEW E 71, 066707 共2005兲

III. THE PATH-INTEGRAL MONTE CARLO SCHEME

Let us consider the classical spin model described by the

J⌫ = −
T
2
冉 冊
ln tanh
⌫
PT
⬎ 0. 共6兲

Hamiltonian 共2兲. In the well-known classical simulated an- When P goes to infinity, the partition functions of the Hamil-
nealing scheme 关11兴, the configuration of the system is ran- tonians 共4兲 and 共5兲 become identical, and the statistical-
domly initialized at an initial temperature T0 and it is then mechanical properties of the two systems become perfectly
updated according to, for instance, a standard Metropolis equivalent. One can then hope to simulate the relaxation dy-
Monte Carlo rule: namics of 共4兲 by applying a classical Metropolis algorithm to

冋 冉 冊册
Pt共flip兲 = min 1,exp −
⌬E
Tt
, 共3兲
a Suzuki-Trotter transformed Hamiltonian in Eq. 共5兲 关7兴. The
accuracy of the quantum simulation will increase for larger
values of P, but the memory and time requirements will also
where Pt共flip兲 is the probability of accepting a given spin flip increase accordingly.
at time t, ⌬E is the energy change induced by the flip, and Tt In the simplest possible Monte Carlo implementation, one
is the temperature at time t. The temperature is gradually tries to flip independently each of the PN spins of the equiva-
reduced according to a specific schedule that can be rather lent classical system, but it is common 关36兴 to perform in
influent on the performance of the algorithm 共see, for in- addition also global moves in which the spins Si,␳ corre-
stance, Refs. 关32,33兴兲. In the present paper we shall concen- sponding to a given site i are simultaneously flipped in all the
trate our attention on simple linear schedules, where the tem- Trotter replicas 共␳ = 1 , . . . , P兲. This type of move does not
perature is reduced by a fixed amount ⌬T every ␦ Monte affect the quantum kinetic energy contribution, but just the
Carlo complete sweeps of the systems 共we chose ␦ = 2 in our average classical energy. In a sense, the introduction of glo-
experiments, while ⌬T was completely determined by the bal moves allows one to simulate a constant-temperature
initial value T0 and by the maximum number of iterations兲. classical relaxation of the average classical energy, superim-
This is not in general the best performing choice, but its posed on the local modifications induced by quantum fluc-
simplicity makes the comparison with quantum annealing tuations at finite temperature.
very transparent. A schedule for the quantum algorithm must also be cho-
The classical Hamiltonian H共兵Si其兲, in analogy with the sen. The principal drawback of the PIMC approach is the
Ising glass case 关7,8兴, can be turned into a quantum Hamil- impossibility of simulating the quantum system directly at
tonian by substituting each classical Ising spin Si with the zero temperature. For this purpose, other quantum Monte
third component of a Pauli SU共2兲 spin operator ␴ˆ zi , and by Carlo schemes should be used, like the Green function
introducing a perturbing transverse field ⌫: Monte Carlo method 关37兴. The simplest possibility for a
PIMC quantum annealing schedule is, once again, to linearly
HQuantum = H共兵␴ˆ zi 其兲 − ⌫ 兺 ␴ˆ xi . 共4兲 adjust the intensity of the transverse field between two given
i extremes ⌫0 and ⌫ f , while keeping the temperature T fixed at
The ⌫-dependent term plays the role of “kinetic energy,” a constant value. More elaborated cooling protocols could be
inducing quantum fluctuations in the spin orientation. The devised, but the linear strategy has been applied with success
basic idea of quantum annealing is to drive the system 共4兲 for the optimization of the Ising spin glass 关7兴 and of the
toward its ground state by adiabatically varying the intensity traveling salesman problem 关9兴. Even for this very basic
of the perturbation field between two extremal values ⌫0 and schedule, four parameters, 共P , T , ⌫0 , ⌫ f 兲, need to be tuned
⌫ f —instead of gradually reducing the temperature, like in carefully in order to achieve a good performance. A more
the case of thermal simulated annealing. complex field-cycling schedule will be discussed in Sec. VII.
In the path integral Monte Carlo 共PIMC兲 scheme, the
statistical-mechanical behavior of the quantum spin model IV. NUMERICAL EXPERIMENTS: COMPARING
共4兲 is approximately turned into a classical simulation prob- CLASSICAL AND QUANTUM ANNEALING
lem by resorting to the Suzuki-Trotter transformation
关34,35兴. Let us consider the following classical model with Let us describe now a first set of numerical experiments,
PN degrees of freedom: performed over a single 3-SAT random instance with N
P P
= 104 and ␣ = 4.24 共i.e., right at the border of the SAT/
1 UNSAT transition兲, for several choices of annealing itera-
HST = 兺 H共兵Si,␳其兲 − J⌫␳兺=1 兺i Si,␳Si,␳+1 .
P ␳=1
共5兲 tions and Trotter replicas. The differences among random
samples extracted from the same ensemble become negli-
The system 共5兲 can actually be seen as composed of P rep- gible for such large sizes, and we can then carry out signifi-
licas 共Trotter replicas兲 兵Si,␳ , ␳ = 1 , . . . , P其 of the original clas- cant experiments without averaging over a large set of in-
sical configuration 兵Si其 at an effective quantum temperature stances.
Tq = PT, coupled among them by a nearest-neighbor trans- Using an efficient ad hoc algorithm 共the WalkSAT heuris-
verse ferromagnetic coupling J⌫. Periodic boundary condi- tic supplemented by a message-passing procedure 关18,19兴兲,
tions must be imposed along the transverse 共Trotter兲 direc- we verified that the chosen formula 共at ␣ = 4.24兲 was actually
tion. The intensity of J⌫ is related to the strength of the satisfiable, as expected from theory for ␣ ⬍ ␣c. An empirical
quantum perturbation term and to the temperature T = Tq / P parameter tuning was then performed for the initial tempera-
by the following relation: ture T0. In the case of linear-schedule classical annealing

066707-3
BATTAGLIA, SANTORO, AND TOSATTI PHYSICAL REVIEW E 71, 066707 共2005兲

most 106兲. In each point, an average has been taken over 50

different realizations of the same experiment; in the case of
QA, a second average was performed among the energies of
the P replicas, which are in general different. It can be seen
that the linear-schedule CA always performs better than the
linear-schedule QA, despite the improvement produced in
the latter by the use of a large number of replicas 共conver-
gence is essentially reached for P 艌 100, see inset of Fig. 1,
but we chose P = 50 to extend as much as possible the simu-
lation time兲. The asymptotic slope of the linear-schedule QA
curves seems indeed to be definitely less steep than that of
CA, independently of the number of replicas involved in the
simulation and of the use of global moves. This behavior is
strikingly worse than that observed in both the Ising spin
glass and the TSP cases 关7,9兴. A detailed study of the influ-
ence of landscape geometry might be helpful in order to
achieve a better understanding of this different performance
success. It should be remarked that large differences exist
between the organization of the optimal states in the combi-
natorial problems analyzed so far. For instance, it is gener-
ally believed that the ground state properties of the TSP 关38兴
are well described by a replica-symmetric ansatz 共all the
ground state assignments are in a single broad valley兲, while
replica symmetry-breaking 共i.e., clustering of ground states兲
FIG. 1. The comparison between optimal linear-schedule classi- is needed to achieve a good characterization of the 3-SAT
cal 共CA兲 and quantum annealing 共QA兲 for a 3-SAT problem with zero-temperature properties 关28兴. Furthermore, the states vis-
N = 104 and ␣ = M / N = 4.24. CA always performs better than QA ited by both CA and QA lie well above the glassy threshold
simulated with P = 50 Trotter replicas. The average performance of 共Gardner energy兲 for the 3-SAT instance, and very few de-
linear QA is worse than that of CA, even if an improvement in the tails are known about the nature of such highly excited
results can be obtained by introducing global moves 共G兲 and by states. An attempt at characterizing the local topology of the
increasing P. 共In the inset the final average energy found by QA visited configurations will be made in Sec. VI.
after 2000 iterations for increasing P is plotted and compared with
the average result of a CA of the same length, dashed line.兲 The
V. MONTE CARLO TIME DYNAMICS AND
solid triangles are the data obtained by the field-cycling QA hybrid
EVOLUTIONARY ANALOGY
strategy described in Sec. VII.
Let us now analyze in more detail the time evolution of
共CA兲, and fixing the maximum number of iterations, we con-
the energy during the annealing dynamics. We denote by
ducted several experiments with different T0. For T0 smaller
than the optimal value T0 = 0.3 the strength of the thermal 具具E典典 the configuration energy averaged over different ex-
fluctuations induced in the system was too small, and the periments and Trotter replicas 共this is the energy reported
dynamics was trapped prematurely at an early stage. On the everywhere in Fig. 1兲; the average among different experi-
other hand, for initial values larger than the optimal value, ments of the best replica energy will be, on the other hand,
too much time was wasted randomly wandering around the denoted as 具E典. In Fig. 2, the Monte Carlo “time” evolution
landscape at excessively high temperatures. A similar multi- profiles of 具具E典典 and 具E典 are shown for a linear-schedule QA
parameter optimization was performed in the case of linear- 共2000 iterations long兲.
schedule quantum annealing 共QA兲. The best results were ob- The strength of the transverse field, and hence of the
tained by taking ⌫0 ⯝ 0.7 and ⌫ f ⯝ 0.001, independently from quantum coupling J⌫ given by Eq. 共6兲 共see inset of Fig. 2兲,
the choice of the number of replicas P. The optimal value of determines the relative importance of the classical and quan-
the effective quantum temperature Tq = PT turned out to be tum terms in the Hamiltonians 共4兲 and 共5兲, and its variation
0.3, coincident with the optimal T0 for CA, a value evidently determines the transition between the three following ob-
more related to the statistics of the energy barriers of the served regimes:
formula landscape than to the details of the relaxation path- 共1兲 Quenching phase, ⌫ ⬇ ⌫0 ⯝ 0.7, J⌫ ⯝ 0. The quantum
way under consideration. system described by the Hamiltonian 共4兲 is quenched at tem-
A comparison between the performance of the optimal perature Tq in the presence of a strong external transverse
CA and the optimal QA at P = 50, both with and without field. The system enters an incoherent mixture of states.
global moves, is shown in Fig. 1. The graph shows the av- From the point of view of the PIMC simulation of Hamil-
erage residual energy 共difference between the final energy tonian 共5兲, when the coupling J⌫ is small, each replica be-
reached by the algorithm and the konwn ground-state en- haves as if roughly independent from the others. The algo-
ergy兲 as a function of the maximum number of total anneal- rithm is then effectively simulating P parallel Monte Carlo
ing iterations, ranging over several decades 共from 102 to al- dynamic processes at a constant temperature Tq. Both the

066707-4
OPTIMIZATION BY QUANTUM ANNEALING: LESSONS… PHYSICAL REVIEW E 71, 066707 共2005兲

tially to zero, but, if global moves are allowed, a further

energy reduction of moderate entity can be observed thanks
to small-range classical oscillations 共the global moves accep-
tance reaches now approximately the 20% level兲.
Considering this three-piece scenario 共that will be con-
firmed by the autocorrelation and geometrical analysis of
Sec. VI兲, the simulated QA could be described as a very
basic kind of evolutionary search 关39,40兴. The P replicas can
be seen as a population of individuals, the spin configuration
of each replica as its genotype, and the classical Hamiltonian
共2兲 as a fitness function. The simulation is not simply equiva-
lent to the selection of the best among P “restarts” 关41兴,
because here contiguous replicas can “mate,” exchanging se-
quences of their genotype thanks to the duplicating action of
the transverse coupling. A global decrease of the average
fitness is induced by the proliferation of spin patterns typical
of low-energy replicas 共the so-called Holland schemata
关42兴兲. Suppose now that some new exceptional individual
appears when J⌫ is already considerably large. The chance of
survival of the spin pattern responsible for its superior fitness
would be rather small, because it would be overwritten with
high probability by the corresponding subsequences in the
most widespread configuration. The population tends then at
a certain point to collapse toward a group of “identical
FIG. 2. The energy evolution during quantum annealing, com- twins.” Global moves cannot alter significantly this picture,
pared to simulated annealing. The variation of the averages 具E典 because they do not cure the problem of the lack of genetic
共average best replica兲 and 具具E典典 共average of the average replica兲 is diversity. In standard evolutionary search, the available gene
shown as a function of the simulation time, for a set of experiments pool is constantly renewed by crossover techniques, but their
with P = 50 and 2000 annealing iterations. The inset shows the time- introduction would not be physically justifiable in our PIMC
dependent value of the coupling J⌫. Three different regimes can be scheme. However, one is in principle allowed to increase the
distinguished, which will be called quenching, search 共driven by mutation rate by switching on again the quantum fluctua-
quantum fluctuations兲, and target selection. tions, a strategy that will be investigated in Sec. VII.
local and global move acceptances are extremely high in this
transient regime. The replicas assume different configura- VI. AUTOCORRELATION ANALYSIS AND LANDSCAPE
tions at similar energies, compatible with the quenching level PROBING
at temperature Tq.
共2兲 Search phase. After the abrupt out-of-equilibrium A better characterization of the dynamical behavior can be
quenching phase, quantum features like delocalization, inter- obtained by looking at the correlations among spin configu-
ference, and tunneling should help the system to evolve to- rations at different times, and by probing the geometry of the
wards attractive low-energy configurations. Looking at the neighborhood of the visited configurations.
Suzuki-Trotter Hamiltonian in Eq. 共5兲, with increasing cou- Let us denote by 兵Si共t兲其 the instantaneous spin configura-
pling strength, the fluctuations of the different replicas be- tion of the sample 3-SAT formula at time t. An autocorrela-
tion function K共t , ␶兲 can be defined as

冓 冔
come correlated. Some spin flips that would have been un-
likely in the absence of the replica-interaction term can now N
1
be accepted, thanks to the kinetic term, and several replicas
can enter in configurations generally not visited by typical
K共t, ␶兲 = 兺 Si共t兲Si共t − ␶兲 ,
N i=1
共7兲
CA trajectories. The acceptance ratio is constantly higher
than in the CA case and around 25%, when considering local where an average over different realizations of the dynamics
moves. It is negligible, on the other hand, for global moves. 共and over replicas in the QA case兲 has to be understood. The
共3兲 Target selection phase, ⌫ ⬇ ⌫ f , J⌫ large. When the autocorrelation function K共t , ␶兲 allows us to visualize in a
transverse field vanishes, quantum fluctuations are gradually compact way the typical behavior of the overlap between
switched off. The system then selects a low-energy target two spin assignments at different evolution instants. In Figs.
state and collapses completely into it. In this classical re- 3 and 4 we plot K共t , ␶兲 as a function of the autocorrelation
gime, when the transverse coupling J⌫ becomes very strong, time ␶ for several fixed values t* of the simulation time t, for
only local spin flips tending to align the largest number of the CA and QA dynamics, respectively 共in the plots, t* is
replicas can be accepted. All the replicas converge then to- generally increasing from bottom to top; see later in this
wards the same configuration, corresponding to the one vis- section兲. The results shown are averaged over 500 different
ited that has minimum energy, if the parameters have been runs, each of 2000 annealing iterations 共and over P = 50 rep-
carefully tuned. The acceptance for local moves falls essen- licas, in the case of QA兲. A K共t* , ␶兲 that decays fast with ␶

066707-5
BATTAGLIA, SANTORO, AND TOSATTI PHYSICAL REVIEW E 71, 066707 共2005兲

FIG. 3. The autocorrelation function K共t* , ␶兲 for CA. The differ- FIG. 4. The autocorrelation function K共t* , ␶兲 for QA. The dif-
ent curves represent the decay with ␶ of several fixed-simulation- ferent curves represent the decay with ␶ of several fixed-simulation-
time snapshots of the autocorrelation function for a CA experiment; time snapshots of the autocorrelation function, for QA experiments
t* is varying in the plot at fixed intervals between 200 and 2000, with and without global moves; t* is varying at fixed intervals be-
from bottom to top. tween 200 and 2000, from bottom to top. The target selection phase
is characterized by a complete collapse into a single configuration,
indicates that at time t* the configuration is still rapidly while damped classical oscillations around the equilibrium position
evolving, and that at every time step a large number of spins are allowed if global moves are introduced.
is being flipped; when the local stability is reached, on the
other hand, K共t* , ␶兲 assumes a flat 共or periodic兲 profile, indi- downhill directions falls to zero when the lowest energies are
cating that the system has entered into some attracting con- approached, indicating that the CA dynamics ends in a local
figuration 共or limit cycle兲. minimum. The number of remaining flat directions is com-
Looking at Fig. 3, for CA, the self-overlap between patible with the observed amplitude of the oscillations shown
兵Si共t*兲其 and 兵Si共t* − ␶兲其 grows with t*. With increasing t*, a in Fig. 3 for large t*.
periodic behavior of K共t* , ␶兲 begins to appear over initially Moving now to the analysis of QA, it must be remarked
short but gradually increasing decay times ␶. In the final part that during most of the simulation time the self-overlap
of the classical relaxation, about 20% of the variables are among configurations at different times is considerably
still allowed to flip at each iteration, even if the average smaller than in the case of CA, and the spin-flip acceptance
energy is no longer changing. The strongly regular oscilla- ratio is larger. Both these properties hint at a regime charac-
tions of K共t* , ␶兲, as well as the nonvanishing asymptotic spin terized by a rapid and disordered evolution of all the replica
flip acceptance ratio, suggests that the system gets trapped spin-configurations. The self-overlap increase becomes faster
into a very small portion of the phase space, and that a frac- upon reducing the transverse magnetic field because the
tion of the variables is still allowed to fluctuate, but only pseudo-evolutionary replication of the “good” spin patterns,
cyclically repeating a limited amount of sequences of flips. operated by the coupling 共6兲, has also a stabilizing effect on
It is possible to characterize the states that are actually them. If only local moves are performed 共solid curves in Fig.
blocking the system, by looking at the neighborhood geom- 4兲, no trace of asymptotic periodic behavior is found, and all
etry during the dynamics. When sitting in a configuration at the replica configurations reach continuously a full overlap
a given energy E, one studies the energy change ⌬E upon with a single final configuration. When global moves are
flipping all the N spins: ⌬E can be positive 共uphill direction兲, switched on, at large t*, after reaching a plateau of approxi-
vanishing 共flat direction兲, or negative 共downhill direction兲. mately 0.93, the target selection phase starts, but now the
The data relative to the configurations sampled by a large self-overlap reduces slightly and damped classical-like oscil-
number of linear CA runs 共each of 2000 annealing iterations, lation set in. Looking at Fig. 5, it is possible to see that the
and always over the same N = 10 000 sample兲 are shown in states reached at the end of the QA schedule are very close to
Fig. 5, where the fractions of downhill, flat, and uphill direc- local minima, with an extremely small number of downhill
tions are plotted 共from top to bottom兲 against the energy of directions. Once again, we assist to a trapping phenomenon,
the visited configurations. One sees that the number of and the global moves can only produce a local optimization

066707-6
OPTIMIZATION BY QUANTUM ANNEALING: LESSONS… PHYSICAL REVIEW E 71, 066707 共2005兲

Nevertheless, the analysis of the local geometry, shown in

Fig. 5, highlights important differences between the CA and
the QA dynamics. The phase space region explored by the
two algorithms are quite different. At the same value of the
energy, the quantum system is visiting configurations with a
significantly larger fraction of downhill and flat directions. If
an abuse of language is tolerated, one could say that the CA
follows narrow canyons, where the number of directions
bringing to a decrease in energy is limited, while the QA
prefers to explore the edges of mid-altitude plateaus. This
phenomenon, which seems to be a genuinely quantum fea-
ture captured by the PIMC simulation, is strongly reminis-
cent of what happens in continuous space, where the choice
of broader potential wells allows the system to reduce the
kinetic contribution to the total energy 共curvature-induced
effects are also well known in the theory of instanton tunnel-
ing 关43兴兲. Quite interestingly, the typical number of spins that
are different among the various Trotter replicas is of the or-
der of the number of flat directions. This means that all the
configurations simultaneously taken by the quantum system
belong to a single broad landscape valley, which is explored
in all its wideness by the quantum system. When the trans-
verse field intensity is vanishing, the target selection transi-
tion toward a classical regime takes place, and the number of
uphill directions increases abruptly, indicating that the dy-
namical collapse is paralleled by a change in the local land-
scape topology. The poor performance of QA in the present
3-SAT case could then be explained by the existence of
broad basins of attraction strewn with deceptive and highly
attractive sinks, that, unlike the cleavages preferred from the
very beginning by the CA, prevent access to lower energy
sectors. To conclude, we remark that genetic-like search is
also known to be strongly perturbed by the presence of mul-
tiple and scattered wells 关44兴.

VII. FIELD-CYCLING STRATEGIES

After the final target selection, a new quantum search

phase can be started by switching on again the field ⌫ and
restoring then a regime dominated by quantum fluctuations
共in a spirit similar to the techniques employed in 关45兴兲.
In a first possible experimental setup 共results not shown兲,
after a linear descending ramp from ⌫0 ⯝ 0.7 to ⌫ f ⯝ 0.001,
the field is raised smoothly to the initial value ⌫0 and then
back again to ⌫ f , while keeping the effective quantum tem-
perature Tq = 0.3 constant. Many such field cycles can be
chained one after the other. It turns out that the linear-
FIG. 5. The local geometry of the visited regions of the phase
schedule QA dynamics is perfectly time reversible, and that
space is probed by counting the fraction of directions in which the
all the ascending and descending ramps produce almost ex-
energy variation is negative, null, or positive. Although both CA
actly the same time evolutions for the energy 共this phenom-
and QA get trapped in a local minimum, the quantum evolution
tends to visit “valleys” that, at the same energies as CA, are more
enon is somehow analogous to the well-known memory and
flat and have a larger number of downhill directions. rejuvenation effects in glassy out-of-equilibrium dynamics
关17兴兲.
around the final configuration reached when just local moves Even if the restoration of quantum fluctuations allows the
are used. Even if the diverging transverse coupling 共and the system to escape from the target local minimum, a further
consequent vanishing of the local acceptance兲 modifies in a ingredient is needed in order to achieve better results. A pos-
noticeable way the behavior of the autocorrelation function, sible expedient to avoid a complete reinitialization is to
the blocking mechanism acting in the target selection phase slightly reduce the temperature T before each new field
of QA is then perfectly analogous to that of CA. ramp; the temperature is still kept constant during each indi-

066707-7
BATTAGLIA, SANTORO, AND TOSATTI PHYSICAL REVIEW E 71, 066707 共2005兲

temperature Tq, finally, is reduced from 0.3 to 0.05 as shown

in the upper panel of Fig. 6. Unlike the case of constant
temperature field cycling, after each ascending ramp and a
short transient phase, we now reach new plateaus, lying at a
distance progressively larger from the energy level that
would be obtained by performing a classical quenching at the
same temperature 共see the arrows in Fig. 6兲. Lower-lying
target states are then selected after each descending ramp.
Over short time scales 共number of MC iterations approxi-
mately smaller than 200 000, when taking P = 50兲, such a
hybrid field-cycling strategy performs definitely better than a
purely classical one. The same experiment has been repeated
in the absence of the transverse magnetic field, and with the
same number of 共now completely decoupled兲 Trotter repli-
cas. The average over the runs and over the Trotter replicas
具具E典典CA, and the average over the runs of the best replica
energy 具E典CA, have been computed. It is possible to see from
Fig. 6 that 具具E典典hybrid obtained with the hybrid strategy lies
clearly below 具E典CA, indicating that quantum effects give ac-
cess to states that can hardly be reached in the same time
even by rare large classical fluctuations. Quantum restarts are
then more effective than classical restarts 关41兴, at least when
short schedules are taken in account.
The analysis of purely classical cycling experiments con-
firms the importance of quantum fluctuations. The fact that
具E典CA ⬎ 具具E典典hybrid shows already that the superior perfor-
mance is not simply due to the gradual reduction of the ini-
tial temperature among the different ramps. Furthermore, the
concatenation of short classical annealings, each new one
starting from a lower T0 and going down to T f = 0, provides
a worse final energy than a single classical annealing
of the same length. Indeed, even if the ith descending clas-
sical annealing ramp 共starting from T共i兲0 兲 produces a configu-
FIG. 6. The energy evolution during a field-cycling hybrid strat- ration with an energy lower than in the random case, the
egy. The strength of the transverse coupling J⌫ is varied cyclically
ith fast ascending ramp to the temperature T共i+1兲
0 ⬍ T共i兲
0 of the
between the values 0.001 and 5, by adjusting the value of the mag-
共i + 1兲-th descending ramp completely erases now any posi-
netic field. The effective temperature Tq is kept constant during
each field ramp, but is reduced in a stepwise way among different
tive effect. The system is indeed pushed at the energy level
ramps, from the initial value of 0.3 down to 0.05. Each ascending typical of a fast quenching at the temperature T共i+1兲
0 and, un-
field-ramp unfreezes the system from a previously reached target like the quantum case, the memory of the previously visited
state and, after a short transient regime, a new search phase is low-energy configurations is completely lost. The presence
entered. The starting plateaus have energy values increasingly of the kinetic energy term is therefore crucial for the survival
smaller than the quenching level at the new simulation temperature of “seed patterns,” peculiar of the target configurations ob-
共the arrows in the graphs indicate the quenching level and the hy- tained at the end of the ith descending ramp, from which new
brid strategy plateau at a given value of the temperature兲. Each new and better low-lying configurations can be grown during the
target state has a better energy than the preceding one, and the final following 共i + 1兲-th stage. In the genetic algorithm jargon,
average energy is better than the value reachable by large classical one could say that the ascending ramps renew the available
fluctuations. gene pool without destroying completely the highly fit genes
of the preceding-step generation.
vidual ramp, realizing thus a hybrid strategy 共a linear- The situation is, however, different for larger time scales
schedule CA, superposed with linear-schedule QA cycles 共MC iterations larger than 200 000 for P = 50兲. Longer field-
over a shorter time scale兲. The time dependence of the en- cycling schedules are obtained by simply rescaling with a
ergy along a successful field-cycling scheme with a total constant factor the duration of all the ramps in a shorter
length of 3100 MC iterations is presented in Fig. 6. The schedule. The asymptotic slope of the field-cycling cooling
共i兲
extrema 共⌫共i兲
0 , ⌫ f 兲 of the ith field ramp are selected in order curve in Fig. 1 becomes remarkably similar to the other QA
to have a smooth variation of J⌫, when the temperature cases. A local topology analysis analogous to the one of the
changes discontinuously among two different ramps. The previous section shows indeed that the valleys explored by
coupling is then varied regularly and cyclically between the the field-cycling strategy are flat and open as the ones found
values J⌫,0 ⯝ 0.001 共corresponding to ⌫0 ⯝ 0.7 when Tq = 0.3兲 in the simpler linear QA case. If the reduction of temperature
and J⌫,f ⯝ 5 共corresponding to ⌫0 ⯝ 0.001 when Tq = 0.3兲. The allows the system to explore the landscape at different length

066707-8
OPTIMIZATION BY QUANTUM ANNEALING: LESSONS… PHYSICAL REVIEW E 71, 066707 共2005兲

scales 关17兴 and to find then better target configurations, the selected by the quantum relaxation bring in the 3-SAT case
attractive power of the visited high-lying local minima con- toward “dead ends,” hiding dangerous sink traps.
tinues to be very strong, and lower energy regions remain Finally, we should stress that both PIMC-QA and CA are
fundamentally inaccessible. definitely worse than ad hoc local search algorithms, notably
Walksat 关18兴. One probably important feature that is missing
to both CA and PIMC-QA, but known to be important in
VIII. DISCUSSION AND CONCLUSIONS Walksat and similar local heuristics, is the so-called focusing.
We have conducted a detailed comparison of the perfor- That is the fact that the proposed random flips should be
mance of path-integral Monte Carlo 共PIMC兲 quantum an- restricted to spins that are exclusively involved in currently
nealing 共QA兲 strategies against a standard thermal simulated UNSAT clauses 共according to the rule of thumb “if it is not
annealing 共CA兲, on a very hard instance of a classical broken, don’t fix it”兲. On the contrary, standard Metropolis
NP-complete problem, 3-SAT. Contrary to the successes pre- sampling, as implemented in both PIMC-QA and CA, does
viously obtained by the same technique on Ising spin glasses not discriminate among the candidate spin flips, and only
关7兴 and on an instance of the traveling salesman problem 关9兴, considers the corresponding energy change ⌬E 关see Eq. 共3兲兴.
QA performs here definitely worse than CA: the ultimate This suggests that a possible better way of implementing QA
large simulation time behavior of QA shows a poorer slope is through a Green’s function Monte Carlo 共GFMC兲 algo-
against inverse annealing rate than CA 共see Fig. 1兲. rithm, whereby a suitable kinetic term and appropriate guid-
In the course of this instructive negative example of QA ing functions, via importance sampling, can in principle take
performance 共perhaps even more instructive because it is care of some form of focusing. Another important advantage
negative兲, we gained some experience and insight on the of GFMC over PIMC would be the fact that annealing could
peculiar dynamical relaxation process behind the PIMC-QA be performed strictly at T = 0, whereas the unavoidably finite
algorithm. In particular, we saw, from overlap autocorrela- temperature T clearly has an influence on the PIMC-QA dy-
tion analysis, that the quantum algorithm leads to “selec- namics 共see Fig. 6兲.
tion,” in the course of annealing, of a target configuration In conclusion, while showing that statements such as
over which all the different replicas eventually collapse, as “quantum is better” have not necessarily anything to do with
the transverse coupling J⌫ induced by the quantum term physical reality, at least for arbitrary choices of the problem
−⌫兺i␴xi grows to infinity. As a byproduct, we also realized landscape and of the associated elementary moves, our work
that restarting repeatedly quantum fluctuations using such still leaves many important questions open. One should
target configurations as intermediate steps leads to a hybrid strive for a reliable predictive theory that, taking as inputs the
strategy that definitely improves over the bare linear sched- appropriate features of the complex energy landscape for the
ule QA, with results comparable to CA for short simulation problem at hand, or for a class of problems, would be able to
times. Nevertheless, even with these considerable improve- anticipate if, how, and where “quantum is better.” Closely
ments of QA, the slope of the annealing curve seems to be connected with this is the ability in designing quantum ki-
essentially unaffected by refinements, and definitely worse netic Hamiltonian terms that implement ad hoc local moves
than that of CA. This apparent “intrinsic” nature of the worse particularly efficient in exploring a given geometry.
performance of PIMC-QA over CA suggests that the 3-SAT ACKNOWLEDGMENTS
landscape 共at least when single spin-flip moves are consid-
ered兲 is in some way more difficult for the quantum algo- We thank Michal Kolář, Mario Rasetti, Lorenzo Stella,
rithm than for the classical one, presumably due to the fact Osvaldo Zagordi, and Riccardo Zecchina for stimulating dis-
that the flat and open landscape sectors that are peculiarly cussions.

关1兴 M. Nielsen and I. L. Chuang, Quantum Computation and 094203 共2002兲.

Quantum Information 共Cambridge University Press, Cam- 关9兴 R. Martoňák, G. E. Santoro, and E. Tosatti, Phys. Rev. E 70,
bridge, 2000兲. 057701 共2004兲.
关2兴 E. Farhi and S. Gutmann, Phys. Rev. A 57, 2403 共1998兲. 关10兴 J. Brooke, D. Bitko, T. F. Rosenbaum, and G. Aeppli, Science
关3兴 E. Farhi et al., Science 292, 472 共2001兲. 284, 779 共1999兲.
关4兴 V. N. Smelyanskiy and U. V. Toussaint, quant-ph/0202155. 关11兴 S. Kirkpatrick, C. D. Gelatt, Jr., and M. P. Vecchi, Science
关5兴 A. B. Finnila et al., Chem. Phys. Lett. 219, 343 共1994兲. 220, 671 共1983兲.
关6兴 T. Kadowaki and H. Nishimori, Phys. Rev. E 58, 5355 共1998兲; 关12兴 W. Van Dam, M. Mosca, U. Vazirani, in Proceedings of the
T. Kadowaki, “Study of Optimization Problems by Quantum 42nd Annual Symposium on Foundations of Computer Science
Annealing,” Ph.D. thesis, Department of Physics, Tokyo Insti- 共2001兲, p. 279.
tute of Technology, 1998. 关13兴 D. R. Mitchell, C. Adami, W. Lue, and C. P. Williams, quant-
关7兴 G. Santoro, R. Martoňák, E. Tosatti, and R. Car, Science 295, ph/0409088.
2427 共2002兲. 关14兴 L. Stella, G. E. Santoro, and E. Tosatti, cond-mat/0502129.
关8兴 R. Martoňák, G. Santoro, and E. Tosatti, Phys. Rev. B 66, 关15兴 M. R. Garey and D. S. Johnson, Computers and Intractability

066707-9
BATTAGLIA, SANTORO, AND TOSATTI PHYSICAL REVIEW E 71, 066707 共2005兲

共Freeman, New York, 1979兲. Cohn 共John Wiley & Sons, New York, 1994兲, p. 105.
关16兴 J.-P. Bouchaud, L. F. Cugliandolo, J. Kurchan, and M. Mézard, 关31兴 D. Mitchell, B. Selman, and H. Levesque, Proc. of the Tenth
in Spin Glasses and Random Fields, edited by A. P. Young National Conference on Artificial Intelligence 共AAAI-92兲
共World Scientific, Singapore, 1997兲. 共AAAI, San Jose, CA, 1992兲, p. 459.
关17兴 J. P. Bouchaud, V. Dupuis, J. Hammann, and E. Vincent, Phys. 关32兴 E. Aarts and J. Korst, Simulated Annealing and Boltzmann
Rev. B 65, 024439 共2001兲.
Machines 共Wiley & Sons, Chichester, 1989兲.
关18兴 B. Selman, H. Kautz, and B. Cohen, Proc. of the Twelfth Na-
关33兴 R. H. Otten and L. P. van Ginneken, The Annealing Algorithm
tional Conference on Artificial Intelligence 共AAAI-94兲 共AAAI,
共Kluwer Academic, Boston, 1989兲.
Seattle, 1994兲, p. 337.
关34兴 M. Suzuki, Prog. Theor. Phys. 56, 1454 共1976兲.
关19兴 D. Battaglia, M. Kolář, and R. Zecchina, Phys. Rev. E 70,
036107 共2004兲. 关35兴 B. K. Chakrabarti, A. Dutta, and P. Sen, Quantum Ising Phases
关20兴 A. Montanari, G. Parisi, and F. Ricci-Tersenghi, J. Phys. A 37, and Transitions in Transverse Ising Models, Lecture Notes in
2073 共2004兲. Physics 共Springer, Berlin, 1996兲.
关21兴 S. A. Cook, Proc. of the Third Annual ACM Symposium on the 关36兴 Y. Okabe and M. Kikuchi, Phys. Rev. B 34, 7896 共1986兲.
Theory of Computing 共ACM, New York, 1971兲, p. 151. 关37兴 N. Trivedi and D. M. Ceperley, Phys. Rev. B 40, R2737
关22兴 H. Kautz, D. McAllester, and B. Selman, Proceedings KR-96 共1989兲.
共1996兲, p. 374. 关38兴 A. G. Percus and O. C. Martin, J. Stat. Phys. 94, 739 共1999兲.
关23兴 H. Kautz and B. Selman, Proc. of the Thirteenth National Con- 关39兴 J. R. Koza, Genetic Programming: On the Programming of
ference on Artificial Intelligence 共AAAI-96兲, AAAI-Press Computers by Means of Natural Selection 共MIT Press, Cam-
共1996兲, p. 1194. bridge, MA, 1992兲.
关24兴 T. Larrabee, IEEE Trans. Comput.-Aided Des. 11, 6 共1992兲. 关40兴 J. H. Holland, Proc. Int. Un. Physiol. Soc. 3, 330 共1962兲.
关25兴 A. P. Kamath, N. K. Karmarkar, K. G. Ramakrishnan, and M. 关41兴 A. Montanari and R. Zecchina, Phys. Rev. Lett. 88, 178701
G. C. Resende, Math. Program. 57, 215 共1992兲. 共2002兲.
关26兴 Special issue on NP-hardness and Phase transitions, edited by 关42兴 J. H. Holland, “Processing and processors for schemata,” in
O. Dubois, R. Monasson, B. Selman, and R. Zecchina, Theor. Associative Information Processing 共American Elsevier, New
Comput. Sci. 265共1–2兲 共2001兲. York, 1971兲.
关27兴 M. Mezard, G. Parisi, and R. Zecchina, Science 297, 812 关43兴 H. Kleinert, Path Integrals in Quantum Mechanics, Statistics
共2002兲. and Polymer Physics, 2nd ed. 共World Scientific, Singapore,
关28兴 M. Mezard and R. Zecchina, Phys. Rev. E 66, 56126 共2002兲. 1995兲.
关29兴 G. Parisi, M. Mézard, and M. Virasoro, Spin Glass Theory and 关44兴 K. A. De Jong, IEEE Trans. Syst. Man Cybern. 10, 566
Beyond 共World Scientific, Singapore, 1987兲. 共1980兲.
关30兴 I. Gent and T. Walsh, Proceedings of ECAI-94, ed. by A. G. 关45兴 Y. Lee and B. J. Berne, J. Phys. Chem. A 104, 86 共2000兲.

066707-10