M. S.

Shell 2009 1/10 last modified 5/31/2012

Advanced sampling ChE210D
Today's lecture: methods for facilitating equilibration and sampling in comple!
frustrated! or slo"#e$ol$ing s%stems
Difficult-to-simulate systems
&racticall% spea'ing! one is al"a%s limited in the length and time scales accessible from simula#
tion. (he former relates to the maimum s%stem si)e that can be used *in terms of number of
atoms+! and the latter to the length of the run that can be performed *in terms of number of
M, integration or M- steps+.
.f the intrinsic! ph%sical time scale for some s%stem of interest is be%ond that accessible from
simulation! "e cannot in$estigate it using straightfor"ard d%namical methods li'e molecular
d%namics because it "ill be challenging to equilibrate the s%stem. /o"e$er! if "e are onl%
interested in thermod%namic and not 'inetic properties! then "e can construct artificial d%nam#
ics in the s%stem that accelerate con$ergence to"ards equilibrium. Such methods ine$itabl%
in$ol$e a Monte -arlo component.
0hat ma'es a s%stem difficult to sample in an equilibrium sense1 (%picall%! these s%stems
ha$e frustrated or rugged energy landscapes. (his means that the underl%ing potential energ%
surface contains man% deep minima separated b% high energ% barriers. 2t lo" temperatures! it
is challenging for the s%stem to mo$e bet"een the rele$ant lo"#energ% regions because it must
surmount these barriers3a rare e$ent process that ta'es time.
2 "ide range of methods ha$e been designed to accelerate equilibration in simulations of
s%stems "ith rugged energ% landscapes. /ere! "e "ill onl% re$ie" a small selection of these.
Some 'inds of s%stems that might fall into this categor% include4
s%stems "ith strong electrostatic interactions
fluids of strongl% orientation#dependent interactions *e.g.! dipoles! h%drogen bonds+
macromolecular and pol%meric s%stems
biomolecular s%stems *proteins! lipids+
self#organi)ing or self#structuring s%stems *micelles! bila%ers+
supercooled liquid and amorphous s%stems at $er% lo" temperature

M. S. Shell 2009 2/10 last modified 5/31/2012
Simulated tempering
(he simulated tempering algorithm is equi$alent to a canonical Monte -arlo simulation in
"hich the temperature changes randoml%. (hat is! there are t"o 'inds of mo$es4
energy fluctuation moves 5 single#particle displacements! orientational displacements!
etc. (he usual canonical acceptance criteria! at the current temperature! are used.
temperature fluctuation moves 5 the temperature is periodicall% increased or de#
creased b% fied amounts so that the temperature can acquire one of a fied set of $al#
ues I
min
, I
min
+I, I
min
+ 2I, , I
max
.
(hese temperature fluctuations help the s%stem equilibrate b% allo"ing it to fluctuate to higher
temperatures "here it can more readil% cross energ% barriers.
.n these simulations! the microstate is no" a function of both the configurations and the
temperature4
(r
N
, I) c
-[0(r
N
)

0hat is the acceptance criterion for a temperature change1 0e "ant all temperatures to
appear "ith equal probabilit%. (hat is! if "e measured the distribution of temperature o$er the
simulation run! "e "ould "ant it to be roughl% uniform. (o do this! "e introduce a "eighting
function in temperature4
(r
N
, I) c
-[0(r
N
)+q(1)

(his means that the acceptance criterion for a random temperature perturbation I
2
= I
1
+ I
is gi$en b%
P
12
acc
= min|1, c
-([
2
-[
1
)0+q(1
2
)-q(1
1
)
]
= min|1, c
-0A[+Aq
]
(he distribution of temperatures is found b% integrating this 6oint probabilit% o$er configura#
tions4
(I) _(r
N
, I)Jr
N

= Z(I, I, N)c
q(1)

= c
q(1)-[A(1)

(o achie$e a uniform distribution in temperature! "e demand that this epression is constant.
(his means that "e should choose4

M. S. Shell 2009 3/10 last modified 5/31/2012
p(I) = [A(I)
0e do not 'no" the free energies at the different temperatures a priori. .nstead! "e must
iterati$el% determine them o$er the course of a simulation. Man% methods can be used to do
this! including4
flat histogram methods 5the flat histogram coordinate is the temperature
multiple histogram-reweighting 5 "e maintain a histogram of energies at each temper#
ature and periodicall% use the re"eighting equations to determine the relati$e free en#
ergies for each temperature
free energy perturbation 5 "e use! for eample! 7ennett8s method bet"een ad6acent
temperatures to compute free energ% differences at periodic inter$als in the simulation
/o" do "e compute properties at a temperature of interest from this approach1 0e can use
the 0/2M#based re"eighting equation applied to all temperatures and all tra6ectories. 0e
first compute a configurational "eight according to
w

=
c
-[0
i
_ c
[
]
A
]
-[
]
0
i
]
]=1

/ere! i is an inde o$er simulation obser$ations. (he in$erse temperature [ corresponds to an
arbitrary re"eighting temperature I. 9ne should choose I
min
I I
max
for good statistical
accurac%.
:or an arbitrar% obser$able X! "e can then compute a$erages and distributions using
(X) =
_ w

X

_ w


(X) w

o
X
i
=X

Replica exchange
(he simulated tempering approach is useful for facilitating equilibrium because it allo"s the
s%stem to eplore multiple temperatures. .t has t"o dra"bac's! ho"e$er4
0e need to compute the free energies A(I
]
) at e$er% temperature in order to properl%
sample all temperatures.

M. S. Shell 2009 ;/10 last modified 5/31/2012
0e need to "ait for the s%stem to tra$erse the entire temperature range man% times in
order to accumulate good statistics at each. (his can be a $er% long time if "e ha$e
man% temperatures.
2n alternati$e approach! "hich has become the method of choice for sampling challenging
s%stems is the replica exchange method. .t o$ercomes these limitations b% ha$ing the follo"ing
general construction4
[ simulations *replicas+ of the same s%stem are performed simultaneousl% at different
temperatures I
]
.
<ach simulation is e$ol$ed independentl%! either through M, or M- methods! at the
corresponding temperature.
2t set inter$als! replica swap mo$es are performed bet"een ad6acent temperature rep#
licas. .n a s"ap mo$e! the instantaneous configurations are echanged bet"een the
t"o temperatures.
Swap move acceptance criterion
(he replica echange simulation performs a Mar'o$ chain in the entire [#s%stem ensemble. 0e
need to determine the acceptance criterion for performing s"ap mo$es. (o do that! "e need
to determine the total probabilit% of one microstate in the entire ensemble. /ere! a microstate
is the list of all of the positions in each of the replicas! R = (r
1
N
, r
2
N
, , r
]
N
). Since the replicas
do not interact *there are no energ% terms bet"een atoms in different replicas+! "e can "rite
(R) = _
]
(r
]
N
)
]

=sing canonical probabilities in each replica!
(R) = _
c
-[
]
0[r
]
N

Z
]
]

-onsider a s"ap mo$e bet"een t"o temperatures 1 and 2. .nitiall% the configuration in tem#
perature 1 is r
1
N
and in temperature 2 is r
2
N
. (he acceptance criterion for the mo$e stems from
the detailed balance equation!
P
12,swap
acc
= min _1,
(R
2
)
(R
1
)
_

M. S. Shell 2009 5/10 last modified 5/31/2012
/ere! R
2
corresponds to the set of configurations in "hich r
1
N
and r
2
N
are transposed to differ#
ent temperatures. 0e ha$e4
(R
1
) =
c
-[
1
0(r
1
N
)
Z
1

c
-[
2
0(r
2
N
)
Z
2
_
c
-[
]
0[r
]
N

Z
]
]
]=3

(R
2
) =
c
-[
1
0(r
2
N
)
Z
1

c
-[
2
0(r
1
N
)
Z
2
_
c
-[
]
0[r
]
N

Z
]
]
]=3

&lugging these probabilities into the acceptance criterion!
P
12,swap
acc
= min j1, c
[
1
0(r
1
N
)+[
2
0(r
2
N
)-[
1
0(r
2
N
)-[
2
0(r
1
N
)
[
>otice that the partition functions in the denominator cancel. (his means that "e do not need
to 'no" the free energies in each temperature "hen e$aluating the acceptance criterion. .t is
because "e s"ap s%stems at t"o temperatures! rather than perturb a single s%stem8s tempera#
ture! that "e no longer need to 'no" the free energ% *as compared to simulated tempering+.
Simplif%ing the abo$e epression!
P
12,s"ap
acc
= min|1, c
A[A0
] "here [ = [
2
- [
1
, u = u(r
2
N
) - u(r
1
N
)
>otice that "e are required to compute the potential energ% difference bet"een the instanta#
neous configurations in the t"o temperature replicas.
S"ap mo$es are performed li'e an% other M- mo$es4 the mo$e is proposed! the acceptance
criterion is computed! a random number is dra"n! and it is decided "hether or not to perform
the mo$e. =nli'e man% other M- mo$es! s"ap mo$es are $er% inepensi$e to perform4 the%
onl% require the current energies in each temperature! "hich is t%picall% maintained through#
out the simulation an%"a%s.
Considerations for acceptance ratios and the temperature schedule
-onsider a s"ap bet"een t"o temperatures I
2
> I
1
. 0e epect that the configuration dra"n
from the higher temperature "ill ha$e a higher energ%. (hat is! u(r
2
N
) > u(r
1
N
). (hus the
quantit% [u "ill most often be negati$e! "hich "ill ma'e the acceptance probabilit% small.
:or $er% large temperature differences! this quantit% becomes e$en more negati$e. (he result
is that the temperatures must be spaced close enough together in order to achie$e a good rate
of accepted s"aps.

M. S. Shell 2009 ?/10 last modified 5/31/2012
0e can thin' about this problem in terms of the distributions
1
(u),
2
(u) bet"een neigh#
boring temperatures. .f there is a substantial o$erlap! "e "ill ha$e a high frequenc% of s"ap
mo$e acceptance4

(hese considerations pla% into the "a% in "hich "e pic' the temperatures in our replica e#
change simulation14
I
min
# t%picall% "e pic' a minimum temperature to be the temperature of interest that is
difficult to simulate
I
max
# one chooses a high temperature "here free energ% barriers can be crossed! but
not so high as to require man% intermediate temperatures
[ # the number of temperatures is usuall% chosen so as to achie$e @50A acceptance of
s"ap mo$es bet"een ad6acent replicas
I
]
# for a s%stem "ith a constant heat capacit%! it can be sho"n that a constant rate of
acceptance bet"een ad6acent temperatures corresponds to an eponential distribution
in temperature! equi$alentl%! a po"er la" in replica number. (hus! one normall% pic's
I
]
= I
min
_
I
max
I
min
]
]
]-1

Scaling with system size
2s the s%stem si)e gro"s! the distribution in energ% at a gi$en temperature becomes increasing#
l% narro" "ith respect to the a$erage energ%! as 1VN. 0ithout o$erlap bet"een the energ%
fluctuations of ad6acent temperatures in the replica echange scheme! fe" s"ap mo$es "ill be
accepted. (hus! as the s%stem si)e increases! more and more intermediate temperature
replicas are needed to achie$e a 50A acceptance ratio. (his ma'es the method challenging to
appl% to $er% large s%stems.
(u)
u
I
2
> I
1

I
1

(u)
I
2
> I
1

I
1

s"aps frequentl% accepted s"aps infrequentl% accepted
u

M. S. Shell 2009 B/10 last modified 5/31/2012
Evaluating properties and distributions
Cigorousl%! all of the configurations at each temperature *regardless of "hether the% s"apped
in from other temperatures+ con$erge to a canonical distribution at that temperature. (hus! "e
could compute the a$erages of properties at each temperature I
]
b% simpl% a$eraging o$er the
configurations at that temperature. 2 trajectory is therefore considered the e$olution of
configurations at a given temperature.
/o"e$er! "e can also compute an a$erage at an% arbitrary temperature bet"een the minimum
and maimum in our simulation b% using a re"eighting approach. 0hen a replica echange
simulation is performed! "e can collect histograms or histories of each potential energ% $isited
in each temperature! u
]
. 0e can then use the re"eighting equations to compute the free
energies and configurational "eights at each temperature. 0ith the computed configurational
"eights! "e can epress the a$erage of an% propert% as a "eighted sum o$er all temperature
tra6ectories4
w
]
=
c
-[0
i]
_ c
[
l
A
l
-[
l
0
i]
]
I=1

(X) =
_ _ w
]
X
]
n
=1
]
]=1
_ _ w
]
n
=1
]
]=1

(X) w
]
o
X
i]
=X
n
=1
]
]=1

mplementation
(o maintain the different replicas! one t%picall% uses a parallel computing scheme! in "hich one
computer node *or processor+ is assigned to each replica. 0hen s"aps are performed! all
nodes stop e$ol$ing the s%stem in time and a master or head node sorts through and ma'es the
s"ap mo$es. (his in$ol$es parallel communication bet"een the different nodes and the head
node.
2t each inter$al "here s"aps are performed! an% number of s"aps could be attempted. (%pi#
call%! one chooses the number of s"ap attempts to be the number of temperature replicas.
(he s"aps can be attempted in serial order! from lo"est/highest to highest/lo"est tempera#
ture! or in random order.
.n a M- simulation! the e$olution of the s%stem in bet"een s"aps can be go$erned b% an%
number of M- mo$es that accomplish changes in the potential energ%! such as single particle
displacements.

M. S. Shell 2009 D/10 last modified 5/31/2012
.n a M, simulation! the e$olution in bet"een s"aps can be performed using short M, tra6ecto#
ries. (hese mo$es can be considered h%brid M-/M, mo$es "ith good energ% conser$ation
such that the% are al"a%s accepted. 2fter a round of s"ap mo$es! one pic's random $elocities
for each atom at e$er% temperature ] from a 7olt)mann distribution at the corresponding I
]
.
(hese $elocities are used to start the short M, tra6ector% before another round of s"ap mo$es
is performed.
2n alternati$e approach in M, "ould be to rescale the velocities in the configurations to the
ne" temperature after a s"ap mo$e. (his approach "as first deri$ed b% Sugita and 9'amoto
E1999F. /ere! random ne" $elocities are not pic'ed at an% time. .nstead! after s"apping
configurations! the momenta are scaled b% a factor I
new
I
olu
/ ! "here I
new
is the ne" temper#
ature into "hich the configuration "as s"apped and I
olu
is the temperature from "hich it
came. (his approach requires a thermostat to be used during the M, tra6ectories. 0hile the
$elocit% rescaling approach is more frequentl% used in the literature than the h%brid approach
*"ith random resampling+! it is not necessaril% more efficient and either approach is $alid.
!ariants
/ere! "e considered a replica echange simulation "here the replicas differed in temperature.
(his is often also called parallel tempering for its similarit% to simulated tempering. /o"e$er!
"e are not limited to temperature. 9ne can ha$e replicas differ in chemical potential or
pressure if the indi$idual simulations are G-M- or NPI simualtions! respecti$el%. Two-
dimensional replica exchange methods allo" replicas to differ in both temperature and chemi#
cal potential or pressure.
2nother "a% to facilitate sampling is to modif% the potential energ% function itself. /ere! one
"ants to perturb the energ% function so that the underl%ing energ% landscape is smoother and
easier to sample. :or eample! one s%stematicall% scale the partial charges from )ero to their
full $alues as one mo$es from replica to replica. (his approach is often called Hamiltonian
exchange (he appropriate form of the acceptance criterion is deri$ed as before! but "ith
different potential energ% functions in each simulation u
]
4
P
12,s"ap
acc
= minj1, c
[
1
0
1
(r
1
N
)+[
2
0
2
(r
2
N
)-[
1
0
1
(r
2
N
)-[
2
0
2
(r
1
N
)
[
.n e$aluating this acceptance criterion! one must e$aluate the cross#energies u
2
(r
1
N
) and
u
1
(r
2
N
) bet"een t"o replicas "ith each s"ap mo$e! since the energ% function is no longer
constant bet"een them.

M. S. Shell 2009 9/10 last modified 5/31/2012
.n all of these cases! "e can use multiple histogram re"eighting techniques to epress the
a$erage or distribution of an% propert% as a "eighted a$erage o$er all temperatures ] and
tra6ector% configurations i.
Extended ensemble molecular dynamics
9ne "a% to enhance the eploration of phase space in M, at lo"er temperatures is bias the
ensemble probabilities to artificiall% broad fluctuations in potential energ%. (his enables the
s%stem to hop o$er higher#energ% barriers more frequentl%. =ltimatel%! the beha$ior of the
s%stem under normal conditions can be reco$ered b% standard re"eighting procedures. 0e
ha$e alread% discussed this approach in the contet of M- simulations. >o"! "e consider an
M, implementation.
0e add to our M, simulation a "eighting function that biases the configurational probabilities!
in the same "a% that "e did during our discussion of biased sampling4
u
w
(r
N
) = u(r
N
) - k
B
Ip(u(r
N
))
/ere! p(u) is a "eighting function "ith dimensionless units! and it onl% depends on the poten#
tial energ%. (he equations of motion are deri$ed as before. :or an atom i4
I

w
= -
Ju
w
Jr

= -
Ju
Jr

+ k
B
I
Jp(u)
Ju
Ju
Jr

= I

_1 - k
B
I
Jp(u)
Ju
]
.n other "ords! the force on each atom in the "eighted ensemble is scaled b% a term in$ol$ing
the deri$ati$e of the "eighting function evaluated at the energ% of the current configuration. .f
the "eighting function is a constant! no scaling occurs.
.n order for this approach to "or'! the "eighting function must be a continuous function. 9ne
t%picall% uses splines or other mathematical constructs so that an anal%tical deri$ati$e can be
computed.
/o" do "e determine the "eighting function1 0e can fit it to estimates of the probabilit%
distribution function. .n the canonical ensemble! "e ha$e the relationship
(u)

w
(u)
c
-q(0)

.f "e "ant the "eighted distribution to be approimatel% flat! "e can choose

M. S. Shell 2009 10/10 last modified 5/31/2012
p(u) = -ln (u) + const
>otice that "e "ill ha$e to fit the "eighting function to discrete histogram calculation! since p
must be continuous. .t is for this reason that it becomes difficult to e$aluate p to high statistical
accurac% using M, techniques. Still! broadening the energ% distribution e$en a little bit ma%
speed equilibration in M, simulations.