Stochastic Thermodynamics

Proceedings of the International School of Physics “Enrico Fermi”
Course CLXXXIV “Physics of Complex Colloids”, edited by C. Bechinger, F. Sciortino and P. Ziherl
(IOS, Amsterdam; SIF, Bologna)
DOI 10.3254/978-1-61499-278-3-155
Stochastic thermodynamics: A brief introduction

C. Van den Broeck
Hasselt University, B-3590 Diepenbeek, Belgium
Summary. — The main purpose of statistical mechanics is to give a microscopic

derivation of macroscopic laws, including in particular the celebrated second law
of thermodynamics. In recent years, there have been spectacular developments in
this respect, including the integral and detailed work fluctuation theorems and the
theory of stochastic thermodynamics. Here we give a brief introduction to these
developments. In the first step, we derive the first and second law of thermodynam-
ics for a Markovian stochastic process at the ensemble level, including two major
advances: 1) the theory can be applied to small-scale systems including the effect
of fluctuations, 2) the theory is not restricted to near-equilibrium dynamics. As
an application, we evaluate the efficiency at maximum power of a two-state quan-
tum dot. We also briefly discuss the connection to information-to-work conversion
(Landauer principle) and the splitting of the second law into an adiabatic and non-
adiabatic component. In a second step we formulate stochastic thermodynamics at
the trajectory level, introducing stochastic trajectory-dependent quantities such as
stochastic entropy, energy, heat, and work. Both the first and the second law can
be formulated at this trajectory level. Concerning the second law, the crucial obser-
vation is that the stochastic entropy production can be written as the logarithm of
the ratio of path probabilities. This in turn implies a detailed and integral work and
fluctuation theorem, linking the probability to observe a given stochastic entropy
production to that of observing minus this entropy change in a reverse experiment.
The usual second law, stipulating the increase on average of the stochastic entropy
production, follows as a subsidiary consequence.

c Società Italiana di Fisica 155
156 C. Van den Broeck
1. – Preliminaries
.
1 1. Introduction. – The main purpose of statistical mechanics is to give a microscopic
derivation of macroscopic laws, including in particular the celebrated second law of ther-
modynamics. The apparent contradiction between the irreversible behavior, enshrined in
the second law, and the reversible microscopic equations of motions has been the object
of numerous and sometimes acrimonious debates. In these short notes, we will avoid this
delicate issue by starting with so-called mesoscopic equations of motion, more precisely
with a Markovian description of the system under consideration. It is well known that
such a description is irreversible from the start. We will however show that one can
surmount several other difficulties that have plagued a statistical mechanical interpreta-
tion of the second law, notably the application to small systems and to systems far from
equilibrium. The basic approach is inspired by the work of Onsager, in the sense that
we will incorporate basic properties imposed by the microscopic laws (reversibility, Liou-
ville theorem, detailed balance) in the irreversible realm, and that of Prigogine since we
derive a thermodynamic formalism valid outside of equilibrium. We shall do this at the
level of a Markovian description with a thermodynamic description including fluctuations
and far-from-equilibrium states. The most surprising development is the formulation of
thermodynamics at the level of single trajectories. It sheds a surprising new light on the
traditional version of the second law: the positive average entropy production observed
at the ensemble level is the result of the fact that at the trajectory level, realizations
with positive entropy production are exponentially more likely than the corresponding
realizations with negative entropy production.
.
1 2. Nutshell thermodynamics [1-3]. – We briefly review some basic elements of tra-
ditional thermodynamics, in order to compare them with the thermodynamic concepts
that we will build for Markov processes. The first observation is that macroscopic sys-
tems, when left alone (confined, but otherwise isolated), reach the so-called equilibrium
state, which is characterized by a limited set of macroscopic state (conserved) variables.
The most basic of these variables is the total (or internal) energy E. Other variables are
the volume V and possibly the number of particles N . Second, we note that the energy
of a system can be changed by an external perturbation. We distinguish contributions
identified as work W (which can be controlled at the macroscopic level), from heat contri-
bution Q (which correspond to uncontrollable energy exchanges via microscopic degrees
of freedom). We use the convention that work and heat are positive when given to the
system. Conservation of energy (first law) stipulates that
(1) ΔE = W + Q
or, for small changes
(2) dE = dW + dQ.
Stochastic thermodynamics: A brief introduction 157
Energy is a state variable, but work and heat are not: they depend on the actual way
the perturbation is applied to change the state of the system(1 ). In particular, Joule’s
famous experiment, establishing the caloric equivalent of heat, corresponds precisely to
the two extreme cases, in which the same transition of a system is achieved (e.g., a
specific amount of water being heated from a lower to a higher temperature), in one case
by only applying work (adiabatic experiment ΔE = W ), and in the other case by pure
heating (ΔE = Q).
Having identified the equilibrium state of a system, one can raise the question as to
the physical nature of system under consideration, and in particular about its interaction
with other systems or with external perturbations. These issues are addressed by the
introduction of a new state function and the formulation of the second law(2 ). The
first key assertion is the existence, for systems in a state of equilibrium, of another
state variable called the entropy S. A (macroscopic) system is thermodynamically fully
characterized by the way this entropy depends on its state, via the so-called fundamental
relation: S = S(E, V, N, . . .). The partial derivative ∂E S = 1/T can be identified as
the inverse (Kelvin) temperature, which is always positive. Hence one can invert the
fundamental relation to obtain the equivalent relation E = E(S, V, N, . . .). Of particular
interest are the so-called quasi-static changes, corresponding to the situation in which the
system changes slowly enough so that it is essentially at equilibrium. Since the relation
E = E(S, V, N, . . .) is valid all along such a transformation, one finds by differentiation
the so-called Gibbs relation
(3) dE = T dS − P dV + μdN.
The partial derivatives of E with respect to S, V and N are new state variables, namely
the temperature T , minus the pressure −P and the chemical potential μ. Comparing this
expression with the first law, one can identify a work contribution dW = −P dV + μdN
(sum of mechanical and chemical work) and a heat contribution dQ = T dS. Since heat
can be measured as the deficit of energy minus work, the entropy becomes measurable in
a quasi-static experiment (to formally complete the argument, one needs to argue that
temperature can be measured, apart from a multiplicative factor, via the efficiency of heat
engines). The second ingredient is the additivity of entropy: the entropy of a collection
of systems is the sum of separate entropies(3 ). The final and major ingredient stipulates
what happens when constraints are released and/or systems brought in contact with each
other. Standard thermodynamics does not describe the details of the ensuing evolution,
not surprisingly so since other variables, and possibly a fully microscopic description, are
(1 ) To stress this fact, one often uses the special notation d̄ for an infinitesimal change of such
a variable, but we will not do so for simplicity of notation.
(2 ) In some textbooks, one also introduces the zeroth law, stating that two systems at equilib-
rium with a third one, are also mutually at equilibrium with each other.
(3 ) One needs to be more careful when there is a strong interaction between the systems or
when for example surface effects are studied.
needed to describe what is happening. It nevertheless gives a general prescription for

the final equilibrium state that is eventually reached. The equilibrium state is the one
with maximum total entropy, compatible with the imposed constraints (e.g., volume,
energy or number of particles). In particular the total entropy change (including the
entropy changes of all systems that participate in the evolution) being the difference
between entropies of final and initial equilibrium states, can never be negative ΔStot ≥ 0.
This is the celebrated second law of thermodynamics, which has numerous applications
in physics (gases, osmotic pressure, blackbody radiation, phase transitions), chemistry
(chemical equilibrium) and biology (electro-chemical equilibrium, Nernst relation). It
implies that systems which are in thermal contact with each other must have the same
temperature (justifying the use of thermometers). In the same way, systems in direct
mechanical and chemical contact have the same pressure and same chemical potential(4 ).
Of particular interest are idealized systems such as heat baths. A heat bath is a ther-
modynamic system that is described by a single state variable, its energy. It furthermore
maintains its equilibrium state (more precisely, its relaxation time scale is fast enough
so that its changes can be considered to be quasi-static), even as it exchanges energy
under the form of heat with its surroundings. We refer to these systems as reservoirs and
denote their properties with subscript r(5 ). For a heat reservoir, one can thus write
(4) dSr = dEr /T = dQr /T.
For a heat and particle reservoir, the energy change has a heat and chemical work com-
ponent dEr = T dSr + μdNr = dQr + dWr,chem , and the entropy change is given by
(5) dSr = dQr /T = (dEr − μdNr )/T.
In the context of stochastic thermodynamics, we will often consider the case of several
baths, which we will then identify with a superscript (ν). Furthermore we will assume
that these reservoirs are infinitely large, so that their temperature T (ν) and chemical
potential μ(ν) remain unchanged even when finite amounts of heat and particles are
exchanged.
For a system (entropy S) in contact with a single heat bath (entropy Sr , temperature
T , chemical potential μ), the second law can now be written as follows (remember that
Q is the heat to the system, hence Qr = −Q is the heat to the reservoir):
(6) ΔStot = ΔS + ΔSr ≥ 0,

(7) ΔS ≥ −ΔSr = Q/T.
The equality sign is reached for a reversible process.
(4 ) The relation between pressures and chemical potentials can however be more complicated
in the presence of specific constraints such as levers or chemical reactions.
(5 ) Note that, in this special case, the heat is equal to the internal energy (apart from a
constant), hence it is a state variable.
The original derivation of the second law was based on the reasonable assumption
that there exists no perpetuum mobile of the second kind. Alternative formulations
state that heat does not spontaneously flow from a cold to a hot reservoir (Clausius
statement), or that heat from a single reservoir cannot, as sole effect, be transformed
into work (Kelvin statement). On the basis of such an assumption, Carnot was able
to obtain a universal bound for the efficiency of thermal machines, which in turn was
used by Clausius to prove the existence of S as a state function. For further refer-
ence, we show in the reverse how the second law limits the efficiency of a thermal
machine. Consider a cycle in which an auxiliary system (the thermal engine) takes
an amount of heat Qh for a hot reservoir at temperature Th , dropping an amount of
heat −Qc in a cold reservoir at temperature Tc , while returning to its original state
at the end of the cycle (hence its entropy also returns to its original value(6 )). The
remainder of the energy is turned into work W = −Qh − Qc . Positivity of the total
entropy produced in the cycle ΔStot = −Qh /Th − Qc /Tc ≥ 0 implies that the effi-
ciency of the heat to work conversion is bounded by the so-called Carnot efficiency:
η = −W/Qh ≤ ηc = 1 − Tc /Th . Note that thermodynamics does not prescribe how
the above conversion is achieved. Carnot provided an explicit construction, but by now
numerous alternatives are available. It is worth noting that several of the more recent
examples are operating with small-scale systems, such that fluctuations or even quantum
effects cannot be ignored.
The application of thermodynamics to quasi-static states was extended to spatially
distributed systems invoking the so-called local equilibrium assumption (which is basi-
cally the quasi-static assumption, not only in time, but also in space) by Prigogine. It
turns out that both the notation and interpretation of the second law that he formulated
on this occasion will be most useful in stochastic thermodynamics. Prigogine introduced
the concepts of entropy flow and entropy production. Entropy flow is the contribution to
the entropy change due to the (reversible) exchange with the environment. Entropy pro-
duction is an additional entropy increase due to irreversible processes inside the system.
In the above example, we identify the entropy flow to the system as minus the entropy
change of the reservoir: Δe S = −ΔSr (7 ). Furthermore, since the reservoir is assumed
to operate quasi-statically, it does not have an internal entropy production. The only
irreversible process is taking place in the system and we call it the entropy production
(6 ) In principle the auxiliary system need not be at equilibrium during this transformation,
hence its classical thermodynamic entropy is not defined. One can however argue that, in a
steady-state operation of a thermal engine, the system returns to its initial state (equilibrium
or non-equilibrium) after each cycle and does therefore not contribute to the overall entropy
production. As we will see, this statement becomes precise in stochastic thermodynamics: the
entropy is a well defined state function, even when the system is out of equilibrium, so its entropy
returns to the same value after each cycle.
(7 ) We mention a possible source of confusion: the reservoir entropy is a state function, while
we stressed that the entropy exchange is not. This apparent contradiction stems from the fact
that when we refer to state variables, we have in mind the state of the system and not the state
of the reservoir.
Δi S = ΔStot . We can thus rewrite the second law for a non-isolated system, exchanging
a reversible entropy flow with its environment, as follows(8 ):
(8) ΔS = Δi S + Δe S with Δi S≥ 0,
(9) dS = di S + de S with di S ≥ 0,
(10) Ṡ = Ṡi + Ṡe with Ṡi ≥ 0.
The first line corresponds to the entropy increase upon changing from an initial equilib-
rium state to a final equilibrium change. Note that the differential and derivative version
assume quasi-static changes (or more generally local equilibrium), otherwise the “usual”
thermodynamic entropy is not well defined.
We finally present yet another formulation of the second law, which is especially
useful if we are interested in the amount of work spent during the transition between
two equilibrium states for a system in contact with a heat bath. We introduce the
state function the free energy F = E − T S. For changes in contact with a single heat
bath, initial and final temperatures of the system are equal to that of the bath, hence
ΔF = ΔE − T ΔS. We find upon combining the second law Δi S = ΔS − Δe S ≥ 0 with
the first law (ΔE = Q + W = T Δe S + W ) that:
(11) T Δi S = W − ΔF ≥ 0.
In words, the amount of work needed for the transition (of the system in contact with a
heat bath) is at least equal to the change in system free energy. The equality is reached
for a reversible process. If the change of free energy is negative ΔF ≤ 0, the work can be
negative, i.e. one can derive an amount of work, namely at most −ΔF , from the change
in the state of the system (which then operates as fuel).
.
1 3. Nutshell equilibrium statistical mechanics [4]. – We briefly review some concepts
from equilibrium statistical mechanics, which will help us with the proper formulation
of stochastic thermodynamics. The system is now described in a fully microscopic way.
To make the connection with a Master equation description, it is convenient to take a
(semi-classical) quantum view: a micro-state of the system is an energy eigenstate, fully
identified by the (set of) parameter(s) m (for example a complete set of quantum num-
bers). Each micro-state has a specific energy m and number of particles nm . One of
the basic features of equilibrium statistical physics is the description in terms of ensem-
bles, introduced by Gibbs(9 ). The equilibrium state of the system is characterized by
(8 ) Note that Prigogine also writes, for obvious aesthetic reasons, the first law in a similar form,
namely dE = di E + de E with di E = 0.
(9 ) We mention in passing the connection with the ergodic theorem, which shows the equivalence
between the ensemble average considered here and the long-time average of a single trajectory.
We also mention that for large systems, fluctuations of macroscopic quantities are exceedingly
small, so that the ensemble average coincides with the most probable or typical value.
a probability distribution pm over these states(10 ). The specific form of this distribu-
tion depends on the situation. For an isolated system, one utilizes the micro-canonical
ensemble, with the system’s energy lying in a small interval I = [E0 , E0 + δE0 ] (with
δE0 E0 , but still large enough to contain many energy levels). All energy states inside
this shell are equally probable. One thus has
(12) pm = peq
m = 1/Ω for m ∈ I
with Ω the number of energy states in the shell I. When the system is in contact with a
heat bath (temperature T , kB T = β −1 and kB is the Boltzmann constant) and a heat or
particle reservoir (temperature T , chemical potential μ), the distributions are canonical
and grand canonical:
(13) m = exp [−β(m − F )] ,

pm = peq
(14) m = exp [−β(m − μnm − G)] ,
pm = peq
where F and G are the Helmholtz free energy and the grand potential, respectively(11 ).
Their explicit expression follows from normalization:

(15) exp (−βF ) = exp (−βm ) ,
m

(16) exp (−βG) = exp [−β(m − μnm )] .
m
We first turn our attention to the energy of the system, being the following ensemble
average:

(17) E= m pm .
m
This energy will be modified when the distribution changes and/or when the energy levels
move. For small changes one can write

(18) dE = (dm pm + m dpm ) = dW + dQ + dWchem .
m
We clearly recognize the first law of thermodynamics, with the following interpretation:
heat corresponds to instantaneous jumps (transitions) between states of different energy,
(10 ) In the full quantum version, we have to use the density matrix ρ, but off-diagonal elements
are anyway zero for equilibrium states in the energy representation.
(11 ) For ideal systems, with the total energy being the sum of single-particle energies, the grand
potential provides a simple starting point to obtain the famous Fermi-Dirac and Bose-Einstein
statistics for the occupation densities of the single-particle states.
resulting in a change in occupation probability. In case the number of particles also

changes, one has to substract the chemical energy part to obtain the heat contribution.
Work (aside from the chemical work) corresponds to moving an occupied energy level.
This can for example be achieved by applying an external field of changing the volume.
We next turn to the definition of entropy. We will use the Shannon expression [6],
since it gives rise to a consistent and elegant formulation in stochastic thermodynamics:

(19) S = −kB pm ln pm .
m
The above definition of the entropy is, at equilibrium, in agreement with the usual
thermodynamic entropy. This can be checked for the three ensembles mentioned above.
In particular, it yields the usual thermodynamic relations for Helmholtz free energy
F = E − T S and the grand potential G = E − T S − μN , where

(20) N= nm pm .
m
Furthermore, the interpretation is compatible with thermodynamics for a quasi-static

change. For example, one finds for a quasi-static change (around a reference equilibrium
state peq , pm = peq
m + dpm ) of a system in contact with a single thermal reservoir
dQ
(21) dS = −kB dpm ln peq
m = βkB m dpm = ,
m m
T
in agreement with the usual thermodynamic formula.

Finally, we mention a problem with the above expression as being the thermody-
namic entropy for an isolated system, which highlights the issue of reversibility versus
irreversibility. This argument applies to both classical and quantum description. The
Liouville theorem stipulates that ρ, the density in phase space (classical systems) or the
density matrix (quantum mechanics), is constant along the trajectory (i.e., the full time
derivative of ρ is zero). In quantum mechanics, it is a result of the unitarity of the
time-evolution operator. Classically, it follows from the Hamiltonian structure of the
equations of motion. We start with the evolution equation for the probability density
ρ (this is a conservation equation for the probability or a non-dissipative Fokker-Planck
equation):
∂ρ ∂ ∂

(22) =− q̇j + ṗj ρ.
∂t j
∂qj ∂pj
In combination with
∂H
(23) q̇j = ,
∂pj
∂H
(24) ṗj = − ,
∂qj
we find for the full derivative

dρ ∂ρ ∂ρ ∂ρ
(25) = q̇j + ṗj + = 0.
dt j
∂qj ∂pj ∂t
It easily follows that the following Shannon entropy
(26) S = −kB Tr ρ ln ρ
is conserved in time, and does at first sight not present a good candidate for the total
entropy. We will not discuss this issue further, nor the proposed arguments of reso-
lution (statistical interpretation, coarse graining, projection, chaos, quantum collapse,
expansion of universe, non-equilibrium initial condition, canonical typicality).
In the stochastic approach presented below, irreversibility is build in from the start.
Furthermore, we do not consider isolated systems, and the relaxation is due to the cou-
pling to reservoirs. The conservation of phase volume has however another fundamental
microscopic consequence, which we will need to incorporate in our mesoscopic theory,
namely the condition of detailed balance. This property states that it is equally proba-
ble, in a system at equilibrium, to observe any kind of transition, as it is to observe its
time-reverse. In other words, there is no arrow of time: in a movie of a system at equilib-
rium one cannot distinguish whether it is played forward or backward. The proof is most
easily given in the microcanonical ensemble. Consider the probability for a transition
from a subspace of phase space to another subspace, during a certain time (of measure
non-zero). For simplicity we consider that both subspaces are even in the momenta. We
note that there is a one-to-one correspondence between every trajectory that makes the
transition, and the time-reverse trajectory with reversed momenta. Because of Liouville
theorem, both sets, the one generating the forward and that producing the time-reversed
transition, have equal phase volume and are thus equally probable in a micro-canonical
distribution. The property of detailed balance can be infered from a handwaving yet
probably deeper argument using the Kelvin statement of the second law. Indeed, if such
a “time-asymmetric” transition would exist in a system at equilibrium, one could extract
work from it by building a work-generating device (kind of “waterwheel”) driven by these
transitions.
.
1 4. Nutshell Master equation [5]. – The state of the system is represented by the
index m. We assume Markovian dynamics with transition rates Wm,m ≥ 0, defined as
the probability per unit time to make a transition from state m to m (m = m)(12 ). The
essence of Markovian dynamics is the assumption that this transition probability, e.g.
for a transition from m to m, does not depend on how the former state m was reached.
As a result, the probability distribution pm (t) to be in state m at time t obeys a Master
(12 ) Take care not to confuse the transition rates Wm,m or the transition matrix W with the
work W .
equation, which we write in a number of different forms:

(27) ṗm = (Wm,m pm − Wm ,m pm )
m

(28) = Jm,m with Jm,m = Wm,m pm − Wm ,m pm
m

(29) = Wm,m pm .
m
In the last line, we introduced the “diagonal” element to compactify notation:

(30) Wm,m = − Wm ,m .
m ,m =m
Note that as a result, the matrix W with elements Wm,m is a “stochastic” matrix(13 ),
the sum of elements in a column adding up to zero:

(31) Wm,m = 0.
m
This property guarantees the conservation of probability. Formulated in an alternative

way, we note that the matrix W has a left eigenvector (1, 1, . . . , 1) with eigenvalue zero.
We call the corresponding right eigenvector the “stationary distribution” pst m:

(32) ṗst
m = Wm,m pst
m = 0.
m
We will assume for simplicity that this distribution is unique(14 ).

In the following, we will consider the possibility of a time-dependent transition matrix
W = W(t). The corresponding distribution is then obviously also time-dependent pst m =
pst
m (t). This distribution still satisfies relation (32), but only at a single instance of time
t, when W = W(t). pst is therefore only a genuine stationary distribution when the rate
matrix is time-independent. In this case, we can in fact construct a Lyapounov function,
showing that any initial distribution relaxes to this stationary distribution. The proof
goes as follows. We introduce the following H-function (terminology borrowed from the
H-function introduced by Boltzmann)
pm
(33) H = D(pm ||pst
m) = pm ln ≥ 0.
m
pst
m
(13 ) “Stochastic” is the name usually reserved for the transition matrix of a discrete time Markov
chains, with the elements of each column summing up to 1.
(14 ) Otherwise the state space consists of separate regions.
This quantity is positive and only zero when pm = pst m ∀ m. This quantity also appears
in information theory, where it is called the Kullback-Leibler distance or relative entropy
between the distributions pm and pst m [6]. Its positivity can easily be shown on the basis
of the inequality
(34) ln x ≤ x − 1 or − ln x ≥ 1 − x, ∀ x ≥ 0.
Furthermore, one shows using the same inequality that (assuming that pst
m is time-
independent!)
dH dpm pm
(35) = ln st
dt m
dt pm
pm
(36) = Wm,m pm ln
pst
m
m,m
pm pstm
(37) = Wm,m pm ln
pst p
m m
m,m
pm pstm
(38) = Wm,m pm ln
pst p
m m
m,m ;m=m

pm pst
m
(39) ≤ Wm,m pm − 1
pst
m pm
m,m ;m=m

pm pst
m
(40) = Wm,m pm − 1 = 0.
pst
m pm
m,m
The H-function has properties reminiscent of minus the entropy: it is positive and de-
creases in time until a minimum is reached at the steady state. There is however, at
this stage, no input whatsoever from physics. We will in fact show below that minus
the time derivative of the H-function corresponds, in an appropriate physical setting, to
a contribution to the entropy production (which we will call the non-adiabatic entropy
production), namely the one related to the relaxation to the steady state. Hence the
H-theorem is not related to the second law proper, but rather to a minimum entropy
production theorem.
2. – Ensemble stochastic thermodynamics

.
2 1. Ensemble stochastic thermodynamics: first law . – We now embark on a proper
formulation of the first law of thermodynamics for a system obeying a Markovian master
equation, proceeding in close analogy with statistical mechanics. We associate to every
state m of the system an energy m , and possibly a number of particles nm . The average
energy and number of particles are given by

(41) E= m pm ,
m

(42) N = nm pm .
m
Note that both m and pm can be time-dependent while nm is time-independent. When

the energy levels are shifted, we are performing work on the system via an external agent
(which we suppose to be a purely non-dissipative work source, which we need not describe
any further). The energy needed to change the probability distribution over the energy
levels, i.e. jumps between different states, is coming from heat exchange with reservoirs
(plus possibly a chemical work contribution if the number of particles also changes).
Hence as in the statistical mechanical description, we can write for the energy flux into
the system
(43) Ė = Ẇ + Q̇ + Ẇchem
with the work and heat (plus chemical work) flux

(44) Ẇ = ˙m pm ,
m

(45) Q̇ + Ẇchem = m ṗm .
m
Note that we give here the rate of change of the energy because we know the time
derivative ṗm (it is given by the Master equation) and ˙m . Usually, the time-dependence
of the energy level is attributed to an external agent acting on a control variable λ = λ(t),
with m = m (λ) (see also discussion further below), hence: ˙m = ∂/∂λ λ̇. Using the
Master equation, the heat and particle rates are given more explicitly by
1
(46) Q̇ + Ẇchem = m ṗm = m Jm,m = m,m Jm,m ,
2
m m,m m,m
1
(47) Ṅ = nm ṗm = nm Jm,m = nm,m Jm,m ,
2
m m,m
m,m
where the net average rate of transitions or flux from m to m (also mentioned above) is
given by (note that Jm,m = −Jm ,m )
(48) Jm,m = Wm,m pm − Wm ,m pm .

For compactness of notation, we also introduced the energy and number of particles
flowing into the system upon making the transition from m to m
(49) m,m = m − m ,
(50) nm,m = nm − nm .
To make the connection with a physical system and to identify amongst other things
the chemical work, we need to say more about the transition rates. We will consider the
simple situation of a system in contact with a number of heat/particle baths denoted by
the superscript (ν). We will assume that each reservoir gives an additive contribution to
the transition rate

(51) W= W(ν) .
ν
For this reason, we can also separate the fluxes Jm,m , and hence also heat and particle
rates (into the system) Q̇ and Ṅ , respectively, into separate contributions coming from
exchanges with each of the reservoirs ν
(ν)
(52) Jm,m = Jm,m ,
ν
(ν) (ν) (ν)

(53) Jm,m = Wm,m pm − Wm ,m pm
and (with the obvious convention that the flux J (ν) has to be used for the quantities
with superscript (ν))

(54) Q̇ = Q̇(ν) ,
ν

(55) Ṅ = Ṅ (ν) ,
ν
(ν)
(56) Ẇchem = Ẇchem .
ν
We now introduce the main assumption that makes the thermodynamic analysis compat-
ible with microscopic theory. It can be viewed as a “quasi-static” or “local equilibrium
assumption”, but now at the level of a stochastic description: we assume that each
transition rate W(ν) obeys—at each moment—detailed balance with respect to the cor-
responding instantaneous equilibrium distribution of the system in its contact with the
corresponding reservoir ν
(ν) (ν)
(57) Wm,m (t) peq,ν eq,ν
m (t) = Wm ,m (t) pm (t).
As equilibrium distributions, we consider the canonical case (heat reservoir)

m (t) = exp −β
peq,ν (m (t) − F eq,ν (t)) ,
(ν)
(58)

(59) exp −β (ν) F eq,ν (t) = exp −β (ν) m (t) .
m
and the grand-canonical situation (heat plus particle reservoir)

(60) peq,ν
m (t) = exp −β (ν)
m (t) − μ (ν)
nm (t) − Geq,ν
(t) ,

(61) exp −β (ν) Geq,ν (t) = exp −β (ν) m (t) − μ(ν) nm (t) .
m
In the grand-canonical scenario, transitions between states involve particle transport and
hence a chemical work contribution. The energy and particle rate of change in the system
due to the coupling with reservoir ν are obviously given by
1 (ν)
(62) Ė (ν) = m,m Jm,m ,
2 m,m
1 (ν)
(63) Ṅ (ν) = nm,m Jm,m .
2 m,m
The corresponding chemical work rate and heat flux read
(ν) 1 (ν) (ν)

(64) Ẇchem = μ nm,m Jm,m
2
m,m
(65) = μ(ν) Ṅ (ν) ,

1 (ν) (ν)
(66) Q̇(ν) = qm,m Jm,m
2
m,m
(67) = Ė (ν) − μ(ν) Ṅ (ν) ,
with
(ν)
(68) qm,m = m,m − μ(ν) nm,m
the heat to the system, coming from reservoir (ν) upon making the transition from m
to m.
We note in passing the the factor 1/2 in all above formulas (and in the formulas that
will follow) stems from the fact the we are counting twice the transitions between levels
m and m . This can be avoided by replacing the sum with a restricted sum (note that
the diagonal contributions m = m are zero)
1
(69) ... = ... .
2
m,m m,m ,m>m
We stress that the above “local equilibrium assumption” deals with the contact be-
tween system and reservoir. There is however no assumption whatsoever on the state of
the system itself, which can be very far from equilibrium(15 ). Note also that, according to
the above condition, the transition rates will automatically change in time, if the energy
levels are shifted. This brings us to the final assumption, namely that the energy needed
for the shift is provided by an idealized work source. The time-dependence of the energy
levels is often represented by introducing a control parameter λ, with m = m (λ). The
idea is that λ represents an external field (e.g. an electric field) or constraint (e.g. the
volume or a confining potential). A time-dependence of this parameter induces a spe-
cific time-dependence of all the energy levels (although there is in principle no objection
against a time-dependence for each energy level separately).
The above assumptions can be justified in a more detailed analysis by a separation
of time scales and by a weak coupling assumption between system and reservoirs. Both
assumptions are consistent with a closed description of the system in terms of the au-
tonomous Master equation dynamics (in which the idealized reservoirs and work source
do not appear).
We close with an important mathematical implication of our “local equilibrium as-
sumption”, which will play a key role in connecting our stochastic analysis with the
“usual” thermodynamic formulation. For simplicity of notation, we will, here and in the
sequel, often omit the explicit time-dependence “(t)” in the expressions. By combining
eqs. (13) and (57), we find
(ν)
Wm,m peq,ν
(70) ln = ln m
= −β (ν) qm,m = β (ν) qm ,m .
(ν)
Wm ,m peq,ν
m
In other words, the log-ratio of the transition rates (time kB ) is equal to minus the heat
flow into the system coming from the corresponding reservoir divided by kB T (ν) .
.
2 2. Ensemble stochastic thermodynamics: second law . – The entropy of the system
is formally identical to the definition that we introduced in statistical mechanics section,
i.e. we are using the Shannon entropy

(71) S = −kB pm ln pm ,
m
(15 ) Even a fully microscopic non-equilibrium dynamics is possible since the Markovian dynamics
include deterministic dynamics as a special case.
but with pm now solution of the Master equation (27). Our purpose is to derive an
entropy balance equation. One finds
(ν)
(72) Ṡ = −kB ṗm ln pm = −kB Wm,m pm ln pm
m m,m ,ν
1 (ν) (ν)
p
m
(73) = kB Wm,m pm − Wm ,m pm ln
2
p m
m,m ,ν
1 (ν) (ν)
W (ν) pm
m,m
(74) = kB Wm,m pm − Wm ,m pm ln (ν)
2 W pm
m,m ,ν m ,m
1 (ν) (ν)
W (ν)
m ,m
(75) +kB Wm,m pm − Wm ,m pm ln (ν) .
2 W
m,m ,ν m,m
We can thus write(16 )
(76) Ṡ = Ṡi + Ṡe
with
di S 1 (ν) (ν)
W (ν) pm
m,m
(77) Ṡi = = kB Wm,m pm − Wm ,m pm ln (ν)
dt 2 W pm
m,m ,ν m ,m
(ν)
(ν)
Wm,m pm
(78) = kB Wm,m pm ln (ν)
m,m ,ν Wm ,m pm
and
de S 1 (ν) (ν)
W (ν)
m ,m
(79) Ṡe = = kB Wm,m pm − Wm ,m pm ln (ν)
dt 2 W
m,m ,ν m,m
(ν)
(ν)
Wm ,m
(80) = kB Wm,m pm ln (ν)
.
m,m ,ν Wm,m
We now show that these expressions are exactly what we expect and deserve. First, we
have written the entropy production in a way that its positivity is obvious [(x−y) ln x/y ≥
0]. Second, it has the structure of a sum of fluxes time forces, just like in macroscopic
(16 ) Note (again) that the short hand notation used here Ṡi = di S/dt and Ṡe = de S/dt could
be misleading since these are not time derivatives of state functions: there are no such things
(state functions) as Si and Se !
irreversible thermodynamics
di S 1 (ν) (ν)
(81) = Jm,m Xm,m ,
dt 2 m,m ,ν
(ν) (ν) (ν)
(82) Jm,m = Wm,m pm − Wm ,m pm ,
(ν)
(ν) Wm,m pm
(83) Xm,m = kB ln (ν)
.
Wm ,m pm
(ν)
The physical interpretation of these quantities is clear: the “flux” Jm,m is the net average
number of transitions per unit time from m to m, and the “force” Xm,m is the log-ratio of
(ν)
the deviation from detailed balance, both with respect to transitions induced by reservoir
ν. We stress that the entropy production is not only non-negative overall, but it is so in
(ν) (ν)
each of its detailed contributions Jm,m Xm,m . Every breaking of detailed balance, with
respect to any reservoir and/or any pairs of states, gives rises to a separately positive
contribution in the entropy production. We also stress that the above expression gives an
explicit value of the rate of entropy production, for any state of the system including far-
from-equilibrium states. We leave it as an exercise to show that the entropy production
is underestimated if the contributions of separate reservoirs are not identified and one
uses for example the total transition matrix W
di S 1
(84) ≥ Jm,m Xm,m .
dt 2 m,m ,ν
We finally turn to the entropy flow. Using the crucial property (57), we find
de S 1 (ν) (ν)
W (ν)
m ,m
(85) = kB Wm,m pm − Wm ,m pm ln (ν)
dt 2 W
m,m ,ν m,m
1 (ν) (ν) (ν)
(86) = kB β Jm,m qm,m
2 m,m ,ν
Q̇(ν)
(87) = .
ν
T (ν)
We conclude that the entropy flow agrees with the standard expression from macroscopic
irreversible thermodynamics, being the sum of the heat flows from each reservoir ν into
the system, Q̇(ν) , divided by the temperature of the corresponding reservoir.
.
2 3. System in contact with two reservoirs: steady state and strong coupling. – We
consider a system at steady state, in contact with two heat and particle reservoirs, with
corresponding temperature and chemical potential, T (ν) and μ(ν) , ν = 1, 2, cf. fig. 1. At
this steady state (which also implies the absence of driving, i.e., no work contribution,
Fig. 1. – Schematic representation of a system in contact with two reservoirs.
asides from chemical work) we have Ė = Ṅ = Ṡ = Ẇ = 0. If the reservoirs have different

temperature and/or chemical potential, this steady state is however not an equilibrium
(1) (2)
state, and we can identify the following heat and particle flows Ė = JE + JE , Ṅ =
(1) (2)
JN + JN :
(1) (2)
(88) JE = JE = Q̇(1) + μ(1) Ṅ (1) = −JE = −Q̇(2) − μ(2) Ṅ (2) ,
(1) (2)
(89) JN = JN = Ṅ (1) = −JN = −Ṅ (2) .
Furthermore, we find from
(90) Ṡ = Ṡi + Ṡe = 0

Q̇(1) Q̇(2)
(91) Ṡe = +
T (1) T (2)
that
(92) Ṡi = −Ṡe = JE XE + JN XN ≥ 0,
with the usual expression of the thermodynamic forces
1 1
(93) XE = − (1) ,
T (2) T
μ(1) μ(2)
(94) XN = (1) − (2) .
T T
We mention that in the domain of linear irreversible thermodynamics, the fluxes and
forces are in linear relation with each other, more precisely
(95) JE = LEE XE + LEN XN ,

(96) JN = LN E XE + LN N Xn .
Positivity of the entropy production requires that the matrix L is non-negative. Fur-
thermore, Onsager derived from the reversibility of the microscopic law the symmetry
of this matrix: LEN = LN E . For the further discussion, we proceed with a constraint
called “strong coupling” that plays a special role for the efficiency of machines both close
to equilibrium and at maximum power. The constraint requires that the two fluxes are
proportional to each other: JE = JN = J. In the region of linear irreversible thermody-
namics, this condition is satisfied provided the Onsager matrix has a zero determinant
det L = 0. For strong coupling, the entropy production simplifies to
(97) Ṡi = JX ≥ 0
with the “collapsed” single thermodynamic force
XN x(2) − x(1)
(98) X = XE + =

with
− μ(ν)
(99) x(ν) = .
T (ν)
This situation is of special interest because reversibility, corresponding to zero entropy

production, can be achieved when X = 0 without requiring that the separate forces XE
and XN be zero. In particular, one does not need that both temperatures and chemical
potentials are equal to reach reversibility. Indeed, the condition X = 0 is satisfied
whenever x(1) = x(2) .
It is revealing to evaluate the efficiency of the thermal machine that can be built with
the two reservoir model. For strong coupling, one finds
(1) (2)
(100) JE = Q̇(1) + μ(1) Ṅ (1) = −JE = −Q̇(2) − μ(2) Ṅ (2) = Ṅ (1) = −Ṅ (2) ,
hence
Q̇(ν)
(101) = x(ν) Ṅ (ν) ,
T (ν)
and finally (JN = Ṅ (1) = −Ṅ (2) , recall also that Carnot efficiency is given by ηc =
1 − T (1) /T (2) , assuming that T (1) ≤ T (2) )
(102) Ẇchem = μ(1) Ṅ (1) + μ(2) Ṅ (2) = −Q̇(1) − Q̇(2)

(103) = (−T (1) x(1) + T (2) x(2) )JN = T (2) [−(1 − ηc )x(1) + x(2) ]JN .
We conclude that the efficiency η, being the ratio of the rate of the chemical work
produced over heat influx from the hot reservoir, in casu reservoir (2), is given by
−Ẇchem x(1)
(104) η= = 1 − (1 − ηc ) (2) .
Q̇(2) x
We thus indeed recover Carnot efficiency under the condition of reversibility x(1) = x(2) .
We stress that the present construction is quite different from the original Carnot ma-
chine: in order to avoid irreversible heat flows (which imply entropy production and
hence make it impossible to attain Carnot efficiency), the original Carnot set-up is run-
ning in a cycle of connection and disconnection (with intermediate adiabatic steps to
change the temperature). In the present construction, the condition of strong coupling
allows to reach reversible operation—hence Carnot efficiency—in a steady-state regime
where the system is at all times in contact with both reservoirs! The analysis is in fact
simplified significantly compared to the original Carnot cycle because it involves a single
steady-state operation.
.
2 4. Two-level system: efficiency at maximum power . – The previous discussion was
based on general thermodynamic arguments. We now switch to a concrete stochastic
model. This allows to calculate explicitly the basic quantity that is left unspecified in
the preceeding analysis, namely the dependence of the particle or energy flow JN and JE
on the system paramaters x(1) and x(2) . As model, we consider a quantum dot with a
single available energy level at the energy (17 ), which can exchange an electron with two
leads ν (temperature T (ν) , chemical potential μ(ν) , respectively). The quantum dot has
only two states, 1 and 0, corresponding to the energy level filled with one electron, or an
empty energy level. The probability per unit time for an electron to jump from lead ν
(ν)
into the empty level is given by W10 = a(ν) f (ν) , where a(ν) is a coupling constant, and
f (ν) the Fermi function
1
(105) f (ν) = f (x(ν) ) =
.
exp x(ν) + 1
The latter describes the occupancy rates of the energy levels in the lead. The proba-
bility for the reverse transition, an electron occupying the energy level of the dot jumping
(ν)
into the lead ν, is equal to W01 = a(ν) (1 − f (ν) ) (the 1 − f (ν) factor coming from the
fact that the corresponding energy level in the lead has to be empty for such a transi-
tion to take place). The steady-state distribution for the corresponding Master equation
(W = W(1) + W(2) ) is
W01
(106) pst
0 = ,
W01 + W01
W10
(107) pst
1 = .
W01 + W01
Inserting this result into the expression of the particle and energy flow, one finds after
(17 ) The other energy levels are supposed to be sufficiently far from the Fermi levels of the
reservoir, so that they are always full or empty and do not play a role.
some algebra

(108) J = J(x(1) , x(2) ) = JE = JN = α f (1) (x(1) ) − f (2) (x(2) ) ,
with
a(1) a(2)
(109) α= .
a(1) + a(2)
The “collapsed thermodynamic” force and entropy production read
x(2) − x(1)
(110) X= ,

(111) Ṡi = α f (1) (x(1) ) − f (2) (x(2) ) (x(2) − x(1) ) ≥ 0.
We now show how stochastic thermodynamics allows to address a question that goes
beyond the usual thermodynamic analysis. From eqs. (104) and (101), we find the
following expression for the power P of our device:
T (2) (2)
(112) P = −Ẇchem = x − (1 − ηc )x(1) J(x(1) , x(2) ).

We raise the following question of interest: for which values of the variables x(1) and x(2)
(obviously control parameters of our device) will the power reach a maximum? These
values x(1),∗ and x(2),∗ are the solutions of the following extremum condition:
∂P
(113) = 0,
∂x(1)
∂P
(114) = 0.
∂x(2)
These conditions give rise to a transcendental equation, which can be solved numerically,
see ref. [7] for more details. An even more interesting question concerns the efficiency
that is reached at maximum power. This efficiency is easily obtained by inserting the
values of x(1), and x(2), into eq. (104). An analytic treatment becomes possible in terms
of an expansion in terms of Carnot efficiency ηC . We just cite the final result for the
efficiency η at maximum power
ηC η2 7 + csch2 (a0 /2) 3

(115) η = + C + ηC + . . . ,
2 8 96
where a0 ≈ 2.39936 is the solution of a0 = 2 coth(a0 /2). It turns out that the coeffi-
cient 1/2 is universal for strong coupling, and is essentially a consequence of Onsager
symmetry [8]. The coefficient 1/8 is also universal under the additional condition of a
left-right symmetry (meaning that the fluxes change sign under inversion of the reservoir
values and couplings) [9]. Like Onsager symmetry, this universality can be linked at a
deeper level with the reversibility of underlying microscopic laws, and more directly to
the fluctuation theorem that we derive below in trajectory stochastic thermodynamics.
3. – Ensemble stochastic thermodynamics: more

.
3 1. General comments. – The theory that we developed so far looks consistent and
appealing, but it may appear to be tailormade for a special class of physical systems,
described by Markovian dynamics satisfying the condition of local equilibrium (57). It is
however possible to show, and this is work in progress, that the theory fits into a much
broader scheme. This is not so surprising: it is a typical strength of thermodynamic
theories to far exceed the domain application/formulation in which they are originally
presented. The Markovian context however allows to easily explore further implications
of the theory and to develop new ideas. In this section, we illustrate this with two
promising extensions.
.
3 2. Landauer principle [10,11]. – We mentioned one of the formulations of the second
law for a system, in contact with a single heat bath, driven from one equilibrium state
into another one, namely
(116) T Δi S = W − ΔF eq ≥ 0.
We added the superscript eq to stress that the free energy is, in conventional thermody-
namics, only well defined for equilibrium states and that the above results only applies
to changes between such states. We now show that stochastic thermodynamics not only
reproduces this result, but gives a generalization for transitions between any two states,
not necessarily at equilibrium. The result is closely related to the so-called Landauer
principle, which establishes a relation between thermodynamics and information. We
start by rewriting first and second law, and the expression for the entropy flow when in
contact with a single heat reservoir (temperature T )
(117) Ẇ = Ė − Q̇,
(118) Ṡi = Ṡ − Ṡe ≥ 0,
(119) Q̇ = T Ṡe .
We now introduce the non-equilibrium free energy
(120) F = E − T S.
Combination with the previous formulas leads in a one-line calculation to
(121) T Ṡi = Ẇ − Ḟ ≥ 0,
Fig. 2. – Left: Szilard conversing with Einstein. Right: schematic representation of the Szilard
engine (with a Brownian variation below), converting one bit of information into kB T ln 2 of
work.
which is the “instantaneous” generalization of eq. (116). To make a clearer comparison,

it is revealing to integrate this result over an interval of time
(122) T Δi S = W − ΔF = W − ΔF eq − T ΔI ≥ 0.
This result is valid for any initial and final condition. To compare with equilibrium
states, we have introduced the quantity I
(123) T I = F − F eq ,
pm
(124) I = kB D(pm ||peq
m ) = kB pm ln ,
m
peq
m
which is a measure of the distance between the actual distribution and an equilibrium
one. The most important upshot of the new result (122) is that the amount of work
−W that can be generated by changing the state of a system is at most equal to the
decrease of equilibrium free energy −ΔF eq plus the decrease in “information content”
−T ΔI. We give a simple illustration which at the same time makes the contact with
the so-called “Landauer principle”, see fig. 2. We consider a single particle in a box
and identify the states m = 0 and m = 1 as being in the left and right half of the
box, respectively. At equilibrium one has peq eq
0 = p1 = 1/2. We however consider the
following non-equilibrium initial state p0 = 1 and p1 = 0. Such an initial condition
could have been prepared (by compressing the particle in the left hand side of the box),
or by performing a measurement, revealing that the particle is in that half side of the
box. This corresponds to the information of 1 bit. In either case, our formula predicts
that we can extract work out of this non-equilibrium initial condition, namely (at most)
−T ΔI = T I(t = 0) = kB T ln 2 (we use the physically relevant limit “0 ln 0” = 0). In
words one can extract at best kB T ln 2 work from erasing one bit. In the same way, one
proves that writing one bit cost at least kB T ln 2 of work [12].
.
3 3. Adiabatic and non-adiabatic entropy. – Let us return to the explicit expression
(77) for entropy production, and in particular to that for the thermodynamic force (83).
One easily verifies that the latter can be split into two separate contributions
(ν) (ν) (ν)

(125) Xm,m = Am,m + Nm,m ,
(ν)
(ν) Wm,m pst
m
(126) Am,m = kB ln (ν)
,
Wm ,m pst
m
pm pst
m
(127) Nm,m = kB ln .
pm pst
m
This results in a splitting of the entropy production into two contributions that are, and
this is somewhat surprising, separately non-negative
(128) Ṡi = Ṡa + Ṡna ≥ 0,

1 (ν) (ν)
(129) Ṡa = Jm,m Am,m ≥ 0,
2
m,m ,ν
1 (ν) 1
(130) Ṡna = Jm,m Nm,m = Jm,m Nm,m ≥ 0.
2
2
m,m ,ν m,m
The proof again follows from the inequality ln x < 1 − x, conservation of probability (31)
and the fact that pst is the instantaneous “steady state”, cf. eq. (32). One can give the
following interpretation: the so-called adiabatic contribution Ṡa to the entropy produc-
tion is the one that survives when one takes the “adiabatic” limit in which the probability
distribution is at all times equal to the (instantaneous) steady state. It is in particular
equal to the full entropy production when operating at a steady state (time-independent
rates). Note that this component will be only zero when operating at full equilibrium.
At non-equilibrium steady states it has a constant non-zero value, expressing the amount
of irreversible dissipation needed to maintain this state.
The contribution Ṡna is a “relaxational” entropy production, internal to the system. It
only depends on the total transition matrix W, and can be rewritten under the following
way:
pm
(131) Ṡna = −kB ṗm ln ≥ 0.
m
pst
m
In the special case of constant transition matrix (so that pst is a time-independent quan-
tity), one verifies that Ṡna = −kB Ḣ. This gives a physical meaning to the H-function:
its time derivative is minus the non-adiabatic entropy production. Only in the special
case of an equilibrium steady state does it correspond to minus the (full) irreversible
entropy production.
.
3 4. Other extensions. – We will be very brief and give an haphazard list of other
developments. One can include driving in the above discussed example of a quantum
dot, by considering a time-dependent energy . The system can then also operate via a
traditional Carnot cycle (system in contact with only one of the reservoirs at any given
time), and one can show that both Carnot efficiency and efficiency at maximum power
are reproduced. One can also discuss energy injection in a steady state by considering
a random perturbation of the energy level. Again all the “familiar” thermodynamic
features are reproduced, including also the issue of work to work conversion engines
(the driving can be used to pump particles up a chemical gradient). Universal features,
similar to those for thermal machines, can be derived [13,14]. Several variants of systems
with two quantum dots have been studied, including a photo-electric cell [15] and photo-
electric cooling [16]. Another model that was studied in the context of efficiency at
maximum power is a maser model [9]. One can relax the condition of “local equilibrium”
for the transition rates. In this case a new contribution appears in the entropy production,
which can be interpreted in terms of a “Maxwell demon” that breaks the detailed balance
of the transition rates [17]. Using Markovian partitions, a connection can be established
between the entropy production discussed here and the concept of Kolmogorov-Sinai
entropy in dynamical systems [18]. In an early paper [19], it is shown that the steady
states close to equilibrium are the states of minimum entropy production. A large section
of the literature deals with stochastic thermodynamics for Langevin equations [20-23].
Note however that the formulation for Master equations includes, as a particular case,
the continuous Markov processes described by Fokker-Planck and Langevin equation,
since they can be obtained —via an appropriate limiting procedure— from a Master
equation. As a historic note, we mention that the “early steps” in the development of
ensemble stochastic thermodynamics were in fact taken at the level of a Master equation
description [19,24-26], with (non-equilibrium) chemical reactions as prototypical example,
followed more than a decade later by similar and further developments in the context of
soft condensed matter, with (driven) overdamped Brownian motion as workhorse [21,22].
The Langevin version of the adiabatic and non-adiabatic entropy production is derived
in ref. [27]. For a further review of the theory and applications in the context of Langevin
and Fokker-Planck equations, see refs. [28, 29].
4. – Trajectory stochastic thermodynamics

.
4 1. Motivation. – We discussed on several occasions one of the traditional formula-
tions of the second law, namely T Δi S = W − ΔF eq ≥ 0, giving a bound for the work
on a system undergoing a change between two equilibrium states, while in contact with
a heat bath. We stress that classical thermodynamics does not specify the value of W ,
it only gives a lower bound. This bound is reached for a reversible (and in particular
infinitely slow) process. As example, consider the expansion of an ideal gas (temperature
T , number of particles N ) from initial volume Vi to final volume Vf . We can extract
V
an amount of work −W ≤ −ΔF eq = T ΔS eq = N kB T ln Vf /Vi = Vif P dV (ΔE eq = 0
for isothermal change in an ideal gas). The limit is reached for a quasi-static isothermal
Fig. 3. – The work w needed to stretch a RNA strand with optical tweezers is a random variable.
expansion. For any other (e.g., faster) scenario, the extracted work will be less. Note
however that the same amount of work will be needed when one repeats the experiment
in the same way, i.e., W is a self-averaging quantity in such a macroscopic experiment.
Consider now a similar experiment but on a small (non-macroscopic) scale, for example
the elongation of a polymer (e.g., RNA strand) with optical tweezers from an initial to
a final equilibrium position, cf. fig. 3. The amount of work can be obtained from force
versus elongation curves (work being force time displacement). We now observe that
this quantity will be different in one run from another, because the system is subject
to fluctuations(18 ). We denote it by w, using a lower script to stress that the quantity
depends on the particular trajectory Π that the system happens to follow in the given
run, w = wΠ . The question now arises concerning the connection with the second law.
One could argue that the second law only applies to macroscopic systems. Hence one
should do the experiment with a macroscopic number N of independent RNA strands.
By the law of large numbers, the work per strand will converge to its average value,
and the second law becomes: N < w > −ΔF eq ≥ 0 (where the free energy difference
ΔF eq refers to that for a collection of N RNA strands). This may look at first like a
satisfactory explanation. However, by repeating the above experiment one can construct
the probability distribution P (w). It looks odd that the second law of thermodynam-
ics, being one of the most fundamental laws in nature, just prescribes an upper bound
for the first moment of this distribution. We now proceed to show that one obtains a
much deeper formulation of the second law as a symmetry property of this probability
distribution, which implies in particular the above inequality for its first moment. We
will derive this symmetry property in the context of stochastic thermodynamics, but we
stress that it has—in our opinion—a much deeper validity, as documented in the liter-
ature in several other specific or general settings (thermostated systems, classical and
quantum dynamics, relativity and quantum field theory).
.
4 2. First law: trajectory thermodynamics [28]. – Let us return to our stochastic de-
scription of a system with states m and probability distribution pm , obeying a Markovian
(18 ) The only and notable exception is an infinitely slow process, in which the work is always
the same and equal to the (equilibrium) free energy difference. We will see below that the
probability distribution for w then indeed reduces to a delta function.
Fig. 4. – Schematic representation of the first law along a trajectory.
Master equation. For simplicity (and because of lack of space and time), we will in the
following assume that there is no particle transport involved (so we need not worry about
the change of particles upon jumping between levels and the associated chemical work).
We will now focus on various properties of the system, not at the ensemble level, but at
the trajectory level. A trajectory generically denoted by Π corresponds to the specifi-
cation of the actual state in the time interval under consideration, i.e., m(t), t ∈ [ti , tf ].
In view of the discrete nature of the state, one can alternatively specify the initial state
mi = m1 = m(ti ), the jumps from mj to mj+1 at specific instances of time tj+1,j ,
j = 1, . . . , N − 1, where N − 1 is the total number of jumps, and mf = mN = m(tf )
being the final state at time tf , see fig. 4. To stress that we are dealing with trajectory-
dependent quantities, we will use a lower script, e.g., Δe for energy difference, w for
work, q for heat heat (and s for entropy, to be defined in the next section). Most of
the time, we suppress for simplicity of notation the reference to the particular trajectory
under consideration (Δe = ΔeΠ , w = wΠ , q = q Π , etc.).
The energy is well defined even at the level of trajectories: the energy at time t is
the energy of the particular state m(t) in which the system resides at that time in the
trajectory under consideration
(132) e = eΠ (t) = m(t) (t).
The energy is in fact a state function, but the state is now the actual micro-state of
the system (compare with the ensemble picture, where E is also a state variable, but
expressed in terms of the statistical state, i.e. the probability distribution over the energy
levels). In particular the energy e(t) at time t for a trajectory Π depends only on the
state of the system at that time, and not on the remainder of the trajectory. For the
same reason the energy difference Δe only depends on final and initial states: the change
Δe between any initial and final times ti and tf is given by
(133) Δe = ef − ei = mf (tf ) − mi (ti ).
Conservation of energy is of course valid trajectorywise, so one can write the first law of
thermodynamics trajectorywise (i.e. valid for any trajectory Π)
(134) Δe = w + q.
As in the ensemble picture, heat and work are not state functions but depend on the
actual trajectory that the system has followed (in addition to the dependence on the way
the perturbation is applied). They are related, respectively, to the jumps between energy
levels and the shift in energy level, as they occur along the trajectory under consideration.
Explicitly, one has for the trajectory Π
(135) w = [m1 (t2,1 ) − m1 (ti )] + [m2 (t3,2 ) − m2 (t2,1 )]

(136) + . . . + [mN (tf ) − mN (tN,N −1 )],

(137) q= qj+1,j ,
jumps
(138) qj+1,j = mj+1 (tj+1,j ) − mj (tj+1,j ).
In case of different thermal reservoirs ν, a trajectory Π needs also to specify which reser-
voir is responsible for which transition, so that one can further specify the subdivision
of the heat flow into contributions due to the different reservoirs

(139) q= q (ν) ,
ν
where q (ν) is the sum of those energy jumps, for which the heat comes from reservoir ν.
It is instructive to reproduce the first law in its differential form. To do so, we rewrite
the eq. (132) for the energy as follows:

Kr
(140) e(t) = m(t) (t) = m (t)δm,m(t) ,
m
so that the two contributions, work rate and heat rate, appear quite naturally as before
from the modulation of the energy level and from the change in state (the “jumps”),
respectively
(141) ė = ẇ + q̇,

Kr
(142) ẇ = ˙m (t)δm,m(t) ,
m

Kr
(143) q̇ = m (t)δ̇m,m(t) .
m
One easily verifies the agreement between the integral and differential formulation of the
first law(19 ).
.
4 3. Second law: trajectory thermodynamics. – The main hurdle is to define entropy
at the trajectory level. This may appear at first to be an oxymoron since, as argued
so forcefully by Boltzmann in his historic debate about the second law, entropy is a
property of the ensemble. One can however attach an entropy to an event taking place
in a sampling of a random variable. This is routinely done in information theory, where
one quantifies the concept of surprise as − ln pm upon observing the outcome m when
its probability is pm [6]. There is no surprise when pm = 1 and the surprise becomes
increasingly large when pm → 0. Upon repeating the experiment, we find that the
average surprise is equal to the entropy S = −Σm pm ln pm . We can of course use the
same concept when both p and m are time-dependent and thus define the stochastic or
trajectory entropy as follows:
(144) s = sΠ (t) = −kB ln pm(t) (t).
In other words, the entropy of the system in a specific trajectory is, at each time, minus
the logarithm of the probability, at that time, to be in the observed state. While being,
like the energy, a “micro-state” variable, function of m(t), it retains an “ensemble qual-
ity”, since its value is also determined in terms of the probability distribution pm (at the
same time t). This definition of the stochastic entropy was introduced by Seifert [29, 30].
We proceed to show how the above definition of stochastic entropy leads to a pleasing
formulation of the second law at the trajectory level, and even more importantly will
result in a profound reassessment of the second law itself. Being a micro-state function,
we find that the change of stochastic system entropy over a finite time interval [ti , tf ] is
given by
(145) Δs = −kB ln pmf (tf ) + kB ln pmi (ti ).
In analogy with the second law, we would like to split this system entropy change into an
entropy flow and an entropy production term. At the ensemble level, the entropy flow is
the average heat flow divided by temperature (of the corresponding reservoir). Heat flow
is however equally well defined at the trajectory level, as we discussed in the previous
(ν)
section. Recalling that qj+1,j is the heat taken from reservoir (ν) upon the jump from
mj to mj+1 , we can immediately write the trajectory version of the entropy flow
qj+1,j
(ν)
(146) Δe s = .
jumps
T (ν)
Kr
(19 ) To find the time-derivative of the Kronecker delta, note that δm,m(t) goes from 0 to 1, and
from 1 to 0, when m(t) jumps into, or our of state m, respectively. Hence the time-derivative
consists of a sum of delta functions, with weights +1 and −1, centered at the time of the jumps.
It will be usefull, when discussing the entropy production, to rewrite this expression using
the basic property (70) as follows:
(ν)
Wj+1,j
(147) Δe s = −kB ln (ν)
.
jumps Wj,j+1
By combining eqs. (145) and (147), we obtain the following expression for the trajectory
entropy production:
(ν)
Wj+1,j
(148) Δi s = Δs − Δe s = −kB ln pmf (tf ) + kB ln pmi (ti ) + kB ln (ν)
.
jumps Wj,j+1
Note that this quantity can have any sign. We will in fact show below that Δi s can
not always be non-negative, i.e., there must exist trajectories for which it is negative.
But before rewriting the above not so beautiful expression into a form which will allow
us to make such sweeping statements, we make a first simple “test”. Let us consider the
experiment that we mentioned before as one motivation for trajectory thermodynamics: a
small-scale system is brought from one equilibrium state into another one, under injection
of an amount of work w. When the experiment is repeated, the detailed trajectory that
the system will follow is different and hence the work will also change. It would be very
pleasing if one finds that the above-introduced entropy production is related to this work,
in a way similar to the macroscopic counterparts, cf. eqs. (116) and (122). We therefore
consider the system in contact with a heat bath at temperature T , and introduce the
stochastic free energy
(149) f = e − T s,
which is obviously also a micro-state function.

Combination of the stochastic second law, cf. eqs. (146) and (148), with the stochastic
first law (134) leads to
(150) T Δi s = w − Δf,
which is the stochastic analogue of eq. (122).

We now identify the expressions for the stochastic entropy and free energy when the
probability distribution pm has an equilibrium form. For a canonical distribution (13)
one finds (instating the subscript eq to stress that we are dealing with the equilibrium
distribution)
e − F eq
(151) seq = ,
T
hence
(152) f eq = F eq .
At equilibrium, the stochastic free energy reduces to the ensemble averaged free energy,
and is independent of the actual micro-state m! In particular, if initial and final condi-
tions are at equilibrium, we find from eq. (150) that
(153) T Δi s = w − ΔF eq .
This result is of particular interest since the statistical properties of the random variables
Δi s and w are now essentially the same (ΔF eq being a given fixed amount). As in the
ensemble average case, we would like to differentiate this result from the situation where
initial and final distributions are not at equilibrium. We define, in analogy to eq. (123)
(154) f − f eq = T i,
pm
(155) i = kB ln eq ,
pm
so that we can rewrite eq. (150) as follows:
(156) T Δi s = w − ΔF eq − T Δi.
We finally turn to an alternative expression of the stochastic entropy production Δi s,

which will allow to derive a deep symmetry relation concerning its stochastic properties.
We start by defining a “forward” experiment by applying a specific time-dependence of
the transition rates W(t), in the interval [ti , tf ] starting from a given initial distribution
pm (ti ) at the initial time ti . In this experiment, one can identify the probability P(Π) to
observe a specific trajectory Π. We now define a time-reversed “tilde” experiment. The
initial probability distribution is the final distribution of the forward experiment. The
rates W̃ are the rates W ran backward in time. We denote by P̃(Π̃) the probability
to observe the time-reversed trajectory Π̃ in this time-reversed experiment(20 ). We
now write what we believe to be the single most important expression in stochastic
thermodynamics
P(Π)
(157) Δi s = kB ln .
P̃(Π̃)
The proof goes as follows. The probability P(Π) to observe the specific trajectory Π,
cf. fig. 4, is equal to the probability pmi (ti ) to start in the state mi times the probability
(20 ) Note that we are in fact dealing here with functional probability densities for paths. Note
further that we assume the variables m to be even function of the momenta, and that there is
no magnetic field.
to stay in this state until the first jump, times the probability to make this jump, and so
on for the other jumps, with a final factor expressing that the system does not leave its
final state mf . The probabilities (per unit time) to make the jumps are of course well
known, namely
(ν)
(158) Wj+1,j .
The probability for not making a jump while in state m during an interval of time [t, t ]
is given by (ti = t + idt with dt = (t − t)/n)
⎡ ⎤

n
n
(159) lim ⎣1 − Wm ,m (ti )dt⎦ = lim [1 + Wm,m (ti )dt]
n→∞ n→∞
i=1 m ,m =m i=1

t
= exp Wm,m (τ )dτ .
t
Turning to the probability of the reverse trajectory, we make the crucial observation that
the latter contributions are exactly the same: the probability for not making transitions
are identical in forward and backward trajectories. The probability for making (the
backward) jumps, on the other hand, are obviously given by
(ν)
(160) Wj,j+1 .
The probability for the starting state mf of the reverse trajectory is, by assumption,
given by pmf (tf ). We thus conclude that
P(Π) (ν)
Wj+1,j
(161) ln = ln pmi (ti ) − ln pmf (tf ) + ln (ν) ,
P̃(Π̃) jumps Wj,j+1
which is indeed identical to Δi s/kB . One should give credit to Crooks, who realized the
above cancelation (in his PhD thesis [31]) and applied it to the heat flux (which is in fact
the entropy flow)
P0 (Π) (ν)
Wj+1,j
(162) Δe s = −kB ln = −kB ln (ν) .
P̃0 (Π̃) jumps Wj,j+1
Here P0 and P̃0 are the probabilities for the trajectory Π and Π̃ disregarding the con-
tribution for the starting probabilities pmi (ti ) and pmf (tf ). In most earlier works, the
latter contribution, which is in fact equal to Δs, was referred to as “the boundary term”
whose physical significance was not fully realized until the work of Seifert.
Before turning to the “dramatic” implications of eq. (153), we close the loop by
returning to the ensemble expression for the entropy production. As Δi s is the entropy
production for a specific trajectory Π, which occurs with probability P(Π), the average
entropy production reads
P(Π)
(163) Δi S =
Δi s = kB P(Π) ln = kB D(P||P̃) ≥ 0.
Π
P̃(Π̃)
We immediately recognize, with contentedness and satisfaction, that this is indeed a

non-negative quantity. One can also verify with relief that it corresponds to the time-
integrated
tf
version of the ensemble entropy production Ṡi given in eq. (77), Δi S =
ti
Ṡ i dt.
As if all this was not pleasing enough, we close with another bonus of the above
result. It was derived from a Master equation description, but the expression for the
stochastic entropy production is expressed in terms of probability for paths, without
explicit reference to the transition matrix. Since Langevin/Fokker-Planck equations can
always be obtained via a limiting procedure for a Master equation, the same result applies
for this description, but the probabilities of the paths now refer to Brownian paths(21 ).
The conclusions that we will derive in the next section do not depend on the detailed
form of the trajectories and therefore apply equally well to any Markovian process, be it
a jump process, a continuous process or a combination of both.
.
4 4. Integral and detailed fluctuation theorem. – We are now ready to “move to the
top”. We will start with the so-called integral fluctuation theorem. Instead of looking
at the “usual” ensemble entropy production Δi S, the average of the stochastic entropy
production Δi s, let us consider the following average:

Δi s P̃ P̃
(164) exp − = = P = 1,
kB P P
Π
where we used the fact that P̃ is a normalized probability (and the sum over all forward
paths implies a sum over all backward paths(22 )). The above expression is called the
integral fluctuation theorem. Note that it holds for any initial probability distribution
(and any perturbation in the sense of time-dependent transition rates). It implies, by
Jensen inequality [6], the “usual” second law
(165) Δi S =
Δi s ≥ 0.
For the special case that the transition is between equilibrium states at the same
temperature, we know that T Δi s = w − ΔF eq , and the above expression reduces to
(21 ) Fokker-Planck equations describe so-called continuous Markov processes with realizations
that are continuous, but nowhere differentiable.
(22 ) To be more precise, we need to show that the Jacobian for the transformation of forward
to backward paths is one. This property in fact derives at the most fundamental level from
Liouville theorem.
the celebrated Jarzynski equality (originally derived in the context of a microscopic

Hamiltonian [32], and later for a mesoscopic Markovian description [33])
(166)
exp (−βw) = exp (−βΔF eq ) .
This result is quite remarkable since it applies independently of the type of perturbation
and the intermediate departure of the system from equilibrium, while it incorporates via
Jensen’s inequality the “traditional” second law information
w ≥ ΔF eq .
For transitions between non-equilibrium states one obtains, using (156), a generalized
Jarzynski equality
(167)
exp (−β(w − T Δi)) = exp (−βΔF eq ) ,
which by Jensen’s inequality implies the earlier discussed Landauer principle, cf. (122).
The above integral fluctuation theorems can be derived from an even more “detailed”
property, which is appropriately called a detailed fluctuation theorem. We ask the fol-
lowing question: given an experimental setting (initial condition and driving), what is
the probability P (Δi s) to observe a change in the stochastic entropy equal to Δi s?
Obviously, this probability can be calculated as follows:

P(Π)
(168) P (Δi s) = P(Π)δ Δi s − kB ln
Π
P̃(Π̃)

P(Π)
(169) = exp (Δi s/kB ) P(Π̃)δ exp Δi s − kB ln
Π
P̃(Π̃)

P̃(Π̃)
(170) = exp (Δi s/kB ) P(Π̃)δ exp −Δi s − kB ln
P(Π)
Π̃
(171) = exp (Δi s/kB ) P̃ (−Δi s),
where

P̃(Π̃)
(172) P̃ (−Δi s) = P(Π̃)δ −Δi s − kB ln .
P(Π)
Π̃
We are thus led to the famous fluctuation theorem

P (Δi s) Δi s
(173) = exp .
P̃ (−Δi s) kB
In words: it is exponentially more likely to see a stochastic entropy increase Δi s in an

experiment, than to see a corresponding decrease −Δi s in the time-reverse experiment.
Before proceeding to a further discussion and illustration of this amazing result, we
need to address a possible source of confusion in the notation and subtlety in the defini-
tion (172): −Δi s is clearly minus the entropy production in the forward path, but how is
it related to the entropy change in the backward path? It turns out that −Δi s is indeed
the entropy production in the backward path Π̃ under the special condition that both ini-
tial and final conditions of the trajectory are stationary states. Indeed we can apply the
general result to find the entropy production Δi s̃ of a path Π̃ in the backward protocol
P̃(Π̃)
(174) Δi s̃ = kB ln .
˜ Π̃)
P̃( ˜
It is obvious that Π̃˜ = Π. However P̃˜ = P only if the initial and final distribution
match: the final distribution of the backward scenario has to be the initial distribution
of the forward trajectory. This will in general only be the case if the initial and final
distributions are stationary (which in an experiment means that we have to stop the
driving and wait until the steady state is reached). Under this condition one concludes
that Δi s̃ = −Δi s, and the fluctuation theorem takes on the additional meaning that
one compares the entropy productions in forward and backward experiments. As a
simple and important application, we cite the so-called the non-equilibrium steady
states. In this case, we have no driving so that furthermore P̃ = P. Hence it is, in this
case, exponentially more likely to see a stochastic entropy increase Δi s than to see a
corresponding decrease −Δi s in the same experiment.
We close with some comments on the solution of eq. (153), which we write now as a
mathematical condition on a probability density P (x) and its twin P̃ (x) (x = Δi s/kB )
P (x)
(175) = exp(x).
P̃ (−x)
We mention the interest of the cumulant generating function

(176) G(λ) = dxP (x) exp (−λx) ,
and a similar definition for G̃ in terms of P̃ . One easily verifies that the detailed fluctu-
ation theorem eq. (175) is equivalent with the following symmetry property:
(177) G(λ) = G̃(1 − λ).
The calculation of the cumulant generating function is in most cases easier than that of
the probability distribution. Furthermore, the detailed fluctuation theorem can often be
proven without an explicit calculation by showing that the equation for the cumulant
generating function possesses the above symmetry [34, 35]. Furthermore, the cumulant
generating function gives direct access to the cumulants of the corresponding quantities,
which can often be evaluated exactly in a large time limit (large deviation function).
Considering the simple case, P = P̃ , one can try as solution a Gaussian probability
distribution (with average
x and dispersion σ 2 ), to find that it is a solution provided
the dispersion is twice the average
2

exp − (x−x)
4x
(178) PG (x) = .
4π
x
One concludes that the general solution of eq. (175) is thus given by this Gaussian times
any non-negative even function (with proportionality factor guaranteeing normalization)
(179) P (x) = PG (x)f (x),

(180) f (x) = f (−x) ≥ 0,
∞
(181) dxPG (x)f (x) = 1.
−∞
Of special interest is the case of a Gaussian distribution with vanishing mean: one finds
P (x) = δ(x). This corresponds to equilibrium with the stochastic entropy production be-
ing always zero. Hence at equilibrium there is no entropy production even trajectorywise.
When the system is not at equilibrium, the distribution will have a finite width. Consider
again a non-equilibrium steady state (so that P = P̃ ). Note the somewhat surprising
implication now of eq. (175): in order to have trajectories with positive stochastic en-
tropy production, there must also be trajectories with a corresponding negative entropy
production, even though the latter are exponentially unlikely compared to the former.
Finally we again note that in transitions between equilibrium states (temperature T ),
one can identify the entropy with work via T Δi s = w − ΔF eq , and obtain the famous
Crooks relation [36]
P (w)
(182) = exp (β(w − ΔF eq )) .
P̃ (−w)
There are by now many illustrations of this beautiful relation. We cite in particular
the early experimental verification for the work upon opening and closing of an RNA hair-
pin [37], the computer simulations of a Joule experiment of dragging a triangle through an
ideal gas [38], and the thought experiment of a photon exchange between black bodies at
equilibrium at a different temperature [39]. Note on the other hand that the verification
of the above relation requires working on an energy scale of the other of kB T .
5. – Perspectives
The above discussion is only the beginning of what can be done with trajectory
thermodynamics [40-47] see also [29, 48-50]. All the additional features and problems
that we discussed at the ensemble level can be reconsidered at the trajectory level, in
particular the splitting of the stochastic entropy into an adiabatic and a non-adiabatic
component, the identification of a stochastic contribution for information processing,

the discussion of various models, etc. There are furthermore questions specific to the
trajectory picture, notably the so-called Feynman-Kac formula [51], which in its origin
considers functionals of Brownian motion, and shows how these are related to the solu-
tion of a differential equation (in particular the Schrödinger equation). The quantities
we are dealing with here are functionals of more general Markov processes, but analo-
gous techniques can be developed for this case. These results allow to rederive Onsager
symmetry and beyond [52], to obtain the famous fluctuation dissipation theorem, linking
response properties to equilibrium correlation functions, and beyond [53-55], and even to
discuss response when perturbing a non-equilibrium steady state or even a general time-
dependent reference state [56]. The fluctuation and work theorem have also been proven
in various other contexts, notably for microscopic classical and quantum Hamiltonian
dynamics, for thermostated systems, and for non-Markovian processes. A lot of work
has also gone into the study of the large deviation properties for trajectory-dependent
quantities in non-equilibrium steady states, such as the (heat or particle) fluxes to the
reservoirs (since these are in principle more easily accessible). The best studied model
in this context is probably the asymmetric exclusion model [57]. It may, at first sight,
appear difficult to verify experimentally the predictions of trajectory thermodynamics.
Yet a number of beautiful experiments have recently been carried out, in which a very
good agreement is obtained with the theoretical predictions [37, 58-61].
We end with a sweeping conclusion. Thermodynamics is probably the only branch of
physics that has found applications in almost all fields of science. Providing a novel and
extended framework for thermodynamics is therefore expected to have very productive
and possibly also very deep implications. It remains to be seen how many of the ideas
presented above supersede the framework in which they were formulated. It is encourag-
ing to see that the fluctuation theorems remain valid (with appropriate interpretation)
for quantum and relativistic dynamics. Furthermore the quantum case raises very inter-
esting new questions dealing with typical quantum features [62-64], such as interaction
between system and reservoir, energy uncertainty principle, the measurement process,
the quantums of transport, entanglement, Bell inequalities and quantum computation.
It is also tempting to speculate that these new insights can be used profitably to make
more efficient machines at the small scale. Many novel constructions for such thermal
engines and refrigerators have indeed recently been proposed in the literature.
∗ ∗ ∗
We acknowledge support from the research network “Exploring the Physics of Small
Devices” of the European Science Foundation.
REFERENCES
[1] Callen H. B., Thermodynamics and an Introduction to Thermostatistics (Wiley) 1985.

[2] Kondepudi D. and Prigogine I., Modern Thermodynamics (Wiley) 1998.
[3] de Groot S. R. and Mazur P., Non-equilibrium thermodynamics (Dover) 1984.
[4] Fermi E., Thermodynamics (Dover) 1956.

[5] Van Kampen N. G., Stochastic Processes in Physics and Chemistry (Elsevier) 2007.
[6] Cover T. M. and Thomas J. A., Elements of Information Theory (Wiley) 2006.
[7] Esposito M., Lindenberg K. and Van den Broeck C., EPL, 85 (2009) 60010.
[8] Van den Broeck C., Phys. Rev. Lett., 95 (2005) 190602.
[9] Esposito M., Kawai R., Lindenberg K. and Van den Broeck C., Phys. Rev. Lett.,
102 (2009) 130602.
[10] Landauer R., IBM J. Res. Dev., 5 (1961) 183.
[11] Esposito M. and Van den Broeck C., EPL, 95 (2011) 40004.
[12] Van den Broeck C., Nat. Phys., 6 (2010) 937.
[13] Esposito M., Kumar N., Lindenberg K. and Van den Broeck C., Phys. Rev. E, 85
(2012) 031117.
[14] Cleuren B., Rutten B. and Van den Broeck C., Phys. Rev. Lett., 108 (2012) 120603.
[15] Rutten B., Esposito M. and Cleuren B., Phys. Rev. B, 80 (2009) 235122.
[16] Cleuren B., Rutten B. and Van den Broeck C., Phys. Rev. Lett., 108 (2012) 120603.
[17] Esposito M. and Schaller G., EPL, (2012) .
[18] Gaspard P., J. Stat. Phys., 117 (2004) 599.
[19] Luo J. L., Van den Broeck C. and Nicolis G., Z. Phys. B, 56 (1984) 165.
[20] Oono Y. and Paniconi M., Prog. Theor. Phys. Suppl., 130 (1998) 29.
[21] Sekimoto K., Prog. Theor. Phys. Suppl., 130 (1998) 17.
[22] Parrondo J. M. R. and Espagnol P., Am. J. Phys., 64 (1996) 1125.
[23] Hatano T. and Sasa S. I., Phys. Rev. Lett., 86 (2001) 3463.
[24] Klein M. J. and Meijer P. H. E., Phys. Rev., 96 (1954) 250.
[25] Schnakenberg, Rev. Mod. Phys., 48 (1976) 571.
[26] Van den Broeck C., Selforganization by Nonlinear Irreversible Processes (Springer) 1986.
[27] Van den Broeck C. and Esposito M., Phys. Rev. E, 82 (2010) 011144.
[28] Sekimoto K., Stochastic Energetics (Springer) 2010.
[29] Seifert U., Eur. Phys. J. B, 64 (2008) 423.
[30] Seifert U., Phys. Rev. Lett., 95 (2005) 040602.
[31] Crooks G. E., Excursions in statistical dynamics, PhD thesis, University of California,
Berkeley (1999).
[32] Jarzynski C., Phys. Rev. Lett., 78 (1997) 2690.
[33] Jarzynski C., J. Stat. Phys., 96 (1999) 415.
[34] Lebowitz J. L. and Spohn H., J. Stat. Phys., 95 (1999) 333.
[35] Peliti L., Phys. Rev. Lett., 101 (2008) 098903.
[36] Crooks G. E., Phys. Rev. E , 61 (2000) 2361.
[37] Bustamante C., Liphardt J. and Ritort F., Phys. Today, 58 (2005) 43.
[38] Cleuren B., Van den Broeck C. and Kawai R., Phys. Rev. Lett., 96 (2006) 050601.
[39] Cleuren B. and Van den Broeck C., EPL, 79 (2007) 30001.
[40] Bochkov G. N. and Kuzovlev Y. E., Physica A, 106 (1981) 443.
[41] Evans D. J., Cohen E. G. D. and Morriss G. P., Phys. Rev. Lett., 71 (1993) 2401.
[42] Gallavotti G. and Cohen E. G. D., Phys. Rev. Lett., 74 (1995) 2694.
[43] Maes C., J. Stat. Phys., 95 (1999) 367.
[44] Kurchan J., J. Phys. A, 31 (1998) 3719.
[45] Ritort F., Séminaire Poincaré, 2 (2003) 195.
[46] Kawai R., Parrondo J. M. R. and Van den Broeck C., Phys. Rev. Lett., 98 (2007)
080602.
[47] Esposito M. and Van den Broeck C., Phys. Rev. Lett., 104 (2010) 090601.
[48] Van den Broeck C., J. Stat. Phys., (2010) P10009.
[49] Chernyak V. Y., Chertkov M. and Jarzynski C., J. Stat. Mech., (2006) P08001.
[50] Harris R. J. and Schutz G. M., J. Stat. Mech., (2007) P07020.

[51] Hummer G. and Szabo A., Proc. Natl. Acad. Sci. U.S.A., 98 (2001) 3658.
[52] Andrieux D. and Gaspard P., J. Stat. Phys., 127 (2007) 107.
[53] Prost J., Joanny J. F. and Parrondo J. M. R., Phys. Rev. Lett., 103 (2009) 090601.
[54] Baiesi M., Maes C. and Wynants B., Phys. Rev. Lett., 103 (2009) 010602.
[55] Seifert U. and Speck T., EPL, 89 (2010) 10007.
[56] Verley G., Chetrite R. and Lacoste D., Phys. Rev. Lett., 108 (2012) 120601.
[57] Bodineau Y. and Derrida B., J. Stat. Phys., 123 (2006) 277.
[58] Speck T., Blickle V., Bechinger C. and Seifert U., EPL, 79 (2007) 30002.
[59] Toyabe S., Sagawa T., Ueda M., Muneyuki E. and Sano M., Nat. Phys., 6 (2010)
988.
[60] Bérut A., Arakelyan A., Petrosyan A., Ciliberto S., Dillenschneider R. and
Lutz E., Nature, 483 (2012) 187.
[61] Bechinger C. and Blickle V., Nat. Phys., 8 (2012) 143.
[62] Maruyama K., Nori F. and Vedral V., Rev. Mod. Phys., 81 (2009) 1.
[63] Esposito M., Harbola U. and Mukamel S., Rev. Mod. Phys., 81 (2009) 1665.
[64] Campisi M., Hanggi P. and Talkner P., Rev. Mod. Phys., 83 (2011) 771.

Stochastic Thermodynamics

Uploaded by

Copyright:

Available Formats

Stochastic Thermodynamics

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Stochastic Thermodynamics

Uploaded by

Copyright:

Available Formats

Proceedings of the International School of Physics “Enrico Fermi”

Stochastic thermodynamics: A brief introduction

Summary. — The main purpose of statistical mechanics is to give a microscopic

or, for small changes

needed to describe what is happening. It nevertheless gives a general prescription for

(4) dSr = dEr /T = dQr /T.

(5) dSr = dQr /T = (dEr − μdNr )/T.

(6) ΔStot = ΔS + ΔSr ≥ 0,

The equality sign is reached for a reversible process.

(13) m = exp [−β(m − F )] ,

resulting in a change in occupation probability. In case the number of particles also

Furthermore, the interpretation is compatible with thermodynamics for a quasi-static

in agreement with the usual thermodynamic formula.

we ﬁnd for the full derivative

It easily follows that the following Shannon entropy

equation, which we write in a number of diﬀerent forms:

In the last line, we introduced the “diagonal” element to compactify notation:

This property guarantees the conservation of probability. Formulated in an alternative

We will assume for simplicity that this distribution is unique(14 ).

2. – Ensemble stochastic thermodynamics

energy and number of particles are given by

Note that both m and pm can be time-dependent while nm is time-independent. When

with the work and heat (plus chemical work) ﬂux

(48) Jm,m = Wm,m pm − Wm ,m pm .

(49) m,m = m − m ,

(50) nm,m = nm − nm .

(ν) (ν) (ν)

As equilibrium distributions, we consider the canonical case (heat reservoir)

and the grand-canonical situation (heat plus particle reservoir)

The corresponding chemical work rate and heat ﬂux read

(ν) 1 (ν) (ν)

(65) = μ(ν) Ṅ (ν) ,

(67) = Ė (ν) − μ(ν) Ṅ (ν) ,

We can thus write(16 )

(76) Ṡ = Ṡi + Ṡe

Fig. 1. – Schematic representation of a system in contact with two reservoirs.

asides from chemical work) we have Ė = Ṅ = Ṡ = Ẇ = 0. If the reservoirs have diﬀerent

Furthermore, we ﬁnd from

(90) Ṡ = Ṡi + Ṡe = 0

(92) Ṡi = −Ṡe = JE XE + JN XN ≥ 0,

with the usual expression of the thermodynamic forces

(95) JE = LEE XE + LEN XN ,

with the “collapsed” single thermodynamic force

This situation is of special interest because reversibility, corresponding to zero entropy

(102) Ẇchem = μ(1) Ṅ (1) + μ(2) Ṅ (2) = −Q̇(1) − Q̇(2)

The “collapsed thermodynamic” force and entropy production read

ηC η2 7 + csch2 (a0 /2) 3

3. – Ensemble stochastic thermodynamics: more

We now introduce the non-equilibrium free energy

Combination with the previous formulas leads in a one-line calculation to

which is the “instantaneous” generalization of eq. (116). To make a clearer comparison,

(ν) (ν) (ν)

(128) Ṡi = Ṡa + Ṡna ≥ 0,

4. – Trajectory stochastic thermodynamics

Fig. 4. – Schematic representation of the ﬁrst law along a trajectory.

(132) e = eΠ (t) = m(t) (t).

Δe between any initial and ﬁnal times ti and tf is given by

(133) Δe = ef − ei = mf (tf ) − mi (ti ).

(135) w = [m1 (t2,1 ) − m1 (ti )] + [m2 (t3,2 ) − m2 (t2,1 )]

(13) m = exp [−β(m − F )] ,

Note that both m and pm can be time-dependent while nm is time-independent. When

(49) m,m = m − m ,

(132) e = eΠ (t) = m(t) (t).

(133) Δe = ef − ei = mf (tf ) − mi (ti ).

(135) w = [m1 (t2,1 ) − m1 (ti )] + [m2 (t3,2 ) − m2 (t2,1 )]