Statistical Mechanics Notes

Leonard Susskind’s Lectures

Lecture 1

Entropy

Definition of entropy over a uniform probability distribution:

S = log M (1)

where M is the number of occupied states. Entropy is conserved, but since it’s hard to keep track
of the system, entropy increases due to our ignorance. Entropy depends on characteristics of the
system and our knowledge about the system.

Maximum entropy is if all states have equal probability:

S = log N (2)

where N is total number of states (i.e. all states are occupied).

Liouville’s Theorem: volume in phase space is conserved over evolution (i.e. number of states
is conserved); uniform probability distribution over states is conserved.

First Law of Thermodynamics: energy is conserved in a closed system

dE
=0 (3)
dt

Entropy for a general probability distribution:

P
S=− i p(i) log p(i) (4)

Entropy in a continuous system is equal to the log of the volume in phase space for occupied volume
with nonzero probability distribution. The probability density integrates to 1:
Z
P (p, q) = 1 (5)

and the general continuous entropy is

Z
S = − P (p, q) log P (p, q)dpdq (6)
Lecture 2

Temperature is the amount of energy needed to change entropy by 1 bit.

dE
T = dS (7)

Entropy and energy are usually monotonically increasing functions of each other.

Second Law of Thermodynamics: Entropy increases as a system goes to equilibrium.

P (i, E) is a probability distribution over states i parameterized by energy E

X
P (i, E) = 1 (8)
i
X
hEi = P (i, E)Ei (9)
i

This is a one-parameter family and higher average energies have broader distributions.

Lecture 3

Ground state: the probability is zero for all states except the state of lowest energy. Entropy is
zero.

Entropy is a measure of the number of states that are important to a distribution so a wider
distribution has a higher entropy.

Thermal Equilibrium: event has already happened, system is stable, heat doesn’t flow.

Second Law of Thermodynamics says that as a system comes to equilibrium, the probability
distribution broadens and entropy increases.

Zeroth Law of Thermodynamics:

• There is a notion of temperature.

• Energy flows from higher temperature to lower temperature.
• In thermal equilibrium, the temperature of all parts of the system is the same.

As energy flows until equilibrium, energy is conserved:

dEa + dEb = 0 (10)
and entropy increases:
dSa + dSb > 0 (11)

The temperature defines the direction of energy/heat flow.

2
Occupation Numbers

System in contact with a heat bath or, alternatively, N identical connected systems.

Figure 1: N identical connected systems.

Occupation number: the number of systems in a particular state. ni denotes the number of
systems in state i.

Constraints on the system:

X
ni = N (12)
i
X
ni E i = N E (13)
i

where E is the average energy per subsystem.

Probability of being in state i:

ni
P (i) = (14)
N

For a given set of occupation numbers the number of arrangements of subsystems is

N!
Q (15)
i ni

without yet considering the constraints.

Stirling’s Approximation:
N ! ≈ e−N N N (16)

3
Lecture 4

Deriving Boltzmann Distribution

The most likely set of occupation numbers ni maximizes entropy subject to constraints
X
Pi = 1 (17)
i
X
Pi Ei = hEi := E (18)
i

Our goal is to minimize X

−S(Pi ) = Pi log Pi = F (19)
i
using Lagrange Multipliers.
hX i hX i
F 0 (p) = F + α Pi − 1 + β Pi Ei − E (20)

Take the derivative of F 0 (p) with respect to an individual pi and set it equal to 0:
∂F 0
= log pi + 1 + α + βEi = 0 (21)
∂pi
Solve for pi to obtain the Boltzmann distribution which maximizes entropy under the con-
straints:
pi = Z1 e−βEi (22)
Use constraint 17 to obtain an expression for Z in terms of β.
X
Pi = 1 (23)
i
1 X −βEi
e =1 (24)
Z
i

−βEi
P
Z(β) = ie (25)
Use constraint 18 to obtain an expression for E in terms of β.
X
E= Pi Ei (26)
i
X 1
= e−βEi (27)
Z
i
1 ∂Z
=− (28)
Z ∂β
∂
E(β) = − ∂β log Z (29)

Note that β and E are functions of each other.

4
Entropy

X
S=− pi log pi (30)
i
X 1
= e−βEi [βEi + log Z] (31)
Z
i

S = βE + log Z (32)

β as Inverse Temperature

The definition of temperature is

dE
T = (33)
dS
Now we can take the derivative of entropy S (32)with respect to energy E.
∂ log Z
dS = βdE + Edβ + ∂β (34)
∂β
Since
∂ log Z
E=− (35)
∂β
we have
dS = βdE (36)
so
1
β= T (37)

Ideal Gas

Ideal gas: molecules are noninteracting point particles inside a box.

States are defined by the position and momentum for each particle.

Kinetic Energy The kinetic energy for component i is

1 2
Ei = p (38)
2m i
for momentum pi

The partition function is Z +∞ P3N 2

Z= dx3N dp3N e−β i=1 (pn /2m) (39)
−∞
First,
VN
Z
dx3N = (40)
N!

5
where the N ! term is to account for overcounting interchangeable states.

The second part is a Gaussian integral:

Z Z
β 2 2
− 2m
dpd p
= dpe−q (41)

for
β 2
q2 = − p (42)
2m
So Z r Z r
−q 2 2m −q 2 2mπ
dpe = dqe = (43)
β β
So the partition function for an ideal gas is

V N  2mπ  3N
2
Z= (44)
N! β
We can use Stirling’s approximation to get

 N   3N
e 2mπ 2
Z= ρ β (45)

N
since ρ = V . Find the energy:
∂
E=− log Z (46)
∂β
3N
E= 2 T (47)

And average energy is

E
N = 23 T (48)

i.e. the particles move around with kinetic energy proportional to temperature.

Lecture 5

Helmholtz Free Energy

A = E − T S = −T log Z (49)
is derived from the equation for entropy
E
S= + log Z (50)
T

Control parameters are macroscopic parameters that an experimenter can change.

Conjugate pairs consist of a control parameter and a conjugate thermodynamical variable. For
example, you can change volume and the response is a change in pressure.

6
 • Independent variables: T, V

• Dependent variables: E, S

Theorem:
∂E  ∂E  ∂E  ∂S 
= − (51)
∂V S ∂V T ∂S V ∂V T
   

To prove this, consider a contour of fixed entropy on the plot of entropy and volume and the fact
that
∂E  ∂E 
∂E = + (52)
∂V T ∂T V
 

Entropy is a monotonic function of temperature.

Pressure

Adiabatic: slow motion and no heat entering the system.

Pressure: force per unit area.

Adiabatic Theorem: If you change the system (i.e. volume) slowly, energy level (but not nec-
essarily the energy value) stays the same: it doesn’t jump to another energy level. The Pi ’s are
conserved so entropy stays the same.

Adiabatic = isentropic

Figure 2: A box with base area A. The piston moves with pressure in the box.

7
Based on figure 2,
dE = −P Adx = −P dV (53)
or
∂E 
 = −P (54)
∂V S

Since it’s easier to calculate how things vary with temperature, use the theorem (equation 51) and
the fact that ∂E
∂S V = T to show that

P =TN
V = ρT (55)

Fluctuations/Variance

Variance
(X − hXi)2 = hX 2 i − hXi2


(56)

Fluctuations in Energy

1 ∂2Z 1  ∂Z 2
h∆Ei2 = − (57)
Z ∂β 2 Z 2 ∂β
∂
= hEiT 2 (58)
∂T
∂
where ∂T hEi is the specific heat or heat capacity CV for constant volume.

h∆Ei2 = CV T 2 kB (59)

Lecture 6

Partition Function of Weakly Interacting Gas

The energy of a weakly interacting gas consists of the kinetic energies of the individual particles
and the potential energy of the pairwise interactions:
X p2 X
E= + U (|xn − xm |) (60)
n
2M n>m

where U (x) < kinetic energy.

For use later, find the integral of potential energy between two particles over all possible positions:
Z
dx31 dx32 U (|x1 − x2 |) (61)

8
Hold particle 1 fixed and integrate over all possible positions of particle 2:
Z
dx3 U (|x|) = U0 (62)

Integrating over the position of particle 1 gives a factor of V . Thus the integral equals V U0 ; it is a
factor of potential energy and volume. Assume that U goes to zero at large distances.

Return to the energy of a weakly interacting gas and write the partition function:
Z
1 p2
Z = dpdx e−β 2M e−βU (x) (63)
N!
V N −β p2 e−βU (x)
Z Z
= dp e 2M dx (64)
N! VN
After separating the partition function into two integrals, the first integral is just the partition
function for the ideal gas, denoted as Z0 .

Taylor expand the potential energy terms:

Z Z
dx −βU (x) dx
N
e ≈ (1 − βU (x)) (65)
V VN
Z
dx
=1−β U (x) (66)
VN
Z
dx X
=1−β U (|xn − xm |) (67)
V N n>m
βN (N − 1)
Z
dx1 dx2
=1− U (|x1 − x2 |)dx3 dx4 ... (68)
2 VN
βN (N − 1)
Z
dx1 dx2
=1− U (|x1 − x2 |)V N −2 (69)
2 VN
βN 2 V N −2
=1− V U0 (70)
2 VN
βN 2
=1− U0 (71)
2V

Now the partition function is !

βN 2
Z = Z0 1− U0 (72)
2V
and the log partition function is
!
βN 2
log Z = log Z0 + log 1 − U0 (73)
2V

Use the following Taylor series expansion

log(1 − x) ≈ −x (74)
to rewrite the log partition function
βN 2
log Z = log Z0 − U0 (75)
2V

9
Properties of weakly interacting gas

Use this approximation of the log partition function to find various quantities.

Energy
−∂ log Z
E= (76)
∂β
3 N2
= NT + U0 (77)
2 2V
3 ρ
= N T + N U0 (78)
2 2

Energy per particle

E 3 ρ
= T + U0 (79)
N 2 2

Pressure Pressure is 
−∂A 
P = (80)
∂V 

T
where
A = −T log Z (81)

−∂A 
P = (82)
∂V 

T
∂ log Z
=T (83)
∂V
N2
= ρT + U0 (84)
2V
1
= ρT + ρ2 U0 (85)
2
where ρT is the pressure of an ideal gas. For small ρ, the second term 12 ρ2 U0 vanishes. This is a
good approximation when potential energy per particle is much smaller than the kinetic energy:
ρU0 << T (86)

U0 is the potential energy between pairs of particles. Repulsion (U0 > 0) increases pressure.
Attraction (U0 < 0) decreases pressure.

10
Mathematical Interlude: Exact and Nonexact Differentials

Consider a function of x and y, F (x, y).

∂F ∂F
dF = dx + dy (87)
∂x ∂y
= Fx dx + Fy dy (88)

If Fx and Fy are derivatives of a unique function, then

∂Fx ∂Fy
− =0 (89)
∂y ∂x
i.e. the curl is zero. This is because the order of partial derivatives doesn’t matter.

Move along F : Z
∆F = Fx dx + Fy dy (90)

If a line integral around a loop is zero, that is equivalent to the curl being zero.

Exact: the curl of the effective force is zero.

Nonexact: the curl of the effective force is nonzero.

For example, Fx = y and Fy = −x are not the partial derivatives of any actual function, so the
curl is not 0. However, these could still be valid rules for how x and y change.

Heat and Work

Any two quantities, such as entropy and volume, determine the state of a gas.

Adiabatic: a process that doesn’t change the energy of a system.

Change in energy holding entropy fixed:

dE = −P dV (91)

11
Change in energy holding volume fixed and changing temperature:

dE = T dS (92)

Change both volume and entropy:

dE = −P dV + T dS (93)

dE = dW + dQ (94)
where dW is work and dQ is heat.

dW and dQ are not exact: if you follow a closed loop, dE = 0 but dW and dQ are not equal to
zero. Therefore the amount of heat put in is not a good description of the system.

Lecture 7

Speed of Sound

Consider the speed of sound in a gas: how fast do molecules in an overdense region spread?

The kinetic energy of a molecule:

3 1
kB T = mv 2 (95)
2 2
3kB T
v2 = (96)
m

The formula for the speed of sound:

∂P 2
c (97)
m∂ρ
∂P
where m∂ρ is called the mass density.

Pressure is
P = ρkB T (98)

Use this to obtain the value of the velocity in equation (96).

∂P kB T
= = L2 (99)
m∂ρ m
√
This approximation is only off by a factor of 3.

This equation for speed of sound tells us that in the range of an ideal gas, the speed doesn’t depend
on pressure or density.

12
Harmonic Oscillation

When you perturb almost anything from its ground state, it will oscillate.

Consider the statistical mechanics of an oscillator in a heat bath. The Hamiltonian is

p2 kx2
H= + (100)
2m 2

The partition function: Z Z

p2 kx2
−β 2m
Z= dpe dxe−β 2 (101)

Ignore numerical constants because they disappear when you take the derivative of log Z.

Perform the substitution

p2
β = q2 (102)
2m
so r
2m
p= q (103)
β

∞
r Z Z
2m 2 kx2
Z= dqe−q dxe−β 2 (104)
β −∞

Where the Gaussian integral can be evaluated as:

r Z r
2m −q 2 2mπ
dqe = (105)
β β

Now perform the substitution

kx2
y2 = β (106)
2

so

r
2
x= (107)
βk

13
which gives us another Gaussian integral
r Z r
2 −y 2 2π
dye = (108)
βk βk

and thus the partition function is

r r
2mπ 2π
Z= (109)
β βk
r
m1
= 2π (110)
k β
where r
m 1
= (111)
k ω
is the inverse frequency.

Find the energy:

log Z = C − log β (112)

−∂ log Z 1
E= = =T (113)
∂β β

For each Gaussian integral, there is a factor of 12 T in the energy.

The average kinetic energy = average potential energy = 12 T .

Quantum Mechanical Oscillator

The energy of a quantum mechanical oscillator is

n~ω (114)

which is n units of Planck’s constant ~ times the frequency ω. Since this is a quantum mechanical
system, there are discrete energy values which are evenly spaced.

The partition function is

X
Z= e−βn~ω (115)
n
X n
= e−β~ω (116)
n
1
= (117)
1 − e−β~ω

14
where we get (117) because of the geometric sum:
X 1
xn = (118)
n
1−x

Find the energy:

−∂ log Z
E= (119)
∂β
1 ∂Z
=− (120)
Z ∂β
~ωe−β~ω
= (121)
1 − e−β~ω

At high temperature, quantum systems become classical. They have lots of energy and the quan-
tization becomes unimportant:
~ωe−β~ω 1
E= −β~ω
≈ =T (122)
1−e β
using the approximation
ex ≈ 1 + x (123)

At low temperature:
E = ~ωe−β~ω (124)

Quantum mechanics suppresses the energy of an oscillator when temperature is low enough.

The crossover/transition from quantum to classical behavior occurs when β~ω = 1 or when ~ω = T :

• ~ω > T : quantum

• ~ω < T : classical

The stiffness of the oscillator is also a factor since

r
k
ω= (125)
m

Second Law of Thermodynamics

• There is an apparent conflict between the reversibility of classical mechanics and the second
law saying that entropy is increasing.

• Coarse-graining: replacing points in phase space by blobs causes the volume to increase
and thus entropy increases due to limits of resolution. Ideally you could follow particles to
infinite resolution, but this isn’t possible.

15
 • As systems evolve in phase space, they grow in a chaotic, fractal-like manner.

• Over time, the trajectory of a single starting point will come close to every point in phase
space. You will never return to the same starting point with infinite precision, but you can
come -close in finite time.

Lecture 8

Poincare Recurrence: If you wait long enough, all of the air molecules will be in one half of the
room. For N particles, the probability of this happening is 21N .

The volume of phase space for N particles in a box with volume V is proportional to V N . Entropy
is the logarithm of the volume of phase space so V N is proportional to the exponent of entropy.
The probability of all particles being in a smaller volume v is

vN
∝ e−S (126)
VN

Magnets

The simplest magnet is a single particle that has spin up or down. Energy is lower if adjacent spins
are parallel so magnetized groups of parallel spins appear.

Ferromagnetic transition occurs when energy lowers until the entire magnet is basically aligned.

At zero temperature, only the lowest energy state matters.

N non-interacting magnets with spin up or down and magnetic moment µ and magnetic field H:

• σ = +1: spin up

• σ = −1: spin down

• energy for a single particle is µH

• n = up spins

• m = down spins

The total energy of this system is

E = (n − m)µH (127)

N!
The number of states with a particular n and m is n!m! .

16
Partition function for this system:
X N!
Z= e−βµH(n−m) (128)
n
n!m!
= (e−βµH + eβµH )N (129)
= 2N coshN (βµH) (130)
where to get 129 set
x = e−βµH (131)
y = eβµH (132)
and then use the binomial theorem
X N!
xn y n = (x + y)N (133)
n
n!m!

The magnetization per particle is the probability of a given spin being up or down:
n−m
M= (134)
N

The total energy is

E = N M µH (135)

Use the partition function to get the energy:

−∂ log Z
E= = −N tanh(βµH)µH (136)
∂β
and also the magnetization:
M = − tanh(βµH) (137)

Ising Model

For the Ising model with no external field, energy is based on pairs of neighboring spins. Aligned
spins have lower energy than antialigned spins.

Energy of one-dimensional Ising model:

X
E = −J σ(n)σ(n + 1) (138)
n

At zero temperature, the Boltzmann distribution infinitely favors the lowest energy state. Spon-
taneous symmetry breaking occurs when even a small external magnetic field on one particle
will influence the lowest energy state of the model.

Symmetry: perform an operation on all coordinates that doesn’t change the energy. Embodied
here through lack of a bias between up and down. Two adjacent up spins have the same energy as
two adjacent down spins.

17
Lecture 9

Energy of a particle in the Ising model:

E = −Jσ (139)

Think of a subsystem as a small system plus a heat bath. Take as the subsystem one spin and
assume that it is in thermal equilibrium with the rest of the environment.

Partition function: X
Z= eβσ = eβJ + e−βJ = 2 cosh βJ (140)
σ=±1

Energy:
E = −J tanh βJ (141)

Average spin:
hσi = tanh βJ (142)

One Dimensional Ising Model

Energy: X
E = −J σi (143)
i

Partition Function: X P
Z= e−Jβ i σi σi+1
(144)
σi

Correlation function: Given that we know a certain spin is up, what is the conditional probability
that a spin at distance n is up; what is the average of the product of spins at two different locations?

Consider the values of bonds µ which indicates parallel or antiparallel neighbors rather than the
values of the spins themselves.

Energy: X
E = −J µi (145)
i

Partition function: X P
Z=2 e− i Jβµi
= 2(2 cosh βJ)N −1 (146)
µ

Average bond:
hµi = hσi σi+1 i = tanh βJ (147)

18
Find the correlation:

hσi σi+n i = hσi σi+1 σi+1 σi+2 σi+2 ...σi+n i = hµ1 µ2 ...µn−1 i (148)

This expression factors completely since µs are independent:

hµ1 µ2 ...µn−1 i = (tanh βJ)n−1 (149)

Think of this as each step losing a factor of tanh βJ.

The equivalence between two different systems is called a duality. Here we have a duality between
uncoupled spins and spins that are coupled with their neighbors.

The one-dimensional Ising model doesn’t display a phase transition but all higher dimension models
do.

Higher dimension Ising models

Mean field approximation: for d-dimensional Ising model where each spin has 2d neighbors
X
E = −Jσ σ (150)
N (i)

Replace the sum over neighbors with 2dσ̄ where σ̄ is the average spin. For large numbers of
neighbors there will only be small fluctuations from the average. Thus energy is

E = −2dJσσ̄ (151)

where σ is the energy of the one particular spin. Now look at the average of one particular spin:

σ̄ = tanh[2βdJ σ̄] (152)

This is also called the self-consistent field approximation.

Impose an external magnetic field B: Energy:

E = −2dJσσ̄ + Bσ = σ[−2dJ σ̄ + B] (153)

Spin average:
σ̄ = tanh[(2βdJ)σ̄ + βB] (154)

Lecture 10

Identify the phase transition of the Ising model by finding the solutions to
yT
= tanh(y − βh) (155)
2dJ

19
where
y = 2βdJ σ̄ (156)
and b is the external magnetic field. This is a change of variable from

σ̄ = tanh(2βdJ σ̄ + βh) (157)

A phase transition occurs at

Tcrit = 2dJ (158)

• Below Tcrit , spontaneous magnetization.

• Above Tcrit , random fluctuations and no magnetization.

Chemical potential: amount of energy it takes to move a particle from a low energy region to a
high energy one. Changing chemical potential can change the density.

The configuration of the system includes the number of particles at a given chemical potential.
Over time, the number of particles with chemical potential µ can change.

Partition function: X
Z= e−βE+N µβ (159)
p,x,N

Liquid-gas phase transition

Keep temperature constant and vary chemical potential until density has changed sufficiently (be-
tween gas and liquid)

• The transition between liquid and vapor is a sudden discontinuity below Tcrit and a smooth
transition above.

Hard-core lattice gas model

Molecular interactions have hard core repulsion and short-range attractions.

Use a lattice to model liquid-gas transition:

• σ = −1: no particle at the site

• σ = +1: particle at the site

Ising model energy: X

E= −Jσi σj (160)
links

20
Changes in energy:

• If all sites start at -1 and one particle is added, energy increases by 8J.

• If two distant particles are added, energy increases by 16J.

• If two close particles that share a bond are added, energy increases by 12J. This models the
short range attractive energy.

Here, the chemical potential is the energy in having a particle.

Energy with external field: X X

E= −Jσi σj + hσ (161)
links sites

The density of particles is

1 + σ̄
ρ= (162)
2

Critical exponents describe properties of systems near critical point/temperature:

(T − Tcrit )γ (163)

These vary for different types of phase transitions and describe classes of systems.

21