334 Chem.

Educator 2002, 7, 334–338

Information Entropy and Uncertainty Relations

Henry E. Montgomery, Jr.

Centre College, Danville, KY 40422-1394, [email protected]

Received July 17, 2002. Accepted September 25, 2002.

Abstract: Information entropy is introduced as a measure of quantum mechanical uncertainty. An uncertainty

relation based on information entropy is obtained as an alternative to the Heisenberg inequality. In two typical
examples, the entropic uncertainty relation is shown to be bounded in situations where the Heisenberg inequality
diverges or grows too large to be useful.

The Heisenberg uncertainty principle [1, 2] introduces the

idea that the probability distributions for the one-dimensional 12
(
∆Ω =  ∫ψ ∗ Ω ) ψ dτ 
2
momentum, px, and position, x, of a quantum mechanical ˆ − Ω
particle cannot be arbitrarily localized. This is usually 
expressed as the Heisenberg inequality
For one-dimensional momentum ( pˆ x = −i! ∂ ∂x ) and position
! ( xˆ = x ), the commutator [ pˆ x , xˆ ] = −i! and the Robertson
∆p x ∆x ≥ (1)
2 inequality reduces to the Heisenberg inequality.
If [ Aˆ , Bˆ ] = k where k is a constant, then the right-hand side
Although Heisenberg did not give general definitions for the
of the Heisenberg inequality is k/2 and a knowledge of ∆A
uncertainties ∆px and ∆x, they are usually associated with the
standard deviation of a set of measurements of the position and gives an estimated lower bound for ∆B using
momentum.
In this paper we discuss some limitations of the Heisenberg k
∆B ≥
inequality that are a consequence of using the standard 2 ∆A
deviation as a measure of uncertainty. We show how a
definition of uncertainty based on the definition of entropy However, if [ Aˆ , Bˆ ] = Cˆ where Cˆ is another operator, then
from information theory leads to an alternative uncertainty
relation. We then use the entropic uncertainty relation to we must know ψ in order to evaluate the right-hand side of eq
discuss two model problems specifically chosen to illustrate 2. But if we know ψ, ∆B can be calculated directly. If the
the shortcomings of the Heisenberg inequality. right-hand side of eq 2 were always a constant, the resulting
inequality would be stronger and thus more useful.
Uncertainty relations The derivation of the Robertson inequality is based on
defining uncertainty as a measure of the distribution of values
For a quantum mechanical system described by a normalized about their average value. This definition is consistent with the
ˆ is
wavefunction ψ, 〈Ω〉, the average value of the operator Ω usual method of determining experimental uncertainty. It is
given by most appropriate when the distribution of values is near-
Gaussian. If the distribution has more than one peak, the
standard deviation is not a good measure of uncertainty. As
Ω = ∫ψ ∗Ω
ˆψ dτ
discussed by Hilgevoord [4], even if there is only one peak, the
standard deviation may be a poor estimate of uncertainty if the
where the integration is over the entire coordinate space. The distribution deviates significantly from Gaussian. It is
standard approach to uncertainty relations is expressed by the reasonable to investigate measures of uncertainty other than
Robertson inequality [3] the standard deviation and see if they lead to other uncertainty
relations.
Intuitively, we equate uncertainty with a lack of
1
( ∆A)( ∆B ) ≥ ∫ψ ∗[ Aˆ , Bˆ ]ψ dτ (2) information. It should come as no surprise that information
2 theory provides a way to measure uncertainty. Information
theory has its primary roots in two classic papers written by
where  and B̂ are Hermitian operators and their commutator Claude Shannon in 1948 [5]. Shannon’s purpose was to
[ Aˆ , Bˆ ] = AB
ˆ ˆ − BA
ˆ ˆ . ∆Ω is the standard deviation of the develop a mathematical theory to quantitatively analyze the
passage of information from a source, through an information
operator Ω̂ , given by channel to a receiver. It has subsequently been applied to areas
ranging from calculation of the ability of a material to be

© 2002 Springer-Verlag New York, Inc., S1430-4171(02)06618-7, Published on Web 10/18/2002, 10.1007/s00897020618a, 760334hm.pdf
Information Entropy and Uncertainty Relations Chem. Educator, Vol. 7, No. X, 2002 335

penetrated by charged particles [6] to analysis of binding sites ∞

1  −ipx 
on nucleotide sequences [7]. φ ( p) = ∫ exp  ψ ( x )dx (7)
If we have a message composed of n signals, each of which 2π! −∞  ! 
occurs with a probability pi,, Shannon defined the information
associated with this discrete probability distribution as With ψ(x) we can associate a position-space density
∗
distribution ρ ( x ) = ψ ( x ) ψ ( x ) and with φ(p) we can
n
H = − ∑ pi ln ( pi ) (3) associate a momentum-space density distribution
∗
i =1
ρ ( p ) = φ ( p ) φ ( p ) . Using eq 4 we calculate the position-
By analogy to Boltzmann’s formulation of entropy in space and momentum-space information entropies as
statistical mechanics, Shannon called H the entropy of the
∞
signal distribution. The story is told that John von Neumann
advised Shannon to use the term entropy because “no one S x = − ∫ ρ ( x ) ln ρ ( x ) dx (8)
−∞
knows what entropy really is, so in a debate you will always
have the advantage” [8]. This probabilistic or information
entropy measures the spread and sharpness of a probability and
distribution, independent of its actual values. For a continuous
variable t and an associated probability density distribution ∞

ρ(t) normalized so that S p = − ∫ ρ ( p ) ln ρ ( p ) dp (9)

−∞
∞

∫ ρ (t )dt = 1 Bialynicki-Birula and Mycielski (BBM) [11] used Fourier

analysis to derive a relationship between Sx and Sp. Their
−∞
derivation is synopsized in the appendix. They showed that Sx
the sum in eq 3 becomes an integral and the information and Sp satisfy the relation
entropy of ρ is given by
S x + S p ≥ 1 + ln π ≅ 2.145 (10)
∞
St = − ∫ ρ ( t ) ln ρ ( t ) dt (4) We call this an entropic uncertainty relation.
−∞ Because the information entropy measures the localization
of a distribution, eq 10 places a limit on the simultaneous
For the reader who desires to learn more about information localization of the position and momentum distributions. If one
theory, there are numerous resources. Chapter 9 of Goldstein of the entropies becomes small, then the other must become
and Goldstein [9] discusses the relationship between entropy large enough to preserve the inequality. This is philosophically
and information at a level understandable by the nonscientist. consistent with the Heisenberg uncertainty principle. BBM
Pierce [10] provides a more technical discussion and discusses [11] also showed that the Heisenberg inequality could be
the applicability to a variety of areas including art and derived from their entropic uncertainty relation.
psychology. Shannon’s original papers [5] contain some The BBM entropic uncertainty relation has a constant lower
difficult areas, but most of his work is straightforward, bound and thus overcomes one of the limitations of the
particularly in his development of the concept of entropy. Heisenberg inequality. These entropic uncertainty relations
To better understand St, it will be helpful to consider a have recently received considerable interest in the literature.
Gaussian distribution with mean t = 0 and standard deviation The interested reader is referred to Majerník and Richterek
σ, [12], Yáñez et al. [13], Majerník and Majerníková [14] and
references therein.
1  −t 2 
ρ (t ) = exp  2  (5) Constant Wavefunction in Position Space
σ 2π  2σ 
We can illustrate one of the shortcomings of the Heisenberg
Performing the integration in eq 4, we obtain inequality by considering a quantum mechanical particle
described by the normalized, position-space wavefunction
1 ψ ( x ) = 1/ L on ћ the interval –L/2 ≤ x ≤ L/2 and ψ(x) = 0
St = 1 + ln ( 2π ) + ln (σ ) (6)
2  elsewhere. We will use units scaled such that ћ = m = 1. This
serves both to simplify the form of the equations and to
As σ increases, ρ(t) becomes less localized and St increases. emphasize that the entropy is a measure of the sharpness of the
This is consistent with the idea that decreased localization probability distribution, independent of dimension. Other ways
should result in increased entropy. In the case of a Gaussian of dealing with the issue of dimension are discussed in section
distribution, σ and St are seen to provide similar information. 4 of [12]. The conjugate momentum-space wavefunction can
For a normalized, one-dimensional, position-space be obtained from the Fourier transform, eq 7 as
wavefunction ψ(x), the conjugate momentum-space
wavefunction φ(p) is given by the Fourier transform

© 2002 Springer-Verlag New York, Inc., S1430-4171(02)06618-7, Published on Web 10/18/2002, 10.1007/s00897020618a, 760334hm.pdf
336 Chem. Educator, Vol. 7, No. X, 2002 Montgomery

2
 2 sin ( Lp 2 ) 
3.5 ∞

3.0
S x +S p ∆p x = ∫ p  2
 dp = ∞ (16)
S p −∞  πL p 
2.5

2.0 and ∆px∆x = ∞. The Heisenberg inequality is certainly

satisfied, but it is difficult to extract anything useful from the
1. 5
product ∆px∆x.
1. 0
S x Infinite Square Well
0.5

0.0 We consider a well of length L, centered at the origin, with a

1 2 3 4 5 6 7 8 9 10
n quantum mechanical particle confined to the interval –L/2 ≤ x
≤ L/2. We will again use units scaled such that ћ = m = 1. The
system is symmetric about x = 0 and the solutions to the
Figure 1. Information entropies for L= 2. Schrödinger equation divide into symmetric cosine functions
and asymmetric sine functions. Because the Fourier transform
of a sine function results in a complex-valued function, we will
2 sin ( Lp 2 )
L2 only consider the symmetric states
1 1
φ ( p) = exp ( −ipx )
2π − L∫ 2
dx = (11)
L πL p
2  ( 2n − 1) πx 
ψ n ( x) = cos   (17)
L  L 
The position-space information entropy is given by

L2 2 2 where n = 1,2,3,…. The conjugate momentum-space

 1   1 
Sx = − ∫  L  ln  L  dx = ln ( L ) (12) wavefunctions are given by
−L 2
n +1 ( 2n − 1) cos ( Lp 2 )
φn ( p ) = 2 ( −1) π L (18)
π 2 ( 2n − 1) − p 2 L2
2
consistent with the idea that the uncertainty in position
increases as the interval length increases. The momentum-
space information entropy results in the rather complicated
Calculation of the entropic uncertainties requires evaluation
integral
of integrals involving logarithms of trigonometric functions
2
  Lp  
2  ( 2n − 1) πx    2 2  ( 2n − 1) πx  
L2
2
 2 sin  2   2
∫   = ln ( 2 L ) − 1
  ln  2 sin ( Lp 2 )  dp (13)
∞ Sx =  cos    ln  cos 
Sp = ∫     −L 2 
L  L   L  L 
−∞ 
πL p   π L p 
(19)
 
Sánchez-Ruiz [15] showed that independent of the value of n. Sp can be evaluated using
numerical integration. Figure 1 shows the information
∞
sin 2 ( x )  sin 2 ( x )  entropies and their sums for L = 2 and increasing n. Sp appears
∫ dx = π (γ − 1)
 x 2 
ln  (14) to be approaching an asymptotic limit which is slightly greater
0
x2   than 2.6. Majerník et al. [16] have shown that the limiting
value is ≈2.6564. We see that Sx + Sp ranges from 2.2120 to
where γ is Euler’s constant γ = 0.5772. Thus Sp = ln(2π/L) + 3.0175 so the BBM inequality is satisfied.
2(1 – γ). Sp = 2.683 at L = 1 and decreases to 0.381 at L = 10 Because the position- and momentum-space wavefunctions
while Sx + Sp = ln(2π) + 2(1 – γ) = 2.683 ≥ 1 + ln π. are again symmetric about the origins of their respective
Because the position- and momentum-space wavefunctions spaces, p x = x = 0 . The standard deviations are readily
are symmetric about the origins of their respective spaces, 〈px〉
evaluated [17] as
= 〈x〉 = 0. Calculation of the position-space standard deviation
gives L/2
 2  ( 2n − 1) πx   2  2  ( 2n − 1) πx  
∆x = ∫ 
L
cos 
 L
 x 
  L
cos 
 L
  dx

L2 −L / 2  (20)
1 3
∆x = ∫ x 2 dx =
L 6
L (15)
=
3L
1−
6
−L 2
π 2 ( 2n − 1)
2
6

When we calculate the momentum-space standard deviation,

and
we find

© 2002 Springer-Verlag New York, Inc., S1430-4171(02)06618-7, Published on Web 10/18/2002, 10.1007/s00897020618a, 760334hm.pdf
Information Entropy and Uncertainty Relations Chem. Educator, Vol. 7, No. X, 2002 337

1
π 2 ( 2n − 1) − 6
30 2
∆p x ∆x = (22)
2 3
25
∆p x
20 independent of L. Figure 2 shows the standard deviations and
their product.
15
∆p x ∆x Although Sp and ∆p are both measures of momentum
uncertainty, as n increases they behave in strikingly different
10
ways. For n > 1 the momentum density distribution, ρn(p), has
5
two distinct peaks, symmetrically located above and below the
∆x mean. ρ2(p) is shown in Figure 3. Superimposed on the graph
0 is a Gaussian distribution having the same standard deviation
1 2 3 4 5
n 6 7 8 9 10
as ρ2(p). Given the difference between the momentum density
distribution and the Gaussian “fit” to it by the standard
deviation, it is not surprising that the standard deviation is not
Figure 2. Standard deviations for L = 2. a good measure of the momentum uncertainty. As n increases,
the separation between the peaks increases. Figure 4 shows
that the increase in standard deviation is linear with increasing
0 . 18
n. The uncertainty measured by the standard deviation is
0 . 16
ρ 2(p ) primarily a measure of the separation between the peaks of the
0 . 14
momentum density distribution.
0 . 12
The momentum-space information entropy measures the
0 . 10
area under the density distribution and primarily depends on
0.08 the area under the two main peaks. Thus, it is bounded and
0.06 well-behaved. Because Sx = ln(2L) – 1, we have a quantitative
0.04 relation between the position-space and momentum-space
0.02 uncertainties, independent of the value of n.
0.00
- 15 - 10 -5 0 5 10 15
Summary
p

We have presented a formulation of uncertainty based on

Figure 3. ρ2(p), the momentum-space distribution for n = 2, L = 2, information entropy and shown how entropic uncertainty
and a Gaussian distribution having the same standard deviation. relations are bounded in situations where the Heisenberg
inequality is not bounded. If we consider ∆px∆x and Sx + Sp to
30
measure orbits in their respective phase spaces, the former are
sometimes not bounded whereas the latter are bounded.
25 Further, the entropic uncertainty relations always have a finite
lower bound.
20 The only case of which the author is aware where the
Heisenberg inequality is bounded and the entropic uncertainty
∆px

15 relation is unbounded is a distribution composed of separated

Dirac delta functions. Everett [18] pointed out that such a
10
distribution satisfies the Heisenberg inequality but results in
infinite information entropy. For typical quantum chemical
5
distributions, such as atomic wavefunctions, the position-space
0
and momentum-space wavefunctions are well behaved and the
1 3 5 7 9 entropic uncertainty relations lead to finite bounds [19].
n This material should be useful in a classroom discussion of
uncertainty and uncertainty principles. It also provides
Figure 4. Momentum standard deviation dependence on n. examples that give a student the opportunity for quantitative
calculations involving one of quantum theory’s fundamental
concepts.
 ( 2n − 1) cos ( Lp 2 ) 
∆p x =  2 πL ( −1)n + 1  p 2 Acknowledgment. The author is grateful to P. M. Wilt and K.
π ( 2n − 1) − p L
2 2 2 2
 
(21) Dutch for their comments on this manuscript and to K. D. Sen
 ( 2n − 1) cos ( Lp 2 ) n +1
 π for introducing him to entropic uncertainty relations. He would
×  2 πL ( −1)  dx = (2n − 1) also like to acknowledge the helpful comments of the
π ( 2n − 1) − p L
2 2 2 2 L
 
anonymous reviewers.

As n increases, ∆x asymptotically approaches ( )

3 6 L . The Appendix

increase in ∆px is linear with n and the product of the standard Consider a one-dimensional function f(x) and its Fourier
deviations is transform g(k) where f(x) and g(k) are normalized so that

© 2002 Springer-Verlag New York, Inc., S1430-4171(02)06618-7, Published on Web 10/18/2002, 10.1007/s00897020618a, 760334hm.pdf
338 Chem. Educator, Vol. 7, No. X, 2002 Montgomery

∞ ∞ and expressing p as a function of q, BBM showed that the

∫ f ( x ) dx = ∫ g ( k ) dk = 1 (A1) derivative of W(q) evaluated at q = 2 reduces to the inequality
−∞ −∞ of eq A2.

In 1957, Everett [18] and Hirshman [20] independently References and Notes
conjectured, but did not prove, that
1. Heisenberg, W. Z. für Phys. 1927, 43, 172–198.
∞ ∞ 2. Heisenberg, W. The Physical Principles of the Quantum Theory;
− ∫ f ( x ) ln  f ( x ) dx − ∫ g ( k ) ln  f ( k ) dk ≥ 1 + ln ( π ) (A2) Translated by Eckart, C.; Hoyt, F. C.; Dover: New York, 1949; pp
13–46.
−∞ −∞
3. Robertson, H. P. Phys. Rev. 1929, 34, 163–164.

Their conjecture was based on the observation that if f(x) 4. Hilgevoord, J. Am J. Phys. 2002, 70, 983.
and g(k) are Gaussian functions, the inequality reduces to an 5. Shannon C. E. Bell Syst. Tech. J. 1948, 27, 379–473; 623–656.
equality whereas any variation from a Gaussian function Reprinted in Shannon, C. E.; Weaver, W. The Mathematical Theory
of Communication; Illinois Press: Urbana, 1949; pp 29–125 and
increases the left side of the inequality. Analogous behavior http://cm.bell-labs.com/cm/ms/what/shannonday/paper.html
for Gaussian functions is also noted for the Heisenberg (accessed Oct 2002).
inequality. 6. Hồ, M.; Weaver, D. F.; Smith, V. H.; Sagar, R. P.; Esquivel, R. O.
Almost 20 years later BBM [11] proved eq A2 using Fourier Phys. Rev. A 1998, 57, 1412–4517.
analysis. Their proof is synopsized below. Define 7. Schneider, T. D.; Stormo, G. D.; Gold, L.; Ehrenfeucht, A. J. Mol.
Biol. 1986, 188, 415–431.
1 1
8. Tribus, M.; McIrvine, E. C. Sci. Am. 1971, 225, 179–188.
∞ p p ∞ q q
f p
=  ∫ f ( x ) dx  and g q
=  ∫ g ( x ) dx  (A3) 9. Goldstein, M.; Goldstein, I. F. The Refrigerator and the Universe;
 −∞   −∞  Harvard University Press: Cambridge, 1993; pp 212–228.
10. Pierce, J. R. An Introduction to Information Theory: Symbols,
Signals and Noise; Dover: New York, 1980.
The (p,q) norm of this Fourier transform pair is defined as the
11. Bialynicki-Birula, I.; Mycielski, J. Comm. Math. Phys. 1975, 44,
smallest number k(p,q) for which the inequality 129–132.
12. Majerník, V.; Richterek, L. Eur. J. Phys. 1997, 18, 79–89.
g q
= k ( p, q ) f p
(A4)
13. Yáñez, R. J.; Van Assche, W.; Dehesa, J. S. Phys. Rev. A 1994, 50,
3065–3079.
holds, where 14. Majerník, V.; Majerníková, E. J. Phys. A: Math. Gen. 2002, 35,
5721–5761.
1 1 15. Sánchez-Ruiz, J. Phys. Rev. A 1998, 57, 1519–1525.
+ = 1 and q ≥ 2 (A5) 16. Majerník, V.; Charvot, R; Majerníková, E. J. Phys. A: Math. Gen.
p q
1999, 32, 2207–2216.
17. Mulder, J. C. Chem. Educator [Online], 2002, 7(2): DOI
Beckner [21] showed that 10.1007/s00897020545a.
18. Everett, H., In The Many-Worlds Interpretation of Quantum
1 −1 Mechanics; DeWitt, B. S. and Graham, N., Eds.; Princeton
2 π 2 q 2 π 2 p University Press: Princeton, 1973; pp 51–52.
k ( p, q ) =     (A6)
 q   p  19. Gadre, S. H.; Sears, S.B. Phys. Rev. A 1985, 32, 2602–2606.
20. Hirschman, I. I. Am. J. Math. 1957, 79, 152–156.
Writing the difference from eq A4 as 21. Beckner, W. Proc. Nat Acad. Sci. USA 1975, 72, 638–641.

W ( q ) = k ( p, q ) f p
− g q
(A7)

© 2002 Springer-Verlag New York, Inc., S1430-4171(02)06618-7, Published on Web 10/18/2002, 10.1007/s00897020618a, 760334hm.pdf