Principle of maximum entropy

The principle of maximum entropy states that the probability distribution which best represents the
current state of knowledge about a system is the one with largest entropy, in the context of precisely stated
prior data (such as a proposition that expresses testable information).

Another way of stating this: Take precisely stated prior data or testable information about a probability
distribution function. Consider the set of all trial probability distributions that would encode the prior data.
According to this principle, the distribution with maximal information entropy is the best choice.

History
The principle was first expounded by E. T. Jaynes in two papers in 1957[1][2] where he emphasized a
natural correspondence between statistical mechanics and information theory. In particular, Jaynes offered a
new and very general rationale why the Gibbsian method of statistical mechanics works. He argued that the
entropy of statistical mechanics and the information entropy of information theory are basically the same
thing. Consequently, statistical mechanics should be seen just as a particular application of a general tool of
logical inference and information theory.

Overview
In most practical cases, the stated prior data or testable information is given by a set of conserved quantities
(average values of some moment functions), associated with the probability distribution in question. This is
the way the maximum entropy principle is most often used in statistical thermodynamics. Another
possibility is to prescribe some symmetries of the probability distribution. The equivalence between
conserved quantities and corresponding symmetry groups implies a similar equivalence for these two ways
of specifying the testable information in the maximum entropy method.

The maximum entropy principle is also needed to guarantee the uniqueness and consistency of probability
assignments obtained by different methods, statistical mechanics and logical inference in particular.

The maximum entropy principle makes explicit our freedom in using different forms of prior data. As a
special case, a uniform prior probability density (Laplace's principle of indifference, sometimes called the
principle of insufficient reason), may be adopted. Thus, the maximum entropy principle is not merely an
alternative way to view the usual methods of inference of classical statistics, but represents a significant
conceptual generalization of those methods.

However these statements do not imply that thermodynamical systems need not be shown to be ergodic to
justify treatment as a statistical ensemble.

In ordinary language, the principle of maximum entropy can be said to express a claim of epistemic
modesty, or of maximum ignorance. The selected distribution is the one that makes the least claim to being
informed beyond the stated prior data, that is to say the one that admits the most ignorance beyond the
stated prior data.

Testable information
The principle of maximum entropy is useful explicitly only when applied to testable information. Testable
information is a statement about a probability distribution whose truth or falsity is well-defined. For
example, the statements

the expectation of the variable is 2.87

and

(where and are probabilities of events) are statements of testable information.

Given testable information, the maximum entropy procedure consists of seeking the probability distribution
which maximizes information entropy, subject to the constraints of the information. This constrained
optimization problem is typically solved using the method of Lagrange multipliers.

Entropy maximization with no testable information respects the universal "constraint" that the sum of the
probabilities is one. Under this constraint, the maximum entropy discrete probability distribution is the
uniform distribution,

Applications
The principle of maximum entropy is commonly applied in two ways to inferential problems:

Prior probabilities

The principle of maximum entropy is often used to obtain prior probability distributions for Bayesian
inference. Jaynes was a strong advocate of this approach, claiming the maximum entropy distribution
represented the least informative distribution.[3] A large amount of literature is now dedicated to the
elicitation of maximum entropy priors and links with channel coding.[4][5][6][7]

Posterior probabilities

Maximum entropy is a sufficient updating rule for radical probabilism. Richard Jeffrey's probability
kinematics is a special case of maximum entropy inference. However, maximum entropy is not a
generalisation of all such sufficient updating rules.[8]

Maximum entropy models

Alternatively, the principle is often invoked for model specification: in this case the observed data itself is
assumed to be the testable information. Such models are widely used in natural language processing. An
example of such a model is logistic regression, which corresponds to the maximum entropy classifier for
independent observations.

Probability density estimation

One of the main applications of the maximum entropy principle is in discrete and continuous density
estimation.[9][10] Similar to support vector machine estimators, the maximum entropy principle may require
the solution to a quadratic programming problem, and thus provide a sparse mixture model as the optimal
density estimator. One important advantage of the method is its ability to incorporate prior information in
the density estimation.[11]

General solution for the maximum entropy distribution with

linear constraints

Discrete case

We have some testable information I about a quantity x taking values in {x1 , x2 ,..., xn }. We assume this
information has the form of m constraints on the expectations of the functions fk; that is, we require our
probability distribution to satisfy the moment inequality/equality constraints:

where the are observables. We also require the probability density to sum to one, which may be viewed
as a primitive constraint on the identity function and an observable equal to 1 giving the constraint

The probability distribution with maximum information entropy subject to these inequality/equality
constraints is of the form:[9]

for some . It is sometimes called the Gibbs distribution. The normalization constant is
determined by:

and is conventionally called the partition function. (The Pitman–Koopman theorem states that the necessary
and sufficient condition for a sampling distribution to admit sufficient statistics of bounded dimension is that
it have the general form of a maximum entropy distribution.)

The λk parameters are Lagrange multipliers. In the case of equality constraints their values are determined
from the solution of the nonlinear equations
In the case of inequality constraints, the Lagrange multipliers are determined from the solution of a convex
optimization program with linear constraints.[9] In both cases, there is no closed form solution, and the
computation of the Lagrange multipliers usually requires numerical methods.

Continuous case

For continuous distributions, the Shannon entropy cannot be used, as it is only defined for discrete
probability spaces. Instead Edwin Jaynes (1963, 1968, 2003) gave the following formula, which is closely
related to the relative entropy (see also differential entropy).

where q(x), which Jaynes called the "invariant measure", is proportional to the limiting density of discrete
points. For now, we shall assume that q is known; we will discuss it further after the solution equations are
given.

A closely related quantity, the relative entropy, is usually defined as the Kullback–Leibler divergence of p
from q (although it is sometimes, confusingly, defined as the negative of this). The inference principle of
minimizing this, due to Kullback, is known as the Principle of Minimum Discrimination Information.

We have some testable information I about a quantity x which takes values in some interval of the real
numbers (all integrals below are over this interval). We assume this information has the form of m
constraints on the expectations of the functions fk, i.e. we require our probability density function to satisfy
the inequality (or purely equality) moment constraints:

where the are observables. We also require the probability density to integrate to one, which may be
viewed as a primitive constraint on the identity function and an observable equal to 1 giving the constraint

The probability density function with maximum Hc subject to these constraints is:[10]

with the partition function determined by

As in the discrete case, in the case where all moment constraints are equalities, the values of the
parameters are determined by the system of nonlinear equations:
In the case with inequality moment constraints the Lagrange multipliers are determined from the solution of
a convex optimization program.[10]

The invariant measure function q(x) can be best understood by supposing that x is known to take values
only in the bounded interval (a, b), and that no other information is given. Then the maximum entropy
probability density function is

where A is a normalization constant. The invariant measure function is actually the prior density function
encoding 'lack of relevant information'. It cannot be determined by the principle of maximum entropy, and
must be determined by some other logical method, such as the principle of transformation groups or
marginalization theory.

Examples

For several examples of maximum entropy distributions, see the article on maximum entropy probability
distributions.

Justifications for the principle of maximum entropy

Proponents of the principle of maximum entropy justify its use in assigning probabilities in several ways,
including the following two arguments. These arguments take the use of Bayesian probability as given, and
are thus subject to the same postulates.

Information entropy as a measure of 'uninformativeness'

Consider a discrete probability distribution among mutually exclusive propositions. The most
informative distribution would occur when one of the propositions was known to be true. In that case, the
information entropy would be equal to zero. The least informative distribution would occur when there is
no reason to favor any one of the propositions over the others. In that case, the only reasonable probability
distribution would be uniform, and then the information entropy would be equal to its maximum possible
value, . The information entropy can therefore be seen as a numerical measure which describes how
uninformative a particular probability distribution is, ranging from zero (completely informative) to
(completely uninformative).

By choosing to use the distribution with the maximum entropy allowed by our information, the argument
goes, we are choosing the most uninformative distribution possible. To choose a distribution with lower
entropy would be to assume information we do not possess. Thus the maximum entropy distribution is the
only reasonable distribution. The dependence of the solution (http://projecteuclid.org/euclid.ba/134037071
0) on the dominating measure represented by is however a source of criticisms of the approach since
this dominating measure is in fact arbitrary. [12]

The Wallis derivation

The following argument is the result of a suggestion made by Graham Wallis to E. T. Jaynes in 1962.[13] It
is essentially the same mathematical argument used for the Maxwell–Boltzmann statistics in statistical
mechanics, although the conceptual emphasis is quite different. It has the advantage of being strictly
combinatorial in nature, making no reference to information entropy as a measure of 'uncertainty',
'uninformativeness', or any other imprecisely defined concept. The information entropy function is not
assumed a priori, but rather is found in the course of the argument; and the argument leads naturally to the
procedure of maximizing the information entropy, rather than treating it in some other way.

Suppose an individual wishes to make a probability assignment among mutually exclusive propositions.
He has some testable information, but is not sure how to go about including this information in his
probability assessment. He therefore conceives of the following random experiment. He will distribute
quanta of probability (each worth ) at random among the possibilities. (One might imagine that he
will throw balls into buckets while blindfolded. In order to be as fair as possible, each throw is to be
independent of any other, and every bucket is to be the same size.) Once the experiment is done, he will
check if the probability assignment thus obtained is consistent with his information. (For this step to be
successful, the information must be a constraint given by an open set in the space of probability measures).
If it is inconsistent, he will reject it and try again. If it is consistent, his assessment will be

where is the probability of the th proposition, while ni is the number of quanta that were assigned to the
th proposition (i.e. the number of balls that ended up in bucket ).

Now, in order to reduce the 'graininess' of the probability assignment, it will be necessary to use quite a
large number of quanta of probability. Rather than actually carry out, and possibly have to repeat, the rather
long random experiment, the protagonist decides to simply calculate and use the most probable result. The
probability of any particular result is the multinomial distribution,

where

is sometimes known as the multiplicity of the outcome.

The most probable result is the one which maximizes the multiplicity . Rather than maximizing
directly, the protagonist could equivalently maximize any monotonic increasing function of . He decides
to maximize

At this point, in order to simplify the expression, the protagonist takes the limit as , i.e. as the
probability levels go from grainy discrete values to smooth continuous values. Using Stirling's
approximation, he finds
All that remains for the protagonist to do is to maximize entropy under the constraints of his testable
information. He has found that the maximum entropy distribution is the most probable of all "fair" random
distributions, in the limit as the probability levels go from discrete to continuous.

Compatibility with Bayes' theorem

Giffin and Caticha (2007) state that Bayes' theorem and the principle of maximum entropy are completely
compatible and can be seen as special cases of the "method of maximum relative entropy". They state that
this method reproduces every aspect of orthodox Bayesian inference methods. In addition this new method
opens the door to tackling problems that could not be addressed by either the maximal entropy principle or
orthodox Bayesian methods individually. Moreover, recent contributions (Lazar 2003, and Schennach
2005) show that frequentist relative-entropy-based inference approaches (such as empirical likelihood and
exponentially tilted empirical likelihood – see e.g. Owen 2001 and Kitamura 2006) can be combined with
prior information to perform Bayesian posterior analysis.

Jaynes stated Bayes' theorem was a way to calculate a probability, while maximum entropy was a way to
assign a prior probability distribution.[14]

It is however, possible in concept to solve for a posterior distribution directly from a stated prior distribution
using the principle of minimum cross entropy (or the Principle of Maximum Entropy being a special case of
using a uniform distribution as the given prior), independently of any Bayesian considerations by treating
the problem formally as a constrained optimisation problem, the Entropy functional being the objective
function. For the case of given average values as testable information (averaged over the sought after
probability distribution), the sought after distribution is formally the Gibbs (or Boltzmann) distribution the
parameters of which must be solved for in order to achieve minimum cross entropy and satisfy the given
testable information.

Relevance to physics
The principle of maximum entropy bears a relation to a key assumption of kinetic theory of gases known as
molecular chaos or Stosszahlansatz. This asserts that the distribution function characterizing particles
entering a collision can be factorized. Though this statement can be understood as a strictly physical
hypothesis, it can also be interpreted as a heuristic hypothesis regarding the most probable configuration of
particles before colliding.[15]

See also
Akaike information criterion
Dissipation
Info-metrics
Maximum entropy classifier
Maximum entropy probability distribution
Maximum entropy spectral estimation
Maximum entropy thermodynamics
Principle of maximum caliber
Thermodynamic equilibrium
Molecular chaos

Notes
1. Jaynes, E. T. (1957). "Information Theory and Statistical Mechanics" (http://bayes.wustl.edu/
etj/articles/theory.1.pdf) (PDF). Physical Review. Series II. 106 (4): 620–630.
Bibcode:1957PhRv..106..620J (https://ui.adsabs.harvard.edu/abs/1957PhRv..106..620J).
doi:10.1103/PhysRev.106.620 (https://doi.org/10.1103%2FPhysRev.106.620). MR 0087305
(https://mathscinet.ams.org/mathscinet-getitem?mr=0087305).
2. Jaynes, E. T. (1957). "Information Theory and Statistical Mechanics II" (http://bayes.wustl.ed
u/etj/articles/theory.2.pdf) (PDF). Physical Review. Series II. 108 (2): 171–190.
Bibcode:1957PhRv..108..171J (https://ui.adsabs.harvard.edu/abs/1957PhRv..108..171J).
doi:10.1103/PhysRev.108.171 (https://doi.org/10.1103%2FPhysRev.108.171). MR 0096414
(https://mathscinet.ams.org/mathscinet-getitem?mr=0096414).
3. Jaynes, E. T. (1968). "Prior Probabilities" (http://bayes.wustl.edu/etj/articles/brandeis.pdf)
(PDF or PostScript (http://bayes.wustl.edu/etj/articles/brandeis.ps.gz)). IEEE Transactions on
Systems Science and Cybernetics. 4 (3): 227–241. doi:10.1109/TSSC.1968.300117 (https://
doi.org/10.1109%2FTSSC.1968.300117). {{cite journal}}: External link in
|format= (help)
4. Clarke, B. (2006). "Information optimality and Bayesian modelling". Journal of Econometrics.
138 (2): 405–429. doi:10.1016/j.jeconom.2006.05.003 (https://doi.org/10.1016%2Fj.jeconom.
2006.05.003).
5. Soofi, E.S. (2000). "Principal Information Theoretic Approaches". Journal of the American
Statistical Association. 95 (452): 1349–1353. doi:10.2307/2669786 (https://doi.org/10.2307%
2F2669786). JSTOR 2669786 (https://www.jstor.org/stable/2669786). MR 1825292 (https://
mathscinet.ams.org/mathscinet-getitem?mr=1825292).
6. Bousquet, N. (2008). "Eliciting vague but proper maximal entropy priors in Bayesian
experiments". Statistical Papers. 51 (3): 613–628. doi:10.1007/s00362-008-0149-9 (https://d
oi.org/10.1007%2Fs00362-008-0149-9). S2CID 119657859 (https://api.semanticscholar.org/
CorpusID:119657859).
7. Palmieri, Francesco A. N.; Ciuonzo, Domenico (2013-04-01). "Objective priors from
maximum entropy in data classification". Information Fusion. 14 (2): 186–198.
CiteSeerX 10.1.1.387.4515 (https://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.387.
4515). doi:10.1016/j.inffus.2012.01.012 (https://doi.org/10.1016%2Fj.inffus.2012.01.012).
 8. Skyrms, B (1987). "Updating, supposing and MAXENT". Theory and Decision. 22 (3): 225–
46. doi:10.1007/BF00134086 (https://doi.org/10.1007%2FBF00134086). S2CID 121847242
(https://api.semanticscholar.org/CorpusID:121847242).
9. Botev, Z. I.; Kroese, D. P. (2008). "Non-asymptotic Bandwidth Selection for Density
Estimation of Discrete Data". Methodology and Computing in Applied Probability. 10 (3):
435. doi:10.1007/s11009-007-9057-z (https://doi.org/10.1007%2Fs11009-007-9057-z).
S2CID 122047337 (https://api.semanticscholar.org/CorpusID:122047337).
10. Botev, Z. I.; Kroese, D. P. (2011). "The Generalized Cross Entropy Method, with Applications
to Probability Density Estimation" (http://espace.library.uq.edu.au/view/UQ:200564/UQ2005
64_preprint.pdf) (PDF). Methodology and Computing in Applied Probability. 13 (1): 1–27.
doi:10.1007/s11009-009-9133-7 (https://doi.org/10.1007%2Fs11009-009-9133-7).
S2CID 18155189 (https://api.semanticscholar.org/CorpusID:18155189).
11. Kesavan, H. K.; Kapur, J. N. (1990). "Maximum Entropy and Minimum Cross-Entropy
Principles". In Fougère, P. F. (ed.). Maximum Entropy and Bayesian Methods (https://archive.
org/details/maximumentropyba00jayn_552). pp. 419 (https://archive.org/details/maximument
ropyba00jayn_552/page/n418)–432. doi:10.1007/978-94-009-0683-9_29 (https://doi.org/10.
1007%2F978-94-009-0683-9_29). ISBN 978-94-010-6792-8.
12. Druilhet, Pierre; Marin, Jean-Michel (2007). "Invariant {HPD} credible sets and {MAP}
estimators" (https://projecteuclid.org/euclid.ba/1340370710). Bayesian Anal. 2: 681–691.
doi:10.1214/07-BA227 (https://doi.org/10.1214%2F07-BA227).
13. Jaynes, E. T. (2003) Probability Theory: The Logic of Science, Cambridge University Press,
p. 351-355. ISBN 978-0521592710
14. Jaynes, E. T. (1988) "The Relation of Bayesian and Maximum Entropy Methods" (http://baye
s.wustl.edu/etj/articles/relationship.pdf), in Maximum-Entropy and Bayesian Methods in
Science and Engineering (Vol. 1), Kluwer Academic Publishers, p. 25-29.
15. Chliamovitch, G.; Malaspinas, O.; Chopard, B. (2017). "Kinetic theory beyond the
Stosszahlansatz" (https://doi.org/10.3390%2Fe19080381). Entropy. 19 (8): 381.
Bibcode:2017Entrp..19..381C (https://ui.adsabs.harvard.edu/abs/2017Entrp..19..381C).
doi:10.3390/e19080381 (https://doi.org/10.3390%2Fe19080381).

References
Bajkova, A. T. (1992). "The generalization of maximum entropy method for reconstruction of
complex functions". Astronomical and Astrophysical Transactions. 1 (4): 313–320.
Bibcode:1992A&AT....1..313B (https://ui.adsabs.harvard.edu/abs/1992A&AT....1..313B).
doi:10.1080/10556799208230532 (https://doi.org/10.1080%2F10556799208230532).
Fornalski, K.W.; Parzych, G.; Pylak, M.; Satuła, D.; Dobrzyński, L. (2010). "Application of
Bayesian reasoning and the Maximum Entropy Method to some reconstruction problems" (ht
tp://przyrbwn.icm.edu.pl/APP/PDF/117/a117z602.pdf) (PDF). Acta Physica Polonica A. 117
(6): 892–899. Bibcode:2010AcPPA.117..892F (https://ui.adsabs.harvard.edu/abs/2010AcPP
A.117..892F). doi:10.12693/APhysPolA.117.892 (https://doi.org/10.12693%2FAPhysPolA.1
17.892).
Giffin, A. and Caticha, A., 2007, Updating Probabilities with Data and Moments (https://arxiv.
org/abs/0708.1593)
Guiasu, S.; Shenitzer, A. (1985). "The principle of maximum entropy". The Mathematical
Intelligencer. 7 (1): 42–48. doi:10.1007/bf03023004 (https://doi.org/10.1007%2Fbf0302300
4). S2CID 53059968 (https://api.semanticscholar.org/CorpusID:53059968).
Harremoës, P.; Topsøe (2001). "Maximum entropy fundamentals" (https://doi.org/10.3390%2
Fe3030191). Entropy. 3 (3): 191–226. Bibcode:2001Entrp...3..191H (https://ui.adsabs.harvar
d.edu/abs/2001Entrp...3..191H). doi:10.3390/e3030191 (https://doi.org/10.3390%2Fe303019
1).
 Jaynes, E. T. (1963). "Information Theory and Statistical Mechanics" (http://bayes.wustl.edu/
etj/node1.html). In Ford, K. (ed.). Statistical Physics. New York: Benjamin. p. 181.
Jaynes, E. T., 1986 (new version online 1996), "Monkeys, kangaroos and N (http://bayes.wu
stl.edu/etj/articles/cmonkeys.pdf)", in Maximum-Entropy and Bayesian Methods in Applied
Statistics, J. H. Justice (ed.), Cambridge University Press, Cambridge, p. 26.
Kapur, J. N.; and Kesavan, H. K., 1992, Entropy Optimization Principles with Applications,
Boston: Academic Press. ISBN 0-12-397670-7
Kitamura, Y., 2006, Empirical Likelihood Methods in Econometrics: Theory and Practice (htt
p://cowles.yale.edu/sites/default/files/files/pub/d15/d1569.pdf), Cowles Foundation
Discussion Papers 1569, Cowles Foundation, Yale University.
Lazar, N (2003). "Bayesian empirical likelihood". Biometrika. 90 (2): 319–326.
doi:10.1093/biomet/90.2.319 (https://doi.org/10.1093%2Fbiomet%2F90.2.319).
Owen, A. B., 2001, Empirical Likelihood, Chapman and Hall/CRC. ISBN 1-58-488071-6.
Schennach, S. M. (2005). "Bayesian exponentially tilted empirical likelihood". Biometrika. 92
(1): 31–46. doi:10.1093/biomet/92.1.31 (https://doi.org/10.1093%2Fbiomet%2F92.1.31).
Uffink, Jos (1995). "Can the Maximum Entropy Principle be explained as a consistency
requirement?" (https://web.archive.org/web/20060603144738/http://www.phys.uu.nl/~wwwgr
nsl/jos/mepabst/mep.pdf) (PDF). Studies in History and Philosophy of Modern Physics. 26B
(3): 223–261. Bibcode:1995SHPMP..26..223U (https://ui.adsabs.harvard.edu/abs/1995SHP
MP..26..223U). CiteSeerX 10.1.1.27.6392 (https://citeseerx.ist.psu.edu/viewdoc/summary?do
i=10.1.1.27.6392). doi:10.1016/1355-2198(95)00015-1 (https://doi.org/10.1016%2F1355-219
8%2895%2900015-1). hdl:1874/2649 (https://hdl.handle.net/1874%2F2649). Archived from
the original (http://www.phys.uu.nl/~wwwgrnsl/jos/mepabst/mep.pdf) (PDF) on 2006-06-03.

Further reading
Boyd, Stephen; Lieven Vandenberghe (2004). Convex Optimization (https://web.stanford.ed
u/~boyd/cvxbook/bv_cvxbook.pdf#page=376) (PDF). Cambridge University Press. p. 362.
ISBN 0-521-83378-7. Retrieved 2008-08-24.
Ratnaparkhi A. (1997) "A simple introduction to maximum entropy models for natural
language processing" (http://repository.upenn.edu/cgi/viewcontent.cgi?article=1083&context
=ircs_reports) Technical Report 97-08, Institute for Research in Cognitive Science,
University of Pennsylvania. An easy-to-read introduction to maximum entropy methods in the
context of natural language processing.
Tang, A.; Jackson, D.; Hobbs, J.; Chen, W.; Smith, J. L.; Patel, H.; Prieto, A.; Petrusca, D.;
Grivich, M. I.; Sher, A.; Hottowy, P.; Dabrowski, W.; Litke, A. M.; Beggs, J. M. (2008). "A
Maximum Entropy Model Applied to Spatial and Temporal Correlations from Cortical
Networks in Vitro" (https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6670549). Journal of
Neuroscience. 28 (2): 505–518. doi:10.1523/JNEUROSCI.3359-07.2008 (https://doi.org/10.1
523%2FJNEUROSCI.3359-07.2008). PMC 6670549 (https://www.ncbi.nlm.nih.gov/pmc/artic
les/PMC6670549). PMID 18184793 (https://pubmed.ncbi.nlm.nih.gov/18184793). Open
access article containing pointers to various papers and software implementations of
Maximum Entropy Model on the net.

Retrieved from "https://en.wikipedia.org/w/index.php?title=Principle_of_maximum_entropy&oldid=1159169311"