Likelihoodist statistics

Likelihoodist statistics or likelihoodism is an approach to statistics that exclusively or primarily uses the
likelihood function. Likelihoodist statistics is a more minor school than the main approaches of Bayesian
statistics and frequentist statistics, but has some adherents and applications. The central idea of
likelihoodism is the likelihood principle: data are interpreted as evidence, and the strength of the evidence is
measured by the likelihood function. Beyond this, there are significant differences within likelihood
approaches: "orthodox" likelihoodists consider data only as evidence, and do not use it as the basis of
statistical inference, while others make inferences based on likelihood, but without using Bayesian
inference or frequentist inference. Likelihoodism is thus criticized for either not providing a basis for belief
or action (if it fails to make inferences), or not satisfying the requirements of these other schools.

The likelihood function is also used in Bayesian statistics and frequentist statistics, but they differ in how it
is used. Some likelihoodists consider their use of likelihood as an alternative to other approaches, while
others consider it complementary and compatible with other approaches; see § Relation with other theories.

Relation with other theories

While likelihoodism is a distinct approach to statistical inference, it can be related to or contrasted with
other theories and methodologies in statistics. Here are some notable connections:

1. Bayesian statistics: Bayesian statistics is an alternative approach to statistical inference that

incorporates prior information and updates it using observed data to obtain posterior
probabilities. Likelihoodism and Bayesian statistics are compatible in the sense that both
methods utilize the likelihood function. However, they differ in their treatment of prior
information. Bayesian statistics incorporates prior beliefs into the analysis explicitly, whereas
likelihoodism focuses solely on the likelihood function without specifying a prior distribution.
2. Frequentist statistics: Frequentist statistics, also known as classical or frequentist inference,
is another major framework for statistical analysis. Frequentist methods emphasize
properties of repeated sampling and focus on concepts such as unbiasedness, consistency,
and hypothesis testing. Likelihoodism can be seen as a departure from traditional frequentist
methods, as it places the likelihood function at the core of statistical inference. Likelihood-
based methods provide a bridge between the likelihoodist perspective and frequentist
approaches by using likelihood ratios for hypothesis testing and constructing confidence
intervals.
3. Fisherian statistics: Likelihoodism has deep connections to the statistical philosophy of
Ronald Fisher.[1] Fisher introduced the concept of likelihood and its maximization as a
criterion for estimating parameters. Fisher's approach emphasized the concept of sufficiency
and the maximum likelihood estimation (MLE). Likelihoodism can be seen as an extension
of Fisherian statistics, refining and expanding the use of likelihood in statistical inference.
4. Information theory: Information theory, developed by Claude Shannon, provides a
mathematical framework for quantifying information content and communication. The
concept of entropy in information theory has connections to the likelihood function and the
AIC criterion. AIC, which incorporates a penalty term for model complexity, can be viewed as
an information-theoretic approach to model selection and balances model fit with model
complexity.
5. Decision theory: Decision theory combines statistical inference with decision-making under
uncertainty. It considers the trade-off between risks and potential losses in decision-making
processes. Likelihoodism can be integrated with decision theory to make decisions based
 on the likelihood function, such as choosing the model with the highest likelihood or
evaluating different decision options based on their associated likelihoods.

Criticism
While likelihood-based statistics have been widely used and have many advantages, they are not without
criticism. Here are some common criticisms of likelihoodist statistics:

1. Model dependence: Likelihood-based inference heavily relies on the choice of a specific

statistical model.[2] If the chosen model does not accurately represent the true underlying
data-generating process, the resulting estimates and inferences may be biased or
misleading. Model misspecification can lead to incorrect conclusions, especially in complex
real-world scenarios where the true model may be unknown or difficult to capture.
2. Difficulty of interpretability: Likelihood-based statistics focus on optimizing the likelihood
function to estimate parameters, but they may not provide intuitive or easily interpretable
estimates. The estimated parameters may not have a direct and meaningful interpretation in
the context of the problem being studied. This can make it challenging for practitioners to
communicate the results to non-technical audiences or make practical decisions based on
the estimates.
3. Sensitivity to sample size: Likelihood-based methods can be sensitive to the sample size of
the data. In situations with small sample sizes, the likelihood function can be highly variable,
leading to unstable estimates. This instability can also affect the model selection process, as
the likelihood ratio test or information criteria may not perform well when sample sizes are
small.
4. Assumption of independence: Likelihood-based inference often assumes that the observed
data are independent and identically distributed (IID). However, in many real-world
scenarios, data points may exhibit dependence or correlation. Ignoring this dependence can
lead to biased estimates or inaccurate hypothesis testing.
5. Lack of robustness: Likelihood-based methods are not always robust to violations of model
assumptions or outliers in the data. If the data deviate from the assumed distribution or if
extreme observations are present, the estimates can be heavily influenced by these outliers,
leading to unreliable results.
6. Computational complexity: Estimating parameters based on likelihood functions can be
computationally intensive, especially for complex models, large datasets, or highly non-
linear systems.[3] Optimization algorithms used to maximize the likelihood function may
require substantial computational resources or may not converge to the global maximum,
leading to suboptimal estimates.
7. Lack of uncertainty quantification: Likelihood-based inference often provides point estimates
of parameters without explicit quantification of uncertainty. While techniques such as
confidence intervals or standard errors can be used to approximate uncertainty, they rely on
assumptions that may not always hold. Bayesian methods, on the other hand, provide a
more formal and coherent framework for uncertainty quantification.

History
Likelihoodism as a distinct school dates to Edwards (1972), which gives a systematic treatment of statistics,
based on likelihood. This built on significant earlier work; see Dempster (1972) for a contemporary review.

While comparing ratios of probabilities dates to early statistics and probability, notably Bayesian inference
as developed by Pierre-Simon Laplace from the late 1700s, likelihood as a distinct concept is due to Ronald
Fisher in Fisher (1921). Likelihood played an important role in Fisher's statistics, but he developed and
used many non-likelihood frequentist techniques as well. His late writings, notably Fisher (1955),
emphasize likelihood more strongly, and can be considered a precursor to a systematic theory of
likelihoodism.

The likelihood principle was proposed in 1962 by several authors, notably Barnard, Jenkins & Winsten
(1962), Birnbaum (1962), and Savage (1962), and followed by the law of likelihood in Hacking (1965);
these laid the foundation for likelihoodism. See Likelihood principle § History for early history.

While Edwards's version of likelihoodism considered likelihood as only evidence, which was followed by
Royall (1997), others proposed inference based only on likelihood, notably as extensions of maximum
likelihood estimation. Notable is John Nelder, who declared in Nelder (1999, p. 264):

At least once a year I hear someone at a meeting say that there are two modes of inference:
frequentist and Bayesian. That this sort of nonsense should be so regularly propagated shows
how much we have to do. To begin with there is a flourishing school of likelihood inference,
to which I belong.

Textbooks that take a likelihoodist approach include the following: Kalbfleisch (1985), Azzalini (1996),
Pawitan (2001), Rohde (2014), and Held & Sabanés Bové (2014). A collection of relevant papers is given
by Taper & Lele (2004).

See also
Akaike information criterion
Foundations of statistics
Likelihood ratio test

References
1. Efron, B. (February 1986). "Why Isn't Everyone a Bayesian?" (https://dx.doi.org/10.2307/268
3105). The American Statistician. 40 (1): 1. doi:10.2307/2683105 (https://doi.org/10.2307%2
F2683105). ISSN 0003-1305 (https://www.worldcat.org/issn/0003-1305).
2. Fitelson, Branden (2007-03-24). "Likelihoodism, Bayesianism, and relational confirmation"
(https://dx.doi.org/10.1007/s11229-006-9134-9). Synthese. 156 (3): 473–489.
doi:10.1007/s11229-006-9134-9 (https://doi.org/10.1007%2Fs11229-006-9134-9).
ISSN 0039-7857 (https://www.worldcat.org/issn/0039-7857).
3. Drignei, Dorin; Forest, Chris E.; Nychka, Doug (2008-12-01). "Parameter estimation for
computationally intensive nonlinear regression with an application to climate modeling" (http
s://dx.doi.org/10.1214/08-aoas210). The Annals of Applied Statistics. 2 (4). doi:10.1214/08-
aoas210 (https://doi.org/10.1214%2F08-aoas210). ISSN 1932-6157 (https://www.worldcat.or
g/issn/1932-6157).

Azzalini, Adelchi (1996), Statistical Inference—Based on the likelihood, Chapman & Hall
Barnard, G. A.; Jenkins, G. M.; Winsten, C. B. (1962), "Likelihood inference and time series",
Journal of the Royal Statistical Society, Series A, 125 (3): 321–372, doi:10.2307/2982406 (ht
tps://doi.org/10.2307%2F2982406), JSTOR 2982406 (https://www.jstor.org/stable/2982406)
Birnbaum, Allan (1962), "On the foundations of statistical inference", Journal of the American
Statistical Association, 57 (298): 269–326, doi:10.2307/2281640 (https://doi.org/10.2307%2F
 2281640), JSTOR 2281640 (https://www.jstor.org/stable/2281640), MR 0138176 (https://mat
hscinet.ams.org/mathscinet-getitem?mr=0138176) (With discussion.)
Dempster, A. P. (1972), "[Book Review] Likelihood. An Account of the Statistical Concept of
Likelihood and Its Application to Scientific Inference. A. W. F. Edwards. Cambridge
University Press, New York, 1972. xvi, 236 pp., illus. $13.50", Science, 177 (4052): 878–
879, doi:10.1126/science.177.4052.878 (https://doi.org/10.1126%2Fscience.177.4052.878)
Edwards, Anthony W. F. (1972), Likelihood (1st ed.), Cambridge University Press
Edwards, Anthony W. F. (1992), Likelihood (2nd ed.), Johns Hopkins University Press,
ISBN 0-8018-4445-2
Fisher, R. A. (1921), "On the "probable error" of a coefficient of correlation deduced from a
small sample", Metron, 1: 3–32
Fisher, Ronald (1955), "Statistical methods and scientific induction", Journal of the Royal
Statistical Society, Series B, 17: 69–78
Hacking, Ian (1965), Logic of Statistical Inference, Cambridge University Press, ISBN 0-521-
05165-7
Held, Leonhard; Sabanés Bové, Daniel (2014), Applied Statistical Inference—Likelihood
and Bayes, Springer
Kalbfleisch, J. G. (1985), Probability and Statistical Inference, vol. 2, Springer-Verlag
Nelder, John A. (1999), "From statistics to statistical science", Journal of the Royal Statistical
Society. Series D (The Statistician), 48 (2): 257–269, JSTOR 2681191 (https://www.jstor.org/
stable/2681191)
Pawitan, Yudi (2001), In All Likelihood: Statistical Modelling And Inference Using Likelihood,
Oxford University Press, ISBN 978-0-19967122-9
Rohde, Charles A. (2014), Introductory Statistical Inference with the Likelihood Function,
Springer, ISBN 978-3-319-10460-7
Royall, Richard M. (1997), Statistical Evidence: A Likelihood Paradigm (https://archive.org/d
etails/statisticalevide0000roya), Chapman & Hall, ISBN 0-412-04411-0
Savage, Leonard J.; et al. (1962), The Foundations of Statistical Inference, Methuen
Publishing
Taper, M. L.; Lele, S. R., eds. (2004), The Nature of Scientific Evidence, University of
Chicago Press

Further reading
Gandenberger, Greg (2016), "Why I am not a likelihoodist", Philosophers' Imprint, 16 (7): 1–
22, hdl:2027/spo.3521354.0016.007 (https://hdl.handle.net/2027%2Fspo.3521354.0016.00
7)

External links
"Likelihood Ratios, Likelihoodism, and the Law of Likelihood" (https://plato.stanford.edu/entri
es/logic-inductive/sup-likelihood.html). Stanford Encyclopedia of Philosophy. Retrieved
2019-03-14.

Retrieved from "https://en.wikipedia.org/w/index.php?title=Likelihoodist_statistics&oldid=1164483768"