Computational Bayesian Statistics

An Introduction

M. Antónia Amaral Turkman

Carlos Daniel Paulino
Peter Müller
 Contents

Preface to the English Version viii

Preface ix

1 Bayesian Inference 1
1.1 The Classical Paradigm 2
1.2 The Bayesian Paradigm 5
1.3 Bayesian Inference 8
1.3.1 Parametric Inference 8
1.3.2 Predictive Inference 12
1.4 Conclusion 13
Problems 14

2 Representation of Prior Information 17

2.1 Non-Informative Priors 18
2.2 Natural Conjugate Priors 23
Problems 26

3 Bayesian Inference in Basic Problems 29

3.1 The Binomial ∧ Beta Model 29
3.2 The Poisson ∧ Gamma Model 31
3.3 Normal (Known µ) ∧ Inverse Gamma Model 32
3.4 Normal (Unknown µ, σ2 ) ∧ Jeffreys’ Prior 33
3.5 Two Independent Normal Models ∧ Marginal Jeffreys’ Priors 34
3.6 Two Independent Binomials ∧ Beta Distributions 35
3.7 Multinomial ∧ Dirichlet Model 37
3.8 Inference in Finite Populations 40
Problems 41

4 Inference by Monte Carlo Methods 45

4.1 Simple Monte Carlo 45
4.1.1 Posterior Probabilities 48
4.1.2 Credible Intervals 49
vi Contents

4.1.3 Marginal Posterior Distributions 50

4.1.4 Predictive Summaries 52
4.2 Monte Carlo with Importance Sampling 52
4.2.1 Credible Intervals 56
4.2.2 Bayes Factors 58
4.2.3 Marginal Posterior Densities 59
4.3 Sequential Monte Carlo 61
4.3.1 Dynamic State Space Models 61
4.3.2 Particle Filter 62
4.3.3 Adapted Particle Filter 64
4.3.4 Parameter Learning 65
Problems 66

5 Model Assessment 72
5.1 Model Criticism and Adequacy 72
5.2 Model Selection and Comparison 78
5.2.1 Measures of Predictive Performance 79
5.2.2 Selection by Posterior Predictive Performance 83
5.2.3 Model Selection Using Bayes Factors 85
5.3 Further Notes on Simulation in Model Assessment 87
5.3.1 Evaluating Posterior Predictive Distributions 87
5.3.2 Prior Predictive Density Estimation 88
5.3.3 Sampling from Predictive Distributions 89
Problems 90

6 Markov Chain Monte Carlo Methods 92

6.1 Definitions and Basic Results for Markov Chains 93
6.2 Metropolis–Hastings Algorithm 96
6.3 Gibbs Sampler 100
6.4 Slice Sampler 107
6.5 Hamiltonian Monte Carlo 109
6.5.1 Hamiltonian Dynamics 109
6.5.2 Hamiltonian Monte Carlo Transition Probabilities 113
6.6 Implementation Details 115
Problems 118

7 Model Selection and Trans-dimensional MCMC 131

7.1 MC Simulation over the Parameter Space 132
7.2 MC Simulation over the Model Space 133
7.3 MC Simulation over Model and Parameter Space 138
7.4 Reversible Jump MCMC 140
Problems 145
 Contents vii

8 Methods Based on Analytic Approximations 152

8.1 Analytical Methods 153
8.1.1 Multivariate Normal Posterior Approximation 153
8.1.2 The Classical Laplace Method 156
8.2 Latent Gaussian Models (LGM) 161
8.3 Integrated Nested Laplace Approximation 163
8.4 Variational Bayesian Inference 166
8.4.1 Posterior Approximation 166
8.4.2 Coordinate Ascent Algorithm 167
8.4.3 Automatic Differentiation Variational Inference 170
Problems 171

9 Software 174
9.1 Application Example 175
9.2 The BUGS Project: WinBUGS and OpenBUGS 176
9.2.1 Application Example: Using R2OpenBUGS 177
9.3 JAGS 183
9.3.1 Application Example: Using R2jags 183
9.4 Stan 187
9.4.1 Application Example: Using RStan 188
9.5 BayesX 194
9.5.1 Application Example: Using R2BayesX 196
9.6 Convergence Diagnostics: the Programs CODA and BOA 200
9.6.1 Convergence Diagnostics 201
9.6.2 The CODA and BOA Packages 203
9.6.3 Application Example: CODA and BOA 205
9.7 R-INLA and the Application Example 215
9.7.1 Application Example 217
Problems 224

Appendix A. Probability Distributions 224

Appendix B. Programming Notes 229
References 234
Index 243
 Preface to the English Version

This book is based on lecture notes for a short course that was given at
the XXII Congresso da Sociedade Portuguesa de Estatística. In the trans-
lation from the original Portuguese text we have added some additional
material on sequential Monte Carlo, Hamiltonian Monte Carlo, transdi-
mensional Markov chain Monte Carlo (MCMC), and variational Bayes,
and we have introduced problem sets. The inclusion of problems makes
the book suitable as a textbook for a first graduate-level class in Bayesian
computation with a focus on Monte Carlo methods. The extensive discus-
sion of Bayesian software makes it useful also for researchers and graduate
students from beyond statistics.
The core of the text lies in Chapters 4, 6, and 9 on Monte Carlo meth-
ods, MCMC methods, and Bayesian software. Chapters 5, 7, and 8 include
additional material on model validation and comparison, transdimensional
MCMC, and conditionally Gaussian models. Chapters 1 through 3 intro-
duce the basics of Bayesian inference, and could be covered fairly quickly
by way of introduction; these chapters are intended primarily for review
and to introduce notation and terminology. For a more in-depth introduc-
tion we recommend the textbooks by Carlin and Louis (2009), Christensen
et al (2011), Gelman et al (2014a) or Hoff (2009).
 Preface

In 1975, Dennis Lindley wrote an article in Advances in Applied Proba-

bility titled “The future of statistics: a Bayesian 21st century,” predicting
for the twenty-first century the predominance of the Bayesian approach to
inference in statistics. Today one can certainly say that Dennis Lindley was
right in his prediction, but not exactly in the reasons he gave. He did not
foresee that the critical ingredient would be great advances in computa-
tional Bayesian statistics made in the last decade of the twentieth century.
The “Bayesian solution” for inference problems is highly attractive, espe-
cially with respect to interpretability of the inference results. However, in
practice, the derivation of such solutions involves in particular the eval-
uation of integrals, in most cases multi-dimensional, that are difficult or
impossible to tackle without simulation. The development of more or less
sophisticated computational methods has completely changed the outlook.
Today, Bayesian methods are used to solve problems in practically all ar-
eas of science, especially when the processes being modeled are extremely
complex. However, Bayesian methods can not be applied blindly. Despite
the existence of many software packages for Bayesian analysis, it is critical
that investigators understand what these programs output and why.
The aim of this text, associated with a minicourse given at the XXII Con-
gresso da Sociedade Portuguesa de Estatística, is to present the fundamen-
tal ideas that underlie the construction and analysis of Bayesian models,
with particular focus on computational methods and schemes.
We start in Chapter 1 with a brief summary of the foundations of Bayesian
inference with an emphasis on the principal differences between the clas-
sical and Bayesian paradigms. One of the main pillars of Bayesian infer-
ence, the specification of prior information, is unfortunately often ignored
in applications. We review its essential aspects in Chapter 2. In Chapter 3,
analytically solveable examples are used to illustrate the Bayesian solution
to statistical inference problems. The “great idea” behind the development
of computational Bayesian statistics is the recognition that Bayesian infer-
x Preface

ence can be implemented by way of simulation from the posterior distribu-

tion. Classical Monte Carlo methods are presented in Chapter 4 as a first
solution for computational problems. Model validation is a very important
question, with its own set of concepts and issues in the Bayesian context.
The most widely used methods to assess, select, and compare models are
briefly reviewed in Chapter 5.
Problems that are more complex than the basic ones in Chapter 4 require
the use of more sophisticated simulation methods, in particular Markov
chain Monte Carlo (MCMC) methods. These are introduced in Chapter 6,
starting as simply as possible. Another alternative to simulation is the use
of posterior approximations, which is reviewed in Chapter 8. The chapter
describes, in a generic fashion, the use of integrated nested Laplace ap-
proximation (INLA), which allows for substantial improvements in both
computation times (by several factors), and in the precision of the reported
inference summaries. Although applicable in a large class of problems, the
method is more restrictive than stochastic simulation. Finally, Chapter 9
is dedicated to Bayesian software. The possibility of resorting to MCMC
methods for posterior simulation underpins the development of the soft-
ware BUGS, which allows the use of Bayesian inference in a large variety
of problems across many areas of science. Rapid advances in technology in
general have changed the paradigm of statistics, with the increasing need
to deal with massive data sets (“Big Data”), often of spatial and temporal
types. As a consequence, posterior simulation in problems with complex
and high-dimensional data has become a new challenge, which gives rise
to new and better computational methods and the development of software
that can overcome the earlier limitations of BUGS and its successors, Win-
BUGS and OpenBUGS. In Chapter 9 we review other statistics packages
that implement MCMC methods and variations, such as JAGS, Stan, and
BayesX. This chapter also includes a brief description of the R package
R-INLA, which implements INLA.
For the compilation of this text we heavily relied on the book Estatística
Bayesiana by Paulino, A. Turkman, and Murteira, published by Fundação
Calouste Gulbenkian in 2003. As all copies of this book were sold a long
while ago, we also extensively used preliminary work for an upcoming
second edition, as well as material that we published in the October 2013
edition of the bulletin of the Sociedade Portuguesa de Estatística (SPE).
This text would not have been completed in its current form without the
valuable and unfailing support of our dear friend and colleague Giovani
Silva. We owe him sincere thanks. We are also thankful to the Sociedade
Portuguesa de Estatística for having proposed the wider theme of Bayesian
 Preface xi

statistics and for the opportunity to give a minicourse at the 22nd confer-
ence of the society. We also acknowledge the institutional support from
the Universidade de Lisboa through the Centro de Estatística e Aplicações
(PEst-OE/MAT/UI0006/2014, UID/MAT/00006/2013), in the Department
of Statistics and Operations Research in the Faculdade de Ciências and
of the Department of Mathematics in the Instituto Superior Técnico. We
would like to acknowledge that the partial support by the Funda cão para a
Ciência e Tecnologia through various projects over many years enabled us
to build up this expertise in Bayesian statistics.
Finally, we would like to dedicate this book to Professor Bento Murteira
to whom the development of Bayesian statistics in Portugal owes a lot. In
fact, Chapter 1 in this book reflects in many ways the flavor of his writings.
 1

Bayesian Inference

1
Before discussing Bayesian inference, we recall the fundamental problem
of statistics: “The fundamental problem towards which the study of Statis-
tics is addressed is that of inference. Some data are observed and we wish
to make statements, inferences, about one or more unknown features of
the physical system which gave rise to these data” (O’Hagan, 2010). Upon
more careful consideration of the foundations of statistics we find many
different schools of thought. Even leaving aside those that are collectively
known as classical statistics, this leaves several choices: objective and sub-
jective Bayes, fiducialist inference, likelihood based methods, and more.2
This diversity is not unexpected! Deriving the desired inference on pa-
rameters and models from the data is a problem of induction, which is one
of the most controversial problems in philosophy. Each school of thought
follows its own principles and methods to lead to statistical inference.
Berger (1984) describes this as: “Statistics needs a: ‘foundation’, by which
I mean a framework of analysis within which any statistical investigation
can theoretically be planned, performed, and meaningfully evaluated. The
words ‘any’ and ‘theoretically’ are key, in that the framework should ap-
ply to any situation but may only theoretically be implementable. Practical
difficulties or time limitations may prevent complete (or even partial) utili-
sation of such framework, but the direction in which ‘truth’ could be found
would at least be known”. The foundations of Bayesian inference are bet-
1
This material will be published by Cambridge University Press as Computational
Bayesian Statistics, by M.A. Amaral Turkman, C.D. Paulino, and P. Müller
(https://tinyurl.com/CompBayes). This pre-publication version is free to view
and download for personal use only. Not for re-distribution, re-sale, or use in derivative
works. c Maria Antónia Amaral Turkman, Carlos Daniel Paulino & Peter Müller, 2019.
2
Subjective Bayes is essentially the subject of this volume. In addition to these schools of
thought, there are even half-Bayesians who accept the use of a priori information but
believe that probability calculus is inadequate to combine prior information with data,
which should instead be replaced by a notion of causal inference.
2 Bayesian Inference

ter understood when seen in contrast to those of its mainstream competitor,

classical inference.

1.1 The Classical Paradigm

Classical statistics seeks to make inference about a population starting from
a sample. Let x (or x = (x1 , x2 , . . . , xn ), where n is a sample size,) denote
the data. The set X of possible samples x is known as the sample space,
usually X ⊆ Rn . Underlying classical inference is the recognition of vari-
ability across samples, keeping in mind that the observed data are only
one of many – possibly infinitely many – data sets that could have been
observed. The interpretation of the data depends not only on the observed
data, but also on the assumptions put forward about the process generating
the observable data. As a consequence, the data are treated as a realization
of a random variable or a random vector X with a distribution Fθ , which
of course is not entirely known. However, there is usually some knowledge
(theoretical considerations, experimental evidence, etc.) about the nature
of the chance experiment under consideration that allow one to conjecture
that Fθ is a member of a family of distributions F . This family of distri-
butions becomes the statistical model for X. The assumption of a model is
also known as the model specification and is an essential part of developing
the desired inference.
Assuming that X is a continuous random variable or random vector, it is
common practice to represent the distributions F by their respective den-
sity functions. When the density functions are indexed by a parameter θ in
a parameter space Θ, the model can be written as F = { f (x | θ), x ∈ X :
θ ∈ Θ}. In many cases, the n variables (X1 , X2 , . . . , Xn ) are assumed inde-
pendent conditional on θ and the statistical model can be written in terms
of the marginal densities of Xi , i = 1, 2, . . . , n:

F = f (x | θ) = Πni=1 fi (xi | θ) : θ ∈ Θ , x ∈ X,
and fi (· | θ) = f (· | θ), i = 1, 2, . . . , n, if additionally the variables Xi
are assumed to be identically distributed. The latter is often referred to as
random sampling.
Beyond the task of modeling and parametrization, classical inference
includes many methods to extract conclusions about the characteristics of
the model that best represents the population and tries to answer questions
like the following: (1) Are the data x compatible with a family F ? (2)
Assuming that the specification is correct and that the data are generated
from a model in the family F , what conclusions can be drawn about the
 1.1 The Classical Paradigm 3

parameter θ0 that indexes the distribution Fθ that “appropriately” describes

the phenomenon under study?
Classical methods – also known as frequentist methods – are evaluated
under the principle of repeated sampling, that is, with respect to the per-
formance under infinitely many hypothetical repetitions of the experiment
carried out under identical conditions. One of the aspects of this principle
is the use of frequencies as a measure of uncertainties, that is, a frequentist
interpretation of probability. See , Paulino et al. (2018, section 1.2), for a
review of this and other interpretations of probability.
In the case of parametric inference, in answer to question (2) above, we
need to consider first the question of point estimation, which, grosso modo,
is: Given a sample X = (X1 , X2 , . . . , Xn ), how should one “guess,” esti-
mate, or approximate the true value θ, through an estimator T (X1 , X2 , . . . ,
Xn ). The estimator should have the desired properties such as unbiasedness,
consistency, sufficiency, efficiency, etc.
For example, with X ≡ Rn , the estimator T (X1 , X2 , . . . , Xn ) based on a
random sample is said to be centered or unbiased if
Z
E{T | θ} = T (x1 , x2 , . . . , xn )Πni=1 f (xi | θ) dx1 dx2 . . . dxn = θ, ∀θ ∈ Θ.
Rn

This is a property related to the principle of repeated sampling, as can be

seen by the fact that it includes integration over the sample space (in this
case Rn ). Considering this entire space is only relevant if one imagines in-
finitely many repetitions of the sampling process or observations of the n
random variables (X1 , X2 , . . . , Xn ). The same applies when one considers
other criteria for evaluation of estimators within the classical paradigm. In
other words, implicit in the principle of repeated sampling is a considera-
tion of what might happen in the entire sample space.
Parametric inference often takes the form of confidence intervals. In-
stead of proposing a single value for θ, one indicates an interval whose
endpoints are a function of the sample,
(T ∗ (X1 , X2 , . . . , Xn ), T ∗∗ (X1 , X2 , . . . , Xn )),
and which covers the true parameter value with a certain probability, prefer-
ably a high probability (typically referred to as the confidence level),
P{T ∗ (X1 , X2 , . . . , Xn ) < θ < T ∗∗ (X1 , X2 , . . . , Xn ) | θ} = 1 − α,
0 < α < 1. This expression pre-experimentally translates a probability
of covering the unknown value θ to a random interval (T ∗ , T ∗∗ ) whose
4 Bayesian Inference

lower and upper limits are functions of (X1 , X2 , . . . , Xn ) and, therefore, ran-
dom variables. However, once a specific sample is observed (i.e., post-
experimentally) as n real values, (x1 , x2 , . . . , xn ), this becomes a specific
interval on the real line (now with real numbers as lower and upper limits).
(T ∗ (x1 , x2 , . . . , xn ), T ∗∗ (x1 , x2 , . . . , xn )),
and the probability
P{∗ T (x1 , x2 , . . . , xn ) < θ < T ∗∗ (x1 , x2 , . . . , xn ) | θ} = 1 − α,
0 < α < 1, is no longer meaningful. In fact, once θ has an unknown, but
fixed, value, this probability can only be 1 or 0, depending upon whether
the true value of θ is or is not in the real interval
(T ∗ (x1 , x2 , . . . , xn ), T ∗∗ (x1 , x2 , . . . , xn )).
Of course, since θ is unknown, the investigator does not know which situ-
ation applies. However, a classical statistician accepts the frequentist inter-
pretation of probability and invokes the principle of repeated sampling in
the following way: If one imagines a repetition of the sampling and infer-
ence process (each sample with n observations) a large number of times,
then in (1 − α) 100% of the repetitions the numerical interval will include
the value of θ.
Another instance of classical statistical inference is a parametric hypoth-
esis test. In the course of scientific investigation one frequently encounters,
in the context of a certain theory, the concept of a hypothesis about the
value of one (or multiple) parameter(s), for example in the symbols
H0 : θ = θ0 .
This raises the following fundamental question: Do the data (x1 , x2 , . . . , xn )
support or not support the proposed hypothesis? This hypothesis is tradi-
tionally referred to as the null hypothesis. Also here the classical solution
is again based on the principle of repeated sampling if one follows the
Neyman–Pearson theory. It aims to find a rejection region W (critical re-
gion) defined as a subset of the sample space, W ⊂ X, such that
(X1 , X2 , . . . , Xn ) ∈ W ⇒ rejection of H0 ,

(X1 , X2 , . . . , Xn ) < W ⇒ fail to reject H0 .

The approach aims to control the probability of a type-I error,
P{(X1 , X2 , . . . , Xn ) ∈ W | H0 is true},
 1.2 The Bayesian Paradigm 5

and minimize the probability of a type-II error,

P{(X1 , X2 , . . . , Xn ) < W | H0 is false}.
What does it mean that the critical region is associated with a type-I er-
ror, equal to, for example, 0.05? The investigator can not know whether a
false or true hypothesis is being rejected when a particular observation falls
into the critical region and the hypothesis is thus rejected. However, being a
classical statistician the investigator is convinced that under a large number
of repetitions and if the hypothesis were true, then only in 5% of the cases
would the observation fall into the rejection region. What does it mean that
the critical region is associated with a type-II error equal to, say 0.10? Sim-
ilarly, when a particular observation is not in the rejection region and thus
the hypothesis is not rejected, then the investigator cannot know whether
a true or false hypothesis is being accepted. Being a classical statistician,
the investigator can affirm that under a large number of repetitions of the
entire process and if the hypothesis were in fact false, only in 10% of the
cases would the observation not fall into the rejection region.
In the following discussion, it is assumed that the reader is familiar with
at least the most elementary aspects of how classical inference approaches
estimation and hypothesis testing, which is therefore not discussed here in
further detail.

1.2 The Bayesian Paradigm

For Lindley, the substitution of the classical paradigm by the Bayesian
paradigm represents a true scientific revolution in the sense of Kuhn (1962)
The initial seed for the Bayesian approach to inference problems was planted
by Richard Price when, in 1763, he posthumously published the work of
Rev. Thomas Bayes titled An Essay towards Solving a Problem in the Doc-
trine of Chances. An interpretation of probability as a degree of belief –
fundamental in the Bayesian philosophy – has a long history, including J.
Bernoulli, in 1713, with his work Ars Conjectandi. One of the first authors
to define probabilities as a degree of beliefs in the truth of a given proposi-
tion was De Morgan, in Formal Logic, in 1847, who stated: (1) probability
is identified as a degree of belief; (2) the degrees of belief can be mea-
sured; and (3) these degrees of belief can be identified with a certain set of
judgments. The idea of coherence of a system of degrees of belief seems
to be due to Ramsey, for whom the behavior of an individual when betting
on the truth of a given proposition is associated with the degree of belief
that the individual attaches to it. If an individual states odds or possibilities
6 Bayesian Inference

(chances) – in favor of the truth or untruth – as r : s, then the degree of be-

lief in the proposition is, for this individual, r/(r + s). For Ramsey, no set of
bets in given propositions is admissible for a coherent individual if it would
lead to certain loss. The strongest exponent of the concept of personal prob-
abilities is, however, de Finetti. In discussing the Bayesian paradigm and its
application to statistics, one must also cite Harold Jeffreys, who, reacting
to the predominantly classical position in the middle of the century, besides
inviting disapproval, managed to resurrect Bayesianism, giving it a logical
basis and putting forward solutions to statistical inference problems in his
time. From there the number of Bayesians grew rapidly and it becomes
impossible to mention all but the most influential – perhaps Good, Savage,
and Lindley.
The well-known Bayes’ theorem is a proposition about conditional prob-
abilities. It is simply probability calculus and is thus not subject to any
doubts. Only the application to statistical inference problems is subject to
some controversy. It obviously plays a central role in Bayesian inference,
which is fundamentally different from classical inference. In the classical
model, the parameter θ, θ ∈ Θ, is an unknown but fixed quantity, i.e., it is a
particular value that indexes the sampling model or family of distributions
F that “appropriately” describes the process or physical system that gener-
ates the data. In the Bayesian model, the parameter θ, θ ∈ Θ, is treated as an
unobservable random variable. In the Bayesian view, any unknown quan-
tity – in this case, the parameter θ – is uncertain and all uncertainties are
described in terms of a probability model. Related to this view, Bayesians
would argue that initial information or a priori information – prior or ex-
ternal to the particular experiment, but too important to be ignored – must
be translated into a probability model for θ, say h(θ), and referred to as the
prior distribution. The elicitation and interpretation of prior distributions
are some of the most controversial aspects of Bayesian theory.
The family F is also part of the Bayesian model; that is, the sampling
model is a common part of the classical and the Bayesian paradigms, ex-
cept that in the latter the elements f (x | θ) of F are in general assumed to
also have a subjective interpretation, similar to h(θ).
The discussion of prior distributions illustrates some aspects of the dis-
agreement between Bayesian and classical statisticians. For the earlier,
Berger, for example, the subjective choice of the family F is often con-
sidered a more drastic use of prior information than the use of prior dis-
tributions. And some would add: In the process of modeling, a classi-
cal statistician uses prior information, albeit in a very informal manner.
Such informal use of prior information is seen critically under a Bayesian
 1.2 The Bayesian Paradigm 7

paradigm, which would require that initial or prior information of an in-

vestigator needs to be formally stated as a probability distribution on the
random variable θ. Classical statisticians, for example, Lehmann, see an
important difference between the modeling of F and the specification of
h(θ). In the earlier case one has a data set x = (x1 , x2 , . . . , xn ) that is gener-
ated by a member of F and can be used to test the assumed distribution.
To understand the Bayesian point of view, recall that for a classical
statistician all problems that involve a binomial random variable X can be
reduced to a Bernoulli model with an unknown parameter θ that represents
a “success” probability. For Bayesians, each problem is unique and has its
own real context where θ is an important quantity about which there is, in
general, some level of knowledge that might vary from problem to problem
and investigator to investigator. Thus, the probability model that captures
this variability is based on a priori information and is specific to a given
problem and a given investigator. In fact, a priori information includes per-
sonal judgements and experiences of most diverse types, resulting from in
general not replicable situations, and can thus only be formalized in sub-
jective terms. This formalism requires that the investigator comply with
coherence or consistency conditions that permit the use of probability cal-
culus. However, different investigators can in general use different prior
distributions for the same parameter without violating coherence condi-
tions.
Assume that we observe X = x and are given some f (x | θ) ∈ F and a
prior distribution h(θ). Then Bayes’ theorem implies3
f (x | θ)h(θ)
h(θ | x) = R , θ ∈ Θ, (1.1)
θ
f (x | θ)h(θ) dθ
where h(θ | x) is the posterior distribution of θ after observing X = x.
Here, the initial information of the investigator is characterized by h(θ),
and modified with the observed data by being updated to h(θ | x). The
denominator in (1.1), denoted f (x), is the marginal (or prior predictive)
distribution for X; that is, for an observation of X whatever the value of θ.
The concept of a likelihood function appears in the context of classical
inference, and is not less important in the Bayesian context. Regarding its
definition, it is convenient to distinguish between the discrete and continu-
ous cases (Kempthorn and Folks, 1971), but both cases lead to the function
3
Easily adapted if x were a vector or if the parameter space were discrete.
8 Bayesian Inference

of θ,
L(θ | x) = k f (x | θ), θ ∈ Θ or
(1.2)
L(θ | x1 , . . . , xn ) = kΠi f (xi | θ), θ ∈ Θ,
which expresses for every θ ∈ Θ its likelihood or plausibility when X = x
or (X1 = x1 , X2 = x2 , . . . , Xn = xn ) is observed. The symbol k represents a
factor that does not depend on θ. The likelihood function – it is not a prob-
ability, and therefore, for example, it is not meaningful to add likelihoods –
plays an important role in Bayes’ theorem as it is the factor through which
the data, x, updates prior knowledge about θ; that is, the likelihood can be
interpreted as quantifying the information about θ that is provided by the
data x.
In summary, for a Bayesian the posterior distribution contains, by way
of Bayes’ theorem, all available information about a parameter:
prior information + information from the sample.
It follows that all Bayesian inference is based on h(θ | x) [or h(θ | x1 , x2 ,
. . . , xn )].
When θ is a parameter vector, that is, θ = (γ, φ) ∈ Γ × Φ, it can be
the case that the desired inference is restricted to a subvector of θ, say
γ. In this case, in contrast to the classical paradigm, the elimination of
the nuisance parameter φ under the Bayesian paradigm follows always the
same principle, namely through the marginalization of the joint posterior
distribution,
Z Z
h(γ | x) = h(γ, φ | x)dφ = h(γ | φ, x)h(φ | x)dφ. (1.3)
Φ Φ

Possible difficulties in the analytic evaluation of the marginal disappear

when γ and φ are a priori independent and the likelihood function factors
into L(θ | x) = L1 (γ | x) × L2 (φ | x), leading to h(γ | x) ∝ h(γ)L1 (γ | x).

1.3 Bayesian Inference

In the Bayesian approach, it is convenient to distinguish between two ob-
jectives: (1) inference about unknown parameters θ, and (2) inference about
future data (prediction).

1.3.1 Parametric Inference

In the case of inference on parameters, we find a certain agreement – at
least superficially – between classical and Bayesian objectives, although
 1.3 Bayesian Inference 9

in the implementation the two approaches differ. On one side, classical in-
ference is based on probabilities associated with different samples, x, that
could be observed under some fixed but unknown value of θ. That is, in-
ference is based on sampling distributions that “weigh” probabilistically
the values that a variable X or statistic T (X) can assume across the sample
space. On the other hand, Bayesian inference is based on subjective prob-
abilities or a posteriori credibilities associated with different values of the
parameter θ and conditional on the particular observed x value. The main
point is that x is fixed and known and θ is uncertain.
For example, once x is observed, a Bayesian being asked about the
hypothesis {θ ≤ 0.5} would directly address the question by calculating
P(θ ≤ 0.5 | x) based on h(θ | x), i.e., without leaving probability cal-
culus. In contrast, a classical statistician would not directly answer the
question. Stating, for example, that the hypothesis H0 : θ ≤ 0.5 is re-
jected at significance level 5% does not mean that its probability is less
than 0.05, but that if the hypothesis H0 were true, (i.e., if in fact θ ≤ 0.5),
then the probability of X falling into a given rejection region W would be
P(X ∈ W | θ ≤ 0.5) < 0.05, and if in fact x ∈ W, then the hypothesis is
rejected.
In O’Hagan’s words (O’Hagan, 2010), while a Bayesian can state prob-
abilities about the parameters, which are considered random variables, this
is not possible for a classical statistician, who uses probabilities on data
and not on parameters and needs to restate such probabilities such that they
seem to say something about the parameter. The question is also related to a
different view of the sample space. For a classical statistician, the concept
of the sample space is fundamental, as repeated sampling would explore
the entire space. A Bayesian would start by objecting to the reliance on
repeated sampling and would assert that only the actually observed value
x is of interest and not the space that x belongs to, which could be totally
arbitrary, and which contains, besides x, observations that could have been
observed, but were not.4
In estimation problems a classical statistician has several alternatives for
functions of the data – estimators – whose sampling properties are inves-
tigated under different perspectives (consistency, unbiasedness, etc.). For
a Bayesian there is only one estimator, which specifically is the posterior
distribution h(θ | x). One can, of course, summarize this distribution in dif-
4
The irrelevance of the sample space also leads to the same issue about stopping rules,
something which Mayo and Kruse (2001) note, recalling Armitage, could cause
problems for Bayesians.
10 Bayesian Inference

ferent ways, using mode, mean, median, or variance. But this is unrelated
to the problem facing a classical statistician, who has to find a so-called op-
timal estimator. For a Bayesian such a problem only exists in the context of
decision theory, an area in which the Bayesian view has a clear advantage
over the classical view. Related to this, Savage claims that in past decades
the central problem in the face of uncertainty is shifting from which infer-
ence one should report, to which decision should be taken. As individual
decisions have been considered outdated by some philosophers, we have
also recently seen a resurgence of the Bayesian approach in the context of
group decisions.
Under a Bayesian approach, confidence intervals are replaced by credi-
ble intervals (or regions). Given x, and once a posterior distribution is deter-
mined, one finds a credible interval for a parameter θ (assume, for the mo-
ment, a scalar). The interval is formed by two values in θ, say [θ(x), θ̄(x)],
or simpler, (θ, θ̄), such that
Z θ
P(θ < θ < θ̄ | x) = h(θ | x) dθ = 1 − α, (1.4)
θ

where 1 − α (usually 0.90, 0.95, or 0.99) is the desired level of credibility.

If Θ = (−∞, +∞), then one straightforward way of constructing a (in this
case, central) credible interval is based on tails of the posterior distribution
such that
Z θ Z +∞
α
h(θ | x) dθ = h(θ | x) dθ = . (1.5)
−∞ θ 2
Equation (1.4) has an awkward implication: The interval (θ, θ) is not
unique. It could even happen that the values θ in the reported interval have
less credibility than values θ outside the same interval. Therefore, to pro-
ceed with the choice of an interval that satisfies (1.4) and at the same time
is of minimum size, Bayesians prefer to work with HPD (highest posterior
density) credible sets A = {θ : h(θ | x1 , x2 , . . . , xn ) ≥ k(α)}, where k(α) is
the largest real number such that P(A) ≥ 1 − α. For a unimodal posterior,
the set becomes a HPD credible interval.
Credible sets have a direct interpretation in terms of probability. The
same is not true for confidence intervals, which are based on a probability
not related to θ, but rather a probability related to the data; more specif-
ically, they are random intervals based on a generic sample, and which
after observing a particular sample become a confidence of covering the
unknown value θ by the resulting numerical interval. In general, this can
not be interpreted as a probability or credibility about θ. Besides other crit-
 1.3 Bayesian Inference 11

ical aspects of the theory of confidence intervals (or regions), there are the
ironical comments of Lindley (1990), who says to know various axioms of
probability – for example, those due to Savage, de Finetti, or Kolmogorov
– but no axiomatic definition of confidence.
For example, when a Bayesian investigates a composite hypothesis H0 :
θ ∈ Θ0 versus a composite alternative H1 : θ ∈ Θ1 , with Θ0 ∩ Θ1 =
∅, Θ0 ∪ Θ1 = θ, she or he uses expressions in terms of probabilities on θ.
When the investigator possesses a distribution h(θ), θ ∈ Θ, representing the
initial credibility attributed to different parameter values, her or his prior
probabilities of the competing hypotheses are determined by
Z Z
P(Θ0 ) = h(θ) dθ, P(Θ1 ) = h(θ) dθ .
Θ0 Θ1

The ratio P(Θ0 )/P(Θ1 ) is known as the prior odds for H0 versus H1 . Af-
ter the experiment resulting in the observations x, and after determining
h(θ | x), a Bayesian statistician calculates the corresponding posterior prob-
abilities
Z Z
P(Θ0 | x) = h(θ | x) dθ, P(Θ1 | x) = h(θ | x) dθ,
Θ0 Θ1

and usually also the posterior odds for H0 versus H1 , that is, P(Θ0 | x)/P(Θ1 |
x). One can therefore say that in the Bayesian framework the inference out-
come is not so much the acceptance or rejection of a the hypothesis H0 –
as is the case in the Neyman–Pearson framework – but rather the updating
of the plausibility that is attributed to the competing hypotheses. Bayesian
inference can be described as a comparison of posterior odds versus prior
odds through
P(Θ0 | x)/P(Θ1 | x)
B(x) = , (1.6)
P(Θ0 )/P(Θ1 )
which is known as the Bayes factor in favor of H0 (or Θ0 ). The Bayes factor
quantifies the evidence in the data x in favor of H0 . Of course, the larger
the Bayes factor, the larger is the increase of the posterior odds relative
to the prior odds and thus the support that the data give to the hypothesis
H0 . In general, the Bayes factor depends on the prior distribution and can
be expressed as a ratio of likelihoods weighted by the prior distributions
conditional on the respective hypothesis on Θ0 and Θ1 (see also Paulino et
al., 2003). In this sense one can not say that the Bayes factor is a measure
of the support for H0 based on only the data.
When the hypothesis about θ is specific to the point of being defined
as H0 : θ = θ0 , evaluation of a Bayes factor or of posterior odds requires
12 Bayesian Inference

that the prior distribution be consistent with this conjecture in the sense
of avoiding zero probability for H0 , implying in general that the prior is a
mixture model. This implication is considered natural by Bayesians such
as Jeffreys, with the argument that a prior distribution needs to integrate
probabilistic judgments that are inherent in the statement of competing hy-
potheses, which in this case attribute some importance to θ0 , as opposed to
other values of θ.
Other Bayesians such as Lindley and Zellner advocate a different ap-
proach, with a certain formal analogy with classical significance tests, in a
way in which the statement of point hypotheses does not interfere with the
prior distribution. Their approach can be described as a quantification of
relative plausibility under the posterior for the value θ0 , via the evaluation
of P = P(θ < R0 (x) | x), where R0 (x) = {θ ∈ Θ : h(θ | x) ≥ h(θ0 | x)} is
the smallest HPD region that contains θ0 . Large (small) values of the pos-
terior relative plausibility P for H0 are evidence in favor of (against) this
hypothesis.
The fundamental tool of the Bayesian approach and the way the joint
model M = { f (x | θ)h(θ), x ∈ X, θ ∈ Θ} is used in the implementation
of inference already suggest that the question of evaluating the adequacy
of a conjectured model in absolute terms might not have an answer in the
sense of Popper (reject/not reject) of the type that is guaranteed by classical
goodness-of-fit tests.
Bayes factors can be used if it is possible to extend the model M (or
parts of it) to a larger family that contains the true model as an unknown
quantity, and that allows comparing models within it. Otherwise, one can
only define various measures of model adequacy for a relative analysis of
a reference model in the context of a class of suitably defined competing
models (see Chapter 5). The unsatisfactory nature of such options has lead
some statisticians to defend a Bayesian approach only when the underlying
model is not put into question, a condition which Gillies (2001) refers to
as fixity of the theoretical framework.

1.3.2 Predictive Inference

Many Bayesians believe that inference should not be restricted to state-
ments about unobservable parameters. They note that parametric inference
is awkward in that actual values are rarely known for parameters and there-
fore such inference can rarely be compared with reality. For Bayesians like
Lindley, the most fundamental problem is to start with a set of observations
(x1 , x2 , . . . , xn ) (yesterday) and infer conclusions, in terms of (subjective)
 1.4 Conclusion 13

probability, about a set of future observations (xn+1 , xn+2 , . . . , xn+m ) (tomor-

row).
For easier exposition we assume m = 1 and that the n + 1 random vari-
ables X1 , X2 , . . . , Xn , Xn+1 are independent and identically distributed, given
θ with probability density function f (x | θ). The problem is to predict the
random variable Xn+1 after observing (X1 = x1 , X2 = x2 , . . . , Xn = xn ).
Trying to predict Xn+1 with sampling model f (x | θ) we face two sources
of randomness: (1) uncertainty that has to do with Xn+1 being a random
variable; (2) the impact of the uncertainty on θ. For example, if we esti-
mate θ with the maximum likelihood estimator Rb θ̂ = θ̂(x1 , x2 , . . . , xn ) and
write P(a < Xn+1 < b | x1 , x2 , . . . , xn )  a f (x | θ̂) dx, as estimate of the
probability of the event a < Xn+1 < b, then this expression ignores the ran-
domness in the substitution of the parameter by its estimate. However, both
types of randomness need to enter the prediction. The method of substitut-
ing an estimate for an unknown parameter in the sampling model (plug-in
procedure) should thus be seen with some caveat.
Although the classical solution of the prediction problem involves much
more than this (Amaral Turkman, 1980), it still could be said that the
Bayesian solution is much cleaner. If one has only prior information, for-
malized as a prior distribution h(θ), the natural tool to use is the already dis-
cussed marginal or prior predictive distribution f (x). The more interesting
case is when one observes x = (X1 = x1 , X2 = x2 , . . . , Xn = xn ) and wishes
to predict Xn+1 , assuming that conditional on θ the latter is independent of
the previous observations [the problem of predicting (Xn+1 , Xn+2 . . . , Xn+m )
is not very different].
R Using a completely probabilistic argument we have
f (xn+1 | x) = θ f (xn+1 | θ)h(θ | x) dθ, where the posterior distribution takes
the place of the prior, as representing the information given the sample.
One can then report summaries of this predictive distribution, including
probabilities of any region in the sample space for Xn+1 or values, a = a(x)
and b = b(x) for any pre-specified probability P(a < Xn+1 < b | x) =
Rb
a
f (xn+1 | x) dxn+1 , which then determine a prediction interval (of HPD
type if desired).

1.4 Conclusion
In summary, from a Bayesian point of view:
• The classical approach to statistical inference proceeds by arguments of
an inductive type, such as, the notion of confidence intervals, which do
not have a direct interpretation as probabilities. The difficulty or impos-
14 Bayesian Inference

sibility of making inference with a direct probabilistic interpretation –

as the parameter θ itself is not even considered as a random variable – is
strongly criticized by Jaynes (2003).
• Under a Bayesian approach, all inference can be derived from a logical
application of probability calculus. Bayesian statistical inference does
not rely on any results that could not be derived from the rules of prob-
ability calculus, in particular Bayes’ theorem. As O’Hagan (2010) puts
it: “Probability theory is a completely self-consistent system. Any ques-
tion of probabilities has one and only one answer, although there may
be many ways to derive it.”
Short of taking an extremist position, it is convenient to recall statements
like that of Dawid (1985), who, besides confessing a clear preference for
Bayesian theory comments that no statistical theory, Bayesian or not, can
ever be entirely satisfactory. A position that some statisticians would argue
for today is not the exclusively Bayesian option of authors like Savage, but
rather an eclectic position shared by Wasserman (2004) when he argues
that, in summary, combining prior judgments with data is naturally done by
Bayesian methods, but to construct methods that guarantee good results in
the long run under repeated observations, one needs to resort to frequentist
methods.

Problems
1.1 Suppose there are N cable cars in San Francisco, numbered 1 to N. You
don’t know the value of N, so this is the unknown parameter. Your prior
distribution on N is a geometric distribution with mean 100; that is
!N−1
1 99
h(N) = ,
100 100
N = 1, 2, . . . . You see a cable car at random. It is numbered x = 203. Assume
that x =number on a randomly picked car and has the probability distribution
f (x | N) = 1/N for x = 1, . . . , N, and f (x | N) = 0 for x > N.
a. Find the posterior distribution h(N | x). Find the Bayes estimate of N, i.e.,
the posterior mean of N, and the posterior standard deviation of N (use
P
results for a geometric series ∞ x
x=k a ; or use a numerical approximation).
b. Find a 95% HPD credible interval for N (you will not be able to exactly
match the 95% – get as close as possible).
1.2 Recording the number of bacteria (yi ) found in n = 6 water samples (of the
P
same volume), we find yi = 2, 3, 8, 6, 4, and 1 (you might need S = yi =
 Problems 15

24). It is thought reasonable to assume that yi follows a Poisson distribution

with mean θ, i = 1, . . . , n. √
Also suppose that the prior h(θ) ∝ 1/ θ is used for the parameter θ (this is
a so-called improper prior – see Chapter 2 for more discussion).
a. Find the posterior distribution h(θ | y), and show how you will obtain a
95% credible interval for θ.
b. Suppose that you are now told that only non-zero outcomes were recorded
in the above experiment, and therefore the correct distribution for yi , i =
1, ..., 6, is the truncated Poisson given by
e−θ θy
f (y | θ) = , y = 1, 2, ...
y!(1 − e−θ )
(i) Write down the likelihood function, and the posterior (using the
same prior as before) up to a constant.
(ii) Find a 95% credible interval using numerical integration (the pre-
vious posterior is no longer in a simple form that allows analytic
evaluation of credible intervals).
1.3 Assume that the waiting time for a bus follows an exponential distribution
with parameter θ. We have a single observation, x = 3. Assume that θ can
only take one of 5 values, Θ = {1, 2, 3, 4, 5}, with prior probabilities h(θ) ∝
1/θ, θ ∈ Θ.
a. Find the posterior mode (MAP), the posterior standard deviation SD(θ |
x), and the (frequentist) standard error of the estimator, i.e., SD(MAP | θ).
b. Find a 60% HPD credible interval A = (θ0 , θ1 ), i.e., find the shortest
interval A with P(A | x) ≥ 0.6.
c. Find h(θ | x, θ ≥ 2), i.e., find the posterior distribution for θ when we
know that θ ≥ 2.
d. Find the posterior mode MAP0 conditional on θ ≥ 2, the conditional pos-
terior standard deviation SD(θ | x, θ ≥ 2), and the (frequentist) standard
error of the estimator, SD(MAP0 | θ).
Compare with the answers under item (a) and comment.
e. How would you justify the choice of the prior distribution h(θ) ∝ 1/θ?
1.4 Your friend always uses a certain coin to bet “heads or tails”5 and you have
doubts whether the coin is unbiased or not. Let θ denote the probability of a
head. You want to test H1 : θ < 0.5 versus H2 : θ = 0.5 versus H3 : θ > 0.5.
Assign a prior probability 1/2 that the coin is unbiased and equal probability
to the other two hypotheses. That is, p(H2 ) = 0.5 and p(H1 ) = p(H3 ) = 0.25.
The prior distribution for θ under H1 and H3 is uniform. That is, h(θ | H1 ) =
U(0, 0.5) and h(θ | H3 ) = U(0.5, 1). Assume you observe n = 1 coin toss.
Let x1 ∈ {0, 1} denote an indicator of “head.”
5
The labels “head” and “tail” refer to the two sides of a 25 cent coin.
16 Bayesian Inference

a. Find the predictive distribution (i.e., the marginal distribution) for the first
toss, under each of the three hypotheses, i.e., find p(x1 = 1 | H1 ), p(x1 =
1 | H2 ), and p(x1 = 1 | H3 ).
b. Find the predictive distribution for the first toss, p(x1 = 1).
c. Still using the data x1 from one coin toss, given x1 = 1,
(i) find the Bayes factors for H1 versus H2 and for H3 versus H2 ;
(ii) find the posterior probability that the coin is unbiased.
(iii) In general it is not meaningful to compute a Bayes factor with a
non-informative uniform prior h(θ) ∝ c (because the choice of c is
arbitrary). Why is it okay here?
 2

Representation of Prior Information

1
The mechanics of the inference process require that all basic ingredients
be properly specified. The first one is a sampling model that can explain
(with more or less accuracy) the data as they arise from some experiment
or observational process and which we wish to analyze. This model en-
compasses a set of unknown aspects about which one could have prior
information that should be included in the analysis, no matter how vague
or significant this information is, and that need to be somehow represented
and quantified.
The process of representing prior information is often complicated by
the involvement of subjective elements that need to be elicited. We address
this here for two cases:
• The first case is when no prior information is available, neither objective
nor subjective (sometimes referred to as “a priori ignorance”) or when
prior knowledge is of little significance relative to the information from
the sampling model (“vague” or “diffuse” prior information). We review
some of the principal methods to derive prior distributions that are in
some sense very little informative, and which commonly are referred to
as non-informative priors.
• The second case assumes a convenient parametric family and then chooses
a member of that family using carefully elicited summary measures for
the desired distribution. An example of this elicitation process arises in
the medical inference problem that is discussed by Paulino et al (2003).
This is the context of so-called natural conjugate priors. Such priors
1
This material will be published by Cambridge University Press as Computational
Bayesian Statistics, by M.A. Amaral Turkman, C.D. Paulino, and P. Müller
(https://tinyurl.com/CompBayes). This pre-publication version is free to view
and download for personal use only. Not for re-distribution, re-sale, or use in derivative
works. c Maria Antónia Amaral Turkman, Carlos Daniel Paulino & Peter Müller, 2019.
18 Representation of Prior Information

can also be used to generate improper non-informative priors. In this

sense it is closely related to the first case.
For more discussion about the problem of prior specification and other
methods to generate vague priors or to elicit subjective priors, see O’Hagan
(2010), Kass and Wasserman (1996), and Paulino et al (2018).

2.1 Non-Informative Priors

Non-informative priors were widely interpreted as formal representations
of ignorance, but today the tendency (motivated by the lack of unique ob-
jective representations of ignorance) is to accept them as conventional de-
fault choices that can be used when insufficient prior information makes
the elicitation of an adequate subjective prior difficult. Unrelated to inter-
pretation, this type of distributions can still play a role as a reference, even
in the presence of strong prior beliefs:
• to derive posterior beliefs for someone who starts with little knowledge,
i.e., when the sample provides overwhelming information about the pa-
rameters, and it is difficult to subjectively determine a reasonable distri-
bution;
• to allow the comparison with classical inference that “only” uses infor-
mation from the sample (all or part of it);
• to evaluate the impact on inference of a subjective prior distribution that
describes actually available information, by comparison with inference
under a default prior.
We review some of the most widely used arguments to construct such prior
distributions.

The Bayes–Laplace Method

Based on the “principle of insufficient reason,” this method, in the absence
of prior information, uses the idea of equiprobability. Depending on the
cardinality of Θ, the argument leads to a discrete uniform or a continuous
uniform prior.
In the case of a finite number of values for θ, e.g., Θ = {θ1 , . . . , θk },
the argument leads to a distribution h(θ) = 1/k, θ ∈ Θ. However, the
same is not true in other situations. If Θ is countably infinite, the resulting
distribution is improper, and the same applies when Θ is an unbounded
uncountable infinite set, which is inconvenient for investigators who do
 2.1 Non-Informative Priors 19

not like un-normalized measures (even if this does not necessarily prevent
the use of Bayes’ theorem, as the posterior distribution, the source of all
inference, might often still be proper).
Another and perhaps more serious critique against the argument that the
lack of information, which some refer to as ignorance, should be repre-
sented by a uniform distribution, is the fact that this is not invariant with
respect to non-linear transformations, thus leading to contradictions. To il-
lustrate, take the model {Ber(θ), θ ∈ (0, 1)} which is part of the exponential
family with natural parameter ψ = ln [θ/(1 − θ)] ∈ R. The use of uniform
distributions for θ (proper) and ψ (improper) is probabilistically inconsis-
tent. In fact, θ ∼ U(0, 1) ≡ Be(1, 1) is equivalent to the reduced logistic
eψ
distribution for ψ, with density h(ψ) = (1+e ψ )2 , ψ ∈ R.

In general, letting ψ = ψ(θ) denote a one-to-one transformation of a

real-valued parameter θ with prior density h(θ), the implied prior on ψ is
  dθ
h(ψ) = h θ(ψ) . (2.1)
dψ
The latter is not uniform when h(θ) is uniform and the Jacobian depends on
ψ, as is the case for non-linear transformations, as in the previous example.

Jeffreys’ Prior
One of the approaches that ensure invariance under one-to-one transfor-
mations is the one proposed by Jeffreys. Jeffreys’ prior is based on Fisher
information for θ ∈ R, defined as
 ! 
 ∂ ln f (X | θ) 2 
I(θ) = E   θ .
∂θ
1
For the univariate case Jeffreys’ prior is defined by h(θ) ∝ [I(θ)] 2 . The fact
that for any real-valued one-to-one transformation ψ of θ ∈ R,
!2
dθ
I(ψ) = I(θ(ψ)) ,
dψ
shows that Jeffreys’ prior for the univariate case has the desired invariance
property and thus, ensures invariance of inference under arbitrary transfor-
mations of the parameter space (that is, under reparametrization).
To understand the non-informative nature of Jeffreys’ prior, note that I(θ)
grows with the square of the change (in expectation over the sample space)
with respect to θ of ln f (X | θ). Also note that the better the model can
differentiate θ from θ+dθ, the larger I(θ), that is, the sample information on
20 Representation of Prior Information

θ. Therefore, considering values of θ with larger (smaller) I(θ) to be a priori

more (less) plausible maximally reduces the impact of prior information.
This explains the non-informative nature of Jeffreys’ prior. And it can be
claimed to be objective as it is derived in an automatic fashion from the
assumed generative model for the data.

Example 2.1 Consider a sampling model with likelihood function

L(θ | x, n) = k θ x (1 − θ)n−x , θ ∈ (0, 1),
where k does not depend on θ. If n is fixed and k = C nx , the model reduces to
a binomial model, X | n, θ ∼ Bi(n, θ), with mean nθ and I(θ) ∝ θ−1 (1 − θ)−1 ,
and thus Jeffreys’ prior
h(θ) ∝ θ−1/2 (1 − θ)−1/2 , θ ∈ (0, 1) i.e., θ ∼ Be(1/2, 1/2),
which by the earlier transformation√argument corresponds to a uniform
distribution U(0, 2π) for ψ = arc sin θ.
If, alternatively, x is fixed and k = C n−1
x−1 we get the negative binomial
model N−x | x, θ ∼ NBin(x, θ), with mean x(1−θ)/θ and I(θ) ∝ θ−2 (1−θ)−1 ,
implying Jeffreys’ prior
h(θ) ∝ θ−1 (1 − θ)−1/2 , θ ∈ (0, 1),
which corresponds to an improper prior distribution that we shall denote
as “Be(0, 1/2),” which is consistent with a “uniform” distribution for
√
1− 1−θ
ψ = ln √ .
1+ 1−θ

Example 2.1 highlights that Jeffreys’ prior, by definition, entirely depends
on the sampling model and not only on the kernel. This dependence on the
sample space motivates some to be vehemently critical of Jeffreys’ prior,
especially because this can lead to different posterior inferences, with re-
spect to the same parameter, depending on the type of the observed experi-
ment; for example, direct binomial or inverse binomial sampling in Exam-
ple 2.1. However, differences might be quite minor for moderate sample
sizes, as the example illustrates. Others argue that such dependence is le-
gitimate because of the vague prior information, which Jeffreys’ prior aims
to represent – that is, the dependence should be seen as a function of the
information that is associated with different types of sampling plans, and
not in absolute terms.
 2.1 Non-Informative Priors 21

The application of Jeffreys’ rule for univariate location parameters, { f (x |

θ) = g(x − θ), θ ∈ Θ ⊆ R} (e.g., a normal model with known variance),
leads to a continuous uniform distribution, which is invariant under linear
transformation (that is, under shifts) and improper if Θ is unbounded. If it
is applied for univariate scale parameters, { f (x | θ) = 1θ g(x/θ), θ ∈ Θ ⊆ R+ }
(e.g., a normal model with known mean), it leads to an improper distribu-
tion h(θ) ∝ θ−1 I(0,+∞) (θ), which is invariant under power transformations
(i.e., under rescaling). Here, invariance refers to the fact that the implied
prior for any ψ = θr is of the same type, h(ψ) ∝ ψ−1 , ψ > 0.
In multiparameter models, Jeffreys’ rule is based on the square root of
the determinant of the Fisher information matrix. However, due to unde-
sirable implications on the posterior distribution, the rule tends to be re-
placed, as even suggested by Jeffreys himself, by an assumption of a priori
independence between parameters (especially when they are of a different
nature) and the use of univariate Jeffreys’ rules for the marginal prior dis-
tributions. For example, in the model {N(µ, σ2 ) : µ ∈ R, σ2 ∈ R+ }, Jeffreys’
prior is
h(µ, σ2 ) ∝ σ−m , µ ∈ R, σ2 > 0,
with m = 3 when the bivariate rule is used and m = 2 when the mentioned
factorization of the joint prior for location, µ, and scale, σ2 is used.

Maximum entropy methods

The notion of entropy is borrowed from physics, where it is associated
with a measure of uncertainty, and was proposed by Jaynes (2003) as a
way to arrive at prior distributions that could represent a state of relative
ignorance. Such a distribution would have to correspond to a maximum
entropy.
Defining the entropy E of a distribution h(θ), θ ∈ Θ, as the expected
value E(h(θ)) = Eh [− ln h(θ)], it is easy to show that in the finite case when
Θ = {θ1 , . . . , θk } the maximum entropy distribution (that is, with maximum
uncertainty) is the discrete uniform h(θi ) = 1/k, i = 1, . . . , k, which has en-
tropy ln k. It suffices tomaximize theLagrange function defined by E(h(θ))
P
and the added term λ ki=1 h(θi ) − 1 , where λ is the Lagrange factor for
the restriction to a probability function.
Next, consider maximizing entropy subject to information that is repre-
sented as pre-specified values for moments or quantiles, e.g., of the form
E(g j (θ)) = u j , j = 1, . . . , m. The same procedure can be used (Lagrange
22 Representation of Prior Information

multipliers method), introducing the additional restrictions to get the ex-

pression
P
exp{ mj=1 λ j g j (θi )}
h(θi ) = Pk Pm ,
l=1 exp{ j=1 λ j g j (θl )}

where the m coefficients λ j arise from the corresponding restrictions.

Example 2.2 In the context of a discrete distribution, assume that Θ =

{1, . . . , k}, and that the median is fixed at one of the possible values, say q.
Thus, we have a restriction imposed with u1 = q and g1 (θ) being an indi-
P
cator for θ ≤ q, that is, given by qi=1 h(i) = 1/2. By the earlier expression


 eλ1
 eλ1 q+(k−q) , if i ≤ q
h(i) = 
 ,
 λ 1 , if q < i ≤ k
e 1 q+(k−q)

where eλ1 = (k − q)/q by the restriction on the median. We get then



 1
 2q , if i ≤ q
h(i) = 
 ,
 1 , if q < i ≤ k
2(k−q)

that is, a piecewise uniform distribution. 

In the case of Θ being a bounded interval on the real line, variational
calculus shows that the maximum entropy distribution is the continuous
uniform distribution which, as we already know, is not invariant under
all injective transformations, which creates problems when using entropy
E(h(θ)) as an absolute measure of uncertainty.
Based on the relationship between entropy and the Kullback–Leibler
measure of information in the discrete case, Jaynes (1968) redefined en-
tropy in the continuous case withh respect ito a non-informative reference
distribution h0 (θ) as E(h(θ)) = Eh − ln hh(θ)
0 (θ)
.
If we assume again initial information represented by restrictions as be-
fore, then variational calculus leads to a solution of the maximization prob-
lem which can then be expressed as
 m 

X
 


h(θ) ∝ h0 (θ) exp 
 λ j g j (θ)
 ,
 
j=1

where the multipliers λ j are obtained from the introduced restrictions.

Example 2.3 Assume that θ is a location parameter that is known to be

 2.2 Natural Conjugate Priors 23

positive, that is, Θ = (0, +∞), and to have mean u. Using as the non-
informative, translation invariant prior distribution the “uniform” distri-
bution on Θ, we get h(θ) ∝ exp(λ1 θ), θ > 0, which implies
h(θ) = −λ1 exp(λ1 θ) I(0,+∞) (θ),
with λ1 < 0, that is, an exponential distribution. Keeping in mind that
the fixed mean is −1/λ1 = u, we find the maximum entropy distribution
θ ∼ Exp(1/u). 

Example 2.4 Again, let θ be a location parameter, with Θ = R and assume

that E(θ) = u1 and Var(θ) = u2 . Using the same (improper) reference
distribution as before, we get h(θ) ∝ exp{λ1 θ + λ2 (θ − u1 )2 }, θ ∈ R.
Simple algebra gives
" !#2 " #
2 λ1 λ21
λ1 θ + λ2 (θ − u1 ) = λ2 θ − u1 − + λ1 u1 − .
2λ2 4λ2
Thus,
 " !#2 


 λ1 


h(θ) ∝ exp 
 λ θ − u −  ,
2λ2 
2 1
 

which is the kernel of a normal distribution with mean u1 − λ1 /(2λ2 ) and

variance −1/(2λ2 ), with λ2 < 0. Checking the two pre-specified moments,
we find that λ1 = 0 and λ2 = −1/(2u2 ) and recognize the maximum entropy
prior as θ ∼ N(u1 , u2 ). 

2.2 Natural Conjugate Priors

A parametric family, where one selects a member of the family in keeping
with elicited summaries, should ideally satisfy the following requirements:
• flexibility to accommodate the largest number of possible prior beliefs;
• interpretability that facilitates the process of summarizing its members;
• simplicity in analytical derivation of posterior and predictive distribu-
tions.
Bayesian updating becomes straightforward if we use a family of prior
distributions, H = {ha (θ) : a ∈ A}, where A is a set of values for the hy-
perparameters, and H is closed under sampling from (any element of)
F = { f (x | θ) : θ ∈ Θ}, that is, if
ha (θ) ∈ H ⇒ h(θ | x) ∝ ha (θ) f (x | θ) ∈ H.
24 Representation of Prior Information

Under these conditions, we call H a natural conjugate family for F .

In other words, the family H is said to be the natural conjugate of F if
L(θ | x) ≡ f (x | θ), for any x, is proportional to a member of H and H is
closed with respect to products, i.e., for any a0 , a1 ∈ A, there is a2 ∈ A
such that
ha0 (θ)ha1 (θ) ∝ ha2 (θ).

Example 2.5 Let x = (xi , i = 1, . . . , n) be an observation of a random

sample from a Bernoulli model Ber(θ), that is,
P P
xi
(1 − θ)n− i xi ,
f (x1 , . . . , xn | θ) = θ i

P P
which is proportional to the kernel of a Be( i xi +1, n− i xi +1) distribution
in θ, which is closed under products. The natural conjugate model for a
Bernoulli sampling model is therefore a family of beta distributions, with
well-known versatility. In summary,
X X
θ ∼ Be(a, b) ⇒ θ | x ∼ Be(A, B), A = a + xi , B = b + n − xi ,
i i

which shows how sampling information enters the easily computed poste-
rior distribution, through the number of successes and failures (that is, the
minimum sufficient statistic), additively with respect to the prior hyperpa-
rameters (a, b).
Since a, b > 0, prior information vanishes (relative to the information
in the likelihood) when a, b → 0. Thus a non-informative (or vague) prior
obtained from the natural conjugate family is the improper Haldane prior,
“Be(0, 0),” defined as h(θ) ∝ θ−1 (1 − θ)−1 , θ ∈ (0, 1), which corresponds
to the “uniform” prior for ψ = ln [θ/(1 − θ)] ∈ R. Consequently, the prior
distribution Be(a, b) can be interpreted as a posterior distribution resulting
from updating a non-informative prior based on a hypothetical sample of
size a + b with a successes. 

In conjugate families, the process of Bayesian updating, that is, the com-
bination of prior and sample information by Bayes’ theorem, is carried out
entirely within them. This allows us to symbolically represent the updating
by a transformation in the space A of hyperparameters, e.g.,
F
a ∈ A → A = a + (A − a) ∈ A.
in Example 2.5. The transformation shows the relative weights of the two
types of information and highlights the interpretative and analytical sim-
plicity of the Bayesian mechanism in the context of a natural conjugate
 2.2 Natural Conjugate Priors 25

family. In the above form, A − a represents the role of the sample infor-
mation in the updating of prior information, represented as a. This is illus-
trated in Example 2.5.

Example 2.6 In Example 2.5, assuming a random sample from a Geo(θ)

sampling model with density function f (xi | θ) = θ(1 − θ) xi , xi ∈ N0 ,
would still lead to the same natural conjugate family and the same non-
informative distribution. However, we would have θ | x ∼ Be(A, B), A =
P
a + n, B = b + i xi for a Be(a, b) prior distribution, which in turn could be
interpreted as the posterior resulting from updating “Be(0, 0)” prior with a
hypothetical sample following a geometric sampling distribution with size
a and b successes. 

Example 2.7 Let x = (xi , i = 1, . . . , n) denote the realization of a random

sample from an Erlang model, that is, Ga(m, λ), where m ∈ N is assumed P
known. The sampling density function is f (x1 , . . . , xn | λ) ∝ λmn e−λ i xi .
P
This is a Ga(mn + 1, i xi ) kernel in λ, which is closed under products.
Thus, the gamma family is the natural conjugate under the Erlang sampling
model, implying the posterior distribution λ | x ∼ Ga(A, B), with A = a +
P
mn, B = b + i xi , corresponding to a Ga(a, b), a, b > 0, prior. This prior
can thus be interpreted as a posterior resulting from a vague “Ga(0, 0)”
prior, defined as h(λ) ∝ λ−1 , λ > 0, and a hypothetical Erlang sample of
size a/m and with sample mean mb/a. 

Example 2.8 Consider now a random sample from a normal distribution

with mean µ and known precision 1/σ2 . The kernel of the corresponding
density function for a realization x = (xi , i = 1, . . . , n) of this sample can
be written as
n 2
f (x1 , . . . , xn | µ) ∝ e− 2σ2 (µ− x̄) ,
which is proportional to the kernel of a normal distribution for µ with mean
x̄ and variance σ2 /n. The product of the two kernels is again a kernel of
the same type 2 .
Thus the natural conjugate family is Gaussian, with µ ∼ N(a, b2 ) ⇒ µ |
x ∼ N(A, B2 ), where by the mentioned identity
!
2 1 n 1 n
A=B a + 2 x̄ , 1/B2 = 2 + 2
b2 σ b σ
2 d1 d2
Note the algebraic identity d1 (z − c1 )2 + d2 (z − c2 )2 = (d1 + d2 )(z − c)2 + d1 +d2 (c1 − c2 )2 ,
d1 c1 +d2 c2
where c = d1 +d2 .
26 Representation of Prior Information

Letting b → +∞ we get the “uniform” in R as a vague distribution,

which in turn implies a posterior distribution µ | x ∼ N( x̄, σ2 /n). In sum-
mary, the prior distribution µ ∼ N(a, b2 ) can be seen as the outcome of
posterior updating of this vague prior with a hypothetical normal sample
of size m, with empirical mean a and (known) variance mb2 . 
The illustrative examples in this chapter were all with univariate param-
eters. However, the argument to identify the natural conjugate family for
models with multivariate parameters, if it exists, proceeds entirely in the
same way as in the described examples. Illustrations with multiparameter
models can be found in Chapter 3.
The big difference is the incomparably bigger difficulty of eliciting a
larger number of summaries that are needed to identify the natural con-
jugate distribution for a multivariate parameter. Unfortunately, there is no
similarly rich set of distributional forms as in the univariate case. Strategies
that try to overcome these limitations in the choice of prior distributions in-
clude the specification of independence across subsets of parameters and
the use of continuous or finite mixtures of natural conjugate distributions.
There are also other methods for prior construction, specifically for mod-
els with multivariate parameters. A specific example is the method of ref-
erence objective priors of Berger and Bernardo, which is described in de-
tail by Bernardo and Smith (2000). However, the method does not always
work, due to the sometimes analytical intractability of the required results,
especially in complex parametric models.

Problems
2.1 Consider a Poisson sampling model f (n | λ) = Poi(λ). Find Jeffreys’ prior
for λ.
2.2 In order to diagnose the cause of a certain symptom, a patient’s physician
analyzes the tetrahydrocortisone content (THE) in the urine of the patient
over 24 hours. The laboratory returns a result of THE = 13 mg/24 h. There
are two possible causes, adenoma (z = 1) and carcinoma (z = 2). It is known
that the distribution of, y = ln(THE), i.e., the logarithms of the amount of
THE in the urine has approximately a normal distribution. We assume
M1 : y ∼ N(µ1 , σ21 )
M2 : y ∼ N(µ2 , σ22 )
for adenoma (M1 ) and carcinoma (M2 ), respectively. A data bank of THE
measures is available with information about six patients with adenoma and
five patients with carcinoma as follows (in mg/24 h):
 Problems 27

adenoma 3.1, 3.0, 1.9, 3.8, 4.1, 1.9

carcinoma 10.2, 9.2, 9.6, 53.8, 15.8
a. First, we use the data to determine the parameters in the sampling model.
That is, use the data to estimate µ j and σ j , j = 1, 2.
Fix the parameters, and determine the posterior probability of this patient
having carcinoma, i.e., p(z = 2 | y), if the initial opinion of the physicians
team, based on the patient’s history, is that the patient has a carcinoma
with probability 0.7.
b. Alternatively, let x ji , i = 1, . . . , 6, denote the historical data and assume
x ji ∼ N(µ j , σ2j ). Using a vague prior, find the posterior distribution h(µ, σ2 |
x) and use it as a prior probability model for the new patient. Again, find
the posterior probability of the new patient having carcinoma.
c. Discuss more variations of the approach in part (b). What does the model
in part (b) assume about the historical patients and the current patient?
What does the model assume about cortisol measurements under the two
conditions? There is no need for more calculations, discuss the problems
only in words.
2.3 For fixed n and r, consider the following two experiments:
E1: X ∼ Bi(n, θ) and
E2: Y ∼ NBin(r, θ),
y+n−1
using a negative binomial with f (y | θ) = Cy θn (1 − θ)y where Ckn is the
binomial coefficient and θ is a Bernoulli success probability.
a. Find Jeffreys’ prior h(θ) in both experiments.
Hint: E(Y) = r(1 − θ)/θ for the negative binomial distribution in E2 (in
contrast to the parametrization used in Appendix A).
b. Use n = 2, r = 1. Using the priors found in part (a), compute p(θ > 0.5 |
X = 1) in E1; and p(θ > 0.5 | Y = 1) in E2.
c. The two probabilities computed in part (b) will differ. Based on parts
(a) and (b), argue that inference based on Jeffreys’ prior can violate the
Likelihood Principle (stated below for reference).
Likelihood Principle: Consider any two experiements with observed data
x1 and x2 such that f (x1 | θ) = c(x1 , x2 ) f (x2 | θ), i.e., the likelihood
functions are proportional as a function of θ. The two experiments bring
the same information about θ and must lead to identical inference (Robert,
1994, ch. 1).3
2.4 Suppose that the lifetimes x1 , . . . , xn of n bulbs are exponentially distributed
with mean θ.

3
Bayesian inference satisfies the likelihood principle since it is entirely based on the
posterior distribution which in turn depends on the data only through the likelihood
f (x | θ).
28 Representation of Prior Information

a. Obtain Jeffreys’ prior h(θ) for θ and show that it is improper.

P
b. Let y = i I(xi < t) denote the number of failures at a fixed time t. Obtain
the likelihood function f (y | θ).
c. Show that if no bulbs fail before a pre-specified time, t > 0, then the
posterior distribution is also improper.
 3

Bayesian Inference in Basic Problems

1
Having understood the fundamental ideas of the Bayesian approach to sta-
tistical inference and the main methods to represent prior information, it is
now time to continue with some illustrations. The basic Bayesian paradigm
is best illustrated in problems in which inference can be represented as
exactly as possible, preferably analytically or at least by simulation from
well-known posterior distributions.
This is done in this chapter by discussing the analysis of eight Bayesian
models which, together with Examples 2.5 – 2.8 and some of the exercises
in this chapter, cover the majority of textbook problems in a basic statistics
course. Since a Bayesian model is defined by a joint distribution of a vector
(X) of observations and of parameters (θ), the statement of each model
includes a sampling model { f (x | θ)} (the model of classical statistics)
and a prior distribution h(θ), which we will indicate by notation like f (x |
θ) ∧ h(θ). See Appendix A for a statement of all the parametric families that
are used in the following discussion.

3.1 The Binomial ∧ Beta Model

Let x = (xi , i = 1, . . . , n) be a realization of conditionally independent bi-
nomial random variables Xi | mi , θ ∼ Bi(mi , θ), i = 1, . . . , n, with known mi
and unknown parameter θ, equipped with a Be(a, b) prior distribution with
fixed hyperparameters. Let Ckn = k!(n − k)!/n! denote binomial coefficients.
1
This material will be published by Cambridge University Press as Computational
Bayesian Statistics, by M.A. Amaral Turkman, C.D. Paulino, and P. Müller
(https://tinyurl.com/CompBayes). This pre-publication version is free to view
and download for personal use only. Not for re-distribution, re-sale, or use in derivative
works. c Maria Antónia Amaral Turkman, Carlos Daniel Paulino & Peter Müller, 2019.
30 Bayesian Inference in Basic Problems

The posterior density for θ has the kernel

 
h(θ | x) ∝ Πni=1 C mxi i θ xi (1 − θ)mi −xi h(θ | a, b)
P P
xi −1 i (mi −xi )−1
∝ θa+ i
(1 − θ)b+ ,
P P
implying that θ | x ∼ Be(A, B), with A = a + i xi and B = b + i (mi − xi ).
This posterior distribution corresponds to using other types of distribu-
tions for some transformations of θ that are of interest in certain applica-
tions, such as Fisher’s F and Z distributions,
B θ
| x ∼ F(2A,2B)
" A 1−θ #
1  B 1  θ 
ln + ln | x ∼ Z(2A,2B) .
2 A 2 1−θ
Let B(a, b) = Γ(a)Γ(b)/Γ(a + b) denote the beta function. Mixed mo-
ments of the posterior for θ are easily found as
B(A + r1 , B + r2 )
E [θr1 (1 − θ)r2 | x] = ,
B(A, B)
which allow evaluation of various summaries of the distribution, such as
the mean E(θ | x) = A/(A + B) and the posterior variance. Another rel-
evant point estimate, the posterior mode, is well defined if A, B > 1, as
A−1
m0 = A+B−2 . Posterior quantiles and probabilities of θ can be evaluated
by incomplete beta functions (there is no explicit form), or in the case of
integer a, b, as binomial distribution functions using
F Be(A,B) (θ0 ) = 1 − F Bi(A+B−1,θ0 ) (A − 1).
Regarding predictive calculation, consider new responses Y j ≡ Xn+ j , j =
1, . . . , k of the same model and assume that they are independent of the
observed ones as Y j | θ ∼ Bi(mn+ j , θ), j = 1, . . . , k, independently. By defi-
nition the posterior predictive distribution for Y = (Y1 , . . . , Yk ) is a mixture
of the sampling model (product of binomials) with respect to the posterior
distribution (beta) on θ. We find the predictive probability function
h m
i B(A + y , B + mn+ − y )
p(y1 , . . . , yk | x) = Πkj=1Cy jn+ j ,
B(A, B)
P P
where y = kj=1 y j , and mn+ = j mn+ j . It is useful to write it as a prod-
uct of two probability functions, a multivariate hypergeometric distribution
conditional on y and the beta-binomial marginal posterior predictive dis-
tribution for y ,
p(y1 , . . . , yk | x) = pHpG({mn+ j },y ) (y1 , . . . , yk | y ) × pBeBin(mn+ ,A,B) (y | x).
 3.2 The Poisson ∧ Gamma Model 31

In fact, this expression highlights the type of dependence that exists in

the multivariate predictive distribution, as well as its univariate nature as
A
BeBin(mn+1 , A, B) for k = 1, with mean E(Y1 | x) = mn+1 A+B and variance
AB m
Var(Y1 | x) = mn+1 (A+B)(A+B+1) (1 + A+B ).
n+1

3.2 The Poisson ∧ Gamma Model

Assume now that x = (xi , i = 1, . . . , n) is a realization of a random sample
from a Poi(θ) model, with corresponding sampling probability function
f (x1 , . . . , xn | θ) ∝ e−nθ θ x . Since this is proportional to the kernel of a
Ga(x + 1, n) for θ, which in turn is closed under products, we see that the
natural conjugate family is the gamma family, that is:
θ ∼ Ga(a, b) ⇒ θ | x ∼ Ga(A, B), A = a + x , B = b + n,
with h(θ) ∝ θ−1 I(0,+∞) (θ) being the corresponding diffuse improper prior
distribution. The posterior h(θ | x) can alternatively be written as α =
2Bθ ∼ χ22A . Posterior probabilities of pre-selected events in Θ = R+ or
relative posterior plausibility for point hypotheses on θ can be evaluated as
incomplete gamma functions or as chi-square distribution functions.
Posterior moments are given by
Γ(A + r) BA
E(θr | x) = .
Γ(A) BA+r
iid
For predictive inference, let Y j ≡ Xn+ j ∼ Poi(θ), j = 1, . . . , k, denote fu-
ture observations, independent of the observed random sample. The poste-
rior predictive distribution for Y = (Y1 , . . . , Yk ) is a mixture of the sampling
model (product of Poisson) with respect to the Ga(A, B) posterior distribu-
tion on θ. Integrating out θ, the posterior predictive probability function
becomes
!y
Γ(A + y )  B A 1
p(y1 , . . . , yk | x) = .
Γ(A)Π j y j ! B + k B+k
P
Since the sampling distribution of y = j Y j is Poi(kθ), an argument anal-
ogous to the previous discussion shows that the posterior predictive distri-
bution for y is a Poi–Ga mixture, with probability function
!y
Γ(A + y )  B A k
p(y | x) = ,
Γ(A)y ! B + k B+k
better known as the generalized negative binomial with parameters (A,
B/(B + k)). The two parameters can be interpreted as the fixed number of
32 Bayesian Inference in Basic Problems

“successes” and the probability of each of them. Compared with the pre-
vious expression, we can rewrite it as p(y1 , . . . , yk | x) Π j y j !(ky )/y !, and
therefore
p(y1 , . . . , yk | x) = p Mk−1 (y , 1k 1k ) (y1 , . . . , yk | y ) × pBiN(A,B/(B+k)) (y | x).
That is, similar to the binomial ∧ beta case, the posterior predictive proba-
bility function can be written as a product of a homogeneous multinomial
probability function and a Poi–Ga marginal probability function. This rep-
resentation highlights the nature of the dependence in the posterior predic-
tive distribution, which reduces to a negative binomial when k = 1.

3.3 Normal (Known µ) ∧ Inverse Gamma Model

Let x = (xi , i = 1, . . . , n) be a random sample of Xi | σ2 ∼ N(µ0 , σ2 ),
i = 1, . . . , n, i.i.d., with known µ0 . The corresponding density function,
P 2
i (xi −µ0 )
f (x1 , . . . , xn | σ2 ) ∝ (σ2 )−n/2 e− 2σ2 ,
as a function of σ2 is proportional to the kernel of an inverse gamma dis-
P
tribution, IGa( n2 − 1, 21 i (xi − µ0 )2 ), which in turn is closed under prod-
ucts. Therefore, the IGa family is a natural conjugate prior for this sam-
pling model, with σ2 ∼ IGa(a, b) ⇔ 1/σ2 ∼ Ga(a, b) implying that
P
σ2 | x ∼ IGa(A, B) with A = a + n2 , B = b + 21 i (xi − µ0 )2 . The
IGa(a, b) prior distribution can therefore be interpreted as an updated vague
“Ga(0, 0)” prior based on a hypothetical sample of size 2a from the corre-
sponding normal model with mean zero and sum of squares 2b.
Parametric inference about the scale or precision can readily be obtained
from the IGa or Ga (or related) posterior distributions. Predictive inference
for a future observation Y from the same sampling model, independent of
Xi , i = 1, . . . , n, is obtained as a scale mixture of N(µ0 , σ2 ) with respect
to the posterior distribution σ2 | x ∼ IGa(A, B). This defines a Student-t
distribution√with 2A degrees of freedom, location parameter µ0 and scale
parameter B/A, written as t(2A) (µ0 , B/A), with predictive density function
Z +∞  
BA (y−µ )2
−(A+3/2) −(1/σ2 ) B+ 20
p(y | x) = √ (σ2 ) e dσ2
2π Γ(A) 0
" !#−1 " #− 2A+1
2A 1 p
−1 (y − µ0 )2 2
= B , 2AB/A 1+
2 2 2AB/A
 3.4. Normal (unknown µ, σ2 ) ∧ Jeffreys’ prior 33

3.4 Normal (Unknown µ, σ2 ) ∧ Jeffreys’ Prior

Assume that x = (xi , i = 1, . . . , n) is an observed random sample from a
N(µ, σ2 ) model with likelihood function
 n ks2 
f (x1 , . . . , xn | µ, σ2 ) ∝ (σ2 )−n/2 exp − 2 (µ − x̄)2 − ,
2σ 2σ2
P
where k = n − 1 and ks2 = i (xi − x̄)2 . This is the kernel of a joint normal-
inverse gamma distribution for (µ, σ2 ), defined as a normal for µ given σ2
and an inverse gamma for σ2 . The family of normal-inverse gamma distri-
butions is closed under products. The natural conjugate family is therefore
defined by density functions of the type h(µ, σ2 | a, v, c, d) = hN(a,σ2 /v) (µ |
σ2 ) hIGa(c,d) (σ2 ).
Jeffreys’ prior under prior independence of µ and σ2 , h(µ, σ2 ) ∝ σ−2 ,
is a limiting case of an improper NIGa. Bayesian updating with the given
likelihood function yields the posterior distribution
n n−1 ks2
h(µ, σ2 | x) ∝ (σ2 )−1/2 e− 2σ2 (µ − x̄)2 × (σ2 )−( 2 +1) e− 2σ2
2 2
implying µ | σ2 , x ∼ N( x̄, σ2 /n) and σ2 | x ∼ IGa( 2k , ks2 ) ⇔ 2σks 2
2 | x ∼ χ(k) .

The marginal posterior distribution for µ is therefore a mixture of nor-

mals with respect to an IGa, which we already know to be a Student t type
distribution. In fact, integrating σ2 with respect to the IGa density function,
we get
"  #−1 " #− k+1
k 1   p 2 −1 (µ − x̄)2 2
h(µ | x) = B , ks /n 1+ ,
2 2 ks2 /n
2
that is, µ | x ∼ t(k) ( x̄, s2 /n) ⇔ s/µ−√x̄n | x ∼ t(k) (0, 1) ⇔ (µ− x̄)
s2 /n
| x ∼ F(1,k) ,
where t(k) (0, 1) is the Student t distribution, well known in classical statis-
k s2
tics. Therefore, E(µ | x) = x̄ (if k > 1) and Var(µ | x) = k−2 n
(if k > 2).
The conditional posterior distribution for σ2 , given µ, can also be found as
2
x̄)2
σ2 | µ, x ∼ IGa( k+12
, ks +n(µ−
2
).
Inferences about the location and scale parameters in the sampling model
are easily obtained from the Student t and inverse gamma distributions (or
χ2 after appropriate transformation). In terms of prediction, consider, for
example, a future random sample of size m from the same model, and as-
sume that we wish to predict its mean Ȳ. The posterior predictive distribu-
tion for Ȳ is a mixture of Ȳ | µ, σ2 ∼ N(µ, σ2 /m) with respect to the joint
posterior h(µ, σ2 | x) = h(µ | σ2 , x) h(σ2 | x). Usingh an algebraic identityi
for the linear combination of the quadratic forms, σ12 m(µ − ȳ)2 + n(µ − x̄)2 ,
34 Bayesian Inference in Basic Problems

we see that this distribution takes the form of a Student t,

 m+n 
Ȳ | x ∼ t(k) x̄, s2 ,
mn
from which one can easily find point and interval estimates.

3.5 Two Independent Normal Models ∧ Marginal Jeffreys’ Priors

Let x j = (x ji , i = 1, . . . , n j ), j = 1, 2, denote the realizations of two in-
dependent random samples from the models N(µ j , σ2j ), and complete the
model with the usually assumed Jeffreys’ priors for the four parameters,
h(µ1 , µ2 , σ21 , σ22 ) ∝ (σ21 σ22 )−1 over the standard joint parameter space.

Comparison of Means
Following an analogous argument as in the previous section, we easily find
that (µ1 , σ21 ) and (µ2 , σ22 ) are a posteriori again independent with univariate
marginal distributions:
µ j − x̄ j
µ j | x j ∼ t(k j ) ( x̄ j , s2j /n j ) ⇔ ν j = √ | x j ∼ t(k j )
s j/ n j
2
kj kjsj
σ2j | x j ∼ IGa( , ),
2 2
Pn j
where k j = n j − 1 and k j s2j = i=1 (x ji − x̄ j )2 .
The standardized difference λ = µ1 − µ2 , written as
λ − ( x̄1 − x̄2 )
τ= q 2 ≡ ν1 sin u + ν2 cos u,
s1 s22
n1
+ n2

where u = arctan( √sn1 1 / √sn2 2 ), is a posteriori distributed as a linear combina-

tion of independent Student t distributions, known as the Behrens–Fisher
distribution, and parametrized by k1 , k2 , and u2 . Its density function, which
is symmetric but not of closed form, is usually evaluated by a Student t
2
Note that the dependence on u implies that there is no duality between the posterior and
sampling distributions for τ, and thus, no numerical identity of Bayesian and classical
inference on the difference of means. This is in contrast to what happens in other
situations under non-informative priors.
 3.6 Two Independent Binomials ∧ Beta Distributions 35
approx
approximation (Patil, 1964), τ | x1 , x2 ∼ BF(k1 , k2 , u) ∼ t(b) (0, a), where
p
b = 4 + c21 /c2 , a = c1 (b − 2)/b, with
k1 k2
c1 = sin2 u + cos2 u, and
k1 − 2 k2 − 2
k12 4
k22
c2 = sin u + cos4 u.
(k1 − 2)2 (k1 − 4) (k2 − 2)2 (k2 − 4)
As an alternative to the use of Patil’s approximation, one could by computer
simulation generate a Monte Carlo sample from the posterior distribution
of τ by simulating from the posterior distributions for ν1 and ν2 . Based on
this sample, one could then empirically evaluate point and interval esti-
mates and test point hypotheses about the difference of means.

Comparing Variances
σ21
Assume now that the parameter of interest is ψ = σ22
. Based on the inde-
d s21
pendent gamma posteriors for {1/σ2j } we find that ψ | x1 , x2 ≡ F
s22 (k2 ,k1 )
,
which allows easy implementation of inferences about ψ.

Comparing Means of Homoscedastic Populations

Assume σ21 = σ22 ≡ σ2 and use Jeffreys’ prior h(µ1 , µ2 , σ2 ) ∝ σ−2 , µ1 , µ2 ∈
R, σ2 > 0. From the results in the previous section we quickly find:
 
λ = µ1 − µ2 | σ2 , x1 , x2 ∼ N x̄1 − x̄2 , σ2 ( n11 + n12 ) ;
2
σ2 | x1 , x2 ∼ IGa( 2k , ks2 ),
P
where k = n1 + n2 − 2 and s2 = k−1 j (n j − 1)s2j is the pooled empirical
variance. This implies, in particular for λ = µ1 − µ2 :
 1 1  λ − ( x̄1 − x̄2 )
λ | x1 , x2 ∼ t(k) x̄1 − x̄2 , s2 ( + ) ⇔ q | x1 , x2 ∼ t(k) ,
n1 n2 s n11 + n12

which is then the basic inference result for the comparison of two normal
populations under homoskedasticity.

3.6 Two Independent Binomials ∧ Beta Distributions

ind
Let t j denote observed counts for T j | θ j ∼ Bi(m j , θ j ), with known {m j },
j = 1, 2. Complete the model with a prior θ j ∼ Be(a j , b j ), j = 1, 2, inde-
36 Bayesian Inference in Basic Problems

pendently. Using the results for the Bi ∧ Be model, we find

θ j | t j ∼ Be(A j , B j ), A j = a j + t j , and B j = b j + m j − t j
Bj θj
⇔ | t j , ∼ F(2A j ,2B j )
Aj 1 − θj
" #
1 Bj 1 θj
⇔ ln + ln | t j ∼ Z(2A j ,2B j ) ,
2 Aj 2 1 − θj
j = 1, 2, independently. We will use these distributional results in the next
two inference problems.

Exact One-Sided Test for Two Proportions

Consider testing H0 : θ1 ≤ θ2 versus H1 : θ1 > θ2 . The evaluation of
posterior odds and the Bayes factor requires the quantities
Z 1 "Z 1 #
P(H0 | t1 , t2 ) = h(θ1 | t1 ) h(θ2 | t2 )dθ2 dθ1 ,
0 θ1

and a similar expression for prior probabilities. In the case of integer a2

and b2 , the integrals can be evaluated using the result F Be(A2 ,B2 ) (θ1 ) = 1 −
F Bi(A2 +B2 −1,θ1 ) (A2 − 1) for beta and binomial distribution functions. We get

1 X
A2 −1

P(H0 | t1 , t2 ) = C A2 +B2 −1 B(A1 + u, B1 + A2 + B2 − 1 − u).

B(A1 , B1 ) u=0 u
If a1 and b1 are also integers, the beta functions can be evaluated in terms
of factorials.

Testing Homogeneity H0 : θ1 = θ2 versus H1 : θ1 , θ2

Consider first H0 : π = 0 ⇔ ln ∆ = 0 using the transformations π = θ1 − θ2
and ∆ = θθ21 /(1−θ
/(1−θ1 )
2)
. First, simulate from the Be(A j , B j ) posterior distributions
on θ j . Transform to posterior samples for π or ∆ (or ln ∆), which then allow
good Monte Carlo approximations of the level of relative posterior plau-
sibility of H0 or HPD intervals. Of course the same approach can be used
for one-sided hypotheses by way of calculating the appropriate proportions
based on the simulated samples.
In the case of large numbers of observed successes and failures, one can
use asymptotic approximations of Fisher’s Z distribution; for example:
" #
approx 1 ν1−1 − 1 1 −1 −1
Z(ν1 ,ν2 ) ∼ N ln −1 , (ν + ν2 ) .
2 ν2 − 1 2 1
 3.7 Multinomial ∧ Dirichlet Model 37

This, together with the distributional result for (1/2) ln[θ j /(1 − θ j )] that was
stated at the beginning of this subsection, give the approximate posterior
distribution
 
 (A1 − 1/2)/(B1 − 1/2) X 
−1 
ln ∆ | t1 , t2 ∼ N ln (A−1  .
approx
, j + B j ) 
(A2 − 1/2)/(B2 − 1/2) j=1,2

This can be used to construct one-sided or two-sided Bayesian tests for the
comparison of proportions.

3.7 Multinomial ∧ Dirichlet Model

This Bayesian model is a multivariate version of the binomial ∧ beta model,
but much less used than the latter. We first review the main properties of
the model relevant to inference.
Let X = (X1 , . . . , Xc ) and θ = (θ1 , . . . , θc ) be random vectors taking
P
values in the subspaces X = {x = (x1 , . . . , xc ) : xi ∈ N0 , x = ci=1 xi ≤ N},
P
with known N, and Θ = {(θ1 , . . . , θc ) : θi ∈ (0, 1), θ = ci=1 θi < 1},
respectively. The space Θ is also known as the c-dimensional simplex Sc .

A (c-Dimensional) Multinomial Model for X

The probability function for X | θ ∼ Mc (N, θ) is
N! xi
f (x | θ) = Πc+1
i=1 θi , x ∈ X,
Πc+1
i=1 xi !
P
with xc+1 = N − x and θc+1 = 1 − θi . The first two moments are defined
as (using, e.g., the moment-generating function):
µ = E(X | θ) = Nθ; Σ = Var(X | θ) = N(Dθ − θθ′ ),
where Dθ = diag(θ1 , . . . , θc ).
Next, we consider the implied distribution on aggregate counts over sub-
sets. Let Ck = { jk−1 + 1, . . . , jk }, k = 1, . . . , s + 1 denote the subsets of a
partition of the set of indices {1, 2, . . . , c, c + 1} into s + 1 subsets with
# Ck = dk , j0 = 0, and j s+1 = c + 1. A corresponding segmentation of X is
defined as
X
X (k) = (Xi , i ∈ Ck ); Mk = Xi , k = 1, . . . , s + 1,
i∈Ck

where the Mk stand for aggregate counts – of Xi – over subsets Ck .

38 Bayesian Inference in Basic Problems

Using the moment-generating function technique and the definition of a

conditional distribution, we find
P
M = (M1 , . . . , M s ) | θ ∼ M s (N, α), α = (α1 , . . . , α s ), αk = i∈Ck θi
ind
X (k) | M, θ, k = 1, . . . , s + 1 ∼ Mdk −1 (Mk , πk ),
with πk = (θi /αk , i ∈ Ck − jk ). These results show that the component-
specific marginal and conditional distributions of a multinomial are also
multinomial (binomial in the univariate case). This is clearly relevant in
several contexts such as analysis of contingency tables. For example, if
X represents the vector of frequencies in a two-dimensional contingency
table, the marginal frequencies of the rows (or columns) are given by M
and the conditional frequencies of rows (or columns), conditional on the
marginal totals, are the indicated products of multinomials.

A Dirichlet Model for θ

The Dirichlet model is defined by the density function θ | a ∼ Dc (a) :
ai −1
ha (θ) = [B(a)]−1 × Πc+1 i=1 θi , θ ∈ Θ = Sc where a = (a1 , . . . , ac , ac+1 ) ∈
c
R+ , θc+1 = 1 − θ , and B(a) = Πc+1 i=1 Γ(ai )/Γ(a ) is the multivariate beta
function.
For theoretical and computational reasons it is better to define the Dirich-
let distribution based on independent gamma distributions through the fol-
P
lowing transformation: θi = νi / c+1 j=1 ν j , i = 1, . . . , c, with random variables
ind
νi , i = 1, . . . , c + 1 ∼ Ga(ai , 1). h i
ri
By their definition, the mixed moments are given by E Πc+1 i=1 θi | a =
B(a + r)/B(a), where r = (r1 , . . . , rc , rc+1 ), from which we find
ai Ei (1 − Ei )
E(θi | a) ≡ Ei = ; Var(θi | a) = ;
a a + 1
Ei E j
cov(θi , θ j | a) = − , i , j.
a + 1
Since (α, π1 , . . . , π s+1 ) is a reparametrization of θ, one can find (from,
e.g., the representation by gamma distributions):
P
α | a ∼ D s ( i∈Ck ai , k = 1, . . . , s + 1)
ind
πk | a, k = 1, . . . , s + 1 ∼ Ddk −1 (ai , i ∈ Ck ).

A Multinomial-Dirichlet Model for X

This distribution, also known as a Pólya distribution, arises as a mixture
of a multinomial with respect to a Dirichlet, X | a ∼ MDc (N, a), with
 3.7 Multinomial ∧ Dirichlet Model 39

probability function
N! B(a1 + x1 , . . . , ac+1 + xc+1 )
p(x | a) = , x ∈ X.
Πc+1
i=1 xi !
B(a1 , . . . , ac+1 )
The first two moments can be expressed as (using, e.g., the properties of
conditional expectations):
!
a a + N aa′
E(X | a) = N ; V(X | a) = N Da − .
a a (a + 1) a
Using a segmentation of X as before, one easily finds that
P
M = (M1 , . . . , M s ) | a ∼ MD s (N; i∈Ck ai , k = 1, . . . , s + 1)
ind
X (k) | M, a, k = 1, . . . , s + 1 ∼ MDdk −1 (Mk ; ai , i ∈ Ck ).

Application to Inference
Assume x = (x1 , . . . , xc ) is a realization of a random vector X | θ ∼
Mc (N, θ). Since f (x | θ) is proportional to the kernel of a Dc (xi + 1, i =
1, . . . , c + 1) distribution that is closed under products, we conclude that
the Dirichlet family is the natural conjugate for a multinomial sampling
model. Thus, if θ | a ∼ Dc (a), then θ | a, x ∼ Dc (A), A = (Ai = ai + xi , i =
1, . . . , c + 1).
Bayesian estimates of (θi , i = 1, . . . , c + 1) can be derived in particular
from the components of the posterior mode (A − 1c+1 )/A (if Ai > 1, ∀i),
where 1c+1 is a c + 1 vector of all 1s, or the posterior mean A/A . Note how
this is a weighted mean of the prior mean, a/a , and the vector of sample
proportions p = (xi /N, i = 1, . . . , c + 1).
In the analysis of contingency tables, inference of interest is often related
to independent structures (or other log-linear models) in which paramet-
P P
ric functions i bi ln θi with i bi = 0 often play a critical role (see, e.g.,
Paulino and Singer, 2006). When the components of A are large, one can
invoke approximate posterior normality of the posterior distribution and
use the resulting χ2 distribution for appropriate quadratic forms, allowing
tests for such structures. For more detail see, e.g., (Paulino et al, 2018, ch.
6). In the special case of a 2 × 2 table the use of such an approach to the
test of independence, which reduces simply to a test of homogeneity of
two binomials, leads to the approach mentioned at the end of the previous
section.
Assume now the aim is to predict a vector Y with Y | m, θ ∼ Mc (m, θ).
The corresponding posterior predictive distribution is a multinomial-Dirichlet,
40 Bayesian Inference in Basic Problems

Y | m, x ∼ MDc (m, A), whose summary by the first two moments can be
obtained by the formulas in Appendix A.

3.8 Inference in Finite Populations

Consider a finite population of known size N, partitioned into c ≤ N groups
P
of unknown sizes Ni , i = 1, . . . , c, with ci=i Ni = N. Assume one se-
lects randomly (without replacement) a sample S of n ≤ N units with
the aim of drawing inference about the population sizes of the groups,
θ = (N1 , . . . , Nc ). Let ni , i = 1, . . . , c, be the observed frequencies of the
P
groups, with ci=1 ni = n, and let x = (n1 , . . . , nc ) which, by assumption,
is now an observation from the multivariate hypergeometric distribution
X | N, n, θ ∼ H pgc−1 (θ, n) (for convenience we use here and in the follow-
ing redundant notation in the definition of random vectors such as X by
explicitely stating all hyperparameters).
Denote by Uk a vector of group membership indicators for the k-th unit.
The possible values of Uk are the standard basis of Rc (that is, binary (c×1)
vectors with exactly one “1”). The inference goal can then be summarized
as
XN X X
θ= Uk = Uk + Uk ≡ X + (θ − X),
k=1 k∈S k<S

highlighting in particular that a posteriori only θ − X is unknown.

One can construct a prior as a hierarchical model defined by
iid
U1 , . . . , U N ∼ Mc−1 (1, φ)
P
with an underlying hyperparameter φ = (φ j , j = 1, . . . , c), with j φ j =
1. In a second level we assume the hyperprior, φ | a ∼ Dc−1 (a), a =
(a1 , . . . , ac ) ∈ Rc+ .
Regarding the first level of the hierarchial prior model, we have thus
θ | φ ∼ Mc−1 (N, φ) and X and θ − X are by definition a priori condition-
ally independent given φ, with distributions of the same type, X | n, φ ∼
Mc−1 (n, φ) and θ − X | n, φ ∼ Mc−1 (N − n, φ). Note also that the hypergeo-
metric sampling model for X can be written as
N
Πcj=1C x jj f (x | n, φ) h(θ− x | n, φ) f (x, θ | n, φ)
f (x | n, θ) = = =
CnN h(θ | φ) h(θ | φ)
= f (x | n, θ, φ).
Using the second-level information, one can identify the following marginal
 Problems 41

(or prior predictive) distributions arising from the multinomial-Dirichlet

model:
θ | a ∼ MDc−1 (N, a); X | n, a ∼ MDc−1 (n, a); (3.1)
θ − X | n, a ∼ MDc−1 (N − n, a).
We also see that updating of the prior information on φ conditional on x is
d
such that φ | x ∼ Dc−1 (a + x). On the other hand, since θ − x | x, φ = θ − x |
φ ∼ Mc−1 (N − n, φ), we have that θ − x | x ∼ MDc−1 (N − n, a + x).
In summary, the posterior distribution of θ − x under hypergeometric
sampling is of the same multinomial-Dirichlet type as the prior distribution.
The posterior distribution for the vector θ of group totals is derived from
that by translation by x, which of course implies the posterior distribution
for the vector of population proportions θ/N (Basu and Pereira, 1982).

Problems
3.1 Exponential ∧ gamma model. Let xi ∼ Exp(θ), i = 1, . . . , n, i.i.d.
a. Assuming θ is distributed as Ga(α, β), with α and β fixed, find the poste-
rior distribution h(θ | x1 , . . . , xn ).
b. Under the same assumptions, compute the predictive distribution p(xn+1 |
x1 , . . . , xn ).
c. Consider now a slightly different situation. Imagine that the exponential
random variables represent waiting times for the recurrence of disease in
n patients surgically treated at time 0. We are at time a, and all patients are
still alive. Some of the n patients may have experienced recurrence. We
record their recurrence time xi (assume patients are not followed up after
the recurrence, i.e., we have no further data). Other patients may still be
healthy, so we don’t know exactly what the recurrence time is. We only
know xi > a. Repeat steps (a) and (b) for this case.
3.2 Binomial (unknown θ, n) ∧ beta/Poisson model. Consider a binomial sam-
pling model x | θ, n ∼ Bi(n, θ).
a. For fixed n and θ ∼ Be(a0 , b0 ), find the marginal distribution p(x). This
distribution is known as beta-binomial (cf. Section 3.1).
b. Assume now that also n is unknown, h(n) ∝ 1/n2 . Plot the joint posterior
distribution h(n, θ | x) for x = 50, and (a0 , b0 ) = (1, 4).
Hint: Use a grid on 0.01 ≤ θ ≤ 0.99 and x ≤ n ≤ 500. Use the R function
lgamma(n+1) to evaluate ln(n!).
3.3 Hierarchical Poisson ∧ gamma model. An animal experiment records tu-
mor counts for mice of two different strains, A and B. Tumor counts are
42 Bayesian Inference in Basic Problems

approximately Poisson distributed. The observed tumor counts for nA = 10

mice of strain A and nB = 13 mice of strain B are
yA = (12, 9, 12, 14, 13, 13, 15, 8, 15, 6); nA = 10
yB = (11, 11, 10, 9, 9, 8, 7, 10, 6, 8, 8, 9, 7); nB = 13.
We assume yAi ∼ Poi(θA ), i.i.d., for i = 1, . . . , nA . And independently yBi ∼
Poi(θB ), i.i.d., for i = 1, . . . , nB . In parts (a) through (c) we will consider
three alternative prior models that reflect our prior information about θA and
θB at different levels.
a. Find the posterior distributions, means, variances, and 95% credible in-
tervals for θA and θB , assuming Poisson sampling distributions for each
group and the following prior distribution:
θA ∼ Ga(120, 10), θB ∼ Ga(12, 1), h(θA , θB ) = h(θA ) · h(θB ).
b. Set up a hierarchical prior for θA and θB that formalizes the notion that
types A and B are similar, that is, a two-level hierarchical model:
θA | ψ ∼ h(θA | ψ) and θB | ψ ∼ h(θB | ψ)
ψ ∼ h(ψ).
Hint: You could, for example, use a hierarchical hyperprior for a common
prior mean ψ for θA and θB .
c. Now modify the prior model from (b) to allow for a positive probability
for θA = θB . Find h(θA = θB | y).
3.4 Rayleigh ∧ gamma model. The Rayleigh distribution with p.d.f. f (x | δ) =
2
δ x e−δx /2 I(0,+∞) (x) is used for some problems in engineering. Assume x =
(xi , i = 1, . . . , n) is a realization of a random sample from this model and
assume a Ga(a, b) prior distribution for δ.
a. Find the posterior distribution h(δ | x). Argue that the gamma family is
the natural conjugate model. Find E(δ | x) and Var(δ | x).
b. Hypothesis testing for H0 : δ = δ0 can be carried out using incomplete
gamma functions. For example, using the Jeffreys prior h(δ) ∝ δ−1 I(0,+∞) (δ),
the posterior distribution conditional on one observation x from the Rayleigh
model is δ | x ∼ Exp(x2 /2).
Find the level of relative posterior plausibility of H0 (recall the definition
from Section 1.3.1).
c. Next we compute a Bayes factor. Assume a sample of five measurements
P
from the Rayleigh model gives i xi2 = 7.54. Assume that it is desired
to test H0 with δ0 = 2 on the basis of 50% prior probability for H0 , and
adopting a Ga(0.02, 0.01) distribution for the prior under the alternative
hypothesis. Note that the prior distribution for δ | H1 is a proper distri-
bution with mean 2 and variance 200, reasonably similar to an improper
 Problems 43

Jeffreys prior. It is quite flat over most of its support, with the exception
0 |x)
of small positive values. Find the Bayes factor B(x) = P(H P(H1 |x) .
d. Finally, we consider prediction for Y, with Y | δ ∼ Ray(δ) independently
of (X1 , . . . , Xn ). Find the posterior predictive density function p(y | x), the
point estimate E(Y | x) and P(Y > 1 | x).
Hint: Use properties of the √ conditional expectation and integration by
parts to prove E(Y | δ) = π/2 δ−1/2 ).
3.5 Uniform ∧ Pareto model. Let x = (xi , i = 1, . . . , n) denote a random sample
from a uniform U(0, θ) model, with density
f (x1 , . . . , xn | θ) = θ−n I[t,+∞) (θ), t = x(n) ≡ max xi .
1≤i≤n

a. Recognize f (x | θ) as the kernel of a Pareto distribution for θ. State the

parameters of the Pareto kernel.
b. Assuming θ ∼ Pa(a, b) with a, b > 0, find h(θ | x).
c. Find the posterior mean, mode, and median for θ.
d. Find a γ-HPD credible interval for θ.
e. Let y = xn+1 denote a new observation from the sampling model, assumed
independent of the already observed data. Find p(y | x1 , . . . , xn ).
Hint: The solution is best written in two cases, for y below and above
some threshold.
3.6 Normal linear regression (unknown β, Σ). Consider a normal linear regres-
sion model for yi , i = 1, . . . , n on xi j , j = 1, . . . , k, with a proper multivariate
normal prior on β and three alternative priors on the covariance matrix. Let
y = (y1 , . . . , yn )′ be an n × 1 vector of responses, X an n × k design matrix
(i.e., a matrix with xi j in the ith row and jth column). Let β = (β1 , . . . , βk ) be
a vector of regression coefficients. We assume
y | β, Σ ∼ N(Xβ, Σ)
β ∼ N(µ, T ).
a. Assume that Σ is known. Let Vβ = (X ′ Σ−1 X)−1 , and β̂ = Vβ X ′ Σ−1 y. Show
that h(β | Σ, y) = N(m, V), with


V −1 = T −1 + Vβ−1 and m = V T −1 µ + Vβ−1 β̂ ,

with the last term simplifying to Vβ−1 β̂ = X ′ Σ−1 y.

b. We now extend the model by assuming an unknown – but diagonal –
covariance matrix:
y | β, σ2 ∼ N(Xβ, σ2 I),
β | σ2 ∼ N(µ, σ2 R),
σ2 ∼ Inv-χ2 (ν0 , s0 ).
Find h(τ | y) for τ = 1/σ2 and h(β | σ2 , y) for fixed R, ν0 , s0 .
44 Bayesian Inference in Basic Problems

c. Replace the prior on β by β ∼ N(µ, T ). Find h(τ | β, y).

Note: Together with h(β | σ2 , y) from question (b), this allows us to define
an algorithm that alternately simulates from these two complete condi-
tional posterior distributions. The algorithm is known as the Gibbs sam-
pler and will be discussed at length in Chapter 6. See also Problem 6.11.
 4

Inference by Monte Carlo Methods

1
The vast majority of statistical inference problems involve complex mod-
els that often leave it unrealistic to consider analytical (or even numerical)
solutions for any inference summaries of interest, which in Bayesian infer-
ence often take the form of integrals. In this context, classical Monte Carlo
methods arise as an attractive alternative. Monte Carlo methods evaluate
the relevant inference based on calculations involving simulated random
samples from probability distributions, which in turn can be generated from
(pseudo-)random variate generators [realizations of a uniform U(0, 1)]. Re-
lated methods of stochastic simulation have been the subject of extended
literature2 and can now be implemented by several available statistical and
mathematical software packages.
This chapter describes the general idea of traditional Monte Carlo meth-
ods in its basic version and in importance sampling, as well as some of the
specialized versions and variations pertaining to Bayesian inference.

4.1 Simple Monte Carlo

Consider the problem of approximating an integral of the form
Z
g(θ) h(θ | x)dθ = E[g(θ) | x], (4.1)

where θ and x can be vectors, assuming the expectation with respect to

h(θ | x) exists. Many posterior summaries can be expressed as (4.1) for
1
This material will be published by Cambridge University Press as Computational
Bayesian Statistics, by M.A. Amaral Turkman, C.D. Paulino, and P. Müller
(https://tinyurl.com/CompBayes). This pre-publication version is free to view
and download for personal use only. Not for re-distribution, re-sale, or use in derivative
works. c Maria Antónia Amaral Turkman, Carlos Daniel Paulino & Peter Müller, 2019.
2
In particular, the books by Devroye (1986), with a free pdf version of the book and
errata available on the author’s homepage (www.nrbook.com/devroye); Ripley
(1987) and Gentle (2004).
46 Inference by Monte Carlo Methods

some integrable function g(θ). This is the case for posterior moments of
the elements of θ, posterior probabilities of subsets of the parameter space,
and posterior predictive densities, where g(θ) is, respectively, θi (for the
mean of the i-th coordinate of θ), IA (θ) for A ⊂ Θ and f (y | θ) for fixed y.
Other quantities of interest can also be expressed using appropriate inte-
grals, including the normalizing constant of a posterior distribution, marginal
posterior densities, Bayes factors, and posterior model probabilities.
If one can simulate a random sample θ1 , . . . , θn from the posterior dis-
tribution h(θ | x), then the simplest Monte Carlo method approximates the
integral (4.1) by the sample mean
 1X
n

Ê g(θ) | x = g(θi ), (4.2)
n i=1
which by the strong law of large numbers converges almost surely to
 
E g(θ) | x . The precision of the estimator can be evaluated by the (esti-
mated) standard error of the Monte Carlo average, given by
 n  2 1/2
1 
X 
 1X
n
 
√  
g(θ ) − g(θ i 
)  
 , (4.3)

n(n − 1) 
i
n 

i=1 i=1
2
when E{[g(θ)] | x] < ∞.
The integral of interest (4.1) can be represented by infinitely many vari-
ations involving consistent changes of the triple of parameter space, inte-
grand, and target distribution, (Θ, g, h) (Ripley, 1987). The Monte Carlo
estimators related to each of these representations come with different pre-
cisions, and corresponding implications for the computational effort (eas-
ier or more difficult random variate generation and larger or smaller Monte
Carlo sample size) required for getting reliable estimates. This suggests op-
portunities for more efficient tools to obtain highly precise estimates with
a relatively low Monte Carlo sample size.3
In summary, if one can generate from the posterior distribution h(θ | x),
then the evaluation of integrals of the type (4.1) is straightforward. Note
also that a simulated Monte Carlo sample considerably simplifies many
otherwise analytically difficult inference summaries. This is the case, for
example, for reparametrizations and marginalization, which are simply car-
ried out by the corresponding transformations of the simulated sample and
the selection of the components of interest, respectively. The following
subsections discuss in more detail the evaluation of posterior probabilities,
3
For details on variance reduction techniques for Monte Carlo estimates see, for example,
Rubinstein (1981) and Robert and Casella (2004).
 4.1 Simple Monte Carlo 47

marginal posterior distributions, credible intervals, and posterior predictive

summaries.

Example 4.1 Phase I clinical trials for chemotherapy agents in cancer

aim to establish the maximum tolerable dose (MTD) that can be adminis-
tered without excessive probability of a dose-limiting toxicity (DLT). As-
suming that higher doses are more effective for the tumor treatment, we
wish to find the highest dose possible without excessive toxicity. However,
some toxicity has to be tolerated, otherwise the treatment would be ineffec-
tive. Investigators might therefore, for example, aim to find the highest dose
with probability of DLT less than π⋆ = 30%. The continual reassessment
method (CRM) is one of the first Bayesian model-based designs for such
trials (O’Quigley et al, 1990). In its most basic form, this method char-
acterizes the dose–toxicity relationship by simple one-parameter models,
such as the hyperbolic tangent model, the logistic model, or the power
model. Let δk , k = 1, . . . , K denote the available dose levels in the trial and
let πk be the probability of a DLT at the k-th dose. The hyperbolic tangent
model assumes
" #a
a exp(δk )
πk (a) = [(tanh(δk ) + 1)/2] = . (4.4)
exp(δk ) + exp(−δk )
To highlight the nature of πk as a function of the unknown parameter a,
we write πk (a). Let π0k denote an a priori expert judgement of the expected
toxicity at the k-th dose level. Recording doses as standardized dose levels
δk = tanh−1 (2π0k − 1), we can match the prior mean E(πk | a = 1) with the
expert prior judgment, i.e., E(πk | a = 1) = π0k .
Suppose then that in developing a new agent, K = 6 dose levels are to
be studied. We assume a hyperbolic tangent dose–toxicity curve with prior
distribution h(a) = Exp(1). The targeted toxicity level is set at π⋆ = 20%.
The dose levels and our prior beliefs regarding the probability of toxicity
at each dose level are given in Table 4.1. Assume, then, that the first four
patients are treated at dose levels ki = 3, 4, 4, and 5, that is, patient i = 1
was treated with dose δ3 , etc. Let yi ∈ {0, 1} denote an indicator for the i-th
patient reporting a DLT. Assume that the observed toxicity outcomes are
yi = 0, 0, 0 and 1. Under (4.4), the sampling model is f (yi = 1 | ki = k, a) =
πk (a).
Let Di = (y1 , . . . , yi ) denote the data up to the ith patient. To carry out the
CRM we need to compute πk ≡ E{πk (a) | Di } for k = 1, . . . , K. This takes
the form of posterior expectations as in (4.1) with θ = a and g(·) = πk (·).
Consider the moment just before the enrollment of the (i + 1) = 5-th patient
48 Inference by Monte Carlo Methods

Table 4.1 CRM: dose levels, prior probabilities of toxicity π0k , and
standardized doses δk .
Dose level
k 1 2 3 4 5 6
Dose (mg/m2 ) 10 20 40 60 75 90
Prob(toxicity) π0k 0.05 0.10 0.20 0.30 0.50 0.70
Standardized dose δk –1.47 –1.1 –0.69 –0.42 0 0.42

and compute πk , k = 1, . . . , K. The posterior distribution h(a | D4 ) is

h(a | D4 ) ∝ e−a (1 − π3 (a))(1 − π4 (a))2 π5 =

e−a (exp(2δ3 ) + 1)−a (exp(2δ4 ) + 1)−2a (1 + exp(−2δt ))−a .
We use Monte Carlo integration to evaluate πk . We generate M = 5000
samples am ∼ h(a | D4 ), m = 1, . . . , M (using, for example, the R macro
P
sim.x() in Appendix B) and evaluate b πk = M1 πk (am ) using Monte Carlo
averages as in (4.2). Figure 4.1 shows the posterior estimates and the esti-
mated posterior distributions h{πk (a) | D4 }. The CRM method would then
πk ≤ π⋆ . In
call for the next patient to be treated at the highest dose k with b
⋆
this case we find k = 3, which would then be assigned to the next enrolled
patient. 

4.1.1 Posterior Probabilities

If g(θ) = IA (θ) is an indicator function for some event A in the parame-
ter space, then the Monte Carlo estimate (4.2) becomes the proportion of
simulated samples that fall into A. As a specific example, consider the eval-
uation of the posterior probability of the shortest HPD set that contains a
specific fixed value θ0 ∈ R,
P(θ0 ) ≡ Ph(θ|x) ({θ : h(θ | x) ≥ h(θ0 | x)}) ,
which we introduced in Chapter 1 as a way to construct Bayesian hypothe-
sis tests for H0 : θ = θ0 (posterior plausibility). The evaluation of this level
of relative posterior plausibility does not require the normalization constant
of a univariate h(θ | x). The corresponding Monte Carlo estimate becomes
1
P̂(θ0 ) = # {θi , 1 ≤ i ≤ n : L(θi | x) h(θi ) ≥ L(θ0 | x) h(θ0 )} . (4.5)
n
 4.1 Simple Monte Carlo 49

15
●

k
0.6

1
2
3
0.5

4
5

10
●
6
0.4

h(πk | y)
p(Y=1)
0.3

5
0.2

●
0.1

0
1 2 3 4 5 6 0.0 0.2 0.4 0.6 0.8 1.0
DOSE πk

(a) πk (b) h(πk (a) | D4 ).

Figure 4.1 Panel (a) shows the posterior mean toxicities

evaluated by Monte Carlo averages b πk . The vertical line segments
show ±0.5 posterior standard deviations. Panel (b) shows the
marginal posterior distributions h(πk | D4 ), evaluated as kernel
density estimates of the histograms of {πk (am ); m = 1, . . . , M}.

4.1.2 Credible Intervals

Consider now a Monte Carlo sample (θi , 1 ≤ i ≤ n) from a univariate pos-
terior distribution h(θ | x), with cumulative distribution function H(θ | x),
and assume that we wish to summarize the posterior distribution with a
credible interval R(γ) at level γ. The construction of such an interval re-
quires complete knowledge of the posterior distribution. In the case of an
unknown normalization constant, one could exploit the Monte Carlo sam-
ple to obtain an approximation of the credible interval from corresponding
empirical quantiles.
A Monte Carlo approximation of a central γ-credible interval R∗ (γ) is
obtained by sorting the Monte Carlo sample and using empirical quantiles.
Specifically, letting (θ(i) , 1 ≤ i ≤ n) denote the ordered sample, the Monte
Carlo estimate of Rc (γ) is
" !# " !#

1 γ 1 γ
R̂∗ (γ) = θ(ℓ) , θ(h) with ℓ = n − ,h= n + , (4.6)
2 2 2 2
where [nα] is the integer part of nα.
The best interval summary of anunimodal, possibly
o asymmetric distribu-
tion is the HPD interval R0 (γ) = θ : h(θ | x) ≥ kγ , where kγ is the largest
50 Inference by Monte Carlo Methods

threshold such that the posterior probability of R0 (γ) is at least γ. The def-
inition makes this interval more difficult to evaluate than intervals with
fixed, predetermined tail areas, even if a closed-form expression for the
posterior density of θ is available. Chen and Shao (1999) propose a Monte
Carlo approach to approximate R0 (γ) which is extremely simple to imple-
ment. Based on an ordered Monte Carlo sample (θ(i) , 1 ≤ i ≤ n), credible
intervals at level γ can be determined by
   
R̂i (γ) = θ(i) , θ(i+[nγ]) , i = 1, . . . , n − nγ ,
 
where nγ denotes the integer part of nγ. Taking into account the minimum
length property of HPD intervals, the Monte Carlo approximation of R0 (γ)
proposed by Chen h and Shao is defined
i as R̂0 (γ) = Ri0 (γ), with i0 determined
 
by i0 = arg mini θ(i+[nγ]) − θ(i) , 1 ≤ i ≤ n − nγ .4
Note that the method is straightforward to modify for HPD intervals for
parametric functions ψ(θ). It suffices to apply it to the transformed Monte
Carlo sample (ψ(θi ), 1 ≤ i ≤ n). However, recall that the HPD nature is not
invariant under non-linear transformations.

4.1.3 Marginal Posterior Distributions

Assume θ = (θ1 , . . . , θk ) ∈ Rk , k > 1, and that the goal is to evalu-
ate marginal posterior densities based on a Monte Carlo sample θ(i) =
(θ(i)1 , . . . , θ(i)k ), 1 ≤ i ≤ n, from h(θ | x) (using subscripts (i) to index sam-
ples and (i)m to index components of θ(i) ∈ Rk ). There are several methods
one could use.
When the goal is the evaluation of a marginal density, say h(θ j | x),
the simplest method is to select the jth component of each multivariate
Monte Carlo sample, create the histogram based on the resulting univari-
ate sample (θ(1) j , . . . , θ(n) j ), and fit a curve to the histogram using some
simple smoothing method. A more sophisticated non parametric smooth-
ing method for the evaluation of the marginal density h(θ(m) | x), where
θ(m) = (θ1 , . . . , θm ) ∈ Rm for some fixed m = 1, . . . , k − 1, is the so-called
kernel method. A description can be found in any book on non parametric
methods (e.g., Silverman, 1986).
To introduce another method based on conditioning, assume for a mo-
ment k = 2 and let Θ denote the support of the posterior density h(θ1 , θ2 | x)
for θ = (θ1 , θ2 ). Let Θ−1 (θ1 ) = {θ2 : (θ1 , θ2 ) ∈ Θ} denote the subset of Θ
4
See Chen et al.(2000, ch. 7) about the asymptotic validity of the approximation R̂0 (γ).
 4.1 Simple Monte Carlo 51

which constitutes the support of h(θ1 , θ2 | x) for fixed θ1 , and let Θ1 (θ2 ) =
{θ1 : (θ1 , θ2 ) ∈ Θ} denote the support of the conditional density h(θ1 | θ2 , x).
For a fixed value θ1∗ of θ1 , let (assuming Fubini’s theorem applies)
R
h(θ1∗ | x) = Θ (θ ) h(θ1∗ | θ2 , x)h(θ2 | x) dθ2
R −1 1∗ nR o
= Θ (θ ) h(θ1∗ | θ2 , x) Θ (θ ) h(θ1 , θ2 | x) dθ1 dθ2 (4.7)
R −1 1∗ 1 2
= Θ h(θ1∗ | θ2 , x)h(θ | x) dθ,
which shows that the ordinates of the marginal posterior density for θ1 can
be interpreted as posterior expected values (with respect to θ, including in
particular θ2 ) of the corresponding ordinates of the conditional posterior
density for θ1 .  
Generalizing this argument for θ = θ(m) , θ(−m) , with θ(−m) = (θm+1 , . . . , θk ),
we get
Z
(m)
h(θ∗ | x) = h(θ∗(m) | θ(−m) , x)h(θ | x) dθ. (4.8)
Θ

The expression implies that the marginal posterior density for θ(m) = (θ1 , . . . ,
θm ) can be approximated with a Monte Carlo estimate based on a ran-
(m) (−m) (m)

dom sample of h(θ | x), θ(i) = (θ(i) , θ(i) ) with θ(i) = θ(i)1 , . . . , θ(i)m and
(−m)

θ(i) = θ(i)m+1 , . . . , θ(i)k , i = 1, . . . , n, as
  1X n  (−m)

(m)
ĥ θ∗ = h θ∗(m) | θ(i) ,x . (4.9)
n i=1
This estimate was proposed by Gelfand and Smith (1990; see also Gelfand
(m)
et al., 1992). It does not make use of the part θ(i) , i = 1, . . . , n, of the
simulated values on which the kernel density estimate is based, but instead
requires complete knowledge of the conditional posterior density of θ(m)
given θ(−m) .
Assuming this condition is met, the estimate (4.9) turns out to be more
efficient than the estimate obtained by the kernel method, as was shown by
Gelfand and Smith (1990). This is because it exploits the knowledge of the
model structure as it is embodied in the required conditional distribution.
Something similar happens with the following estimate of the posterior
mean of θ(m) that, accounting for (4.9), is given by
1 X  (m) (−m) 
n
(m)
θ̂ = E θ | θ(i) , x . (4.10)
n i=1
This estimator, using the availability of the indicated conditional expec-
tation, is more precise than the classical Monte Carlo estimator obtained
 52 Inference by Monte Carlo Methods

from a sample of the marginal distribution of θ(m) , due to properties of con-

ditional expectations.5

4.1.4 Predictive Summaries

Noting that the ordinates of the posterior predictive density of Y are the
 
expectations p(y | x) = Eθ|x f (y | θ, x) , one can easily derive the Monte
Carlo approximation
1X
n
p̂(y | x) = f (y | θi , x) (4.11)
n i=1
based on a posterior Monte Carlo sample of draws from h(θ | x).
For Monte Carlo estimation of summaries associated with the predictive
model p(y | x), one needs a random sample from this distribution. This
is possible by what is known as the method of composition (Tanner, 1996;
section 3.3) if one can simulate from the sampling model for y. The method
generates a sample (y1 , . . . , yn ) from p(y | x) as follows.
1. Generate an i.i.d. sample of size n from h(θ | x), (θ1 , . . . , θn ).
2. For each i, generate yi from f (y | θi , x), i = 1, . . . , n.
Based on this Monte Carlo sample, one can then easily evaluate approxi-
mations for various predictive summaries. For example, an estimate of the
predictive mean and predictive HPD intervals for future observations y ∈ R
is obtained in the same way as we did for the posterior mean and HPD
credible intervals for θ on the basis of a posterior Monte Carlo sample of
θ, as we discussed before.

4.2 Monte Carlo with Importance Sampling

Although there are random variate generators for many specific distribu-
tions (see, e.g., Ripley, 1987), it is in many cases not possible to generate
an i.i.d. sample from the posterior h(θ | x), making it necessary to consider
alternative strategies. One of the possible strategies is based on simulating
from a distribution which is “similar” to the desired posterior distribution.
5
The argument is analogous to the Rao–Blackwell theorem, which establishes that the
expected value of an estimator conditional on a sufficient statistic is an estimator with
equal bias but more efficient than the original one. Even if the context here is not the
same, the same term, Rao–Blackwellization, which is used for the conditioning is also
used for the conditioning in (4.10) and (4.9).
 4.2 Monte Carlo with Importance Sampling 53

An example of this type of methods is known as importance sampling,

which we describe in the following. Until around 1990, importance sam-
pling used to be the method of choice for posterior simulation, but is now
much less widely used. We still include it here, for historical context, and
also because the underlying ideas remain useful in many applications.
Let p(θ) be a density whose support, say Θ p , contains the support of h(θ |
x) = c f (x | θ)h(θ) and which is proposed as a tool for the desired sampling.
Assume the quantity of interest is the expectation of g(θ) with respect to
the posterior distribution on θ. It can be written using the distribution p(θ)
as Z Z
h(θ | x)
g≡ g(θ) h(θ | x)dθ = g(θ) p(θ)dθ,
p(θ)
that is, as expected value of the original function g with respect to p, but
adjusted by a multiplicative factor h(θ | x)/p(θ). The latter expectation al-
ways exists by the assumption about the support of the distribution p(θ). In
the light of what was discussed in the previous section, the idea of simu-
lating from p(θ) instead of from h(θ | x) naturally leads to estimating the
quantity of interest by (Θ p , (gh)/p, p), that is, estimating it as an integral of
(gh)/p with respect to p over Θ p .
Also, this new representation of the quantity of interest requires only that
the posterior be known up to a proportionality constant c, and the same is
true for p(θ). In fact,
Z R
g(θ) f (x | θ)h(θ)dθ
g(θ)h(θ | x)dθ = R (4.12)
f (x | θ)h(θ)dθ
R R
g(θ) f (x|θ)h(θ)
p(θ)
p(θ)dθ g(θ)w(θ)p(θ)dθ
= R f (x|θ)h(θ) = R ,
p(θ)
p(θ)dθ w(θ)p(θ)dθ
with w(θ) = f (x | θ) h(θ)/p(θ). The density p(·) is known as the importance
function, possibly because it allows one to better explore the region of the
parameter space that is most important for the evaluation of the integral
of interest. Accordingly, the process of simulation from this distribution is
known as importance sampling.
Assume, then, that (θ1 , . . . , θn ) is an importance sample from p(θ). Us-
ing wi = w(θi ), one can now apply Monte Carlo methods to approximate
 
E g(θ) | x as
1 X
n
 
Ê g(θ) | x = Pn wi g(θi ). (4.13)
i=1 wi i=1
54 Inference by Monte Carlo Methods

The form of this estimator shows that importance sampling could be seen
as weighted sampling, with the weights wi for g(θi ) known as importance
sampling weights. Under appropriate assumptions, that is, in particular
Rthat the support of p(θ) includes the support of h(θ | x) and the integral
g(θ)h(θ | x)dθ exists and is finite, Geweke (1989) shows that with an
i.i.d. importance sample θi from p(θ),
Z
1 X
n
Pn wi g(θi ) → g(θ)h(θ | x)dθ a.s.,
i=1 wi i=1

with a Monte Carlo standard error that can be estimated by

"Xn  Xn 2 #1/2
1 1
σn = Pn
b g(θi ) − Pn wi g(θi ) w2i . (4.14)
j=1 w j i=1 j=1 w j i=1

The result assumes a finite variance of the Monte Carlo estimator, that is,
of the posterior expected value of the product of [g(θ)]2 and the importance
weight h(θ | x)/p(θ) (this is identical to the expected value with respect to
p of the square of g(θ)h(θ | x)/p(θ)).
The rate of convergence for the importance sampling estimator depends
on the ratio between the importance distribution and the target distribution
h(θ | x). Note that the estimator gives more weight to θ values with p(θ) <
h(θ | x) and less when the opposite inequality holds. If the importance
ratio is unbounded, as happens if the tails of p are lighter than the tails of
h(· | x), then the weights can vary substantially and give great importance to
few simulated values in a range (tails) with low probability under the target
distribution. Depending on the function g, the variance of the estimator
could even be infinite, implying far worse performance than an estimator
based directly on the target distribution, if that were possible.
In summary, whether Monte Carlo with importance sampling is a promis-
ing technique for variance reduction depends on the selected importance
sampling density (for more details, see Robert and Casella, 2004). Desir-
able properties for a good importance sampling density are: (1) easy ran-
dom variate generation; (2) heavier tails than the target distribution h(· | x);
and (3) good approximation of h(· | x). Shaw (1988) developed a class of
univariate distributions that are designed to be suitable as importance func-
tions (see also Smith, 1991). In the case of multivariate parameters, often
multivariate normals or Student t distributions are used.6

6
The program BAYESPACK provides an R interface for Fortran subroutines that
implement numerical integration by importance sampling, based on methods described
 4.2 Monte Carlo with Importance Sampling 55

Example 4.2 Recall the setup from Example 4.1. The single-parameter
model (4.4) was deliberately chosen to allow meaningful inference also
with small sample sizes in a phase I trial. Still keeping a parsimonious
model, but allowing for some more flexibility Neuenschwander et al (2008)
propose a design based on a two-parameter probit regression,
f (yi = 1 | di = δk ) = πk with πk = 1 − Φ [−a − b ln(δk /δo )] . (4.15)
Here, a and b are probit regression coefficients and δo is a reference dose.
The model allows easy interpretation of the parameters, with a determined
by the prior odds at the reference dose δo , and b determines the shift in
log odds for doses away from δo . The model is completed with a bivariate
normal prior (a, b) ∼ N(µ, Σ). We set the prior mean to µ = (1.65, 1.327)
to match a prior expectation E(π1 ) = 0.01 and E(πK ) = 0.95, respectively,
and Σ = I to have wide prior support over a range of plausible dose–
response curves.
Noting that it is impossible to define a single precise target toxicity π⋆ ,
Neuenschwander et al (2008) extend the notion of a single target toxicity
level π⋆ to an ordinal scale over four sub-intervals of toxicity probabil-
ities. Neuenschwander et al (2008) partition the range of toxicity proba-
bilities πk into four intervals: under-dosing, Iu = (0, 0.20]; targeted toxi-
city, It = (0.20, 0.35]; excessive toxicity, Ie = (0.35, 0.60]; and unaccept-
able toxicity, Ia = (0.60, 1.00]. Based on the posterior probabilities of πk ,
k = 1, . . . , K, being in these four intervals they propose a pragmatic design
that prescribes de-escalation, escalation, and continued enrollment at the
current dose, depending on these four probabilities.
Consider a study with a dose grid δ = (12.5, 25, 50, 100, 150, 200, 250)
with reference dose δ0 = 250. Assume that the first n = 7 patients were as-
signed doses ki = 1, 1, 2, 2, 3, 2, 3, with recorded outcomes yi = 0, 0, 0, 0, 1, 0, 1
We use importance sampling to find P(πk ∈ It | Dn ) and similar for Ie and
Ia , for k = 1, . . . , K. Let θ = (a, b). We use a bivariate normal importance
function p(θ) = N(m, V), with m being the posterior mode (MAP) and V
being the negative inverse of the Hessian of the log posterior at the mode,7
i.e., S = −H −1 . Figure 4.2 summarizes posterior inference. The trial design
would then determine the dose with maximum p(πk ∈ It | Dn ), restricting
the search to all doses with p(πk ∈ Ie ∪ Ia | Dn ) < 0.25. 

by Genz and Kass (1997). The package can be downloaded from

http://cran.r-project.org/src/contrib/Archive/bayespack.
7
See Section 8.1.1 for more discussion of normal approximations of the posterior
distribution.
56 Inference by Monte Carlo Methods
60

1.0

1.0
A A A
● A
● U
●
A
50

0.8

0.8
●
40

0.6

0.6
U

p(πk ∈ Ix | y)
Frequency

E(πk | y)
30

0.4

0.4
E
20

A
T T

0.2

0.2
●
10

E E
U
● T E
O
E A T E E T

0.0

0.0
A U U
T U
T U
T
0

U
0.2 0.4 0.6 0.8 1.0 2 4 6 8 2 4 6 8
w DOSE DOSE

(a) w(m) (b) πk (c) p(πk ∈ I | Dn )

Figure 4.2 Ordinal toxicity intervals: Panel (a) shows the

importance sampling weights w(m) . Panel (b) shows the posterior
estimated probabilities of DLT, πk = E(πk | Dn ) after n = 7
patients. The short vertical line segments show ±1 posterior
standard deviation for πk . Panel (c) plots the posterior
probabilities p(πk ∈ I | Dn ) for the four intervals I = Iu , It , Io , and
Ia (marked with “U,” “T,” “O,” and “A,” respectively).

In a similar fashion as in the previous section, we now proceed to de-

scribe applications of this method (and variations) for inference related to
credible intervals, Bayes factors and marginal posterior densities.8

4.2.1 Credible Intervals

For more generality we now assume that the parameter vector in the sam-
pling model is partitioned as θ = (γ, φ), where γ is a univariate parameter
of interest and φ is a vector of nuisance parameters (in the discussion cor-
responding to Section 4.1., there is no φ and thus θ = γ).
Let (γi , φi ), 1 ≤ i ≤ n, denote a random sample from an importance
sampling density p(γ, φ), designed for use with a posterior distribution
h(γ, φ | x) ∝ L(γ, φ | x)h(γ, φ). The c.d.f. of the marginal posterior dis-
tribution of γ in γ∗ is then
h i
H (γ∗ | x) = E I(−∞,γ∗ ) (γ) | x
R
I(−∞,γ∗ ) (γ) L(γ,φ|x)h(γ,φ)
p(γ,φ)
p(γ, φ)dγdφ
= R L(γ,φ|x)h(γ,φ) .
p(γ,φ)
p(γ, φ)dγdφ
8
Additional discussion can be found in the books by Paulino et al (2018) and Chen et al
(2000).
 4.2 Monte Carlo with Importance Sampling 57

It can be approximated by the weighted Monte Carlo average:

1X
n
Ĥ (γ∗ | x) = wi I(−∞,γ∗ ) (γi ), (4.16)
n i=1
with weights
L (γi , φi | x) h (γi , φi ) /p (γi , φi )
wi = Pn      .
1
n j=1 L γ j , φ j | x h γ j , φ j /p γ j , φ j

(Note that in variation from (4.13), we now include the normalization of

the weights in wi already.) Denote the sample ordered by the value of γi


as γ(i) , φ(i) , 1 ≤ i ≤ n, where φ(i) is the value that was simulated together
with γ(i) (that is, not the ith order statistic of φ, if φ were univariate), and


let w(i) denote the respective weight for the pair γ(i) , φ(i) . We then define a
weighted empirical distribution function of γ as



 0, γ∗ < γ(1)
 Pi
Ĥ (γ∗ | x) = 
 j=1 w( j) /n, γ (i) ≤ γ∗ < γ(i+1) ,

 1, γ ≥γ
∗ (n)

which naturally matches the empirical distribution function if

p(γ, φ) = h(γ, φ | x) (and thus w(i) = c, ∀i).
Let γα = inf {γ : H(γ | x) ≥ α} denote the α-quantile of the marginal
posterior of γ. Its Monte Carlo estimate is
(
γ(1) , α = 0
γ̂α = P 1 Pi
γ(i) , 1n i−1
j=1 w( j) < α ≤ n j=1 w( j) .

The central 1 − α credible interval, R∗ (1 − α), is (consistently) estimated by

 
R̂∗ (1 − α) = γ̂ α2 , γ̂1− α2 . (4.17)
Similarly, letting
 
R̂i (1 − α) = γ̂ ni , γ̂ i+[n(1−α)] , i = 1, . . . , n − [n(1 − α)]
n

denote a sequence of (1 − α) credible intervals for γ, the HPD interval

R0 (1 − α) can be estimated by R̂0 (1 − α) = R̂i0 (1 − α), where R̂i0 (1 − α) is
the minimum length interval in this sequence.
For credible intervals regarding any real-valued function ψ(γ, φ), it suf-
fices to order the values ψi = ψ(γi , φi ) and estimate the quantiles by the
same scheme, which requires only the rearrangement of the same weights
w(i) .
58 Inference by Monte Carlo Methods

4.2.2 Bayes Factors

Credible intervals are often used as a means to construct Bayesian signifi-
cance tests, as was mentioned in Chapter 1. However, the idea of compar-
ing statements by their posterior probabilities implies that hypothesis tests
(as well as the usual model comparison procedures) should make use of
Bayes factors, which in general could be complicated ratios of marginal
likelihoods.
Specifically, the Bayes factor in favor of H0 versus the model H1 is the
ratio of marginal likelihoods B(x) = p(x | H0 )/p(x | H1 ), where
Z
p(x | Hk ) = f (x | θk , Hk )h(θk | Hk ) dθk , k = 0, 1.
Θk

This is the ratio of the normalizing constants for f (x | θk , Hk ) h(θk | Hk ),

k = 0, 1. Therefore the methods that we shall describe for evaluating B(x)
are equally applicable for the evaluation of any ratio of normalization con-
stants.
We start by considering the case when the densities with respect to which
we integrate in numerator and denominator are of the same dimension. In
that case notation need not distinguish parameters under H0 and H1 , and
we write θ ≡ θ0 = θ1 . For simpler notation we also use
h(θ | x, Hk ) = h̄k (θ)/ck ,
with support Θk . Here, h̄k is the un-normalized product of prior times like-
lihood, and ck = p(x | Hk ), k = 0, 1 is the normalization constant. Assume,
then, that one wishes to evaluate the ratio
R
c0 h̄ (θ)dθ
Θ 0
B(x) = =R 0
c1 h̄1 (θ)dθ
Θ1

for analytically intractable (or difficult to evaluate) ck .

Let pk (θ) denote an importance
n sampling
o density for h̄k , assumed com-
pletely known, and let θ1(k) , . . . , θn(k)k , k = 0, 1, denote corresponding im-
portance samples. Direct application of Monte Carlo with importance sam-
pling for each ck gives the following approximation:
 
nk h̄ θ(k)
ĉ0 1 X k i
B̂1 (x) = , ĉk =  (k)  , k = 0, 1. (4.18)
ĉ1 nk i=1 pk θ
i

By the law of large numbers this is a consistent estimator for B(x).

Note that alternatively one could evaluate ck with independent simple
Monte Carlo methods using sampling from the prior distribution, to give
 4.2 Monte Carlo with Importance Sampling 59
 
1 Pnk (k)
nk i=1 f x | θi , Hk , with the simulated values now being a random sam-
ple of size nk from the prior h(θ | Hk ) on θ under Hk . However, the prior
could be substantially different from the likelihood function, in particular
over subsets which are a priori little plausible, implying that this could
be a rather poor estimate of ck . See Chapter 5 for alternative methods to
evaluate ck .
In the special case of Θ0 ⊂ Θ1 it is possible to use a Monte Carlo ap-
proach for the evaluation of B(x), using only a sample generated from h1 (θ)
(which could be obtained without knowing c1 , as we shall see in Chapter
6). In fact,
Z " #
c0 h̄0 (θ) h̄0 (θ)
B(x) = = dθ = Eh1 .
c1 Θ1 c1 h̄1 (θ)
 
Assume now that θi(1) , i = 1, . . . , n1 is a random sample from h1 . Then
 
n1 h̄ θ(1)
1 X 0 i
B̂2 (x) =   (4.19)
n1 i=1 h̄1 θ(1)
i

is a (unbiased and consistent) Monte Carlo estimate of B(x). Note that this
approximation is most efficient when the tails of h1 (θ) are heavier than the
tails of h0 (θ) and less efficient when there is little overlap of h0 (θ) and h1 (θ)
(that is, small Eh1 [h0 (θ)]).
In many problems, the Bayes factor involves densities of different di-
mension and therefore requires different strategies. We will discuss some
in Chapter 7. For a more general discussion of this problem, see the books
by Chen et al (2000) and Paulino et al (2018).

4.2.3 Marginal Posterior Densities

As we explained in Section 4.1.3, the evaluation of marginal posterior den-
sities by the conditional approach of Gelfand and Smith (1990) requires full
knowledge of the corresponding conditional posterior densities (not just the
kernels), which unfortunately is not commonly available, and availability
of a Monte Carlo sample from the marginal posterior distribution for θ(−m) .
If the latter is not available but it is possible to instead simulate from a
substitute p(·) of h(θ(−m) | x), then one can define an importance sampling
estimate of the marginal density for θ(m)
Pn  
  i=1 wi h θ∗(m) | θ(i)
(−m)
,x
ĥ θ∗(m) | x = Pn , (4.20)
i=1 wi
60 Inference by Monte Carlo Methods

where
 (−m)   (−m) 
wi = h θ(i) | x /p θ(i) ,
i = 1, . . . , n. Alternatively, with an argument as in (4.8), one could use



wi = h θ(i) | x /p θ(i) , i = 1, . . . , n, with an importance sampling density
p(·) for h(θ | x).
(m)
As yet another alternative, one could generate θ(i) , i = 1, . . . , n, by the
(−m) (m) (−m)
composition method using p(θ ) and h(θ | θ , x). However, since we
(m)
used p(·) instead of the marginal posterior for θ(−m) , the samples θ(i) are not
(m) (m)
a random sample from h(θ | x). Instead, the θ(i) are a weighted sample of
P
an approximation of h(θ(m) | x), with weights pi = wi / ni=1 wi , i = 1, . . . , n.
To get from this a random sample of size L, say, one could simulate in the
spirit of a bootstrap method L values θ((m) j)∗ , j = 1, . . . , L, from the discrete
(m)
distribution (θ(i) , pi ), i = 1, . . . , n. Smith and Gelfand (1992) therefore used
the name weighted bootstrap.
This method to evaluate marginal posterior densities, similar to Gelfand
and Smith (1990), does not apply when the normalization constant for the
conditional posterior density h(θ(m) | θ(−m) , x) is unknown.  (m)  In this case,
Chen (1994) proposes a weighted estimator for h θ∗ | x , based on the
following identity
  R  
h θ∗(m) | x = Θ (θ(m) ) h θ∗(m) , θ(−m) | x dθ(−m)
R −m ∗  h(θ(m) ,θ(−m) |x) (4.21)
= Θ w θ(m) | θ(−m) h(θ∗(m) ,θ(−m) |x) h(θ | x)dθ,

where Θ−m (θ(m) ) = {θ(−m) : (θ(m) , θ(−m) ) ∈ Θ} is the subspace of Θ with fixed
θ(m) and w(θ(m) | θ(−m) ) is a completely known conditional density with the
same as or greater support than h(θ(m) | θ(−m) , x), given by Θm (θ(−m) ) =
{θ(m) : (θ(m) , θ(−m) ) ∈ Θ}. The criteria for selecting the function w are the
same as for selecting an importance sampling density.
The form of (4.21) highlights that the normalization constant of the joint
posterior distribution (and, a fortiori, conditional) cancel out and that the
marginal density in question can be unbiasedly estimated based on a sam-
(m) (−m)
ple θ(i) = (θ(i) , θ(i) ), 1 ≤ i ≤ n, of h(θ | x), by
 
  1X n  (m) (−m)  h θ∗(m) , θ(i)
(−m)
|x
(m)
ĥ θ∗ | x = w θ(i) | θ(i)  . (4.22)
n i=1 h θ(m) , θ(−m) | x
(i) (i)
 4.3 Sequential Monte Carlo 61

4.3 Sequential Monte Carlo

Time series data and related models give rise to challenging posterior in-
tergration problems. Some of these problems can be approached by Monte
Carlo simulation methods known as sequential Monte Carlo (SMC), which
is the subject of this section. We start with a brief review of dynamic state
space models, including popular normal dynamic linear models which al-
low analytic solutions and motivate some of the SMC schemes.

4.3.1 Dynamic State Space Models

With time series data yt collected over time, it is natural to index parameters
correspondingly as θt , t = 1, . . . , T . This allows, in particular, introduction
of dependence of yt across time as independent sampling conditional on
θt , with an evolution of θt over time inducing the desired marginal depen-
dence. Assuming normal linear models for both, the sampling model for
yt (observation equation, OE) and the evolution of θt over time (evolution
equation, EE) this defines the normal dynamic linear model (NDLM)
OE : yt = Ft′ θt + vt , vt ∼ N(0, Vt ),
(4.23)
EE : θt = Gt θt + wt , wt ∼ N(0, Wt ),
with θ0 ∼ N(m0 , C0 ). Here, yt ∈ R p is the observed data, θt ∈ Rq is a latent
state vector, and Ft and Gt are known design matrices that could possibly
include a regression on covariates.
Posterior updating in the NDLM is straightforward by the following fi-
nite recursion. Let Dt = (y1 , . . . , yt ) denote the data up to time t. Condi-
tioning on Dt defines a sequence of posterior distributions, including h(θt |
Dt−1 ) and h(θt | Dt ), for t = 1, . . . , T . Starting with h(θ0 | D0 ) = N(m0 , C0 ),
we easily find for t = 1, . . . , T :
θt | Dt−1 ∼ N(at , Rt ), at = Gt mt−1 , Rt = Gt Ct−1G′t + Wt
yt | Dt−1 ∼ N( ft , Qt ), ft = Ft′ at , Qt = Ft′ Rt Ft + Vt
(4.24)
θt | Dt ∼ N(mt , Ct ), mt = at + At et Ct = Rt − At Qt A′t
et = yt − ft and At = Rt Ft Q′t
A similar set of recursive equations obtains the posterior distributions
h(θt−k | Dt ) = N(at (−k), Rt (−k)),
for k = 1, . . . , t − 1. Starting with at (0) = mt and Rt (0) = Ct for k = 0, and
62 Inference by Monte Carlo Methods

defining Bt = Ct G′g+1 R−1

t+1 , we have

at (−k) = mt−k + Bt−k [at (−k + 1) − at−k+1 ],

Rt (−k) = Ct−k + Bt−k [Rt (−k + 1) − Rt−k+1 ]B′t−k . (4.25)
Without the initial prior at t = 0, or equivalently, C0−1 = 0, the recursive
equations determine the maximum likelihood estimate, and are known as
the Kalman filter. As posterior updating the same equations are known as
forward filtering (FF) for (4.24) and backward smoothing (BS) for (4.25).
See, for example, Prado and West (2010: ch. 4) for a detailed discussion of
the NDLM and many useful variations, including in particular the case of
unknown variances in the observation and evolution equations.
The NDLM (4.23) is a special case of more general dynamic state space
models. The characteristic feature is the hidden Markov nature of the model.
The data yt are independent conditional on latent states θt , and the prior for
the latent {θt , t = 1, 2, . . .} includes a Markov assumption with conditional
independence of θt and θt−k , k > 1, given θt−1 . The general dynamic state
space model is defined by a sampling model for yt given θt and a transition
probability for the Markov prior on the states:
f (yt | θt ) and p(θt | θt−1 ), (4.26)
for t = 1, 2, . . . . The model is completed with the initial prior h(θ0 ). Like
the NDLM, the general dynamic state space model gives rise to a sequence
of posterior distributions ht = h(θt | Dt ), predictive distributions, and more.
Many applications include additional (static) parameters φ in f (yt | θt , φ)
and/or p(θt | θt−1 , φ). See Section 4.3.4.

4.3.2 Particle Filter

Short of the special normal and linear features of the NDLM, posterior
inference in general dynamic state space models (4.26) is not analytically
tractable and posterior simulation is needed. Monte Carlo posterior simula-
tion methods for the sequence of posterior distributions ht = h(θt | Dt ) are
naturally set up in a sequential fashion, including a step to update a poste-
rior Monte Carlo sample Bt = {θt,i , i = 1, . . . , M} from ht to a Monte Carlo
sample Bt+1 from ht+1 . Such methods are known as SMC methods. A good
review appears, for example, in Doucet et al (2001) and, more recently, in
Doucet and Lee (2018).
For reference, we state the relevant posterior and posterior predictive
 4.3 Sequential Monte Carlo 63

distributions and define notation,

ht ≡ h(θt | Dt ) posterior at time t,
h′t ≡ h(θt | Dt−1 ) prior distribution at time t, (4.27)
ft ≡ p(yt | Dt−1 ) forecast distribution.
One of the earliest proposals for SMC is the auxiliary particle filter de-
scribed by Pitt and Shephard (1999). The algorithm starts with a Monte
Carlo sample B0 = {θ0,i , i = 1, . . . , M} from h(θ0 ), and then iteratively up-
dates Bt−1 into a Monte Carlo sample B′t = {θt,i ′
} from the prior distribution
′
ht = h(θt | Dt−1 ) and then a Monte Carlo sample Bt = {θt,i } from the poste-
rior ht at time t. In other words, the elements of B0 (“particles”) are pushed
through a sequence of updating steps to generate the desired Monte Carlo
samples from ft and ht . The key identities are the representation of h′t as a
convolution of ht−1 and the transition model as
Z
h′t (θt ) = h(θt | Dt−1 ) = p(θt | θt−1 ) dht−1 (θt−1 ) (4.28)

and posterior updating

ht (θt ) = h(θt | Dt ) ∝ h′t (θt ) f (yt | θt ). (4.29)
Rather than generating Monte Carlo samples from the target distributions,
the algorithm generates Monte Carlo samples from importance sampling
densities together with corresponding weights Wt = {wti , i = 1, . . . , M} for
Bt and Wt′ = {w′ti , i = 1, . . . , M} for B′t . Let b ht ≈ ht and b h′t ≈ h′t denote
b
the importance sampling densities. Then wti = ht (θt,i )/ht (θt,i ) and w′ti =
h′t (θt,i )/b
h′t (θt,i ), and posterior integrals with respect to the target distributions
can be approximated as
Z
1 X
M
g(θt )h(θt | Dt )dθt ≈ P wti g(θt,i ),
wti i=1
with the Monte Carlo average going over all particles θt,i ∈ Bt . In particular,
this allows us to approximate h′t by using the representation from (4.28) as
1 X
h′t (θt ) ≈ b
h′t (θt ) = P wt−1,i p(θt | θt−1,i ). (4.30)
wti
A basic particle filter could then proceed as follows. Assume Bt−1 and Wt−1
are available. First, generate B′t by sampling from b h′t (θt ). This is done by:
(1) sampling θt−1,i from Bt−1 with probabilities proportional to wt−1,i ; and (2)
′
generate θt,i ∼ p(θt′ | θt−1,i ) and record weights w′ti = 1/M. Finally, in step
′
(3), define Bt by setting θt,i ≡ θt,i and wti ∝ f (yt | θt,i ) proportional to the
 64 Inference by Monte Carlo Methods

likelihood factor for yt . This is essentially the idea of the auxiliary variable
particle filter, except that by combining steps (1) and (3), the algorithm
makes the sampling more efficient.
Similar to (4.30), we define
X
ht ≈ bht ∝ wt−1,i f (yt | θt )p(θt | θt−1,i ) (4.31)
i

as a Monte Carlo approximation of ht , based on Bt−1 and Wt−1 . In words,

the trick is to (1) augment b ht (θt ) to a joint model b h(θt , i); (2) approximate
b
h(θt , i) ≈ g(θt , i) by replacing θt in f (yt | θt ) by µt,i = E(θt,i′
| θt−1,i ); and (3)
generate (θt , i) ∼ g, and use weights wt,i = f (yt | θt,i )/ f (yt | µt,i ). The main
advantage of the algorithm is that we select the term i in b h′t in a way that
already anticipates the later multiplication with the likelihood factor. We
formally state steps 1 through 3:
1. Augment b
ht (θt ) in (4.31) to
b
h(θt , i) ∝ wt−1,i f (yt | θt )p(θt | θt−1,i ).
Substitute an approximation f (yt | θt ) ≈ f (yt | µt,i ), for example, using
µti = E(θt | θt−1,i ), to obtain
g(θt , i) ∝ wt−1,i f (yt | µt,i )p(θt | θt−1,i ). (4.32)
2. Let g(i) ∝ wt−1,i f (yt | µt,i ) denote the implied marginal on i under g(θt , i).
Generate i ∼ g(i) and θt,i | i ∼ p(θt | θt−1,i ).
bh(θ ,i) f (y |θ )
3. Record weights wti = g(θt,it,i ,i) = f (ytt|µt,it,i ) .

4.3.3 Adapted Particle Filter

Pitt and Shephard (1999) refer to the algorithm with f (yt | µti ) ≈ f (yt | θti )
as the basic adapted particle filter. Exploiting the specific structure of the
state space model, we can sometimes construct better adapted schemes.
The aim is to achieve weights wti with minimum variance.
Perfect adaptation, i.e., constant weights, is possible for a state space
model with normal sampling model and evolution (indexing the normal
model with mean and variance):
f (yt | θt ) = N(θt , 1) and p(θt | θt−1 ) = N[µ(θt−1 ), σ2 (θt−1 )], (4.33)
assuming for simplicity unit variance in the sampling model. In this case
g(θt , i) in step (i) from before can be replaced by b
h itself:
b
h(θt , i) ∝ wt−1,i N(yt | θt , 1)N(θt | µ(θt−1,i ), σ2 (θt−1,i )), (4.34)
 4.3 Sequential Monte Carlo 65

without the likelihood approximation in g(θt , i). Instead, the normal as-
sumption allows us to use Bayes’ theorem and replace the last two factors
to obtain
b
h(θt , i) ∝ wt−1,i N(θt | mi , vi ) N(yt | µ(θt−1,i ), σ2 (θt−1,i ) + 1), (4.35)
| {z }
λi

where (mi , vi ) are the posterior moments in a normal–normal model with

sampling model and prior given by those last two factors. In other words,
we replace sampling model times prior, f (yt | θt , . . .) × p(θt | θt−1 , . . .), in
(4.34) by posterior times marginal, h(θt | yt , . . .) × p(yt | . . .), in (4.35) (cf.
Example 2.8). This simplifies the auxiliary particle filter to
1′ . b
h(θt , i) ∝ wt−1,i λi N(θt | mi , vi )
2′ . Generate i ∼ b h(i) ∝ wt−1,i λi and θt | i ∼ N(θt | mi , vi ).
3′ . Record wti = 1/M.
A similar simplification with perfect adaptation is possible for any conju-
gate sampling model and evolution.

Example 4.3 Consider an ARCH model with additional independent nor-

mal residuals:
f (yt | θi ) = N(θt , σ2 ), f (θt+1 | θt ) = N(0, β0 + β1 θt2 ). (4.36)
With fixed static parameters (β0 , β1 , σ2 ), the model is a special case of
(4.33) and allows a perfectly adapted particle filter. Let N(x | m, s2 ) de-
note a normal p.d.f. with moments (m, s2 ), evaluated for x. In this case
2
λi = N(yt | 0, β0 + β1 θt,i + σ2 ) and (mi , vi ) are just the posterior moments
for θt+1 under the likelihood and prior in (4.36) 
In a general dynamic state space model, with arbitrary sampling model
f (yt | θt ), one can still implement steps (1′ ) and (2′ ) using a second-order
Taylor series approximation of ln f (yt | θt ). The expansion is in θt (Pitt and
Shephard, 1999). See Problem 4.9.

4.3.4 Parameter Learning

Recall the general dynamic state space model from (4.26). Often the model
might include unknown static parameters φ, in addition to the dynamic
parameters θt , defining a model:
p(yt | θt , φ) and p(θt | θt−1 , φ), (4.37)
66 Inference by Monte Carlo Methods

completed with a prior on the static parameters and the initial state (φ, θ0 ) ∼
h(φ, θ0 ). For example, in the ARCH model, φ = (β0 , β1 , σ2 ).
There are several approaches to generalizing particle filter methods to
include parameter learning. Liu and West (2001) reduce static parameter
learning to the earlier problem by allowing a negligibly small but positive
evolution noise for φ, thus including it as φt in the state vector. Polson
et al (2008) use a (approximate) sufficient statistic to define the “practical
filter.” For a review and more detailed discussion see, for example, Prado
and West (2010).

Problems
4.1 Simple Monte Carlo integration. Implement inference for Example 4.1.
a. Verify the posterior means πk shown in Figure 4.1, and add central 50%
credible intervals for πk , k = 1, . . . , K.
b. Assume the next patient is treated at dose k5 = 3, and we record no tox-
icity, y5 = 0. Find the updated posterior means πk = E(πk | D5 ) and
determine the treatment allocation k⋆ for the next patient.
4.2 Importance sampling: probit regression. Refer to Example 4.2.
a. Based on Figure 4.2 and the design rule as it is summarized in the exam-
ple, which dose k⋆ is assigned to the next, (n + 1)-st patient?
b. Let θ = (a, b). Evaluate the marginal posterior distributions h(πk | Dn )
using importance sampling, using a bivariate normal importance function
p(θ) = N(m, V), as in the example. Plot h(πk | Dn ), k = 1, . . . , K. In the
plot indicate the four intervals for under-dosing, target dose, excessive
dose, and unacceptable toxicity by vertical dotted lines on the interval
boundaries.
c. Let {θ(m) ; m = 1, . . . , M} denote the importance sample from p(θ). Figure
4.2(a) shows the importance weights w(m) ≡ h(θ(m) | y)/p(θ(m) ) for an
importance Monte Carlo sample {θ(m) ; m = 1, . . . , M} of size M = 100.
Redo the importance sampling estimate, now with M = 1000, and plot
the histogram of weights. What do you observe?
d. Now consider an alternative bivariate Student t importance function, p2 (θ) =
t2 (m, V; ν) with ν = 4. Plot a histogram of the importance sampling
weights w2(m) = h(θ(m) | y)/p2 (θ(m) ) and compare.
4.3 Importance sampling: nonlinear regression. Let x denote the number of
plants per unit area and y denote the yield per plant. To model the relation-
ship between yield and planting density, we use the sampling model
f (yi | θ) = N(µi , σ2 ) with µi = 1/(α + βxi + γxi2 ).
 Problems 67

Let τ = ln(σ2 ) and θ = (α, β, γ, τ). In this model, 1/α = lim x→0 y is inter-
preted as “genetic potential” and 1/β can be interpreted as “environmental
potential.”
The model is completed with a non-informative prior h(θ) ∝ 1. Use the data
set onion in the R package SemiPar. The data set includes the variables
dens, yield, and location. We use data from location 0 (Purnong
Landing) only. Scale y = yield/100 by 100, and standardize x as x =
(d − d̄)/sd , where d =density, and d̄, s2d are the sample mean and variance.
a. Set up a multivariate normal importance sampling density p(θ). Use a
N(m, V) with m being the posterior mode (MAP) and V = −H −1 being
the negative inverse Hessian of the log posterior at the mode.9 Carry out
importance sampling to estimate the posterior means and standard de-
viations of θ. Report numerical uncertainties and show a histogram of
importance sampling weights.
b. Repeat the same importance sampling estimation with a multivariate Stu-
dent t distribution with ν = 4 d.f. (see Appendix A). See also the hint for
Problem 4.4(b).
c. Why would one recommend the Student t importance function? Explain.
4.4 Importance sampling: hierarchical model. Carlin and Gelfand (1991) con-
sider a hierarchical event rate model for the number of pump failures (yi )
over given exposure times (ti ), i = 1, . . . , n = 10. The data are shown in the
following table.
yi 5 1 5 14 5
ti 94.32 15.72 62.88 125.76 5.24

yi 19 1 1 4 22
ti 31.44 1.048 1.048 2.096 10.48
We assume a Poisson regression,
yi ∼ Poi(λi ti ),
with a hierarchical prior for Li = ln(λi ):
Li ∼ N(η, σ2 ), and η ∼ N(µ, τ2 ), γ ≡ 1/σ2 ∼ Ga(a, b).
We fix the hyperparameters at (µ, τ2 , a, b) = (−1, 1, 1, 1).
Let g = ln(γ) and let θ = (η, g, L1 , . . . , Ln ) denote the complete parameter
vector (now with g = ln(γ)). Let θ̂ and S denote mean and covariance matrix
of a multivariate normal approximation of the joint posterior distribution
h(θ | y) (for example, posterior mode and negative inverse Hessian of the
log posterior at the mode, as in Problem 4.3). Under the multivariate N(θ̂, S )
9
We will discuss the use of such multivariate normal approximations in Chapter 8.
68 Inference by Monte Carlo Methods

distribution for θ let m = (θ̂1 , θ̂2 ) and V denote the corresponding marginal
moments for ψ = (η, g).
Let p1 (ψ) = t2 (m, V; ν) (use ν = 2), and p2 (Li | η, γ) ∝ N(Li | η, σ2 )Poi(yi |
ti eLi ). Here, Poi(y | µ) denotes the Poisson probability mass function with
rate µ evaluated for y and similar for N(x | µ, σ2 ). Use an importance sam-
pling density
p(θ) = p1 (η, g) Πni=1 p2 (Li | η, g). (4.38)
We can generate from p(·) by first generating ψ = (η, g) from the bivariate
t distribution p1 (ψ), and then Li from p2 using, for example, a grid-based
method using evaluation of p2 on a grid (using the R function sample();
see Appendix B).
a. Using the importance sampling function (4.38), find the expression for
the importance sampling weights wi .
Hint: Do not forget the normalization constant in p2 (Li | η, g).
b. Let y = (y1 , . . . , yn ) denote the data. Implement importance sampling
to evaluate posterior moments, λ̄i = E(λi | y) and s2i = Var(λi | y),
i = 1, . . . , n. Show a histogram of the importance sampling weights, a
table of estimated posterior moments λ̄i (si ), and corresponding numerical
uncertainties.
Hint: To generate from the bivariate t distribution tν (m, V) with ′
2
√ V = LL ,
you could use: z ∼ N2 (0, I), y = Lz, u ∼ χ (ν), and ψ = m + ν/uy. Then,
" #− ν+2
1 ′ −1
2 ν+2
p(ψ) ∝ 1 + (ψ − m) V (ψ − m) = [1 + z′ z/u]− 2
ν
c. Use suitable Monte Carlo methods to evaluate and plot the marginal pos-
terior distributions h(λi | y), i = 1, . . . , n.
d. The problem greatly simplifies with λi ∼ Ga(c, ηc ). Why?
4.5 The skewed normal distribution defines a skewed continuous random vari-
able. Let ϕm,s denote a normal p.d.f. with moments (m, s) and let Φ(z) denote
a standard normal c.d.f. The skewed normal can be defined by its p.d.f.,
 x − µ
f (x | µ, σ, α) ∝ ϕµ,σ (x) Φ α .
σ
For α > 0 (α < 0), the distribution is right (left) skewed. We write X ∼
SN(µ, σ, α) for a skewed normal random variable X. For the following ques-
tions use the data set stockreturns.txt10 of (simulated) stock returns.
We assume the sampling model
xi ∼ SN(µ, σ, α),
10
Available on the book’s homepage at
sites.google.com/view/computationalbayes/home
 Problems 69

i = 1, . . . , n, i.i.d., and complete the model with a conditionally conjugate

prior on µ, σ and a gamma prior on α:
1/σ2 ∼ Ga(a/2, aσ20 /2), h(µ | σ) = N(µ0 , κσ2 ) and h(α) = Ga(c, d),
with fixed hyperparameters κ, c, d, µ0 , a, and σ20 .
a. Let γ = 1/σ2 . For fixed α suggest an importance sampling strategy to
estimate posterior means (conditional on α). Let p(µ, γ) denote the im-
portance sampling density, and let w denote the importance sampling
weights. Propose a choice for p(·), and find a bound M such that w ≤ M.
b. Claim
h(α | x) = 2E {Φ(W)}
where the expectation is with respect to a t-distributed random variable
W. Show the claim, and then use the result to find the marginal posterior
mode of α over a grid α ∈ {1, 1.1, 1.2, 1.3, 1.4, 1.5}.
4.6 In a study, N = 97 alcoholics were classified as X1 – professional situation
(with or without employment), X2 – daily alcohol consumption (yes or no),
and X3 – expression of some phobia (yes or no), as listed in Table 4.2 (from
Paulino and Singer, 2006). The aim of the study was inference on possible
associations between the three binary variables X1 , X2 , and X3 . We assume a
multinomial sampling model for the observed counts, (ni jk , i, j, k = 0, 1) ∼
M7 (N, θi jk , i, j, k = 0, 1), and complete the model with an improper prior
h(θ) ∝ Πi, j,k θi−1
jk . The model implies a Dirichlet posterior distribution h(θ |
n) = D7 (A) with A = (10, 24, 6, 12, 13, 17, 4, 7).
We are interested in the hypothesis of conditional independence of X2 and
X3 given X1 , that is
θi11 θi22
HIC : θi jk = θi j· θi·k /θi·· , ∀i, j, k ⇔ ψi ≡ ln = 0, ∀i.
θi12 θi21
In short, ψi = a′i ln θ with a1 = (1, −1, −1, 1, 0, 0, 0, 0)′ and
a′2 = (0, 0, 0, 0, 1, −1, −1, 1)′ . The parameters of interest ψ = (ψ1 , ψ2 ) have a
distribution that is not easily available in closed form.

Table 4.2 observed frequencies ni jk

Professional Daily use phobia (X3 )
situation (X1 ) (X2 ) Yes No
Unemployed Yes 10 24
No 6 12
Employed Yes 13 17
No 4 7
70 Inference by Monte Carlo Methods

a. Use Monte Carlo simulation to estimate h(ψ1 , ψ2 | n). First, generate M =

10, 000 draws from the Dirichlet posterior distribution θ(m) ∼ h(θ | n), m =
1, . . . , M. Next, compute ψ(m) for each draw. Use the resulting posterior
Monte Carlo sample ψ(m) , m = 1, . . . , M to plot a bivariate histogram or
contour plot.
b. Use an approximation of h(ψ1 , ψ2 | n) on a bivariate grid to find a 95%
HPD credible region of ψ = (ψ1 , ψ2 ).
c. What evidence does the credible region provide about the hypothesis
HIC ?
4.7 Importance sampling. Let b gn denote
R the importance sampling estimate (4.13)
of the posterior expectation g = g(θ)h(θ | x)dθ. Using the central limit the-
n 
orem, Geweke (1989) shows n1/2 (gn −g) → N(0, σ2 ), with σ2 = E g(θ) − g 2
σ2n in (4.14). Show
Convergence is in distribution. Recall the definition of b
2 2
σn → σ . State the mode of convergence.
nb
Hint: See the discussion in Geweke (1989).
4.8 Consensus Monte Carlo. Consider a large data set y that is too big or just
impractical to process at one time. The idea of consensus Monte Carlo (Scott
et al, 2016) is to break the prohibitively big data y into small “shards,”
y s , s = 1, . . . , S , carry out posterior computation for each shard separately,
and then appropriately combine the subset posteriors to recover, or at least
approximate, what posterior inference under the complete data could have
been.
Assume y s | θ ∼ N(θ, Σ s ), s = 1, . . . , S , for known Σ s (for example, Σ s =
σ2 I), with conjugate prior h(θ) = N(m, T ).
a. Define a subset posterior as h s (θ | y) ∝ f (y s | θ) h(θ)1/S , i.e., as posterior
conditional on the subset y s under the modified prior h s (θ) ∝ h(θ)1/S .
Show that h s (θ | y s ) = N(m s , V s ), and that h(θ | y) = Π s h s (θ | y s ) =
N(m, V). Find m s , V s , m, and V.
b. Let θ s ∼ h s(θ | y) denote
 draws from the subset posteriors, s = 1, . . . , S .
P
Let θ = V s V s−1 θ s . Show that θ ∼ N(m, V), i.e., θ is a draw from the
joint posterior.
For multivariate normal posteriors, the described algorithm provides an exact
draw from the joint posterior. In general, one can argue that it provides a
reasonable approximate draw from the joint posterior that can be obtained
by carrying out computation for small shards only.
4.9 Adapted auxiliary particle filter. Recall steps (1′ ), (2′ ), and (3′ ) of the per-
fectly adapted auxiliary particle filter for model (4.33) in Section 4.3.3. Now
replace the normal sampling model in (4.33) by a general sampling model
f (yt | θt ).
a. Let ℓ(θt ) = ln f (yt | θt ) denote the log-likelihood factor for θt . Find (µt , σt )
 Problems 71

for the quadratic approximation ℓ(θt ) ≈ −1/(2σ2t )(θt −µt )2 , using a Taylor
series expansion of ℓ(θt ).
b. Use this quadratic approximation to derive a variation of (4.35).
c. Using that variation of (4.35), work out appropriate variations (1′′ ), (2′′ ),
and (3′′ ) of the adapted particle filter (the weights in step (3′′ ) are no
longer constant).
4.10 Consider a stochastic volatility model,
yt = ǫt β exp(θt /2) and θt+1 = φθt + ηt ,
with ǫt ∼ N(0, 1) and ηt ∼ N(0, σ2η ). For this problem we fix φ = 0.9702, ση =
0.178, and β = 0.5992 (Pitt and Shephard, 1999: section 4.2).
a. Simulate a data set y1 , . . . , yT , using θ0 = 0 and T = 200. State the random
variate seed that you use. For example, if you use R, use set.seed(1963).
Plot yt versus t as a time series.
b. Let ℓ(θt ) = ln f (yt | θt ) denote the log-likelihood factor for θt . Find (µt , σt )
for the quadratic approximation ℓ(θt ) ≈ −1/(2σ2t )(θt − µt )2 (see the previ-
ous problem).
c. Implement the adapted auxiliary particle filter from the previous problem.
As in Section 4.3, let Dt = (y1 , . . . , yt ) denote the data up to time t. Plot the
posterior means θt = E(θt | Dt ) against time, and add posterior medians
and 25% and 75% quantiles.
 5

Model Assessment

1
This chapter deals with approaches for model assessment aiming to imple-
ment a selection and comparison leading to the choice of the best model(s).
We start in Section 5.1 with a description of model diagnostics such as
Bayesian p-values associated with various discrepancy measures, resid-
uals, conditional predictive ordinates, and corresponding pseudo-Bayes
factors. In Section 5.2 we then consider several measures of predictive
performance, known by their acronyms as AIC, SIC (or BIC), DIC, and
WAIC, and ways of using Bayes factors for selection and comparison tasks.
Finally, in Section 5.3, using Monte Carlo samples to represent posterior
distributions in complex models, we review ways of estimating relevant
quantities for model assessment in this context.

5.1 Model Criticism and Adequacy

Verifying or critically examining a model is a stage of statistical analysis
that aims to evaluate the adequacy of model fitting to data and known fea-
tures of the problem at hand. In this exercise, we aim to quantify discrep-
ancies with the data, evaluate whether these are or are not due to chance,
and ascertain the degree of sensitivity of inference with respect to various
elements of the model, and propose ways to revise the model.

Bayesian P-values
Box (1980, 1983) proposed to base a critical analysis of a model on the
comparison of the data x and the marginal (or prior predictive) p(x) =
1
This material will be published by Cambridge University Press as Computational
Bayesian Statistics, by M.A. Amaral Turkman, C.D. Paulino, and P. Müller
(https://tinyurl.com/CompBayes). This pre-publication version is free to view
and download for personal use only. Not for re-distribution, re-sale, or use in derivative
works. c Maria Antónia Amaral Turkman, Carlos Daniel Paulino & Peter Müller, 2019.
 5.1 Model Criticism and Adequacy 73
   
Eh f (x | θ) by way of marginal p-values P p(X) ≥ p(x) . This approach
does not work with improper priors. In particular, it prevents the use of
hypothetical replicate data based on simulation from h(θ). Even when h(θ)
is proper, Monte Carlo methods tend to be in general unstable (Gelfand,
1996), and therefore not reliable, for the evaluation of p(x).
One of the most widely used strategies for a critical model examination
is based on the posterior predictive distribution under the model,
Z
p(y | x) = f (y | θ) h(θ | x) dθ. (5.1)
Θ

Here, y are hypothetical future data (see the following for details) and x
are the observed data. The data y should expectedly (or not) reflect the
observed data in the case of a good (bad) fit of the model. Any systematic
discrepancies between the two data sets, y and x, are evidence for some
flaw in the model.
The discrepancy between the observed data and data that are observable
under the model can be evaluated by summary variables V(x, θ), which
should be chosen to relate to the specific model aspects that the investigator
wishes to evaluate. Examples of such variables include standardized mean
squared deviations defined as
X [xi − E(Xi | θ)]2
V(x, θ) = (5.2)
i
Var(Xi | θ)
and the actual log-likelihood, ln f (x | θ), as well as the special cases of
statistics V(x) and functions g(θ) of the parameters only.
The posterior predictive distribution is typically determined by simula-
tion from the joint distribution of (y, θ), conditional on x. Consider, then,
{(yk , θk ), k = 1, . . . , m)}, a set of values generated from this joint distribu-
tion. The comparison of the real data x and predictive data via the sum-
mary variable V can be shown graphically by a scatter plot of the val-
ues {(V(yk , θk ), V(x, θk ), k = 1, . . . , m)}, or by a histogram of {V(yk , θk ) −
V(x, θk ), k = 1, . . . , m}. For a model that fits the data well, the points in the
scatter plot should be symmetric with respect to the 45-degree line and the
histogram should include 0.
One of the summary measures of discrepancy between V(x, θ) and the
distribution of V(Y, θ) is the posterior predictive p-value, sometimes re-
ferred to as the Bayesian p-value, defined as
PB = P [V(Y, θ) ≥ V(x, θ) | x]

= E(Y,θ) I{V(Y, θ) > V(x, θ)} | x . (5.3)
74 Model Assessment

Note that a very small or very large value of PB (for example, below 1% or
above 99%) implies that V(x, θ) falls into one or the other extreme tail of
the posterior distribution of V(Y, θ), given x (in both cases, being in general
implausible under the model). In this way, both cases are evidence for a
bad fit of the model to the data with respect to the characteristics that are
reflected in the chosen summary variable. For this reason, the Bayesian p-
value can, alternatively, be defined as PB = P [V(Y, θ) ≤ V(x, θ) | x], which
can be regarded as resulting from (5.3) by taking the negative of the chosen
variable. Typically, the Bayesian p-value, PB , is calculated as a proportion
of simulated values from the posterior predictive distribution of (Y, θ) given
x which satisfy V(yk , θk ) ≥ V(x, θk ). Such simulated values can be obtained
by first generating from h(θ | x) and then future data y from the sampling
model.
When the discrepancy measure is a statistic V(X), the posterior predic-
tive evaluation can alternatively be applied separately for each observation,
making use of the marginal predictive distributions p(yi | x). The corre-
sponding marginal p-values with respect to V(Xi ) are defined as PBi =
P [V(Yi ) ≥ V(xi ) | x], i = 1, . . . , n, which reduces to PBi = P(Yi ≥ xi | x) if
V is the identity. This is useful to detect outliers and to check whether there
is a global correspondence between the model and observed data. p-values
centered at extremes or too concentrated in the middle of the range of val-
ues are evidence for overdispersion and underdispersion, respectively, rel-
ative to the predictive data.
Extreme values of a measure of statistical significance, such as PB , need
not necessarily prompt one to completely abandon the model if the char-
acteristic that is expressed by the corresponding variable V is not of great
practical relevance. Even if it were, the features that are subject to criticism
resulting from this model inspection should suggest possible directions for
model revision to find models that are more adequate for the context of the
actual data.
Residuals and Other Measures of Model Adequacy and Diagnostic
Other measures can be constructed by comparing characteristics of the
predictive distribution under the model conditional on observed data with
other also observed data. This is possible in the context of cross-validation,
where one uses a training sample x = (x1 , . . . , xn ) to generate a posterior
distribution h(θ | x), and another sample y = (y1 , . . . , yℓ ), independent of
x, to evaluate the validity of the model by inspection of the corresponding
posterior predictive p(y | x) or some of its characteristics. For example,
the predictive mean and variance for each component Y j of the vector Y, of
 5.1 Model Criticism and Adequacy 75

which y is a realization, are useful to define normalized Bayesian predictive

residuals,
y j − E(Y j | x)
dj = p , j = 1, . . . , ℓ, (5.4)
Var(Y j | x)
which can be used, similar to what is done in classical inference, as instru-
ments for informal model validation.
This approach assumes the existence of independent samples, which
does, however, not usually happen in practice. Of course, if the original
sample is large, there is always the possibility to split it into two parts and
use one as a training sample to construct a posterior distribution and the
other as a test sample to obtain a predictive distribution conditional on the
earlier sample.
If it is not viable to split the entire sample x to implement such cross-
validation, then one can use a jackknife (leave-one-out) approach which
consists of repeating the cross-validation n times, always leaving one ob-
servation out of the training subsample. That observation plays then the
validation role of the test sample. This is known as leaving-one-out cross-
validation.
Denote with x(−i) = (x1 , . . . , xi−1 , xi+1 , . . . , xn ), the vector of all obser-
vations except xi . We can obtain the conditional predictive distributions
p(yi | x(−i) ),
Z
p(yi | x(−i) ) = f (yi | θ, x(−i) ) h(θ | x(−i) ) dθ, (5.5)

and the resulting normalized leaving-one-out Bayesian residuals,

xi − E(Yi | x(−i) )
di′ = p , i = 1, . . . , n, (5.6)
Var(Yi | x(−i) )
where the means and variances of the corresponding conditional predictive
distributions are calculated analytically or by simulation.
On the basis of these residuals, one can then again proceed with informal
validation. The value of p(yi | x(−i) ) that is calculated for xi is commonly
known as conditional predictive ordinate (CPO) and can be used in an in-
formal diagnostic. In fact, the values are evidence of the likelihood of each
observation given all other observations, and, therefore, low CPO values
correspond to badly fitted observations. In this sense, the higher the sum of
the logarithmic CPO values, also known as log pseudo-marginal likelihood
76 Model Assessment

(LPML)
X
n
LPML = ln CPOi = ln Πni=1 p(xi | x(−i) ), (5.7)
i=1

the more adequate is a model. For other diagnostic instruments see, in par-
ticular, Gelfand (1996) and Gelman and Meng (1996).

Example 5.1 Henderson and Velleman (1981) describe a study of the per-
formance of car models in terms of fuel consumption. We use a subset of
the data from this study. The data report efficiency, E f , measured as miles
per gallon, weight in pounds (X1 ), power in horse power (X4∗ ), and number
of gears in the transmission, at levels 3, 4, or 5, jointly represented by indi-
cator variables for 4 (X2 ) and 5 (X3 ), respectively. After some preliminary
analysis they consider normal linear regression models for the transformed
response variable Y = 100/E f , gallons per 100 miles (Y here is unrelated
to the earlier notation y for the predictive data).
One of the models involves the explanatory variables X1 , (X2 , X3 ), and
X4 = X4∗ /X1 (power per unit weight) in a multiple regression model:
X
4
M1 : µ ≡ E(Y) = β0 + β j X j + β5 X2 X4 + β6 X3 X4 ,
j=1

which corresponds to three different regression functions of X1 and X4 , one

for each number of gears, which differ by intercept and slope with respect
to X4 .
iid
The regression model Yi ∼ N(µ, σ2 ), i = 1, . . . , n = 29, is completed with
the usual non-informative prior distribution, β j ∼ N(0, 104 ) and 1/σ2 ∼
Ga(10−3 , 10−3 ). Bayesian analysis for this linear regression model under
the natural conjugate prior or the usual non-informative prior for (µ, σ2 )
can be implemented analytically (see, e.g., Paulino et al., 2018: sec. 4.3).
In the context of this Bayesian regression model we illustrate some of
the quantities that were described in this chapter and variations thereof,
with µ parametrized in terms of the regression coefficients, by using Monte
Carlo simulation. We define reduced models that compete with the de-
scribed model by simplifying µ by way of removing the interaction terms
(M2 ) and also by removing the main effects for the indicators for the num-
ber of gears (M3 ). Denote by θ the parameter vector, composed of the re-
gression coefficients and the residual variance, in each model.
Figure 5.1 shows scatter plots under models M1 and M2 of the measures
of discrepancy based on the standardized mean squared deviations V(·, θk )
 5.1 Model Criticism and Adequacy 77

from (5.2), calculated with the simulated values θk , generated from the pos-
terior distribution of θ (given the observed values for Y and {X j }) for the
actual data versus the predicted data.
M1 M2
150

150
100

100
V(x,θ)

V(x,θ)
50

50
0

0 20 40 60 80 0 20 40 60 80
V(x,θ) V(x,θ)

Figure 5.1 Scatterplots for models M1 and M2

The point cloud of the diagrams is in both cases more concentrated in

the region above the 45-degree line, indicating that the predictive residuals
under the two models tend to be larger than the corresponding residuals
under the observed data. The asymmetric nature of the scatter plot with
respect to the 45-degree line seems a little less pronounced for M2 than for
M1 . The Bayesian p-values associated with the standardized mean squared
deviations V(·, θk ) are evaluated with the observed data and the poste-
rior distribution of V(Y ∗ , θ) for the predicted data Y ∗ . The corresponding
Bayesian p-values are tabulated for the three models in Table 5.1, and
suggest that the reduced models perform better than the encompassing
model, in terms of fitting the data. The sum of the logarithmic CPOs sug-
gest the same, with a slight advantage for model M2 . Following a proposal
by Gelfand (1996), these quantities were estimated by the harmonic mean
of the assessed sampling density with respect to a simulated sample from
the posterior of θ (see Section 5.3.1). 
It is convenient to distinguish the diagnostic summaries that were dis-
cussed in this section from the information criteria in the following sec-
tion. While the use of these summaries is very similar, the information
criteria have a common underlying form and are only meaningful for the
comparison of models, whereas many diagnostic summaries in the earlier
discussion can also be meaningfully evaluated for individual observations.
78 Model Assessment

Table 5.1 Diagnostic measures for the competing models

P
Model PB ln CPO
M1 0.844 −27.577
M2 0.778 −23.665
M3 0.687 −23.817

5.2 Model Selection and Comparison

The comparison of statistical models can be implemented by several crite-
ria. The specific choice depends on the considerations that the investigator
has in mind with the analysis of the observed data, as was already implied
in the earlier discussion of model assessment. Undoubtedly one of the cen-
tral ideas in model evaluation is predictive precision under the model.
An ideal measure of model fit should reflect the predictive use in terms
of some external validation, that is, with new data that are generated from
a true, naturally unknown, model. As before, we denote such hypothetical
data y = {yi } and the already observed data x = {xi }. A measure of predic-
tive accuracy for each yi could be the logarithm of the posterior predictive
 
distribution ln p(yi | x) = ln Eθ|x f (yi | θ) .
Given that the data-generating model is unknown and given the fictitious
nature of {yi }, we can get an approximation of such a measure by using
instead the real data, leading to a posterior predictive intra-sample accuracy
measure:
X n
Ã(x) = ln p(xi | x) = ln Πni=1 p(xi | x)
i=1
 
X
n
 1 Xm

≃ ln   f (xi | θk ) , (5.8)
i=1
m k=1
where θk , k = 1, . . . , m are simulated from the posterior distribution h(θ |
x).
This type of within-sample approximation of an out-of-sample measure
of predictive accuracy is easy to obtain but tends to in general overestimate
the measure, because it involves the observed data twice. This issue sug-
gests the use of corrections to Ã(x), which combined with the application
of transformations give rise to distinct measures of predictive performance
that are traditionally named information criteria.
 5.2 Model Selection and Comparison 79

5.2.1 Measures of Predictive Performance

We now describe some of these measures that are most widely recom-
mended, practically and theoretically. By the specific form of how all the
information criteria are defined, especially by flipping the sign of the cor-
responding measure of accuracy, the lower the value of these criteria, the
better the model.

Akaike Information Criterion (AIC)

The measure of predictive accuracy that is used in the criterion due to
Akaike (1973) is based on the approximation of Ã(x) by the maximum
log-likelihood, ln f (x | θ̂), following a plug-in strategy, as is quite typical
in frequentist statistics, and penalizing by the number p of parameters in θ,
that is,
ÃAIC = ln f (x | θ̂) − p, (5.9)
where θ̂ is the maximum likelihood estimate of θ. The AIC measure of
information is obtained from ÃAIC by a linear transformation as in the like-
lihood ratio statistic of Wilks, that is
AIC = −2 ln f (x | θ̂) + 2p. (5.10)
In summary, the measure is composed of two components, one associated
with the goodness of fit of the model, and the other with the complexity of
the model, quantified as twice the dimension of the parameter vector. By
definition, the criterion does not depend on the sample size but its deriva-
tion assumes in fact a large sample context. On the other hand, any weak or
strong prior information about the parameters in the sampling model is sim-
ply ignored in the AIC, in contrast to Bayesian criteria that are described
in the following. For a more detailed discussion about this originally not
Bayesian criterion, and particularly about its variants, see Burnham and
Anderson (2002: ch. 2 and 6).

Bayesian Information Criterion (SIC/BIC)

Schwarz (1978) proposed a criterion (SIC) to select models by a large
sample approximation of the respective marginal distribution of the data
 
p(x) = Eh(θ) f (x | θ) . The criterion is more commonly known as BIC
(Bayes information criterion), possibly because it is based on a weight-
ing of the sampling distribution with the prior density, reflecting an origin
80 Model Assessment

that is quite distinct from that of measures of posterior predictive accuracy.

For large sample sizes n, ln p(x) ≃ ln f (x | θ̂) − (p/2) ln n, where θ̂ is the
maximum likelihood estimate of θ. The BIC criterion is defined as
BIC = −2 ln f (x | θ̂) + p ln n. (5.11)
The criterion considers models with large values of the mentioned approx-
imation of p(x) to be preferable, or equivalently, models with small values
of BIC. For moderate and large samples, the term in the BIC related to the
model dimension is larger than the corresponding term in the AIC, thus
heavily penalizing more complicated models.
To avoid simulation-based maximization, Carlin and Louis (2009) sug-
gested a modification of this criterion using the posterior mean of the log-
likelihood instead of the maximum. This modified version of the BIC is
thus defined as
 
BICCL = −2Eθ|x ln f (x | θ) + p ln n. (5.12)

Deviance Information Criterion (DIC)

Keeping in mind that the prior distribution and the type of model struc-
ture tend to affect the degree of over-fitting, the DIC criterion proposed by
Spiegelhalter et al (2002) modified the measure of predictive accuracy ÃAIC
in two aspects related to its two components. The maximum likelihood es-
timate θ̂ is substituted by a Bayesian estimate θ̄ (usually the posterior mean
of θ) and the number of parameters is replaced by an effective dimension
of the model, pD , defined by the following argument.
For a definition of model complexity, Spiegelhalter et al (2002) start with
a relative information measure for the sampling model, defined as
 
−2 ln f (x | θ)/g(x) , where g(x) denotes some function of only the data,
used for standardization. For example, g(x) = f (x | θ̃) where θ̃ is an es-
timate of θ in a saturated model, implying that the information measure
could be seen as a parametric function associated with the likelihood ra-
tio statistic. Seen as a function of θ given the data x, this measure is often
called Bayesian deviance and denoted D(θ).
Accordingly the value of this relative information measure for the same
distribution when θ is estimated by θ̄ is denoted D(θ̄), and thus the amount
of information about the sampling distribution that the estimation of θ by θ̄
does not account for is described by the difference:
f (x | θ)
D(θ) − D(θ̄) = −2 ln , (5.13)
f (x | θ̄)
 5.2 Model Selection and Comparison 81

which is now independent of the normalizing factor g(x). Since θ is un-

known and a random variable under a Bayesian perspective, Spiegelhalter
et al (2002) substitute D(θ) by its posterior expectation D(θ) = Eθ|x [D(θ)],
and define the effective number of parameters by
pD = D(θ) − D(θ̄). (5.14)
This definition is justified in the following. In words, pD is the difference
between posterior mean deviance and deviance under the posterior mean
when θ̄ = E(θ | x). This quantity naturally depends on the data, the param-
eter focus, and the prior information.
Alternatively other Bayesian estimates of θ and D(θ) could be used, with
naturally different results than under the posterior means. One advantage
of the earlier definition is easy computation. As long as the model includes
a closed-form likelihood function, one can always use the approximation
 
1X  1 X
m m

pD ≃ D(θk ) − D   θk  , (5.15)
m k=1 m k=1
where θk are a Monte Carlo sample from h(θ | x). Another advantage is the
fact that pD ≥ 0 for any model with log-concave likelihood (by Jensen’s
inequality).
In some standard models (without hierarchical structure) with a likeli-
hood dominating the prior, it can be shown that pD is approximately equal
to the actual number of parameters. The interpretation of pD as a measure
of dimensionality of the model attempts to extend the same notion to more
complex models for which it is not easy to determine the number of pa-
rameters, such as models that include latent variables. However, in some
models such as finite mixture models or some hierarchical models the im-
plied pD can be negative. See, for example, Celeux et al (2006).
Using pD as a measure of model complexity (and for the moment ignor-
ing the standardization function g), the corresponding measure of predic-
tive accuracy becomes
ÃDIC = ln f (x | θ̄) − pD . (5.16)
This leads to the DIC criterion (deviance information criterion) proposed
by Spiegelhalter et al (2002) as
DIC = D(θ̄) + 2pD = D(θ) + pD = 2D(θ) − D(θ̄). (5.17)
In practice, the normalizing factor in the deviance is usually omitted
(that is, g(x) = 1) when the Bayesian models under consideration are all
82 Model Assessment

based on the same sampling model and differ only in the parametric struc-
ture. Otherwise, one must be careful because the DIC measure depends on
the chosen function g(·). Regarding this, it is by no means straightforward
to determine the maximum likelihood for the saturated structure in most
multiparameter models.

Widely Applicable Information Criterion (WAIC)

The measure of intra-sample predictive accuracy associated with the WAIC
(widely applicable information criterion) criterion, due to Watanabe (2010),
does not involve plug-in approximations as the previously discussed crite-
ria, and as such is more Bayesian than that used by the DIC. It is evaluated
with an overfitting correction as
X
n
 
ÃWAIC = Ã(x) − pW = ln Eθ|x f (xi | θ) − pW . (5.18)
i=1

Just like Ã(x) previously, also the complexity penalty term involves an ex-
pression in terms of the individual data. One of the proposals for pW is
similar to the one used in the DIC, expressed as
X
n
   
pW1 = −2 {Eθ|x ln f (xi | θ) − ln Eθ|x f (xi | θ) }
i=1
n 
 
X 1X
m
 1 Xm

≃ −2 ln f (xi | θk ) − ln  f (xi | θk ) , (5.19)
i=1
m k=1 m k=1
with pW1 ≥ 0 by construction.
Another proposal for pW is based on the posterior variance of ln f (xi | θ)
for all data points, and defined as
X
n
 
pW2 = Varθ|x ln f (xi | θ)
i=1
n 
X i2 
1 Xh
m
≃ ¯ i) ,
lk (xi ) − l(x (5.20)
i=1
m − 1 k=1
¯ i ) = 1 Pmk=1 lk (xi ).
where lk (xi ) = ln f (xi | θk ) and l(x m
Using any of these proposals for the complexity penalty term, inter-
preted as “effective model dimensionality,” depending on either data or
prior information, Gelman et al. (2014a, 2014b) propose a transformation
of Watanabe’s information criterion to the same scale as other proposals,
 5.2 Model Selection and Comparison 83

being expressed by
X
n
 
WAIC = −2 ln Eθ|x f (xi | θ) + 2pW . (5.21)
i=1

5.2.2 Selection by Posterior Predictive Performance

Bayesian analysis is usually based on a given model and, consequently,
related inference is of a conditional nature. However, unavoidable uncer-
tainty about any of the components of a Bayesian model are a sufficiently
strong reason to consider a range of possible joint models for data and pa-
rameters, which is then subject to some preliminary screening.
After the inspection and assessment of the models, one should then pro-
ceed with a comparison aiming to select the best models with respect to
the adopted criteria. This stage of statistical analysis need not necessarily
culminate in the selection of a single model as this might lead to mislead-
ing conclusions. To the contrary, one should certainly discard some models
with unacceptable performance with respect to the chosen criteria, but keep
the remaining ones for further consideration.2
The traditional practice of carrying out model selection by way of hy-
potheses tests has slowly been changing over the last 40 years, perhaps
due to an increasing understanding of the limitations of such a process as
compared to the optimization of other measures of model performance.
Application of the earlier described information criteria respects the par-
simony paradigm that requires that the selected models should combine an
adequate representation of the data (with a good fit) to as simple a struc-
ture as possible (low dimensionality). The paradigm was expressed way
back (in the fourteenth century) as Occam’s razor: “shave away all that is
unnecessary.”
Keeping in mind the need to control the number of models under consid-
eration, a strategy in the spirit of this principle should be guided by the fol-
lowing desiderata: Drop any model that predicts the data worse than nested
simpler models or much worse than the model that produces the currently
best prediction. The strategy is compatible with several ways of evaluating
the quality of predictions, be it by diagnostic summaries, by information
criteria, or others.
2
This could even include the implementation of the desired inference by appropriate
combination of the selected models, known as Bayesian model averaging.
84 Model Assessment

Use of Diagnostic Summaries

In the face of several competing models, diagnostic tools or measures of
model adequacy such as the Bayesian p-value or standardized Bayesian
residuals (obtained by cross-validation or by jackknife) are useful to com-
paratively evaluate competing models in terms of their performance. For
example, using the sum of squares (or of absolute values) of these residu-
als, one should favor models that report smaller values.
Another means of evaluating comparative adequacy among several mod-
els is the sum of the log conditional predictive ordinates, with the idea of
selecting the models with the highest values. When comparing two models,
H1 versus H2 , by means of CPO one can use
p(xi | x(−i) , H1 )
PBF = Πni=1 ≡ Πni=1 Qi . (5.22)
p(xi | x(−i) , H2 )
This is known as the pseudo-Bayes factor (PBF). Keeping in mind that the
product of CPOs for each model is used as a substitute for the respective
marginal likelihood (recall the definition of the Bayes factor), then values
of the pseudo-Bayes factor greater (less) than 1 are evidence for model
H1 (H2 ). Besides this, since observations with Qi greater (less) than 1 are
evidence for H1 (H2 ), a plot of ln(Qi ) versus i is useful to visualize which
observations are better fit by which of the two competing models.

Use of Information Criteria

In the context of model comparison the relevant quantities are not the ac-
tual values of the predictive performance measures in absolute terms, but
the relative values within the set of the, say, J models under consideration.
Thus for each of these measures the differences across pairs of models are
most useful. Let IC generically denote one of the measures that were in-
troduced in the previous subsection. Since all measures were defined such
that smaller values correspond to better models, it is useful to determine
for each model the differences r j (IC) = IC j − ICmin , j ∈ J.
The evaluation of these differences allows an easier comparison and
ranking of the models under consideration. The smaller r j the higher the
degree of empirical evidence for model j, with the best model correspond-
ing to r j = 0. We will use j = o (as in “optimal”) to denote the index of the
best model. For example, the differences r j for DIC are:
r j (DIC) = DIC j − DICo ,
DIC j = Eθ j |x [D j (θ j )] + pD j = 2Eθ j |x [D j (θ j )] − D j (Eθ j |x [θ j ]).
 5.2 Model Selection and Comparison 85

Considering the respective differences under the BIC criterion, using

{p∗j } for the dimension of the parameter vectors under each model, we get:
∗
f j (x | θˆj )n−p j /2
r j (BIC) = BIC j − BICo = −2 ln
fo (x | θˆ0 )n−po /2
∗

p j (x)
≃ −2 ln = −2 ln B jo (x). (5.23)
po (x)
That is, the differences r j (BIC) are related with the Bayes factor between
the two models under comparison, say H j and Ho , in the sense that for large
sample sizes they can be approximated by −2 ln B jo (x), using an argument
of Schwarz (1978).

Example 5.2 Continuing with the earlier example, consider now compar-
ative evaluation of the three multiple regression models with respect to
their predictive performance. The PBF for the pairwise comparisons are
PBF(M1 /M2 ) = 0.809; PBF(M1 /M3 ) = 0.941, and PBF(M2 /M3 ) = 1.164.
These values support the earlier result that M2 was the best and M1 the
worst of the three models, in terms of this criterion based on conditional
predictive ordinates.
A comparison with respect to the information criteria shown in Table
5.2, reports model M2 as the best in terms of the Bayesian summaries DIC
and WAIC, which, however, is dominated by M3 under the BIC. The latter
is no surprise, knowing that BIC favors the more parsimonious models.

Table 5.2 DIC, BIC and WAIC criteria for models M1 , M2 and M3
Model DIC (pD ) BIC (p) WAIC (pW2 )
M1 48.69 (8.27) 67.36 (8) 46.77 (5.38)
M2 47.32 (6.19) 61.33 (6) 46.70 (4.78)
M3 47.58 (4.12) 56.93 (4) 47.40 (3.48)

Pooling the results here and in the previous example, the applied criteria
show that the overall best of the three models is the one with intermediate
complexity as measured by the number of parameters. 

5.2.3 Model Selection Using Bayes Factors

The methods for model selection that were discussed in the previous sub-
section are useful for screening, with the aim of retaining for further con-
sideration those models that are found to perform well with respect to the
86 Model Assessment

available information. But there is no concern that any model stands for the
reality under study.
In contrast, some methods correspond to a perspective (possibly con-
troversial) that the set of considered models must include a so-called true
model, in the sense of being the one that generated the observed data (or,
at least, which constitutes a sufficiently good approximation of the true
model). The method of using Bayes factors to select one of a finite number
of models fits into this perspective. This and other aspects that could be
criticized should not prevent one from using this method in some contexts,
with due caution.
Any set of Bayesian models M = {M j , j ∈ J} (with different sampling
models and prior distributions) implies a set of corresponding prior predic-
tive models
Z
p(x | M j ) ≡ p j (x) = f j (x | θ j )h j (θ j ) dθ j , j ∈ J. (5.24)

If J is a discrete set of indicators for the members of a set of models that

is assumed to contain the unknown true model, the global predictive distri-
bution is
X
p(x) = P(M j )p(x | M j ) , (5.25)
j∈J

where P(M j ) is the prior probability of M j being the true model and is
updated in the face of data x to
P(M j | x) = P(M j )p(x | M j )/p(x), j∈J. (5.26)
The Bayes factor in favor of Mk versus M j is thus the ratio
P(Mk | x)/P(M j | x) p(x | Mk )
Bk j (x) = = , (5.27)
P(Mk )/P(M j ) p(x | M j )
and the posterior odds for Mk are
P(Mk | x) P(Mk )
=P , (5.28)
1 − P(Mk | x) j,k P(M j )B jk (x)

where B jk (x) = 1/Bk j (x) is the Bayes factor in favour of M j versus Mk . The
P
formula for these odds becomes ( j,k B jk (x))−1 when one uses a uniform
prior over the space of models. Simple examples of the application of this
method can be found in Paulino et al (2018).
In general, the evaluation of prior predictive distributions, and thus of
Bayes factors, requires simulation – see the preceding chapter and the next
 5.3 Further Notes on Simulation in Model Assessment 87

section. Model comparison based on Bayes factors requires the use of prac-
tical rules for the choice of thresholds on the strength of evidence in favor
of a model. One such rule is proposed by Kass and Raftery (1995).

5.3 Further Notes on Simulation in Model Assessment

Following Chapter 4, Sections 5.1 and 5.2 already included some recom-
mendations on the use of stochastic simulation to evaluate diagnostic sum-
maries and measures of fit and predictive performance by means of Monte
Carlo methods. In this section we complete these recommendations with
the discussion of some additional questions related to simulation-based im-
plementations of such Bayesian inference.
To define the context of the discussion, assume that one has a Monte
Carlo sample {θ(⋆j) ; j = 1, . . . , m} from some posterior distribution h(θ | x),
which was previously generated by some simulation method. The follow-
ing subsections describe some aspects of simulation specifically related to
model assessment.

5.3.1 Evaluating Posterior Predictive Distributions

Let y denote a new data set, independent of the already observed data. The
predictive density p(y | x) can be estimated, as suggested in Section 4.1.4,
by
1X
m
p̂(y | x) = f (y | θ(⋆j) ).
m j=1

Using a similar argument, for the estimation of a conditional predictive

density
Z
p(xi | x(−i) ) = f (xi | θ, x(−i) ) h(θ | x(−i) ) dθ,

one can, in principle, simulate {θ(⋆j) ; j = 1, . . . , m} for each posterior distri-

bution h(θ | x(−i) ). However, this is impractical for high-dimensional x. An
ideal, more practicable method should obtain the desired density estimate
using only a single sample {θ(⋆j) ; j = 1, . . . , m} from h(θ | x).
Gelfand (1996) suggests to use the harmonic mean of the set { f (xi |
x(−i) , θ(⋆j) ), j = 1, . . . , m} to estimate p(xi | x(−i) ). Indeed, noting
p(x)h(θ | x) = h(θ) f (x | θ) = h(θ) f (xi | x(−i) , θ) f (x(−i) | θ),
88 Model Assessment

we get
"Z #−1
p(x) 1
p(xi | x(−i) ) = = h(θ | x) dθ ,
p(x(−i) ) f (xi | x(−i) , θ)
and thus, if {θ(⋆j) ; j = 1, . . . , m} is a sample from h(θ | x), we have
 −1
 1 Xm
1 
p̂(xi | x(−i) ) =  ⋆ 
 . (5.29)
m j=1 f (xi | x(−i) , θ( j) ) 

The expression simplifies in the case of (X1 , . . . , Xn ) being conditionally

independent, in which case f (xi | x(−i) , θ) = f (xi | θ).
Note that a similar argument allows us to estimate moments of the poste-
rior predictive distribution (and, in particular, standardized Bayesian resid-
uals), based on sample moments for each simulated value from h(θ | x), as
well as other diagnostic summaries.

5.3.2 Prior Predictive Density Estimation

The evaluation of Bayes factors requires the prior predictive (or marginal)
distribution p(x). We review some approaches that have been proposed in
the literature to obtain it, assuming it exists.
If {θ(⋆j) ; j = 1, . . . , m} is a prior sample from h(θ), then by the definition
of p(x) it follows that the same can be approximated as
1X
m
p̂(x) = f (x | θ(⋆j) ).
m j=1

This estimate of p(x) has been shown, however, to be in general very inef-
ficient, as we already mentioned in Section 4.2.2.
Newton and Raftery (1994) suggested the harmonic mean of { f (x | θ(⋆j) )},
that is
h1 Xm
1 i−1
p̂(x) = ⋆ , (5.30)
m j=1 f (x | θ( j) )

with {θ(⋆j) ; j = 1, . . . , m} being a posterior sample from h(θ | x). Assuming

that h(θ) is a proper distribution, the result is easily shown by using Bayes’
theorem to write
Z Z
1 1 1
= h(θ) dθ = h(θ | x) dθ,
p(x) p(x) f (x | θ)
which motivates the estimation of p(x) by the harmonic mean. However,
 5.3 Further Notes on Simulation in Model Assessment 89

also this estimator is numerically unstable due to possibly small values of

f (x | θ).
Gelfand and Dey (1994) suggested a generalization of (5.30) which
gives rise to a more stable estimator. The generalization uses a (proper)
distribution g(θ), which should be a good approximation of the posterior
h(θ | x) and should be easy to generate from (for example, a multivariate
normal posterior approximation with mean and covariance matrix based on
a posterior Monte Carlo sample). Then use
Z Z
1 1 g(θ)
= g(θ) dθ = h(θ | x) dθ.
p(x) p(x) f (x | θ) h(θ)
Using a posterior Monte Carlo sample, {θ(⋆j) , j = 1, . . . , m}, a corresponding
estimator for p(x) is
 −1
 1 Xm g(θ(⋆j) ) 
p̂(x) =    . (5.31)
⋆ ⋆ 
m j=1 f (x | θ( j) ) h(θ( j) )

More discussion of this and other prior predictive density estimates can
be found in Kass and Raftery (1995).

5.3.3 Sampling from Predictive Distributions

Suppose we want to sample from the posterior predictive distribution p(y |
x), for y = (y1 , . . . , yn∗ ). If for each element θ(⋆j) of a posterior Monte Carlo
sample {θ(⋆j) }mj=1 from h(θ | x) we simulate a data set y⋆( j) from f (y | θ(⋆j) ),
then marginally, y⋆( j) is a sample from p(y | x). In particular, y⋆r,( j) , the r-th
element of the sample y⋆( j) , is an observation for p(yr | x). This posterior
predictive sampling scheme is useful, in general, for the investigation of
model adequacy, including in particular the use of standardized Bayesian
residuals dr .
But how can one sample from the conditional posterior predictive distri-
bution p(yi | x(−i) )? By a similar argument, one could use a sample {θ(⋆⋆ m
j) } j=1
⋆⋆ ⋆⋆
from h(θ | x(−i) ) and then, for each θ( j) , obtain a sample from f (yi | θ( j) ).
However, clearly this would be computation-intensive and inefficient for
large sample sizes.
The question arises, then, how to sample from h(θ | x(−i) ) for all i, with-
out having to repeat the entire scheme for each observation. Note that for
each θ(⋆j) , we have for x = (xi , x(−i) ),
p(xi | x(−i) ) 1
h(θ(⋆j) | x(−i) ) = ⋆ h(θ(⋆j) | x) ∝ h(θ(⋆j) | x).
f (xi | x(−i) , θ( j) ) f (xi | x(−i) , θ(⋆j) )
90 Model Assessment

Thus if one resamples {θ(⋆j) }mj=1 , with probabilities proportional to ω j =

{ f (xi | x(−i) , θ(⋆j) )}−1 and with replacement, the resulting sample is approxi-
mately a sample from h(θ | x(−i) ). Often h(θ | x(−i) ) ≈ h(θ | x) and thus the
resampling might become unnecessary.
Finally, to sample from the marginal predictive p(x), assuming it is
proper, one can generate θ̃ j from h(θ) and then x̃ j from f (x | θ̃ j ).

Problems
5.1 Replicate Example 5.1. The data are available in R as the data frame mtcars.
Evaluate PB and the log pseudo-Bayes factor (5.22).
5.2 Let yi be the number of misprints on page i of the book Bayesian Theory by
Bernardo and Smith (2000). Consider two alternative models:
Model H1 : yi | λ ∼ Poi(λ · Ni ), i = 1, . . . , n
λ ∼ Ga(1, β)

Model H2 : yi | θ ∼ Bin(Ni , θ)
θ ∼ Be(1, β − 1),
where Ni is the number of characters per page, β > 1 is a fixed hyperparam-
eter, and the gamma distribution is parametrized such that E(λ) = 1/β. That
is, under both models 1/β is interpreted as the expected number of typos per
word.
a. Find the Bayes factor B = p(y | H2 )/p(y | H1 ) for choosing between
competing models H1 and H2 .

b. Consider the following data

Ni 5607 5878 6200 5460 5576
yi 0 1 0 0 1
(ctd.) 5772 5770 6009 6027 5793
0 0 1 0 0
Fix 1/β = 0.0001 and evaluate the Bayes factor.
P
c. Let V(Y, θ) = i I(Yi = 0) denote the number of typo-free pages. Using V
evaluate a Bayesian p-value for models H1 and H2 .
5.3 For the following data we will consider two competing models: H1 : a linear
regression, versus H2 : a quadratic regression.
xi −1.9 −0.39 0.79 −0.20 0.42 −0.35
yi −1.7 −0.23 0.50 −0.66 1.97 0.10
(ctd.) 0.67 0.63 −0.024 1.2
0.60 1.13 0.943 2.6
 Problems 91

Model H1 :
yi = β1 + β2 xi + ǫi , i = 1, . . . , n; ǫi ∼ N(0, 1)
β1 ∼ N(0, 1), β2 ∼ N(1, 1),
with β1 and β2 a priori independent.
Model H2 :
yi = γ1 + γ2 xi + γ3 xi2 + ǫi , i = 1, . . . , n; ǫi ∼ N(0, 1)
γ1 = N(0, 1), γ2 ∼ N(1, 1), γ3 ∼ N(0, 1),
with γ1 , γ2 , γ3 a priori independent.
R
a. Find the marginal
R distributions p(y | H1 ) = f (y | β, H1 ) h(β) dβ. and
p(y | H2 ) = f (y | γ, H2 ) h(γ) dγ.
b. Write down the Bayes factor B = p(y | H2 )/p(y | H1 ) for comparing
model H1 versus model H2 and evaluate it for the given data set.
c. We now replace the prior distributions by improper constant priors: h(β) =
c1 in modelR H1 ; and h(γ) = c2 in model R H2 . We can still formally evaluate
integrals3 f (y | β, H1 ) h(β) dβ and f (y | γ, H2 ) h(γ) dγ and define a
Bayes factor,
R
f (y | γ)h(γ) dγ
B̃ = R .
f (y | β)h(β) dβ
Show that the value of the Bayes factor B̃ depends on the – arbitrarily
chosen – constants c1 and c2 .
d. Evaluate the Bayes factor using the harmonic mean estimator (5.31) and
compare with the exact evaluation from b.
5.4 Refer to Problem 5.3. Evaluate the log pseudo-Bayes factor (5.22), BIC,
AIC, WAIC, and DIC to compare the two models.

3
Although the marginal distributions might be improper, i.e., meaningless.
 6

Markov Chain Monte Carlo Methods

1
As discussed in Chapter 4, the implementation of Bayesian inference of-
ten involves the use of simulation-based methods, based on a Monte Carlo
sample of values that are generated from a typically multivariate posterior
distribution, h(θ | x), θ ∈ Θ. The use of simulation-based methods is one
way of dealing with the analytically often intractable form of Bayesian
inference summaries. Depending on the complexity of the posterior distri-
 
bution h(·), the evaluation of posterior summaries like E g(θ) | x can be
carried out by classical Monte Carlo (MC) methods, by generating i.i.d.
samples from the target distribution itself, or from some appropriate im-
portance sampling distribution, whose construction involves the target dis-
tribution.
For more complex problems it has become more common, especially
since the 1990s, to use more general MC methods based on the simulation
of a (homogeneous) Markov chain that is constructed to have an ergodic
distribution π(θ) equal to the target distribution, π(θ) ≡ h(θ | x). Such meth-
ods, known as Markov chain Monte Carlo (MCMC), thus use dependent
samples of θ, implying also more complex asymptotics and the need for
larger simulation sample sizes compared to classical MC methods.
The rediscovery of MCMC methods2 by statisticians in the 1990’s (in
particular, Gelfand and Smith, 1990) led to considerable progress in simulatio
based inference methods and, in particular, Bayesian analysis for models
that were too complex for earlier methods.
Given the nature of MCMC methods, it is not possible to fully under-
1
This material will be published by Cambridge University Press as Computational
Bayesian Statistics, by M.A. Amaral Turkman, C.D. Paulino, and P. Müller
(https://tinyurl.com/CompBayes). This pre-publication version is free to view
and download for personal use only. Not for re-distribution, re-sale, or use in derivative
works. c Maria Antónia Amaral Turkman, Carlos Daniel Paulino & Peter Müller, 2019.
2
The earlier literature includes Metropolis et al (1953), Hastings (1970) and Geman and
Geman (1984).
 6.1 Definitions and Basic Results for Markov Chains 93

stand them and the details of their application in Bayesian statistics with-
out a knowledge of basic results for Markov chains which are therefore
summarized in the following section, in a very brief form given the nature
of this text.3 For simplicity, we use in this section (and beyond) a generic
notation for the states of a chain. In the following sections we describe
the most commonly used methods, including Metropolis–Hastings chains,
Gibbs samplers, slice sampling, and Hamiltonian Monte Carlo. The last
section is dedicated to implementation questions related to MCMC meth-
ods, including the question of one versus multiple parallel chains and con-
vergence diagnostics.

6.1 Definitions and Basic Results for Markov Chains

A stochastic process is a set of random variables that are defined over the
same probability space {U(t), t ∈ T }, where T is some subset of R which,
without loss of generality, may be considered as a set of temporal indexes.
When this set T is T = {0, 1, 2, . . .}, then the stochastic process is usually
written as {Un , n ≥ 0}. This is the typical setup in stochastic simulation.
The set U of values of the variables Un is known as state space.
Knowing the past and present states of a process generally informs about
the plausibility of future states. When conditional on a given present state,
the plausibility of future states does not depend on the past, and we say
that the process has Markov dependence. A process {Un , n ≥ 0} with this
conditional independence property is known as a Markov chain, and can
be defined by

Un+1 ⊥ (U0 , . . . , Un−1 ) | Un ⇔

P(Un+1 ∈ A | U0 = u0 , . . . , Un = u) = P(Un+1 ∈ A | Un = u) ≡ Pn (u, A),
for all events A and n ≥ 0. The probability Pn (u, A) is known as (one-step)
transition function at time n. Equivalently, considering A = (−∞, v], a
Markov chain can be defined by the conditional distribution functions
FUn+1 (v | U0 = u0 , . . . , Un = u) = FUn+1 (v | Un = u) ≡ Fn (u, v),
for all v, u ∈ U. When the transition function is invariant with respect to n,
we write P(u, A) (or F(u, v)) and the Markov chain is called homogeneous.
In the upcoming discussion we are only interested in homogeneous Markov
3
For more discussion of this issue, see Ross (2014) and Tierney (1996) and references
therein.
94 Markov Chain Monte Carlo Methods

chains, and will therefore henceforth not explicitly indicate the qualifier
homogeneous.
For a discrete state space, the Markov chain is entirely defined by the
conditional probabilities P(u, {v}), i.e.

P(Un+1 = v | U0 = u0 , . . . , Un = u) =
P(Un+1 = v | Un = u) ≡ p(u, v), ∀n ≥ 0, u, v ∈ U.
In the case of a finite state space the transition probabilities p(·, ·) can be
recorded as a matrix P of probabilities for one step. When U is uncount-
ably infinite and F(u, v) is absolutely continuous, then the transition func-
tion can be defined by a density p(u, v) = ∂F(u,v)
∂v
.
For the moment we assume a discrete Markov chain. We have
X X
P(Un+1 = v) = P(Un = u)p(u, v) = P(U0 = u)pn (u, v),
u u
P
where pn (u, v) = P(Un = v | U0 = u) = u pn−1 (u, z)p(z, v), n ≥ 1
defines a transition function for n steps (in matrix form the product Pn ).
The construction of a Markov chain is therefore completely determined by
the transition function, as long as an initial distribution is given.
We say that a probability distribution π(u), u ∈ U is a stationary distri-
bution if
X
π(v) = π(u)p(u, v).
u

In particular, an initial distribution P(U0 = u) = π(u) is stationary if and

only if the marginal distribution of Un is invariant across n, i.e. P(Un =
u) = π(u), ∀n ≥ 0.
The existence and uniqueness of stationary distributions depends upon
whether the chain has certain characteristics known as irreducibility and
recurrence. A chain is irreducible if the chain can within a finite number
of transitions reach any state, starting from any initial state. A chain is said
to be recurrent if it returns infinitely many times to any starting state. It is
said to be positive recurrent if the expected time of the first return to any
state u is finite for all states u. Irreducibility implies positive recurrence if
U is finite.
An irreducible and recurrent Markov chain (with discrete U) has a unique
stationary distribution. On the other hand, if there is a stationary distribu-
tion π(v) such that limn→∞ pn (u, v) = π(v), then the stationary distribution
is unique and limn→∞ P(Un = v) = π(v). In that case, independently of the
 6.1 Definitions and Basic Results for Markov Chains 95

initial distribution, for large enough n the marginal distribution of Un is

approximately π.
Convergence to the stationary distribution π is not guaranteed for an
irreducible and positive recurrent chain. However, with the additional con-
dition of aperiodicity, defined as min{n ≥ 1 : pn (u, u) > 0} = 1 (it suffices
if there exists u such that p(u, u) > 0), such a chain is then called ergodic
and has a limiting behavior pn (u, v) −→ π(v) for all u, v ∈ U, thus assuring
n→∞
convergence of P(Un = u) to π(u) for all u.
In addition, if g(U) is a function defined on the state space of an ergodic
Markov chain with finite expectation under π, then we have
1X
n
 
g(Ut )−→ Eπ g(U) , a.s.
n t=1 n→∞

This result, commonly known as ergodic theorem, generalizes the strong

law of large numbers to Markov chains with the stated characteristics. Un-
der additional conditions also an extension of the central limit theorem
holds, that is, hconvergence (in distribution) to a normal distribution for the
√ P  i
sequence n n1 nt=1 g(Ut ) − Eπ g(U) .
When the states of a chain are absolutely continuous random variables,
the definition of the described properties needs to be modified to refer to
events A ⊆ U instead of individual states, similar to the definition of a
transition function, and are subject to some measure theoretic technical
details. For example, the description of the dynamics of a chain involves
the condition of visits to an event A with positive probability. A probability
measure Π is said to be stationary if for any event A,
Z
Π(A) = P(u, A)Π(du),
U

which in terms of densities corresponds to

Z
π(v) = p(u, v)π(u)du.
U

Convergence results for chains with uncountably infinite state spaces

are analogous to the earlier statements, with the difference that the results
require stronger conditions.4
Another property of Markov chains that is of practical importance in the
analysis of the limiting behaviour concerns the reversibility of the proba-
bilistic dynamics. Specifically, a chain is said to be reversible if for any
4
For example, positive Harris recurrence is required for ergocidity – see e.g. the book
Paulino et al (2018) and references therein.
96 Markov Chain Monte Carlo Methods

event A and state u in the state space U (discrete or not),

P(Un+1 ∈ A | Un = u) = P(Un+1 ∈ A | Un+2 = u).
Reversibility of a chain with transition function p(·, ·) and stationary distri-
bution π(·) is equivalent to
π(u)p(u, v) = π(v)p(v, u), ∀u, v ∈ U. (6.1)
This condition is known as detailed balance condition. It can be inter-
preted as a balance implied by the chain in the sense that being in u and
passing to v and being in v and passing to u are equally plausible, for ev-
ery pair (u, v) of states. In particular, a chain that satisfies this condition
for a probability density π is not only reversible but also has the same π as
stationary distribution.

6.2 Metropolis–Hastings Algorithm

In this and the following sections we continue to use the notation that was
introduced in Section 6.1. Considering the usually multivariate nature of
the states of the discussed chains, we change the “time” index to a su-
perindex for the random vectors, such as U (t) , reserving the subindex for
the elements of the vector (generally scalars), U (t)
j when needed. Note that
this deviates from the notation in Chapters 4 and 5. In fact, letting U repre-
sent the k-dimensional parameter θ (k ≥ 2), then U (t) and U (t) j denote what
5
was then denoted by θ(t) and θ(t) j . We continue to denote the stationary
distribution by π(u), u ∈ U.6
The fundamental element of the Metropolis–Hastings algorithm is a con-
ditional distribution q(ũ | u) ≡ q(u, ũ) which plays the role of generating
the simulated values. The basic requirement for q(· | ·), sometimes referred
to as proposal or instrumental distribution, is easy random variate genera-
tion. The values ũ that are generated from this distribution are subject to
stochastic inspection, based on q(· | ·) and π(·), which determines whether ũ
is accepted or rejected, in the sense that a rejected ũ is replaced by the most
recent accepted value. The process is described in the following algorithm.

5
Note that for k = 1, U (t) corresponds to θt in these earlier chapters, reserving θ(t) for the
t-th smallest value of a sample (θ1 , . . . , θn ).
6
For convenience, we will use the term “density function” to refer to a distribution
independently of the nature of its support. That is, in the case of discrete random
variables it represents the probability mass function.
 6.2 Metropolis–Hastings Algorithm 97

Algorithm 1 Metropolis–Hastings (M-H) Algorithm

1. Given u(t) , t = 0, 1, 2, . . . generate a value Ũ ∼ q(ũ | u(t) ).

π(ũ)q(u|ũ)
2. Evaluate the M-H ratio R(u(t) , Ũ), with R(u, ũ) = π(u)q(ũ|u) ,
and record the probability α(u, ũ) = min{R(u, ũ), 1}.
3. Accept as the next state of the chain



Ũ, with probability α(u(t) , Ũ)
(t+1)
U = (6.2)
u(t) , with probability 1 − α(u(t) , Ũ).

Several notes about the algorithm are stated in the following.

Note 1: support of π. The acceptance probability for ũ in iteration t + 1
requires π(u(t) ) > 0. This is guaranteed ∀t ∈ N if the initial value u(0) of the
chain satisfies it, as all simulated values with π(ũ) = 0 are rejected due to
α(u(t) , ũ) = 0. We set R(u, ũ) = 0 when π(ũ) = 0 = π(u). Thus, once within
the support of π, the chain does not leave from it almost surely.
Note 2: normalizing constants. The nature of the M-H ratio shows that
the algorithm can be implemented when π(·) and q(· | u) are known up to
normalizing constants, that is, factors that do not involve u in the case of
q(· | u). On the other hand, values of ũ with π(ũ)/q(ũ | u(t) ) greater than the
same ratio for the previous value, π(u(t) )/q(u(t) | ũ), are always accepted, as
α(u(t) , ũ) = 1.
Note 3: repetitions. A chain {u(t) } generated by this algorithm can include
repetitions, and it is a special case of a Markov chain, as the distribution of
U (t+1) conditional on all previous values depends only on U (t) . Convergence
of this chain to a target distribution π(u) depends, as can be expected, on
the proposal distribution.
Note 4: transition function. As this is the most common case in applica-
tions, we consider here only the absolutely continuous case (with respect to
Lebesgue measure) with uncountably infinitely many states, in which case
π(u) is a density of the stationary distribution.
Let Q(·, ·) denote the transition function of a Markov chain, with density
q(· | ·), i.e., Q(u, dũ) = q(ũ | u)dũ. In that case, step 3 of the M-H algorithm
defines a transition function,
h i
P(u, dũ) ≡ P U (t+1) ∈ dũ | U (t) = u = α(u, ũ)q(ũ | u)dũ + r(u)δu (dũ),
R (6.3)
where δu (dũ) denotes a Dirac measure in dũ and r(u) = 1 − α(u, ũ)q(ũ |
u) dũ is the probability that the chain remains in u. The transition function
98 Markov Chain Monte Carlo Methods

(6.3) is characterized by a transition density

p(u, ũ) = α(u, ũ)q(ũ | u) + r(u)δu (ũ), (6.4)
which, by definition of α and δu , satisfies the detailed balance condition
with π, π(u)p(u, ũ) = π(ũ)p(ũ, u). Thus, an M-H chain is reversible with a
stationary distribution precisely equal to the desired target distribution π.
Note 5: convergence. Convergence of an M-H Markov chain to the sta-
tionary distribution π depends on the regularity conditions that were dis-
cussed in the previous section.
Let S = {u : π(u) > 0} denote the support of π. The use of a proposal
distribution q(· | ·) with q(ũ | u) > 0, ∀(u, ũ) ∈ S × S guarantees that the
chain {U (t) } is irreducible with respect to π. Since π is a stationary distri-
bution of the M-H chain, it is therefore positive recurrent (and also Harris
recurrent), and the ergodic theorem of Section 6.1 applies. 7
In summary, for an M-H chain that converges to a target distribution π,
the states of the chain beyond a certain time can be considered as approxi-
mate simulations from π, even if in the implementation they were generated
from the proposal distribution. This implies that summaries of π can be de-
termined empirically from a (computer-generated) sample from the chain.

The perhaps most attractive feature of the M-H algorithm is its versa-
tility, considering the few and weak requirements for π and q to guarantee
convergence of the chain to π. Note, however, that the mere fact of con-
vergence does not yet imply that the algorithm is efficient in the sense of
achieving practical convergence in a relatively small number of iterations.
That is, it does not necessarily describe a fast mixing Markov chain.
A well-chosen instrumental distribution should generate values that cover
the support of the target distribution in a reasonable number of iterations,
and the proposals should be neither accepted nor rejected too often. These
features are related to the dispersion of the proposal distribution that gen-
erates the simulated values. Specifically, if q is too dispersed relative to π,
the proposed values are rejected frequently and the support of π can only
be representatively sampled after many iterations, implying slow conver-
gence. In the opposite case of narrow dispersion, only a small subset of
S is visited across many iterations, with a high acceptance rate that might
be falsely interpreted as quick convergence, when in fact a large number of
additional iterations is needed to explore other parts of S. For these reasons
7
See e.g. Tierney (1994) or Robert and Casella (2004). If additionally the chain is
aperiodic – and this is guaranteed if r(u) > 0, ∀u ∈ S – then one can prove convergence
(in a suitable sense) of the n-step transition function, Pn (u, ·), to Π as n → ∞.
 6.2 Metropolis–Hastings Algorithm 99

one should always start with a preliminary analysis of π, such that q could
be chosen to approximate the target distribution as well as possible.
Given the generic nature of the M-H algorithm, we now describe two of
the most commonly used special cases. 8

Independence M-H Algorithm

As implied by the name of the algorithm, the proposal distribution is inde-
pendent of the current state, i.e., q(ũ | u) = q(ũ). This implies the accep-
tance probability
( )
π(ũ) q(u(t) )
α(u(t) , ũ) = min , 1 , t ≥ 0.
π(u(t) ) q(ũ)
Similar to what we noted about the general M-H algorithm, ergodicity of
the chain {U (t) } requires that the support of the proposal distribution q, now
without conditioning, contain the support of π.
For instance consider simulation from a posterior distribution, i.e., π(θ) =
h(θ | x) ∝ L(θ | x)h(θ) and the states are {U (t) ≡ θ(t) }. An illustration of an
independence M-H chain in this context is the special case with q(θ) = h(θ).
In this case the support of q covers the support of π, even if the two distri-
butions could be very different. Also, in this case, the M-H ratio reduces to
L(ũ|x)
a likelihood ratio R(θ(t) , ũ) = L(θ (t) |x) .

Random Walk M-H Algorithm

This algorithm is defined by an instrumental distribution Ũ = U (t) + εt ,
where εt is a random error with distribution q∗ independent of U (t) . This
defines a random walk with transition density q(ũ | u) = q∗ (ũ − u). Usual
choices for q∗ include uniform distributions on a ball centered around the
origin, Gaussian, and Student t distributions.
Note that if the proposal distribution is symmetric, i.e., q(ũ | u) = q(u |
ũ), then the M-H ratio simplifies to R(u, ũ) = π(ũ)
π(u)
, highlighting that the tar-
get distribution need only be known up to a normalization constant. Sym-
metry occurs when q∗ (y) depends on y only through |y|. When a chain is
based on a random walk Ũ ∼ q∗ (|ũ − u(t)) |) this becomes the Metropolis
algorithm introduced in Metropolis et al (1953) in the context of a problem
of the physics of particles with a discrete state space.9
8
See, e.g., Givens and Hoeting (2005) for other cases.
9
Nicholas Metropolis and Stanislaw Ulam were jointly the founders of what they named
Monte Carlo methods.
100 Markov Chain Monte Carlo Methods

6.3 Gibbs Sampler

The generic character of the M-H algorithm was evident in its description
in the previous section, in particular in the fact that it was not necessary
to even indicate the dimension of u in the target distribution π(u). In con-
trast, the Gibbs sampler algorithm, 10 which is the subject of this section,
is specifically designed for k-dimensional distributions (k ≥ 2).
The algorithm constructs a Markov chain that is set up to converge to
a desired target distribution π(u), u = (u1 , u2 , . . . , uk ) ∈ U. This is imple-
mented by iteratively sampling from the conditional distributions (typically
univariate) given all other elements, also referred to as full conditional dis-
tributions. The algorithm successively replaces the elements of a state vec-
tor u in cycles of k steps, with the j-th step replacing u j by a value sampled
from the conditional distribution π(v j | {ui , i < j}), j = 1, 2, . . . , k. For ex-
ample, assuming k = 3, a cycle of three steps substitutes the current state u
by v = (v1 , v2 , v3 ) with
step 1 step 2 step 3
(u1 , u2 , u3 ) −→ (v1 , u2 , u3 ) −→ (v1 , v2 , u3 ) −→ (v1 , v2 , v3 ).
In general, given a currently imputed state u(t) = (u(t) (t)
1 , . . . , uk ) one tran-
sition of the Gibbs generates iteratively the values u(t+1) for the next state
vector of the chain using the full conditional distributions
U1(t+1) ∼ π(u1 | u(t) (t) (t)
2 , u3 , . . . , uk )

U2(t+1) ∼ π(u2 | u(t+1)

1 , . . . , uk(t) )
↓ (6.5)
(t+1)
Uk−1 ∼ π(uk−1 | u(t+1)
1 , u2(t+1) , . . . , uk−2
(t+1) (t)
, uk )
Uk(t+1) ∼ π(uk | u1(t+1) , u(t+1)
2
(t+1)
, . . . , uk−1 ).
The next transition repeats the k-step cycle, now starting from u(t+1) . The
scheme is summarized in Algorithm 2, below.

Example 6.1 Let x = (xi , i = 1, . . . , n) be a random sample from a Weibull

model with unknown scale and shape parameters denoted δ and α, respec-
tively. The likelihood function is

α−1 −δ Pi xiα
L(δ, α | x) = (δα)n Πni=1 xi e , δ, α > 0.
Assume that δ and α are a priori independent with gamma, Ga(a, b), and
10
The name for this method originated from Geman and Geman (1984) in an application
to inference for so-called Gibbs random fields, referring to the physicist J. W. Gibbs.
 6.3 Gibbs Sampler 101

Algorithm 2 Gibbs sampler

1. Given a current state u(t) = (u(t) (t)

1 , . . . , uk ), starting with t = 0, generate
each component, u(t+1)
j , of the next state vector for the chain using

U (t+1)
j ∼ π(u(t+1)
j | u(t+1)
1 , . . . , u(t+1) (t) (t)
j−1 , u j+1 , . . . , uk ),

for j = 1, 2, . . . , k.
2. At the end of k steps, take u(t+1) = (u1(t+1) , . . . , u(t+1)
k ) and repeat step 1 for
t ≡ t + 1.

log-normal, LN(c, d) distributions, respectively, with fixed hyperparame-

ters, a, b, d > 0 and c ∈ R.
The joint posterior density has the kernel

α 2 P α

h(δ, α | x) ∝ αn+c/d−1 Πni=1 xi e−(ln α) /2d δa+n e−δ b+ i xi .
This implies complete conditional distributions proportional to
P α

1. h(δ | α, x) ∝ δa+n e−δ b+ i xi ;  

α − (ln α)2 +δ Pi xiα
2. h(α | δ, x) ∝ αn+c/d−1 Πni=1 xi e 2d .
P
Thus, the complete conditional distribution for δ is Ga(a + n, b + i xiα )
and the generation of values for δ for any given α can be carried out by a
computation-efficient gamma random variate generator. The complete con-
ditional distribution for α does not appear in a standard form, thus requir-
ing the use of more sophisticated methods of random variate generation.
For example, adaptive rejection sampling based on the concave nature of
the log-normal density. See also Note 4 in this section, and Appendix B for
generic grid-based random variate generation. 
Let u generically denote the currently imputed state vector at the begin-
ning of the j-th step. The Gibbs sampler proceeds as if it were generating
a vector ũ = (u1 , . . . , u j−1 , ũ j ,u j+1 , . . . , uk ) with



π(ũ j | u− j ), if ũ− j = u− j
Ũ | u ∼ q j (ũ | u) =   (6.6)
0, otherwise.
The term Gibbs sampler refers not only to the described version. There
are many variations about the sequential updates and about the simulation.
Among others, this includes the following:
102 Markov Chain Monte Carlo Methods

Gibbs Sampler with Blocking

Although the typical description of the Gibbs sampler uses univariate full
conditional distributions, the scheme can include a more flexible number
of steps in the cycle, with each step using the conditional distribution for
a subvector of any dimension. This variation has the particular advantage
of allowing us to group higher correlated variables together, such that gen-
erating from the complete conditional distributions for the entire subvector
can accelerate convergence of the algorithm. It is usually used whenever
the complete conditional posterior distribution for any subvector of the pa-
rameter vector is available for efficient random variate generation.

Gibbs Sampler with Hybridization

In general, there might be no known efficient random variate generators
for some of the complete conditional distributions. In such cases, one can
always resort to other transition probabilities that can then be combined
with the Gibbs sampler to define a hybrid Gibbs sampler. An example is
the use of M-H transition probabilities as discussed, for example, in Givens
and Hoeting (2005, Section 7.2.5).
After this brief introduction to some variations, it is helpful to highlight
some of the aspects of the Gibbs sampler algorithms in general.
Note 1: no proposal distribution. As shown, random variate generation
in the Gibbs sampler is based on the target distribution itself. This avoids
the often difficult problem of finding a “good” proposal distribution, as it
is needed in the M-H algorithm. However, generating a single variable in
each iteration of the Gibbs sampler is not a good device for fast mixing
over the support of the target distribution.
Note 2: Gibbs and M-H algorithms. Despite differences between the
Gibbs and the M-H algorithms, there is a close connection that is best seen
in (6.6). Consider step j of a cycle of the Gibbs sampler starting with u(t) .
Simulation involves the conditional distribution q j (ũ | u) = π(ũ j | u− j )
with ũ being a vector that differs from u only in the j-th component, that
is ũ− j = u− j . The distribution q j (ũ | u) plays the role of a proposal dis-
tribution. By the definition of a joint distribution, and earlier introduced
notation, we can write π(u) = π(u− j )π(u j | u− j ), where the first and sec-
ond factor refer, respectively, to the marginal distribution for U− j and the
conditional distribution for U j given U− j . The factors are also exactly the
distributions of Ũ− j and the conditional distribution of U j given Ũ− j . Thus,
π(ũ) π(ũ j | u− j ) q j (ũ | u)
= ≡ ,
π(u) π(u j | ũ− j ) q j (u | ũ)
 6.3 Gibbs Sampler 103

implying for this step the M-H ratio

π(ũ) q j (u | ũ)
R j (u, ũ) = = 1.
π(u) q j (ũ | u)
Each cycle of the Gibbs sampler can therefore be seen as a composition
of k M-H transition probabilities, with acceptance probability for each step
equal to 1. Note that when alternatively an entire cycle of the Gibbs sam-
pler was interpreted as a single M-H transition function, a corresponding
global acceptance probability could be calculated for the transition density
between the initial and the final step of the cycle. This acceptance proba-
bility would not reduce to a constant 1.11
Note 3: bivariate case. The definition of the Gibbs sampler in the bivari-
ate case consists of the steps U1(t+1) ∼ π1 (· | u(t) (t+1)
2 ) and U 2 ∼ π2 (· | u(t+1)
1 )
(t)
for t ≥ 0, and highlights clearly that the sequence {(U1 , U2(t) )} defines a
Markov chain. Also each of the subsequences is a Markov chain, e.g., U2(t)
is a Markov chain with transition density
Z
P(u2 , ũ2 ) = π1 (w | u2 )π2 (ũ2 | w) dw,

which depends on the past only through the previous value of U2 . In other
words, the definition of the marginal densities and the transition density for
U2 implies
Z
π2 (ũ2 ) = π2 (ũ2 | w)π1 (w) dw =
Z "Z # Z
π2 (ũ2 | w)π1 (w | u2 ) dw π2 (u2 ) du2 = P(u2 , ũ2 )π2 (u2 ) du2 ,

which shows that π2 is a stationary distribution for the subchain U2(t) .

One of the basic conditions for convergence of a multivariate {U (t) } is
that the support U of the joint distribution π(·) be the Cartesian product
of the supports U j of the marginal distributions π j (·). This implies that
the chain is irreducible and, in the bivariate case, the same holds for the
marginal subchains. If, additionally, the transition function is absolutely
continuous with respect to Lebesgue, with the density taking the form of a
11
It suffices to consider the case k = 2 and the corresponding transition density
P(u, ũ) = π1 (ũ1 | u2 )π2 (ũ2 | ũ1 ), where π j refers to the marginal distribution of the
element U j of the pair (here conditional on the other element). In this case the M-H
acceptance ratio π(ũ1 )/π(ũ1 | u2 ) π(u1 | ũ2 )/π(u1 ).
104 Markov Chain Monte Carlo Methods

product of complete conditional densities,

p(u, v) = π1 (v1 | u2 , . . . , uk ) π2 (v2 | v1 , u3 , . . . , uk ) × . . .

× πk (vk | v1 , v2 , . . . , vk−1 ),
the chain is (Harris) recurrent, implying that π is the stationary distribution
of the chain {U (t) } and its marginals are the limiting distributions of the
respective subchains, with the resulting applicability of the ergodic theo-
rem.12
In summary, the structure and convergence of the Gibbs sampler high-
light that the full conditional distributions suffice to characterize and gen-
erate from the joint distribution. It is helpful to explore (counter-)examples
for the incompatibility of conditional distributions to appreciate the rele-
vance of the conditions that assure convergence to the target distribution in
a Gibbs sampler (see also Paulino et al., 2018 and the references therein).
In particular, one needs to be careful with the normalization constant in
the complete conditional distributions. In particular, an (improper) infinite
normalization constant makes the existence of a proper joint distribution
impossible, a condition which is not always detected by inspection of the
generated Markov chains, but it occurs not uncommonly with Bayes mod-
els with improper prior distributions (see also Robert and Casella, 2004:
ch. 10, and references therein).
Note 4: The simulation from complete conditional distributions depends
naturally on the specific structure of these conditionals. In the simplest
case, resorting to the inverse c.d.f. method or efficient ad hoc methods for
certain known distributions might allow efficient random variate genera-
tion for the corresponding steps in the Gibbs sampler. In more complicated
cases, the required sampling is still possible with more sophisticated ap-
proaches described in the references that were indicated in the introduction
to Chapter 4.
In many statistical inference problems, the target distribution can be dif-
ficult to evaluate; for example, if it involves analytically intractable inte-
grals. As a consequence, no easy method for random variate generation for
the steps in the Gibbs sampler might be available. One often successful
method (like in missing data problems) is to augment the target distribu-
tion π(u) to f (u, Z) by introducing additional latent variables Z such that
12
And also convergence of the n-step transition function to π if the chain is additionally
aperiodic.
 6.3 Gibbs Sampler 105

π(u) remains the marginal distribution of the joint f (u, Z). 13 In some cases,
the augmented model f (u, Z) allows a much easier implementation of the
Gibbs sampler, now involving the distributions f (u | z) and f (z | u).

Example 6.2 Consider a diagnostic test for some disease with a binary
outcome (positive or negative). Taking a random sample of N patients, let
X be the number of positive results, and assume that X | φ ∼ Bi(N, φ).
Most diagnostic tests are subject to classification errors implying that the
probability of a positive can be written as φ = ασ + (1 − α)(1 − ε), where
θ = (α, σ, ε) and α is the prevalence of the disease, σ is the sensitivity of
the test (probability of a true positive), and ε is the specificity of the test
(probability of a true negative).
The vector θ is typically unknown and is the parameter vector of interest
in the inference problem. However, given the obvious overparametrization
of the sampling model, it is not possible to report inference on θ (or func-
tions of θ, except φ, for example) without additional data (e.g., from a fur-
ther test regarded as a gold standard) or a priori information related to the
type of diagnostic test and the disease under consideration. Suppose that
one only has access to such prior information, represented as independent
beta distributions for the components of θ, with fixed hyperparameters. The
posterior distribution takes the following analytically intractable form:
h(θ | x) ∝ f (x | θ) αa p −1 (1 − α)b p −1 σcs −1 (1 − σ)ds −1 εce −1 (1 − ε)de −1 ,
θ ∈ (0, 1)3 . Intending to implement posterior inference using MCMC, the
Gibbs sampler is not a particularly easy approach for this posterior dis-
tribution because the conditional posterior distributions are complicated,
requiring specialized random variate generation methods. However, the
implementation of a Gibbs sampler is greatly facilitated by the following
model augmentation with latent data. Let Y = (X, Z1 , Z2 ), where Z1 (resp.
Z2 ) are unobserved (latent) data that report the number of individuals with
true positive (negative) results. A model for Y that remains consistent with
the observed data X is defined as
f (y | θ) = f (x | θ) f (z1 | x, θ) f (z2 | x, θ),
where now Z1 | x, θ ∼ Bi(x, ασ/φ) and Z2 | x, θ ∼ Bi(N −x, (1−α)ε/(1−φ)).
Note that the parameters of the conditional distributions for the latent
variables correspond to the so-called positive predicted values V+ = ασ/φ
13
The construction of f (u, Z) could also be referred to as demarginalization or
augmentation of π(u).
 106 Markov Chain Monte Carlo Methods

and negative predicted values V− = (1 − α)ε/(1 − φ). The posterior density,

now with the augmented data y, is then
h(θ | y) ∝ f (x | φ)(V+ )z1 (1 − V+ ) x−z1 (V− )z2 (1 − V− )N−x−z2 ×
αa p −1 (1 − α)b p −1 σcs −1 (1 − σ)ds −1 εce −1 (1 − ε)de −1 .
The expression is considerably simplified to a product of three beta densi-
ties. In fact, if we introduce a transformation of the data from y = (x, z1 , z2 )
to y∗ = (m, z1 , z2 ), where m = z1 + N − x − z2 is the number of individuals
in the sample who have the disease, we find from f (y | θ) that
f (y∗ | θ) = f (m | θ) f (z1 | m, θ) f (z2 | m, θ),
such that M | θ ∼ Bi(N, α), Z1 | m, θ ∼ Bi(m, σ) and Z2 | m, θ ∼ Bi(N −
m, ε).
This factorization of the likelihood, considering factorization of the joint
prior and of the binomial and beta factors, implies that the components of
θ are also a posteriori independent, with the following distributions:
α | y ∼ Be(A p , B p ), A p = a p + m = a p + z1 + N − x − z2 ,
B p = b p + N − m = b p + x − z1 + z2
σ | y ∼ Be(C s , D s ), C s = c s + z1 , D s = d s + N − x − z2
ε | y ∼ Be(Ce , De ), Ce = ce + z2 , De = de + x − z1 .
These are the complete conditional distributions for the parameters condi-
tional on the augmented data y. Since the parts z1 and z2 of the augmented
data are not observed, one needs to impute these based on the parame-
ters. The latter can be done using the respective sampling distributions
conditional on the observed part x of the data. This leads to a Gibbs type
algorithm for the joint posterior distribution h(θ, z1 , z2 | x), defined by the
following two steps.
Data Augmentation
1. Imputation step: given θ(0) = (α(0) , σ(0) , ε(0) ), compute V+(0) = V+ (θ(0) )
and V−(0) = V− (θ(0) ) and generate
z(1) (0) (1) (0)
1 ∼ Bi(x, V+ ), z2 ∼ Bi(N − x, V− ).

2. Posterior step: based on (z(1) (1)

1 , z2 ), generate from the posterior distribu-
tion for θ given the augmented data. That is, generate θ(1) as
α(1) ∼ Be(A p , B p ), σ(1) ∼ Be(C s , D s ), ε(1) ) ∼ Be(Ce , De ).
Starting from θ(1) repeat the cycle of the two steps repeatedly.
 6.4 Slice Sampler 107

This algorithm was introduced, still without any reference to the Gibbs
sampler, by Tanner and Wong (1987) under the aforementioned title, where
they prove that h(θ | x, z1(t) , z(t)
2 ) converge as t → ∞ to h(θ | x), under quite
general conditions.


6.4 Slice Sampler

As we saw, complex target distributions π can complicate conditional ran-
dom variate generation even when pointwise evaluation of the density func-
tion π(u) in any u ∈ U remains possible. In this case, yet another strategy
to implement MCMC is to introduce an auxiliary variable Z with the aim
of facilitating simulation of a chain for (U, Z) ∼ f (u, z) = π(u) f (z | u).
Here, Z should be chosen such that the chain for the augmented distribu-
tion f (u, z) converges and such that the exploration of the support U of the
corresponding subchain and the evaluation of the desired summaries of π
are possible.
Defining Z such that Z | U = u ∼ U(0, π(u)), we get (U, Z) ∼ U(S)
where S = {(u, z) : u ∈ U, z ∈ [0, π(u)]} and U(S) refers to a uniform dis-
tribution over the set S. Thus one way of obtaining a MC sample from π is
to generate a Markov chain with the stationary distribution being precisely
the multivariate uniform on S.
The slice sampler is an iterative method to generate a random walk on
S, moving alternating in two directions using uniform distributions. In the
first step over the real line using Z | U = u ∼ U(0, π(u)) and in the second
step over U using U | Z = z ∼ U(S(z)), with S(z) = {u ∈ U : π(u) ≥ z}
– note that the marginal density f (z) is therefore proportional to Lebesgue
measure of S(z).
If the chain converges, this method thus generates an approximate sam-
ple from π by considering the corresponding subchain, requiring only the
evaluation of π in the locations simulated by the uniform distribution. Actu-
ally, the sampling schemes only requires π up to a normalization constant14 .
In summary, the sampling algorithm is defined as follows.
14
In fact, writing π(u) = cπ∗ (u) and letting Z ∗ = Z/c, the approach is equivalent to using
(U, Z ∗ ) ∼ U({(u, z∗ ) : u ∈ U, z∗ ∈ [0, π∗ (u)]}), implying Z ∗ | U = u ∼ U(0, π∗ (u)) and
U | Z ∗ = z∗ ∼ U({u : π∗ (u) ≥ z∗ }).
 108 Markov Chain Monte Carlo Methods

Algorithm 3 Slice Sampler

Given (u(t) , z(t) ), starting with t = 0, simulate
1. z(t+1) ∼ U(0, π(u(t) ));
2. u(t+1) ∼ U({u : π(u) ≥ z(t+1) });
and repeat the cycle of these two steps incrementing t each time; note
that π(u) here denotes the density or its kernel, whichever is easier to
evaluate.

We conclude with a few more comments about this approach whose

name is attributed to Neal (1997) and which goes back to work published
in Neal (2003) and Damien et al (1999).
Note 1: univariate case. For univariate U the slice sampler can be easily
illustrated by a graphical representation of the density π(u) with U and Z
marked on the horizontal and vertical axes, respectively (see Figure 6.1).
The point (u(t) , π(u(t) ) defines on the vertical axis a slice over which the

z π(u)

z (t+1)×

×
u(t+1) u(t) u
Figure 6.1 Slice sampler for a univariate distribution π(u).

value z(t+1) is generated. The intersection of the line Z = z(t+1) with π(u)
defines the point(s) that delimit the horizontal slice S(z(t+1) ) (an interval
or union of intervals) over which u(t+1) is generated. In practice, the main
difficulty with this algorithm is the second step since the support S(z) of the
distribution on the horizontal slice could be complicated for a multimodal
π(u), which might need the use of other simulation methods in this step
 6.5 Hamiltonian Monte Carlo 109

(e.g., rejection sampling). In any case, by the nature of the algorithm it

works better than many other algorithms (such as, for example, M-H) with
multimodal target densities, in the sense of an efficient exploration of the
support of such target distribution.
Note 2: slice sampler and Gibbs. The structure of the slice sampler for
univariate U highlights that it can be seen as a special case of a two-step
Gibbs sampler for the model augmentation of π(u) to f (u, z) = π(u) f (z |
u), a uniform in S. The sequence
R {U (t) } is therefore a Markov chain with
transition density P(u, ũ) = f (z | u) f (ũ | z) dz and stationary distribution
π(u).
This interpretation remains essentially valid also for multivariate target
distributions, except for a naturally larger number of steps. As a conse-
quence, the convergence conditions for a slice sampler can be seen to fol-
low from those for the Gibbs sampler, based on the introduction of a vector
of auxiliary variables to deal with more complex target distributions (for a
discussion, see, for example, Robert and Casella, 2004).

6.5 Hamiltonian Monte Carlo

6.5.1 Hamiltonian Dynamics
A common problem with some of the earlier discussed MCMC schemes is
the often local nature of the transitions. For example, with a Metropolis–
Hastings random walk transition function, we often only move a small step
in the parameter space. With a Gibbs sampler we only update one param-
eter at a time. High posterior correlations can lead to very slowly mixing
Markov chains. An interesting alternative that can allow us to quickly move
around the parameter space is Hamiltonian Monte Carlo (Neal, 2011). The
basic idea is very simple. There are three important steps to the construc-
tion. Let π(θ) denote the target distribution of interest, e.g., a posterior dis-
tribution h(θ | x).
First, we add a (entirely artificial) time index to θ, making it θ(t). Even-
tually, after the transition, we will drop the t index again to obtain the new
parameter values.
Next, we start the construction with a differential equation system for
dθ(t)/dt that is known to keep a given target function invariant. That is,
if we simulate transitions following the solution of this system, then these
transitions would keep the target function unchanged. If we use ln π as
the target function, then this gives us transitions that move along equal
contour lines of π(θ). Such a differential equation system is, for example,
110 Markov Chain Monte Carlo Methods

Hamilton’s equations of Hamiltonian mechanics. All we need to do is to

equate potential energy with the log posterior distribution. Then we sim-
ulate Hamiltonian dynamics. We are guaranteed that the simulated states
have all equal posterior density. That is, we move on contours of the joint
posterior distribution.
There is one more clever twist to the setup. Let N(x | m, S ) denote a mul-
tivariate normal p.d.f. for the random vector x with mean m and covariance
matrix S . We first augment the probability model to π(θ, p) = π(θ) N(p |
0, I), using a multivariate normal distribution for p (and it could be any
other – but this makes the following derivations easiest). The notation p
for the additional variable is chosen in anticipation of the upcoming inter-
pretation of the augmented model. Note that this model augmentation is
unusual in the sense that θ and the latent variable p are independent. That
will turn out to greatly simplify the algorithm. Let
1 ′
H(θ, p) = − ln π(θ) + pp (6.7)
2
denote the negative log augmented target distribution (ignoring constant
factors). We use H(θ, p) as the target function (potential) for the Hamil-
tonian equations, interpreting θ as location and p as momentum. That is
all! All we have left to do is to state the equations and then implement
an approximate numerical solution to the differential equation. Let θ =
(θ1 , . . . , θd ) and p = (p1 , . . . , pd ) denote the two d-dimensional location
and momentum. Hamilton’s equations are
dθi ∂H d pi ∂H
= and =− . (6.8)
dt ∂pi dt ∂θi
Changing (θ(t), p(t)) according to these equations leaves the potential un-
changed. Since we set the potential to be the log (augmented) target dis-
tribution we are guaranteed to move on equal probability contours. That is
the trick. The particular choice (6.7) simplifies the equations further, with
dθi ∂H d pi ∂H ∂ ln π
= = pi and =− = .
dt ∂pi dt ∂θi ∂θi
There is a beautiful interpretation of the setup. In the application of (6.8)
to mechanics, the parameters θ become the location of an object and p is
the momentum (that is, velocity × mass). For example, think of the object
as a ball on a slope. Then ln π(θ) is the potential, that is the energy due to
location, and 21 p2 is the kinetic energy. It is good to think of the object as a
ball, as we entirely ignore friction. Hamiltonian mechanics describes how a
ball in location θ and with momentum p at time t will move. Its movement
 6.5 Hamiltonian Monte Carlo 111

is determined by keeping the sum of potential and kinetic energy constant.

Compare this to Figure 6.2, later.

Leapfrog Approximation
To implement Hamiltonian dynamics we use a discretization of the differ-
ential equation system (6.8) known as the leapfrog method. Starting from
(θ(t), p(t)) we generate (θ(t + ǫ), p(t + ǫ) by using a discrete approximation
over two subintervals of length ǫ/2 each:
 ǫ ǫ ∂ ln π(θ(t))
pi t + = pi (t) +
2 2 ∂θ
 iǫ 
θi (t + ǫ) = θi (t) + ǫ pi t +
2
 ǫ  ǫ ∂ ln π(θ(t + ǫ))
pi (t + ǫ) = pi t + + . (6.9)
2 2 ∂θi
Let T ǫ (θ(t), p(t)) = (θ(t + ǫ), p(t + ǫ)) denote the discrete approximation
implemented in (6.9). It is easily verified that the approximation is perfectly
reversible, that is, T −ǫ (θ(t + ǫ), p(t + ǫ)) = (θ(t), p(t)) again, or T −ǫ (θ, p) =
T ǫ−1 (θ, p). Even easier than turning back time, all we have to do to send
the ball back to where it came from is to turn it around. That is, p ≡ −p,
or T ǫ−1 (θ, p) = T ǫ (θ, −p). Note that the time index in θ(t) and p(t) is only
used for the implementation of T ǫ (·). After the last step in (6.9), we drop
the time index again.

Example 6.3 Logistic regression. In a toxicity study for some new com-
pound, various dose levels of the compound are administered to batches
of animals. Let i = 1, . . . , k index the batches, let xi index the dose for the
i-th batch, let ni denote the number of animals in the i-th batch, and yi the
number of animals in the i-th batch that show a response. We assume that
yi is a binomial random variable, yi ∼ Bi(ni , πi ), where  πi is the probability
of response at dose xi . We assume πi = 1/ 1 + eα+βxi . This is known as a
logistic regression model. The probabilities πi define the sampling model
(likelihood).
We complete the model with a prior h(α, β) = N(0, cI), where I is a
(2 × 2) identity matrix, and c = 100, i.e., a very vague prior. We observe
112 Markov Chain Monte Carlo Methods

the following data:

Number Number
i Dose xi animals ni responses yi
1 −0.86 6 1
2 −0.30 5 2
3 −0.05 5 3
4 0.73 5 5
In this example, we use (6.9) to define transitions that leave the augmented
log posterior (6.7) invariant. For the moment these are deterministic tran-
sitions – we will later add randomness. This will be easy. Let θ = (α, β)
denote the parameter vector. Let πi = 1 − πi and ȳi = ni − yi denote
the probabilities and number of non-responses. Then f (y | θ) = Πi πyi i πȳi i ,
h(θ) = N(0, I) and
α2 + β2 X   X  
ln h(θ | y) = c − − yi ln 1 + eα+βxi − ȳi ln 1 + e−α−βxi .
2 i i

with gradient
! X ! X !
−α 1 1
∇ ln h(θ | y) = − yi πi + ȳi πi .
−β xi xi
Using (6.9) with target distribution π(θ) = h(θ | y), we implement M = 8
steps of size ǫ = 0.1. The transitions are deterministic, and we expect to
stay on a constant level of the augmented model (6.7). The algorithm re-
quires an initial value of p. Think of θ as the location of a ball on log pos-
terior contours, and p as the momentum that you give the ball by kicking it.
Hamiltonian mechanics and the approximation in (6.9) describes the tra-
jectory of the ball over M small time intervals of length ǫ. In this analogy
you have to think of the posterior distribution as a valley, with the posterior
mode being the lowest point. This is because ln h(θ | y) appears in (6.7)
with negative sign. Figure 6.2 shows a contour plot of the log posterior
“valley,” and three possible trajectories. All start at θ = (−2, −4), kicking
the ball with p = (1, 0), (1, −3), and (1, 1.4). On the four-dimensional con-
tours of the augmented model H(θ, p) the trajectories would move along
contour lines. Next we will introduce a random transition function, build-
ing on this deterministic transition of the leapfrog approximation, which we
will then use in Problem 6.12 to implement MCMC posterior simulation for
this example.

 6.5 Hamiltonian Monte Carlo 113

0
●

●
0.1
● 0.2

0.4
●

−2
0.6
●
0.8
●
●
● ●
●
● ●
●
● 0.9

−4
● ● ●

β
●
● 0.7
●
● ● ●
● ●
0.5
−6

0.3
−8

−3.0 −2.5 −2.0 −1.5 −1.0 −0.5 0.0 0.5

α

Figure 6.2 Log posterior ln h(θ | y) with trajectories for M = 8

steps of the leapfrog approximation (6.9). All trajectories start at
the same point θ, and correspond to three different initial values
for p.

6.5.2 Hamiltonian Monte Carlo Transition Probabilities

Recall that (exact) Hamiltonian dynamics leaves the augmented probability
model h(θ, p) invariant. However, the approximation T ǫ (·) does not – with
the approximation error depending on the step size. But this is not a prob-
lem for our application. We use T ǫ to generate a proposal in a Metropolis–
Hastings transition probability. Starting with (θ, p), we generate a proposal
(θ̃, p̃) with



T ǫ (θ, p) w. pr. 21
(θ̃, p̃) = 
T (θ, p) w. pr. 1 . (6.10)
 −ǫ 2

By the earlier comment, the proposal distribution is symmetric, leaving the

acceptance probability α = min(1, R) with ln(R) = H(θ̃, p̃)−H(θ, p). In this
implementation, a possible approximation error in (6.9) is a feature, not a
problem, as it allows us to move across contour lines of H(·).
The final trick of Hamiltonian Monte Carlo is particularly clever. We
alternate the M-H step (6.10) with a Gibbs transition probability by gener-
ating p from the complete conditional under π(·),
p ∼ π(p | θ) = N(0, I).
Here, we exploit the fact that location and momentum are independent in
(6.7), making the Gibbs step particularly easy.
 114 Markov Chain Monte Carlo Methods

In summary, the MCMC algorithm proceeds with the following two tran-
sition probabilities.
1. Generate pi ∼ N(0,  1), i = 1, . . . , d.


T ǫ (θ, p) w. pr. 1/2
2. Generate (θ̃, p̃) = 
 .
T −ǫ (θ, p) w. pr. 1/2
n o
With probability α = min 1, π( θ̃)
π(θ)
, accept the proposal, θ ≡ θ̃.
To justify the acceptance probability imagine a minor modification of step
2. After generating (θ̃, p̃) as described we replace p̃ by p† ∼ π(p | θ̃) and
recognize α as the M-H acceptance probability for (θ̃, p† ). Finally, we do
not need to record p in step 2 since p is immediately overwritten in step 1 of
the next iteration. Also, if desired, T ǫ can be changed to implement multiple
leapfrog steps. The algorithm remains unchanged (and the approximation
error increases with multiple steps).
The algorithm can be further simplified by noting that pi (t + ǫ) in the
last step in the leapfrog approximation (6.9), that is T ǫ or T −ǫ in step 2, is
not needed. The value pi (t + ǫ) is never used in the MCMC described here.
It is immediately replaced by a new value, pi ∼ N(0, 1) in step 1 of the
following iteration. The first two lines of (6.9) can be collapsed into
( )
ǫ ∂ ln π(θ(t))
θi (t + ǫ) = θi (t) + ǫ pi (t) + . (6.11)
2 ∂θi
And, even easier, we can further collapse (6.11) with pi ∼ N(0, 1) in step 1
to get
( )
ǫ 2 ∂ ln π(θ(t)) 2
θi (t + ǫ) | θi (t) ∼ N θi (t) + ,ǫ .
2 ∂θi
That is, we replace steps 1 and 2 with a single step.
1′ . Generate a proposal,
( )
ǫ 2 ∂ ln π(θ) 2
θ̃i | θ ∼ N θi + ,ǫ .
2 ∂θi
Accept with probability α = min{1, R} with acceptance ratio
n o
ǫ 2 ∂ ln π(θ̃) 2
π(θ̃) d N θi | θ̃i + 2 ∂θ̃i , ǫ
R= Π n o,
π(θ) i=1 N θ̃i | θi + ǫ 2 ∂ ln π(θ) , ǫ 2
2 ∂θi

the first factor being the ratio of target distributions and the second fac-
tor the ratio of proposal distributions for the proposed and the reciprocal
moves. This and other simplifications are discussed by Welling and Teh
 6.6 Implementation Details 115

(2011). In particular, they observe the following clever interpretation of

the last version of the HMC algorithm. First, letting δ = ǫ 2 , rewrite the
proposal distribution as
δ ∂ ln π(θ) √
θ̃i = θi + + δZ
2 ∂θi
where Z ∼ N(0, 1) is a standard normal random variable. Stated in this
form, note that for large δ the gradient term dominates, whereas for small
δ the standard normal term dominates.

6.6 Implementation Details

As we discussed before, MCMC methods aim to iteratively generate the
states of a Markov chain such that it approaches its limiting distribution
with increasing number of iterations. And the chain is set up such that this
limiting distribution equals a desired target distribution π(u). If, in a given
iteration t, the chain is already in (or “close to”) its limiting distribution,
then the generated states at that time and beyond can be used as approxi-
mate draws from the target distribution, which in our discussion is usually
a posterior distribution h(θ | x).
However, subsequent realizations of the same chain (over time) are not
a random sample from the target distribution, because of the serial correla-
tion of the generated vectors θ(t) . This is the main difference to classical MC
methods that we discussed in Chapter 4. While the summaries of the MC
sample are the same, the quantification of the corresponding numerical un-
certainty requires different summaries – for example, the sample standard
deviation of the ergodic averages is not the same as the standard error in
the i.i.d. case.15 Also, the asymptotic justification of the ergodic averages
introduces additional requirements.
Besides the importance of understanding convergence conditions for the
previously described MCMC schemes, it is equally important that these
conditions are not immediately useful from an implementation point of
view. This is the case, because the practical verification of these conditions
is often problematic and also because the methods themselves do not in-
clude diagnostics about when simulation can be considered sufficient to
evaluate the desired inference summaries.
15
One strategy is to use the mentioned sample standard error after a correction factor that
accounts for the serial correlations (see, e.g., Robert and Casella, 2004: chapter 12).
Another option, mentioned below, is to sufficiently thin out the chain such that the
remaining states can be considered as a random sample.
116 Markov Chain Monte Carlo Methods

For these reasons it is in practice necessary to resort to empirical di-

agnostics to evaluate the simulation output to understand when practical
convergence is achieved (with the desired target), keeping in mind that no
such methods are exact. Without such diagnostics it is difficult to have any
confidence in the reported summaries.

Single Chain Versus Multiple Chains

Many diagnostics aim to monitor convergence of the chain to its stationary
distribution or of the ergodic averages to the corresponding expectations.
Naturally, approaches differ about the extent to which they make use of the
simulation output, and whether they use a single long chain or multiple,
parallel independent chains. The latter two choices are described in the
following.

Single Chain Approach

Let θ =(0)
(θ1(0) , . . . , θk(0) )
denote the initial state of the chain.
• Generate one long realization of the chain of size t = ℓ + k⋆ m iterations,
where
– ℓ is the number of initial iterations used to reach practical convergence
(as determined by one of the diagnostics in the following). These ini-
tial iterations are also known as initial burn-in, and can be longer or
shorter depending on whether the chain is faster or slower mixing for
a specific model;
– m is the desired MC sample size, and k∗ is the spacing between states
of the chain {θ(t) } that are being saved, aiming to mitigate serial auto-
correlation between the saved states such that the resulting MC sam-
ple is approximately i.i.d. (k⋆ can be determined from a plot of auto-
correlations across different lags).
• In summary this results in extracting from the original chain a subset
∗ ∗ ∗
of m realizations θ(ℓ+k ) , θ(ℓ+2k ) , . . . , θ(ℓ+mk ) , which we shall now denote
∗
θ(1) , . . . , θ(m) , where θ( j) ≡ θ(ℓ+ jk ) , and which shall be used to evaluate the
desired inference summaries as discussed in Chapters 4 and 5.
The question arises of how to choose the sample size m, the number of
initial iterations ℓ and the spacing k⋆ between iterations. Since the choices
are highly dependent on the particular details in each problem, there are no
general answers. Note that the value of ℓ in particular is dependent on the
initial state of the chain and the level of mixing. The simulation sample size
m is dependent on the desired precision for the inference summaries. The
spacing k⋆ is highly dependent on the correlation structure and is meant
 6.6 Implementation Details 117

to achieve faster convergence of ergodic averages, although at the cost of

reduced efficiency (the ergodic average after thinning out has less precision
than the total average (after burn-in) – see MacEachern and Berliner, 1994).

Multiple Chains
• Generate m chains with t∗ (usually, t∗ ≪ t) iterations each, starting
with m initial values, usually generated to be distinct and well separated
across the support of the target distribution.
∗
• Use the final iterations θ(t ) of each chain to form a MC sample θ(1) , . . . , θ(m)
– the choice of t∗ , beyond some initial burn-in, and the choice of m are
subject to similar considerations as before.
Both approaches have their advantages and limitations, which explains
the great variety of choices that are found in the literature. 16 The first
scheme allows us to reduce the computational cost and can lead to a chain
closer to h(θ | x) than schemes using multiple chains, using the same total
number of iterations. The second scheme makes it easier to control conver-
gence to h(θ | x) by reducing dependence on the initial states and allows a
more direct exploration of the target support. It is more likely to detect if
the apparent convergence of a chain is merely an indication that it is trapped
around some local mode, far from the target distribution. If this is the case,
one can reparametrize the model (difficult in general) or one can change
the initial values. In summary, the scheme with multiple chains seems by
and large appropriate for an exploratory evaluation before running one long
chain for definite inference.

Convergence diagnostics
A variety of methods have been proposed to diagnose convergence. Some
are already included as default in specific or more general software for
Bayesian inference. Others can be added by writing problem-specific code
for a given inference problem. Keeping with the scope of this text, we limit
the discussion to a brief review of some of the most widely used methods.
The most widely used tools to monitor convergence to the stationary
distribution are plots of simulated values across iterations, known as tra-
jectory (or trace) plots, and the analysis of such plots across different time
windows to inspect any changes in the mixing of the chain over the sup-
port of the target posterior distribution. Another tool is the superposition of
16
The issue is discussed at length by Geyer (1992) and in the following discussion, where
many pros and cons of each method are presented.
118 Markov Chain Monte Carlo Methods

estimated marginal posterior densities based on increasing numbers of iter-

ations to detect when the estimate stabilizes. Yet another type of method is
the use of non-parametric tests to verify if the process appears to become
stationary. For example, one could use a Kolmogorov–Smirnov test for the
univariate marginal distributions across two subsamples corresponding to
non-overlapping ranges of iterations in the thinned-out chain.
To monitor convergence of ergodic averages for scalar functions g(θ),
P
given by S m = mi=1 g(θ(i) )/m (recall Chapter 4), one possible approach is to
P  
graph the cumulative sums Dℓ = ℓi=1 g(θ(i) ) − S m , ℓ = 1, . . . , m. For fast-
mixing chains this summary tends to look like noise centered around zero,
in contrast to slowly mixing chains for which this plot appears more regular
with long excursions to values far from zero. One possible variation of this
graphical summary is to use ergodic averages of conditional means when
g is a function of only a subset of the parameters, say g(α) for θ = (α, β).
P  
That is, S mc = mi=1 E g(α) | β(i) , where the conditional means are used,
which in the case of a Gibbs sampler are often readily available.
There are other methods to verify convergence based on different ideas
which appeared in the literature around the same time (beginning of the
1990’s), which are often known by the respective author names. These are
the methods of Gelman–Rubin, Raftery–Lewis, Geweke, and Heidelberger–
Welch. A comparative description of these methods appears in Cowles and
Carlin (1996). All are implemented in the R packages CODA (acronym
for COnvergence Diagnostic and output Analysis) developed by Best et al
(1995) and Plummer et al (2006) and BOA (Bayesian Output Analysis)
by Smith (2007). Chapter 9 of this text discusses Bayesian software and
includes examples of the use of these packages with MCMC methods.

Problems
Most of the following problems require programming to implement MCMC
simulation. See Appendix B for some suggestions on how to implement
such simulation in R. Later, in Chapter 9, we will discuss public domain
software to implement MCMC for most problems. However, it is useful to
implement MCMC and understand implementation details for at least some
more stylized problems. It is therefore recommended not to use MCMC
software, such as OpenBUGS, JAGS, or Stan in the following problems.
A large number of good case studies can be found in the manuals for
BUGS and R-INLA, including some substantially more complex infer-
ence problems than the following exercises. See www.openbugs.net/
Examples/ and also the problem set in Chapter 9.
 Problems 119

6.1 Metropolis–Hastings. Verify that the transition function (6.4) does indeed
satisfy the detailed balance condition (6.1).
6.2 Data augmentation. The table below shows the observed frequencies y j re-
lated to the observed phenotype defined by the blood group of an individual,
for a sample of n = 435 individuals. Here j ∈ {1, 2, 3, 4} indexes the four
blood groups O, A, B, AB.
j Blood Frequency Probability
group yj pj
1 O 176 r2
2 A 182 p2 + 2pr
3 B 60 q2 + 2qr
4 AB 17 2pq

The probabilities p j are determined by the laws of genetics, with p, q, and r

being the probabilities of the genes of type A, B, and O, respectively, with
p + q + r = 1.
a. Find the likelihood function f (y | θ) under this model for θ = (p, q) and
using r = 1 − p − q.
b. The observed phenotype (blood group) depends on the genotype which
cannot be directly observed. Following is the relationship between geno-
type and phenotype:
k Phenotype Genotype Probability p j
1 O OO r2
2 A AA p2
3 A AO 2pr
4 B BB q2
5 B BO 2qr
6 AB AB 2pq

Let zi ∈ {1, . . . , 6} denote the unobserved genotype for individual i, i =

1, . . . , n, and let z = (z1 , . . . , zn ). Write a complete data likelihood f (y |
z, θ).
c. Using the latent variables z from (b) and completing the model with a
suitable prior h(θ), propose a Gibbs sampling scheme to generate (θ, z) ∼
h(θ, z | y).
6.3 Binomial (unknown θ, n) ∧ beta/Poisson model. Refer to Problem 3.2. Find
the conditional posterior distributions h(θ | n, x) and h(n | θ, x). As before,
use x = 50, and (a0 , b0 ) = (1, 4).
a. Describe and implement a Gibbs sampling algorithm to generate (nm , θm ) ∼
h(n, θ | x). Plot the joint posterior h(n, θ | x), and plot on top of the same
figure the simulated posterior draws (nm , θm ), m = 1, . . . , 50 (connected
120 Markov Chain Monte Carlo Methods

by line segments showing the moves).

Hint: Use the R function sample(.) to generate from h(n | θ, x). When
evaluating h(n | θ, x), evaluate the function first on the log scale over a
grid on n, subtract the maximum (to scale it), and then only exponentiate
(to avoid numerical problems). See Appendix B.
b. In the same problem implement Metropolis–Hastings posterior simula-
tion. Add the simulated posterior draws on top of the plot from (a).

6.4 Missing data. Consider a bivariate normal sampling model:

(xi , yi ) ∼ N(µ, Σ), i = 1, . . . , n.
Here, µ = (µ1 , µ2 ) is the (bivariate) mean and Σ is the (2 × 2) covariance
matrix. We assume an improper prior,
h(µ, Σ) ∝ |Σ|−(d+1)/2 ,
where d = 2 is the dimension of µ.
a. Posterior distribution: Let y = {xi , yi , i = 1, . . . , n} denote the observed
data. Find the posterior distribution h(µ | Σ, y) and h(Σ | y).

b. Missing data posterior: Assume we observe the following data

i 1 2 3 4 5 6 7 8 9 10 11 12
x 1 1 −1 −1 2 2 −2 −2 − − − −
y 1 −1 1 −1 − − − − 2 2 −2 −2
with missing observations marked as “–”.
Let y denote the observed data. Let z = {x9 , . . . , x12 , y5 , . . . , y8 } denote the
missing data. Find p(z | Σ, µ, y) and h(µ, Σ | y, z).
Hint: Write h(µ, Σ | y, z) as h(Σ | y, z) · h(µ | Σ, y, z).
c. Data augmentation – algorithm. Using the conditional posterior distribu-
tions found in part (b), describe a data augmentation scheme to implement
posterior simulation from h(µ, Σ | y).
• Set up a Gibbs sampler for h(µ, Σ, z | y). Let θk = (µk , Σk , zk ) denote
the simulated Monte Carlo sample, k = 1, . . . , K.
• Simply drop the zk . The remaining (µk , Σk ) are an MC sample from
h(µ, Σ | y).
d. Data augmentation – simulation. Implement the data augmentation de-
scribed in part (c). Plot trajectories of generated µ j , j = 1, 2, against
iteration number and estimated marginal posterior distributions h(µ j | y).
e. Convergence diagnostic. Propose some (ad hoc) convergence diagnostic
to decide when to terminate posterior simulation in the program used for
part (d).
 Problems 121

(The answer need not be perfect – any reasonable, practical, creative sug-
gestions is fine. See Section 9.6 for more discussion of convergence diag-
nostics.)
6.5 We consider a hierarchical event rate model with a Poisson sampling model
yi ∼ Poi(λi ti ), and a prior model
λi ∼ Ga(α, β) and β ∼ Ga(c, d),
where (α, c, d) are fixed hyperparameters.17
a. Find the conditional posterior distributions:
1. h(λi | β, λ j , j , i, y).
2. h(β | λ1 , . . . , λn , y).
b. Using the data from Problem 4.4, implement a Gibbs sampler to simulate
from the posterior in the above model.
c. Alternatively, marginalize with respect to λi to find the marginal likeli-
hood f (yi | β) and h(β | y).
6.6 Assume that x, y are jointly distributed random variables with support (0, B),
with conditional p.d.f.
f (x | y) ∝ e−xy , 0 < x < B, (6.12)
f (y | x) ∝ e−xy , 0 < y < B. (6.13)
a. Propose a Gibbs sampler to generate an MC sample from f (x, y) (this is
easy – there are no tricks).
b. Now consider B = ∞, that is, the support for x and y is the entire posi-
tive real line. Show that there is no (proper) joint distribution f (x, y) with
conditionals (6.12) and (6.13).
c. Justify the statement “Under the setup of (b) we can not apply the Gibbs
sampler.”
6.7 Probit regression. Consider a probit regression model for a binary response
yi on covariates xi , i = 1, . . . , n. Let Φ(·) denote the standard normal c.d.f.
P(yi = 1 | xi ) = Φ(xi′ β), (6.14)
where β = (β0 , β1 , β2 ) and xi = (1, xi1 , xi2 ). We observe the following data.18
These are historical data – do you recognize them?
17
Compare with Problem 4.4 for a variation of the same problem.
18
The (identical) data are available as oring.dta on the book homepage
sites.google.com/view/computationalbayes/home
122 Markov Chain Monte Carlo Methods

xi1 xi2 yi xi1 xi2 yi xi1 xi2 yi xi1 xi2 yi

66 50 0 70 100 0 53 200 1 75 200 1
70 50 1 57 200 1 67 200 0 75 200 1
69 50 0 63 200 1 75 200 0 76 200 0
68 50 0 70 200 1 70 200 0 58 200 1
67 50 0 78 200 0 81 200 0 31 200 0
72 50 0 67 200 0 76 200 0
73 100 0 53 200 1 79 200 0

Let y = (y1 , . . . , yn ) denote the observed data. Assuming an improper prior

h(β) = 1, find the marginal posterior distributions h(β j | y), j = 0, 1, 2 and
plot the posterior predictive probability p(yn+1 = 1 | y, xn+1 ) as a function of
xn+1 .
Start by introducing latent scores zi to replace (6.14) by
yi = I(zi > 0) and zi ∼ N(xi′ β, 1). (6.15)
Here, I(A) is the indicator function of event A. Then,
a. Show that (6.15) is equivalent to (6.14).
b. Find the conditional posterior h(zi | β, y) and h(β | z, y).
c. Propose a Gibbs sampling scheme to simulate from the posterior distri-
bution h(β, z | y).
d. Plot a histogram of simulated β values as an estimate of h(β j | y).
e. Show that E{h(β j | y, z)} = h(β j | y). With respect to which distribution is
the expectation?
f. Use (e) to produce estimates of h(β j | y).
g. Argue why the estimate in (f) is “better” than (d). How do you formalize
“better”?
h. Plot P(yn+1 = 1 | xn+1 , y).
Fix xn+1,2 = 100 and plot the posterior predictive probability P(yn+1 = 1 |
xn+1 , y) for a grid of values for xn+1,1 , say 30 < xn+1,1 < 80.
6.8 Probit regression. Refer to Example 4.2 in Section 4.2. Use Gibbs sampling
with data augmentation to implement inference in this application, using
latent variables similar to Problem 6.7. Plot h(a | y) and h(b | y).
6.9 Mixture model. Consider the following mixture of normal models for an
unknown density g(x):
X
J
g(x) = w j N(µ j , σ2 ), (6.16)
j=1

with a Dirichlet prior on the weights

(w1 , . . . , w J ) ∼ D(α, α, . . . , α),
 Problems 123

i.e., h(w) ∝ Π Jj=1 wα−1

j and independent normal priors for the normal loca-
tions,
µ j ∼ N(0, τ),
j = 1, . . . , J, independently. Here α and τ are fixed hyperparameters. Use,
for example, α = τ = 1. Complete the prior with a gamma prior on the
precision, (1/σ2 ) ∼ Ga(a, b).
We use the mixture of normal priors for a density estimation problem, i.e.,
we assume a sampling model
xi ∼ g(xi ), i = 1, . . . , n, i.i.d.
with prior (6.16) for g(·).
a. Joint posterior. Let µ = (µ1 , . . . , µ J ), w = (w1 , . . . , w J ), and x = (x1 , . . . , xn ).
Find the joint posterior distribution h(µ, w, σ | x) (up to a normalizing
constant is fine).
b. Hierarchical model. Rewrite (6.16) as a hierarchical model using indica-
tors si ∈ {1, . . . , J}:
f (xi | si = j, µ, w, σ2 ) = . . . and P(si = j | µ, w, σ2 ) = . . . (6.17)
What are the appropriate probabilities to substitute in . . .?
c. Gibbs sampling. Using the hierarchical model (6.17), define a Gibbs sam-
pling scheme to simulate from the posterior distribution h(w, µ, s, σ | x).
State the list of complete conditional posterior distributions that are sam-
pled in each iteration of the Gibbs sampler.
d. Metropolis–Hastings. Now consider posterior simulation without using
the hierarchical model extension, i.e., using the original posterior distri-
bution h(µ, w | x) without the model augmentation with the latent indica-
tors si .
Define an MCMC scheme with the following transition probabilities:
• For each j, j = 1, . . . , J, update µ j by . . .
• Update w by . . .
• Update σ2 .
e. Data. Use the dataset galaxy in R. You can get it as the data galaxies
in the package MASS.
Estimate g(x) using a mixture of J = 5 normals using the Gibbs part (c).
Show:
1. convergence diagnostic to judge practical convergence. You can use an
implementation from the R package coda or boa (see Section 9.6 for
more discussion of convergence diagnostics);
2. estimated g(x) on a grid (use, 0, 0.01, 0.02, . . . , 0.99, 1.0);
124 Markov Chain Monte Carlo Methods

0.20
0.15
0.10
0.05
0.00

10 15 20 25 30 35
VELOCITY

Figure 6.3 Galaxy data. The histogram shows the data points.
The curve shows a kernel density estimate.

3. trajectory plot of ln h(w, µ, s, σ | x) against iteration (up to a constant

is okay);
4. trajectory plot of σ against iteration.
f. Do the same using the Metropolis–Hastings of (d). Estimate g(x) using a
mixture of J = 5 normals using the M-H.

6.10 This problem describes a real inference problem that arises in bioinformatics
research. The probability model is more complex than other examples, but
does not introduce any serious difficulties. 19
Likelihood (sampling model). Let xi , i = 1, . . . , n, denote a set of multinomial
random variables, with outcomes xi ∈ {1, . . . , N}. Let y = (y1 , . . . , yN ) denote
a contingency table summarizing the multinomial experiment, i.e., y j is the
frequency of outcome j. Let π j = P(xi = j) denote the unknown probability
P
of observing outcome j, π j = 1.0. We write
y ∼ MN−1 (n; π1 , . . . , πN ).
Prior. Due to the nature of the experiment we believe a priori that some of
the π j are much larger than others. We refer to this subset of outcomes with
much larger probability as the “prevalent” outcomes A1 , and to the set of not
so likely outcomes as “rare” outcomes A0 . We will formally define A0 and
A1 in the following.
Parametrization. To define a prior probability model for θ = (π1 , . . . , πN ) it
is convenient to introduce latent indicators z j , and a change of variables for
19
The problem is a stylized form of the analysis for SAGE data discussed by Morris et al
(2003).
 Problems 125

the π j :


π∗ q j
 if z j = 1
πj = 
 (6.18)
(1 − π∗ ) r j if z j = 0,
P P
with q j = 1 and r j = 1 (let q j = 0 if z j = 0, and r j = 0 if z j = 1). In
words, the latent variable z j is an indicator for outcome j being a prevalent
outcome, i.e., A1 = { j : z j = 1} and A0 = { j : z j = 0}. The probability
of observing some prevalent outcome is π∗ , with π∗ close to 1.0; q j is the
probability of outcome j given that we observe a prevalent outcome, and r j
is the probability of j given a rare outcome. For later reference we define
M1 = #A1 and M0 = N − M1
as the number of prevalent and rare outcomes, respectively.
Prior probability model h(z, π∗ , q, r). We assume
P(z j = 1) = ρ, (6.19)
a beta prior on the total probability of prevalent outcomes:
π∗ ∼ Be(a∗ , b∗ ), (6.20)
and a Dirichlet prior for the partitioning of π∗ into the cell probabilities π j ,
j ∈ A1 . Let q̃h , h = 1, . . . , M1 denote the non zero weights q j :
(q̃1 , . . . , q̃ M1 ) ∼ D M1 −1 (a1 , . . . , a1 ). (6.21)
The use of equal Dirichlet parameters a1 reflects the symmetric nature of
prior beliefs about (q1 , . . . , q M1 ). Similarly for r j , j ∈ A0 :
(r̃1 , . . . , r̃ M0 ) ∼ D M0 −1 (a0 , . . . , a0 ). (6.22)

Hyperparameters. The hyperparameters ρ, a∗ , b∗ , a1 , a0 are fixed. Use, for

example,
ρ = 0.1, a∗ = 9, b∗ = 1, a1 = 0.1, and a0 = 10.
Data. posted on the book homepage20 as sage.dta. The file gives the ob-
served values xi ∈ {0, N}, i = 1, . . . , n, with n = 700 and N = 77.
a. Joint probability model. Write out the joint probability model
p(π⋆ , q1 , r1 , . . . , qN , rN , z1 , . . . , zN , y1 , . . . , yN ).
20
sites.google.com/view/computationalbayes/home
126 Markov Chain Monte Carlo Methods

b. Graphical model (optional).21 Show a graphical model representation of

the probability model. Use circles for each random variable, and connect
any two r.v.’s that are not conditionally independent given all other vari-
ables.
c. Conditional posterior distributions. Find the complete conditional poste-
rior distributions h(z j | . . .), h(π∗ | . . .), h(q | . . .), and h(r | . . .).
Here . . . denotes “all other parameters and the data y.”
d. Marginalizing with respect to q and r. Find the posterior distribution p(z |
π∗ , y), marginalizing with respect to q and r.
e. MCMC I. Consider a Gibbs sampling scheme based on sampling from the
complete conditional posterior distributions found in (c), i.e., an MCMC
scheme with steps: zi ∼ p(zi | . . .); π∗ ∼ h(π∗ | . . .); q ∼ h(q | . . .);
r ∼ h(r | . . .).
Show that MCMC I violates irreducibility (of course, do not implement
MCMC I – it would not work).
f. MCMC II. Implement a Gibbs sampling scheme based on sampling from
the conditional posterior distributions found in part (d): zi ∼ p(zi | z−i , π∗ , y),
i = 1, . . . , n; q ∼ h(q | . . .); r ∼ h(r | . . .); and π∗ ∼ h(π∗ | . . .). Show
trajectories of simulated values π∗(t) against iteration t; a boxplot of the
(marginal) posterior distributions h(π j | y) (use one figure with multiple
boxplots, e.g., using the R command boxplot(.)); a plot of the poste-
rior means E(π j | y) against the maximum likelihood estimates π̂ j = y j /n.
In the latter include the 45-degree line and discuss the shrinkage pattern
that you (should) see.
g. MCMC III. Consider the following MCMC scheme. We describe the MCMC
by constructive definition of the transition function, i.e., h(θt+1 | θt ):
1. Metropolis–Hastings step to change zi , q, r, i = 1, . . . , N: Generate a
proposal (z′i , q′ , r′ ) using p(z′i = 1) = 0.5, q′ ∼ h(q | z′ , . . . , y) and
r′ ∼ h(r | z′ , . . . , y), where z′ is the currently imputed z, with zi replaced
by z′i , i.e., z′ = (z1 , . . . , zi−1 , z′i , zi+1 , . . . , zN ).
Compute an appropriate acceptance probability α, and set (zi , q, r) ≡
(z′i , q′ , r′ ) with probability α and keep (zi , q, r) unchanged otherwise.
2. π∗ ∼ h(π∗ | . . .)
Find the correct expression for the acceptance probability α (in step 1)
(no need to implement MCMC III).
6.11 Normal linear regression (see also Problem 3.6). Consider a normal linear
regression model yi = x′i β + ǫi , i = 1, . . . , n, with ǫi ∼ N(0, σ2 ), indepen-
dently. Here, yi ∈ R is the response and xi = (xi1 , . . . , xip ) ∈ R p is a covariate
vector. Letting X denote the (n × p) design matrix with xi in the ith row and
21
This was not discussed in the text.
 Problems 127

ǫ = (ǫ1 , . . . , ǫn ), we can alternatively write the model as

y = X β + ǫ. (6.23)
We complete the model with a prior. Let τ2 = 1/σ2 denote the residual
precision:
h(τ2 ) = Ga(ν0 /2, ν0 σ20 /2) and h(β ) = N(β0 , Σ0 ), (6.24)
independently. That is, we assume an informative prior for β (for example,
this could be based on historical data from a related earlier study).
Here, X ∼ Ga(a, b) denotes a gamma distribution with mean E(X) = a/b.
a. State the joint posterior h(β , τ2 | y) (up to a normalizing constant is okay).
No need to simplify.
b. Find the conditional distribution h(β | τ2 , y). Use notation β b for the least
squares solution. Recognize the distribution as a well-known parametric
family.
c. Find h(τ2 | β , y). You can use notation RSS(β ) = (y − X β )′ (y − X β ).
Recognize the distribution as a well-known parametric family.
d. Propose a Gibbs sampling posterior MCMC scheme based on the condi-
tionals from (b) to generate a Monte Carlo sample,
(β (m) , τ2(m) ) ∼ h(β , τ2 | y), m = 1, . . . , M.
6.12 Hamiltonian Monte Carlo. Refer to Example 6.3 in Section 6.5.1.
a. Posterior. Plot the posterior distribution h(α, β | y). Use a contour plot
(or heatplot, or any other format that allows you to add the points for the
simulations from part (b)).
b. Hamiltonian Monte Carlo. Let θ = (α, β). Propose a Hamiltonian Monte
Carlo algorithm to simulate a Monte Carlo sample:
θm ∼ h(θ | y),
m = 1, . . . , M (the θm need not be independent, as in any MCMC). Use
M = 100.
1. Describe the algorithm by stating all transition probabilities.
2. Implement the algorithm.
3. Plot θm by adding them to the plot from part (a).
c. Metropolis–Hastings. Implement a Metropolis–Hastings MCMC to gen-
erate θm ∼ h(θ | y), m = 1, . . . , M.
1. Describe the algorithm by stating all transition probabilities.
2. Plot θm by adding them to the plot from part (a).
128 Markov Chain Monte Carlo Methods

6.13 Approximate Bayesian computation (ABC). Consider a posterior h(θ | y) ∝

h(θ) · f (y | θ) in a statistical inference problem with data y and unknown
parameters θ. We set up the following MCMC simulation. Let q(θ̃ | θ) denote
a proposal distribution, for example q(θ̃ | θ) = N(θ, cI), a multivariate normal
centered at θ.
Start with θ0 . For i = 1, . . . , M, simulate the following transition function.
1. θ̃ ∼ q(θ̃ | θi );
2. z̃ ∼ f (z̃ | θ̃), using the sampling distribution under θ̃.
3. Let



θ̃ w. pr. a = min{1, A(θi , θ̃)}
θi+1 = 
 (6.25)
θi w. pr. 1 − a,

with
h(θ̃) I {d(z̃, y) < ǫ} q(θi | θ̃)
A(θi , θ̃) = ,
h(θi ) 1 q(θ̃ | θi )
where d(z, y) is some distance measure for two (hypothetical or actually ob-
served) data sets z and y, for example, d(z, y) = ||z − y||2 .
Find the stationary distribution π(θ) for this Markov chain.
Hint: Introduce a variable zi , initializing with z0 ∼ f (z0 | θ0 ), and then mod-
ify (6.25) to a proposal distribution for an augmented state vector (θi , zi ),
find the stationary distribution π(θ, z) for the Markov chain with this aug-
mented state vector. Finally, argue that the desired π(θ) is the marginal under
π(θ, z).22
6.14 Thall et al (2003) consider inference for a phase I oncology trial for the
combination of two cytotoxic agents. Here, we focus on estimating the dose–
response surface π(x; θ) = P(y = 1 | x = (x1 , x2 ), θ) for the probability of
toxicity (y = 1) as a function of the doses of the two agents (x1 and x2 ). We
assume standardized doses, 0 ≤ x j ≤ 1, and let yi ∈ {0, 1} denote an indicator
of the i-th patient recording toxicity after being treated with combination
therapy with doses xi1 and xi2 of the two agents. The response surface is
indexed by unknown parameters θ = (a1 , b1 , a2 , b2 , a3 , b3 ),
 b3
a1 x1b1 + a2 x2b2 + a3 x1b1 x2b2
π(x, θ) =  b3 . (6.26)
1 + a1 x1b1 + a2 x2b2 + a3 x1b1 x2b2
The model is chosen to allow easy incorporation of information about single-
agent toxicities. For x2 = 0 the model reduces to the single-agent dose–
toxicity curve π((x1 , 0), θ) ≡ π1 (x1 , θ) and similarly for π2 . The parameters
(a3 , b3 ) characterize the two-agent interactions.
22
This is one of the variations of ABC. See Marin et al (2012) for a review.
 Problems 129

We complete the model with independent gamma priors:

a j ∼ Ga(α1 j , α2 j ) and b j ∼ Ga(β1 j , β2 j ),
j = 1, 2 as an informative prior with hyperparameters (α1 j , α2 j , β1 j , β2 j ),
j = 1, 2 are chosen to match the known single-agent toxicity curves as
closely as possible with prior means E(a1 , b1 , a2 , b2 ) = (0.4286, 7.6494,
0.4286, 7.8019) and marginal variances (0.1054, 5.7145, 0.0791, 3.9933). These
moments were carefully elicited by Thall et al (2003). For the interaction
parameters, we use a log-normal prior ln a3 ∼ N(µa3 , σ2a3 ) and ln b3 ∼
N(µb3 , σ62b3 ). As default choices we propose to use µa3 = µb3 = 0.25 and
σ2a3 = σ2b3 = 3.
Let Yn = (xi , yi ; i = 1, . . . , n) denote observed indicators for toxicity yi for n
patients treated at dose combinations xi = (xi1 , xi2 ), i = 1, . . . , n.
a. Find hyperprior parameters to match the described prior information.
b. Propose a Hamiltonian Monte Carlo scheme for posterior simulation.
Find ∂ ln h(θ | Yn )/∂θ j and describe the transition probabilities.
Hint: In the derivation of the gradient it is useful to use notation like
b1 b2
πi ≡ π(xi , θ), πi = (1 − πi )., and ȳi = 1 − yi . Use xi3 = (xi1 xi2 ) (keeping in
mind that the definition of xi3 involves b1 and b2 ).
c. Using the data set CTX.dta from the book’s homepage23 implement pos-
terior simulation. Plot the estimated mean toxicity surface π̄(x) = E{π(x; θ)}
over a grid for (x1 , x2 ) ∈ [0, 1]2 (the expectation is with respect to h(θ |
x)).
6.15 Getting it all right diagnostic (Geweke, 2004). Let f (y | θ) and h(θ) denote
sampling model for data y and a prior for parameters θ in a Bayesian infer-
ence problem. Let h(θ | y) ∝ h(θ) f (y | θ) denote the posterior distribution
and let pθ,y (θ, y) denote the joint probability model on (θ, y).
Almost all MCMC posterior simulation schemes can be described as gen-
erating a sequence θRm using some transition kernel q(θm | θm−1 , y). That is,
P(θm ∈ A | θm−1 ) = A q(θm | θm−1 , y)dθm (for fixed y – we only indicate y in
the kernel in anticipation of the next construction). The transition kernel q(·)
is constructed such that it has unique invariant distribution π(θ) ≡ h(θ | y).
Consider the following Markov chain in x = (θ, y). Initialize with θ0 ∼ h(θ)
and y0 ∼ f (y | θ0 ). Set m = 0, and iterate over the following steps, m =
1, . . . , M:
1. Generate θm ∼ q(θm | θm−1 , ym−1 ).
2. Generate ym ∼ f (ym | θm ).
a. Find the invariant distribution for this Markov chain.
b. Let P(n) (x0 , A) = P(xn ∈ A | x0 ) denote the n-step transition probability
23
sites.google.com/view/computationalbayes/home
130 Markov Chain Monte Carlo Methods

under this chain. Show that P(n) → pθ,y in total variation norm. That is,
lim ||P(n) (x0 , ·) − pθ,y || = 0
n→∞

in total variation norm, for all x0 . You may assume that q(θm | θm−1 , ym−1 ) >
0 for all θm ∈ Θ in the parameter space Θ (and all θm−1 , ym−1 ); and simi-
larly f (y | θ) > 0 for all y ∈ X in the sample space X and θ ∈ Θ.
Use at most three sentences.
c. Can you suggest an alternative way of generating an MC sample xn ∼
pθ,y (θ, y)? 24
Hint: There is a very easy answer.

24
Geweke (2004) uses ergodic Monte Carlo averages under (a) and under (b) to construct
an interesting diagnostic.
 7

Model Selection and Trans-dimensional

MCMC

1
Earlier (Sections 4.2.2. and 4.2.3) we discussed how the evaluation of the
marginal likelihood function (or prior predictive distribution) for model se-
lection can be implemented by means of importance sampling. Since the
1990s, various methods have been proposed in the literature to implement
such schemes.2 This works for problems with moderate dimensional pa-
rameter vectors, but breaks down in higher dimensional problems. More
complex problems usually require the use of Markov chain Monte Carlo
(MCMC) methods to generate posterior Monte Carlo samples (or, to be
more precise, approximate posterior samples), as discussed in previous
chapters. Using such posterior Monte Carlo samples, one can then approx-
imately evaluate the marginal likelihood function. It is convenient to dis-
tinguish two cases. The first case is when the simulated values are gener-
ated from the parameter space for each model separately. The other case
is when the simulated values are generated from the model space, that is,
model indicators (separately or jointly with the parameter space). A popu-
lar algorithm to implement the latter is reversible jump MCMC (RJ).
In the following discussion, M j (or for short, j, when the context clarifies
the use) will denote competing models, and θ j will denote parameters that
index the sampling distribution under model M j , and we will use notation
f (y | θ j , M j ) or for short f j (y | θ j ) for the sampling model under model M j ,
h(θ j | M j ), or h j (θ j ) for the prior under model M j , p(x | M j ), or p j (x) for
the marginal under model M j , and h(M j ) or just h( j) for the prior model
probability.
1
This material will be published by Cambridge University Press as Computational
Bayesian Statistics, by M.A. Amaral Turkman, C.D. Paulino, and P. Müller
(https://tinyurl.com/CompBayes). This pre-publication version is free to view
and download for personal use only. Not for re-distribution, re-sale, or use in derivative
works. c Maria Antónia Amaral Turkman, Carlos Daniel Paulino & Peter Müller, 2019.
2
See Chen et al (2000) and Andrieu et al (2004) for useful reviews of these methods.
132 Model Selection and Trans-dimensional MCMC

7.1 MC Simulation over the Parameter Space

Besides the basic Monte Carlo methods that we already discussed in Sec-
tions 4.2.2. and 4.2.3, methods that correspond to the first of the above
cases include those by Chib (1995) and Chib and Jeliazkov (2001). The
latter methods approximate the posterior distribution of the parameters in
each model, h(θ j | y, M j ), by Gibbs and Metropolis–Hastings samplers,
respectively. Importantly, the evaluation includes normalization constants
which then allow the evaluation of the marginal distribution f (y | M j ) via
Bayes’ theorem (written as candidate’s formula, as explained in the follow-
ing).
We describe the approach for the evaluation of the marginal likelihood
function f (x) for one model, dropping for the moment the explicit condi-
tioning on the model indicator M j . The method of Chib (1995) uses the
factorization of the joint posterior distribution of a parameter vector θ into
factors corresponding to b ≥ 2 subvectors θ(i) , i = 1, . . . , b, assuming that
the complete conditional posterior distributions for all subvectors are avail-
able in closed form. The Gibbs sampler is carried out b−1 times to estimate
the first b − 1 factors of

h(θ(1) , θ(2) , . . . , θ(b) | y) = h(θ(1) | y) h(θ(2) | θ(1) , y) × . . .

× h(θ(b) | θ(b−1) , . . . , θ(1) , y) ≡ Πbi=1 h(θ(i) | θ(1) , . . . , θ(i−1) , y). (7.1)
Details of this approach are explained in Chib (1995), including variations
for the case when complete conditionals are not available for all blocks. We
only briefly summarize the approach. Let θ−(1) denote R the θ vector with the
∗ ∗
subvector θ(1) removed. First, note that h(θ(1) | y) = h(θ(1) | θ−(1) , y)h(θ−(1) |
∗
y)dθ−(1) allows an approximate evaluation of h(θ(1) | y) as a Monte Carlo
∗
average of h(θ(1) | θ−(1) , y) over a posterior Monte Carlo sample {θ(1) , θ−(1) }
(discarding the θ(1) values in the Monte Carlo sample without using them).
Assume that the posterior Monte Carlo sample is generated by MCMC
simulation. Assuming b > 2, to evaluate reduced conditional ordinates
∗ ∗
like h(θ(2) | θ(1) , y) continue running the same MCMC, but now keeping
∗
θ(1) fixed at θ(1) . In a Gibbs sampler implementation this is easily done by
skipping one transition probability, without any additional programming.
∗
Using the resulting posterior sample {θ(1) , θ(2) , θ−(1,2) }, we then approximate
∗ ∗
h(θ(2) | θ(1) , y) as the Monte Carlo average over the posterior sample of
∗ ∗
h(θ(2) | θ(1) , θ−(1,2) , y). A similar scheme allows the evaluation of all reduced
conditional ordinates in (7.1).
Once the posterior density for θ is evaluated at a given point, say, θ∗ ,
(e.g., some value with high posterior plausibility), the marginal distribu-
 7.2 MC Simulation over the Model Space 133

tion can be estimated via an inversion of Bayes’ theorem (also known as

candidate’s formula). Using, for computational reasons, log transformed
densities, we have
ln p̂(y) = ln f (y | θ∗ ) + ln h(θ∗ ) − ln ĥ(θ∗ | y),
where ĥ(θ∗ | y) = Πb−1 ∗ ∗ ∗ ∗ ∗
i=1 ĥ(θ(i) | θ(1) , . . . , θ(i−1) , y) × h(θ(b) | θ(i) , i , b, y).
The evaluation of ĥ(θ∗ | y) by this method requires closed-form expres-
sions for all complete conditional distributions. In general this might not
be available. Chib and Jeliazkov (2001) propose an alternative approach
for such cases, using instead a Metropolis–Hastings proposal distribution
q(θ̃ | θ) with closed form expression.

7.2 MC Simulation over the Model Space

Methods corresponding to the second case mentioned at the beginning of
this chapter include, in particular, many popular methods for variable se-
lection. Consider variable selection in a normal linear regression with re-
sponse variable Y and p explanatory variables x j . There are 2 p models. To
make variable selection feasible we proceed with a preliminary selection
of a smaller class of appropriate models in a first step. In a second step
we then select the members in this class that best fulfill the criteria of the
particular inference problem or we proceed using all models by means of
a weighting scheme for the particular inference of interest. Naturally there
are various criteria and methods that can be used for such constructions.

The SSVS Method

Consider variable selection in a normal linear regression model with a
P
linear predictor η = pj=1 β j x j . George and McCulloch (1993) propose a
method known as SSVS (stochastic search variable selection), based on a
hierarchical prior structure with the first level defining for each regression
coefficient a mixture of two zero mean normals. The variances are fixed
(on the basis of some preliminary fit) such that one of the two terms in the
mixture is tightly concentrated around 0 and the other term is diffuse. More
specifically, introducing a binary parameter υ j corresponding to each β j



h1 (β j ) ≡ N(0, b2j ) if υ j = 0
βj | υj ∼ 
 (7.2)
h2 (β j ) ≡ N(0, a2 b2 ) if υ j = 1,
j j

where b j and a j are fixed values of small and large magnitude, respectively.
Assuming conditional independence the implied prior distribution for β =
(β j , j = 1, . . . , p) in the first level of the hierarchical model is a multivariate
134 Model Selection and Trans-dimensional MCMC
h i
normal β | υ ∼ N p (0, Bυ ), with Bυ = diag (1 − υ j )b2j + υ j a2j b2j being a
diagonal matrix.3
At the second level of the hierarchy, the SSVS model assumes an inverse
gamma prior distribution for σ2 with fixed hyperparameters. More impor-
tantly, for the indicators υ j the model assumes υ j ∼ Ber(w j ), implying the
mixture prior
h(β j | w j ) = (1 − w j )h1 (β j ) + w j h2 (β j ),
where w j = P(υ j = 1) is the prior probability for a non-zero estimate
for β j , i.e., for including x j in the model. This way, each of the 2 p mod-
els is identified by a vector of latent indicators υ = (υ1 , . . . , υ p ). The prior
model probabilities are written as the product of p Bernoulli probabilities,
Ber(w j ), with {w j } representing the prior beliefs about including each co-
variate.
Once the hierarchical model is completely specified, SSVS proceeds by


using a Gibbs sampler 4 to generate a sequence β(t) , σ(t) , υ(t) , t = 1, . . . , T,
that approximates the joint posterior distribution, with mainly the subse-
quence υ(t) , t = 1, . . . , T , being of interest. Inspection of this sequence
allows in particular the identification of a subset of covariates with the
highest inclusion probabilities, which indicate the most promising models
in the light of the data and the assumed prior. Note that any specific value
of υ might not occur with high frequency in the sample, simply due to the
limited size of the sample (T can be much less than 2 p when p is large). In
summary, the nature of SSVS is mainly an exploratory tool to restrict the
class of models for further consideration. Only when a small number of
covariates makes it practicable (say, p < 7), can a similar process be used
to assess the posterior distribution for υ, rather than just restricting the set
of models under consideration.
Several approaches that are in some way related have appeared in the
literature. For example, Kuo and Mallick (1998) use a linear predictor of
P
the form η = pj=1 υ j β j x j , making the variable selection explicit, and a
prior distribution for β that is independent of υ. Dellaportas et al (2002)
keep a dependent prior for β and υ, but use the latter type of linear predictor.
The earlier-described hierarchical prior in the SSVS is not conjugate for
normal linear models (given any subset of included covariates). A conju-
3
George and McCulloch (1993, 1997) consider more general multivariate normal prior
distributions, including correlations in Bυ and dependence on a sampling variance σ2 .
4
See George and McCulloch (1993, 1997) for recommendations about the choice of the
tuning parameters, the complete conditional posterior distributions, and extension to
other families such as generalized linear models.
 7.2 MC Simulation over the Model Space 135

gate prior is achieved by scaling Bυ with σ2 Hoff(2009: ch. 9). With this
change in the prior p(β | υ, σ2 ), it becomes possible to evaluate the ker-
nel of the marginal posterior distribution for υ, h∗ (υ | y) = c−1 h(υ | y)
(George and McCulloch, 1997), from which one can then directly gener-
ate a Markov chain in υ, without involving (β, σ2 ). For example, one could
implement a Gibbs sampler using the complete conditional distributions
h(υ j | υ− j , y).
Relying on a predefined subset of models, for example the subset MG of
models that occur in a preliminary sample υ(k) , k = 1, . . . , K, one can then
obtain an estimate of the posterior probability for each model via h∗ using
ĥ(MG | y) ∗
ĥ(υ | y) = ĉ h∗ (υ | y) ≡ h (υ | y), (7.3)
h∗ (MG | y)
where
X 1 X
K
h∗ (MG | y) = h∗ (υ | y), ĥ(MG | y) = I M (υ(k) ).
υ∈MG
K k=1 G

In (7.3) the histogram estimate b

h(Mg | y) is used to estimate c by match-
ing it with the unnormalized h∗ (MG | y). By explicitly using information
from the particular model specification, such estimates are naturally more
precise than an estimate using the relative frequency of models in a Gibbs
sampler Monte Carlo sample, obtained from the original form of the SSVS.
The difference between these two evaluations can be more noticeable in
problems with large p.

MC3 Method
Generating a Markov chain with a stationary distribution given by the ker-
nel h∗ (υ | y) can be done by an M-H algorithm with some proposal dis-
tribution5 q(υ̃ | υ(t) ) for any υ(t) . Using the special case of a Metropolis
algorithm with symmetric proposal q, the acceptance ratio for a proposal υ̃
generated from q simplifies to R(υ(t) , υ̃) = h∗ (υ̃ | y)/h∗ (υ(t) | y). A simple
example of this class of Metropolis algorithms uses a proposal distribution
that is only non-zero for υ̃ that differs from υ(t) in only one component,
q(υ̃ | υ(t) ) = 1/p, where p is the dimension of the least parsimonious model
corresponding to υ = 1 p with all 1s. This algorithm was proposed for mod-
els with discrete data by Madigan and York (1995) under the name MC3 ,
(Markov chain Monte Carlo model composition). Raftery et al (1997) used
5
Under suitable conditions, this algorithm can be implemented very computation
efficiently, similar to what happens in a Gibbs sampler (George and McCulloch, 1997).
136 Model Selection and Trans-dimensional MCMC

MC3 Bayesian model averaging in multiple regression with a conjugate

(normal-gamma) prior on (β, σ2 ).

Bayesian Lasso, Horseshoe, and Dirichlet–Laplace Model

The SSVS model R (7.2) can be characterized as a scale mixture of nor-
mals h(β j ) = N(0, b2j γ j )dG(γ j ) with respect to a discrete mixing measure
G(γ j ) = w j δ1 + (1 − w j )δa2j . Here, δ x denotes a point mass at x. The mixture
can be alternatively written as a hierarchical model:
h(β j | γ j ) = N(0, b2j γ j ) and γ j ∼ G(γ j ).
Several other approaches, including the Bayesian lasso (Park and Casella,
2008), the horseshoe (Carvalho et al, 2010) and the Dirichlet–Laplace model
(Bhattacharya et al, 2015) use similar scale mixture of normal construc-
tions. The Bayesian lasso uses
h(β j | σ2 , γ j ) = N(0, σ2 γ j ) and γ j ∼ Exp(λ2 /2).
λ −λ|β j |/σ
Marginalizing with respect to γ j the implied prior h(β j | σ2 ) = 2σ e
is the double exponential distribution. The log double exponential prior
density introduces the L1 penalty that is characteristic for the popular lasso
method (Tibshirani, 1996). Thus, the Bayesian lasso gives an interpretation
of the lasso as the maximum a posteriori estimate in a corresponding Bayes
model. The horseshoe prior uses
h(β j | γ j ) = N(0, γ2j ) and γ j | τ ∼ C+ (0, τ),
completed with τ | σ ∼ C+ (0, σ), where C+ (0, s) denotes a Cauchy dis-
tribution with scale s and truncated to the positive half-line. The name
“horseshoe” derives from the implied prior on the shrinkage coefficient
in writing E(β j | y) as shrinking the maximum likelihood estimator to zero.
In a stylized setup the prior on that shrinkage coefficient is a Be(1/2, 1/2)
distribution which takes the shape of a horseshoe. Finally, the Dirichlet–
Laplace prior has
!
1
h(β j | γ j , ψ j , τ) = N(0, ψ j γ2j τ2 ), ψ j ∼ Exp , (γ1 , . . . , γ p ) ∼ D(a)
2
with a = (a, . . . , a) and τ ∼ Ga(pa, 21 ). Here, D(a) denotes a symmetric
Dirichlet distribution with all parameters equal to a. The a priori indepen-
dent scaling coefficients ψ j imply marginally double exponential distribu-
tions, like in the Bayesian lasso. The main feature of the Dirichlet–Laplace
prior are the scaling coefficients γ j with the dependent Dirichlet prior. The
 7.2 MC Simulation over the Model Space 137

Dirichlet–Laplace prior is most suitable for a massive sparse signal, that

is, a context with p → ∞, but only a few non-zero β j . Using a = p−(1+β)
the Dirichlet prior introduces a similar mechanism as the horseshoe prior.
This is because the Dirichlet distribution with total mass pa < 1 piles up
probability mass in the corners of the simplex, leading to effective variable
selection.

MC Simulation over the Model Space: Beyond Variable Selection

When the class of models Mυ (or υ, for short) under consideration is such
that it is possible to evaluate p(y | υ) (typically obtained by eliminating
parameters via appropriate integration), then it is possible to work in the
context of the model space only, requiring only a prior distribution h(υ).
This is the case, for example, under SSVS and MC3 for variable selection
under a conjugate hierarchical prior.
Using MCMC allows, then, to generate a Markov chain υ(t) , t = 1, 2, . . .
which converges to the posterior distribution h(υ | y) ≡ P(Mυ | y), the
posterior probability for model Mυ . One could, for example, adopt an M-H
algorithm that generates in iteration t +1 a proposal υ̃ from q(· | υ(t) ), which
is accepted with probability
 p(y | υ̃) h(υ̃) q(υ(t) | υ̃) 
α(υ(t) , υ̃) = min 1, × .
p(y | υ(t) ) h(υ(t) ) q(υ̃ | υ(t) )
In the case of rejection, we keep υ(t+1) = υ(t) . A simple estimate of the pos-
terior probability of each model is then given by the inclusion probability
in the Monte Carlo sample that is generated by this chain. On the basis of
these posterior probabilities, one can estimate Bayes factors as a ratio of
the posterior odds versus the prior odds between any pair of models.
The efficiency of the algorithm depends not only on the appropriate pro-
posal distribution, but also on a fast evaluation of the marginal likelihood.
Note that in the case of analytically available exact marginal likelihood,
one can obtain alternative posterior estimates in the subclass B s of models
that appear in the Monte Carlo sample by way of the standardization
p(y | υ) h(υ)
ĥ(υ | y) = P ,
υ∈B s p(y | υ) h(υ)

with the corresponding advantages of its use in Bayesian model averaging.

138 Model Selection and Trans-dimensional MCMC

7.3 MC Simulation over Model and Parameter Space

Consider now the general model selection problem, beyond the specific ex-
ample of variable selection. Let M = {M j , j ∈ J} denote the set of models
under consideration. For each M j let θ j ∈ Θ j denote a parameter vector
of dimension p j . The ijoint model and parameter space is then defined as
S h
N = j∈J {M j } × Θ j and the aim is to develop a simulation method for
the target distribution being the posterior distribution of the pairs (model,
parameters):
h(M j , θ j | y) ∝ f (y | M j , θ j ) h(θ j | M j ) h(M j ) ≡ h∗ (M j , θ j | y), (7.4)
P R
where k∈J f (y | Mk , θk ) h(θk | Mk ) h(Mk )dθk is the reciprocal of the
normalization constant, and h∗ is an un-normalized version of the poste-
rior distribution. The aim is then to simulate from (7.4) using computer
simulation.

The Pseudo-Prior Method of Carlin–Chib

Carlin and Chib (1995) achieve the desired simulation by setting up a Gibbs
sampler over a larger space, namely the product space M × Πk∈J Θk . That
is, they set up a super parameter vector ω = (M, θ1 , . . . , θ J ) that includes in
addition to the parameters for the chosen model M = M j also parameters
for all other models k , j. The vectors θk , k , j remain hypothetical in
the sense that they are not used in the evaluation of the likelihood. How-
ever, the prior probability model is defined over the entire super parameter
vector, including what Carlin and Chib refer to as “pseudo-priors” for the
parameters θk , k , j. See the following for details. The output is a Markov
chain of states (M, θ1 , . . . , θ J ), from which we keep for each j the subsam-
ple (M = M j , θ j ).
In this description we assumed that all models in M use different pa-
rameter vectors, which are assumed a priori independent, given the model,
and are concatenated to a composite parameter vector θ = (θ1 , . . . , θ J ),
such that for each model f (y | M j , θ) = f (y | M j , θ j ) and h(θ | M j ) = h(θ j |
M j ) × Πk, j h(θk | M j ), with proper prior distributions. The factors h(θk | M j )
are the pseudo-priors. Their role is to allow the construction of transition
probabilities across models M and M̃. See the following.
Upon convergence this Gibbs sampler algorithm generates a sample from
the posterior distribution h(M, θ | y) which implies for each model

h(M j , θ | y) ∝ f (y | M j , θ j ) h(θ j | M j ) Πk, j h(θk | M j ) h(M j )

≡ h∗ (M j , θ | y), (7.5)
 7.3 MC Simulation over Model and Parameter Space 139

implying that the prior distributions h(θk | M j ) for the parameters corre-
sponding to models k , j are not required to specify (M j , θ j ). Further, one
can easily show that the marginal likelihood for each model can be written
as
h i h i
f (y | M j ) = Eθ f (y | M j , θ) | M j = Eθ j f (y | M j , θ j ) | M j ,
from which one can see that for the purpose of model comparison via Bayes
factors the choice of the pseudo-priors is irrelevant.
The pseudo-prior method makes use of the complete conditional distri-
butions for the J + 1 blocks
h∗ (M, θ | y)
∀M ∈ M h(M | θ, y) = P (7.6)
h∗ (Mk , θ | y)
 k∈J


h(θ j | M j , y), M = M j
∀ j ∈ J h(θ j | θ− j , M, y) = 
h(θ | M ), (7.7)
 j k M = M , k , j.
k

The simulation based on these distributions is more efficient if, for each
model M j , one has conjugacy of the likelihood and the respective prior
h(θ j | M j ), and if one chooses pseudo-priors h(θk | M j ), k , j, that are
similar to h(θk | Mk , y). Note that under conjugacy for model M j one can
exactly evaluate the posterior distribution for the respective parameter and
the marginal likelihood f (y | M j ) and, on the basis of the latter, evaluate the
posterior probability of M j , which allows a separate and easy evaluation of
the posterior quantities.
Once a Monte Carlo sample {(M (t) , θ(t) )} from h(M, θ | y) is available,
it can be used to evaluate posterior probabilities for each model h(M j | y)
(as relative sample frequencies of M = M j ) and, therefore, Bayes factors
between any pair of models. Inference related to h(θ j | M j , y) is obtained
from the subsample of θ(t) consisting of the components θ(t) j associated with
model M j .
For details about the implementation, see Carlin and Louis (2009: sec-
tion 4.5.1). Dellaportas et al (2002) propose a modification of the pseudo-
prior method for the special case of variable selection.

A Metropolized Pseudo-Prior MCMC

This approach was proposed by Dellaportas et al (2002) with the aim of
reducing the computational effort of the Carlin and Chib method related
to the generation from (#M − 1) pseudo-priors in each cycle of the Gibbs
sampler (here #M is the number of models under consideration). This is
 140 Model Selection and Trans-dimensional MCMC

achieved by hybridizing the Gibbs sampler by substituting h(M | θ, y) in the

step of the model updating with a proposal distribution that is independent
of the parameters θ. Let q(M j′ | M j ) denote the proposal distribution when
the current model is M j . The “Metropolization” of the model updating step
then creates a sampler that is independent of the current parameters.
In h(Mℓ , θ | y)/h(Mk , θ | y), k , ℓ, all other pseudo-priors cancel out and
only the pseudo-priors h(θk | Mℓ ) and h(θℓ | Mk ) remain. Also, recall that
h∗ (·) denotes the un-normalized product of prior times likelihood. Thus, the
M-H acceptance ratio for a proposals M j −→ M j′ is
h∗ (M j′ , θ j′ | y) h(θ j | M j ) q(M j | M j′ )
R(M j , M j′ ) = ,
h∗ (M j , θ j | y) h(θ j′ | M j ) q(M j′ | M j )
highlighting that in each iteration only one pseudo-prior is needed. Thus,
each cycle of the hybrid MCMC consists of the following three steps:
1. Given a current model M j , generate a value h(θ j | M j , y).
2. Propose a transition to M j′ by generating from q(M j′ | M j ) and generate
a value from h(θ j′ | M j ).
3. Accept the proposal with probability
α(M j , M j′ ) = min{1, R(M j , M j′ )}.
More details about this method, including in particular the choice of pro-
posal distribution, are found in Dellaportas et al (2002).

7.4 Reversible Jump MCMC

Reversible jump MCMC (RJ) is another method to set up simulation from
(7.4). As the perhaps most widely used approach of this type, we discuss
it in more detail. RJ is due to Green (1995) and implements an M-H al-
gorithm to generate from the posterior distribution on the pair (model, pa-
S
rameter), defined over the space j∈J {M j , Θ j }. Without loss of generality,
assume Θ j = R p j . The RJ defines a process that can jump between models
of different dimensions, including a mapping between model-specific pa-
rameters. By trans-dimensional simulation the algorithm generates a sam-
ple from the joint posterior distribution of an indicator M for a model in
S
a given set and a parameter θ which takes values in the set j∈J Θ j such
that θ ∈ Θ j (i.e., θ = θ j ) when M = M j . The set J of model indicators is
allowed to be infinite.
The transition from a current state (M j , θ j ) to ( M̃, θ̃ M̃ ) follows an M-
H transition probability that is implemented with a proposal distribution
 7.4 Reversible Jump MCMC 141

which is composed of two factors related to the jump between models (qm )
followed by a generation of a parameter for the proposed model (q p ), de-
fined as
q( M̃, θ̃ M̃ | M j , θ j ) = qm ( M̃ | M j , θ j ) q p (θ̃ M̃ | M j , θ j , M̃).
Assume M̃ = M j′ . The proposal distribution qm is in many cases selected
to be independent of the parameters under the current model, θ j . That is,
qm ( M̃ | M j , θ j ) = q j j′ is a function of only j, j′ . The distinguishing feature
of the RJ method is in the proposal distribution q p , wich has to accommo-
date the transition from model M j to M̃, including a possible change in
dimension of the parameter vector. Details are introduced in the following.
In short, the reversible jump builds on an M-H algorithm by adding two
important elements. First, we introduce dimension padding by adding aux-
iliary variables. Second, each transition probability can include a determin-
istic transformation. The latter is important to create a well-mixing Markov
chain. For example, consider an RJ for a normal linear regression that al-
lows the use of a line with (M2 ) or without (M1 ) intercept, that is E(yi | xi )
is α + βxi under M2 or βxi under M1 . It is important to adjust the currently
imputed value of the slope β when we propose to add a new intercept α , 0
in the line. That is essentially all that is to be said about the RJ. The rest
are details.
We first introduce the RJ for the simple problem of regression with and
without an intercept. Of course, one would not really use RJ in this prob-
lem. But it provides an easily understood context with all the same details
that are needed in any RJ. Let f (y | α, β, M2 ) and f (y | β, M1 ) denote the
likelihood function under the model with and without intercept, respec-
tively, and let h(α, β | M2 ), h(β | M1 ), and h(M) denote the prior distribu-
tions. In terms of the earlier general notation for model-specific parameter
vectors θ1 = (β) and θ2 = (α, β).
Move up: Assume the current state is ω = (M1 , θ1 ). Propose a new model
M̃ with P( M̃ = 2) = q12 and P( M̃ = 1) = q11 = 1−q12 . If M̃ = M1 , continue
with the usual MCMC step, without change of dimension. Otherwise M̃ =
M2 and:
1. Generate (β̃, α̃) by:
(a) generating an auxiliary variable u ∼ q(u);
(b) using a deterministic mapping (β̃, α̃) = T (β, u).
 142 Model Selection and Trans-dimensional MCMC

2. Accept the proposal (β̃, α̃) with probability Aup = min{ρup , 1} and ratio
h( M̃) h(β̃, α̃ | M̃) f (y | α̃, β̃) q21 ∂T (β, u)
ρup (β, u, β̃, α̃) = .
h(M1 ) h(β̃ | M1 ) f (y | β) q12 q(u) ∂(β, u)
prior ratio × likelihood × proposal × Jacobian

3. With probability Aup , set ω = ( M̃, β̃, α̃) = (M2 , θ2 ).

Move down: Similarly, assume the current state is (M2 , θ2 ). Propose a new
model M̃ with P( M̃ = 2) = q22 and P( M̃ = 1) = q21 = 1 − q22 . If M̃ =
M2 , continue with the usual MCMC step, without change of dimension.
Otherwise:
4. Compute (β̃, u) = T −1 (β, α).
5. Acceptance probability Adown = min{1, ρdown }, with
ρdown = 1/ρup (β̃, u, β, α).
The inclusion of the auxiliary variable in step 1(a) achieves the desired
dimension padding and allows transdimensional MCMC. The use of the
deterministic mapping in step 1(b) allows us to design algorithms with bet-
ter mixing. For example, in the application to a regression with and without
intercept, the slope β should be adjusted when proposing to add an intercept
and vice versa. In general, both moves, up and down, could involve auxil-
iary variables, set up such that the dimensions of current state and proposal,
possibly both augmented by auxiliary variables, match. A general RJ algo-
rithm can include more than two types of transition probabilities. The only
condition is that for the proposal generated by any transition probability
there needs to be a matching reciprocal type of transition probability that
could generate a proposal to come back. If we were to propose, for ex-
ample, an up-move of no return, that is, with no possibility to reverse the
proposal, then formally in ρup the proposal probability in the numerator
would evaluate as zero, leaving acceptance probability Aup = 0. That is,
the algorithm would never accept such a move of no return. It is therefore
usually best to design RJ moves in reciprocal pairs. Examples are the up
and down move, birth and death moves that add or remove a term in a
mixture model, and many more.
A similar trans-dimensional MCMC algorithm could be implemented
for any posterior distribution over variable-dimension parameter spaces. In
practice, the use of a good mapping T (·) is critical to achieve a reasonably
fast-mixing Markov chain. That is, the construction requires a good esti-
mate for proposed parameters θ̃ in the proposed model M̃. In many more
 7.4 Reversible Jump MCMC 143

complex problems, the lack of a good mapping makes the implementation

of RJ impractical, since proposed moves would be almost always rejected.
One class of problems in which RJ has been found to be viable are mixture
models. It is reasonably easy to construct proposals for the parameters of a
larger (smaller) size mixture when proposing moves up (down) (Richard-
son and Green, 1997). See also problem 7.2 at the end of the chapter.
Finally, note the similarities between RJ and the pseudo-prior algorithm.
Consider an RJ transition probability that involves a proposal to move from
a current state vector θ ∈ Θ j to a proposal θ̃ ∈ Θ j+1 . Assume the transition
probability involves an auxiliary variable generated from q(u) and a deter-
ministic mapping T (θ, u). The same elements can be used to construct a
suitable pseudo-prior h(θ j+1 | θ j , M j ). The only formal difference is that the
RJ does not require a finite number of competing models M j , whereas the
pseudo-prior method does.
RJ with General “Up” and “Down” Moves.
For reference we include a description of a general RJ that involves “up”
and “down” moves similar to the earlier stylized example. For example,
in the mixture of normal problem the “up” move could involve splitting a
term in the mixture into two new terms, and the “down” move could involve
merging two terms.
Consider an RJ for a generic target distribution π(θ) on a variable-dimension
parameter space Θ = ∪n Θn with Θn ⊆ Rn . We let θ ∈ Θ denote the
state vector and write θn when we want to highlight θn ∈ Θn . We let
πn (θn ) = π(θ | Θn ) denote the target distribution Rrestricted to Θn and as-
sume that it has a density fn (θn ). That is, πn (Fn ) = F fn (θ) dθ for any event
n
F n ⊆ Θn .
We consider an RJ including transition probabilities Pu (θ, A) to propose
a move from θ ∈ Θn to θ ∈ Θn+1 (“up”), and Pd (θ, A) to move from
θ ∈ Θn+1 to θ ∈ Θn (“down”). For easier notation we assume that the
jump in dimension is from n to n + 1 and n − 1, respectively (little changes
if it were transitions from Θn to Θn+d and Θn−d , respectively). For the fol-
lowing construction the reader might find it helpful to keep in mind the
specific example of a mixture of normal model with split (“up”) and merge
(“down”) moves. And there might be other transition probabilities that in-
volve moves within Θn . For example, in a mixture of normal problem, we
might include Gibbs sampling transition probabilities to update locations
and weights for a fixed size mixture (like in Problem 6.9). In the following,
we only focus on Pu and Pd which move across dimensions. We construct
general RJ transition probabilities in this problem.
 144 Model Selection and Trans-dimensional MCMC

Algorithm 4 Reversible Jump (RJ).

1
Assume the current state is θn ∈ Θn . Let qn,n−1 = 2
if n ≥ 2 and qn,n−1 = 0
for n = 1.
Move up: With probability 1 − qn,n−1 , propose an “up move,” Pu .
1. Select one of Nnup possible moves. For example, in a mixture problem
one might select the term to be split. Let qup,m (θn ) denote the probability
of selecting move m, m = 1, . . . , Nnup . For example, qup,m = 1/Nnup .
2. Generate an auxiliary variable u ∼ qaux (θn ). Often, qaux involves a trun-
cation. Do not forget the normalization constant if it is a function of θn .
3. Use a deterministic function to define a proposal θ̃n+1 = T (θn , u). Record
J = det (∂T/∂θn ∂u).
4. Evaluate the acceptance probability Aup (θn , θ̃n+1 ) = min{1, ρ(θn , θ̃n+1 )}
π(θ̃n+1 ) qn+1,n qdown,m′ (θn+1 ) 1
ρ(θn , θ̃n+1 ) = |J|. (7.8)
π(θn ) qn,n+1 qup,m (θn ) qaux (u)
| {z } | {z } | {z }
target proposal auxiliary
See the following for the details of the down move.
5. With probability Aup set θ = θ̃n+1 , otherwise θ = θn .
Move down: With probability qn,n−1 , carry out a “down move,” Pd .
1. We select one of Nndown possible down moves with probabilities
qdown,m (θn ), m = 1, . . . , Nndown . For example, in a mixture problem, one
might select two adjacent terms to be merged. Let m′ denote the selected
move.
2. Record (θ̃n−1 , u) = T −1 (θn ).
3. Compute the acceptance probability
( )
1
Ad (xn , θ̃n−1 ) = min 1, .
ρ(θ̃n−1 , θn )
4. With probability Ad set θ = θ̃n−1 , otherwise keep θ = θn .

Note how we had to carefully record all moves in steps 1 through 3 and
account for the corresponding proposal probabilities in (7.8).
Nothing much changes if the “down” move also involves an auxiliary
v ∈ Rq . It only complicates notation.
 Problems 145

Problems
As in Chapter 6, most of the problems require some programming to im-
plement MCMC simulation. See Appendix B for some suggestions on how
to implement such simulation in R.

7.1 Normal linear regression – variable selection. Use the setup and notation of
Problem 6.11.
a. Let β̄ = E(β | τ2 , y) and V = Var(β | τ2 , y) denote the posterior mean and
P
covariance matrix for β conditional on τ2 . Let RSS(β ) = i (yi − x′i β )2
denote the residual sum of squares for β . Prove the following result
!
1 n τ2
f (y | τ2 ) ∝ |V| 2 h(β̄ ) τ 2 exp − RSS(β̄ ) , (7.9)
2
as a function of τ2 . That is, the proportionality constant is a function of y.
Hint: You could use Bayes’ theorem for h(β | y, τ2 ) and substitute for
β = β̄ .
b. Now consider variable selection. Let γ = (γ1 , . . . , γ p ) denote a vector of
P
indicators γ j ∈ {0, 1}, let pγ = j γ j , and let Xγ denote the (n × pγ )
submatrix of X with the columns selected by γ j = 1. Similarly, let βγ
denote the subvector of β selected by γ , and similar for β0γ and Σ0,γ .
We change model (6.23) to include variable selection by using
y = Xγ βγ + ǫ, (7.10)
with ǫ = (ǫ1 , . . . , ǫn ), and a modified prior h(βγ | γ) = N(β0,γ , Σ0,γ ) and
hyperprior h(γ j = 1) = π, j = 1, . . . , p, independently across j.
Use (7.9) to find h(γ | τ2 , y).
c. Propose a posterior MCMC scheme to generate a posterior Monte Carlo
sample (γ (m) , τ2(m) ) ∼ h(γ , τ2 | y).
How would you augment the Monte Carlo sample with β(m) ∼ h(β |
γ (m) , τ2(m) , y), if desired?
7.2 Variable selection. Ročková and George (2014) propose the following im-
plementation of SSVS for a Gaussian linear model:
f (y | β, σ) = N(Xβ, σ2 I),
where y = (y1 , . . . , yn ) is a response vector, and X is an (n × p) matrix of
covariates.
We use the following SSVS prior for the regression coefficients β. Let
cov(β) = Dσ,γ = σ2 diag(a1 , . . . , a p ) with ai = (1 − γi )ν0 + γi ν1
denote the prior covariance matrix for β. Here, γ j ∈ {0, 1}, ν0 is small such
√
that β j ≈ ν0 is practically dropping the jth covariate; and ν1 is large.
146 Model Selection and Trans-dimensional MCMC

That is, γ = (γ1 , . . . , γ p ) has the interpretation of a vector of indicators for

variable selection. We assume that
hβ (β | σ, γ) = N(0, Dσ,γ ),
with hγ (γ j = 1 | θ) = θ. The model is completed with hyperpriors,
hσ (σ2 | γ) = IGa(ν/2, νλ/2), hθ (θ) = Be(a, b),
and ν0 , ν1 are fixed hyperparameters.
Ročková and George introduce an EM-type (Dempster et al, 1977) iterative
algorithm to maximize h(β, θ, σ | y). Let (β(t) , θ(t) , σ(t) ) denote the parameter
vector after t iterations. The algorithm proceeds by iteratively maximizing
 
Q(β, θ, σ | β(t) , θ(t) , σ(t) , y) = Eγ ln h(β, θ, σ, γ | y) . (7.11)
The expectation (Eγ ) is with respect to h(γ | β(t) , θ(t) , σ(t) , y), using the cur-
rently imputed values (β(t) , θ(t) , σ(t) ). That is, remove γ from ln h(β, θ, σ, γ |
y) by marginalizing with respect to h(γ | β(t) , θ(t) , σ(t) , y). As a result, the
Q function has no argument γ anymore – it is removed by the expectation,
leaving a function of (β, θ, σ) only.
In the context of the general EM algorithm, the evaluation of Q is known as
the E-step, and the maximization of Q with respect to β, θ, σ is known as the
M-step. The maximization defines (β(t+1) , θ(t+1) , σ(t+1) ).
It can be shown that (β(t) , θ(t) , σ(t) ) converges to the mode of p(β, θ, σ | y).
a. Find h(γ | β, θ, σ, y).
b. Show that h(γ | β, θ, σ, y) is independent across γ j , j = 1, . . . , p.
For the next two questions, factor the posterior distribution and the Q
function in (7.11) as
h(β, θ, σ, γ | y) = C × f (y | β, σ, γ) × hσ (σ2 ) × hβ (β | σ2 , γ)
× hγ (γ | θ) × hθ (θ)
Q = Eγ [ln h(· | y)] = ln C
+ Qy (β, σ) + Qσ (σ2 ) + Qβ (β, σ2 ) + Qγ (θ) + Qθ (θ).
Here, Qy = Eγ [ln f ] and Q x = Eγ [ln h x ] (x = γ, σ, β, θ) and recall again
that the expectation is with respect to h(γ | β(t) , θ(t) , σ(t) , y). Note that, for
example, the only argument of Qγ is just θ, after taking the expectation
with respect to γ.
c. Find Qy , Qσ and Qθ .
d. Find Qβ and Qγ .
7.3 Mixture model. Recall the mixture of normal model (6.16) from Problem
6.9. Let θ = (J, w1 , . . . , w J , µ1 , . . . , µ J , σ2 ).
X
J
gθ (x) = w j N(µ j , σ2 ). (7.12)
j=1
 Problems 147

We now complete the model with a prior on J. Let Poi+ (λ) denote a Poisson
distribution restricted to positive integers. We assume
J ∼ Poi+ (λ).
Use λ = 5.
a. Joint posterior. Let µ = (µ1 , . . . , µ J ), w = (w1 , . . . , w J ), and x = (x1 , . . . , xn ),
and let θ = (µ, w, σ2 , J) denote the complete parameter vector. Find the
joint posterior distribution h(θ | x).
b. Hierarchical model. Rewrite the mixture as a hierarchical model using
indicators si ∈ {1, . . . , J}.
f (xi | si = j) = N(µ j , σ2 ) and P(si = j | w, J) = w j . (7.13)
Let s−i = (s1,...,i−1 , si+1,...,n ).
Find h(µ j | s, w, x), h(si | s−i , µ, w, J, xi ), and h(w | s, J, x).
c. Reversible jump MCMC. Propose a RJ MCMC for posterior simulation
across variable J.
1. Split move. Propose a transition probability for incrementing J, i.e.,
J˜ = J + 1.
Describe the construction of a proposal (step by step), and state the
acceptance probability for the proposal.
Hint: You could: (1) select a term j to split; assume without loss of
generality j = J (rearrange indexes if needed, to simplify notation);
(2) generate a bivariate auxiliary variable (u, v); (3) define T (θ, u, v) by
µ̃ J = µ J + u, µ̃ J+1 = µ J − u, w̃ J = w J v and w̃ J+1 = w J (1 − v).
2. Merge move. Propose a transition probability for decrementing J, i.e.,
J˜ = J − 1 (“merge move”). Describe the construction of a proposal
(step by step) and state the acceptance probability for the proposal.
d. Implementation. Implement an RJ using the complete conditional distri-
butions from part (b) and the RJ move from (c).
Use the data set galaxy in R. You can get it, for example, in the R pack-
age ElemStatLearn. The data is shown in Figure 6.3.
• Plot J against iteration (trajectory of imputed J).
• Estimate g(x) = E[gθ (x) | x] for x on a grid x.
• Evaluate convergence
R∞ diagnostics for σ2 , g10 = gθ (10), g20 = gθ (20),
and G30 = 30 gθ (s)ds.6
7.4 Pseudo-prior. The following example is about inference in a clinical trial us-
ing a historical control. The trial is a study of a proposed treatment for uterine
papillary serous carcinoma, a very rare disease. The study drug is adminis-
tered in three doses, including 0 (control), 1, and 2. To facilitate a study in a
6
Use the R package boa or coda (or any other) for convergence diagnostics. See Section
9.6.
148 Model Selection and Trans-dimensional MCMC

realistic time frame we consider pooling with historical data on control. Let
yi denote the outcome (PFS, progression free survival, in months) for the ith
patient, numbering all patients, including patients from the historical study
as i = 1, . . . , n, with n = 56, including n0 = 40 patients in the historical data
and n1 = 16 in the new study. Let Ii ∈ {0, 1} denote an indicator for a patient
to be in the historical study and zi ∈ {0, 1, 2} for the dose of the study drug.
All patients in the historical data are on control, i.e., zi = 0, i = 1, . . . , n0 .
We use a Weibull regression with (or without in part (b)) a study effect. That
is, a sampling model
yi ∼ Weib(λ, a), with ln λ = β0 + β1 zi + β2 Ii .
a. Set up a prior β j ∼ N(m j , s2j ), j = 0, 1, 2, with m2 = 0, s0 = 0.1 and
s1 = s2 = 0.5. Determine suitable values for m0 and m1 to match expert
opinion that PFS at doses z = 0, 1, 2 should be around 7, 11, and 14
months (you might not be able to exactly match these means). The data
are in the file uterine.txt which is linked on the book’s homepage.7
In addition to zi , yi , and Ii , the file reports an indicator si for observed
event time (si = 1) versus censoring time (si = 0). Carry out posterior
inference.
b. Now we change the prior on β2 , the study effect. Let δ x denote a point
mass at x. We assume



δ0 if M = 0
β2 ∼ 
N(m2 , s2 ) if M = 1,
2

with h(M = 1) = 0.5. That is, β2 = 0 under model M = 0, and β2 ∼

N(m2 , s22 ) under model M = 1. The modified model allows for pooling of
all data (β2 = 0), with prior probability 0.5.
Implement posterior MCMC with a pseudo-prior mechanism for transdi-
mensional MCMC across M = 0 and M = 1.
c. Alternatively, evaluate a Bayes factor for model M = 0 versus M = 1.
Use an appropriate Monte Carlo strategy to approximate the Bayes factor
(see Section 4.2.2).
7.5 RJ: multivariate mixture model. Zhang et al (2004) develop an RJ algorithm
for a multivariate mixture of a normal model,
X
K
yi | K, µ, Σ, w ∼ wk N(µk , Σk ). (7.14)
k=1

Here, yi = (yi1 , . . . , yiD )′ is a D-dimensional response vector. The covariance

matrix Σk is represented by its singular value decomposition Σk = Ek Λk Ek′ ,
where Ek = (ek1 , . . . , ekD ) is an orthogonal matrix with columns equal to the
7
sites.google.com/view/computationalbayes/home.
 Problems 149

eigenvectors of Σk , and Λk = diag(κk1 , . . . , κkD ) is a diagonal matrix with the

corresponding eigenvalues. We assume that Ek = E is fixed across compo-
nents and thus Σk = EΛk E ′ . We fix E using the SVD, S = EΛE ′ , of the
empirical covariance matrix S , and complete the prior specification for Σk
by assuming
−1
κkd ∼ Ga(a/2, b/2).
The model is completed with K ∼ Poi+ (λ), µ j ∼ N(0, B) and w ∼ DK−1 (a),
with fixed hyperparameters λ, B, and a.
For fixed K, MCMC transition probabilities to update w, µ and κ are very
similar to the univariate mixture model, as in Problem 7.3. However, updat-
ing K requires an RJ-type move. Let ω = (w, µ, κ) denote the parameters
except for K.
We make a random choice to propose a combine or a split move. Let qKd
and qKu = 1 − qKd denote the probability of proposing a combine and a split
move, respectively, for a currently imputed point configuration of size K. We
use qKd = 0.5 for K ≥ 2, and qKd = 0 for K = 1. In the following transition
probabilities, let µ⋆kd = e′d µk and y⋆d = e′d y denote the coordinates of µd and
yd in the eigenvector basis.
Split move: We randomly select a component j to split into two new compo-
nents. The probability of choosing component j is qK s ( j) = K1 . Without loss
of generality, assume that component j = 1 is split into new components
j1 = 1 and j2 = 2 (relabeling components 2, . . . , K into 3, . . . , K + 1). We
define new parameters ω̃:
w̃1 = w1 α, w̃2 = w1 (1 − α)
q q
w̃2 1/2 1/2
µ̃⋆1d = µ⋆1d − w̃1 κ1d rd µ̃⋆2d = µ⋆1d + w̃w̃12 κ1d rd
κ̃1d = βd (1 − rd ) w̃w11 κ1d
2 κ̃2d = (1 − βd )(1 − rd2 ) ww̃12 κ1d ,
where α ∼ Be(1, 1), βd ∼ Be(1, 1) and rd ∼ Be(2, 2) are auxiliary variables.
For later reference, let qu (α, β, r) = p(α)Πd p(βd )p(rd ).
Let θ = (w1 , µ1 , κ1d ) and u = (α, r, β) denote the current state (ignoring the
terms that are not affected by the split) and the auxiliary variables, and let
θ̃ = (w̃1 , w̃2 , µ̃1 , µ̃2 , κ̃1d , κ̃2d ) denote the proposal. In short, the split proposes
θ̃ = T (θ, u).
It is easy to verify that the marginal moments E(y⋆d ) and cov(y⋆d ) remain
unchanged under the proposed extended mixture.
Combine move: We randomly select a pair ( j1 , j2 ) of components to merge.
2
The probability of choosing ( j1 , j2 ) is qKc ( j1 , j2 ) = K(K−1) , j1 < j2 . With-
out loss of generality, assume ( j1 , j2 ) = (1, 2) are merged to form the new
component j = 1. The following deterministic transformation defines pa-
rameter values after the merge. To highlight the relationship with the earlier
150 Model Selection and Trans-dimensional MCMC

split move (and simplify the answer to the following question), we label the
current state vector ω̃ and the proposal ω:
w1 = w̃1 + w̃2 (7.15)
w1 µ1 = w̃1 µ̃1 + w̃2 µ̃2
w1 (κ1d + µ⋆2 ⋆2 ⋆2
1d ) = w̃1 (κ̃1d + µ̃1d ) + w̃2 (κ̃2d + µ̃2d ),

and the implied auxiliary variables

r r 
βd 
 w̃1 w̃2 

α = ..., = . . . and 
 +  rd = . . . (7.16)
1 − βd w̃2 w̃1 
Here, we wrote the equations for µ1 , κ1d , and βd without the final simplifica-
tion – just for beauty, and to highlight how to derive them as the inverse of
the split. For later reference we denote the mapping of (7.15) and (7.16) as
S (θ̃) = (θ, u).
a. Fill in the missing expressions . . . on the r.h.s. of (7.16) and show that
S = T −1 .
b. Find the acceptance ratio for the split move.
c. Find the acceptance ratio for a merge move.
d. See Dellaportas and Papageorgiou (2006) for an alternative RJ for a mul-
tivariate normal mixture of normals. Discuss the relative advantages and
limitations of the two posterior simulation schemes.
7.6 Detail balance for RJ. In this problem we verify the detail balance condition
for the RJ MCMC with general “up” and “down” transition probabilities (in
Section 7.4).
To verify detail balance for this general RJ, we start with the general state-
ment of DB.
Z X Z X
qm (θ)Pm (θ, B) π(dθ) = qm (θ)Pm (θ, A) π(dθ)
A m u m

for any (measureable) A, B ⊂ Θ. We only worry about transdimensional

moves, A = Fn+1 ∈ Θn+1 and B = Fn ∈ Θn :
Z X
qm (θn+1 )Pm (θn+1 , Fn ) π(dθn+1 ) =
Fn+1 m
Z X
qm (θn )Pm (θn , Fn+1 ) π(dθn ) (7.17)
Fn m

Complete the argument to verify (7.17).

Hint: Your argument could proceed along the following steps. (1) A suf-
ficient condition for (7.17) is that the equation holds for pairs (m, m′ ) of
matching down and reciprocal up moves, Pd and Pu . (2) Write Pu as an in-
tegral over u. (3) Use the densities fn and fn+1 , write θn+1 = T (θn , u), and
 Problems 151

use indicators for Fn and Fn+1 to replace the range of integration. (4) Use a
change of variables to make both sides into integrals over (θn , u). (5) Finally,
argue that equality of the integrands is a sufficient condition for equality of
the integrals, and verify that the RJ acceptance probability satisfies the latter
condition.
 8

Methods Based on Analytic Approximations

1
Since the 1980s, investigators have aimed to find efficient, and preferably
simple, approaches to overcome the technical problems of calculus that
arise in Bayesian inference. A variety of strategies have been proposed, in-
cluding in particular multivariate normal approximations of posterior dis-
tributions, the Laplace approach, numerical quadrature methods, classical
Monte Carlo methods, and Markov chain Monte Carlo (MCMC) methods.
Advances in data collection give rise to the need for modeling ever more
complex data structures. This includes, spatio-temporal models, dynamic
linear models, generalized linear mixed models, generalized additive mod-
els, log-Gaussian Cox processes, geo-additive models, and more. All these
models are part of a much larger class of models known as latent Gaus-
sian models (LGMs). See, for example, Blangiardo and Cameletti (2015).
Theoretically, it is always possible to implement MCMC algorithms for
such LGMs. However, such implementations come with many problems in
terms of convergence and computation time.
Recently, Rue et al (2009) developed an analytical approach based on in-
tegrated and nested Laplace approximations (INLAs), which allows deter-
ministic approximation of marginal posterior distributions in these models.
The method provides a particularly efficient implementation of Bayesian
inference in LGMs. The INLA approach has two major advantages over
MCMC techniques. The first is computation time. Using INLA one can get
results within seconds or minutes in models that would take hours or even
days for inference using MCMC algorithms. The second advantage is that
INLA treats LGMs in a unified way, allowing substantial automation of
inference, independent of the specific model.
1
This material will be published by Cambridge University Press as Computational
Bayesian Statistics, by M.A. Amaral Turkman, C.D. Paulino, and P. Müller
(https://tinyurl.com/CompBayes). This pre-publication version is free to view
and download for personal use only. Not for re-distribution, re-sale, or use in derivative
works. c Maria Antónia Amaral Turkman, Carlos Daniel Paulino & Peter Müller, 2019.
 8.1 Analytical Methods 153

To better understand the methods proposed by Rue et al (2009), this

chapter starts with a review of the main techniques for analytical approx-
imation that were developed to implement Bayesian inference, and then
presents the INLA method and its implementation in an associated R pack-
age.

8.1 Analytical Methods

8.1.1 Multivariate Normal Posterior Approximation
One possible strategy to overcome computational problems that arise in
Bayesian inference is based on asymptotic properties of the posterior dis-
tribution. In fact, for large n and under certain regularity conditions, the
posterior distribution for a k-dimensional parameter vector θ is approxi-
mately multivariate normal (Walker, 1969).
Consider a posterior density h(θ | x), written as
h(θ | x) ∝ exp{ln h(θ) + ln f (x | θ)}.
Using a Taylor series expansion of second order for the logarithm of the
two terms in the previous expression, around the respective maxima (as-
suming those are unique), we get
1
ln h(θ) = ln h(m0 ) − (θ − m0 )t H0 (θ − m0 ) + R0 (8.1)
2
1
ln f (x | θ) = ln f (x | θ̂n ) − (θ − θ̂n )t H(θ̂n )(θ − θ̂n ) + Rn ,
2
where m0 is the prior mode and θ̂n is the maximum likelihood estimate of
θ given the data x,
∂2 ln h(θ) ∂2 ln f (x | θ)
H0 = − , H(θ̂n ) = −
∂θi ∂θ j θ=m0
∂θi ∂θ j θ=θ̂n

and R0 , Rn , are the remainders of the respective series expansions. Subject

to certain regularity conditions, which guarantee that the remainder terms
are small for large n (see, e.g., Bernardo and Smith, 1994), we get
( )
1 t
h(θ | x) ∝ exp − (θ − mn ) Hn (θ − mn ) ,
2
Hn = H0 + H(θ̂n ),
mn = Hn−1 (H0 m0 + H(θ̂n )θ̂n ). (8.2)
This expansion suggests that the posterior distribution can be approxi-
mated, for large enough sample size and subject to regularity conditions, by
154 Methods Based on Analytic Approximations

a multivariate normal distribution Nk (mn , Σ̂n ) with mean mn and covariance

matrix Σ̂n = Hn−1 .
With increasing sample size the prior precision, represented by H0 , is
completely dominated by H(θ̂n ), which arises from the data, that is Hn ≈
H(θ̂n ). Thus, also mn ≈ θ̂n , and one can use it to approximate the posterior
distribution by a multivariate normal, centered at the maximum likelihood
h i−1
estimate and covariance matrix Σ̂ = H(θ̂n ) , that is, the inverse of the
observed information matrix. 2
The two separate expansions (around separate modes) in (8.1) are useful
to note the asymptotic match with the observed information matrix. Alter-
natively, one can consider one expansion of the log posterior ln h(θ | x)
around its mode mn , that is, around the posterior mode. Assume, then, a
sequence of posterior distributions {hn (θ | x), n = 1, 2, . . .} for θ and that
Ln (θ | x) = ln hn (θ | x) and mn are such that
Ln′ (mn ) = ∂Ln (θ | x)/∂θ|θ=mn = 0
and
Σn = (−Ln′′ (mn ))−1 ,
where [Ln′′ (mn )]i j = (∂2 Ln (θ | x)/∂θi ∂θ j )|θ=mn . Bernardo and Smith (1994:
ch. 5) prove that, under some conditions on hn (θ | x) and for sufficiently
large n, the posterior distribution can be approximated by a multivariate
normal distribution N(mn , Σn ).
This approach has the advantage that practically all posterior summaries
can be calculated based on a Gaussian route. However, a major problem
arises in the need to verify, for each application, the adequacy of this mul-
tivariate normal posterior approximation.

Example 8.1 Assume that X follows a binomial distribution with param-

eters n (known) and θ, and that θ is assumed to follow a beta prior with
parameters (a0 , b0 ). We know that the posterior for θ is again a beta dis-
tribution with parameters (an , bn ), where an = a0 + x and bn = b0 + n − x.
Thus
hn (θ | x) ∝ θan −1 (1 − θ)bn −1 , 0 ≤ θ ≤ 1.
Of course, if one wishes to make  inference
 on θ or any function of θ, for
θ
example, the logit, that is, ρ = ln 1−θ , one need not resort to sophisticated
2
Since the observed information matrix H(θ̂n ) converges to the Fisher information matrix,
one can also define an approximation with the inverse of the Fisher information matrix
as covariance matrix.
 8.1 Analytical Methods 155

computational methods, as exact solutions are readily available. However,

the example is useful to illustrate the earlier methods.
We find ln hn (θ | x) ∝ (an − 1) ln θ + (bn − 1) ln(1 − θ) and thus
(an − 1) (bn − 1) ′′ (an − 1) (bn − 1)
Ln′ (θ) = − , Ln (θ) = − − (8.3)
θ (1 − θ) θ2 (1 − θ)2
and
(an − 1) (an − 1)(bn − 1)
mn = , −{Ln′′ (mn )}−1 = . (8.4)
an + bn − 2 (an + bn − 2)3
It is easy to verify that the regularity conditions are satisfied, and there-
fore the posterior distribution can be well approximated, for large n, by a
normal distribution, N(mn , σ2n ), with σ2n = −{Ln′′ (mn )}−1 .
In the case of a uniform prior (a0 = b0 = 1), we have
x x/n(1 − x/n)
mn =, σ2n = ,
n n
that is, the posterior distribution for θ given X = x can be approximated by
a normal distribution, N(x/n, (x/n)(1 − x/n)/n). Note the duality of this re-
sult with what is obtained – by the central limit theorem – about the asymp-
totic distribution of X/n given θ. In fact, we know that for X ∼ Bi(n, θ) and
large n the distribution of X/n given θ is well approximated by a normal
distribution N(θ, θ(1 − θ)/n).
If approximate inference on the log odds ρ is desired, one could pro-
ceed with several approaches. Start by verifying that the exact posterior
distribution for ρ can be obtained by a simple transformation. In fact, note
hn (ρ | x) ∝ e an ρ (1 + e ρ )−(an +bn ) , ρ ∈ R. (8.5)
The posterior mean of ρ evaluated from (8.5) is ψ(an ) − ψ(bn ) where the
function ψ(x) is the derivative of the logarithm of Γ(x)3 . We find
ln hn (ρ | x) ∝ an ρ − (an + bn ) ln(1 + eρ )
eρ eρ
Ln′ (ρ) = an − (an + bn ) , Ln
′′
(ρ) = −(a n + bn ) , (8.6)
(1 + eρ ) (1 + eρ )2
and thus
an 1 1
mn = ln and − {Ln′′ (mn )}−1 = + . (8.7)
bn an bn
3
See Gradshteyn and Ryzhik (2007: 943) for integral representations and series
expansions of this function.
156 Methods Based on Analytic Approximations

In summary, the posterior distribution for ρ can be approximated by a nor-

mal distribution, N (ln(an /bn ), 1/an + 1/bn ). In the case of a vague prior
(a0 = b0 = 0), we get:
θ̂ 1
mn = ln , σ2n = with θ̂ = x/n.
1 − θ̂ nθ̂(1 − θ̂)
That is, the posterior for ρ given X = x can be approximated by a normal
distribution N(mn , σ2n ). Note again the duality of this result with the one
obtained, by the central limit theorem, for the asymptotic distribution of
X/n
ln 1−X/n given θ. In fact, we know that for X ∼ Bi(n, θ), the distribution of
X/n
   
θ 1
ln 1−X/n given θ is well approximated, for large n, by an N ln 1−θ , nθ(1−θ)
distribution.


8.1.2 The Classical Laplace Method

Tierney and Kadane (1986) proposed an analytic approach to evaluate ex-
pressions of the form
Z
E[g(θ) | x] = g(θ)h(θ | x) dθ, (8.8)

using the Laplace method to approximate integrals. This method consists,

essentially, of the following: Assume that ψ is a regular function of a k-
dimensional parameter θ and that −ψ has a maximum in θ̂. The Laplace
method approximates integrals of the form
Z
I= f (θ) exp(−nψ(θ))dθ, (8.9)

by a series expansion of ψ around θ̂. In general, an expansion up to second

order suffices. This is what is used in the following argument.
• Consider first the case k = 1.
Expanding ψ(θ) around θ̂ up to second order and substituting it in
exp(−nψ(θ)), we get
!
n(θ − θ̂)2 ′′
exp(−nψ(θ)) ≈ exp −nψ(θ̂) − ψ (θ̂) ,
2
using ψ′ (θ̂) = 0.
Note that the exponential function is proportional to the density func-
tion of a normal distribution with mean θ̂ and variance (nψ′′ (θ̂))−1 , and
 8.1 Analytical Methods 157

therefore
Z +∞ !  1
nψ′′ (θ̂)
exp − (θ − θ̂) = 2π(nψ′′ (θ̂))−1 2 .
2
−∞ 2
Thus the integral I in (8.9) can be approximated by
ˆ + O(n−1 )},
I ≈ I{1 (8.10)
where
√ 1
Iˆ = 2πn− 2 σ̂ f (θ̂) exp(−nψ(θ̂))
and σ̂ = [ψ′′ (θ̂)]−1/2 .
• In the k-dimensional case a similar argument leads to Iˆ of the form,
k k 1
Iˆ = (2π) 2 n− 2 det(Σ̂) 2 f (θ̂) exp(−nψ(θ̂)),
where Σ̂−1 = ∇2 ψ(θ̂) is the Hessian matrix of ψ in θ̂.
Of course, higher-order expansions of f and ψ would obtain better ap-
proximations, for example:
Z (
p −nψ̂ ˆ 1  2 ˆ′′
f (θ)e −nψ(θ)
dθ = (2π)σe f+ σ f − σ4 fˆ′ ψ̂′′′ +
2n
)
5 ˆ ′′′ 2 6 1 ˆ (4) 4 
+ f (ψ̂ ) σ − f ψ̂ σ + O(n−2 ), (8.11)
12 4
where fˆ, ψ̂, etc., are the respective functions evaluated in θ̂, which is, as
already mentioned, a value where the maximum of −ψ(θ) is achieved and
σ2 = [ψ′′ (θ̂)]−1 .
Assume, then, that one wants to evaluate the posterior expected value of
some function g(θ) of the parameters. By (8.8) we see that E[g(θ) | x] can
be obtained as a ratio of two integrals, that is,
R
g(θ) f (x | θ)h(θ)dθ
E[g(θ) | x] = R . (8.12)
f (x|θ)h(θ)dθ
The basic idea is to apply separate Laplace approximations to the numer-
ator and denominator integrals and consider the ratio of these approxima-
tions. Tierney and Kadane (1986) obtain the following two corresponding
approximations for E[g(θ) | x].
• Use the Laplace approximation for exp(−nψ(θ)) = f (x | θ)h(θ), in
numerator and denominator, with f (θ) = g(θ) in the numerator and
f (θ) = 1 in the denominator of (8.12) to get
E[g(θ) | x] = g(θ̂)[1 + O(n−1 )]. (8.13)
158 Methods Based on Analytic Approximations

Note that this corresponds to approximating E[g(θ) | x] by the mode

g(θ̂), where θ̂ is the posterior mode, since θ̂ was defined as the point of
maximum of −ψ(θ).
• Assuming that g(θ) is positive almost everywhere and, to simplify, that
θ is a real-valued parameter, we get
E[g(θ) | x] = (σ⋆ /σ̂) exp{−n[ψ⋆ (θ⋆ ) − ψ(θ̂)]}(1 + O(n−2 )).
To arrive at this approximation, consider
R
exp{−nψ⋆ (θ)}dθ
E[g(θ) | x] = R , (8.14)
exp{−nψ(θ)}dθ
where
−nψ(θ) = ln h(θ) + ln f (x | θ),
−nψ⋆ (θ) = ln g(θ) + ln h(θ) + ln f (x | θ). (8.15)
Define θ̂, θ⋆ and σ̂, σ⋆ such that
−ψ(θ̂) = sup{−ψ(θ)}, σ̂ = [ψ′′ (θ)]−1/2 |θ=θ̂ ,
θ

−ψ (θ ) = sup{−ψ⋆ (θ)}, σ⋆ = [ψ⋆′′ (θ)]−1/2 |θ=θ⋆ .

⋆ ⋆
(8.16)
θ

Assuming that ψ(·), ψ⋆ (·) are sufficiently regular functions, the Laplace
approximations for the numerator and denominator integrals in (8.14)
(using in both cases f (θ) = 1) are, respectively,
√ √
2πσ⋆ n−1/2 exp{−nψ⋆ (θ⋆ )} and 2πσ̂n−1/2 exp{−nψ(θ̂)}.
From there we get the following approximation for E[g(θ) | x],
E[g(θ) | x] ≈ (σ⋆ /σ̂) exp{−n[ψ⋆ (θ⋆ ) − ψ(θ̂)]}. (8.17)
The approximation errors for the two integrals are of order n−1 . However,
the relevant terms of the two errors are identical and therefore cancel
out in the ratio. Therefore the final approximation has a relative error of
order n−2 .
To arrive at the previous approximation, we imposed a quite restrictive
condition, namely that g be positive almost everywhere. One can proceed
in various ways for approximations with real-valued function g in more
general problems. Tierney et al (1989) suggest to start with an approxi-
mation of the moment-generating function of g(θ) (E[exp{sg(θ)}]), using
Laplace approximation for positive functions, and then from there to ob-
tain an approximation of the expected value of g(θ) as the derivative of the
 8.1 Analytical Methods 159

logarithm of the moment generating function in s = 0. The error for this

approximation is of order O(n−2 ).
Another approach, proposed by Tierney et al (1989), is to rewrite the
integrands in (8.12) such that the expectation E[g(θ) | x] can be written as
R
fN (θ) exp{−nψN (θ)}dθ
E[g(θ) | x] = R , (8.18)
fD (θ) exp{−nψD (θ)}dθ
with appropriate fN , fD , ψN , and ψD ,4 and then use the Laplace approxima-
tion (8.11) for both integrals.

Example 8.2 Getting back to the first example, if one wishes to estimate
θ
g(θ) = ρ = ln 1−θ , one could use a Laplace approximation. Since g(θ) can
take negative values, we either use the first approximation that corresponds
to estimating the mean by the mode, obtaining
an − 1
E(ρ | x) = ln ,
bn − 1
or we use one of the alternative approaches proposed by Tierney et al
(1989).
Using the approach in (8.18) with ψN = ψD = ψ, that is, −nψ = ln h(θ) +
ln f (x | θ); fN (θ) = g(θ); fD (θ) = 1, we get
an − 1 1 (an − bn )
E(ρ | x) = ln +
bn − 1 2n (an − 1)(bn − 1)
1 (an − 1)(bn − 1)
− 2 [(an − 1)2 − (bn − 1)2 ]. (8.19)
n (an + bn − 2)
Compare this result, for specific values of an and bn , with the exact value.
Tanner (1996) suggests obtaining an approximation of the mean of ρ
based on the transformation
1 bn θ
λ= ln .
2 an (1 − θ)
It is easily seen that the distribution of λ is the distribution of Fisher’s z
with probability density function
e2an λ
h(λ) ∝ ,
(2bn + 2an e2λ )(an +bn )
4
For example, if one were to use ψN = ψD ; fN (θ) = g(θ); fD (θ) = 1 then (8.18) would
reduce to (8.13) only. See Tierney et al (1989) or Robert (1994) for more details.
160 Methods Based on Analytic Approximations

with approximate mean

" #
1 1 − (2an )−1
ln .
2 1 − (2bn )−1
Thus we obtain as approximate posterior mean for ρ,
an − 0.5
ln ,
bn − 0.5
which is more precise than the previous estimates. 

These ideas are easily extended to the multiparameter case, with im-
mediate application to the evaluation of marginal posterior distributions,
posterior moments, and predictive densities.
Assume θ ∈ Θ = Rk and that we wish to find the marginal posterior
distribution for θ1 . Partition the parameter vector as θ = (θ1 , θ−(1) ) where
θ−(1) = (θ2 , ..., θk ) ∈ Θ−(1) . The marginal posterior distribution for θ1 can be
written as the ratio of two integrals:
Z
h1 (θ1 | x) = h(θ1 , θ−(1) | x)dθ−(1) (8.20)
Θ−(1)
R
Θ−(1)
h(θ1 , θ−(1) ) f (x | θ1 , θ−(1) )dθ−(1)
= R . (8.21)
Θ
h(θ) f (x | θ)dθ
Applying Laplace approximation to the numerator and denominator inte-
grals in (8.20), we obtain the approximation
!1/2
det(Σ̂∗ (θ1 )) h(θ1 , θ̂−(1) ) f (x | θ1 , θ̂−(1) )
h1 (θ1 | x) ≈ , (8.22)
2πn det(Σ̂) h(θ̂) f (x | θ̂)
where θ̂ maximizes h(θ) f (x | θ), and Σ̂ is the negative inverse of the cor-
responding Hessian matrix evaluated in θ̂, θ̂−(1) maximizes h(θ1 , θ−(1) ) f (x |
θ1 , θ−(1) ) for fixed θ1 , and Σ̂∗ (θ1 ) is the negative inverse of the corresponding
Hessian matrix evaluated in θ̂−(1) .
In Section 8.3 we will see how this result becomes useful in the INLA
approach proposed by Rue and collaborators.

Using similar arguments, one can further show that (8.17) remains valid
when θ ∈ Rk with
σ̂ = |∇2 ψ(θ̂)|−1/2 and σ⋆ = |∇2 ψ⋆ (θ⋆ )|−1/2 ,
where
2 2 ⋆
[∇2 ψ(θ)]i j = ∂∂θψ(θ)
i ∂θ j
and [∇2 ψ⋆ (θ)]i j = ∂∂θψi ∂θ(θ)j .
 8.2 Latent Gaussian Models (LGM) 161

For more discussion of this method, see the references cited in the book
by Paulino et al (2018). Despite being a powerful technique, the approach
has some limitations. In the multivariate case the application can become
difficult and impractical, in particular when the integrands are multimodal
and/or the derivatives are difficult to obtain. In many problems, even with
moderate dimension parameter vectors, it is convenient to consider re-
parametrizations for better approximations.

8.2 Latent Gaussian Models (LGM)

The class of models known as LGMs can be represented by a hierarchical
three-level structure. The first level is the top-level sampling model,
x | θ, ψ ∼ f (x | θ, ψ) = Πni=1 f (xi | θ, ψ), (8.23)
The second level assumes that the parameter vector θ follows a Gaussian
Markov random field (GMRF), with respect to an undirected graph G =
(V = {1, ..., n}, E) (Rue and Held, 2005), that is
θ | ψ ∼ N(0, Σ(ψ))
θl ⊥ θm | θ−(lm) , ∀{l, m} < E (8.24)
where θ−(lm) is the vector θ without the components θl and θm , implying
that θl and θm are conditionally independent if they do not share an edge.
The third level specifies a prior distribution h(ψ) for unknown hyperparam-
eters ψ, including hyperparameters in the covariance matrix of θ and in the
sampling model (8.23).
Many problems allow partitioning the hyperparameter vector ψ into ψ =
(ψ1 , ψ2 ) such that the LGM can be stated as
x | θ, ψ ∼ f (x | θ, ψ) = Πni=1 f (xi | θi , ψ2 ) (sampling model for x)
θ | ψ ∼ N(0, Σ(ψ1 )) (GMRF prior for θ)
ψ ∼ h(ψ) (hyperprior,)
The notation θi in the sampling model indicates that xi only depends on one
or a few components of the latent field, e.g., θi . Most components of θ will
not be observed. Also ψ1 is a vector of hyperparameters for the covariance
matrix and ψ2 are, for example, dispersion parameters. In such models, the
vector θ could be very high dimensional, in contrast to ψ which usually is
low dimensional (1 – 5).

Example 8.3 In a longitudinal study, n patients with the same disease

were assigned to two different treatments. Clinical evaluations were recorded
 162 Methods Based on Analytic Approximations

at three different times after treatment. The aim of the study was to evaluate
whether the disease developed differently between the two groups, whether
this depends on age, duration of the disease before treatment, or on an-
other time-dependent covariate that was recorded at the same three times
as the clinical evaluations of the disease.
A Bayesian analysis of this regression problem with repeat measure-
ments is easily implemented as an LGM. Let X jk denote the result of the
clinical evaluation of patient j (with j = 1, ..., n) at time k (with k = 1, 2, 3).
Let z j = (z1, j , z2, j , z3, j ) denote the covariate vector of treatment, age, and
duration of the disease, respectively, for patient j, and let z jk denote the
time-dependent covariate.
We assume the following model for X jk :
• X jk ∼ N(µ jk , σ2 ), independent conditional on the parameters
• µ jk = β0 + β1 z1, j + β2 z2, j + β3 z3, j + β4 z jk + a j + b jk
iid iid
• a j ∼ N(0, σ2a ), b jk ∼ N(0, σ2b ), where a = (a j , j = 1, ..., n) and b =
(b jk , j = 1, ..., n, k = 1, 2, 3) are random effects at the level of patients
and at the level of repeat measurements within a patient (see Section
9.2.1 on including b jk ). The random effects are introduced to induce de-
pendence that arises from the longitudinal nature of the data;
iid
• β = (β1 , β2 , β3 , β4 ), β0 , βi , i = 1, . . . , 4 ∼ N(0, σ2β );
• Let τ = 1/σ2 , τa = 1/σ2a , τb = 1/σ2b , and τβ = 1/σ2β denote the preci-
sions. We assume τ ∼ Ga(c, d) and τ x ∼ Ga(c x , d x ) for x = a, b, β.
The model can be written as a three-level hierarchical model. Let N(x |
m, s2 ) indicate an N(m, s2 ) density for the random variable X.
1. X | z, θ, ψ ∼ f (x | z, θ, ψ) = Π j,k N(x jk | µ jk , 1/ψ2 );
2. θ = (β0 , β, a, b) with θ ∼ N(0, Σ(ψ1 )), i.e. θ | ψ ∼ GMRF(ψ1 );
3. ψ = (ψ1 , ψ2 ) with ψ1 = (τβ , τa , τb ), assuming that the parameters are a
priori independent and ψ2 = τ.


Note that the assumed prior independence of the fixed effects parameters
is not needed. One of the attractive properties of the GMRF (see Rue and
Held, 2005) is that the precision matrix Q = Σ−1 is sparse. In fact, one can
show
θl ⊥ θm | θ−(lm) ⇔ Qlm = 0,
 8.3 Integrated Nested Laplace Approximation 163

where Qlm is the lm element of the precision matrix Q. This has computa-
tional advantages when using numerical methods that are specifically de-
signed for sparse matrices.
An alternative way to state the LGM, and which is central to the INLA
approach, is based on recognizing the model as a special case of a re-
gression model with additive structure (Fahrmeir and Tutz, 2001). In these
models, the dependent variable (Xi ) is assumed to have an exponential fam-
ily distribution with mean µi that is linked to a predictor ηi with additive
structure (see the following) via a link function g(µi ) = ηi , and the sampling
model can still additionally be controlled by hyperparameters ψ2 . The gen-
eral form of the predictor is
nβ
X nf
X
ηi = β0 + β j z ji + wki f (k) (uki ) + ǫi , (8.25)
j=1 k=1

where β0 is an intercept, β = (β1 , ..., βnβ ) is a vector of linear coefficients

for the covariates z, and the functions ( f (1) , ..., f (n f ) ) of covariates u can
represent non linear effects of continuous covariates, seasonal effects, and
random effects of various natures. These functions can have associated
weights ({wki }), fixed and known for each observation. Random effects
without specific structure are included as ǫi .
A latent Gaussian model is then obtained by assuming for θ = {β0 , {β j },
{ f (k) (uki )}, {ηi }} a multivariate normal prior with precision matrix Q(ψ1 ),
that is, a GMRF. This parametrization of θ, including ηi , is useful since
it allows one to link each observation with one component of the random
field.
The definition of the latent Gaussian model is completed with the spec-
ification of a final hyperprior for the model hyperparameters (ψ1 , ψ2 ).

8.3 Integrated Nested Laplace Approximation

In LGMs, the posterior distribution of interest is
h(θ, ψ | x) ∝ h(θ | ψ)h(ψ)Πi f (xi | θi , ψ)
 
 1 T X 
∝ h(ψ)|Q(ψ)| exp − θ Q(ψ)θ +
n/2
ln( f (xi | θi , ψ)) .
2 i

The principal aim of INLA is to obtain an analytic expression for the

marginal posterior distributions of the latent parameters and the hyperpa-
rameters in the Gaussian model.
164 Methods Based on Analytic Approximations

The marginal distributions can be written as:

Z Z
h(θi | x) = h(θi , ψ | x)dψ = h(ψ | x)h(θi | ψ, x)dψ
Z
h(ψk | x) = h(ψ | x)dψ−(k) ,

where ψ−(k) denotes the vector ψ without the component ψk . The evaluation
of these distributions requires first estimates of h(ψ | x) and h(θi | ψ, x) to
then obtain
Z
h̃(θi | x) = h̃(ψ | x)h̃(θi | ψ, x)dψ (8.26)
Z
h̃(ψk | x) = h̃(ψ | x)dψ−(k) , (8.27)

where h̃(. | .) is an approximation of the respective density. The approxi-

mation of marginal posterior distributions (8.26) and (8.27) is then imple-
mented in three steps: an approximation for h(θi | ψ, x); an approximation
for h(ψ | x); and finally numerical integration. The name INLA comes from
exactly these steps. Note that
h(θ, ψ | x) h(ψ)h(θ | ψ) f (x | θ, ψ)
h(ψ | x) = ∝ .
h(θ | ψ, x) h(θ | ψ, x)
If e
h(θ | ψ, x) were a Gaussian approximation for h(θ | ψ, x) with mode b
θ(ψ),
then an approximation for h(ψ | x) can be obtained by

e h(ψ)h(θ | ψ) f (x | θ, ψ)
h(ψ | x) ∝ ,
e
h(θ | ψ, x) θ=b
θ(ψ)

which is the Laplace method as proposed by Tierney and Kadane (1986)

for the approximation of the marginal posterior (proceeding similar to the
argument in (8.22)). Regarding the approximation e h(θ | ψ, x), using
 


 1 T X 


h(θ | ψ, x) ∝ exp 
 − θ Qθ − ln( f (xi | θi , ψ))
 ,
 2 
i

one can obtain a normal approximation for h(θ | ψ, x) by an iterative

process, considering a second-order Taylor series expansion of ln( f (xi |
θi , ψ)) = gi (θi | ψ) around the ith component µ(0)
i (ψ) of an initial value for
the mean vector µ(0) (ψ). See Rue et al (2009) for details.
Regarding the approximation of h(θi | ψ, x) in (8.26), there are several
possible approaches:
 8.3 Integrated Nested Laplace Approximation 165

1. Directly use a normal approximation for h(θ | ψ, x) and a Cholesky de-

composition for the precision matrix Q(ψ1 ), that is, Q(ψ1 ) = L(ψ1 )LT (ψ1 ),
where L(ψ1 ) is a lower triangular matrix, to obtain marginal variances.
In this way, the only additional work is to calculate the marginal vari-
ances. However, this normal approximation for h(θi | ψ, x) is generally
not very good.
2. Let θ−i denote θ without θi ; then
h(θi , θ−i | ψ, x) h(ψ)h(θ | ψ) f (x | θ, ψ)
h(θi | ψ, x) = ∝ .
h(θ−i | θi , ψ, x) h(θ−i | θi , ψ, x)
Using a normal approximation for h(θ−i | θi , ψ, x), an estimate for h(θi |
ψ, x) is then
h(ψ)h(θ | ψ) f (x | θ, ψ)
h̃(θi | ψ, x) ∝ , (8.28)
h̃(θ−i | θi , ψ, x) θ−i =θc
−i (θi ,ψ)

where θc −i (θi , ψ) is the mode of h̃(θ−i | θi , ψ, x). This approach gives a

better approximation than the earlier one, but the complication is that it
needs re-calculation for each θ and ψ, since the precision matrix depends
on θi and ψ.
3. To overcome this problem, Rue et al (2009) suggested several modifica-
tions that lead to alternative Laplace approaches which they call com-
plete Laplace approximation and simplified Laplace approximation.
Under the complete approach, one avoids the optimization, using in-
stead of the mode the conditional mean E(θ−i | θi ) based on a normal
approximation h̃(θ | ψ, x). Besides this, only the θ j which are “close”
are used, using the intuition that only those should impact the marginal
posterior distribution for θi . Such a region of interest around θi is con-
structed by considering θ j to be close to θi if |ai j (ψ)| > 0.001, with ai j
defined by
E(θ j | θi ) − µ j (ψ) θi − µi (ψ)
= ai j (ψ) ,
σ j (ψ) σi (ψ)
where µi , σi , µ j , σ j , are based on the Gaussian approximation h̃(θ | ψ, x).
Finally, the simplified approach is based on a third-order Taylor series
expansion of the log numerator and denominator in (8.28), substituting
again for θ−i the conditional expectation (instead of the mode). In the nu-
merator, the third-order terms allow correcting the approximation with
respect to asymmetry. Details of this approach are explained by Rue et al
(2009).
166 Methods Based on Analytic Approximations

Naturally, the computational details in the implementation of these meth-

ods are non-trivial. An extensive description of some of these details are
given by Rue et al (2009) and Blangiardo et al (2013). An efficient im-
plementation of this approach is available in the public domain package
R-INLA (see www.r-inla.org).

8.4 Variational Bayesian Inference

8.4.1 Posterior Approximation
Variational Bayesian inference (VB) (Jordan et al, 1999) implements ap-
proximate posterior inference by approximating the joint posterior distri-
bution h(θ | x) within a predefined class of distributions D (variational
family).
The most widely used variational family D is the class of independent
distributions, D = {q : q(θ) = Π j q j (θ j )}, known as the mean-field varia-
tional family. Here, independence is across the elements or subvectors of
θ = (θ1 , . . . , θ p ). We briefly review the setup of VB, in particular mean-field
VB. For a more extensive recent review see, e.g., Blei et al (2017).
The criterion to select the best approximation within a variational family
D is Kullback–Leibler (KL) divergence. That is,
q∗ = arg min KL{q(θ)||h(θ | x)}. (8.29)
q∈D

Recall that KL is defined as KL(q(θ)||h(θ | y)) = Eq ln{q(θ)/h(θ | y)}, with

the expectation being with respect to q(θ). Using Bayes theorem to substi-
tute h(θ | x) and noting that Eq p(x) = p(x), we find
KL{q(θ)||h(θ | x)} = Eq {ln(q(θ)} − Eq {ln[ f (x | θ)h(θ)]} + ln p(x) (8.30)
= Eq {ln[q(θ)/h(θ)]} − Eq {ln[ f (x | θ)]} + ln p(x).
All expectations are with respect to q(θ). Therefore, (8.29) is equivalent to
maximizing
n o
q∗ = arg max Eq [ln f (x | θ)] − KL(q(θ)||h(θ)) . (8.31)
q | {z }
ELBO

The criterion is known as evidence lower bound (ELBO). This is because

(8.30) can be stated as
KL{q(θ)||h(θ | x)} = KL{(q(θ)||h(θ)} − Eq [ln f (x | θ)] + Eq [ln p(x)]
= ln p(x) − ELBO ≥ 0,
and therefore ln p(x) ≥ ELBO (and “evidence” is another name for the
 8.4 Variational Bayesian Inference 167

marginal distribution p(x)). Equation (8.31) also reveals the nature of VB

as another form of balancing maximization of the (log-) likelihood and
shrinkage toward the prior.

8.4.2 Coordinate Ascent Algorithm

An easy algorithm to find q∗ is an iterative conditional maximization known
as coordinate ascent variational inference (CAVI). By assumption, q ∈ D
factors as q(θ) = Π pj=1 q j (θ j ). CAVI is defined as iterative optimization of
q j , keeping all other qk , k , j, at their currently imputed choice. The algo-
rithm repeatedly cycles over j = 1, . . . , p, until q∗ remains unchanged for
an entire cycle. As a greedy optimization, the algorithm delivers a local op-
timum only. Most importantly, the optimization of q j in each step is easy.
Let h(θ j | θ− j , x) denote the complete conditional posterior distribution of
θ j given the remaining parameters, and let E− j (·) denote an expectation
over θ− j with respect to q− j (θ− j ) = Πk, j qk (θk ), that is, an expectation with
respect to the current solution for qk , k , j. Then the optimal choice for q j
at each step of the CAVI is given by
q∗j (θ j ) ∝ exp{E− j ln h(θ j | θ− j , x)}. (8.32)
Blei et al (2017) give an elegant and easy argument for this result. The
optimization is with respect to KL{q(θ)||h(θ | y)} in (8.29). Using h(θ | x) =
h(θ j | θ− j , x) h(θ− j | x) and the assumed factorization of q(θ) we have
E{ln h(θ | x)} = E j {E− j ln h(θ j | θ− j , x)} + E− j ln h(θ− j | x).
Dropping terms that do not depend on q j , we are left with optimizing q j
with respect to
ELBO(q j ) ≡ E j {E− j ln h(θ j | θ− j , x)} − E j ln q j (θ j ).
Here, E− j is defined as before, and E j is an expectation with respect to q j .
Note that ELBO(q j ) = −KL{q j (θ j )||q∗j (θ j )} for the q∗j from (8.32). As such,
it is maximized for q j = q∗j . That is all. In summary, the algorithm is as
shown in Algorithm 5.
168 Methods Based on Analytic Approximations

Algorithm 5 Coordinate Ascent Algorithm (CAVI)

Input: posterior h(θ | x); variational family D = {q : q(θ) = Π j q j (θ j )};
initial solution q∗ ∈ D
Output: variational approximation q∗ (θ) = Π q∗j (θ j )
1: repeat
2: for j = 1 to p do
3: q∗j ∝ exp{E− j ln h(θ j | θ− j , x)}.
4: end for
5: until no change

The optimization becomes particularly easy when h(θ j | θ− j , x)} is an

exponential family model. In that case,
h(θ j | θ− j , x) ∝ a(θ j ) exp(η′j t(θ j )), (8.33)
with η j = η j (θ− j , x) being some function of (θ− j , x). If we use a variational
family D with q j in the same exponential family as (8.33), then line 3 in the
algorithm greatly simplifies. First, taking the expectation E− j requires only
E− j [η j (θ− j , x)]. Second, and most important, recording q∗j we only need to
record the updated hyperparameters η j . Since the (unconstrained) solution
q⋆ is in D, the use of the assumed family D constitutes no restriction.
This situation arises quite commonly in conditionally conjugate models,
including in particular hierarchical models with conditionally conjugate
model choices. The simplification is best appreciated in an example. We
outline the steps for an example from Gelfand et al (1990), which also
appears as the rats example in the OpenBUGS manual, www.openbugs.
net/w/Examples.

Example 8.4 Gelfand et al (1990) discuss this example of a hierarchical

normal/normal model. The data are weights for n = 30 young rats, mea-
sured weekly for J = 5 weeks. The data are given in table 3 of Gelfand et al
(1990). Let yi j denote the recorded weight for rat i in week j. Let x j denote
the measurement times in days. We assume a normal sampling model,
yi j | αi , βi , σy ∼ N(µi j , τy ) with µi j = µ + αi + βi x j ,
with a normal prior for animal-specific growth curve parameters
αi ∼ N(µα , τα ) and βi ∼ N(µβ , τβ ),
where the second parameter of the normal distribution is the precision (us-
ing the parametrization from WinBUGS). The model is completed with a
 8.4 Variational Bayesian Inference 169

hyperprior
τy ∼ Ga(γy , δy ), τα ∼ Ga(γα , δα ), τβ ∼ Ga(γβ , δβ ) and p(µα ) = p(µβ ) = c,
with fixed hyperparameters (γy , δy , γα , δα , γβ , δβ ), and assuming indepen-
dence across parameters in the hyperprior.
Let ω = (µα , µβ , τy , τα , τβ , αi , βi , i = 1, . . . , n) denote the complete pa-
rameter vector, and let θi = (αi , βi ) denote the animal-specific random ef-
fects. In a slight abuse of notation, let N(x | m, P) indicate a multivariate
normal distribution for the random variable x with mean m and precision
matrix P and similarly for Ga(x | c, d) (note the use of the precision matrix
instead of the covariance matrix as the second parameter in the normal
distribution). We use a variational family D with variational distributions
of the form
q(ω) = Πi N(θi | mi , Pi ) Π x=y,α, β Ga(τ x | c x , d x ) Π x=α, β N(µ x | m x , P x ).
The rationale for this choice will be evident in a moment, when we work
out the updates (8.32). As before, let E−x denote an expectation under q
with respect to all parameters except for x (where the symbol x here is
a placeholder for any of the parameters). Let h(ω | y) denote the joint
posterior distribution, and let h(x | ω−x , y) denote the complete conditional
posterior for parameter x. For reference, we note the joint posterior. Let
yi = (yi j , j = 1, . . . , J) denote the repeat measurements for animal i. Let X
denote the (J × 2) design matrix for the linear regression for one animal,
let H = XX ′ , and let θ̂i denote the least squares fit for the θi . Also, let
T = diag(τα , τβ ) and µ = (µα , µβ ), and let N = nJ denote the total sample
size. We have

h(ω | y) ∝ Π x=y,α, β Ga(τ x | γ x , δ x ) Πi N(yi | Xθi , τy I) N(θi | µ, T ) =

N
−n
Π x=y,α, β Ga(τ x | γ x , δ x ) τy2 Πi N(θ̂i | θi , τy H) N(θi | µ, T ),
with h(µα ) = h(µβ ) = c being subsumed in the proportionality constant.
Following (8.32) we find then the following updating equations by first de-
riving the complete conditional posterior h(x | ω−x , y) for each parameter
and then E−x ln h(x | ω−x , y), where, again, x is a placeholder for any of
the parameters and E−x is an expectation with respect to ω−x under the
variational distribution q. Taking the latter expectation, keep in mind the
independence under q, which greatly simplifies the evaluation. We start
with the update for q(θi ). Below we write ω−i as short for ω−θi and similar
for E−i . Note that h(θi | ω−i , y) = N(θi , V̄), with
V̄ = T + τy H and θi = V̄ −1 (T µ + τy H θ̂i ).
170 Methods Based on Analytic Approximations

Therefore, ln h(θi | ω−i , y) = c − 12 θi′ V̄θi + θi′ V̄θi . Let τ x = c x /d x (x = α, β, y),

T̄ = diag(τα , τβ ) and µ = (mα , mβ ) denote expectations under q−i . Also, let
Pi = E−i V̄ = T̄ + Hτi . Then,
1 n o
E−i ln h(θi | ω−i , y) = c − θi′ Pi θi + θi′ −i T µ + τy H θ̂i
2
1 ′
= c − θi Pi θi + θi′ i P−1 i (T̄ µ + τy H θ̂i ) (8.34)
2
and therefore by (8.32), q∗ (θi ) = N(mi , Pi ) with mi = P−1
i (T̄ µ + τy H θ̂i ) and
Pi as above.
Similar simplifications apply for updating q(τy ) q(τα ), q(τβ ), q(µα ), and
q(µβ ). In all cases it turns out that the expectation under q−x in the equiva-
lent of (8.34) is easy. This is no coincidence. Because the complete condi-
tional distributions are exponential families, ln h(x | ω−x , y) always reduces
to an expression of the type t(x)η x (ω−x ), and the expectation of η x (·) is usu-
ally easy since q−x includes independence across all other parameters.

In Problem 8.6 we consider a finite mixture. VB for mixture models has
been generalized to infinite mixture of normal models by several recent
papers, including those by Blei and Jordan (2006) and Lin (2013), who
also introduce a sequential scheme to accommodate big data.

8.4.3 Automatic Differentiation Variational Inference

Implementing Algorithm 5 requires careful consideration of the target dis-
tribution and problem-specific choice of a suitable variational family D.
Alternatively, Kucukelbir et al (2017) develop an implementation of varia-
tional inference that lends itself to automation. The two key elements of the
algorithm are the use of a transformation to map all original parameters to
the real line, and the use of mean-field independent normal variational fam-
ily D, now indexed only by the normal location and scale parameters for
the p independent univariate normal distributions, η = (µ1 , . . . , µ p , σ1 , . . . ,
σ p ). One more mapping transforms to a multivariate standard normal dis-
tribution, making it possible to evaluate the expectations that appear in the
ELBO and using gradient ascent with automatic differentiation to carry out
the optimization.
The algorithm is implemented in Stan, a public domain program that
implements posterior MCMC using Hamiltonian Monte Carlo. See Section
9.4 for a brief introduction to Stan and Problem 9.6 in the same chapter for
an example of using variational inference in STAN.
 Problems 171

Problems
8.1 Normal approximation. Under a particular genetic model, animals of a given
species should be one of four specific phenotypes with probabilities p1 =
(2 + θ)/4, p2 = (1 − θ)/4, p3 = (1 − θ)/4 and p4 = θ/4, respectively. Let
y1 , . . . , y4 denote the number of animals of each phenotype, and N = y1 +. . .+
y4 . Assuming multinomial sampling and a Be(a, b) prior on θ the posterior
density h(θ | y) takes the form
h(θ | y) ∝ (2 + θ)y1 (1 − θ)y2 +y3 +b−1 θy4 +a−1 , 0 ≤ θ ≤ 1,
that is,
L(θ | y) ≡ ln h(θ | y) = C + y1 ln(2 + θ) + (y2 + y3 + b − 1) ln(1 − θ) +
+ (y4 + a − 1) ln(θ)
′ y1 y2 + y3 + b − 1 y4 + a − 1
L (θ) = − +
2+θ 1−θ θ
′′ y1 y2 + y3 + b − 1 y4 + a − 1
−L (θ) = + + ,
(2 + θ)2 (1 − θ)2 θ2
where C is the log normalization constant. Answer (a) through (d) below
using the following two data sets: N = 197, y = (125, 18, 20, 34) and N = 20,
y = (14, 0, 1, 5).
a. Find a (truncated) normal approximation, N(m, V), 0 ≤ θ ≤ 1, by solving
H ′ = 0 to determine m and using V −1 = −H ′′ (m). Let p1 (θ) denote the
p.d.f. Let m1 and V1 denote posterior mean and variance under the trun-
cated normal approximation (use simulation to find the moments of the
truncated normal – note that m1 , m, due to the truncation).
b. As an alternative approximation, find a beta distribution, Be(a, b), by
matching mean and variance (obtained as in part (a)). Let p2 (θ) denote
the beta p.d.f. Let m2 (= m, by construction) and V2 = V denote the
posterior moments under the beta approximation.
c. Now use p1 and p2 as importance sampling densities to carry out impor-
tance sampling to evaluate the posterior mean and variance. Let (m3 , V3 )
and (m4 , V4 ) denote the approximate posterior moments, using impor-
tance sampling densities p1 and p2 , respectively.
d. With p2 as the importance sampling density, carry out importance sam-
pling to find the normalization constant for h(θ | y). Then use numerical
integration on a grid over 0 ≤ θ ≤ 1 (for example, as a Rieman sum, or
using the trapezoidal rule) to evaluate posterior mean and variance. Let
(m0 , V0 ) denote the estimates.
Make a table to compare (m j , V j ), j = 0, 1, . . . , 4.
8.2 LGM. Refer to Example 8.3. Identify the three levels of the LGM. That is,
state the probability models f (xi | θi , ψ2 ), p(θ | ψ), and h(ψ).
172 Methods Based on Analytic Approximations

8.3 LGM (Rue et al, 2009). Consider a stochastic volatility model for daily
pound–dollar exchange rates, with sampling model
yt | ηt ∼ N(0, exp(ηt )),
t = 1, . . . , nd , log variances ηt = µ + ft , prior
ft | f1 , . . . , ft−1 , φ ∼ N(φ ft−1 , 1/τ),
and hyperpriors
′
eφ ′
τ ∼ Ga(1, 0.1), µ ∼ N(0, 1), and φ = 2 ′ − 1, with φ ∼ N(3, 1).
1 + eφ
Show how the model can be stated as an LGM. Identify the three levels of
the LGM. That is, state the probability models f (xi | θi , ψ2 ), p(θ | ψ), and
h(ψ).
8.4 (Rue et al. 2009). Let yi ∈ {0, 1} denote indicators for pre-malignant (yi = 1)
versus malignant (yi = 0) incidences of cervical cancer, i = 1, . . . , nd . Let
di ∈ {1, . . . , 216} and ai ∈ {1, . . . , 15} index geographical region and age
group of the ith case. Rue and Held (2005) use a logistic binary regression
for pi ≡ P(yi = 1),
logit(pi ) = ηi = µ + faai + fdsi + fdui
where f a = ( f1a , . . . , fAa ), A = 15, is a smooth effect of age groups, f s =
( f1s , . . . , fSs ), S = 216, is a smooth spatial field of geographic region effects,
and f u = ( f1u , . . . , fSu ) are district-specific random effects. For f a we assume
a second-order random walk model with precision κa ,
κa P15 a a a 2
f a | κa ∝ (κa )(15−2)/2 e− 2 j=3 ( f j −2 f j−1 + f j−2 ) .
For f s we assume a conditional autoregressive model f s ∼ CAR(0, κ s , G),
where G is a (S × S ) binary adjacency matrix with G st = 1 if district s is
neighboring district t. Subject to some constraints, the CAR model implies
a multivariate normal prior for f s | κ s ∼ N(0, Σ(κ s , G)). This is all that is
needed in this problem. The random effects fdu are independent N(0, 1/κu ).
The model is completed with independent Ga(1, 0.01) priors for κa , κ s , and
κu , and µ ∼ N(0, 0.01), with the second parameter of the normal distribution
being a precision here (matching the parametrization used in INLA).
Show how the model can be stated as an LGM. Identify θ, ψ, and the three
levels of the LGM. That is, state the probability models f (xi | θi , ψ2 ), p(θ |
ψ), and h(ψ).
8.5 Variational Bayes. For Example 8.4 in Section 8.4.2, find the updating equa-
tions for q(τy ) q(τα ), q(τβ ), q(µα ), and q(µβ ).
8.6 Variational Bayes: mixture model. Consider a mixture of normal model f (x |
PK
θ) = k=1 πk N(x | µk , σ2 ), with θ = (π, µ, σ2 ), where µ = (µ1 , . . . , µK ) and
π = (π1 , . . . , πK ). Let γ = 1/σ2 . We complete the model with conditionally
 Problems 173

conjugate priors h(µk ) = N(0, τ), h(γ) = Ga(a, b) and π ∼ DK−1 (α, . . . , α),
with fixed hyperparameters (τ, a, b, α) and prior independence.
a. Find the complete conditional posterior h(θ j | θ− j , x) for µk , πk and γ.
b. For the moment, fix σ2 = 1 and π = (1/K, . . . , 1/K). Consider the mean-
field variation family D with q(µk ) = N(mk , sk ), q(ci = k) = φik , and de-
rive one iteration of CAVI, that is, find the updating equations for (mk , sk )
and φk j .
Hint: See the discussion in Blei et al (2017).
c. Now include γ and π in the parameter vector. Extend D by defining q(γ) =
Ga(c, d) and q(π) = D(e1 , . . . , eK ). Find the updating equations for c, d,
and e.
8.7 In R, load the package MASS which provides the data set galaxies. Scale
the data as x = c(scale(galaxies)). Assume a mixture of K = 5 nor-
PK
mal models, f (xi | θ) = k=1 πk N(µk , σ2 ) with θ = (µ, π, σ2 ). Complete the
model with priors as in the previous problem, using τ = a = b = c = d = e =
1. Use variational inference to generate an approximate Monte Carlo poste-
rior sample, Θ = {θm , m = 1, . . . , M} with θm ∼ h(θ | x), approximately.
a. At each iteration, evaluate the ELBO. Plot it against the iteration.
b. Use Θ to approximate the posterior predictive distribution

b 1 XX m
f (x) = π N(µk , (σm )2 ).
M m k k

Plot the data, together with b

f . Add pointwise central 50% credible bounds
for b
f (x).
 9

Software

1
Using computational methods, like those discussed in previous chapters,
Bayesian data analysis can be implemented for problems arising from a
wide variety of scientific areas. This development has been importantly
supported by the generosity of many investigators who have made soft-
ware implementations of such methods freely available to the scientific
community. The software R includes a variety of packages, or sometimes
just functions, that can be used for Bayesian inference. Interested readers
should consult the page http://cran.r-project.org/web/views/
Bayesian.html. One can find there, for example, the package bayesSurv,
built specifically for Bayesian inference in survival models, for example.
Besides these R packages and the four software packages discussed below,
there are several other public domain programs for Bayesian inference, in-
cluding NIMBLE (de Valpine et al, 2017; Rickert, 2018), bayesm (Rossi
et al, 2005), and the Bayesian regression software by George Karabatsos
(Karabatsos, 2015). Rickert (2018) includes an implementation of infer-
ence for Problem 4.4 and 6.5 in NIMBLE.
For this chapter we selected four general-purpose public domain soft-
ware packages that implement methods based on stochastic simulation. Al-
though they are independent of R, all four can be used through an interface
in R. These are OpenBUGS (Thomas et al, 2006), JAGS (Plummer, 2003),
Stan (Carpenter et al, 2017), and BayesX (Brezger et al, 2005; Belitz et al,
2013). The chapter includes a brief summary of the principal characteristics
of each software and illustrates each in the same example. We also show
how to monitor and diagnose convergence of chains using the software
1
This material will be published by Cambridge University Press as Computational
Bayesian Statistics, by M.A. Amaral Turkman, C.D. Paulino, and P. Müller
(https://tinyurl.com/CompBayes). This pre-publication version is free to view
and download for personal use only. Not for re-distribution, re-sale, or use in derivative
works. c Maria Antónia Amaral Turkman, Carlos Daniel Paulino & Peter Müller, 2019.
 9.1 Application Example 175

packages CODA and BOA. Both are R packages. Finally, we show how
the same example can be analyzed using R-INLA (www.r-inla.org).
There are several books that show the use of these software packages, in-
cluding those by Ntzoufras (2009), Kruschke (2011, 2014), Korner-Nievergelt
et al (2015) and Blangiardo and Cameletti (2015). Each of these packages
comes with extensive documentation.

9.1 Application Example

Here, we introduce the common example that will be analyzed with the
different methods.

Example 9.1 In a longitudinal study, n patients with the same disease

were assigned to two different treatments. A clinical evaluation after the
treatment is made at three different time points. The aim of the study is
to investigate whether disease progression differs between the two groups
and whether it depends on the age of the patient, the duration of the dis-
ease prior to treatment, and another longitudinal covariate that is also
measured at the same three time points as the clinical evaluation of the
disease progression.
Bayesian inference for regression with repeat measurements can be con-
sidered in the context of LGM. In fact, let X jk denote the random variable
that represents the clinical evaluation of patient j (with j = 1, ..., n) at time
k (with k = 1, 2, 3). Let z j = (z1, j , z2, j , z3, j ) denote the vector of covariates
including treatment, age, and prior disease duration for patient j, and let
z jk denote the time-dependent covariate.
The following model is assumed for X jk :
1. X jk ∼ N(µ jk , σ2 ), conditionally independent given the parameters;
2. µ jk = β0 + β1 z1, j + β2 z2, j + β3 z3, j + β4 z jk + a j + b jk ;
iid iid
3. a j ∼ N(0, σ2a ), b jk ∼ N(0, σ2b ), with a = (a j , j = 1, ..., n) and b =
(b jk , j = 1, ..., n, k = 1, 2, 3) representing random effects at the level of
patients and the repeat observations nested within patients, respectively
(see Section 9.2.1 on including b jk ). The random effects are introduced
to represent the dependence structure that arises from the longitudinal
nature of the data;
iid
4. β = (β1 , β2 , β3 , β4 ), β0 , βi , i = 1, . . . , 4 ∼ N(0, σ2β );
5. τ = (σ2 )−1 ∼ Ga(c, d), τa = (σ2a )−1 ∼ Ga(ca , da ),
τb = (σ2b )−1 ∼ Ga(cb , db ), τβ = (σ2β )−1 ∼ Ga(cβ , dβ ).

176 Software

9.2 The BUGS Project: WinBUGS and OpenBUGS

The project BUGS (Bayesian inference using Gibbs sampling) was initi-
ated in 1989 as a contract of Andrew Thomas with the Medical Research
Council Biostatistics Unit in Cambridge. The project arose out of work
that was developed by Spiegelhalter (1986) and Lauritzen and Spiegelhal-
ter (1988) about graphical models and their applications in artificial intel-
ligence, and the recognition of the importance of these structures in the
formulation of Bayesian models. It is interesting to note that at the same
time, but independently, the seminal work of Gelfand and Smith (1990)
was developed in Nottingham, even if this was done with a rather different
perspective.
A prototype of BUGS was first publicly presented in the IV. Valencia
Meeting in 1991. According to Lunn et al (2009), the real push for the de-
velopment came in 1993 after the INSERM Workshop on MCMC Methods,
which was followed by a workshop in Cambridge on the use of BUGS,
which prompted the book MCMC in Practice by Gilks, Richardson and
Spiegelhalter in 1996. In the first versions of the software, only Gibbs sam-
plers were used and only for models with log-concave complete conditional
posterior distributions. The big step forward was at the beginning of 1996
when the project moved to Imperial College London. The WinBUGS ver-
sion was developed, which permitted interactive diagnostics and inference.
The introduction of Metropolis–Hastings methods allowed dropping the
restriction to log-concavity.
The analysis of more complex models was then achieved with the intro-
duction of the slice sampler and the Jump interface for WinBUGS, which
allowed the implementation of MCMC with reversible jumps. GeoBUGS
was developed for spatial data, PKBugs for pharmacokinetic models, and
WBDiff to deal with systems of ordinary differential equations. In 2004,
Andrew Thomas started to work on an open-source version of BUGS at the
University of Helsinki, giving rise to the project OpenBUGS. OpenBUGS
is expected to be the future of the BUGS project after version 1.4.3. was
launched as the last version of WinBUGS in 2007. A history of the project,
including the technical development, can be read in Lunn et al (2009). The
availability of BUGS is without doubt partially responsible for the rapid
dissemination of Bayesian ideas at the end of the last century, as was pre-
dicted by Lindley. The software BUGS was for many years without any
competitors, despite some known limitations of the program, in particular
computation times.
The syntax of BUGS is simple and attractive, by using a text definition
 9.2 The BUGS Project: WinBUGS and OpenBUGS 177

of a Bayesian inference model. Distributions are indicated by the symbol

∼ and logical and/or deterministic relationships by the symbol < −. It in-
cludes deterministic structures that are typically used in programs, such
as for loops. Being a declarative language, the sequence of instructions
is irrelevant. This implies the inconvenience of not allowing, for example,
instructions like if–then–else, which is a great limitation of the language.
The step() function can be used to overcome this limitation, but is unclear
programming style.
With WinBUGS not being further developed, it is now advisable to use
OpenBUGS. The R2OpenBUGS package, an adaptation of the R2WinBUGS
package (Sturtz et al, 2005) by Neal Thomas, serves as the interface be-
tween OpenBUGS and R. To analyze Bayesian models in R with R2OpenBUG
one needs to first write the code to program the statistical model in BUGS.
It is useful to do this in OpenBUGS, which includes the possibility to verify
the syntax and correct possible errors.
There are various manuals for BUGS, which can be downloaded from
the site www.openbugs.net/w/Manuals. The basic manual is OpenBUGS
User Manual, which should be consulted to understand how models are
defined and to understand the functionality of BUGS. A list of accepted
sampling distributions and prior distributions, and how to specify them, is
found in appendix I of this manual. In the case of a sampling distribution
that is not included in this list, there is the possibility that the user con-
structs his or her own likelihood. The section Advanced Use of the BUGS
Language in the manual explains how to do this. The same approach can
be used to specify a prior distribution for a parameter that is not listed in
the appendix.
Next we illustrate the use of BUGS with the example that was introduced
earlier in this chapter.

9.2.1 Application Example: Using R2OpenBUGS

After opening an R session, install the package using the command
install.packages("R2OpenBUGS", dependencies=TRUE,
repos="http://cran.us.r-project.org")

To simplify the commands we will always assume that all files that are
needed for the program are saved in the working directory of the R session
(use setwd() if needed). The steps to analyze the model in R2OpenBUGS
are then the following:
178 Software

1. Write the code for the model and save it in a file with extension .txt.
In this case, we saved it in the file Cexemplo1BUGS.txt.
model{
for(i in 1:147){
X[i]~dnorm(mu[i],tau)
mu[i]<-beta0+beta[1]*z1[i]+beta[2]*z2[i]
+beta[3]*z3[i]+beta[4]*z[i]+a[ID[i]]+b[i]
b[i]~dnorm(0,tau_b)
}
for(j in 1:49){
a[j]~dnorm(0,tau_a)
}
for(k in 1:4){
beta[k]~dnorm(0,0.0001)
}
beta0~dnorm(0,0.0001)
tau~dgamma(0.05,0.05)
tau_a~dgamma(0.05,0.05)
tau_b~dgamma(0.05,0.05)
sigma<-1/sqrt(tau)
sigma_a<-1/sqrt(tau_a)
sigma_b<-1/sqrt(tau_b)
}

The first for loop corresponds to the statement of the probability model
given in item 1 of Section 9.1. In BUGS the second parameter (tau) of
the normal distribution is the precision (inverse variance). The loop also
includes the definition of the linear predictor mu and the statement of the
model for the random effects b that appear in items 2. and 3. of the same
section. The number of patients is n = 49, but since all were observed at
three distinct time points, the total number of observations is 147.
The second for loop is the model for the random effects a. Those effects
are specific to each patient, who are identified by the variable ID in the first
loop.
The third for loop defines the prior distribution for the fixed effect pa-
rameters. After this, the prior distribution for the remaining model para-
meters are given. All prior distributions reflect vague prior information.
Finally, to allow monitoring of standard deviations, those are defined as a
function of the respective precisions. Note that the sequence of these com-
mands is completely arbitrary.
To verify the file that defines the model to be used by OpenBUGS, one
can use the command
file.show("Cexemplo1BUGS.txt")
 9.2 The BUGS Project: WinBUGS and OpenBUGS 179

2. Reading in the data. For better convergence of the chains, it is advis-

able that the covariates that appear in the definition of the linear predictor
be centered. If the data file does not yet include centering, this can be done
in R.
> Cexemplo1<-read.table("Cexemplo1.txt",header=T)
> dim(Cexemplo1)
[1] 147 9
> head(round(Cexemplo1,4))

X gender z1 z2 year z3 ID z all

1 14 1 0 -0.89796 1 2.5979 1 19.1329 1
2 10 2 0 -0.89796 1 -0.4020 2 -13.0671 2
3 8 2 0 0.10204 1 -0.9020 3 -7.3671 3
4 10 2 0 -3.89796 1 -1.2020 4 51.2329 4
5 10 2 0 -7.89796 1 -1.7020 5 18.2329 5
6 20 2 0 -3.89796 1 0.5979 6 -0.8671 6

The covariate z is recorded at three time points. The variable ID is a pa-

tient identifier with values between 1 and 49. The variable all indexes ob-
servations with values between 1 and 147 and is included for convenience,
as will be seen later. The variable year is the time of evaluation, with val-
ues between 1 and 3, also included for convenience only. The continuous
covariates in the file, z2, z3, and z are already centered. The variable z1 is
an indicator for the assigned treatment and takes values 0 or 1.
3. Define the vectors of the data matrix to be used by the model. The
data must be supplied as a vector, matrix, or list.
#Create separate objects for each variable
X<-Cexemplo1$X
ID<-Cexemplo1$ID
z3<-Cexemplo1$z3
z1<-Cexemplo1$z1
z2<-Cexemplo1$z2
z<-Cexemplo1$z
#Create a list of data which will be passed on to OpenBUGS
Cexemplo1.data<-list("X","ID","z1","z2","z3","z")

4. Define the parameters that will be monitored.

Cexemplo1.params <- c("beta0","beta",
"tau","tau_a","tau_b","sigma_a","sigma_b","sigma")

If one wishes to monitor mu, or any other parameter that was defined in the
model specification, for example a, or b, those must appear in the above
list.
5. Define the initial values of parameters and hyperparameters in the
model. In this case we need to define initial values for beta0, beta, tau,
180 Software

tau_a, tau_b, a, b. Those are defined in a list that needs to include initial
values for each chain.
Inits<-list(tau=1,tau_a=1,tau_b=1,beta=c(0,0,0,0), b=c(0,0,0,0,
0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0),a=c(0,0,0,0,0,0,0,0,0,
0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
0,0,0,0,0,0,0,0,0),beta0=0)

Using more than one chain, one needs to create a list of the same type for
each chain, with different initial values, arranged as objects with names,
for examples, “Inits1” and “Inits2”, and then define a list of lists using
Inits<-list(Inits1,Inits2,...)

6. The program OpenBUGS can then be run through R using the func-
tion bugs(), after having verified that the R2OpenBUGS package is loaded
in the current R session.
library(R2OpenBUGS)
Cexemplo1_openBUGS.fit<- bugs(data=Cexemplo1.data, inits=list(Init
parameters.to.save=Cexemplo1.params,
"Cexemplo1BUGS.txt", n.chains=1, n.iter=40000,
n.burnin=20000, debug=FALSE,save.history=FALSE,DIC=TRUE)

It is recommended to verify the arguments for bugs() with the command

?bugs().
7. To obtain summaries of marginal posterior distributions for the param-
eters that were declared earlier in the vector Cexemplo1.params, write
Cexemplo1_OpenBUGS.fit$summary

which generates in this case the output (the final column for the 97.5%
quantile is cut to fit the page)
mean sd 2.5% 25% 50% 75%
beta0 17.4290 1.9208 13.9200 16.1000 17.2800 18.6600
beta[1] 4.2329 2.8568 -2.6160 2.5750 4.4480 6.0600
beta[2] 0.1422 0.1391 -0.1303 0.0474 0.1448 0.2352
beta[3] 4.3456 0.9455 2.3960 3.7600 4.3430 4.9100
beta[4] -0.1029 0.0366 -0.1726 -0.1281 -0.1046 -0.0798
tau 1.5607 3.6155 0.0636 0.0847 0.1521 1.1012
tau_a 0.0162 0.0038 0.0097 0.0135 0.0159 0.0185
tau_b 2.2989 5.4749 0.0638 0.0870 0.1604 1.4452
sigma_a 8.0226 0.9547 6.4140 7.3560 7.9320 8.5990
sigma_b 2.1790 1.2991 0.2279 0.8318 2.4965 3.3910
sigma 2.2725 1.2624 0.2750 0.9528 2.5640 3.4360
deviance 583.7390 241.1451 37.2990 403.3750 695.3000 782.2000
 9.2 The BUGS Project: WinBUGS and OpenBUGS 181

The output provides summaries of the marginal posterior distributions of

the model parameters, including mean, standard deviation, and the 0.025,
0.25, 0.5, 0.75, and 0.975 quantiles. Inspecting these elements one can,
for example, report an estimate for β3 (coefficient of the covariate z3) as
4.3456 and a 95% credible interval as (2.396,6.373). To obtain HPD inter-
vals one can use CODA or BOA (as will be demonstrated in the subsections
explaining these packages).
8. Some more information is available in Cexemplo1_OpenBUGS.fit.
Using the command
names(Cexemplo1_OpenBUGS.fit)

we see the members of this list:

[1] "n.chains" "n.iter"
[3] "n.burnin" "n.thin"
[5] "n.keep" "n.sims"
[7] "sims.array" "sims.list"
[9] "sims.matrix" "summary"
[11] "mean" "sd"
[13] "median" "root.short"
[15] "long.short" "dimension.short"
[17] "indexes.short" "last.values"
[19] "isDIC" "DICbyR"
[21] "pD" "DIC"
[23] "model.file"

For example, get information about the DIC with

Cexemplo1_OpenBUGS.fit$DIC
[1] 429.7
Cexemplo1_OpenBUGS.fit$pD
[1] -154.1

The negative value of pD could indicate an excessive number of parameters

in the model. In this case, this could be due to the inclusion of the random
effects b in the model.
9. Finally, it is critical to assess convergence by using some of the meth-
ods discussed in Chapter 6. Either the CODA package, or the BOA package
in R can be used (see Section 9.6).
10. A complete analysis of the model should include model selection
and validation. All this can be done in R. It suffices to have the simulated
values of the model parameters. Those values are available in the form of a
list, array, or matrix. The latter, for example, is obtained with the command
A<-Cexemplo1_OpenBUGS.fit$sims.matrix
dim(A)
182 Software

[1] 20000 12
head(A)# shows the first 6 rows of A
beta0 beta[1] beta[2] beta[3] beta[4] tau
[1,] 15.45 8.683 0.066 4.086 -0.118 0.422
[2,] 16.44 5.370 0.123 4.398 -0.117 0.071
[3,] 19.35 2.387 0.214 4.675 -0.037 2.613
[4,] 19.12 -0.014 0.152 4.254 -0.084 0.361
[5,] 17.49 3.548 0.068 5.744 -0.104 2.452
[6,] 19.98 3.522 0.384 3.826 -0.106 0.092
tau_a tau_b sigma_a sigma_b sigma deviance
[1,] 0.017 0.078 7.640 3.589 1.539 536.6
[2,] 0.016 2.567 7.895 0.624 3.749 790.8
[3,] 0.019 0.074 7.169 3.665 0.619 249.5
[4,] 0.016 0.107 7.853 3.061 1.665 570.3
[5,] 0.015 0.079 8.067 3.558 0.639 286.1
[6,] 0.020 3.179 7.058 0.561 3.298 791.9

Note that only parameters that were declared in the object Cexemplo1.para
appear. To carry out residual analysis, for example, it is important to mon-
itor mu.
If the random effects b are removed (note that the file Cexemplo1BUGS.tx
with the model definition and the initial values need to be changed accord-
ingly) we get
> exemplo1_OpenBUGS1.fit<- bugs(data=Cexemplo1.data,
inits=list(Inits1), parameters.to.save=Cexemplo1.params1,
"Cexemplo1BUGS_semb.txt", n.chains=1,
n.iter=40000,n.burnin=20000, debug=FALSE,save.history=FALSE,
DIC=TRUE)

> exemplo1_OpenBUGS1.fit$summary
mean sd 2.5% 25% 50% 75% 97.5%
beta0 17.210 1.763 13.760 16.040 17.190 18.360 20.790
beta[1] 4.781 2.393 0.063 3.186 4.731 6.417 9.495
beta[2] 0.152 0.145 -0.117 0.052 0.149 0.249 0.450
beta[3] 4.146 0.923 2.384 3.527 4.121 4.751 6.053
beta[4] -0.107 0.036 -0.177 -0.131 -0.107 -0.083 -0.036
tau 0.077 0.011 0.057 0.070 0.077 0.085 0.101
tau_a 0.016 0.004 0.010 0.014 0.016 0.019 0.025
sigma_a 7.974 0.936 6.363 7.319 7.897 8.544 10.020
sigma 3.625 0.265 3.150 3.439 3.608 3.794 4.190
deviance 795.126 12.624 772.600 786.200 794.300 803.200 822.000

> exemplo1_OpenBUGS1.fit$DIC
[1] 843.2
> exemplo1_OpenBUGS1.fit$pD
[1] 48.03
> A1<-exemplo1_OpenBUGS1.fit$sims.matrix
> dim(A1)
[1] 20000 10
 9.3 JAGS 183

> head(A1)
beta0 beta[1] beta[2] beta[3] beta[4] tau
[1,] 20.62 -0.8067 0.1487 4.631 -0.1005 0.0838
[2,] 18.39 2.8580 0.0692 5.767 -0.0569 0.0826
[3,] 19.56 2.5330 0.3619 3.045 -0.1277 0.0784
[4,] 16.61 6.1660 -0.1078 4.284 -0.1168 0.0827
[5,] 18.11 2.8790 0.1680 4.372 -0.2055 0.0636
[6,] 15.06 5.1330 0.0461 3.707 -0.1142 0.0639
tau_a sigma_a sigma deviance
[1,] 0.0188 7.291 3.455 773.9
[2,] 0.0141 8.431 3.479 781.4
[3,] 0.0247 6.368 3.572 796.0
[4,] 0.0162 7.844 3.478 783.5
[5,] 0.0251 6.310 3.964 819.5
[6,] 0.0168 7.704 3.955 818.8

> Cexemplo1_OpenBUGS1$DIC
[1] 843.1
> Cexemplo1_OpenBUGS1$pD
[1] 48.12

Note that pD is now positive, with pD = 48.12.

9.3 JAGS
In 2003 Martyn Plummer, of the International Agency for Research on
Cancer, created JAGS (Just Another Gibbs Sampler) as a clone of BUGS
written in C++, which fixed certain limitations of BUGS. Models writ-
ten in BUGS syntax can be used in JAGS, practically without any change.
This has the advantage that the same code can be run on both platforms.
There are two parts to a model definition in JAGS: the model description
(model{}), as in BUGS, and the definition of the data (data{}). The lat-
ter can be used, for example, to define transformations of the data, define
summary statistics, simulate data sets, etc. The latest version of JAGS was
launched in July 2017 (JAGS 4.3.0). To understand how JAGS works, it
is important to read the manual (Plummer, 2012). The R package R2jags
(https://cran.r-project.org/web/packages/R2jags/R2jags.pdf) serves
as an interface between R and JAGS. One of the main advantages of JAGS
compared to OpenBUGS is the faster run time.

9.3.1 Application Example: Using R2jags

The following example illustrates how to use R2jags to study the model in
the example given in Section 9.1 without the random effects b. The program
is installed and loaded with the commands
184 Software

install.packages("R2jags", dependencies=TRUE,
repos="http://cran.us.r-project.org")

library(R2jags)

1. The same model as in OpenBUGS can be loaded as a file, or can be

defined in an R script, as a function, as follows:
exemplo1.model<-function(){
for(i in 1:147){
X[i]~dnorm(mu[i],tau)
mu[i]<-beta0+beta[1]*z1[i]+beta[2]*z2[i]+beta[3]*z3[i]
+beta[4]*z[i]+a[ID[i]]
}
for(j in 1:49){
a[j]~dnorm(0,tau_a)
}
for(k in 1:4){
beta[k]~dnorm(0,0.0001)
}
beta0~dnorm(0,0.0001)
tau~dgamma(0.05,0.05)
tau_a~dgamma(0.05,0.05)
sigma<-1/sqrt(tau)
sigma_a<-1/sqrt(tau_a)
}

2. Reading in the data, the definition of the model variables, the decla-
ration of parameters that are to be monitored, and the definition of initial
values is done exactly as before.
Cexemplo1.data <- list("X","ID","z3","z1","z2","z")
Cexemplo1.params1 <- c("beta0","beta","tau","tau_a",
"sigma_a","sigma")

Inits1<-list("tau"=1,"tau_a"=1,"beta"=c(0,0,0,0),
"a"=c(0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0),
"beta0"=0)

3. Before using R2jags for the first time, we need to set a random variate
seed, for example, by the R command
set.seed(123)

4. To estimate the model in JAGS, use the function jags().

exemplo1_JAGS.fit <- jags(data = Cexemplo1.data,
inits = list(Inits1), parameters.to.save = Cexemplo1.params1,
n.chains = 1, n.iter = 40000,
n.burnin = 20000, model.file = exemplo1.model)
 9.3 JAGS 185

5. Summary statistics of the marginal posterior distributions of the pa-

rameters are obtained by the command
> print(exemplo1_JAGS.fit)
Inference for Bugs model at "C:/Users...model1f5469ec52a3.txt",
fit using jags,
1 chains, each with 40000 iterations (first 20000 discarded),
n.thin = 20, n.sims = 1000 iterations saved
mu.vect sd.vect 2.5% 25% 50% 75% 97.5%
beta[1] 4.751 2.537 -0.248 3.038 4.783 6.507 9.450
beta[2] 0.160 0.143 -0.121 0.069 0.159 0.249 0.447
beta[3] 4.163 0.915 2.467 3.552 4.132 4.777 6.081
beta[4] -0.107 0.036 -0.177 -0.132 -0.108 -0.081 -0.031
beta0 17.212 1.810 13.561 16.050 17.151 18.387 20.717
sigma 3.676 0.808 3.133 3.441 3.604 3.789 4.267
sigma_a 7.920 1.159 6.289 7.335 7.921 8.529 9.778
tau 0.077 0.013 0.055 0.070 0.077 0.084 0.102
tau_a 0.699 13.406 0.010 0.014 0.016 0.019 0.025
deviance 797.267 28.396 773.207 786.080 794.391 802.998 823.233

DIC info (using the rule, pD = var(deviance)/2)

pD = 403.2 and DIC = 1200.4
DIC is an estimate of expected predictive error
(lower deviance is better).

Note the thinning out of saved simulations (n.thin=20). In fact, one

of the arguments of the function jags() is n.thin. Its default value
is max(1, floor((n.iter - n.burnin)/1000)); note also the ele-
vated value of pD.
6. Another feature of R2jags is the possibility to leave the choice of
initial values up to the program, by specifying NULL, as illustrated:
exemplo1_JAGS.fit2 <- jags(data = Cexemplo1.data, inits = NULL,
parameters.to.save = Cexemplo1.params1,
n.chains = 2, n.iter = 40000,
n.burnin = 20000, model.file = exemplo1.model)

> print(exemplo1_JAGS.fit2)
Inference for Bugs model at
"C:/Users.../model1f5477c03ee1.txt",
fit using jags,
2 chains, each with 40000 iterations (first 20000 discarded),
n.thin = 20
n.sims = 2000 iterations saved
mu.vect sd.vect 2.5% 25% 50% 75% 97.5%
beta[1] 4.859 2.596 -0.346 3.166 4.939 6.604 9.950
beta[2] 0.157 0.139 -0.113 0.068 0.151 0.242 0.453
beta[3] 4.191 0.918 2.422 3.595 4.173 4.805 6.059
beta[4] -0.105 0.036 -0.175 -0.130 -0.106 -0.081 -0.035
beta0 17.218 1.816 13.857 15.996 17.207 18.435 20.716
186 Software

sigma 3.659 0.715 3.164 3.436 3.608 3.798 4.235

sigma_a 7.943 1.049 6.363 7.317 7.886 8.587 9.873
tau 0.077 0.012 0.056 0.069 0.077 0.085 0.100
tau_a 0.507 14.898 0.010 0.014 0.016 0.019 0.025
deviance 796.175 22.494 773.690 786.454 794.169 802.528 822.873
Rhat n.eff
beta[1] 1.001 2000
beta[2] 1.003 590
beta[3] 1.001 2000
beta[4] 1.001 2000
beta0 1.001 2000
sigma 1.006 2000
sigma_a 1.040 2000
tau 1.006 2000
tau_a 1.040 2000
deviance 1.022 2000

For each parameter, n.eff is a crude measure of effective

sample size, and Rhat is the potential scale reduction
factor (at convergence, Rhat=1).

DIC info (using the rule, pD = var(deviance)/2)

pD = 253.1 and DIC = 1049.3
DIC is an estimate of expected predictive error
(lower deviance is better)

Note that there are now two extra columns, “Rhat” and “n.eff,” whose
meanings are explained in the output.
A plot of simulated parameter values can be obtained by
traceplot(exemplo1_JAGS.fit2)

7. The plots show obvious convergence problems. If needed, such as

in this example, one can continue simulation until convergence using the
following command (which can only be used with at least two chains –
notice n.chains=2 in the call to jags).
exemplo1_JAGS.fit2.upd <- autojags(exemplo1_JAGS.fit2)

print(exemplo1_JAGS.fit2.upd)
Inference for Bugs model at
"C:/Users.../model1f5477c03ee1.txt",
fit using jags,
2 chains, each with 1000 iterations (first 0 discarded)
n.sims = 2000 iterations saved
mu.vect sd.vect 2.5% 25% 50% 75% 97.5%
beta[1] 4.681 2.562 -0.227 2.972 4.702 6.345 9.832
beta[2] 0.156 0.143 -0.117 0.059 0.151 0.253 0.440
beta[3] 4.179 0.924 2.410 3.575 4.181 4.771 5.951
beta[4] -0.106 0.037 -0.178 -0.132 -0.106 -0.081 -0.036
beta0 17.288 1.777 13.735 16.117 17.333 18.443 20.676
 9.4 Stan 187

sigma 3.631 0.265 3.139 3.444 3.611 3.801 4.210

sigma_a 8.007 0.955 6.314 7.363 7.936 8.582 10.080
tau 0.077 0.011 0.056 0.069 0.077 0.084 0.102
tau_a 0.016 0.004 0.010 0.014 0.016 0.018 0.025
deviance 795.438 12.641 773.639 786.407 794.608 802.980 823.357
Rhat n.eff
beta[1] 1.001 2000
beta[2] 1.001 2000
beta[3] 1.001 2000
beta[4] 1.001 2000
beta0 1.001 2000
sigma 1.002 2000
sigma_a 1.001 2000
tau 1.002 2000
tau_a 1.001 2000
deviance 1.002 2000

For each parameter, n.eff is a crude measure of effective

sample size, and Rhat is the potential scale reduction
factor (at convergence, Rhat=1).

DIC info (using the rule, pD = var(deviance)/2)

pD = 79.9 and DIC = 875.4
DIC is an estimate of expected predictive error
(lower deviance is better).

Note the decrease of the values for pD and DIC. Another function that can
be used for the same purpose, and which requires only one chain, is the
function update().
In Section 9.6 we will discuss how to evaluate convergence diagnostics
for this example, including also for posterior simulation using other pack-
ages.

9.4 Stan
In 2010, Andrew Gelman, Columbia University, New York, and collabora-
tors were working on a Bayesian analysis of multi-level generalized linear
models, as described by Gelman and Hill (2006). The implementation of
these models in WinBUGS or JAGS turned out to be extremely challeng-
ing due to the complex model structure. For example, Matt Schofield noted
that simulation for a multi-level time series model that was used for a cli-
mate model using measurements of growth rings on trees, did not converge
even after hundreds of thousands of iterations (Schofield et al, 2016). To
resolve the problem, Gelman and collaborators developed a new Bayesian
software, which they called Stan, in honor of Stanislaw Ulam, one of the
creators of the Monte Carlo methods. The first version was made available
188 Software

for users in August 2012. Stan does not use Gibbs samplers, but instead
uses Hamiltonian Monte Carlo (Neal, 2011). With the no-U-turn sampler
algorithm that they developed (Hoffman and Gelman, 2014), all parame-
ters are simulated in one block. Using this strategy, convergence problems
were substantially mitigated. In contrast to BUGS, Stan is written as an
imperative language.
Stan allows the use of all basic operators from C++, in addition to a large
number of other special functions, including in particular Bessel functions,
gamma and digamma functions, and a variety of functions related to inverse
functions that are used in generalized linear models. A complete list of the
basic operators and special functions that are implemented in Stan is given
by Carpenter et al (2017). The list of implemented probability distributions,
which also appears in the same reference, is extensive. This allows great
flexibility in model construction.
A bit of history of the development of Stan, as well as implementation
details, can be found in Stan Modeling Language: User’s Guide and Ref-
erence Manual, which corresponds to version 2.6.2. An interface between
R and Stan is implemented in RStan (Stan Development Team, 2014).

9.4.1 Application Example: Using RStan

To install RStan, follow the instructions at https://github.com/stan-d
rstan/wiki/RStan-Getting-Started.
1. As in the other packages, one has to first define the model in Stan.
The model definition is saved in a text file, typically with the suffix .stan.
The model definition is a little more elaborate than in the previous pack-
ages. Being an imperative language, the sequence of commands is im-
portant. For our example, the model definition in Stan, saved in the file
exemplo1.stan.txt, is
data {
int<lower=1> J; // length of data
int<lower=1> N; // number of patients
real X[J]; // response variable
real z2[J]; // covariate
real z3[J]; // covariate
real z[J]; // covariate
int<lower=0,upper=1> z1[J]; // z1 takes values 0 and 1
int<lower=1> ID[J]; // identification
}
parameters {
real<lower=0> tau;
real<lower=0> tau_a;
 9.4 Stan 189

real beta0;
real beta[4];
real a[N];
}
transformed parameters {
real<lower=0> sigma_a;
real<lower=0> sigma;
sigma_a = sqrt(1/tau_a);
sigma = sqrt(1/tau);
}
model {
vector[J] mu;
for(i in 1:J){
mu[i] = beta0+beta[1]*z1[i]+beta[2]*z2[i]
+beta[3]*z3[i]+a[ID[i]]+beta[4]*z[i]};
beta0 ~ normal(0,100);
beta ~ normal(0,100);
tau ~ gamma(0.05,0.05);
tau_a ~ gamma(0.05,0.05);
a ~ normal(0, sigma_a);
X ~ normal(mu, sigma);
}
generated quantities {
vector[J] log_lik;
vector[J] m;
for(i in 1:J) {
m[i] = beta0+beta[1]*z1[i]+beta[2]*z2[i]
+beta[3]*z3[i]+a[ID[i]]+beta[4]*z[i]};
log_lik[i] = normal_log(X[i] | m[i], sigma);
}
}

We explain the program block by block:

• In the data block the data to be used when running Stan have to be
declared. For example, J and N are integers with minimum value 1; z1
is a binary covariate with integer value between 0 and 1. The response
variable and the covariates are real vectors. Stan also allows data in the
form of a matrix, ordered vectors, arrays, and more.
• The parameters block declares all unknown quantities that Stan will
estimate. In this case we decided to include the precision parameters tau
and tau_a to be consistent with the earlier model definition. Addition-
ally, we declare variable types for all quantities that are to be estimated.
For example, tau and tau_a are postive reals; beta and a are real vec-
tors of dimensions 4 and N, respectively.
• In the transformed parameters block, all quantities that are func-
tions of the data and/or the parameters have to be declared if they are to
190 Software

be used later. In this case we define sigma and sigma_a as the square
root of the inverse precision parameters.
• The model block defines the actual model. The vector mu is defined
as a function of the covariates and the random effects a. We define the
model for the response variable (note that the second parameter of the
normal distribution is the standard deviation here). We also define prior
distributions for the parameters and hyperparameters in the model. In
Stan the definition of prior distributions is not obligatory. In the absence
of such definitions, Stan proceeds with uniform prior distributions.
• The generated quantities block is not required. Here we define the
log-likelihood in a way that individual terms can be saved by Stan. These
terms are collected in an object with the name log-lik; the simulated
values of this object can then be used to calculate the widely applicable
information criterion (WAIC) or the LOO (Vehtari et al, 2017), as we
shall see.

2. The data are introduced as a list, as before, or if the data are read from
a file it suffices to create an object with the variable names, as follows:
Cexemplo1<-read.table("Cexemplo1.txt",header=T)
attach(Cexemplo1)
J<-nrow(Cexemplo1) #J=147
N<-length(unique(ID)) #N=49
# object to be used by Stan
Cexemplo1_data<-c("N","J","X","ID","z3","z2","z1","z")

3. Next we can call the function stan() from the package rstan to
simulate from the posterior distribution:
library(rstan)
exemplo1.fit_stan <- stan(file="exemplo1.stan.txt",
data=Cexemplo1_data, iter=40000, chains=2)

Call ?stan to see the arguments of the function stan. Particularly use-
ful are the arguments sample_file and diagnostic_file that allow
indicating the names of files where simulated samples of all model pa-
rameters and convergence diagnostics can be saved. If these names are not
given, these elements are not saved. They can, however, be extracted later.
rstan provides information about the execution time for each chain in
the following form (here only for one chain):
COMPILING THE C++ CODE FOR MODEL ’example1’ NOW.

SAMPLING FOR MODEL ’exempl1’ NOW (CHAIN 1).

Chain 1, Iteration: 1 / 40000 [ 0%] (Warmup)

 9.4 Stan 191

Chain 1, Iteration: 4000 / 40000 [ 10%] (Warmup)

Chain 1, Iteration: 8000 / 40000 [ 20%] (Warmup)
Chain 1, Iteration: 12000 / 40000 [ 30%] (Warmup)
Chain 1, Iteration: 16000 / 40000 [ 40%] (Warmup)
Chain 1, Iteration: 20000 / 40000 [ 50%] (Warmup)
Chain 1, Iteration: 20001 / 40000 [ 50%] (Sampling)
Chain 1, Iteration: 24000 / 40000 [ 60%] (Sampling)
Chain 1, Iteration: 28000 / 40000 [ 70%] (Sampling)
Chain 1, Iteration: 32000 / 40000 [ 80%] (Sampling)
Chain 1, Iteration: 36000 / 40000 [ 90%] (Sampling)
Chain 1, Iteration: 40000 / 40000 [100%] (Sampling)
# Elapsed Time: 46.431 seconds (Warm-up)
# 75.594 seconds (Sampling)
# 122.025 seconds (Total)

The execution time for 40,000 iterations is much longer than the ex-
ecution time in OpenBUGS or JAGS, which for this particular problem
is practically instantaneous. However, the number of iterations needed to
achieve convergence is lower using Stan. Above we used the same number
of iterations, but this was not necessary. In fact, if the number of iterations
is not specified in iter=, we find convergence after only 2000 iterations
in this example.
4. To get summary statistics for the marginal posterior distributions for
the parameters of interest, use the command
print(exemplo1.fit_stan,
pars=c("beta0","beta","sigma","sigma_a","tau","tau_a","lp__"))

to get (a final column for the 97.5% quantile is cut to fit the page) the
following:
Inference for Stan model: example1.
2 chains, each with iter=40000; warmup=20000; thin=1;
post-warmup draws per chain=20000, total post-warmup draws=40000.

mean se_mean sd 2.5% 25% 50% 75%

beta0 17.22 0.02 1.79 13.67 16.04 17.23 18.41
beta[1] 4.80 0.03 2.52 -0.17 3.12 4.80 6.48
beta[2] 0.16 0.00 0.14 -0.12 0.06 0.16 0.25
beta[3] 4.19 0.01 0.93 2.38 3.55 4.18 4.82
beta[4] -0.10 0.00 0.04 -0.18 -0.13 -0.11 -0.08
sigma 3.62 0.00 0.26 3.16 3.44 3.61 3.79
sigma_a 8.00 0.01 0.94 6.39 7.34 7.92 8.57
tau 0.08 0.00 0.01 0.06 0.07 0.08 0.08
tau_a 0.02 0.00 0.00 0.01 0.01 0.02 0.02
lp__ -388.89 0.06 6.36 -402.47 -392.90 -388.50 -384.42
n_eff Rhat
beta0 5771 1
beta[1] 5754 1
192 Software

beta[2] 7261 1
beta[3] 7555 1
beta[4] 10087 1
sigma 21771 1
sigma_a 23821 1
tau 21546 1
tau_a 25151 1
lp__ 11599 1

For each parameter, n_eff is a crude measure of effective sample s

and Rhat is the potential scale reduction factor on split chains (
convergence, Rhat=1).

Here, se_mean is the Monte Carlo standard error of the mean. If no param-
eters are specified then the summary statistics are given for the marginal
posterior distributions of all unknown quantities, including in particular
also a, log_lik, m.
The quantity lp_ which appears as the last element in pars= and whose
summary statistics are found in the last row of the output, is the log pos-
terior density (un-normalized), which is calculated by Stan for the imple-
mentation of Hamiltonian Monte Carlo. It can also be used for model as-
sessment (see, for example, Vethari and Ojanen, 2012).
The print() function provides summary statistics based on all chains
that are generated, whereas summary() provides summary statistics for
each chain separately.
5. To obtain the simulated parameter values, use the function extract().
With the argument permuted=TRUE, a list of all simulated values for all
parameters is created (if only simulated values for some parameters are de-
sired, those need to be given in the argument pars=); using permuted=FALS
an array is created with the first dimension corresponding to iterations, the
second dimension corresponding to chains, and the third dimension corre-
sponding to the parameters. See, for example, the following:
#####using permuted=TRUE##############
samples_stan<-extract(exemplo1.fit_stan,
pars=c("beta0", "beta", "sigma", "sigma_a"),
permuted = TRUE, inc_warmup = FALSE, include = TRUE)
> class(samples_stan)
[1] "list"
> names(samples_stan)
[1] "beta0" "beta" "sigma" "sigma_a"
> length(samples_stan$beta0)
[1] 40000 #20000 each chain
> head(samples_stan$beta0)
[1] 16.18767 18.64417 20.43510 16.69809 14.35278 15.39996
> dim(samples_stan$beta)
 9.4 Stan 193

> head(round(samples_stan$beta,3))

iterations [,1] [,2] [,3] [,4]

[1,] 8.994 0.468 2.437 -0.126
[2,] 4.310 0.309 4.425 -0.093
[3,] 2.127 0.079 3.700 -0.156
[4,] 3.394 0.245 1.680 -0.131
[5,] 10.541 0.359 3.814 -0.086
[6,] 8.314 0.357 4.001 -0.068
#########using permuted=FALSE#########
> samples_stan_array<-extract(exemplo1.fit_stan,
+ pars=c("beta0", "beta", "sigma", "sigma_a"),
+ permuted = FALSE, inc_warmup = FALSE, include = TRUE)
> class(samples_stan_array)
[1] "array"
> dim(samples_stan_array)
[1] 20000 2 7 # 20000 each chain
# 2 chains, 7 parameters
> samples_stan_array[1:4,1:2,1:3]
, , parameters = beta0

chains
iterations chain:1 chain:2
[1,] 16.29099 17.81893
[2,] 16.68243 17.31063
[3,] 16.49383 17.31063
[4,] 16.20388 16.70740

, , parameters = beta[1]

chains
iterations chain:1 chain:2
[1,] 6.530125 5.718621
[2,] 4.949012 6.479835
[3,] 6.000288 6.479835
[4,] 6.204705 7.421142

, , parameters = beta[2]

chains
iterations chain:1 chain:2
[1,] 0.1718956 0.07575568
[2,] 0.1657402 0.18286167
[3,] 0.1793824 0.18286167
[4,] 0.1347633 0.15160846

6. The traceplot() function plots the simulated parameter values for

each chain. Figure 9.1 is obtained by the following command:
traceplot(exemplo1.fit_stan,
pars=c("beta"), nrow = 5, ncol = 2, inc_warmup = FALSE)
194 Software

Figure 9.1 Simulated values.

7. To obtain the WAIC (Vehtari et al, 2017), proceed as follows, after

installing the package loo from CRAN:
> library(loo)
> log_lik1 <- extract_log_lik(exemplo1.fit_stan)
> waic(log_lik1)
Computed from 40000 by 147 log-likelihood matrix

Estimate SE
elpd_waic -422.4 9.9
p_waic 40.5 4.7
waic 844.8 19.8

For details on how to use Rstan, see https://cran.r-project.org/

web/packages/rstan/vignettes/rstan_vignette.pdf.

9.5 BayesX
BayesX is a program that was developed in the Department of Statistics
at Ludwig Maximilian Universität München by Andreas Brezger, Thomas
Kneib, and Stefan Lang, with the first versions appearing in 2002. The
software is written specifically for structured additive regression models
(Brezger et al, 2005). This family of models (STAR), which was introduced
 9.5 BayesX 195

in Chapter 8, includes various well-known and widely used regression

models, such as generalized additive models (GAM), generalized additive
mixed models (GAMM), generalized geoadditive mixed models (GGAMM),
dynamic models, spatio temporal models, and more, under one unified
framework (Umlauf et al, 2015). BayesX, written in C++, also allows the
analysis of regression models when the response variable does not neces-
sarily belong to the exponential family. It also allows the analysis of quan-
tile regression, survival regression by modeling the hazard function (ex-
tensions of the Cox model), and analysis of multi-state models and multi-
level models. The methodology manual for BayesX (www.statistik.lmu.
de/~bayesx/manual/methodology_manual.pdf) contains a brief description
of the regression models that are allowed in BayesX.
The particular arguments of the BayesX function to implement a sta-
tistical model include the specification of a distributional family for the
response variable, the estimation method, and other control parameters de-
fined by the function bayesx.control(), as will be seen. Similar to the
glm() function in R, the distributional family that is (implicitly) specified
by omission is a Gaussian family. A list of allowed probability distribu-
tions in BayesX, besides the usual exponential family distributions, can be
found in Umlauf et al (2015). A particular feature of BayesX is that, be-
sides MCMC, BayesX allows us to carry out inference for mixed models
using restricted maximum likelihood estimation (REML) and a penalized
likelihood method (STEP).
In principle, inference for the STAR models that are implemented in
BayesX could also be implemented using WinBUGS/OpenBUGS, or JAGS.
However, the BayesX authors report substantial reduction in execution
times compared to WinBUGS/OpenBUGS, besides a faster convergence of
the Markov chains (Brezger et al, 2005), which have better mixing proper-
ties. To simplify the use and subsequent analysis of results from BayesX,
Kneib et al (2014) created an R package, also called BayesX, which allows
reading and processing results from BayesX. However, with this package,
the users still have to read data, adapt the models, and obtain output files
using BayesX. To simplify this task, Umlauf et al (2015) introduce a new
R package, R2BayesX, which now provides a complete interface between
R and BayesX.
The R2BayesX manual can be found at https://cran.r-project.
org/web/packages/R2BayesX/R2BayesX.pdf. To install R2BayesX
it suffices, as usual, to use the commands
install.packages("R2BayesX", dependencies=TRUE,
repos="http://cran.us.r-project.org")
196 Software

9.5.1 Application Example: Using R2BayesX

The syntax used in R2BayesX to implement inference for a Bayesian model
is in all aspects very similar to the syntax used in R to implement statistical
models.
1. To implement the desired model it suffices, after reading in the data,
to write the formula of the model and call the bayesx() function.
Cexemplo1 <- read.table("Cexemplo1.txt",header=T)
library(R2BayesX)
modelo2_BayesX <- X~z2+z3+z1+z+sx(ID,bs="re")
exemplo1_BayesX <- bayesx(formula = modelo2_BayesX,
data = Cexemplo1, family = "gaussian", method = "MCMC",chains=2
seed = c(123,456),
control = bayesx.control(model.name = "bayes2x.estim",
outfile=’C:/...’, iterations = 40000L,
burnin = 20000L,dir.rm=T))

Explaining the code:

• The formula that is saved as modelo2_BayesX is the formula of the
linear predictor for the expected response value. Following the initial
model specification, the fixed effects z2, z3, z1, z enter linearly in
the linear predictor. BayesX is, however, particularly useful to model
non linear covariate effects by using the function sx(). For example,
if we had reason to believe that the covariate z2 has a non linear ef-
fect, and we would like to use a P-spline to model this non linear ef-
fect, then z2 should enter in the model with a model statement using
sx(z2,bs="ps"), where the argument bs selects the type of basis for
this term. We used it above to specify the random effects. See how the
random effect a is introduced using the sx() function. Recall that these
random effects are patient-specific, with patients indexed by ID. There-
fore, the first argument for the function sx() is ID. To specify that the
random effects are i.i.d. normal with mean zero and standard deviation
σa , we write bs="re". To specify a model with the random effects b,
which we used initially, one would write sx(all,bs="re") (recall that
all indexes all observations with a running index from 1 to 147). A list
of all arguments for BayesX can be found in table 4 of Umlauf et al
(2015).
• The function bayesx() fits the specified model. The function has sev-
eral arguments, many of which are optional arguments with default choices
The only required arguments are the first two: formula and data. By
 9.5 BayesX 197

default the distributional family is Gaussian, the method is MCMC,2

the number of iterations is 12,000, with a burn-in of 2000, and thin-
ning out by saving every tenth iteration, and the number of chains is
1. These values, as well as others, can be changed by the argument
control in the bayesx.control function; as usual, to find out the
arguments of this function and their use, use the help function in R by
typing ?bayesx.control. The first argument of this function is the
model.name, which takes the name of the model and is also used to
form the name of the files that are used to save the results of the model
estimation using BayesX; these files will be saved in the directory spec-
ified in outfile.

2. Using this code, we get the following output of summary statistics for
the marginal distributions of the parameters:
> summary(exemplo1_BayesX)
### Chain_1
Call:
bayesx(formula = formula, data = data, weights = weights,
subset = subset, offset = offset, na.action = na.action,
contrasts = contrasts, control = control, model = model,
chains = NULL, cores = NULL)

Fixed effects estimation results:

Parametric coefficients:
Mean Sd 2.5% 50% 97.5%
(Intercept) 17.2313 1.8036 13.6702 17.2744 20.7688
z2 0.1557 0.1371 -0.1174 0.1566 0.4250
z3 4.2146 0.9665 2.3040 4.2043 6.2029
z1 4.7691 2.4910 -0.1460 4.7646 9.6957
z -0.1043 0.0366 -0.1756 -0.1055 -0.0319

Random effects variances:

Mean Sd 2.5% 50% 97.5% Min Max
sx(ID):re 64.925 15.702 41.081 62.706 99.701 26.824 169.6

Scale estimate:
Mean Sd 2.5% 50% 97.5%
Sigma2 13.2389 1.9332 9.9936 13.0804 17.373

N = 147 burnin = 20000 DIC = 194.7823 pd = 48.29928

method = MCMC family = gaussian iterations = 40000
step = 10
### Chain_2
Call:
2
BayesX also implements REML and STEP; for details, see Umlauf et al (2015)
198 Software

bayesx(formula=formula, data=data, weights=weights,

subset=subset, offset = offset, na.action = na.action,
contrasts = contrasts, control = control,
model = model, chains = NULL, cores = NULL)

Fixed effects estimation results:

Parametric coefficients:
Mean Sd 2.5% 50% 97.5%
(Intercept) 17.1458 1.7125 13.9773 17.0971 20.3820
z2 0.1591 0.1438 -0.1282 0.1612 0.4407
z3 4.1544 0.9413 2.3008 4.1405 6.0312
z1 4.9990 2.5100 -0.2337 5.0116 9.6973
z -0.1025 0.0351 -0.1751 -0.1016 -0.0367

Random effects variances:

Mean Sd 2.5% 50% 97.5% Min Max
sx(ID):re 64.569 15.502 40.542 62.367 101.027 30.008 136.28

Scale estimate:
Mean Sd 2.5% 50% 97.5%
Sigma2 13.2323 1.9739 9.9179 13.0191 17.518

N = 147 burnin = 20000 DIC = 195.1921 pd = 48.54314

method = MCMC family = gaussian iterations = 40000 step = 10
###
Object consists of 2 models

When the argument control gives a file name for the output, folders are
automatically created in the indicated directory, one for each chain, con-
taining various files with the simulated parameter values (with file exten-
sion .raw) and text files with the summary statistics. In the current exam-
ple, two folders are created with the names Chain_1_bayes2x.estim
and Chain_2_bayes2x.estim. All these data files can later be used for
graphical summaries, to evaluate diagnostics, etc.
3. The command
getscript(exemplo1_BayesX)

returns an R script to create plots using these files.

4. Alternatively, the simulated posterior samples of the parameters can
be obtained with the samples() function as follows:
AA<-samples(exemplo1_BayesX)
> class(AA)
[1] "mcmc.list"
> names(AA)
[1] "Chain_1" "Chain_2"
> length(AA[[1]])
 9.5 BayesX 199

[1] 10000
> length(AA[[2]])
[1] 10000

plot(AA)

The object AA is a list that contains in AA[[1]] a sample from the poste-
rior distribution of the fixed effect parameters from chain 1, and similarly in
AA[[2]] for chain 2. Since we save 20,000 iterations and thin out to every
tenth, there are 2000 simulated values for each of the five parameters. The
order in which the simulated values appear is the same as that in which they
were introduced in the formula. Thus AA[[1]][1:2000] contains poste-
rior simulated values, starting first with beta0; AA[[1]][2001:4000]
which contains posterior simulated values for beta1, the coefficient for
the covariate z2, etc.
5. The command plot(AA) plots the superimposed series of simulated
values for each parameter and corresponding marginal posterior density
estimates.
6. To get the posterior simulated values of the variances (in this case, the
variance of a normal distribution), and the corresponding graphical sum-
maries, write
> Va<-samples(exemplo1_BayesX,term = "var-samples")
> length(Va[[1]])
[1] 2000
> plot(Va)

Figure 9.2 shows the plot of plot(Va).

7. To get simulated values for the random effects a and for σ2a use the
term="sx(ID)" argument in the samples function. This way one can ob-
tain posterior samples for specific parameters; for example, use term="z3"
for the parameter corresponding to z3.
8. A useful feature is that the folder that is created for the results from
each chain also includes a latex document with a model summary, in-
cluding information about the assumed prior distributions and a summary
of the results. For example, consulting this document when no specific pri-
ors were specified, one finds that diffuse priors were used for the fixed
effects, i.i.d. normal random effects, and inverse gamma priors for the vari-
ance components with hyperparameters a = 0.001 and b = 0.001. If de-
sired, the specification of hyperparameters can be adjusted in the control
argument.
200 Software

Figure 9.2 Trace plot of simulated values and marginal posterior

density for σ2 .

The information from this latex document can also be obtained with
the command
bayesx_logfile(exempl1_BayesX)

9.6 Convergence Diagnostics: the Programs CODA and BOA

As already mentioned in Section 6.6, MCMC methods generate realiza-
tions from a homogeneous Markov chain with an equilibrium distribution
that matches the desired posterior distribution h(θ | x). The aim is then to
obtain one (or more) sequence(s) of parameter values which can be consid-
ered a representative sample from the joint posterior distribution. To obtain
one such sample, it is necessary to assess, based on a given number of itera-
tions t, if the chain is already in (or “close to”) the equilibrium state. If that
is the case, then the currently imputed states and the following iterations
can be considered to be approximate Monte Carlo samples from the poste-
rior distribution h(θ | x), of course correlated due to the Markovian nature
of the chain. There are several methods, graphical and based on statistical
tests, that allow us to diagnose convergence of the chain or chains to the
stationary distribution. Some were already mentioned in Section 6.6. The
packages CODA (Plummer et al, 2006) and BOA (Smith, 2007) implement
these methods, allowing one to carry out a quick and efficient assessment
of convergence.
 9.6 Convergence Diagnostics: the Programs CODA and BOA 201

9.6.1 Convergence Diagnostics

The CODA program as well as BOA allow the evaluation of convergence
diagnostics, in particular the methods by Gelman and Rubin (1992), Geweke
(1992), Raftery and Lewis (1992), and Heidelberger and Welch (1983),
which will be briefly described in the following. A more detailed descrip-
tion can be found in the articles cited in Cowles and Carlin (1996) or
Paulino et al (2018).

Gelman and Rubin Diagnostic

Gelman and Rubin suggest using the variance components of multiple par-
allel chains, initialized from different starting points. The method involves
the following steps:
1. Simulate m ≥ 2 chains, each one with 2n iterations, starting from initial
points that are generated from a distribution that is overdispersed relative
to the target (stationary) distribution.
2. Discard the first n iterations of each chain.
3. Let g denote the scalar quantity of interest that is desired to be estimated
(g is typically a function of the parameter θ).
4. On the basis of the simulated values of g, calculate the variance compo-
nents W and B, that is, the variance within each chain and the variance
across chains, respectively.
5. Estimate the mean of g under the target distribution as a sample average
of all mn simulated values of g.
6. Estimate V, the variance of g(θ) under the target distribution, as a weighted
average of W and B. √
7. Evaluate the scale reduction factor R̂ = V/W.
8. The ratio converges to 1 as n → ∞. Values R̂ ≈ 1 are evidence that
each of the m sequences of n simulated states is approximating the target
distribution.

Geweke’s Diagnostic
t
Let θ , t = 1, ..., N, be a sequence of states generated by an MCMC simu-
lation and let g(θ) be a function of θ that is to be estimated. The trajectory
g1 , g2 , . . . formed by gt = g(θt ) defines a time series.
The method of Geweke (1992) is based on the application of usual time
series methods to test for the convergence of the simulated sequence. As-
P
suming sufficiently large N, we calculate the sample average ga = n1a g(θt )
P
using the first na iterations, as well as the average gb = n1b g(θt ) using the
 202 Software

last nb iterations. If the chain is stationary, then the mean of the first part
of the chain should be similar to the mean of the latter part of the chain.
Letting N → ∞ with fixed na /N and nb /N, one can show that
(ga − gb )
q → N(0, 1),
(s2a /na ) + (s2b /nb )

where s2a and s2b are independent estimates of the asymptotic variances of
ga and gb , adjusted for the autocorrelation of the time series. Using this
statistic, we can now assess whether or not there is evidence for conver-
gence.

Raftery and Lewis Diagnostic

Assume that it is desired to estimate the posterior q-quantile of some func-
tion of the parameters, with numerical uncertainty r and probability s to
be within the bounds defined by r. The method of Raftery and Lewis cal-
culates the necessary number of iterations N and initial burn-in M needed
to achieve the specified criteria. The calculation is based on assuming that
a derived binary sequence of indicators of the function being above (1) or
below (0), the desired quantile is approximately Markov. Besides N and M
the diagnostic reports Nmin , the minimum size of a pilot sample and refers
to I = (M + N)/Nmin as the factor of dependence, interpreted as the propor-
tional increase in the number of iterations that can be attributed to the serial
dependence. High values of this factor (> 5) are evidence for influential
starting values, high correlation between the coefficients of the parameter
vector, or a poorly mixing Markov chain over the posterior support. The
method should be used with one or more pilot sequences.

Heidelberg and Welch Method

Heidelberg and Welch proposed a test statistic, based on the Cramer-von
Mises test, to test the null hypothesis that the simulated Markov chain does
come from a stationary distribution.
The convergence diagnostic is applied for each variable that is being
monitored, and is evaluated as follows:
1. Generate a chain of size N and define a level α.
2. For each monitored variable, evaluate the test statistic using the N itera-
tions. According to the result of this test, take a decision about rejecting
the null hypothesis or not.
3. If the null hypothesis is rejected, evaluate the test statistic again after
 9.6 Convergence Diagnostics: the Programs CODA and BOA 203

discarding the first 10% of iterations. Repeat the process while the null
hypothesis is rejected.
4. If we continue to reject the null hypothesis as the remaining number of
iterations reaches 50% of the initial N iterations, then the Markov chain
simulation has to continue, as the chain has not yet reached equilibrium.
In that case CODA reports the test statistic and indicates that the chain
has failed the test of stationarity.
5. Otherwise, the part of the chain that passed the stationarity test is used to
estimate the mean (m) and asymptotic standard error (s) of the average
and a half-width test is applied to that part of the chain as follows. If
1.96s < mǫ, for small ǫ (CODA uses a default of α = 0.05 and ǫ = 0.1),
then the chain passes the overall test. Otherwise, the condition 1.96s ≥
mǫ means that the Markov chain simulation needs to be continued.

9.6.2 The CODA and BOA Packages

The CODA package was originally written for S-PLUS, as part of a bio-
statistics PhD thesis by Cowles (1994). Later it was taken over by the
BUGS team (Best et al, 1995), who created an interface by saving the sim-
ulated values from BUGS in a CODA format, which could then later be
analyzed by CODA.
The CODA package for R arose from an effort to move the functions that
were written for S-PLUS to an R environment. Difficulties in this transla-
tion lead to substantial rewriting of the functions, and the creation of the
BOA package. The development of the latter started in 2000, exactly when
all CODA functions were already rewritten to be used in R. As a conse-
quence, CODA is now available for use in R (Plummer et al, 2006).
Both packages can be directly installed from CRAN with the commands:
install.packages("coda",repos="http://cran.us.r-project.org")
install.packages("boa",repos="http://cran.us.r-project.org")

Both packages, CODA and BOA from CRAN, have a function codamenu()
and boa.menu(), respectively, which allows using the programs with a
menu interface, for casual users with limited knowledge of R. For exam-
ple, in CODA:
> library(coda)
> codamenu()
CODA startup menu

1: Read BUGS output files

2: Use an mcmc object
204 Software

3: Quit

Selection: 2

Enter name of saved object (or type "exit" to quit)

1:A1_2_mcmc #ver posteriormente como foi definido.
Checking effective sample size ...OK
CODA Main Menu

1: Output Analysis
2: Diagnostics
3: List/Change Options
4: Quit

After reading in the data, the user is presented with a list of analysis op-
tions, summary statistics, graphical representations, and convergence diag-
nostics.
The BOA menu is quite similar. See details of the boa.menu() in Smith
(2007).
Alternatively to the menu mode, it is possible to carry out an analysis
using R commands. To have an interface between CODA and R such that
functions can be used from the R command line, a new R class mcmc was
created. BOA accepts the simulated values as a vector or matrix as input
parameter, and can save them as an mcmc object.
The latest version of the manuals for CODA and BOA are available from
https://cran.r-project.org/web/packages/coda/coda.pdf
https://cran.r-project.org/web/packages/boa/boa.pdf.

The manuals describe the functions to summarize the MCMC simulation

results, including graphical summaries, as well as diagnostic tests for con-
vergence to the limiting distribution of the chains.
The CODA functions require an object of the class mcmc which contains
all simulated values for all parameters that are being monitored. Such an
object can easily be created using the function as.mcmc() with a matrix
of simulation output.
To use the BOA functions, the MCMC simulation output needs to be
given in a matrix, with the parameters in the columns and one row for each
iteration. A list with row and column names is the argument dimnames()
of the class matrix object.
 9.6 Convergence Diagnostics: the Programs CODA and BOA 205

9.6.3 Application Example: CODA and BOA

We illustrate the use of CODA and BOA to study convergence of the chains
that were previously simulated using the packages R2OpenBUGS, R2jags,
RStan and R2BayesX.

A. Use with R2OpenBUGS

Recall from Section 9.2.1 how the matrix of simulated values for param-
eters that were previously declared for monitoring can be obtained. It is
obtained as the component $sims.matrix of the object
A1=Cexemplo1_OpenBUGS.fit. The latter contains all output from the
call to bugs(), as is illustrated in the following example:
Cexemplo1_OpenBUGS.fit<- bugs(data=Cexemplo1.data,
inits=list(Inits1,Inits2), parameters.to.save=Cexemplo1.params1,
"Cexemplo1BUGS_semb.txt", n.chains=2, n.iter=40000,
n.burnin=20000, debug=FALSE,save.history=FALSE,DIC=TRUE)

> A1<-Cexemplo1_OpenBUGS.fit$sims.matrix
> dim(A1)
[1] 40000 10
> head(round(A1,4))
beta0 beta[1] beta[2] beta[3] beta[4] tau tau_a
[1,] 15.73 6.349 0.0098 3.843 -0.1046 0.0819 0.0192
[2,] 18.55 2.689 0.0214 4.742 -0.1315 0.0953 0.0195
[3,] 16.41 6.330 0.2284 4.585 -0.0643 0.0664 0.0218
[4,] 14.18 5.653 -0.1744 4.911 -0.1551 0.0793 0.0127
[5,] 19.86 2.291 0.0826 4.259 -0.1209 0.0875 0.0180
[6,] 19.00 1.449 -0.0214 5.277 -0.0964 0.0778 0.0190
sigma_a sigma deviance
[1,] 7.207 3.495 797.3
[2,] 7.168 3.240 792.8
[3,] 6.779 3.882 800.5
[4,] 8.883 3.552 781.2
[5,] 7.452 3.381 793.0
[6,] 7.247 3.585 783.9
> class(A1)
[1] "matrix"

We see that A1 has 40,000 rows (20,000 iterations for each of two chains)
and ten columns with the names of the monitored parameters. Note that the
states for the first chain are in lines 1 through 20,000, and for the second
chain in rows 20,001 through 40,000.
Thus, to use CODA with two chains, we need to define two objects of
type mcmc, one for each chain, and then combine them into a single object
using the function as.mcmc.list, as is shown in the following example.
> library(coda)
206 Software

> A1_1chain<-as.mcmc(A1[1:20000,])
> A1_2chain<-as.mcmc(A1[20001:40000,])
> A1_2_mcmc<-as.mcmc.list(list(A1_1chain,A1_2chain))

We can plot superimposed trajectories for the two chains, posterior den-
sities (illustrated in Figure 9.3 for the parameters β1 and β2 ), autocorre-
lations, trajectories of quantiles, and the correlation matrix between the
components. The plots are obtained using the following commands:
plot(A1_mcmc)
plot(A1_2_mcmc[,2:3]) #only for cols that appear in the figure
autocorr.plot(A1_2_mcmc) #autocorrelation
cumuplot(A1_2_mcmc) #evaluate quantiles (0.025,0.5,0.975)
crosscorr.plot(A1_2_mcmc)#plot correlation matrix

The convergence diagnostics from Section 9.6.1 are easily evaluated with
the respective CODA functions.

Figure 9.3 Trajectory plots and posterior densities for β1 and β2 .

1. Gelman and Rubin’s Diagnostic

> gelman.diag(list(A1_1chain,A1_2chain),
confidence = 0.95, transform=FALSE, autoburnin=TRUE,
multivariate=TRUE)
 9.6 Convergence Diagnostics: the Programs CODA and BOA 207

Potential scale reduction factors:

Point est. Upper C.I.

beta0 1 1
beta[1] 1 1
beta[2] 1 1
beta[3] 1 1
beta[4] 1 1
tau 1 1
tau_a 1 1
sigma_a 1 1
sigma 1 1
deviance 1 1

Multivariate psrf
1

The command included the option autoburnin=TRUE. Therefore, only

the second half of the series is used for the evaluation of the scale reduc-
tion factor. The factors evaluate to 1, indicating no evidence against con-
vergence of the series. A plot of the evolution of Gelman and Rubin’s scale
reduction factor versus iteration is obtained by gelman.plot(). Figure
9.4 illustrates this plot for β1 and β2 .

beta[1] beta[2]
1.4

median median
1.08

97.5% 97.5%
1.3
1.06
shrink factor

shrink factor

1.2
1.04

1.1
1.02
1.00

1.0

0 5000 10000 20000 0 5000 10000 20000

last iteration in chain last iteration in chain

Figure 9.4 Gelman and Rubin’s scale reduction factor.

208 Software

2. Geweke’s Diagnostic
> geweke.diag(A1_2_mcmc)
[[1]]

Fraction in 1st window = 0.1

Fraction in 2nd window = 0.5

beta0 beta[1] beta[2] beta[3] beta[4] tau

0.003827 -1.383019 -0.608487 -0.695510 -0.948153 0.047654
tau_a sigma_a sigma deviance
-0.231638 -0.349071 0.069021 -0.278292

[[2]]

Fraction in 1st window = 0.1

Fraction in 2nd window = 0.5

beta0 beta[1] beta[2] beta[3] beta[4] tau

-2.2450 2.7206 0.6372 -0.5620 -0.5291 0.3862
tau_a sigma_a sigma deviance
0.1276 -0.4537 -0.3752 1.4152

The output of this function is the Z-scores for the tests of equal means
between the earlier and later part of the series for each variable. Since the
values for the first chain are all within the interval (−1.96, 1.96), we do
not reject the null hypothesis of equal means for all monitored parameters.
However, the same is not true for β0 and β1 in the second chain. Keep
in mind that any parameter failing the test is evidence for the chain as a
whole not having reached stationarity. This is the case since convergence
is a property of the chain, not of individual parameters.
3. Raftery and Lewis’ Diagnostic
This method was considered for short pilot sequences of Markov chains.
Thus, we will apply it for the first 4000 iterates only for each chain (iter-
ates during the initial burn-in, in this case 20,000, were not saved in the
object A1). The method requires the designation of a quantile that is to be
estimated. By default, CODA uses the 0.025 quantile. We use the method
assuming one wishes to estimate the median, that is, the 0.5 quantile.
> raftery.diag(A1_1chain[1:4000,],q=0.5,r=0.01,s=0.95,
converge.eps=0.001)

Quantile (q) = 0.5

Accuracy (r) = +/- 0.01
Probability (s) = 0.95
 9.6 Convergence Diagnostics: the Programs CODA and BOA 209

You need a sample size of at least 9604 with

these values of q, r and s

#using 10000 iterates

> raftery.diag(A1_1chain[1:10000,],q=0.5,r=0.01,s=0.95)

Quantile (q) = 0.5

Accuracy (r) = +/- 0.01
Probability (s) = 0.95

Burn-in Total Lower bound Dependence

(M) (N) (Nmin) factor (I)
beta0 2 9324 9604 0.971
beta[1] 2 9268 9604 0.965
beta[2] 2 9354 9604 0.974
beta[3] 1 9619 9604 1.000
beta[4] 2 9099 9604 0.947
tau 2 9354 9604 0.974
tau_a 2 9520 9604 0.991
sigma_a 2 9558 9604 0.995
sigma 2 9384 9604 0.977
deviance 2 9272 9604 0.965

> raftery.diag(A1_2chain[1:10000,],q=0.5,r=0.01,s=0.95)

Quantile (q) = 0.5

Accuracy (r) = +/- 0.01
Probability (s) = 0.95

Burn-in Total Lower bound Dependence

(M) (N) (Nmin) factor (I)
beta0 2 9794 9604 1.020
beta[1] 2 9771 9604 1.020
beta[2] 2 9459 9604 0.985
beta[3] 1 9588 9604 0.998
beta[4] 2 9302 9604 0.969
tau 2 9736 9604 1.010
tau_a 2 9406 9604 0.979
sigma_a 2 9399 9604 0.979
sigma 2 9751 9604 1.020
deviance 2 9276 9604 0.966

The dependence factor for both chains is approximately 1, indicating no

problems. According to these results, 10,000 iterations would suffice to
estimate the median with a tolerance of r = 0.01 and a probability of s =
0.95 of being within these tolerance limits.
4. Heidelberg and Welch Method
To apply this method we need to fix values for ǫ and α. As mentioned
before, CODA uses by default ǫ = 0.1 and α = 0.05. The default values
210 Software

can be changed by using the arguments of the function heidel.diag().

Just for illustration, we use ǫ = 0.01.
> heidel.diag(A1_2_mcmc, eps=0.01, pvalue=0.05)
[[1]]

Stationarity start p-value

test iteration
beta0 passed 1 0.503
beta[1] passed 1 0.052
beta[2] passed 1 0.592
beta[3] passed 1 0.822
beta[4] passed 1 0.504
tau passed 1 0.402
tau_a passed 1 0.999
sigma_a passed 1 0.936
sigma passed 1 0.435
deviance passed 1 0.503

Halfwidth Mean Halfwidth

test
beta0 passed 17.2739 2.47e-02
beta[1] passed 4.7013 3.46e-02
beta[2] failed 0.1565 1.93e-03
beta[3] passed 4.2236 1.29e-02
beta[4] passed -0.1047 4.93e-04
tau passed 0.0774 1.52e-04
tau_a passed 0.0163 5.29e-05
sigma_a passed 7.9973 1.30e-02
sigma passed 3.6216 3.60e-03
deviance passed 794.9228 1.72e-01

[[2]]

Stationarity start p-value

test iteration
beta0 passed 2001 0.2585
beta[1] passed 1 0.0766
beta[2] passed 1 0.8299
beta[3] passed 1 0.1795
beta[4] passed 1 0.8124
tau passed 1 0.9457
tau_a passed 1 0.8847
sigma_a passed 1 0.9781
sigma passed 1 0.9562
deviance passed 1 0.5130

Halfwidth Mean Halfwidth

test
beta0 passed 17.2694 0.025928
beta[1] passed 4.6858 0.034335
 9.6 Convergence Diagnostics: the Programs CODA and BOA 211

beta[2] failed 0.1561 0.001879

beta[3] passed 4.2275 0.012846
beta[4] passed -0.1046 0.000507
tau passed 0.0773 0.000154
tau_a passed 0.0162 0.000052
sigma_a passed 8.0069 0.013041
sigma passed 3.6251 0.003658
deviance passed 795.0646 0.175413

With ǫ = 0.01 the parameter β2 passed the test of stationarity (for both
chains), but failed the half-width test, indicating the need to continue sim-
ulation to achieve the desired precision. If we augment ǫ to 0.05, all pa-
rameters pass both stationarity and half-width tests.
5. HPD Intervals
Finally, recall that in CODA one can obtain HPD intervals for all mon-
itored parameters, using the function HPDinterval(). This is illustrated
here with 95% HPD intervals for each of the chains.
> HPDinterval(A1_2_mcmc, prob = 0.95)
[[1]]
lower upper
beta0 13.860000 20.80000
beta[1] 0.036880 9.84600
beta[2] -0.128400 0.42490
beta[3] 2.487000 6.08700
beta[4] -0.178200 -0.03300
tau 0.056580 0.10000
tau_a 0.009279 0.02371
sigma_a 6.246000 9.88700
sigma 3.122000 4.14500
deviance 771.400000 820.30000
attr(,"Probability")
[1] 0.95

[[2]]
lower upper
beta0 13.960000 20.87000
beta[1] 0.002304 9.64300
beta[2] -0.114500 0.42770
beta[3] 2.402000 6.02500
beta[4] -0.178100 -0.03235
tau 0.055750 0.09912
tau_a 0.009449 0.02374
sigma_a 6.223000 9.83100
sigma 3.122000 4.15300
deviance 771.100000 819.80000
attr(,"Probability")
[1] 0.95
212 Software

We now briefly illustrate how to evaluate the same diagnostics in BOA.

A1<-Cexemplo1_OpenBUGS.fit$sims.matrix
#results in matrix form
nomes<-list(c(1:20000,1:20000),c("beta0","beta[1]","beta[2]",
"beta[3]","beta[4]","tau","tau_a",
"sigma_a","sigma","deviance"))
dimnames(A1)<-nomes
A1_1<-A1[1:20000,] #define first chain
A1_2<-A1[20001:40000,] #define a second chain
#-----------------------------------------------#
#autocorrelation
#-----------------------------------------------#
boa.acf(A1_1,lags=1)
boa.acf(A1_2,lags=1)
#-----------------------------------------------#
#Geweke’s method #
#-----------------------------------------------#
boa.geweke(A1_1, p.first=0.1, p.last=0.5)
boa.geweke(A1_2, p.first=0.1, p.last=0.5)
#-----------------------------------------------#
#Heidelberg and Welch method #
#-----------------------------------------------#
boa.handw(A1_1, error=0.05, alpha=0.05)
boa.handw(A1_2, error=0.05, alpha=0.05)
#-----------------------------------------------#
#HPD intervals #
#-----------------------------------------------#
#the function boa.hpd() computes HPD intervals
#for one parameter.
#to find intervals for all (monitored) parameters
#one can use, e.g.
hpd_boa<-function(x) boa.hpd(x,0.05)
apply(A1_1,2,hpd_boa)
apply(A1_2,2,hpd_boa)

B. Using R2jags
1. The output from jags() can be converted into an mcmc object with the
command:
exemplo1_JAGS.fit2.mcmc <- as.mcmc(exemplo1_JAGS.fit2 )

2. As before, with the mcmc object one can then use a variety of com-
mands for convergence diagnostics in CODA:
library(coda)
plot(exemplo1_JAGS.fit2.mcmc)
autocorr.plot(exemplo1_JAGS.fit2.mcmc)
gelman.plot(exemplo1_JAGS.fit2.mcmc)
gelman.diag(exemplo1_JAGS.fit2.mcmc)
geweke.diag(exemplo1_JAGS.fit2.mcmc)
 9.6 Convergence Diagnostics: the Programs CODA and BOA 213

raftery.diag(exemplo1_JAGS.fit2.mcmc)
heidel.diag(exemplo1_JAGS.fit2.mcmc)

Note that the object exemplo1_JAGS.fit2 already includes both chains,

and the same is true for the mcmc object exemplo1_JAGS.fit2.mcmc.
Thus it is not necessary to work with separate objects for both chains, as
we had to in R2OpenBUGS.
3. The function jags() returns the simulated values for the monitored
parameters also in matrix form:
exemplo1_JAGS.fit2$BUGSoutput$sims.matrix

Thus, for using BOA, we proceed exactly as we did in the case of two
chains with output from ROpenBUGS.

C. Using RStan
As mentioned before (item 5, section 9.4.1), the simulated parameter values
for a chain that is set up using the command stan() can be obtained by
applying the function extract() to the output of stan().
> samples_stan_array<-extract(exemplo1.fit_stan,
pars=c("beta0", "beta", "sigma", "sigma_a", "tau", "tau_a"),
permuted = FALSE, inc_warmup = FALSE, include = TRUE)
> class(samples_stan_array)
[1] "array"
> dim(samples_stan_array)
[1] 20000 2 9 #20000 cada cadeia, 2 cadeias, 9 parameters

To use CODA or BOA, we start by defining matrices for each of the

chains.
samples_coda_1<-as.matrix(samples_stan_array[1:20000,1,1:9]))
samples_coda_2<-as.matrix(samples_stan_array[1:20000,2,1:9]))

For CODA, the matrices are transformed to objects of class mcmc:

samples_coda_1<-mcmc(samples_coda_1)
samples_coda_2<-mcmc(samples_coda_2)
gelman.diag(list(samples_coda_1,samples_coda_2))
geweke.diag(samples_coda_1)
geweke.diag(samples_coda_2)
raftery.diag(samples_coda_1)
raftery.diag(samples_coda_2)
heidel.diag(samples_coda_1)
heidel.diag(samples_coda_2)

For BOA, we need to first define dimnames for the row and column names:
214 Software

samples_coda_1<-as.matrix(samples_stan_array[1:20000,1,1:9])
samples_coda_2<-as.matrix(samples_stan_array[1:20000,2,1:9])
dimnames(samples_coda_1)<- list(1:20000,
c("beta0", "beta[1]","beta[2]", "beta[3]", "beta[4]",
"sigma", "sigma_a", "tau", "tau_a"))
dimnames(samples_coda_2)<- list(1:20000,
c("beta0", "beta[1]","beta[2]", "beta[3]", "beta[4]",
"sigma", "sigma_a", "tau", "tau_a"))

We illustrate this with Geweke’s diagnostic

> boa.geweke(samples_coda_1,p.first=.1,p.last=0.5)
Z-Score p-value
beta0 -0.1895212 0.84968432
beta[1] -1.1536020 0.24866338
beta[2] -0.3998341 0.68927871
beta[3] -0.3581599 0.72022368
beta[4] -1.3735690 0.16957554
sigma -0.7696775 0.44149123
sigma_a -1.7314080 0.08337903
tau 0.6671540 0.50467380
tau_a 1.7048218 0.08822767
> boa.geweke(samples_coda_2,p.first=.1,p.last=0.5)
Z-Score p-value
beta0 -0.51871293 0.6039609
beta[1] 0.15164978 0.8794632
beta[2] 1.35185008 0.1764233
beta[3] -0.57649303 0.5642820
beta[4] 0.61505637 0.5385175
sigma -0.93391998 0.3503452
sigma_a -0.03298591 0.9736858
tau 1.23723600 0.2159995
tau_a 0.02936042 0.9765771

D. Using R2BayesX
Recall (item 4, Subsection 9.5.1) that the function samples() applied to
the output of bayesx() returns the simulated values from the posterior
distribution, including the parameters that are indicated in the argument
term. The argument CODA controls the type and class of the output object.
Thus with the following commands we get a list of type mcmc, which
can be used for convergence diagnostics in CODA:
> AA_coda<-samples(exemplo1_BayesX,
term=c("linear-samples","var-samples","sd(ID)"),coda=TRUE)
> class(AA_coda)
[1] "mcmc.list"
> names(AA_coda)
[1] "Chain_1" "Chain_2"

#illustrating
 9.7 R-INLA and the Application Example 215

> gelman.diag(AA_coda)
Potential scale reduction factors:

Point est. Upper C.I.

Intercept 1 1
z2 1 1
z3 1 1
z1 1 1
z 1 1
1 1

Multivariate psrf
1

On the other hand, to use BOA for convergence diagnostics, proceed as

follows:
> AA_boa<-as.matrix(AA_data)
> AA_boa_1<-AA_boa[,1:6]
> AA_boa_2<-AA_boa[,7:12]

#illustrating
> library(boa)
> boa.geweke(AA_boa_1,p.first=0.1,p.last=0.5)
Z-Score p-value
Chain_1.Param.Intercept -0.2281478 0.8195313
Chain_1.Param.z2 1.2278951 0.2194863
Chain_1.Param.z3 -0.2358216 0.8135711
Chain_1.Param.z1 1.1215734 0.2620438
Chain_1.Param.z 0.8195813 0.4124548
Chain_1.Var 0.6110576 0.5411614
> boa.geweke(AA_boa_1,p.first=0.1,p.last=0.5)
Z-Score p-value
Chain_1.Param.Intercept -0.2281478 0.8195313
Chain_1.Param.z2 1.2278951 0.2194863
Chain_1.Param.z3 -0.2358216 0.8135711
Chain_1.Param.z1 1.1215734 0.2620438
Chain_1.Param.z 0.8195813 0.4124548
Chain_1.Var 0.6110576 0.5411614

9.7 R-INLA and the Application Example

In Section 8.3. we described the INLA approach to analyzing Bayesian
hierarchical models without the use of simulation. Being an approxima-
tion method, with this method we need not worry about convergence prob-
lems as are inherent in MCMC simulation methods, and discussed in the
previous section. However, this does not imply that the obtained poste-
rior approximations are always good. We have to study the quality of the
216 Software

approximation. Rue et al (2009) proposed two strategies to evaluate the

approximation error of posterior distributions: one is based on the calcu-
lation of the effective number of parameters and the other is based on the
Kullback–Leibler divergence criterion. Details of these strategies are dis-
cussed in sections 4.1. and 4.2. of the cited article. These strategies are
implemented in R-INLA. We will see how to evaluate them. .
As discussed in Chapter 8, the INLA method is designed for Bayesian
inference in latent Gaussian models, a large class of models that includes
many models from additive GLMMs to log-Gaussian Cox models and
spatio-temporal models. Together with the use of stochastic partial differ-
ential equations (SPDEs) (Lindgren et al, 2011), one can model all kinds of
spatial data, including area referenced data, geo-referenced data, and point
process data (Lindgren and Rue, 2015).
The R-INLA software is an R package developed to implement approx-
imate Bayesian inference using INLA. The package is a further devel-
opment of a stand-alone program INLA. It is written in C using the li-
brary GMRFLib (www.math.ntnu.no/~hrue/GMRFLib/doc/html) to
implement fast and exact simulation for Gaussian random fields.
R-INLA is available for the operating systems Linux, Mac and Win-
dows. On the site www.r-inla.org, besides installation instructions for
R-INLA, one can find code, examples, articles, and reviews where the the-
ory and application of INLA are discussed, and there are also many other
materials of interest, in particular a discussion forum and answers to fre-
quently asked questions.
To install R-INLA directly from R use the command:
install.packages("INLA",
repos="http://www.math.ntnu.no/inla/R/stable")

As with any other R package, to load R-INLA in each work session, write
library(INLA)

As there are frequent updates of R-INLA, one should use

inla.upgrade(testing=FALSE)

to get the most recent and most stable version of the package.
Using R-INLA there is a large number of probability distributions that
can be used for the response variable. The list can be seen with the com-
mand
> names(inla.models()$likelihood)
 9.7 R-INLA and the Application Example 217

A complete description of these distributions, with examples, can be

found at www.r-inla.org/models/likelihoods. Similarly, to get in-
formation about prior distributions for model parameters and structured
or not structured random effects, see www.r-inla.org/models/priors
and www.r-inla.org/models/latent-models. The lists with the cor-
responding distributions can also be obtained with the commands
> names(inla.models()$prior)
> names(inla.models()$latent)

To better understand how R-INLA works, we continue with the applica-

tion example.

9.7.1 Application Example

1. As in BayesX, using the Bayesian model from Section 9.1, the model is
translated into R as the object created by the formula
> INLA_formula <- X ~ z1 + z2 + z3 + z +
f(ID, model="iid",
hyper=list(prec=list(prior="loggamma",param=c(1,0.005))))
> class(INLA_formula)
[1] "formula"

As before, fixed effects that appear linearly in the model have been cen-
tered. Using the function f(), which appears in the definition of the for-
mula, we define the structural effects (the various types are defined at
www.r-inla.org/models/latent-models). In the case at hand we
have only the patient-specific (variable ID) random effects which are in-
troduced in the model as a. The model iid specifies a normal distribution
with zero mean and precision τa . The prior distribution that is specified in
the argument hyper is for ln(τa ). Being a log-gamma it corresponds to a
gamma distribution for the precision τa .
2. Next we call the inla() function to run the INLA algorithm and
obtain the desired results to eventually proceed with Bayesian inference,
as follows:
> ?inla
> resultado_INLA <- inla(INLA_formula,family="normal",
control.predictor=list(compute=TRUE),
control.compute =list(waic=TRUE,dic=TRUE,cpo=TRUE),
data = Cexemplo1,
control.family=
list(hyper=list(prec=list(prior="loggamma",param=c(1,0.005)))
218 Software

The first line in the above code lists all arguments of the inla() func-
tion, of which the only required ones are the object that states the formula,
in this case INLA_formula, and the object that contains the data, in this
case data = Cexemplo1. By not specifying the other arguments they are
assumed by R-INLA to be specified by omission.
The directive control.predictor=list(compute=TRUE) specifies
to compute the marginal distributions of the linear predictor. There are
other arguments for this function, which can be listed by the command
?control.predictor

control.family() declares the prior distributions for the parameters in

the family of sampling models. In this case, we declared a prior distribution
for the precision parameter τ. See ?control.family for details on how
to do this for parameters of specific distributions.
To compute the WAIC and deviance information criterion (DIC) criteria
and also conditional predictive ordinate (CPO), we need to declare
control.compute =list(waic=TRUE,dic=TRUE,cpo=TRUE)

3. The function inla() returns an object of class inla, here saved as

resultado_INLA. This object is a list with many elements that can be
explored by the command names(resultado_INLA). Summary results
of the INLA approach can be obtained by the command
> summary(resultado_INLA)
...
Time used:
Pre-processing Running inla Post-processing Total
0.1719 0.4375 0.0975 0.7069

Fixed effects:
mean sd 0.025quant 0.5quant 0.975quant mode kld
(Intercept) 17.250 1.737 13.820 17.251 20.669 17.253 0
z1 4.772 2.448 -0.050 4.770 9.599 4.766 0
z2 0.154 0.138 -0.118 0.154 0.427 0.153 0
z3 4.169 0.908 2.394 4.164 5.972 4.154 0
z -0.106 0.036 -0.176 -0.106 -0.035 -0.107 0

Random effects:
Name Model
ID IID model

Model hyperparameters:
mean sd 0.025quant 0.5qu
Precision for the Gaussian observations 0.0789 0.0113 0.0587 0.07
Precision for ID 0.0170 0.0039 0.0106 0.01
0.975quant mode
 9.7 R-INLA and the Application Example 219

Precision for the Gaussian observations 0.1027 0.0771

Precision for ID 0.0256 0.0160

Expected number of effective parameters(std dev): 46.38(0.8267)

Number of equivalent replicates : 3.17

Deviance Information Criterion (DIC) ...: 841.57

Effective number of parameters .........: 47.65

Watanabe-Akaike information criterion (WAIC) ...: 844.77

Effective number of parameters .................: 41.31

Marginal log-Likelihood: -497.49

Posterior marginals for linear predictor and fitted values computed

Compare these results with the corresponding results from simulation meth-
ods. In particular, compare the WAIC values with those obtained from
RStan.

4. Note that the results also include, for each configuration of hyperpa-
rameters, an estimate of the effective number of parameters. This estimate
essentially corresponds to the expected number of independent parameters
in the model. In our case, we have 7 + 49 = 56 parameters, but since the
random effects are correlated, the effective number of parameters is lower,
≈ 47, as can be seen. As mentioned, this is one of the strategies, proposed
in Rue et al (2009), to evaluate the accuracy of the approximation. In par-
ticular, if the effective number of parameters is low compared to the sample
size, then one expects the approximation to be good. In this case the ratio
of sample size (147) and effective number of parameters (46.38) is approxi-
mately 3.17, suggesting a reasonably good approximation. In fact, the ratio
can be interpreted as the number of “equivalent replicates” corresponding
to the number of observations for each expected number of effective pa-
rameters.
Another reported quantity is the mean Kullback–Leibler divergence (in
the column kld). This value describes the difference between the normal
approximation and the simplified Laplace approximation (recall the dis-
cussion in Chapter 8 about the various approximation strategies used in
INLA) for the marginal posterior distributions. Small values indicate that
the posterior distribution is well-approximated by a normal.
5. The default approximation strategy in inla() is the simplified Laplace
approach. Other approximation and integration methods can be defined us-
ing the argument control.inla in the function inla(). For example, if
one wanted the complete Laplace approach used, which is recommended
220 Software

for higher accuracy in the estimation of tails in the marginal distributions,

one would use inla() with the argument
control.inla=list(strategy="laplace",npoints=21)

6. Besides the results shown earlier, R-INLA can also report two types
of goodness-of-fit measures, namely CPO p(yi | y−i ) and the probability
integral transforms P(Yinova ≤ yi | y−i ) (PIT). To add these in the out-
put, just add cpo=TRUE in the list of the argument control.compute
for the function inla. The values are then returned as part of the output
from inla. A list of all possible values can be obtained by the command
names(resultados_INLA). Of the 51 possible ones, we list only a few:
> names(resultado_INLA)
[1] "names.fixed"
[2] "summary.fixed"
[3] "marginals.fixed"
[4] "summary.lincomb"
[5] "marginals.lincomb"
[6] "size.lincomb"
[7] "summary.lincomb.derived"
[8] "marginals.lincomb.derived"
[9] "size.lincomb.derived"
[10] "mlik"
[11] "cpo"
[12] "po"
[13] "waic"
...
[18] "summary.linear.predictor"
[19] "marginals.linear.predictor"
[20] "summary.fitted.values"
[21] "marginals.fitted.values"
...
[27] "offset.linear.predictor"
...
[51] "model.matrix"

One can then save the values of these two measures of fit in an object, for
example CPO_PIT below, and use, for further analysis, for graphs, etc.
> CPO_PIT<-resultado_INLA$cpo
> names(CPO_PIT)
[1] "cpo" "pit" "failure"
> class(CPO_PIT)
[1] "list"
> summary(CPO_PIT$cpo)
Min. 1st Qu. Median Mean 3rd Qu. Max.
0.0003652 0.0486900 0.0741300 0.0673100 0.0905200 0.0924000
> summary(CPO_PIT$pit)
Min. 1st Qu. Median Mean 3rd Qu. Max.
 9.7 R-INLA and the Application Example 221

0.0004578 0.2769000 0.5046000 0.5013000 0.7493000 0.9961000

> summary(CPO_PIT$failure)
Min. 1st Qu. Median Mean 3rd Qu. Max.
0 0 0 0 0 0

Extreme CPO values indicate unusual “surprise” observations, and extreme

PIT values indicate outliers. A histogram of the PIT probabilities that ap-
pears far from uniform is evidence for a model misfit. Corresponding to
the information in CPO_PIT$failure above, no observation is considered
surprising or discordant.
7. To obtain a graphical representation, one can use the plot() com-
mand. The function has various boolean arguments. By default, the argu-
ments are “TRUE.” Thus, the command
plot(resultado_INLA)
# or in separate windows
plot(resultado_INLA,single = TRUE)

generates plots of the posterior densities of fixed and random effects, of

posterior densities for the precision parameters, of a sequence of means and
0.025 and 0.975 posterior quantiles of the random effects and linear predic-
tors, of the CPO and PIT values, and corresponding histograms. If desired,
selected plots can be produced by choosing “FALSE” for the boolean ar-
gument corresponding to the graphs that are not wanted. For example, to
get only the posterior densities for the precision parameters, use:
plot(resultado_INLA,
plot.fixed.effects = FALSE,
plot.lincomb = FALSE,
plot.random.effects = FALSE,
plot.hyperparameters = TRUE,
plot.predictor = FALSE,
plot.q = FALSE,
plot.cpo = FALSE)

to get Figure 9.5.

8. R-INLA works with precision parameters (inverse variance). How-
ever, in general, inference for standard deviations is of more interest. It is
possible to use a set of functions in INLA to calculate quantiles, expected
values of original parameters, and also to obtain samples from marginal
posterior distributions. To get posterior means of the standard deviations
√ √
σ = 1/ τ and σa = 1/ τa , use the following commands:
> names(resultado_INLA$marginals.hyperpar)
[1] "Precision for the Gaussian observations"
[2] "Precision for ID"
222 Software

PostDens [Precision for the Gaussian observations]

30
15
0

0.04 0.06 0.08 0.10 0.12 0.14 0.16

PostDens [Precision for ID]

80
40
0

0.01 0.02 0.03 0.04 0.05

Figure 9.5 Posterior densities for τ and τa

> tau<-resultado_INLA$marginals.hyperpar$
+"Precision for the Gaussian observations"
> sigma<-inla.emarginal(function(x) 1/sqrt(x), tau)
> sigma
[1] 3.588387
> tau_a<-resultado_INLA$marginals.hyperpar$"Precision for ID"
> sigma_a<-inla.emarginal(function(x) 1/sqrt(x), tau_a)
> sigma_a
[1] 7.813781

Alternatively, to get the posterior distribution of random effect standard

deviations, one can use the commands:
> sigmas<-inla.contrib.sd(resultado_INLA,nsamples=1000)
> names(sigmas)
[1] "samples" "hyper"
> sigmas$hyper
mean sd 2.5% 97.5%
sd for the Gaussian observations 3.59079 0.25221 3.1267 4.1120
sd for ID 7.79427 0.88068 6.2625 9.6867
> head(sigmas$samples)
sd for the Gaussian observations sd for ID
[1,] 3.407485 8.287859
[2,] 3.775560 6.945835
 9.7 R-INLA and the Application Example 223

[3,] 3.912179 9.931287

[4,] 3.282005 10.068471
[5,] 3.736729 7.386682
[6,] 3.808289 9.027061

The object sigmas above includes in samples a vector of posterior simu-

lated standard deviations.
9. To get random samples from marginal posteriors, use the function
inla.rmarginal() in the following way (illustrated for β3 ):
> names(resultado_INLA$marginals.fixed)[
1] "(Intercept)" "z1" "z2" "z3" "z"
> dens_z3<-resultado_INLA$marginals.fixed$z3
> amostra_z3<-inla.rmarginal(1000,dens_z3)
> summary(amostra_z3)
Min. 1st Qu. Median Mean 3rd Qu. Max.
0.5421 3.5580 4.1580 4.1630 4.7880 7.1570

More information about the functions that are used to operate on marginal
distributions is obtained via the help command ?inla.marginal.
10. One of these functions, inla.hpdmarginal(), allows getting HPD
credible intervals for the model parameters. To obtain these intervals for the
parameters corresponding to the fixed effects in the linear predictor, one
can either get them individually, or get all in one command, as illustrated
here for a 95% interval,
> HPD<-NULL
> for(i in 1:5){
HPD[[i]]<-inla.hpdmarginal
(0.95, resultado_INLA$marginals.fixed[[i]])}
> HPD
[[1]]
low high
level:0.95 13.82332 20.66469

[[2]]
low high
level:0.95 -0.05260815 9.58653

[[3]]
low high
level:0.95 -0.1184688 0.4263571

[[4]]
low high
level:0.95 2.384431 5.958083

[[5]]
224 Software

low high
level:0.95 -0.1759522 -0.03584334

and for the model hyperparameters,

> names(resultado_INLA$marginals.hyper)
[1] "Precision for the Gaussian observations"
[2] "Precision for ID"
> HPDhyper<-NULL
> for(i in 1:2){
+ HPDhyper[[i]]<-inla.hpdmarginal
+ (0.95, resultado_INLA$marginals.hyper[[i]])}
> HPDhyper
[[1]]
low high
level:0.95 0.0574843 0.1011766

[[2]]
low high
level:0.95 0.009948865 0.02469588

As expected, the HPD intervals for the fixed effect parameters practically
coincide with the equal tail intervals in the summary results. The same is
not true for the precision parameters.

Problems
9.1 FEV Data. Rosner (1999) reports on a study of lung capacity and smoking
on n = 654 youths aged 3 – 19 years. Lung capacity yi is measured as forced
expiratory volume (FEV). Covariates include age (xi1 ) and an indicator of
smoking (xi2 ). The same data are also analyzed by Chistensen et al. (2011:
ch. 7).
Set up a regression of yi on age (xi1 ), adjusting intercept and slope for smok-
ing (xi2 ), that is, a normal linear regression for yi = b0 + b1 xi1 + b2 xi2 +
b3 xi1 xi2 + ǫi j . The data are available as the data set fev in the R package
tmle. Set up inference using OpenBUGS, JAGS, Stan, or INLA. Does FEV
differ by smoking? Does smoking affect the increase of lung capacity with
age? Is the interaction term needed?
9.2 Consider any of the examples in the OpenBUGS manual, www.openbugs.
net/w/Examples. Implement inference in JAGS or Stan or OpenBUGS.
9.3 Consider the Epilepsy study reported in the OpenBUGS manual, www.openbug
net/Examples/Epil.html. Implement inference using R-INLA. Show
the marginal posterior distributions for the logistic regression coefficients
a j . Compare versus a simplified model without extra-Poisson variation (that
is, without the b jk random effect).
9.4 Consider the Salmonella dose–response study in the OpenBUGS manual,
 Problems 225

www.openbugs.net/Examples/Salm.html. Implement inference using

R-INLA. Plot the estimated dose-response curve of mean outcome versus
dose. Include pointwise 95% credible bands.
9.5 Consider the surgical institution rating example in the OpenBUGS man-
ual, www.openbugs.net/Examples/Surgical.html. Implement infer-
ence using R-INLA. Show a boxplot of marginal posterior distribution for
failure rates ri , marking posterior mean, median, and quartiles under the de-
pendent model that borrows strength across hospitals.
9.6 Variational Bayes. Consider the rats example in the OpenBUGS manual,
www.openbugs.net/Examples/Rats.html. Code the example in Stan
and use the following code fragment to implement posterior inference.
# assume the STAN model is coded in the file "rats.stan"
rats_model <- stan_model(file = "rats.stan")
# MCMC posterior simulation
rats_fit <- sampling(rats_model, data=data,
iter=2000, warmup=1000, chains=4, seed=123)
# use vb() to implement variational inference
rats_fit_vb <- vb(rats_model, data=data,
output_samples=2000, seed=123)

Hint: Use a centered parametrization with an overall mean α0 and animal-

specific offsets αi ∼ N(0, σ2α ). See also the implementation in
https://gist.github.com/bakfoo/2447eb7d551ff256706a
 Appendix A

Probability Distributions

For reference we include a brief summary of distributions that are used

throughout the text. Where multiple parametrizations are in use, the follow-
ing summary lists the specific parametrizations that are used in the text. We
use f (·) generically for probability mass functions for discrete random vari-
ables and density functions for continuous random variables. We use Ckn =
k!(n − k)!/n! to denote binomial coefficients, B(a, b) = Γ(a)Γ(b)/Γ(a + b)
P
for the beta function, and in general B(a) = Π j Γ(a j )/Γ( j a j ) for a =
(a1 , . . . , ak ). We write µ, σ2 for mean and variance, µi and σ2i for marginal
mean and variance of the i-th component, σi j for covariance, and Σ for the
covariance matrix of a random vector. Notation like a• indicates summation
P
over all coordinates of a, i.e., c+1 i=1 ai when a = (a1 , . . . , ac+1 ), assuming
that the dimension of a is understood from the context. Finally, Sc denotes
the c-dimensional simplex and N0 are the non-negative integers.

Discrete Distributions
Binomial: x ∼ Bi(n, p), x ∈ {0, . . . , n};
f (x) = C nx p x (1 − p)n−x , n ∈ N, 0 < p < 1
µ = np, σ2 = np(1 − p).
Bernoulli: Ber(p) = Bi(n = 1, p).
Beta-binomial: x ∼ BeBin(n, a, b), x ∈ {0, . . . , n}
f (x) = C nx B(a + x, b + n − x)/B(a, b), a > 0, b > 0, n ∈ N
µ = na/(a + b), σ2 = nab(a + b + n)/[(a + b)2 (a + b + 1)].
Note: A BeBin(n, a, b) distribution is a mixture of a Bi(n, θ) with respect
to θ ∼ Be(a, b).
Negative binomial: x ∼ NBin(r, θ), x ∈ N0 ;
f (x) = C xx+r−1 (1 − θ)r θ x , 0 < θ < 1, r ∈ N;
µ = rθ/(1 − θ) and σ2 = rθ/(1 − θ)2 .
Geometric: Geo(θ) = NBin(1, θ).
 Probability Distributions 227

Beta-negative binomial: x ∼ BeNBin(r, α, β), x ∈ N0 ,

B(α + r, β + x)
r+x
f (x) = Cr−1 , r ∈ N, α > 0, β > 0
B(α, β)
β rβ
h α+β+r−1 rβ
i
µ = r α−1 (α > 1) and σ2 = α−1 α−2
+ (α−1)(α−2)
(α > 2).
Note: A BeNBin(r, α, β) distribution is the mixture of an NBin(r, θ) dis-
tribution with respect to θ ∼ Be(α, β).
Poisson: x ∼ Poi(λ), x ∈ N0 ;
f (x) = e−λ λ x /x!, λ > 0
µ = σ2 = λ.
Poisson-gamma: x ∼ PoiGa(n, α, β), x ∈ N0 ;
!α !x
Γ(α + x) β n
f (x) = , α > 0, β > 0, n ∈ N
x!Γ(α) β + n β+n
µ = nα/β and σ2 = nα(β + n)/β2 .
Note: A PoiGa(n, α, β) distribution is the mixture of a Poi(nλ) distribu-
tion with respect to λ ∼ Ga(α, β).
Hypergeometric: x ∼ HpG(N, M, n), x ∈ {n, . . . , n} with n = min{n, M},
n = max{0, n − (N − M)} ;

C xM Cn−x
N−M
f (x) = , N ∈ N, M ∈ {0, . . . , N}, n ∈ {1, . . . , N};
CnN
 
µ = nM
N
and σ2 = n N−n M
N−1 N
1− M
N
.

Continuous Distributions
Uniform: x ∼ U(α, β), α ≤ x ≤ β;
f (x) = 1/(β − α), α < β, α, β ∈ R;
µ = (β + α)/2 and σ2 = (β − α)2 /12.
Note: We also write U(S ) for a uniform distribution over a set S .
Beta: x ∼ Be(α, β), 0 ≤ x ≤ 1;
Γ(α + β) α−1
f (x) = x (1 − x) β−1 , α > 0, β > 0;
Γ(α)Γ(β)
µ = α/(α + β) and σ2 = αβ/[(α + β)2 (α + β + 1)].
228 Probability Distributions

Normal: x ∼ N(µ, σ2 ), x ∈ R;
1 2 2
f (x) = √2πσ e−(x−µ) /2σ , µ ∈ R, σ2 > 0;
E(x) = µ and Var(x) = σ2 .
Note: Alternative parametrizations use the precision, 1/σ2 , or the stan-
dard deviation, σ, as the second parameter. WinBUGS, JAGS, and INLA
use precision, STAN and R use standard deviation.
Gamma: x ∼ Ga(α, β), x ≥ 0;
βα
f (x) = Γ(α) , e−βx xα−1 , α > 0, β > 0;
µ = α/β and σ2 = α/β2 .
Exponential: Exp(λ) = Ga(1, λ).
Chi-square: χ2 (n) = Ga(n/2, 1/2).
Erlang: Erl(k, λ) = Ga(k, λ), k ∈ N.
Inverse gamma: x ∼ IGa(α, β), x ≥ 0;
βα −β/x −α−1
f (x) = Γ(α) e x , α > 0, β > 0;
µ = β/(α − 1), α > 1 and σ2 = β2 /[(α − 1)2 (α − 2)], α > 2.
Weibull: x ∼ Weib(λ, k), x ≥ 0;
 k−1 k
f (x) = λk λx e−(x/λ) , λ > 0, k > 0;
µ = λΓ(1 + 1k ) and σ2 = λ2 Γ(1 + 2k ) − µ2 .

√ uses δ = k/λ instead of λ.

Note: An alternative parametrization
Rayleigh: Ral(σ) = Weib(λ = σ 2, k = 2).
2
Note: With δ = 1/σ2 , we get f (x) = δe−δx /2 .
Student t : x ∼ t(λ, δ; n), x ∈ R;
" #− ν+1
(x − λ)2 2
f (x) = c 1 + , c = [B(ν/2, 1/2)]−1 (νδ)−1/2
νδ
ν
µ = λ, ν > 1 and σ2 = ν−2
δ, ν > 2.
Gamma-Gamma: x ∼ GaGa(n, α, β) x > 0;
βα xn−1
f (x|n, α, β) = , α > 0, β > 0, n ∈ N
B(n, α) (β + x)α+n
µ = nβ/(α − 1), α > 1 and σ2 = nβ2 (n + α − 1)/[(α − 1)2 (α − 2)], α > 2.
Note: A GaGa(n, α, β) distribution is the mixture of a Ga(n, λ) distribu-
tion with respect to λ ∼ Ga(α, β).
Normal/IGa: (x, w) ∼ NIGa(λ, v, a, b), x ∈ R, w > 0
f (x, w) = f (x | w) f (w) with x | w ∼ N(λ, w/v) and w ∼ IGa(a, b),
λ ∈ R, v > 0, a > 0, b > 0.
Note: Marginally, x ∼ t(λ, δ, ν) is Student t with δ = b/(av), ν = 2a.
 Probability Distributions 229

Pareto: x ∼ Pa(a, b), x ≥ b;

f (x) = aba /xa+1 , a > 0, b > 0,
µ = ab/(a − 1), a > 1 and σ2 = b2 a/[(a − 1)2 (a − 2)], a > 2.
Fisher–Snedecor: x ∼ F(α, β) , x > 0

αα/2 β β/2
f (x) = xα/2−1 (αx + β)−(α+β)/2 , α > 0, β > 0;
B (α/2, β/2)

µ = β/(β − 2), β > 2, and σ2 = 2β2 (α + β − 2) [α(β − 2)2 (β − 4)], β > 4.
Note: x ∼ F(α, β) can be derived as the distribution of
1. x = (x1 /α)/(x2 /β), where x1 ∼ χ2(α) independently of x2 ∼ χ2(β) ; or,
2. x = βy/[α(1 − y)] where y ∼ Be(α/2, β/2).

Multivariate Distributions
Dirichlet: θ ∼ Dc (a), θ ∈ Sc ,
Γ(a• ) ai −1
f (θ) = c+1
Πc+1
i=1 θi , ai ∈ R+ , i = 1, . . . , c + 1; θc+1 = 1 − θ• ,
Πi=1 Γ(ai )
µi = a/a• , σ2i = µi (1 − µi )/(a• + 1) and σi j = −µi µ j /(a• + 1), i , j.
P
Multinomial: x ∼ Mc (n, θ), x = (x1 , . . . , xc+1 ), xi ∈ N0 , ci=1 xi ≤ n and
Pc
xc+1 = n − i=1 xi
n! xi
f (x) = Πc+1
i=1 θi , θ ∈ Sc , θc+1 = 1 − θ•
Πc+1
i=1 xi !

µ = nθ and Σ = n(diag(θ1 , . . . , θc ) − θθ′ ).

Multinomial-Dirichlet: x ∼ MDk (n, α) for x = (x1 , . . . , xk ), with xi ∈
P P
{0, 1, 2, . . . , n}, ki=1 xi ≤ n, xk+1 = n − ki=1 xi
n! B({αi + xi })
f (x) = , α = (α1 , . . . , αk , αk+1 ), αi > 0, n ∈ N
Πk+1
i=1 xi !
B(α)
µi = nmi σ2i = nmi (1 − mi ) αα•• +n
+1
and σi j = −nmi m j αα•• +1
+n
, where m j =
α j /α• .
Note: An MDk (n, α) distribution is the mixture of an Mk (n, θ) distribu-
tion with respect to θ ∼ Dk (α).
Multivariate normal: x ∼ Nk (m, V) for x ∈ Rk
1 ′ −1
f (x) = (2π)−k/2 |V|−1/2 e− 2 (x−m) V (x−m)
, m ∈ Rk ,
230 Probability Distributions

V is a positive definite k × k matrix;

µ = m and Σ = V.
Wishart: X ∼ W(s, A), X a (k × k) non-negative definite matrix, and
s > k + 1, with
π−k(k−1)/4
f (X) = c|A| s |X| s−(k+1)/2 e−tr(AX) , c =  
Πkj=1 Γ 2s+1−
2
j

µ = sA−1 and Var(Xi j ) = s(v2i j + vii v j j ) with V = [vi j ] = A−1 .

Multivariate Student t: x ∼ tk (m, V; ν) for x ∈ Rk
" #− ν+k2
Γ((ν + k)/2) −k/2 −1/2 1 ′ −1
f (x) = ν |V| 1 + (x − m) V (x − m) ,
Γ(ν/2) [Γ(1/2)]k ν
with m ∈ Rk , V a positive definite k × k matrix;
ν
µ = m, ν > 1, and Σ = ν−2
V ν > 2.
Note: A tk (m, V; ν) distribution is the mixture of an Nk (m, wV) distribu-
tion with respect to w ∼ IGa(ν/2, ν/2).
Multivariate normal/Wishart: (x, W) ∼ NWik (m, v, α, Ω) for x ∈ Rk
and W a positive definite k × k matrix,
f (x, W) = f (x | W) f (W) with x | W ∼ Nk (m, W/v)), W ∼ Wik (α, A),
α > k−1
2
, A positive definite k × k matrix.
Note: Marginally x ∼ tk (µ, α A−1 /v, 2α).
 Appendix B

Programming Notes

Several of the problems in Chapter 6 and elsewhere require some program-

ming, in particular the implementation of general univariate and bivariate
random variate generation when a p.d.f. can be evaluated pointwise, and
the implementation of iterative simulation in an MCMC simulation. For
reference, we include brief R code fragments of possible implementations.

Random Variate Generation

Let X denote a random variable with univariate p.d.f. f (x). Assuming that
ln f (x) can be evaluated pointwise, the following function implements ran-
dom variate generation. Evaluation up to a constant factor (or offset on the
logarithmic scale) suffices. Let lpdf(x) denote a function that evaluates
the log p.d.f. The function below assumes that lpdf(x) is vectorized, that
is, for a vector x it returns a vector of ln f (x) values.
#######################################################
# simulating from a univariate distribution

sim.x <- function(n=1, xgrid=NULL, lpdf, other=NULL)

{ ## simulates x ~ p(x) with
## lpdf = log p(x) (a function)
## need not be standardized
## must evaluate p(x) for a vector of x values
## n = number of r.v. generations
## K = grid size
## xgrid = grid (equally spaced)
## other = (optional) additional parameters to be
## used in the call to p(x), i.e., r
if (is.null(xgrid)){
cat("\n *** Error: need to specify xgrid for sim.x().\n")
exit(-1)
}
delta <- xgrid[2]-xgrid[1] # grid size..

if (is.null(other)) # no optional additional arguments

lp <- lpdf(xgrid)
else
lp <- lpdf(xgrid,other)
232 Programming Notes

pmf <- exp(lp-max(lp))

x <- sample(xgrid,n,replace=T,prob=pmf) +
runif(n,min=-delta/2,max=+delta/2) # smear out x

return(x)
}

Bivariate Random Variate Generation

Let X = (X1 , X2 ) denote a bivariate random vector with p.d.f. f (x1 , x2 ). As-
suming that ln f (x1 , x2 ) can be evaluated pointwise, the following function
implements random variate generation. As before, evaluation up to a con-
stant suffices. Assume the function lpdf(x) evaluates the log p.d.f. for a
bivariate vector x.

#######################################################
# simulating from a bivariate distribution

sim.xy <- function(n=1, xgrid=NULL, ygrid=NULL, lpdf)

{ ## simulates (x,y) ~ p(x,y) with
## lpdf = log p(x,y) (a function)
## need not be standardized
## n = number of r.v. generations
## K = grid size
## xgrid = grid (equally spaced)
## ygrid = grid (equally spaced)
if ( is.null(xgrid) | is.null(ygrid) ){
cat("\n *** Error: need to specify xgrid for sim.x().\n")
exit(-1)
}
dx <- xgrid[2]-xgrid[1]
dy <- ygrid[2]-ygrid[1]

xy <- cbind(sort(rep(xgrid,K)), rep(ygrid,K))

## a (K*K x 2) matrix with one row for each 2-dim grid point.
lp <- apply(xy,1,lpdf)
pmf <- exp(lp-max(lp))
idx <- sample((1:nrow(xy)), n,replace=T,prob=pmf)
dx <- runif(n, min=-dx/2, max=+dx/2) # smear out x
dy <- runif(n, min=-dy/2, max=+dy/2) # smear out y
XY <- xy[idx,] + cbind(dx,dy)
return(XY)
}

MCMC Simulation
The last code fragment shows a typical Gibbs sampler implementation.
The main function gibbs() implements the iteration over the steps of
the MCMC simulation. The code implements a Gibbs sampler with two
transition probabilities, generating parameters b and s2 from the respective
complete conditional posterior distributions.
 Programming Notes 233

init <- function()

{ # initialize par values
fit <- lsfit(x, dta$Z) # fits a least squares regr
b <- fit$coef
s2 <- var(fit$resid)
th <- c(b,s2)
return(th=list(b=b,s2=s2))
}

gibbs <- function(niter=100)

{
th <- init()
thlist <- NULL
for(iter in 1:niter){
th$b <- sample.b(th)
th$s2 <- sample.s2(th)
thlist <- rbind(thlist,c(th$b,th$s2))
}
return(thlist)
}

sample.b <- function(th)

{# replace b by a draw from p(b | s2,y) = N(m,V)
## m = (X’X)^-1 X’y = least squares fit
## V^-1 = (X’X)*tau + S0^-1
## m = V * (tau* X’y)
tau <- 1/th$s2
Vinv <- H*tau + S0inv
V <- solve(Vinv)
m <- V %*% (tau*t(W)%*%dta$Z)
R <- chol(V) # V = t(R) %*% R
b <- rnorm(2,m=m) %*% R # b ~ N(m, V) as desired :-)
return(b)
}

sample.s2 <- function(th)

{ ## replace s2 by a draw from p(s2 | b,y)
## let tau=1/s2, then p(tau | b,y) = IG(a/2, b/2)
## with a=n and b=S2=sum(Z[i]-Zhat[i])^2
## also, when v ~ IG(a,b) <=> 1/v ~ G(a,b), shape=a, rate=b
b <- th$b
n <- nrow(dta)
Zhat <- W %*% c(b) # need c(b) to make sure it’s a col
S2 <- sum((dta$Z-Zhat)^2)
a1 <- a0+n
b1 <- b0+S2
tau <- rgamma(1,shape=a1/2, rate=b1/2)
s2 <- 1/tau
return(s2)
}
 References

Amaral Turkman, M. A. 1980. Applications of predictive distributions. PhD thesis,

University of Sheffield. (Cited on page 13.)
Andrieu, C., Doucet, A., and Robert, C. P. 2004. Computational advances for and from
Bayesian analysis. Statistical Science, 19(1), 118–127. (Cited on page 131.)
Basu, D., and Pereira, C. A. B. 1982. On the Bayesian analysis of categorical data:
the problem of nonresponse. Journal of Statistical Planning and Inference, 6(4),
345–362. (Cited on page 41.)
Belitz, C., Brezger, A., Kneib, T., Lang, S., and Umlauf, N. 2013. BayesX: Software for
Bayesian Inference in Structured Additive Regression Models. Version 2.1. (Cited
on page 174.)
Berger, J. O. 1984. The robust Bayesian viewpoint (with discussion). Pages 63–144
of: Kadane, J. B. (ed.), Robustness of Bayesian Analyses. North-Holland. (Cited on
page 1.)
Bernardo, J., and Smith, A. F. M. 2000. Bayesian Theory. Wiley. (Cited on pages 26
and 90.)
Best, N., Cowles, M., and Vines, S. 1995. CODA Manual Version 0.30. (Cited on
pages 118 and 203.)
Bhattacharya, A., Pati, D., Pillai, N. S., and Dunson, D. B. 2015. Dirichlet–Laplace pri-
ors for optimal shrinkage. Journal of the American Statistical Association, 110(512),
1479–1490. (Cited on page 136.)
Blangiardo, M., and Cameletti, M. 2015. Spatial and Spatio-Temporal Bayesian Models
with R-INLA. Wiley. (Cited on pages 152 and 175.)
Blangiardo, M., Cameletti, M., Baio, G., and Rue, H. 2013. Spatial and spatio-temporal
models with r-inla. Spatial and Spatio-Temporal Epidemiology, 4(Supplement C),
33–49. (Cited on page 166.)
Blei, D. M., and Jordan, M. I. 2006. Variational inference for Dirichlet process mix-
tures. Bayesian Analysis, 1(1), 121–143. (Cited on page 170.)
Blei, D. M., Kucukelbir, A., and McAuliffe, J. D. 2017. Variational inference: a review
for statisticians. Journal of the American Statistical Association, 112(518), 859–877.
(Cited on pages 166, 167, and 173.)
Box, G. 1980. Sampling and Bayes inference in scientific modelling and robustness.
Journal of the Royal Statistical Society, A, 143, 383–430. (Cited on page 72.)
Box, G. 1983. An apology for ecumenism in statistics. Pages 51–84 of: Box, G.,
Leonard, T., and Wu, C.-F. (eds.), Scientific Inference, Data Analysis, and Robust-
ness. Academic Press. (Cited on page 72.)
 References 235

Brezger, A., Kneib, T., and Lang, S. 2005. BayesX: analyzing Bayesian structural
additive regression models. Journal of Statistical Software, Articles, 14(11), 1–22.
(Cited on pages 174, 194, and 195.)
Burnham, K. P., and Anderson, D. R. 2002. Model Selection and Multimodel Inference:
A Practical Information-Theoretic Approach. 2nd edn. Springer.
Carlin, B. P., and Chib, S. 1995. Bayesian model choice via Markov chain Monte Carlo.
Journal of the Royal Statistical Society, B, 57(3), 473–484. (Cited on page 138.)
Carlin, B. P., and Gelfand, A. E. 1991. An iterative Monte Carlo method for non-
conjugate Bayesian analysis. Statistics and Computing, 1(2), 119–128. (Cited on
page 67.)
Carlin, B. P., and Louis, T. A. 2009. Bayesian Methods for Data Analysis. CRC Press.
(Cited on pages viii and 80.)
Carpenter, B., Gelman, A., Hoffman, M., et al. 2017. Stan: a probabilistic programming
language. Journal of Statistical Software, Articles, 76(1), 1–32. (Cited on pages 174
and 188.)
Carvalho, C. M., Polson, N. G., and Scott, J. G. 2010. The horseshoe estimator for
sparse signals. Biometrika, 97(2), 465–480. (Cited on page 136.)
Celeux, G., Forbes, F., Robert, C. P., and Titterington, D. M. 2006. Deviance informa-
tion criteria for missing data models. Bayesian Analysis, 1(4), 651–673. (Cited on
page 81.)
Chen, M.-H. 1994. Importance-weighted marginal Bayesian posterior density esti-
mation. Journal of the American Statistical Association, 89, 818–824. (Cited on
page 60.)
Chen, M.-H., and Shao, Q. 1999. Monte Carlo estimation of Bayesian credible and
HPD intervals. Journal of Computational and Graphical Statistics, 8, 69–92. (Cited
on page 50.)
Chen, M.-H., Shao, Q., and Ibrahim, J. G. 2000. Monte Carlo Methods in Bayesian
Computation. Springer. (Cited on pages 56, 59, and 131.)
Chib, S. 1995. Marginal likelihood from the Gibbs output. Journal of the American
Statistical Association, 90(432), 1313–1321. (Cited on page 132.)
Chib, S., and Jeliazkov, I. 2001. Marginal likelihood from the Metropolis–Hastings
output. Journal of the American Statistical Association, 96(453), 270–281. (Cited
on pages 132 and 133.)
Christensen, R., Johnson, W., Hanson, T., and Branscum, A. 2011. Bayesian Ideas and
Data Analysis: An Introduction for Scientists and Statisticians. CRC Press. (Cited
on page viii.)
Cowles, M. K. 1994. Practical issues in Gibbs sampler implementation with applica-
tion to Bayesian hierarchical modelling of clinical trial data. PhD thesis, University
of Minnesota. (Cited on page 203.)
Cowles, M. K., and Carlin, B. P. 1996. Markov chain Monte Carlo convergence diag-
nostics: a comparative review. Journal of the American Statistical Association, 91,
883–904. (Cited on pages 118 and 201.)
Damien, P., Wakefield, J., and Walker, S. 1999. Gibbs sampling for Bayesian non-
conjugate and hierarchical models by using auxiliary variables. Journal of the Royal
Statistical Society, B, 61(2), 331–344. (Cited on page 108.)
Dawid, A. P. 1985. The impossibility of inductive inference. (invited discussion of
236 References

‘Self-calibrating priors do not exist’, by D. Oakes.). Journal of the American Statis-

tical Association, 80, 340–341. (Cited on page 14.)
Dellaportas, P., and Papageorgiou, I. 2006. Multivariate mixtures of normals with un-
known number of components. Statistics and Computing, 16(1), 57–68. (Cited on
page 150.)
Dellaportas, P., Forster, J. J., and Ntzoufras, I. 2002. On Bayesian model and vari-
able selection using MCMC. Statistics and Computing, 12(1), 27–36. (Cited on
pages 134, 139, and 140.)
Dempster, A. P., Laird, N. M., and Rubin, D. B. 1977. Maximum likelihood from
incomplete data via the EM algorithm. Journal of the Royal Statistical Society, B,
39(1), 1–38. (Cited on page 146.)
de Valpine, P., Turek, D., Paciorek, C. J., et al. 2017. Programming with models: Writ-
ing statistical algorithms for general model structures with NIMBLE. Journal of
Computational and Graphical Statistics, 26(2), 403–413. (Cited on page 174.)
Devroye, L. 1986. Non-Uniform Random Variate Generation. Springer. (Cited on
page 45.)
Doucet, A., and Lee, A. 2018. Sequential Monte Carlo methods. Pages 165–190 of:
Drton, M., Lauritzen, S. L., Maathuis, M., and Wainwright, M. (eds.), Handbook of
Graphical Models. CRC. (Cited on page 62.)
Doucet, A., Freitas, N. D., and Gordon, N. 2001. Sequential Monte Carlo Methods in
Practice. Springer. (Cited on page 62.)
Fahrmeir, L., and Tutz, G. 2001. Multivariate Statistical Modeling Based on General-
ized Linear Models. Springer. (Cited on page 163.)
Gelfand, A. E. 1996. Model determination using sampling-based methods. Pages 145–
161 of: Gilks, W. R., Richardson, S., and Spiegelhalter, D. J. (eds.), Markov Chain
Monte Carlo in Practice. Chapman & Hall. (Cited on pages 73, 76, 77, and 87.)
Gelfand, A. E., and Dey, D. K. 1994. Bayesian model choice: asymptotics and exact
calculations. Journal of the Royal Statistical Society, B, 56, 501–514. (Cited on
page 89.)
Gelfand, A. E., and Smith, A. F. M. 1990. Sampling-based approaches to calculating
marginal densities. Journal of the American Statistical Association, 85, 398–409.
(Cited on pages 51, 59, 60, and 176.)
Gelfand, A. E., Hills, S., Racine-Poon, A., and Smith, A. F. M. 1990. Illustration of
Bayesian inference in normal data models using Gibbs sampling. Journal of the
American Statistical Association, 85(412), 972–985. (Cited on page 168.)
Gelfand, A. E., Smith, A. F. M., and Lee, T. 1992. Bayesian analysis of constrained pa-
rameter and truncated data problems using Gibbs sampling. Journal of the American
Statistical Association, 87, 523–531.
Gelman, A., and Hill, J. 2006. Data Analysis Using Regression and Multi-
level/Hierarchical Models. Cambridge University Press. (Cited on page 187.)
Gelman, A., and Meng, X. L. 1996. Model checking and model improvement. Pages
189–202 of: Gilks, W. R., Richardson, S., and Spiegelhalter, D. J. (eds.), Markov
Chain Monte Carlo in Practice. Chapman & Hall. (Cited on page 76.)
Gelman, A., and Rubin, D. B. 1992. Inference from iterative simulation using multiple
sequences. Statistical Science, 7, 457–72. (Cited on page 201.)
Gelman, A., Carlin, J. B., Stern, H. S., et al. 2014a. Bayesian Data Analysis. Vol. 3.
Chapman and &/CRC Press. (Cited on page viii.)
 References 237

Gelman, A., Hwang, J., and Vehtari, A. 2014b. Understanding predictive information
criterion for Bayesian models. Statistics and Computing, 24, 997–1016.
Geman, S., and Geman, D. 1984. Stochastic relaxation, Gibbs distribution and the
Bayesian restoration of images. IEEE Transactions on Pattern Analysis and Machine
Intelligence, 6, 721–741. (Cited on pages 92 and 100.)
Gentle, J. E. 2004. Random Number Generation and Monte Carlo Methods. 2nd edn.
Springer. (Cited on page 45.)
Genz, A., and Kass, R. E. 1997. Subregion adaptative integration of functions having
a dominant peak. Journal of Computational and Graphical Statistics, 6, 92–111.
(Cited on page 55.)
George, E. I., and McCulloch, R. 1997. Approaches for Bayesian variable selection.
Statistica Sinica, 7, 339–373. (Cited on pages 134 and 135.)
George, E. I., and McCulloch, R. E. 1993. Variable selection via Gibbs sampling. Jour-
nal of the American Statistical Association, 88(423), 881–889. (Cited on pages 133
and 134.)
Geweke, J. 1989. Bayesian inference in econometric models using Monte Carlo inte-
gration. Econometrica, 57(02), 1317–39. (Cited on pages 54 and 70.)
Geweke, J. 1992. Evaluating the accuracy of sampling-based approaches to calculating
posterior moments. In: Bayesian Statistics 4. Clarendon Press. (Cited on page 201.)
Geweke, J. 2004. Getting it right. Journal of the American Statistical Association,
99(467), 799–804. (Cited on pages 129 and 130.)
Geyer, C. J. 1992. Practical Markov chain Monte Carlo (with discussion). Statistical
Science, 7, 473–511. (Cited on page 117.)
Gillies, D. 2001. Bayesianism and the fixity of the theoretical framework. Pages 363–
379 of: Corfield, J., and Williamson, J. (eds.), Foundations of Bayesianism. Kluwer
Academic Publishers. (Cited on page 12.)
Givens, G. H., and Hoeting, J. A. 2005. Computational Statistics. Wiley. (Cited on
page 99.)
Gradshteyn, I., and Ryzhik, I. 2007. Table of Integrals, Series, and Products, Jeffrey,
A., and Zwillinger, D. (eds.). Academic Press.
Green, P. J. 1995. Reversible jump Markov chain Monte Carlo computation and
Bayesian model determination. Biometrika, 82, 711–732. (Cited on page 140.)
Hastings, W. K. 1970. Monte Carlo sampling methods using Markov chains and their
applications. Biometrika, 57, 97–109. (Cited on page 92.)
Heidelberger, P., and Welch, P. 1983. Simulation run length control in the presence of
an initial transient. Operations Research, 31, 1109–1144. (Cited on page 201.)
Henderson, H. V., and Velleman, P. F. 1981. Building multiple regression models inter-
actively. Biometrics, 37, 391–411. (Cited on page 76.)
Hoff, P. D. 2009. A First Course in Bayesian Statistical Methods. Springer. (Cited on
page viii.)
Hoffman, M. D., and Gelman, A. 2014. The No-U-Turn sampler: Adaptively setting
path lengths in Hamiltonian Monte Carlo. Journal of Machine Learning Research,
15(1), 1593–1623. (Cited on page 188.)
Jaynes, E. T. 1968. Prior probabilities. IEEE Transactions on Systems, Science and
Cybernetics, 4, 227–291. (Cited on page 22.)
Jaynes, E. T. 2003. Probability Theory: The Logic of Science. Cambridge University
Press. (Cited on pages 14 and 21.)
238 References

Jordan, M. I., Ghahramani, Z., Jaakkola, T. S., and Saul, L. K. 1999. An introduction
to variational methods for graphical models. Machine Learning, 37(2), 183–233.
(Cited on page 166.)
Karabatsos, G. 2015. A menu-driven software package for Bayesian regression analy-
sis. The ISBA Bulletin, 22, 13–16. (Cited on page 174.)
Kass, R. E., and Raftery, A. E. 1995. Bayes factors. Journal of the American Statistical
Association, 90, 773–795. (Cited on page 87.)
Kass, R. E., and Wasserman, L. 1996. The selection of prior distributions by formal
rules. Journal of the American Statistical Association, 91, 1343–1370. (Cited on
page 18.)
Kempthorn, O., and Folks, L. 1971. Probability, Statistics and Data Analysis. Iowa
State University Press. (Cited on page 7.)
Kneib, T., Heinzl, F., Brezger, A., Bove, D., and Klein, N. 2014. BayesX: R Utilities
Accompanying the Software Package BayesX. R package version 0.2-9. (Cited on
page 195.)
Korner-Nievergelt, F., von Felten, S., Roth, T., et al. 2015. Bayesian Data Analysis in
Ecology Using Linear Models with R, BUGS, and Stan. Academic Press. (Cited on
page 175.)
Kruschke, J. 2011. Doing Bayesian Data Analysis: A Tutorial with R and BUGS. Aca-
demic Press/Elsevier. (Cited on page 175.)
Kruschke, J. 2014. Doing Bayesian Data Analysis: A Tutorial with R, JAGS and Stan.
Academic Press/Elsevier. (Cited on page 175.)
Kucukelbir, A., Tran, D., Ranganath, R., Gelman, A., and Blei, D. M. 2017. Automatic
differentiation variational inference. Journal of Machine Learning Research, 18(1),
430–474. (Cited on page 170.)
Kuhn, T. S. 1962. The Structure of Scientific Revolutions. University of Chicago Press.
(Cited on page 5.)
Kuo, L., and Mallick, B. 1998. Variable selection for regression models. Sankhya: The
Indian Journal of Statistics, Series B, 60(1), 65–81. (Cited on page 134.)
Lauritzen, S. L., and Spiegelhalter, D. J. 1988. Local computations with probabilities
on graphical structures and their application to expert systems. Journal of the Royal
Statistical Society, B, 50(2), 157–224. (Cited on page 176.)
Lin, D. 2013. Online learning of nonparametric mixture models via sequential vari-
ational approximation. Pages 395–403 of: Proceedings of the 26th International
Conference on Neural Information Processing Systems. USA: Curran Associates
Inc. (Cited on page 170.)
Lindgren, F., and Rue, H. 2015. Bayesian spatial modelling with R-INLA. Journal of
Statistical Software, Articles, 63(19), 1–25. (Cited on page 216.)
Lindgren, F., Rue, H., and Lindstrom, J. 2011. An explicit link between Gaussian
fields and Gaussian Markov random fields: the stochastic partial differential equation
approach. Journal of the Royal Statistical Society, B, 73(4), 423–498. (Cited on
page 216.)
Lindley, D. V. 1990. The 1988 Wald memorial lectures: the present position in Bayesian
statistics. Statistical Science, 5, 44–89. (Cited on page 11.)
Liu, J., and West, M. 2001. Combined parameter and state estimation in simulation-
based filtering. Pages 197–223 of: Doucet, A., de Freitas, N., and Gordon, N. (eds.),
Sequential Monte Carlo Methods in Practice. Springer. (Cited on page 66.)
 References 239

Lunn, D., Spiegelhalter, D., Thomas, A., and Best, N. 2009. The BUGS project: Evo-
lution, critique and future directions. Statistics in Medicine, 28(25), 3049–3067.
(Cited on page 176.)
MacEachern, S., and Berliner, L. 1994. Subsampling the Gibbs sampler. The American
Statistician, 48, 188–190.
Madigan, D., and York, J. 1995. Bayesian graphical models for discrete data. Interna-
tional Statistical Review, 63, 215–232. (Cited on page 135.)
Marin, J.-M., Pudlo, P., Robert, C. P., and Ryder, R. J. 2012. Approximate Bayesian
computational methods. Statistics and Computing, 22(6), 1167–1180. (Cited on
page 128.)
Mayo, D., and Kruse, M. 2001. Principles of inference and their consequences. Pages
381–403 of: Corfield, J., and Williamson, J. (eds.), Foundations of Bayesianism.
Kluwer Academic Publishers. (Cited on page 9.)
Metropolis, N., Rosenbluth, A. W., Rosenbluth, M. N., Teller, A. H., and Teller, E.
1953. Equation of state calculations by fast computing machines. J. Chem. Phys,
21, 1087–1092. (Cited on pages 92 and 99.)
Morris, J. S., Baggerly, K. A., and Coombes, K. R. 2003. Bayesian shrinkage estimation
of the relative abundance of mRNA transcripts using SAGE. Biometrics, 59, 476–
486. (Cited on page 124.)
Neal, R. M. 1997. Markov Chain Monte Carlo methods based on “slicing” the density
function. Technical Report. University of Toronto. (Cited on page 108.)
Neal, R. M. 2003. Slice sampling (with discussion). Annals of Statistics, 31, 705–767.
(Cited on page 108.)
Neal, R. M. 2011. MCMC using Hamiltonian dynamics. Chap. 5 of: Brooks, S.,
Gelman, A., Jones, G., and Meng, X.-L. (eds.), Handbook of Markov Chain Monte
Carlo. Chapman & Hall / CRC Press. (Cited on pages 109 and 188.)
Neuenschwander, B., Branson, M., and Gsponer, T. 2008. Critical aspects of the
Bayesian approach to phase I cancer trials. Statistics in Medicine, 27, 2420–2439.
(Cited on page 55.)
Newton, M. A., and Raftery, A. E. 1994. Approximate Bayesian inference by the
weighted likelihood bootstrap (with discussion). Journal of the Royal Statistical
Society, B, 56, 1–48. (Cited on page 88.)
Ntzoufras, I. 2009. Bayesian Modeling Using WinBUGS. (Cited on page 175.)
O’Hagan, A. 2010. Bayesian Inference, Vol. 2B. 3rd edn. Arnold. (Cited on pages 1, 9,
14, and 18.)
O’Quigley, J., Pepe, M., and Fisher, L. 1990. Continual reassessment method: A prac-
tical design for phase 1 clinical trials in cancer. Biometrics, 46(1), 33–48. (Cited on
page 47.)
Park, T., and Casella, G. 2008. The bayesian lasso. Journal of the American Statistical
Association, 103(482), 681–686. (Cited on page 136.)
Patil, V. H. 1964. The Behrens–Fisher problem and its Bayesian solution. Journal of
the Indian Statistical Association, 2, 21. (Cited on page 35.)
Paulino, C. D., and Singer, J. M. 2006. Análise de Dados Categorizados. Editora
Edgard Blücher.
Paulino, C. D., Soares, P., and Neuhaus, J. 2003. Binomial regression with misclassifi-
cation. Biometrics, 59, 670–675. (Cited on page 17.)
240 References

Paulino, C. D., Amaral Turkman, M. A., Murteira, B., and Silva, G. 2018. Estatística
Bayesiana. 2nd edn. Fundacão Calouste Gulbenkian. (Cited on pages 18, 39, 56,
59, 86, 95, 161, and 201.)
Pitt, M. K., and Shephard, N. 1999. Filtering via simulation: Auxiliary particle fil-
ters. Journal of the American Statistical Association, 94(446), 590–599. (Cited on
pages 63, 64, and 65.)
Plummer, M. 2003. JAGS: a program for analysis of Bayesian graphical models using
Gibbs sampling. In: Hornik, K., Leisch, F., and Zeileis, A. (eds.), 3rd International
Workshop on Distributed Statistical Computing (DSC 2003). (Cited on page 174.)
Plummer, M. 2012. JAGS Version 3.3.0 User Manual. http://mcmc-jags.
sourceforge.net, accessed on July 22, 2018. (Cited on page 183.)
Plummer, M., Best, N. G., Cowles, M. K., and Vines, S. K. 2006. CODA: Conver-
gence diagnostics and output analysis for MCMC. R News, 6(1), 7–11. (Cited on
pages 118, 200, and 203.)
Polson, N. G., Stroud, J. R., and Müller, P. 2008. Practical filtering with sequential
parameter learning. Journal of the Royal Statistical Society, B, 70(2), 413–428.
(Cited on page 66.)
Prado, R., and West, M. 2010. Time Series: Modeling, Computation, and Inference.
Chapman & Hall/CRC Press. (Cited on page 66.)
Raftery, A. L., and Lewis, S. 1992. How many iterations in the Gibbs sampler? Pages
763–74 of: Bernardo, J., Berger, J., Dawid, A., and Smith, A. (eds.), Bayesian Statis-
tics IV. Oxford University Press. (Cited on page 201.)
Raftery, A. E., Madigan, D., and Hoeting, J. A. 1997. Bayesian model averaging for
linear regression models. Journal of the American Statistical Association, 92(437),
179–191. (Cited on page 135.)
Richardson, S., and Green, P. J. 1997. On Bayesian analysis of mixtures with an un-
known number of components (with discussion). Journal of the Royal Statistical
Society, B, 59(4), 731–792. (Cited on page 143.)
Rickert, J. 2018. A first look at NIMBLE. Blog: https://rviews.rstudio.com/
2018/07/05/a-first-look-at-nimble/, accessed on July 16, 2018. (Cited on
page 174.)
Ripley, B. D. 1987. Stochastic Simulation. Wiley. (Cited on pages 45 and 46.)
Robert, C. P. 1994. The Bayesian Choice. Springer. (Cited on pages 27 and 159.)
Robert, C. R., and Casella, G. 2004. Monte Carlo Statistical Methods. 2nd edn. New
York: Springer. (Cited on pages 46 and 98.)
Rosner, B. 1999. Fundamentals of Biostatistics. Duxbury. (Cited on page 224.)
Ross, S. M. 2014. Introduction to Probability Models, 11th ed. Academic Press. (Cited
on page 93.)
Rossi, P. E., Allenby, G. M., and McCulloch, R. 2005. Bayesian Statistics and Market-
ing. Wiley. (Cited on page 174.)
Ročková, V., and George, E. I. 2014. EMVS: The EM approach to Bayesian variable se-
lection. Journal of the American Statistical Association, 109(506), 828–846. (Cited
on page 145.)
Rubinstein, R. Y. 1981. Simulation and the Monte Carlo Method. 1st edn. Wiley. (Cited
on page 46.)
Rue, H., and Held, L. 2005. Gaussian Markov Random Fields: Theory and Applica-
tions. Chapman & Hall. (Cited on page 161.)
 References 241

Rue, H., Martino, S., and Chopin, N. 2009. Approximate Bayesian inference for latent
Gaussian models by using integrated nested Laplace approximations. Journal of the
Royal Statistical Society, B, 71(2), 319–392. (Cited on pages 152, 153, 164, 165,
166, 172, 216, and 219.)
Schofield, M. R., Barker, R. J., Gelman, A., Cook, E. R., and Briffa, K. 2016. A model-
based approach to climate reconstruction using tree-ring data. Journal of the Amer-
ican Statistical Association, 2016, 93–106. (Cited on page 187.)
Schwarz, G. 1978. Estimating the dimension of a model. Annals of Statistics, 6, 461–
466. (Cited on pages 79 and 85.)
Scott, S., Blocker, A., Bonassi, F., et al. 2016. Bayes and big data: the consensus Monte
Carlo algorithm. International Journal of Management Science and Engineering
Management, 11(2), 78–88. (Cited on page 70.)
Shaw, J. E. H. 1988. Aspects of numerical integration and summarization. Pages 625–
631 of: Bernardo, J. M., DeGroot, M. H., Lindley, D. V., and Smith, A. F. M. (eds.),
Bayesian Statistics 3. Oxford: University Press. (Cited on page 54.)
Silverman, B. W. 1986. Density Estimation for Statistics and Data Analysis. London:
Chapman and Hall.
Smith, A. F. M. 1991. Bayesian computation methods. Phil. Trans. R. Soc. Lond. A,
337, 369–386.
Smith, A. F. M., and Gelfand, A. E. 1992. Bayesian statistics without tears. The Amer-
ican Statistician, 46, 84–88. (Cited on page 60.)
Smith, B. 2007. BOA: An R package for MCMC output convergence assessment and
posterior inference. Journal of Statistical Software, 21, 1–37. (Cited on pages 118,
200, and 204.)
Spiegelhalter, D. J. 1986. Probabilistic prediction in patient management and clinical
trials. Statistics in Medicine, 5(5), 421–433. (Cited on page 176.)
Spiegelhalter, D. J., Best, N. G., Carlin, B. P., and van der Linde, A. 2002. Bayesian
measures of model complexity and fit (with discussion). Journal of the Royal Sta-
tistical Society, B, 64, 583–639. (Cited on pages 80 and 81.)
Stan Development Team. 2014. RStan: The R Interface to Stan, Version 2.5.0. (Cited
on page 188.)
Sturtz, S., Ligges, U., and Gelman, A. 2005. R2WinBUGS: a package for running
WinBUGS from R. Journal of Statistical Software, 12(3), 1–16. (Cited on page 177.)
Tanner, M. A. 1996. Tools for Statistical Inference. 3rd edn. New York: Springer Verlag.
(Cited on page 159.)
Tanner, M. A., and Wong, W. H. 1987. The calculation of posterior distributions by data
augmentation. Journal of the American Statistical Association, 82(398), 528–540.
(Cited on page 107.)
Thall, P. F., Millikan, R. E., Müller, P., and Lee, S.-J. 2003. Dose-finding with two
agents in phase i oncology trials. Biometrics, 59(3), 487–496. (Cited on pages 128
and 129.)
Thomas, A., O’Hara, B., Ligges, U., and Sturtz, S. 2006. Making BUGS open. R News,
6(01), 12–17. (Cited on page 174.)
Tibshirani, R. 1996. Regression shrinkage and selection via the Lasso. Journal of the
Royal Statistical Society, B, 58(1), 267–288. (Cited on page 136.)
Tierney, L. 1994. Markov chains for exploring posterior distributions. Annals of Statis-
tics, 22, 1701–1728. (Cited on page 98.)
242 References

Tierney, L. 1996. Introduction to general state-space Markov chain theory. Pages 61–74
of: Gilks, W., Richardson, S., and Spiegelhalter, D. (eds.), In Markov Chain Monte
Carlo in Practice. Chapman. (Cited on page 93.)
Tierney, L., and Kadane, J. 1986. Accurate approximations for posterior moments and
marginal densities. Journal of The American Statistical Association, 81(03), 82–86.
(Cited on pages 156 and 164.)
Tierney, L., Kass, R., and Kadane, J. 1989. Fully exponential laplace approximations
to expectations and variances of nonpositive functions. Journal of the American
Statistical Association, 84(407), 710–716. (Cited on pages 158 and 159.)
Umlauf, N., Adler, D., Kneib, T., Lang, S., and Zeileis, A. 2015. Structured additive re-
gression models: An r interface to BayesX. Journal of Statistical Software, Articles,
63(21), 1–46. (Cited on pages 195, 196, and 197.)
Vehtari, A., and Ojanen, J. 2012. A survey of Bayesian predictive methods for model
assessment, selection and comparison. Statist. Surv., 6, 142–228.
Vehtari, A., Gelman, A., and Gabry, J. 2017. Practical Bayesian model evaluation using
leave-one-out cross-validation and WAIC. Statistics and Computing, 27(5), 1413–
1432. (Cited on pages 190 and 194.)
Walker, A. M. 1969. On the asymptotic behaviour of posterior distributions. Journal of
the Royal Statistical Society, B, 31(1), 80–88. (Cited on page 153.)
Wasserman, L. 2004. All of Statistics. Springer-Verlag. (Cited on page 14.)
Watanabe, S. 2010. Asymptotic equivalence of Bayes cross validation and widely appli-
cable information criterion in singular learning theory. Journal of Machine Learning
Research, 11(Dec.), 3571–3594. (Cited on page 82.)
Welling, M., and Teh, Y. W. 2011. Bayesian learning via stochastic gradient Langevin
dynamics. Pages 681–688 of: Proceedings of the 28th International Conference on
International Conference on Machine Learning. Omnipress. (Cited on page 114.)
Zhang, Z., Chan, K. L., Wu, Y., and Chen, C. 2004. Learning a multivariate Gaussian
mixture model with the reversible jump MCMC algorithm. Statistics and Comput-
ing, 14(4), 343–355. (Cited on page 148.)
 Index

ABC, see approximate Bayesian Geweke, 201

computation Heidelberg and Welch, 202
approximate Bayesian computation, 128 Raftery and Lewis, 202
Bayes factor, 11, 85 coordinate ascent variational inference,
importance sampling estimate, 58 167
pseudo, see pseudo-Bayes factor CPO, see conditional predictive ordinate
Bayes’ theorem, 7 INLA, 220
Bayesian data augmentation, 119, 120
credible interval, 10 detailed balance condition, 96
estimation, 8 deviance, 81
inference, 6 Bayesian, 80
p-value, 73 deviance information criterion, see
testing, 11 information criterion, DIC
Bayesian residuals, 89 diagnostic summary, 84
leaving-one-out, 75 DIC, see information criterion, DIC
predictive, 75 Dirichlet distribution, 37, 229
BayesX, 194 discrepancy measure, 73
BOA, 214 effective number of parameters, 81
CODA, 214 ELBO, see evidence lower bound
Bernoulli distribution, 226 entropy, 21
beta distribution, 30, 35, 41, 227 ergodic theorem, 95
beta-binomial distribution, 226 Erlang distribution, 25, 228
beta-negative binomial distribution, 227 evidence lower bound, 166
binomial distribution, 24, 30, 41, 226 exponential distribution, 41, 228
two samples, 35
BOA, 181, 200, 203 Fisher–Snedecor distribution, 229
BUGS, see OpenBUGS frequentist
confidence intervals, 3
CAVI, see coordinate ascent variational estimation, 3
inference testing, 4
Chi-square distribution, 228 full conditional distribution, 100
classical inference, 2
CODA, 181, 200, 203 gamma distribution, 31, 41, 42, 228
conditional predictive ordinate, 72, 75, geometric distribution, 25, 226
84, 218, 220 Gibbs sampler
conjugate family, 24 basic, 101
consensus Monte Carlo, 70 blocking, 102
convergence diagnostic, 116, 200 data augmentation, 106
Gelman and Rubin, 201 hybridization, 102
244 Index

HPD credible sets, 10 independence chain, 99

hypergeometric distribution, 227 random walk proposal, 99
importance sampling, see Monte Carlo, mixture model, 122, 146
importance sampling multivariate, 148
information criterion Monte Carlo
AIC, 79 consensus, see consensus Monte Carlo
BIC, 80 importance sampling, 54
DIC, 81, 218 marginal distribution, 51
WAIC, 82, 218 method of Chib, 132
INLA, 152, 163, 215 simple, 46
integrated nested Laplace approximation, multinomial distribution, 37, 229
see INLA multinomial-Dirichlet distribution, 229
inverse gamma distribution, 32, 228 multivariate normal/Wishart distribution,
230
JAGS, 183
BOA, 212 NDLM, see normal dynamic linear
CODA, 212 model
negative binomial distribution, 226
Laplace approximation, 156 normal distribution, 228
complete, 165 multivariate, 230
simplified, 165 two samples, 34
latent Gaussian models, 152, 161 unknown mean, 25
leaving-one-out cross-validation, 75 unknown mean and variance, 33
LGM, see latent Gaussian models unknown variance, 32
likelihood principle, 27 normal dynamic linear model, 61
linear regression, 43, 126, 145 normal/inverse gamma distribution, 228
log pseudo-marginal likelihood, 76
LPML, see log pseudo-marginal Occam’s razor, 83
likelihood OpenBUGS, 176
marginal distribution p-value
importance sampling estimate, 59 Bayesian, 84
Markov chain Pareto distribution, 43, 229
aperiodic, 95 particle filter
definition, 93 adapted, 64
ergodic, 95 auxiliary, 63
homogeneous, 93 parameter learning, 65
irreducible, 94 Poisson distribution, 31, 41, 227
Metropolis–Hastings, see Poisson-gamma distribution, 227
Metropolis–Hastings posterior approximation
positive recurrent, 94 multivariate normal, 153
reversible, 95 posterior distribution, 7
Markov chain Monte Carlo marginal, 8
data augmentation, see data posterior plausibility, 12
augmentation posterior predictive distribution, 87
Gibbs sampler, see Gibbs sampler predictive distribution
Hamiltonian, 109 posterior, 13, 73
missing data, 120 prior, 7
reversible jump, 140, 150 prior
MCMC, see Markov chain Monte Carlo, conjugate, 23
92 Jeffreys, 19, 33, 34
measures of discrepancy, 76 maximum entropy, 21
Metropolis–Hastings, 96 non-informative, 18
 Index 245

probability
frequentist, 3
subjective, 6
probit regression, 121
pseudo-Bayes factor, 84
pseudo-prior, 138
metropolized, 139
R-INLA, 216
Rao–Blackwellization, 52
Rayleigh distribution, 42, 228
repeated sampling, 3
slice sampler, 107
Stan, 187
BOA, 213
CODA, 213
state space model, 62
stationary distribution, 94
Student t distribution, 68, 228
multivariate, 230
t distribution, see Student t distribution
transition function, 93
uniform distribution, 43, 227
variable selection
Bayesian lasso, 136
Dirichlet–Laplace, 136
horseshoe, 136
MC3 , 135
SSVS, 133, 145
variational Bayes, 166
Stan, 225
variational family, 166
variational inference, 166
mean-field, 166
Weibull distribution, 228
WinBUGS, 176
BOA, 205
CODA, 205
Wishart distribution, 230