Chapter 2 The Model Matrix and Random Effects - Bayesian Hierarchical Models in Ecology
Chapter 2 The Model Matrix and Random Effects - Bayesian Hierarchical Models in Ecology
https://bookdown.org/steve_midway/BHME/Ch1.html 1/18
4/5/24, 6:39 PM Chapter 2 The Model Matrix and Random Effects | Bayesian Hierarchical Models in Ecology
Figure 2.1: Although hurricanes are not statistical models, they are a good example
of understanding something that cannot perfectly be modeled, and therefore has
some stochasticity inherent to it.
https://bookdown.org/steve_midway/BHME/Ch1.html 2/18
4/5/24, 6:39 PM Chapter 2 The Model Matrix and Random Effects | Bayesian Hierarchical Models in Ecology
2.1.2 Distributions
https://bookdown.org/steve_midway/BHME/Ch1.html 3/18
4/5/24, 6:39 PM Chapter 2 The Model Matrix and Random Effects | Bayesian Hierarchical Models in Ecology
## [1] 0 1 2 1 0 1 2 2 3 1
## [1] 2 1 8 6 2 4 4 2 2 2
Recall that the linear contribution to our model includes the predictors (explanatory
variables) with additive effects. Although this is a linear model, it need not be
thought of literally as a straight line. The linear component can accomodate both
continuous and categorical data. Conceptually,
R takes care of both the design matrix and the parameterization, but this is not
always true in JAGS , so it is worth a review.
“For each element of the response vector, the design matrix n index indicates
which effect is present for categorical (discrete) explanatory variables and what
amount of an effect is present in the case of continuous explanatory variables.”
https://bookdown.org/steve_midway/BHME/Ch1.html 4/18
4/5/24, 6:39 PM Chapter 2 The Model Matrix and Random Effects | Bayesian Hierarchical Models in Ecology
(Kéry 2010)
In other words the design matrix is a matrix that tells the model if and where an
effect is present, and how much. The number of columns in the matrix are equal to
the number of fitted parameters. Ultimatly, the design matrix is multiplied with the
parameter vector to produce the linear predictor. Let’s use the simulated data in
Chapter 6 of Marc Kery’s book as an example (Kéry 2010).
6 1 1 1 40
8 1 1 2 45
5 2 1 3 39
7 2 1 1 50
9 3 2 2 52
11 3 2 3 57
If you would like to code this dataset into R and play along, use the code below.
massi = μ + ϵi
This model has a covariate of one for every observation, because every
observation has a mass. This model is also sometimes referred to as an intercept
only model. The model describes individual snake mass as a (global) mean plus
individual variation (deviation or “error”).
https://bookdown.org/steve_midway/BHME/Ch1.html 5/18
4/5/24, 6:39 PM Chapter 2 The Model Matrix and Random Effects | Bayesian Hierarchical Models in Ecology
massi = α + β × regioni + ϵi
2
ϵi ∼ N (0, σ )
Let’s pause for a moment on the t-test and think about how we write models. You
are familiar with the notation that I have thus used, where the model and residual
error are expressed as two separate equations. What if we were to combine them
into one equation? How would that look?
2
massi ∼ N (α + β × regioni , σ )
This notation may take a little while to get used to, but it should make sense. This
expression is basically saying that snake mass is thought to be normally distributed
with a mean that is the function of the mean mass plus region effect, and with
some residual error. Expressing models with distributional notation may seen odd
at first, but it may help you better understand how distributions are built into the
processes we are modeling, and what part of the process belongs to which part of
the distribution. We will stick with this notation (not exclusively) for much of this
course.
model.matrix(mass ~ region)
https://bookdown.org/steve_midway/BHME/Ch1.html 6/18
4/5/24, 6:39 PM Chapter 2 The Model Matrix and Random Effects | Bayesian Hierarchical Models in Ecology
## (Intercept) region
## 1 1 1
## 2 1 1
## 3 1 1
## 4 1 1
## 5 1 2
## 6 1 2
## attr(,"assign")
## [1] 0 1
But we said earlier the regions were not inherent quantities and just categories or
indicator variables. Above, R is taking the region variable as numeric because it
does not know any better. Let’s tell R that region is a factor.
model.matrix(mass ~ as.factor(region))
## (Intercept) as.factor(region)2
## 1 1 0
## 2 1 0
## 3 1 0
## 4 1 0
## 5 1 1
## 6 1 1
## attr(,"assign")
## [1] 0 1
## attr(,"contrasts")
## attr(,"contrasts")$`as.factor(region)`
## [1] "contr.treatment"
The model matrix yields a system of equations. And for the matrix above, the
system of equations we would have can be expressed 2 ways. The first way shows
the equation for each observation.
6 = α × 1 + β × 0 + ϵ1
https://bookdown.org/steve_midway/BHME/Ch1.html 7/18
4/5/24, 6:39 PM Chapter 2 The Model Matrix and Random Effects | Bayesian Hierarchical Models in Ecology
8 = α × 1 + β × 0 + ϵ2
5 = α × 1 + β × 0 + ϵ3
7 = α × 1 + β × 0 + ϵ4
9 = α × 1 + β × 1 + ϵ5
11 = α × 1 + β × 1 + ϵ6
The second way adopts matrix (vector) notation to economize the system.
6 1 0 ϵ1
⎛ ⎞ ⎛ ⎞ ⎛ ⎞
⎜ 8 ⎟ ⎜1 0⎟ ⎜ ϵ2 ⎟
⎜ ⎟ ⎜ ⎟ ⎜ ⎟
⎜ 5 ⎟ ⎜1 0⎟ α ⎜ϵ ⎟
3
⎜ ⎟ ⎜ ⎟ ⎜ ⎟
= × ( ) +
⎜ ⎟ ⎜ ⎟ ⎜ ⎟
⎜ 7 ⎟ ⎜1 0⎟ β ⎜ ϵ4 ⎟
⎜ ⎟ ⎜ ⎟ ⎜ ⎟
⎜ 9 ⎟ ⎜1 1⎟ ⎜ϵ ⎟
5
⎝ ⎠ ⎝ ⎠ ⎝ ⎠
11 1 1 ϵ6
In both cases the design matrix and parameter vector are combined. And using
least-squares estimation to estimate the parameters will result in the best fits for α
and β.
2.1.4 Parameterization
Although parameters are inherent to specific models, there are cases in the linear
model where parameters can be represented in different ways in the design matrix,
and these different representations will have different interpretations. Typically, we
refer to either a means or effects parameterization. These two parameterizations
really only come into play when you have a categorical variable—the
representation of continuous variables in the model matrix includes the quantity or
magnitude of that variable (and defaults to an effects parameterization). A means
parameterization may be present or needed when categorical variables are
present. Let’s take the above t-test example with 2 regions. As we notated the
model above, α = μ for region 1, and β represented the difference in expected
mass in region 2 compared to region 1. In other words, β is an estimate of the
effect of being in region 2. As you guessed, this is an effects parameterization
because region 1 is considered a baseline or reference level, and other levels are
compared to this reference, hence the coefficients represent their effect. Effects
parameterization is the default in R (e.g., lm() ).
https://bookdown.org/steve_midway/BHME/Ch1.html 8/18
4/5/24, 6:39 PM Chapter 2 The Model Matrix and Random Effects | Bayesian Hierarchical Models in Ecology
In a means parameterization, α has the same interpretation—the mean for the first
group. However, in a means parameterization, all other coefficients represent the
means of those groups, which is actually a simpler imterpretation than the effects
parameterization. Going back to our t-test, in a means parameterization β would
represent the mean mass for snakes in region 2. Thus, no addition or subtraction of
effects is required in a means parameterization. Both the means and effects
parameterization will yield the same estimates (with or without some simple math),
but it is advised to know which parameterization you are using. Additionally, as you
work in JAGS you may need to use one parameterization over another, and
understanding how their work and are interpreted is critically important.
Let’s look at one more model using the snake data. A simple linear regression will
fit one continuous covariate (x) as a predictor on y. Let’s look at the model for the
snout-vent length ( svl ) as the predictor on mass .
massi = α + β × svli + ϵi
The model matrix for this model (an effect parameterization) can be reported as
such.
model.matrix(mass ~ svl)
## (Intercept) svl
## 1 1 40
## 2 1 45
## 3 1 39
## 4 1 50
## 5 1 52
## 6 1 57
## attr(,"assign")
## [1] 0 1
And we see that each snake has an intercept (mean) and an effect of svl .
These examples using the fake snake data were taken from Kery (2010), in which
he provides more context and examples that you are encouraged to review.
https://bookdown.org/steve_midway/BHME/Ch1.html 9/18
4/5/24, 6:39 PM Chapter 2 The Model Matrix and Random Effects | Bayesian Hierarchical Models in Ecology
1. What is variance?
2. How do we deal with variance?
3. What can random effects do?
4. How do I know if I need random effects?
5. What is a hierarchical model? (later)
Variance was historically seen as noise or a distraction—it is the part of the data
that the model does not capture well. At times it has been called a nuisance
parameter. The traditional view was always that less variance was better. However:
The good news is that there is (often) information in variance. Two problems,
however, are that our models don’t always account for the information connected to
variance, and that variance is often ubiquitous and can occur at numerous places
within the model. For instance, variance can occur among observations and within
observations, along with how we parameterize our models and how we take our
measurements.
One way to deal with variance concerns how we treat the factors in our model.
(Recall that a factor is a categorical predictor that has 2 or more levels.)
Specifically, we can treat our factors as fixed or random, and the underlying
difference lies in how the factor levels interact with each other.
Fixed effects are those model effects that are thought to be separate, independent,
and otherwise not similar to each other.
Likely more common—at least in use—than random effects, if only for the fact
that they are a statistical default in most statistical softwares.
https://bookdown.org/steve_midway/BHME/Ch1.html 10/18
4/5/24, 6:39 PM Chapter 2 The Model Matrix and Random Effects | Bayesian Hierarchical Models in Ecology
Random effects are those model effects that can be thought of as units from a
distribution, almost like a random variable.
Random effects are less commonly used, but perhaps more commonly
encountered (than fixed effects).
Each factor level can be thought of as a random variable from an underlying
process or distribution.
Estimation provides inference about specific levels, but also the population-
level (and thus absent levels!)
Exchangeability
If n is large, factor level estimates are the same or similar to fixed effect
estimates.
If n is small, factor levels estimates draw information from the population-level
information.
https://bookdown.org/steve_midway/BHME/Ch1.html 11/18
4/5/24, 6:39 PM Chapter 2 The Model Matrix and Random Effects | Bayesian Hierarchical Models in Ecology
“The statistical literature is full of confusing and contradictory advice.” (Gelman and
Hill 2006)
Factor levels are very low (e.g., 2); it’s hard to estimate distributional
parameters with so little information (but still little risk).
Or when you don’t want factor levels to inform each other.
e.g., male and female growth (combined estimate could be meaningless and
misleading)
https://bookdown.org/steve_midway/BHME/Ch1.html 12/18
4/5/24, 6:39 PM Chapter 2 The Model Matrix and Random Effects | Bayesian Hierarchical Models in Ecology
Random Effects Equation Notation There will be a much more extensive treatment
of model notation and you will need to become familari with notation to successfully
code models; however, now is as good a time as any to introduce some basics of
random effects statistical notation. (Note that statistical notation and code notation
are different, but may be unspecified and unclear when referring to model
notation.)
SLR, no RE:
y i = α + β × xi + ϵ i
y i = α j + β × xi + ϵ i
y i = α + β j × xi + ϵ i
y i = α j + β j × xi + ϵ i
1 1 2 2
yi = α + β × x + β × x + ϵi
j i j i
For the most part, we will use subscript i to index the observation-level (data) and
subscript j to index groups (and indicate a random effect).
https://bookdown.org/steve_midway/BHME/Ch1.html 13/18
4/5/24, 6:39 PM Chapter 2 The Model Matrix and Random Effects | Bayesian Hierarchical Models in Ecology
Because random effects are used to model variation at different levels of the data,
they add a hierarchical structure to the model. Let’s start with non-hierarchical
models and review how data inform parameters. Consider a simple model with no
hierarchical strucure (Figure 2.3). The observations (yi . . . yn ) all inform one
parameter and the data are said to be pooled.
Perhaps we want to group the data in some logical way. We might add a fixed
effect, and then assign certain observations to different parameters (Figure 2.4). In
this case, our parameters, θ are subscripted to indicate that they are different
levels within a factor. Despite this apparent connection, the data and the
parameters inherent to one group inform only that group, and there is said to be no
pooling or sharing of information.
https://bookdown.org/steve_midway/BHME/Ch1.html 14/18
4/5/24, 6:39 PM Chapter 2 The Model Matrix and Random Effects | Bayesian Hierarchical Models in Ecology
https://bookdown.org/steve_midway/BHME/Ch1.html 15/18
4/5/24, 6:39 PM Chapter 2 The Model Matrix and Random Effects | Bayesian Hierarchical Models in Ecology
Once you understand some examples of how hierarchical structures reflect the
ecological systems we seek to model, you will find hierarchical models to be an
often realistic representation of the system or process you want to understand—or
at least more realistic than the traditional model representations.
“The existence of the process model is central to the hierarchical modeling view.
We recognize two basic types of hierarchical models. First is the hierarchical model
that contains an explicit process model, which describes realizations of an actual
ecological process (e.g., abundance or occurrence) in terms of explicit biological
quantities. The second type is a hierarchical model containing an implicit process
model. The implicit process model is commonly represented by a collection of
random effects that are often spatially and or temporally indexed. Usually the
implicit process model serves as a surrogate for a real ecological process, but one
that is diffcult to characterize or poorly informed by the data (or not at all).” Royle
and Dorazio (2008)
https://bookdown.org/steve_midway/BHME/Ch1.html 16/18
4/5/24, 6:39 PM Chapter 2 The Model Matrix and Random Effects | Bayesian Hierarchical Models in Ecology
“However, the term, hierarchical model, by itself is about as informative about the
actual model used as it would be to say that I am using a four-wheeled vehicle for
locomotion; there are golf carts, quads, Smartcars, Volkswagen, and Mercedes, for
instance, and they all have four wheels. Similarly, plenty of statistical models
applied in ecology have at least one set of random effects (beyond the residual)
and therefore represent a hierarchical model. Hence, the term is not very
informative about the particular model adopted for inference about an animal
population.” Kéry and Schaub (2011)
References
Gelman, Andrew, and Jennifer Hill. 2006. Data Analysis Using Regression and
Multilevel/Hierarchical Models. Cambridge university press.
Kéry, Marc, and Michael Schaub. 2011. Bayesian Population Analysis Using
Winbugs: A Hierarchical Perspective. Academic Press.
Leslie, Paul W, and Michael A Little. 2003. “Human Biology and Ecology: Variation
in Nature and the Nature of Variation.” American Anthropologist 105 (1). Wiley
Online Library: 28–37.
Lindley, Dennis V, and Adrian FM Smith. 1972. “Bayes Estimates for the Linear
Model.” Journal of the Royal Statistical Society: Series B (Methodological) 34 (1).
Wiley Online Library: 1–18.
Royle, J Andrew, and Robert M Dorazio. 2008. Hierarchical Modeling and Inference
in Ecology: The Analysis of Data from Populations, Metapopulations and
Communities. Elsevier.
Wagner, Tyler, Daniel B Hayes, and Mary T Bremigan. 2006. “Accounting for
Multilevel Data Structures in Fisheries Data Using Mixed Models.” Fisheries 31 (4).
Taylor & Francis: 180–87.
https://bookdown.org/steve_midway/BHME/Ch1.html 17/18
4/5/24, 6:39 PM Chapter 2 The Model Matrix and Random Effects | Bayesian Hierarchical Models in Ecology
https://bookdown.org/steve_midway/BHME/Ch1.html 18/18