What? General linear model Examples When? Why? How? Who?

Introduction
Linear Models: Introduction Yao-ban Chan
What? General linear model Examples When? Why? How? Who?
Statistics
Statistics is a collection of tools for quantitative research, the main
aspects of which are:
Questions

Study
design

Data
collection

Data
display

Modelling
Formulation
Estimation
Checking
Inference

Conclusions
(answers)

Reporting
results
Linear Models: Introduction Yao-ban Chan
What? General linear model Examples When? Why? How? Who?
What is a linear model?
A linear model is one of many types of models that we can use in
the modelling phase. It assumes that the data we measure have
some sort of linear relationship to other explanatory sets of data
(give or take a small amount of error).
Generally speaking the linear model is the nicest model we can
use. It makes certain assumptions, which are mainly used to make
the model easy to analyse. It is (relatively) restrictive, although it
encompasses many situations.
Many of you will have seen at least one kind of linear model
linear regression. However linear models are much more exible
than that.
Linear Models: Introduction Yao-ban Chan
What? General linear model Examples When? Why? How? Who?
In this course, we will take an in-depth look at the theory behind
linear models, and how this theory can be applied. We will also
show how some practical calculations can be done using the R
statistical package.
Linear Models: Introduction Yao-ban Chan
What? General linear model Examples When? Why? How? Who?
The general linear model
The general linear model encompasses all linear models. It is a way
of describing data that we have measured from some statistical
experiment. Suppose that we have run some kind of statistical
experiment that has produced data:
We have n subjects, labelled 1 to n;
We wish to analyse or predict the behaviour of a measurement
or property of the subject (y variable);
Denoted by y
1
, y
2
, . . . , y
n
.
Each subject has certain other properties that we know or
have pre-determined (x variables);
Subject i has k of these properties x
i 1
, x
i 2
, . . . , x
ik
.
Linear Models: Introduction Yao-ban Chan
What? General linear model Examples When? Why? How? Who?
The general linear model can be stated thus:
y
i
=
0
+
1
x
i 1
+
2
x
i 2
+ . . . +
k
x
ik
+
i
for all i = 1, 2, . . . , n.
Linear Models: Introduction Yao-ban Chan
What? General linear model Examples When? Why? How? Who?
We call y the response variable and the xs the design constants
(technically, they are not variable). The s are parameters of the
model, and is an error term.
All the terms excluding consist of the model. The model
attempts to explain the variation in the measured ys (if there were
no variation then the data would be rather boring!). However, not
all variation can be explained by deterministic data alone (and if it
could, the data would again be pretty boring!). There will always
be an error term .
Ideally, the error term is very small. In that case, the linear model
explains the response variable to a high degree of accuracy. We
also want the expected error to be 0 - if not, there is something
going on which the model is not covering!
Linear Models: Introduction Yao-ban Chan
What? General linear model Examples When? Why? How? Who?
Matrix formulation
By putting the data and other terms into matrices:
y =

y
1
y
2
.
.
.
y
n

, =

2
.
.
.

, =

2
.
.
.

Linear Models: Introduction Yao-ban Chan

1 x
11
x
12
. . . x
1k
1 x
21
x
22
. . . x
2k
.
.
.
.
.
.
.
.
.
.
.
.
1 x
n1
x
n2
. . . x
nk

then we can express the general linear model in matrix form:

y
1
y
2
.
.
.
y
n

1 x
11
x
12
. . . x
1k
1 x
21
x
22
. . . x
2k
.
.
.
.
.
.
.
.
.
.
.
.
1 x
n1
x
n2
. . . x
nk

2
.
.
.

2
.
.
.

y = X +
Linear Models: Introduction Yao-ban Chan
What? General linear model Examples When? Why? How? Who?
Note the dimensions of the matrices:
y is n 1;
X is n (k + 1);
is (k + 1) 1; and
is n 1.
Linear Models: Introduction Yao-ban Chan
What? General linear model Examples When? Why? How? Who?
Examples
Linear models can be used for many things, including (but not
limited to):
Which conditions aect the rate of banana ripening?
Is it better to wrap them in newspaper, or submerge them in
water?
Optimizing the choice of ISPs based on customer service
Comparing time spent in dierent companies customer service
queue
At dierent times of days and dierent days
Linear Models: Introduction Yao-ban Chan
What? General linear model Examples When? Why? How? Who?
Example
Examining the best brand of alkaline battery
Plugging them into dierent appliances and waiting for them
to run out
The eect of lifestyle factors on blood pressure
Taking into account factors like gender, age, BMI, height,
hours of work, hours of sleep, and number of dependents
Observing the performance of short-term memory for numbers
Looking at factors such as gender, exposure to mathematics,
duration of interval and presentation of the numbers
Linear Models: Introduction Yao-ban Chan
What? General linear model Examples When? Why? How? Who?
Examples plant data
Heights of 9 plants.
1. No other information.
height (y)
22 13 24 35 29 27 29 18 23
. . ... . : .
+---------+---------+---------+-height
10 20 30 40
Linear Models: Introduction Yao-ban Chan
What? General linear model Examples When? Why? How? Who?
Model: y
i
= +
i

y
1
.
.
.
y
9

22
13
24
35
29
27
29
18
23

1
.
.
.
1

[] +

1
.
.
.

y = X +
Linear Models: Introduction Yao-ban Chan
What? General linear model Examples When? Why? How? Who?
2. Soil moisture (x) given.
Moisture (x) Height (y)
204 22
121 13
261 24
460 35
468 29
299 27
308 29
235 18
188 23
Linear Models: Introduction Yao-ban Chan
What? General linear model Examples When? Why? How? Who?
150 200 250 300 350 400 450
15
20
25
30
35
Moisture
H
e
i
g
h
t
Linear Models: Introduction Yao-ban Chan
What? General linear model Examples When? Why? How? Who?
Model: y
i
=
0
+
1
x
i
+
i

22
13
24
35
29
27
29
18
23

1 204
1 121
1 261
1 460
1 468
1 299
1 308
1 235
1 188

1
.
.
.

y = X +
Linear Models: Introduction Yao-ban Chan
What? General linear model Examples When? Why? How? Who?
3. Three varieties.
Variety
1 2 3
22 13 27
24 18 29
29 23 35
Linear Models: Introduction Yao-ban Chan
What? General linear model Examples When? Why? How? Who?
1.0 1.5 2.0 2.5 3.0
15
20
25
30
35
Variety
H
e
i
g
h
t
Linear Models: Introduction Yao-ban Chan
What? General linear model Examples When? Why? How? Who?
Model I: y
ij
=
i
+
ij

y
1,1
y
1,2
y
1,3
y
2,1
y
2,2
y
2,3
y
3,1
y
3,2
y
3,3

22
24
29
13
18
23
27
29
35

1 0 0
1 0 0
1 0 0
0 1 0
0 1 0
0 1 0
0 0 1
0 0 1
0 0 1

1,1
.
.
.
.
.
.

3,3

y = X +
Linear Models: Introduction Yao-ban Chan
What? General linear model Examples When? Why? How? Who?
Model II: y
ij
= +
i
+
ij

y
1,1
y
1,2
y
1,3
y
2,1
y
2,2
y
2,3
y
3,1
y
3,2
y
3,3

22
24
29
13
18
23
27
29
35

1 1 0 0
1 1 0 0
1 1 0 0
1 0 1 0
1 0 1 0
1 0 1 0
1 0 0 1
1 0 0 1
1 0 0 1

1,1
.
.
.
.
.
.

3,3

y = X +
Linear Models: Introduction Yao-ban Chan
What? General linear model Examples When? Why? How? Who?
When is a model linear?
A model is linear when the response variable is predicted to be a
linear form of the parameters . Linearity in x is not needed.
Sometimes, a suitable transformation can make a non-linear model
into a linear model. However, we must be careful with the errors!
Linear Models: Introduction Yao-ban Chan
What? General linear model Examples When? Why? How? Who?
Which of the following are linear models?

0
+
1
x

0
+
1
x +
2
x
2

1
if male

2
if female

0
+
1
e

2
x

0
e

1
x
+
i
, where
i
refers to the levels (> 1) of a factor (A)
+
i
+
j
+()
ij
, where
i
(
j
) refers to the levels of factor
A (B) and ()
ij
refers to the interaction between A and B
Linear Models: Introduction Yao-ban Chan
What? General linear model Examples When? Why? How? Who?
Why should we use linear models?
Easy to analyse;
Easy to formulate;
Often very appropriate!
Linear Models: Introduction Yao-ban Chan
What? General linear model Examples When? Why? How? Who?
How do we t linear models?
This is the subject of the course I wouldnt want to spoil your
surprise!
Linear Models: Introduction Yao-ban Chan
What? General linear model Examples When? Why? How? Who?
Who should t linear models?
You!
(by the end of the course, hopefully)
Linear Models: Introduction Yao-ban Chan