Newsom 1

USP 656 Multilevel Regression

Winter 2006
Jkj..khh.kh.khk.jhb.jkk..kjhhlhlhhj!j!lj!j!j " situ"tion involving r"n#om e$$e%ts "n"l&ses. ' $i(e# e$$e%ts
'N)*' re$ers to "ssum+tions "bout the in#e+en#ent v"ri"ble "n# the error #istribution $or the
v"ri"ble. 'n e(+eriment"l #esign is the e"siest e("m+le $or illustr"ting the +rin%i+"l. Usu"ll& the
rese"r%her is intereste# in onl& gener"li,ing the results to e(+eriment"l v"lues use# in the stu#&. -or
inst"n%e " #rug stu#& using 0 mg 5 mg or .0 mg o$ "n e(+eriment"l #rug. /his is when " $i(e#
e$$e%ts 'N)*' woul# be "++ro+ri"te. 0n this %"se the e(tr"+ol"tion is to other stu#ies or tre"tments
th"t might use the s"me v"lues o$ the #rug 1i.e. 0 mg 5 mg "n# .0 mg2. 3owever i$ the rese"r%her
w"nts to m"ke in$eren%es be&on# the +"rti%ul"r v"lues o$ the in#e+en#ent v"ri"ble use# in the stu#&
" r"n#om e$$e%ts mo#el is use#. ' %ommon e("m+le woul# be the use o$ +ubli% "rt works
re+resenting low mo#er"te "n# high "bstr"%tness 1e.g. st"tue o$ " w"r hero vs. " +ivoting geometri%
#esign2. /he rese"r%her woul# like to m"ke in$eren%es be&on# the three +ie%es use# so the "rt
+ie%es "re %on%e+tu"li,e# "s +ie%es r"n#oml& #r"wn $rom " l"rger universe o$ +ossible +ie%es "n#
the in$eren%es "re m"#e to " l"rger universe o$ "rt work "n# r"nge o$ "bstr"%tness v"lues. Su%h "
gener"li,"tion is more o$ "n in$erenti"l le"+ "n# %onse4uentl& the r"n#om e$$e%ts mo#el is less
+ower$ul. R"n#om e$$e%ts mo#els "re sometimes re$erre# to "s 5Mo#el 006 or 5v"ri"n%e %om+onent
mo#els.6 'n"l&ses using both $i(e# "n# r"n#om e$$e%ts "re %"lle# 5mi(e# mo#els.6
Fixed and Random Coefficients in Multilevel Regression
/he r"n#om vs. $i(e# #istin%tion $or v"ri"bles "n# e$$e%ts is im+ort"nt in multilevel regression. 0n
multilevel regression mo#els both level7. "n# level72 predictors "re "ssume# to be $i(e#. 3owever
level7. intercepts and slopes "re t&+i%"ll& "ssume# to v"r& r"n#oml& "%ross grou+s. 8e%"use o$ the
"ssum+tions "bout their error #istributions we %"ll their v"ri"n%es oo=var(U0j) "n# 11=var(U1j),
5r"n#om %oe$$i%ients.6
/his me"ns th"t we %"n think "bout 0j "n# 1j "s "kin to the r"n#om variables 0 #es%ribe#
"bove. 0nste"# o$ "ttem+ting to gener"li,e be&on# the +"rti%ul"r v"lues o$ the in#e+en#ent v"ri"ble
we "re "ttem+ting to gener"li,e be&on# the +"rti%ul"r grou+s in the stu#&. -or inst"n%e we m"& h"ve
50 %om+"nies but we wish to gener"li,e to " l"rger universe o$ %om+"nies when we e("mine the
me"ns 1inter%e+ts2 or the (7& rel"tionshi+ 1slo+es2.
0n 39M v"ri"n%es $or the inter%e+ts "n# slo+es "re estim"te# b& #e$"ult. /h"t is when &ou
$irst %onstru%t " mo#el U0j "n# U1j "re estim"te# b& #e$"ult. 3owever the rese"r%her h"s the %hoi%e
o$ setting U0j "n# U1j to be ,ero. 8& setting them to ,ero we "re testing " mo#el in whi%h we "ssume
0j "n# 1j #o not v"r& r"n#oml& "%ross grou+s. 0n $"%t their v"ri"n%e is "ssume# to be ,ero so the&
"re "ssume# to be %onst"nt or 5nonv"r&ing6 "%ross grou+s. -or e("m+le $i(e# nonv"r&ing inter%e+ts
woul# im+l& the grou+ "ver"ge $or the #e+en#ent v"ri"ble is "ssume# to be e4u"l in e"%h grou+.
'lthough we t&+i%"ll& re$er to this %onstr"int "s 5$i(ing the inter%e+ts6 or 5$i(ing the slo+es6 the term is
somewh"t loosel& "++lie# be%"use we "re re"ll& "ssuming the& "re $i(e# and nonvarying.
Intercept Model and Random Effects ANOVA
0n the sim+lest 39M mo#el there "re no +re#i%tors. We h"ve one level7. e4u"tion "n# one level72
e4u"tion:
0 ij j ij
Y R = +
where Y is the #e+en#ent v"ri"ble 0j is the inter%e+t "n# Rij is the resi#u"l or error. 0j %"n be
thought o$ "s the me"n o$ e"%h grou+. /he level72 e4u"tion "lso h"s no +re#i%tors in it;s sim+lest
$orm:
0 00 0 j j
U = +
where 0j is the #e+en#ent v"ri"ble 00 is the level72 inter%e+t "n# U0j is the level72 error. 0n this
e4u"tion 00 re+resents the gr"n# me"n or the me"n o$ the inter%e+ts. U0j re+resents the #evi"tion o$
e"%h me"n $rom the gr"n# me"n. When the "ver"ge #evi"tion is l"rge there "re l"rge grou+
#i$$eren%es.
Newsom 2
USP 656 Multilevel Regression
Winter 2006
0t %"n be shown th"t this is re"ll& the 'N)*' mo#el. 'N)*' e("mines the #evi"tions o$ grou+ me"ns
$rom the gr"n# me"n. 8e%"use we "ssume th"t the grou+ me"ns re+resente# b& 0j "n# thus their
#evi"tions "re r"n#oml& v"r&ing this mo#el is e4uiv"lent to the random effects 'N)*' mo#el. We
%"n rewrite our two 39M e4u"tions "s " single e4u"tion i$ we +lug in the level72 terms into level7.
e4u"tion.
00 0 ij j ij
Y U R = + +
0n this e4u"tion 00 re+resents the gr"n# me"n the U0j term re+resents the #evi"tions $rom the me"n
1or me"n #i$$eren%es or tre"tment e$$e%t i$ &ou woul# like2 "n# Rij re+resents the within7grou+ error or
v"ri"tion. 0n more m"them"ti%"ll& oriente# 'N)*' te(tbooks &ou will see "n 'N)*' mo#el written
something like:
ij j ij
Y e = + +
0$ the mo#el is "n r"n#om e$$e%ts 'N)*' j is "ssume# to h"ve " r"n#om #istribution. 0n multilevel
regression be%"use we "ssume U0j to v"r& r"n#oml& the sim+le 39M mo#el with no level7. or level72
+re#i%tors is e4uiv"lent to the r"n#om e$$e%ts 'N)*' mo#el.
When we "## +re#i%tors to the level7. e4u"tion the& "re %ov"ri"tes "n# the mo#el be%omes "
r"n#om e$$e%ts 'N<)*' in whi%h the me"ns "re "#juste# $or the %ov"ri"te.
Newsom 3
USP 656 Multilevel Regression
Winter 2006
Summary a!le
Random vs"
Fixed
#efinition Example $se in Multilevel
Regression
Varia!les Random variable: 1.2 is "ssume# to be me"sure# with
me"surement error. /he s%ores "re " $un%tion o$ " true
s%ore "n# r"n#om error! 122 the v"lues %ome $rom "n#
"re inten#e# to gener"li,e to " mu%h l"rger +o+ul"tion
o$ +ossible v"lues with " %ert"in +rob"bilit& #istribution
1e.g. norm"l #istribution2! 1=2 the number o$ v"lues in
the stu#& is sm"ll rel"tive to the v"lues o$ the v"ri"ble "s
it "++e"rs in the +o+ul"tion it is #r"wn $rom. Fixed
variable: 1.2 "ssume# to be me"sure# without
me"surement error! 122 #esire# gener"li,"tion to
+o+ul"tion or other stu#ies is to the s"me v"lues! 1=2
the v"ri"ble use# in the stu#& %ont"ins "ll or most o$ the
v"ri"ble;s v"lues in the +o+ul"tion.
0t is im+ort"nt to #istinguish between " v"ri"ble th"t is
varying "n# " v"ri"ble th"t is random. ' $i(e# v"ri"ble
%"n h"ve #i$$erent v"lues it is not ne%ess"ril& inv"ri"nt
1e4u"l2 "%ross grou+s.
Random variable:
+hotogr"+hs
re+resenting in#ivi#u"ls
with #i$$ering levels o$
"ttr"%tiveness
m"ni+ul"te# in "n
e(+eriment %ensus
tr"%ks
Fixed variable:
gen#er r"%e or
intervention vs. %ontrol
grou+.
Pre#i%tor
v"ri"bles in M9R
gener"ll&
"ssume# to be
$i(e#
Effects Random effect: 1.2 #i$$erent st"tisti%"l mo#el o$
regression or 'N)*' mo#el whi%h "ssumes th"t "n
in#e+en#ent v"ri"ble is r"n#om! 122 gener"ll& use# i$
the levels o$ the in#e+en#ent v"ri"ble "re thought to be
" sm"ll subset o$ the +ossible v"lues whi%h one wishes
to gener"li,e to! 1=2 will +rob"bl& +ro#u%e l"rger
st"n#"r# errors 1less +ower$ul2. Fixed effect: 1.2
st"tisti%"l mo#el t&+i%"ll& use# in regression "n# 'N)*'
"ssuming in#e+en#ent v"ri"ble is $i(e#! 122
gener"li,"tion o$ the results "++l& to simil"r v"lues o$
in#e+en#ent v"ri"ble in the +o+ul"tion or in other
stu#ies! 1=2 will +rob"bl& +ro#u%e sm"ller st"n#"r#
errors 1more +ower$ul2.
Random effect:
r"n#om e$$e%ts 'N)*'
r"n#om e$$e%ts
regression
Fixed effect: $i(e#
e$$e%ts 'N)*' $i(e#
e$$e%ts regression
0nter%e+t onl&
mo#els in M9R
"re e4uiv"lent to
r"n#om e$$e%ts
'N)*' or
'N<)*'.
Coefficients Random coefficient: term "++lies onl& to M9R
"n"l&ses in whi%h inter%e+ts slo+es "n# v"ri"n%es %"n
be "ssume# to be r"n#om. M9R "n"l&ses most
t&+i%"ll& "ssume r"n#om %oe$$i%ients. )ne %"n
%on%e+tu"li,e the %oe$$i%ients obt"ine# $rom the level7.
regressions "s " t&+e o$ r"n#om v"ri"ble whi%h %omes
$rom "n# gener"li,es to " #istribution o$ +ossible v"lues.
>rou+s "re %on%eive# o$ "s " subset o$ the +ossible
grou+s.
Fixed coefficient: " %oe$$i%ient %"n be $i(e# to be non7
v"r&ing 1inv"ri"nt2 "%ross grou+s b& setting its between
grou+ v"ri"n%e to ,ero.
R"n#om %oe$$i%ients must be v"ri"ble "%ross grou+s.
<on%e+tu"ll& $i(e# %oe$$i%ients m"& be inv"ri"nt or
v"r&ing "%ross grou+s.
Random coefficient:
the level72 +re#i%tor
"ver"ge in%ome is
use# to +re#i%t s%hool
+er$orm"n%e in e"%h
s%hool. 0nter%e+t
v"lues $or s%hool
+er$orm"n%e "re
"ssume# to be "
s"m+le o$ the inter%e+ts
$rom " l"rger +o+ul"tion
o$ s%hools.
Fixed coefficient:
slo+es or inter%e+ts
%onstr"ine# to be e4u"l
over #i$$erent s%hools.
8oth use# in
M9R. Slo+es "n#
inter%e+t v"lues
%"n be
%onsi#ere# to be
$i(e# or r"n#om
#e+en#ing on
rese"r%hers
"ssum+tions "n#
how the mo#el is
s+e%i$ie#. /he
v"ri"n%e o$ the
slo+es or
inter%e+ts "re
%onsi#ere#
r"n#om
%oe$$i%ients.