Newsom 1

USP 656 Multilevel Regression

Winter 2006
Jkj..khh.kh.khk.jhb.jkk.bkbbbb.b.b.b vlues o! tht rn"om vrible. Most o! the time in #N$%# n"
regression nl&sis we ssume the in"e'en"ent vribles re !i(e".
Random and Fixed Effects
)he terms *rn"om+ n" *!i(e"+ re use" in the ,onte(t o! #N$%# n" regression mo"els- n" re!er to
,ertin t&'e o! sttisti,l mo"el. #lmost lw&s- reser,hers use !i(e" e!!e,ts regression or #N$%#
n" the& re rrel& !,e" with sitution involving rn"om e!!e,ts nl&ses. # !i(e" e!!e,ts #N$%#
re!ers to ssum'tions bout the in"e'en"ent vrible n" the error "istribution !or the vrible. #n
e('erimentl "esign is the esiest e(m'le !or illustrting the 'rin,i'l. Usull&- the reser,her is
intereste" in onl& generli.ing the results to e('erimentl vlues use" in the stu"&. /or instn,e-
"rug stu"& using 0 mg- 5 mg- or 00 mg o! n e('erimentl "rug. )his is when !i(e" e!!e,ts #N$%#
woul" be ''ro'rite. 1n this ,se- the e(tr'oltion is to other stu"ies or tretments tht might use
the sme vlues o! the "rug 2i.e.- 0 mg- 5 mg- n" 00 mg3. 4owever- i! the reser,her wnts to mke
in!eren,es be&on" the 'rti,ulr vlues o! the in"e'en"ent vrible use" in the stu"&- rn"om
e!!e,ts mo"el is use". # ,ommon e(m'le woul" be the use o! 'ubli, rt works re'resenting low-
mo"erte- n" high bstr,tness 2e.g.- sttue o! wr hero vs. 'ivoting geometri, "esign3. )he
reser,her woul" like to mke in!eren,es be&on" the three 'ie,es use"- so the rt 'ie,es re
,on,e'tuli.e" s 'ie,es rn"oml& "rwn !rom lrger universe o! 'ossible 'ie,es n" the
in!eren,es re m"e to lrger universe o! rt work n" rnge o! bstr,tness vlues. Su,h
generli.tion is more o! n in!erentil le'- n"- ,onse5uentl&- the rn"om e!!e,ts mo"el is less
'ower!ul. Rn"om e!!e,ts mo"els re sometimes re!erre" to s *Mo"el 11+ or *vrin,e ,om'onent
mo"els.+ #nl&ses using both !i(e" n" rn"om e!!e,ts re ,lle" *mi(e" mo"els.+
Fixed and Random Coefficients in Multilevel Regression
)he rn"om vs. !i(e" "istin,tion !or vribles n" e!!e,ts is im'ortnt in multilevel regression. 1n
multilevel regression mo"els- both level60 n" level62 predictors re ssume" to be !i(e". 4owever-
level60 intercepts and slopes re t&'i,ll& ssume" to vr& rn"oml& ,ross grou's. 7e,use o! the
ssum'tions bout their error "istributions- we ,ll their vrin,es- oo=var(U0j) n" 11=var(U1j),
*rn"om ,oe!!i,ients.+
)his mens tht we ,n think bout 0j n" 1j s kin to the rn"om variables 1 "es,ribe"
bove. 1nste" o! ttem'ting to generli.e be&on" the 'rti,ulr vlues o! the in"e'en"ent vrible-
we re ttem'ting to generli.e be&on" the 'rti,ulr grou's in the stu"&. /or instn,e- we m& hve
50 ,om'nies- but we wish to generli.e to lrger universe o! ,om'nies when we e(mine the
mens 2inter,e'ts3 or the (6& reltionshi' 2slo'es3.
1n 48M- vrin,es !or the inter,e'ts n" slo'es re estimte" b& "e!ult. )ht is- when &ou
!irst ,onstru,t mo"el- U0j n" U1j re estimte" b& "e!ult. 4owever- the reser,her hs the ,hoi,e
o! setting U0j n" U1j to be .ero. 7& setting them to .ero- we re testing mo"el in whi,h we ssume
0j n" 1j "o not vr& rn"oml& ,ross grou's. 1n !,t- their vrin,e is ssume" to be .ero- so the&
re ssume" to be ,onstnt or *nonvr&ing+ ,ross grou's. /or e(m'le- !i(e"- nonvr&ing inter,e'ts
woul" im'l& the grou' verge !or the "e'en"ent vrible is ssume" to be e5ul in e,h grou'.
#lthough we t&'i,ll& re!er to this ,onstrint s *!i(ing the inter,e'ts+ or *!i(ing the slo'es-+ the term is
somewht loosel& ''lie"- be,use we re rell& ssuming the& re !i(e" and nonvarying.
Intercept Model and Random Effects ANOVA
1n the sim'lest 48M mo"el- there re no 're"i,tors. We hve one level60 e5ution n" one level62
e5ution9
0 ij j ij
Y R = +
where Y is the "e'en"ent vrible- 0j is the inter,e't- n" Rij is the resi"ul or error. 0j ,n be
thought o! s the men o! e,h grou'. )he level62 e5ution lso hs no 're"i,tors in it:s sim'lest
!orm9
Newsom 2
USP 656 Multilevel Regression
Winter 2006
0 00 0 j j
U = +
where 0j is the "e'en"ent vrible- 00 is the level62 inter,e't- n" U0j is the level62 error. 1n this
e5ution- 00 re'resents the grn" men or the men o! the inter,e'ts. U0j re'resents the "evition o!
e,h men !rom the grn" men. When the verge "evition is lrge- there re lrge grou'
"i!!eren,es.
1t ,n be shown tht this is rell& the #N$%# mo"el. #N$%# e(mines the "evitions o! grou' mens
!rom the grn" men. 7e,use we ssume tht the grou' mens- re'resente" b& 0j n"- thus- their
"evitions re rn"oml& vr&ing- this mo"el is e5uivlent to the random effects #N$%# mo"el. We
,n rewrite our two 48M e5utions s single e5ution- i! we 'lug in the level62 terms into level60
e5ution.
00 0 ij j ij
Y U R = + +
1n this e5ution- 00 re'resents the grn" men- the U0j term re'resents the "evitions !rom the men
2or men "i!!eren,es or tretment e!!e,t- i! &ou woul" like3- n" Rij re'resents the within6grou' error or
vrition. 1n more mthemti,ll& oriente" #N$%# te(tbooks- &ou will see n #N$%# mo"el written
something like9
ij j ij
Y e = + +
1! the mo"el is n rn"om e!!e,ts #N$%#- j is ssume" to hve rn"om "istribution. 1n multilevel
regression- be,use we ssume U0j to vr& rn"oml&- the sim'le 48M mo"el with no level60 or level62
're"i,tors is e5uivlent to the rn"om e!!e,ts #N$%# mo"el.
When we "" 're"i,tors to the level60 e5ution- the& re ,ovrites n" the mo"el be,omes
rn"om e!!e,ts #N;$%# in whi,h the mens re "juste" !or the ,ovrite.
Newsom 3
USP 656 Multilevel Regression
Winter 2006
Summary a!le
Random vs"
Fixed
#efinition Example $se in Multilevel
Regression
Varia!les Random variable9 203 is ssume" to be mesure" with
mesurement error. )he s,ores re !un,tion o! true
s,ore n" rn"om error< 223 the vlues ,ome !rom n"
re inten"e" to generli.e to mu,h lrger 'o'ultion
o! 'ossible vlues with ,ertin 'robbilit& "istribution
2e.g.- norml "istribution3< 2=3 the number o! vlues in
the stu"& is smll reltive to the vlues o! the vrible s
it ''ers in the 'o'ultion it is "rwn !rom. Fixed
variable9 203 ssume" to be mesure" without
mesurement error< 223 "esire" generli.tion to
'o'ultion or other stu"ies is to the sme vlues< 2=3
the vrible use" in the stu"& ,ontins ll or most o! the
vrible:s vlues in the 'o'ultion.
1t is im'ortnt to "istinguish between vrible tht is
varying n" vrible tht is random. # !i(e" vrible
,n hve "i!!erent vlues- it is not ne,essril& invrint
2e5ul3 ,ross grou's.
Random variable:
'hotogr'hs
re'resenting in"ivi"uls
with "i!!ering levels o!
ttr,tiveness
mni'ulte" in n
e('eriment- ,ensus
tr,ks
Fixed variable:
gen"er- r,e- or
intervention vs. ,ontrol
grou'.
Pre"i,tor
vribles in M8R
generll&
ssume" to be
!i(e"
Effects Random effect9 203 "i!!erent sttisti,l mo"el o!
regression or #N$%# mo"el whi,h ssumes tht n
in"e'en"ent vrible is rn"om< 223 generll& use" i!
the levels o! the in"e'en"ent vrible re thought to be
smll subset o! the 'ossible vlues whi,h one wishes
to generli.e to< 2=3 will 'robbl& 'ro"u,e lrger
stn"r" errors 2less 'ower!ul3. Fixed effect9 203
sttisti,l mo"el t&'i,ll& use" in regression n" #N$%#
ssuming in"e'en"ent vrible is !i(e"< 223
generli.tion o! the results ''l& to similr vlues o!
in"e'en"ent vrible in the 'o'ultion or in other
stu"ies< 2=3 will 'robbl& 'ro"u,e smller stn"r"
errors 2more 'ower!ul3.
Random effect9
rn"om e!!e,ts #N$%#-
rn"om e!!e,ts
regression
Fixed effect9 !i(e"
e!!e,ts #N$%#- !i(e"
e!!e,ts regression
1nter,e't onl&
mo"els in M8R
re e5uivlent to
rn"om e!!e,ts
#N$%# or
#N;$%#.
Coefficients Random coefficient: term ''lies onl& to M8R
nl&ses in whi,h inter,e'ts- slo'es- n" vrin,es ,n
be ssume" to be rn"om. M8R nl&ses most
t&'i,ll& ssume rn"om ,oe!!i,ients. $ne ,n
,on,e'tuli.e the ,oe!!i,ients obtine" !rom the level60
regressions s t&'e o! rn"om vrible whi,h ,omes
!rom n" generli.es to "istribution o! 'ossible vlues.
>rou's re ,on,eive" o! s subset o! the 'ossible
grou's.
Fixed coefficient: ,oe!!i,ient ,n be !i(e" to be non6
vr&ing 2invrint3 ,ross grou's b& setting its between
grou' vrin,e to .ero.
Rn"om ,oe!!i,ients must be vrible ,ross grou's.
;on,e'tull&- !i(e" ,oe!!i,ients m& be invrint or
vr&ing ,ross grou's.
Random coefficient9
the level62 're"i,tor-
verge in,ome- is
use" to 're"i,t s,hool
'er!ormn,e in e,h
s,hool. 1nter,e't
vlues !or s,hool
'er!ormn,e re
ssume" to be
sm'le o! the inter,e'ts
!rom lrger 'o'ultion
o! s,hools.
Fixed coefficient9
slo'es or inter,e'ts
,onstrine" to be e5ul
over "i!!erent s,hools.
7oth use" in
M8R. Slo'es n"
inter,e't vlues
,n be
,onsi"ere" to be
!i(e" or rn"om-
"e'en"ing on
reser,hers
ssum'tions n"
how the mo"el is
s'e,i!ie". )he
vrin,e o! the
slo'es or
inter,e'ts re
,onsi"ere"
rn"om
,oe!!i,ients.