Quantile Regression for Longitudinal Data

Roger Koenker

CEMMAP and University of Illinois, Urbana-Champaign

LSE: 17 May 2011

Roger Koenker (CEMMAP & UIUC) Quantile Regression for Longitudinal Data LSE: 17.5.2010 1/9
Classical Linear Fixed/Random Effects Model

Consider the model,

yij = x>
ij β + αi + uij j = 1, ...mi , i = 1, ..., n,

or
y = Xβ + Zα + u.
The matrix Z represents an incidence matrix that identifies the n distinct
individuals in the sample. If u and α are independent Gaussian vectors
with u ∼ N(0, R) and α ∼ N(0, Q). Observing that v = Zα + u has
covariance matrix Evv> = R + ZQZ> , we can immediately deduce that
the minimum variance unbiased estimator of β is,

β̂ = (X> (R + ZQZ> )−1 X)−1 X> (R + ZQZ> )−1 y.

Roger Koenker (CEMMAP & UIUC) Quantile Regression for Longitudinal Data LSE: 17.5.2010 2/9
A Penalty Interpretation of β̂
Proposition. β̂ solves min(α,β) ky − Xβ − Zαk2R−1 + kαk2Q−1 , where
kxk2A = x> Ax.

Proof.
Differentiating we obtain the normal equations,

X> R−1 Xβ̂ + X> R−1 Zα̂ = X> R−1 y

Z> R−1 Xβ̂ + (Z> R−1 Z + Q−1 )α̂ = Z> R−1 y

Solving, we have β̂ = (X> Ω−1 X)−1 X> Ω−1 y where

Ω−1 = R−1 − R−1 Z(Z> R−1 Z + Q−1 )−1 Z> R−1 .

But Ω = R + ZQZ> , see e.g. Rao(1973, p 33.).

This result has a long history: Henderson(1950), Goldberger(1962),

Lindley and Smith (1972), etc.
Roger Koenker (CEMMAP & UIUC) Quantile Regression for Longitudinal Data LSE: 17.5.2010 3/9
Quantile Regression with Fixed Effects

Suppose that the conditional quantile functions of the response of the jth
observation on the ith individual yij takes the form:

Qyij (τ|xij ) = αi + x>

ij β(τ) j = 1, ...mi , i = 1, ..., n.

In this formulation the α’s have a pure location shift effect on the
conditional quantiles of the response. The effects of the covariates, xij are
permitted to depend upon the quantile, τ, of interest, but the α’s do not.
To estimate the model for several quantiles simultaneously, we propose
solving,
Xq X n Xmi
min wk ρτk (yij − αi − x>
ij β(τk ))
(α,β)
k=1 j=1 i=1

Note that the usual between/within transformations are not permitted.

Roger Koenker (CEMMAP & UIUC) Quantile Regression for Longitudinal Data LSE: 17.5.2010 4/9
Penalized Quantile Regression with Fixed Effects

25
20

i=1
15

i=2
yit

i=3
10
5
0

0 2 4 6 8

Time invariant, individual specific intercepts

xit are quantile independent;
slopes are quantile dependent.

Roger Koenker (CEMMAP & UIUC) Quantile Regression for Longitudinal Data LSE: 17.5.2010 5/9
Penalized Quantile Regression with Fixed Effects

When n is large relative to the mi ’s shrinkage may be advantageous in

controlling the variability introduced by the large number of estimated α
parameters. We will consider estimators solving the penalized version,

X
q X
n X
mi X
n
min wk ρτk (yij − αi − x>
ij β(τk )) + λ |αi |.
(α,β)
k=1 j=1 i=1 i=1

For λ → 0 we obtain the fixed effects estimator described above, while as

λ → ∞ the α̂i → 0 for all i = 1, 2, ..., n and we obtain an estimate of the
model purged of the fixed effects. In moderately large samples this requires
sparse linear algebra. Example R code is available from my webpages.

Roger Koenker (CEMMAP & UIUC) Quantile Regression for Longitudinal Data LSE: 17.5.2010 6/9
Shrinkage of the Fixed Effects

8 10

8
6
6

4
4

2
α

α
2

0
−2 0

−4
0 1 2 3 4 5 0 1 2 3 4 5

λ λ

Shrinkage of the fixed effect parameter estimates, α̂i . The left panel illustrates an
example of the `1 shrinkage effect. The right panel illustrates an example of the `2
shrinkage effect.

Roger Koenker (CEMMAP & UIUC) Quantile Regression for Longitudinal Data LSE: 17.5.2010 7/9
Dynamic Panel Models and IV Estimation

Galvao (2010) considers dynamic panel models of the form:

Qyit (τ|yi,t−1 , xit ) = αi + γ(τ)yi,t−1 + x>

it β(τ) t = 1, ...Ti , i = 1, ..., n.

In “short” panels estimation suffers from the same bias problems as seen
in least squares estimators Nickel (1981) Hsiao and Anderson (1981);
using the IV estimation approach of Chernozhukov and Hansen (2004) this
bias can be reduced.

Roger Koenker (CEMMAP & UIUC) Quantile Regression for Longitudinal Data LSE: 17.5.2010 8/9
Correlated Random Effects

Abrevaya and Dahl (JBES, 2008) adapt the Chamberlain (1982) correlated
random effects model and estimate a model of birthweight like that of
Koenker and Hallock (2001).
The R package rqpd implements both this method and the penalized fixed
effect approach. Available from R-Forge with the command:

install.packages("rqpd", repos="http://R-Forge.R-project.org")

This is a challenging, but very important, problem and hopefully there will
be new and better approaches in the near future.

Roger Koenker (CEMMAP & UIUC) Quantile Regression for Longitudinal Data LSE: 17.5.2010 9/9