Factor Models

Lecture 15: Factor Models

MIT 18.S096

Dr. Kempthorne

Fall 2013

MIT 18.S096 Factor Models 1

 Linear Factor Model
Macroeconomic Factor Models
Fundamental Factor Models
Factor Models
Statistical Factor Models: Factor Analysis
Principal Components Analysis
Statistical Factor Models: Principal Factor Method

Outline

1 Factor Models
Linear Factor Model
Macroeconomic Factor Models
Fundamental Factor Models
Statistical Factor Models: Factor Analysis
Principal Components Analysis
Statistical Factor Models: Principal Factor Method

MIT 18.S096 Factor Models 2

 Linear Factor Model
Macroeconomic Factor Models
Fundamental Factor Models
Factor Models
Statistical Factor Models: Factor Analysis
Principal Components Analysis
Statistical Factor Models: Principal Factor Method

Linear Factor Model

Data:
m assets/instruments/indexes: i = 1, 2, . . . , m
n time periods: t = 1, 2, . . . , n
m-variate random vector for each time period:
xt = (x1,t , x2,t , . . . , xm,t )0
E.g., returns on m stocks/futures/currencies;
interest-rate yields on m US Treasury instruments.

Factor Model
xi,t = αi + β1,i f1,t + β2,i f2,t + · · · + βk,i fk,t + i,t
= αi + β 0i f t + i,t where
αi : intercept of asset i
f t = (f1,t , f2,t , . . . , fK ,t )0 : common factor variables at period t (constant over i)
β i = (β1,i , . . . , βK ,i )0 : factor loadings of asset i (constant over t)
i,t : the specific factor of asset i at period t.

MIT 18.S096 Factor Models 3

 Linear Factor Model
Macroeconomic Factor Models
Fundamental Factor Models
Factor Models
Statistical Factor Models: Factor Analysis
Principal Components Analysis
Statistical Factor Models: Principal Factor Method

Linear Factor Model

Linear Factor Model: Cross-Sectional Regressions
xt = α + Bf t + t ,
for each t ∈ {1, 2 . . . , T }, where
 0 
α1 β1 1,t
   
 α2   β 02   
 2,t 
  
α =  .  (m × 1); B =  .  = βi,k (m × K ); t =  .  (m × 1)
   
 ..   ..   .. 
0
αm βm m,t
α and B are the same for all t.
{f t } is (K −variate) covariance stationary I (0) with
E [f t ] = µf
Cov [f t ] = E [(f t − µf )(f t − µf )0 ] = Ωf
{t } is m-variate white noise with:
E [t ] = 0m
Cov [t ] = E [t 0t ] = Ψ
Cov [t , t 0 ] = E [t 0t 0 ] = 0 ∀t 6= t 0
Ψ is the (m × m) diagonal matrix with entries (σ12 , σ22 , . . . , σm2 ) where

σi2 = var (i,t ), the variance of the ith asset specific factor.
The two processes {f t } and {t } have null cross-covariances:
0
MIT t − µf )(tFactor
E [(f18.S096 0 − 0Models
m) ] = 4
 Linear Factor Model
Macroeconomic Factor Models
Fundamental Factor Models
Factor Models
Statistical Factor Models: Factor Analysis
Principal Components Analysis
Statistical Factor Models: Principal Factor Method

Linear Factor Model

Summary of Parameters
α: (m × 1) intercepts for m assets
B: (m × K ) loadings on K common factors for m assets
µf : (K × 1) mean vector of K common factors
Ωf : (K × K ) covariance matrix of K common factors
Ψ = diag (σ12 , . . . , σm
2 ): m asset-specific variances

Features of Linear Factor Model

The m−variate stochastic process {xt } is a
covariance-stationary multivariate time series with
Conditional moments:
E [xt | f t ] = α + Bf t
Cov [xt | f t ] = Ψ
Unconditional moments:
E [xt ] = µx = α + Bµf
Cov [xt ] = Σx = BΩf B 0 + Ψ
MIT 18.S096 Factor Models 5
 Linear Factor Model
Macroeconomic Factor Models
Fundamental Factor Models
Factor Models
Statistical Factor Models: Factor Analysis
Principal Components Analysis
Statistical Factor Models: Principal Factor Method

Linear Factor Model

Linear Factor Model: Time Series Regressions
xi = 1T αi + Fβ i + i ,
 asseti ∈ {1, 2 .
for each . . , m}, where
f 01
    
xi,1 i,t f1,1 f2,1 ··· fK ,1
 .   .  . .. .. .. ..
 .   .

 .
  
 .   .  . . . . .
   
   
 0
xi =  xi,t  i =  i,t F =  ft = f1,t f2,t ··· fK ,t
      
 
 ..   ..  . ... ... ... ..
       
 ..
   
 .   .    . 
x1,T i,T f 0T f1,T f2,T ··· fK ,T
αi and β i = (β1,i , . . . , βK ,i ) are regression parameters.
i is the T -vector of regression errors with Cov (i ) = σi2 IT
Linear Factor Model: Multivariate Regression
X = [x1 | · · · |xm ], E = [1 | · · · |m ], B = [β 1 | · · · |β m ],
X = 1T α0 + FB + E
(note that B equals the transpose of cross-sectional B)
MIT 18.S096 Factor Models 6
 Linear Factor Model
Macroeconomic Factor Models
Fundamental Factor Models
Factor Models
Statistical Factor Models: Factor Analysis
Principal Components Analysis
Statistical Factor Models: Principal Factor Method

Outline

1 Factor Models
Linear Factor Model
Macroeconomic Factor Models
Fundamental Factor Models
Statistical Factor Models: Factor Analysis
Principal Components Analysis
Statistical Factor Models: Principal Factor Method

MIT 18.S096 Factor Models 7

 Linear Factor Model
Macroeconomic Factor Models
Fundamental Factor Models
Factor Models
Statistical Factor Models: Factor Analysis
Principal Components Analysis
Statistical Factor Models: Principal Factor Method

Macroeconomic Factor Models

Single Factor Model of Sharpe (1970)

xi,t = αi + βi RMt + i,t i = 1, . . . , m t = 1, . . . , T
where
RMt is the return of the market index in excess of the
risk-free rate; the market risk factor.
xi,t is the return of asset i in excess of the risk-free rate.
K = 1 and the single factor is f1,t = RMt .
Unconditional cross-sectional covariance matrix of the assets:
Cov (xt ) = Σx = σM2 ββ 0 + Ψ where
2
σM = Var (RMt )
β = (β1 , . . . , βm )0
Ψ = diag (σ12 , . . . , σm
2
)

MIT 18.S096 Factor Models 8

 Linear Factor Model
Macroeconomic Factor Models
Fundamental Factor Models
Factor Models
Statistical Factor Models: Factor Analysis
Principal Components Analysis
Statistical Factor Models: Principal Factor Method

Estimation of Sharpe’s Single Index Model

Single Index Model satisfies the Generalized Gauss-Markov
assumptions so the least-squares estimates (α̂i , β̂i ) from the
time-series regression for each asset i are best linear unbiased
estimates (BLUE) and the MLEs under Gaussian assumptions.
xi = 1T α̂i + RM βˆi + ˆi
Unbiased estimators of remaining parameters:
σ̂i2 = (ˆ0i ˆi )/(T − 2)
2
PT PT
σ̂M = [ t=1 (RMt − R̄M )2 ]/(T − 1) with R̄M = ( t=1 RMt )/T
Ψ̂ = diag (σ̂12 , . . . , σ̂m
2
)
Estimator of unconditional covariance matrix:
\
Cov (xt ) = Σ̂x = σ̂M ˆ 0 + Ψ̂
2 ββ̂

MIT 18.S096 Factor Models 9

 Linear Factor Model
Macroeconomic Factor Models
Fundamental Factor Models
Factor Models
Statistical Factor Models: Factor Analysis
Principal Components Analysis
Statistical Factor Models: Principal Factor Method

Macroeconomic Multifactor Model

The common factor variables {f t } are realized values of macro
econonomic variables, such as
Market risk
Price indices (CPI, PPI, commodities) / Inflation
Industrial production (GDP)
Money growth
Interest rates
Housing starts
Unemployment
See Chen, Ross, Roll (1986). “Economic Forces and the Stock Market”
Linear Factor Model as Time Series Regressions
xi = 1T αi + Fβ i + i , where
F = [f 1 , f 2 , . . . f T ]0 is the (T × K ) matrix of realized values of
(K > 0) macroeconomic factors.
Unconditional cross-sectional covariance matrix of the assets:
Cov (xt ) = BΩf B 0 + Ψ
where B = (β 1 , . . . , β m )0 is (m × K )
MIT 18.S096 Factor Models 10
 Linear Factor Model
Macroeconomic Factor Models
Fundamental Factor Models
Factor Models
Statistical Factor Models: Factor Analysis
Principal Components Analysis
Statistical Factor Models: Principal Factor Method

Estimation of Multifactor Model

Multifactor model satisfies the Generalized Gauss-Markov
assumptions so the least-squares estimates α̂i and β̂ i (K × 1)
from the time-series regression for each asset i are best linear
unbiased estimates (BLUE) and the MLEs under Gaussian
assumptions.
xi = 1T α̂i + Fβ̂ i + ˆi
Unbiased estimators of remaining parameters:
σ̂i2 = (ˆ0i ˆi )/[T − (k + 1)]
Ψ̂ = diag (σ̂12 , . . . , σ̂m
2
)
PT
Ω̂f = [ t=1 (f t − f̄)(f t − f̄)0 ]/(T − 1)
PT
with f̄ = ( t=1 f t )/T
Estimator of unconditional covariance matrix:
\
Cov 2 BΩ
(xt ) = Σ̂x = σ̂M ˆ0 + Ψ
ˆ ˆf B ˆ

MIT 18.S096 Factor Models 11

 Linear Factor Model
Macroeconomic Factor Models
Fundamental Factor Models
Factor Models
Statistical Factor Models: Factor Analysis
Principal Components Analysis
Statistical Factor Models: Principal Factor Method

Outline

1 Factor Models
Linear Factor Model
Macroeconomic Factor Models
Fundamental Factor Models
Statistical Factor Models: Factor Analysis
Principal Components Analysis
Statistical Factor Models: Principal Factor Method

MIT 18.S096 Factor Models 12

 Linear Factor Model
Macroeconomic Factor Models
Fundamental Factor Models
Factor Models
Statistical Factor Models: Factor Analysis
Principal Components Analysis
Statistical Factor Models: Principal Factor Method

Fundamental Factor Models

The common-factor variables {f t } are determined using
fundamental, asset-specific attributes such as
Sector/industry membership.
Firm size (market capitalization)
Dividend yield
Style (growth/value as measured by price-to-book,
earnings-to-price, ...)
Etc.
BARRA Approach (Barr Rosenberg)
Treat observable asset-specific attributes as
factor betas
Factor realizations {f t } are unobservable, but
are estimated.
MIT 18.S096 Factor Models 13
 Linear Factor Model
Macroeconomic Factor Models
Fundamental Factor Models
Factor Models
Statistical Factor Models: Factor Analysis
Principal Components Analysis
Statistical Factor Models: Principal Factor Method

Fama-French Approach (Eugene Fama and Kenneth French)

For every time period t, apply cross-sectional
sorts to define factor realizations
For a given asset attribute, sort the assets at
period t by that attribute and define quintile
portfolios based on splitting the assets into 5
equal-weighted portfolios.
Form the hedge portfolio which is long the top
quintile assets and short the bottom quintile
assets.
Define the common factor realizations for period
t as the period-t returns for the K hedge
portfolios corresponding to the K fundamental
asset attributes.
Estimate the factor loadings on assets using time
series regressions, separately for each asset i.
MIT 18.S096 Factor Models 14
 Linear Factor Model
Macroeconomic Factor Models
Fundamental Factor Models
Factor Models
Statistical Factor Models: Factor Analysis
Principal Components Analysis
Statistical Factor Models: Principal Factor Method

Barra Industry Factor Model

Suppose the m assets (i = 1, 2, . . . , m) separate into K
industry groups (k = 1, . . . , K )
For each asset i , define the factor loadings (k = 1, . . . K )
1 if asset i is in industry group k
βi,k =
0 otherwise
These loadings are time invariant.
For time period t, denote the realization of the K factors as
f t = (f1t , . . . , fKt )0
These K − vector realizations are unobserved.
The Industry Factor Model is
Xi,t = βi,1 f1t + · · · + βi,K fKt + it , ∀i, t
where var ( ) = σ2 , ∀i
it i
cov (it , fkt ) = 0, ∀i, k, t
cov (fk 0 t , fkt ) = [Ωf ]k 0 ,k , ∀k 0 , k, t
MIT 18.S096 Factor Models 15
 Linear Factor Model
Macroeconomic Factor Models
Fundamental Factor Models
Factor Models
Statistical Factor Models: Factor Analysis
Principal Components Analysis
Statistical Factor Models: Principal Factor Method

Barra Industry Factor Model

Estimation of the Factor Realizations
For each time period t consider the cross-sectional regression for
the factor model:
xt = Bf t + t (α = 0 so it does not appear)
with  0 
x1,t β1 1,t
  
 x2,t   β 02     2,t 
xt =  . (m × 1); B =  .  = βi,k (m × K ); t =  ..  (m × 1)
     
 ..  .. 

  . 
xm,t β 0m m,t
0
where E [t ] = 0m , E [t t ] = Ψ, and Cov (f t ) = Ωf .
Compute f̂ t by least-squares regression of xt on B with regression parameter f t .
B is (m × K ) matrix of indicator variables (same for all t)
B 0 B = diag (m1 , . . . mK ),
where mk is the count of assets i in industry k, and K
P
k=1 mk = m.
0 0
f̂ t = (B B)−1 B xt (vector of industry averages!)
t = xt − B f̂ t
ˆ
MIT 18.S096 Factor Models 16
 Linear Factor Model
Macroeconomic Factor Models
Fundamental Factor Models
Factor Models
Statistical Factor Models: Factor Analysis
Principal Components Analysis
Statistical Factor Models: Principal Factor Method

Barra Industry Factor Model

Estimation of Factor Covariance Matrix

¯ ¯0
Ω̂f = T 1−1 T
P
t=1 (f̂ t − f̂)(f̂ t − f̂)
f̂¯ = T1 T ˆ
P
t=1 f t
Estimation of Residual Covariance Matrix Ψ̂

Ψ̂ = diag (σ̂12 , . . . , σ̂m

2)

where
σ̂i2 = T 1−1 T i,t − ¯ˆi ]2
P
t=1 [ˆ
¯ˆi = T1 T
P
t=1 
ˆi,t
Estimation of Industry Factor Model Covariance Matrix
Σ̂ = B 0 Ω̂f B + Ψ̂

MIT 18.S096 Factor Models 17

 Linear Factor Model
Macroeconomic Factor Models
Fundamental Factor Models
Factor Models
Statistical Factor Models: Factor Analysis
Principal Components Analysis
Statistical Factor Models: Principal Factor Method

Barra Industry Factor Model

Further Details
Inefficiency of least squares estimates due to
heteroscedasticity in Ψ.
Resolution: apply Generalized Least Squares (GLS) estimating
Ψ in the cross-sectional regressions.
The factor realizations can be rescaled to represent factor
mimicking portfolios
The Barra Industry Factor Model can be expressed as a
seemingly unrelated regression (SUR) model

MIT 18.S096 Factor Models 18

 Linear Factor Model
Macroeconomic Factor Models
Fundamental Factor Models
Factor Models
Statistical Factor Models: Factor Analysis
Principal Components Analysis
Statistical Factor Models: Principal Factor Method

Outline

1 Factor Models
Linear Factor Model
Macroeconomic Factor Models
Fundamental Factor Models
Statistical Factor Models: Factor Analysis
Principal Components Analysis
Statistical Factor Models: Principal Factor Method

MIT 18.S096 Factor Models 19

 Linear Factor Model
Macroeconomic Factor Models
Fundamental Factor Models
Factor Models
Statistical Factor Models: Factor Analysis
Principal Components Analysis
Statistical Factor Models: Principal Factor Method

Statistical Factor Models

The common-factor variables {f t } are hidden (latent) and their

structure is deduced from analysis of the observed returns/data
{xt }. The primary methods for extraction of factor structure are:
Factor Analysis
Principal Components Analysis
Both methods model the Σ, the covariance matrix of
{xt , t = 1, . . . , T } by focusing on the sample covariance matrix Σ̂,
computed as follows:
X = [x1 : · · · xT ] (m × T )
X∗ = X · (IT − T1 1T 10T ) (‘de-meaned’ by row)
Σ̂x = T1 X∗ (X∗ )0

MIT 18.S096 Factor Models 20

 Linear Factor Model
Macroeconomic Factor Models
Fundamental Factor Models
Factor Models
Statistical Factor Models: Factor Analysis
Principal Components Analysis
Statistical Factor Models: Principal Factor Method

Factor Analysis Model

Linear Factor Model as Cross-Sectional Regression
xt = α + Bf t + t ,
for each t ∈ {1, 2 . . . , T } ( m equations expressed in vector/matrix form) where
α and B are the same for all t.
{f t } is (K −variate) covariance stationary I (0) with E [f t ] = µf , Cov [f t ] = Ωf
{t } is m-variate white noise with E [t ] = 0m and Cov [t ] = Ψ = diag (σi2 )
Invariance to Linear Tranforms of f t
For any (K × K ) invertible matrix H define
f ∗t = Hft and B ∗ = BH −1
Then the linear factor model holds replacing f t and B
xt = α + B ∗ f ∗t + t = α + BH −1 Hf t + t
= α + Bf t + t
and replacing µf and Ωf with
Ω∗f = Cov (f ∗t ) = Cov (Hf t ) = HCov (f t )H 0 = HΩf H 0
µ∗f = Hµf
MIT 18.S096 Factor Models 21
 Linear Factor Model
Macroeconomic Factor Models
Fundamental Factor Models
Factor Models
Statistical Factor Models: Factor Analysis
Principal Components Analysis
Statistical Factor Models: Principal Factor Method

Factor Analysis Model

Standard Formulation of Factor Analysis Model
Orthonormal factors: Ωf = IK
1
This is achieved by choosing H = ΓΛ− 2 , where
Ωf = ΓΛΓ0 is the spectral/eigen decomposition
with orthogonal (K × K ) matrix Γ and diagonal matrix
Λ = diag (λ1 , . . . , λK ), where λ1 ≥ λ2 ≥ · · · ≥ λK > 0.
Zero-mean factors: µf = 0K
This is achieved by adjusting α to incorporate the mean
contribution from the factors:
α∗ = α + Bµf
Under these assumptions the unconditional covariance matrix is
Cov (xt ) = Σx = BB 0 + Ψ
MIT 18.S096 Factor Models 22
 Linear Factor Model
Macroeconomic Factor Models
Fundamental Factor Models
Factor Models
Statistical Factor Models: Factor Analysis
Principal Components Analysis
Statistical Factor Models: Principal Factor Method

Factor Analysis Model

Maximum Likelihood Estimation
For the model
xt = α + Bf t + t
α and B are vector/matrix constants.
All random variables are Normal/Gaussian:
xt i.i.d. Nm (α, Σx )
f t i.i.d. NK (0K IK )
t i.i.d. Nm (0m , Ψ)
Cov (xt ) = Σx = BB 0 + Ψ

Model Likelihood
L(α, Σx ) = p (x1 , . . . , xT | α, Σ)
QT
= [p(xt | α, Σ)]
Qt=1
T −m/2 |Σ|− 21 exp − 1 (x − α)0 Σ−1 (x − α) ]


= t=1 [(2π) t x t
T
h P2 i
= (2π)−Tm/2 |Σ|− 2 exp − 21 T t=1 (xt − α) 0 Σ−1 (x − α)
x t
MIT 18.S096 Factor Models 23
 Linear Factor Model
Macroeconomic Factor Models
Fundamental Factor Models
Factor Models
Statistical Factor Models: Factor Analysis
Principal Components Analysis
Statistical Factor Models: Principal Factor Method

Factor Analysis Model

Log Likelihood of the Factor Model

l(α, Σx ) = log L(α, Σx )
= − TK2 log (2π) − K2 log (|Σ|)
− 21 T 0 −1
P
t=1 (xt − α) Σx (xt − α)
Maximum Likelihood Estimates (MLEs)
The MLEs of α, B, Ψ are the values which
Maximize l(α, Σx )
Subject to: Σx = BB 0 + Ψ
The MLEs are computed numerically applying the
Expectation-Maximization (EM) algorithm*
* Optional Reading: Dempster, Laird, and Rubin (1977), Rubin and Thayer (1983).

MIT 18.S096 Factor Models 24

 Linear Factor Model
Macroeconomic Factor Models
Fundamental Factor Models
Factor Models
Statistical Factor Models: Factor Analysis
Principal Components Analysis
Statistical Factor Models: Principal Factor Method

Factor Analysis Model

ML Specification of the Factor Model
Apply EM algorithm to compute α̂ and B̂ and Ψ̂.
Estimate factor realizations {f t }
Apply the cross-sectional regression models for each time
period t:
xt − α̂ = B̂f t + ˆt
Solving for f̂ as the regression parameter estimates of the
regression of observed xt on the estimated factor loadings
matrix. Taking account of the heteroscedasticity in , apply
GLS estimates:
0 −1
f̂ t = [B̂ Ψ̂ ˆ −1 [B̂ 0 Ψ̂−1 (xt − α̂)]
B]
(Optional) Consider coordinate rotations of orthonormal
factors as alternate interpretations of model.
MIT 18.S096 Factor Models 25
 Linear Factor Model
Macroeconomic Factor Models
Fundamental Factor Models
Factor Models
Statistical Factor Models: Factor Analysis
Principal Components Analysis
Statistical Factor Models: Principal Factor Method

Factor Analysis Model

Further Details of ML Specification

Estimated factor realizations can be rescaled to represent
factor mimicking portfolios
Likelihood Ratio test can be applied to test for the number of
factors.
˜ − l(α̂, B̂, Ψ)]
Test Statistic: LR(K ) = 2[l(α̃, Σ) ˆ
where H0 : K factors are sufficient to model Σ and
α̃ and Σ̃ are the MLEs with no factor-model restrictions.

MIT 18.S096 Factor Models 26

 Linear Factor Model
Macroeconomic Factor Models
Fundamental Factor Models
Factor Models
Statistical Factor Models: Factor Analysis
Principal Components Analysis
Statistical Factor Models: Principal Factor Method

Outline

1 Factor Models
Linear Factor Model
Macroeconomic Factor Models
Fundamental Factor Models
Statistical Factor Models: Factor Analysis
Principal Components Analysis
Statistical Factor Models: Principal Factor Method

MIT 18.S096 Factor Models 27

 Linear Factor Model
Macroeconomic Factor Models
Fundamental Factor Models
Factor Models
Statistical Factor Models: Factor Analysis
Principal Components Analysis
Statistical Factor Models: Principal Factor Method

Principal Components Analysis (PCA)

An m−va riaterandom variable:
x1
 .. 
x =  . , with E [x] = α ∈ <m , and Cov [x] = Σ (m × m)

xm
Eigenvalues/eigenvectors of Σ:
λ1 ≥ λ2 ≥ · · · ≥ λm ≥ 0: m eigenvalues.
γ 1 , γ 2 , . . . , γ m : m orthonormal eigenvectors:
Σγ i = λi γ i , i = 1, . . . , m
γ 0i γ i = 1, ∀i
γ0γ = 0, ∀i 6= i 0
Pm i i 0 0
Σ = i=1 λi γ i γ i
Principal Component Variables:
pi = γ 0i (x − α), i = 1, . . . , m
MIT 18.S096 Factor Models 28
 Linear Factor Model
Macroeconomic Factor Models
Fundamental Factor Models
Factor Models
Statistical Factor Models: Factor Analysis
Principal Components Analysis
Statistical Factor Models: Principal Factor Method

Principal Components Analysis

Principal Components in Vector/Matrix Form
m−Variate x : E [x] = α, Cov [x] = Σ
Σ = ΓΛΓ0 , where
Λ = diag (λ1 , λ2 , . . . , λm )
Γ = [γ 1 : γ 2 : · · · : γ m ]
Γ0 Γ = Im
p1
 

p= 

.. 
.  = Γ0 (x − α), m−Variate PC variables
pm
E [p] = E [Γ0 (x − α)] = Γ0 E [(x − E [x])] = 0m
Cov [p] = Cov [Γ0 (x − α)] = Γ0 Cov [x]Γ
= Γ0 ΣΓ = Γ0 (ΓλΓ0 )Γ = Λ
p is a vector of zero-mean, uncorrelated random variables that
provides an orthogonal basis for x.
MIT 18.S096 Factor Models 29
 Linear Factor Model
Macroeconomic Factor Models
Fundamental Factor Models
Factor Models
Statistical Factor Models: Factor Analysis
Principal Components Analysis
Statistical Factor Models: Principal Factor Method

Principal Components Analysis

m−Variate x in Principal Components Form
x1
 
. 
x= 
 ..  = α + Γp, where E [p] = 0m , Cov [p] = Λ
xm
Partition Γ = [Γ1 Γ2 ] where Γ1 corresponds to the K (< m)
largest eigenvalues
 of
 Σ.
p1
Partition p = where p1 contains the first K elements.
p2
x = α + Γ1 p1 + Γ2 p2 = α + Bf + 
where
B = Γ1 (m × K )

f = p1 (K × 1)

 = Γ2 p2 (m × 1)
Like factor model except Cov [] = Γ2 Λ2 Γ02 , where Λ2 is diagonal matrix of last
(m − K ) eigenvalues.
MIT 18.S096 Factor Models 30
 Linear Factor Model
Macroeconomic Factor Models
Fundamental Factor Models
Factor Models
Statistical Factor Models: Factor Analysis
Principal Components Analysis
Statistical Factor Models: Principal Factor Method

Empirical Principal Components Analysis

The principal components analysis of

X = [x1 : · · · xT ] (m × T )
consists of the following computational steps:
Component/row means : x̄ = ( T1 )X1T
‘De-meaned’ matrix: X∗ = X − x̄10T
1 ∗ ∗ 0
Sample covariance matrix: Σ̂x = T X (X )
0
Eigenvalue/vector decomposition: Σ̂x = Γ̂Λ̂Γ̂
yielding estimates of Γ and Λ.
Sample Principal Components:
0
P = [p1 : · · · : pT ] = Γ̂ X∗ . (m × T )

MIT 18.S096 Factor Models 31

 Linear Factor Model
Macroeconomic Factor Models
Fundamental Factor Models
Factor Models
Statistical Factor Models: Factor Analysis
Principal Components Analysis
Statistical Factor Models: Principal Factor Method

Empirical Principal Components Analysis

PCA Using Singular Value Decomposition
Consider the Singular Value Decomposition (SVD) of the
de-meaned matrix:
X∗ = VDU0
where
V: (m × m) orthogonal matrix, VV0 = Im .
U: (m × T ) row-orthonormal matrix, UV0 = Im .
D: (m × m) diagonal matrix, D = diag (d1 , . . . , dm )
with d1 ≥ d2 ≥ · · · ≥ 0.
Exercise: Show that
Λ̂ = T1 D2
Γ̂ = V
0
P = Γ̂ X∗ = DU0
MIT 18.S096 Factor Models 32
 Linear Factor Model
Macroeconomic Factor Models
Fundamental Factor Models
Factor Models
Statistical Factor Models: Factor Analysis
Principal Components Analysis
Statistical Factor Models: Principal Factor Method

Alternate Definition of PC Variables

Given the m−variate x : E [x] = α and Cov [x] = Σ
Define the First Principal Component Variable as
p1 = w0 x = (w1 x1 + w2 x2 + · · · + wm xm )
where the coefficients w= (w1 , w2 , . . . , wm )0 are chosen to
maximize: Var (p1 ) P = w 0 Σx w
subject to: |w| = m2 2
i=1 wi = 1.
Define the Second Principal Component Variable as
p2 = v0 x = (v1 x1 + v2 x2 + · · · + vm xm )
where the coefficients v= (v1 , v2 , . . . , vm )0 are chosen to
maximize: Var (p2 )P = v0 Σx v
subject to: |v| = m
2 2 0
i=1 vi = 1, and v w = 0.
Etc., defining up to pm , The coefficient vectors are given by
[w : v : · · · ] = [γ 1 : γ 2 : · · · ] = Γ
MIT 18.S096 Factor Models 33
 Linear Factor Model
Macroeconomic Factor Models
Fundamental Factor Models
Factor Models
Statistical Factor Models: Factor Analysis
Principal Components Analysis
Statistical Factor Models: Principal Factor Method

Principal Components Analysis

Further Details
PCA provides a decomposition
Pm of the Total Variance:
Total Variance (x) = i=1 Var (xi ) = trace(Σx )
0 0
= trace
Pm (ΓΛΓ ) = trace(ΛΓ Γ) = trace(Λ)
= Pk=1 λk
m
= k=1 Var (pk )
= Total Variance (p)
The transformation from x to p is a change in coordinate
system which shifts the origin to the mean/expectation
E [x] = α and rotates the coordinate axes to align with the
Principal Component Variables. Distance in the space is
preserved (due to orthogonality of the rotation).

MIT 18.S096 Factor Models 34

 Linear Factor Model
Macroeconomic Factor Models
Fundamental Factor Models
Factor Models
Statistical Factor Models: Factor Analysis
Principal Components Analysis
Statistical Factor Models: Principal Factor Method

Outline

1 Factor Models
Linear Factor Model
Macroeconomic Factor Models
Fundamental Factor Models
Statistical Factor Models: Factor Analysis
Principal Components Analysis
Statistical Factor Models: Principal Factor Method

MIT 18.S096 Factor Models 35

 Linear Factor Model
Macroeconomic Factor Models
Fundamental Factor Models
Factor Models
Statistical Factor Models: Factor Analysis
Principal Components Analysis
Statistical Factor Models: Principal Factor Method

Factor Analysis Model

For {xt , t = 1, . . . , T }, the factor model is:

xt = α + Bf t + t
α and B are vector/matrix constants.
All random variables are Normal/Gaussian:

xt i.i.d. Nm (α, Σx )
f t i.i.d. NK (0K IK )
t i.i.d. Nm (0m , Ψ)
Cov (xt ) = Σx = BB 0 + Ψ

Principal Factor Method of Estimation

To fit a K −factor model with fixed K < m, define
X = [x1 : · · · xT ] (m × T )

MIT 18.S096 Factor Models 36

 Linear Factor Model
Macroeconomic Factor Models
Fundamental Factor Models
Factor Models
Statistical Factor Models: Factor Analysis
Principal Components Analysis
Statistical Factor Models: Principal Factor Method

Principal Factor Method of Estimation

Step 1: Conduct the computational steps of principal
components analysis:
Component/row means :x̄ = ( T1 )X1T
‘De-meaned’ matrix: X∗ = X − x̄10T
Sample covariance matrix: Σ̂x = T1 X∗ (X∗ )0
0
Eigenvalue/vector decomposition: Σ̂x = Γ̂Λ̂Γ̂
yielding estimates of Γ and Λ.
Step 2: Specify initial estimates (index s = 0)
α̃0 = x̄
1
B̃ 0 = Γ̂(K ) (Λ̂(K ) ) 2 , where
Γ̂(K ) is submatrix of Γ̂ (first K columns)
Λ̂(K ) is submatrix of Λ̂ (first K columns)
Ψ̃0 = diag (Σ̂x ) − diag (B̃ 0 B ˜ 00 )
Σ̃0 = B̃ 0 B˜ 00 + Ψ̃0
MIT 18.S096 Factor Models 37
 Linear Factor Model
Macroeconomic Factor Models
Fundamental Factor Models
Factor Models
Statistical Factor Models: Factor Analysis
Principal Components Analysis
Statistical Factor Models: Principal Factor Method

Principal Factor Method of Estimation

Step 3: Adjust the sample covariance matrix to

∗
Σ̂x = Σ̂x − Ψ̃0
Compute the eigenvalue/vector decomposition:
∗ 0
Σ̂x = Γ̃Λ̃Γ̃
yielding updated estimates of Γ and Λ
Repeat Step 2 with these new estimates
˜ 01 + Ψ̃1
obtaining B̃ 1 , Ψ̃1 , Σ̃1 = B̃ 1 B
Step 4: Repeat Step 3 generating a sequence of estimates
(B̃ s , Ψ̃s , Σ̃s ) s = 1, 2, . . ., until successive changes in
Ψ̃s are sufficiently negligible.
Step 5: Use the estimates from the last iteration in Step 4.

MIT 18.S096 Factor Models 38

MIT OpenCourseWare
http://ocw.mit.edu

18.S096 Topics in Mathematics with Applications in Finance

Fall 2013

For information about citing these materials or our Terms of Use, visit: http://ocw.mit.edu/terms.