Machine Learning in Finance Workshop 2020

AGENCY MBS PREPAYMENT

MODEL USING NEURAL
NETWORKS
Jiawei “David” Zhang
MSCI
AGENCY MBS PREPAYMENT MODEL
USING NEURAL NETWORKS
Columbia-Bloomberg Machine Learning in Finance 2020
David Zhang
MSCI Securitized Products Research

© 2016 MSCI Inc. All rights reserved.

Please refer to the disclaimer at the end of this document.
— GENERAL —
 Speaker
David Zhang is a Managing Director and Head of Securitized Products Research at MSCI. His
team is responsible for developing models and analytics to support investment analysis, risk
management, and regulatory compliance. Before joining MSCI, Dr Zhang was Managing
Director and head of Securitized Products modeling at Credit Suisse for more than a decade. At
Credit Suisse he was responsible for supporting risk, regulatory and client analytics as well as
sales/trading quantitative strategies. Dr Zhang’s group developed one of the most widely used
MBS models by fixed income institutional investors. Their work was consistently awarded top
ranking by various industry and client surveys, including Institutional Investor All-America
Research Team ranking in Agency prepayment. They also won the award for best paper by the
American Real Estate Society for research on effectiveness of government mortgage programs
The regulatory projects Dr Zhang lead at Credit Suisse included developing models for CCAR
and PPNR (Pre-Provision Net revenue), Dodd-Frank IHC (Intermediate Holding Company) and
related VaR, RWA and RBPL modeling, and FRTB (Fundamental Review of Trading
Zhang, PhD
Book). Prior to Credit Suisse, Dr Zhang worked at FreddieMac, CIBC Oppenheimer, and
ewYork University of Chicago. He holds leadership positions at PRMIA (Professional Risk Management
MANAGING DIRECTOR, HEAD OF
International Association) and GCREC (Global Chinese Real Estate Congress). He is a frequent
speaker at industry and academic conferences, and his research on risk, financial modeling and
SECURITIZED PRODUCTS RESEARCH
real estate has been published in many academic journals. Dr Zhang has a Ph.D. from Princeton
212. 981. 7464 University.
[email protected]

MSCII 3
SUMMARY: NEURAL NETWORKS AGENCY MBS PREPAYMENT MODEL

Why a machine learning model for Agency MBS?

• Prepayment is a highly complex and non-linear process with many idiosyncratic risk factors, among the most
complex financial models
• Recent development in computational software and hardware enable us to make significant advancement in AI
prepayment models
• Machine learning models have excelled in many areas, such as image recognition, natural language processing,
fraud detection, etc.

What is the model and what have we learned?

• Deep neural network model applied to pool level agency MBS prepayment data, compared with MSCI1 (the
human model)
• Results show the deep learning model is able to capture very complex prepayment patterns and signals with
extremely high computational efficiency

4
— GENERAL —
MACHINE LEARNING IN FINANCE

• Consumer credit risk models via Machine-Learning Algorithms (Dr. Andrew Lo,
2010)
Using machine-learning model for consumer credit default and delinquency
Generalized classification and regression trees
Accurately forecasted credit events 3 to 12 months in advance

• Risk and risk management in credit card industry (Dr. Andrew Lo, 2016)
Analyzed very large dataset consisting of credit card data from six large banks.
Decision trees and random forests model perform better than logistic regression at short time horizon

• Deep learning for mortgage risk (Dr. Kay Giesechke, 2015-2018)

Using deep neural network to model mortgage prepayment, delinquency and foreclosure
Loan level data
Compared NNM with a logit model

5
— GENERAL —
 US BOND MARKET

US Bond Market 2018 (42.4$Tn)

Federal Agency
Securities Money Markets
1.05

Mortgage Related
9.66

US Bond Market 2007 (29.5$Tn}

Money Markets
1.79

AgencyMBS
5.80

MSCI •- 6
— GENERAL —
MODELING OBJECTIVE

Forecast prepayment rate for agency RMBS pools

SMM : Single Monthly Mortality Rate

CPR: Conditional Prepayment Rate

Agencies report last month’s prepayment speed on the 4th business day of each month.

Prepayment types:
─ Rate refinance
─ House turnover
─ Cash-out
─ Curtailment
─ Buyout

7
— GENERAL —
 MORTGAGE PREPAYMENT MODELING: SCIENCE AND CRAFTSMANSHIP

Difficulties with mortgage prepayment modeling

• Large data sets: ~20-2000 G data, Agency MBS covers ~400,000 pools/100+mm loans
performance over 20-30 years, pool/loan variables ~30-100

• Multiple, highly non-linear and interactive risk drivers (“layered risk”)

• Loan size vs. prepayment is function of moneyness
• Age vs. prepayment is function of past moneyness history
• Loan purpose (refi vs purchase) vs. prepayment is function of origination year
• ….

• Regime changes
• Mortgage credit and borrower risk appetite cycles, and business practice affect
absolute level and risk drivers for prepayment/default

8
— GENERAL —
 Agency MBS prepayment

• Complex behaviors 50
•• •

.,,...
40
– 30–100 risk factors: rates, loan • •• I •

Model CPR
size, GEO, purpose, property, HPA….
30 •1•• • •
– “layered risk”- non-linear interaction
(e.g., loan size vs moneyness,
20
10
0 •
..·- • •
• •
.. • •
purpose vs. origination year, ..) 0 10 20 30 40 50 60 70 80
Actual CPR
– Regime changes: behavior, policy
– Statistical noises Model CPR distribution
1,000

• Large data set to model 800

Pool count
600
– 400,000 pools/100m loans, 30yr 400
200

• Modelers as craftsman? 0

– Idiosyncratic modeler risk CPR range

90-100bps ITM FNCL 4s pool speeds

MSCII 9
 AI agency MBS prepayment model

Deep neural network model

Architecture* Model training reedforward pass: compare
3
ilr.t ual Oll tp11tWith pred ir.tP.rl
Hidden 1 Select a set of training data: output ilnd determinP. cost
Input layer
Actual
inputs togP.thP.r with Output function based on the error
correct/actual output rnlc::uIaterJ

2 Pass training Al Model

data through
the network
to obtain
predicled
Predicted MeP.ts

--~o•Y•••
output
Output Ctiteria?
r1 r..
:!'.1 = mox(A 1K I B,.,0) :!'.'...- mlll:'.(Ani' ,-i + Bn, 0) Twea~ weiqttls tmcl biases
Vilhtes ot hirJden lilyers ilrP. generilterJ hy Mtiviltion using gradient function
functions with previous layers as inputs, and weights and
biases as parameters 4 Backward propagation: oplirniLe weiqhls am.II.Jiases(p,:Hc1melers)
until some stopping condition has been met by passing the error
•Example of a Neural Network archite.cture Q Node si nal throu h the network usin radient function

• Feed forward neural network • Competing vs. “human” /MSCI production model
– Applied successfully in many other fields – Forecast accuracy
– Layers and nodes, hyper-parameters – New signals, new discoveries
– Ensemble techniques, bagging and boosting – x100-1000 Efficiency gains: 3hrs vs. weeks/months

MSCII 10
 CPR

0
20
40
60
80
Aug-03
Mar-04
Oct-04
May-05

MSCII
Dec-05
Jul-06
Feb-07
Sep-07
Apr-08

Actual
Nov-08
Jun-09

• Higher modeling accuracy

Jan-10
Aug-10
Mar-11

NNM
Oct-11
May-12
Dec-12
Jul-13
Feb-14
Sep-14
Apr-15
Nov-15

– Across cohorts and multiple dimensions of risk factors

– Highly adaptive to high dimensionality and non-linearity
CPR
0
20
40
60
80

Aug-03
Sep-04
Oct-05
Nov-06
Dec-07
Jan-09
Feb-10
CA

Mar-11
CPR

Apr-12
May-13
Jun-14
Jul-15
Aug-16
AI vs. “human” models: higher accuracy

Sep-17
NNM

Dec-03
Jan-05
Feb-06
Mar-07
Apr-08
May-09
Jun-10
NY

Jul-11
Hmodel

Aug-12
Sep-13
Oct-14
Nov-15
11

Dec-16
Jan-18
 AI vs. “human” models: higher accuracy

FICO SATO
30 60
Actual Actual
25 50
NNM NNM
20
40
CPR

15

CPR
30
10
5 20

0 10
720 740 760 780 720 740 760 780
0
Pre-2008 Post-2008 -1.5 -1 -0.5 0 0.5 1

Current loan size CLTV

35 30
Actual Actual
30 ----------------
-~------- NNM 25 NNM
25 .........
20
20 ..... _.........
------
CPR

CPR
15
15
10
10
5 5

0 0
50,000 100,000 150,000 200,000 250,000 300,000 350,000 55 65 75 85 95 105

MSCII 12
 AI vs. “human” models: higher accuracy

Ranking-based sample error tracking for FNCL 4s

45 45
40 40
, I

35 35
,,,
30 30
, ,,
25 25 ,,,'

CPR
CPR

20 20 ,, ,, ,,
15
10
15
10 --
---------
------ _,

,,
5 5 , ,, ----
0 0
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
Actual NNM Actual Hmodel

• Higher modeling accuracy

– Across cohorts and multiple dimensions of risk factors
– Highly adaptive to high dimensionality and non-linearity

MSCII 13
 AI vs. “human” models: higher accuracy

25
• “Media effect”
20 – When rates hit
historical low, new &
lower coupons ramp
15 up faster
CPR

– Highly non-linear
10 behavior and depends
on multiple risk factors

0
May-13

May-13
Jul-12

Jul-13

Jul-12

Jul-13
Dec-12

Jun-13

Dec-13
Sep-12

Aug-12

Apr-13

Aug-13
Jan-13
Mar-13

Sep-13

Oct-12

Feb-13

Oct-13

Sep-12

Jan-13
Mar-13

Sep-13
Nov-12

Nov-13

Nov-12

Nov-13
3.5 4 4.5

Actual NNM Hmodel

MSCII 14
 AI vs. “human” models: new signals

80 60 12

70
50 10
60
40 8
50
CPR

CPR

CPR
40 30 6

30
20 4
20
10 2
10

0 0 0
-60
-40
-20
-10
-100

0
10
20
30
40
60
80
100
120
140
160
200
240
75k 100k 150k 200k 250k 300k 400k 500k

Incentive (bps)

75k 100k 150k 200k 300k 400k 500k 60 bps(LHS) -60 bps(RHS)

• Accuracy vs overfitting: loan size example

– Understand sensitivities of risk drivers and economic rationale
– Apply regularization to penalize overfitting

MSCII 15
 AI vs. “human” models: new signals

Model speeds for OTM 60bps vs loan size (in thousands) Housing turnover vs loan size and year*
12 0.80%
0.70%
10
0.60%
8
0.50%
6 0.40%
0.30%
4
0.20%
2
0.10%
0 0.00%
50 100 150 200 250 300 350 400 450 500 550

2005Yr 2012Yr <= 75k 76k-150k 151k-250k 251k-350k 351k+

• Is low loan balance still safe investment for extension risk?

– Sensitivity tests for the AI model indicate relationship between loan size and housing turnover has flipped after the recession
– This is verified by Black Knight’s proprietary data

MSCII * Source: Black Knight, used with permission 16

CONCLUSION

NN model vs. “Human Model”

• Accurate forecasts and successfully flag prepayment

anomalies over the study period
• Accurate model large numbers of risk factors
• Accurate model highly non-linear and interactive risk factors
• Highly efficient modeling process - hundreds times of
increases in modeling efficiency
• Was able to find/flag prepayment signals that eluded human
models

17
— GENERAL —
ACKNOWLEDGEMENT

Joy Zhang:
Executive Director, Securitized Product Research, MSCI
Jan Zhao:
Principal, Advanced Analytics, Ernst & Young
Fei Teng:
Senior Quantitative Analysts, Quantitative Advisory Services , Ernst & Young
Siyu Lin:
Senior Quantitative Analysts, Quantitative Advisory Services , Ernst & Young
Henry Li:
Executive Director, in Quantitative Advisory Services practice, Ernst & Young

18
— GENERAL —
Appendix

19
 MORTGAGE PREPAYMENT MODELING: SCIENCE AND CRAFTSMANSHIP

4: Agency MBS prepayment regimes since 2003

2003: historical l'ow FN 30 CPR/ Rate Incentive Bracket (Age= 10-30) 2009: historical low mortgage
mortgage rates and rates, but muted refinancing

100 , refinancing wave

---- Mid 2004-,Mid 2006: Strong home price
appreciation, (HPA), fast housing turnovet", high cash -
Mid 2007-2009: Severe
home price depreciation
_response.High premium coupons
prepay ever slower than, lower
out refinancing and fast discount prepayment speed (HPD), low housing turnover, premium coupons, because of
80 -----~--------------st------------ no cash out refinancing and - much tighter lending standard for
slow rate tet"m refinancing

60
ct:
Q.
0 40

20

0
Jan-03 Aug-03 Mar-04 Nov-04 Jun-05 Jan-06 Sep-06 Apr-07 Nov-07 Jul-08 feb-09 Oct-09
Month
-(-50,-25) -(75,100) -(150,175)

MSCI •- 20
— GENERAL —
 MORTGAGE PREPAYMENT MODELING: SCIENCE AND CRAFTSMANSHIP

Exhibit 5: HARP CL TV curve history and long term model assumptions

The CLTV curve represents the ratio of refinance speeds across CLTV spectrum, using sub-50 CL TV cohort as
benchmark, with all other pool variables (for example, loan size, moneyness, FICO, etc.) holding constant

1.80

1.60

1.40
-model(noHARP)
1.20 -model(Long Term)
--HARP1
1.00
- -201202
0.80 --201203
-201212
0.60
---•201204

0.40 -201206

~--------- - -201503
0.20

0.00
60 70 80 90 100 110 120
Current LTV

The HARP program caused temporary inversion of the CLTV Prepayment Curve

HARP: Home Affordable Refinance Program

MSCI •- NNM 21
— GENERAL —
MORTGAGE PREPAYMENT MODELING: SCIENCE AND CRAFTSMANSHIP

V2002 Major Premium Coupons At 2003 Refi Wave

100 In 2003 refi wave, nigher coupons prepay raster.
But in 2009 refi wave, higher coupons prepay
lower e weake ccediLQi!J:Ual!!!!!J.
f,.OQ!f..hd}!li
~rh~e:r..---===t~--'---------.....;::.,.. _________________ _
80
coupons was not much an issue in 2003's refi, but
rt become a big refi hurdle in 2009's refi.
60
Q.
u 40

20

0
Jan-03 Feb-03 Mar-03 Apr-03 May-03 Jun-03 Jul-03 Aug-03 Sep-03 Oct-03 Nov-03 Dec-03
Month

-5.5-6-6.5

V2008 Major Premium Coupons At 2009 Refi Wave

40

30

!f
u
20

In 2003 refi wave,.,bigber cou oos rel'!!',,y.,fa.,,st,,e.,_c..,Be,,ut"-'-"inc,..--2.e.0,.,0..,9,_r._,e.:.,fi..,w"a"v_,.,e

___________________________ _
10
higher coupons prepay slower. The weaker credrt quality of higher
coupons was not much an issue in 2003"s refi. but it become a big refi
hurdle in 2009's refi.
0
Jan-09 Feb-09 Mar-09 Apr-09 May-09 Jun-09 Jul-09 Aug-09 Sep-09 Oct-09
Month

-5.5-6-6.5

MSCI — GENERAL —
22
 MORTGAGE PREPAYMENT MODELING: SCIENCE AND CRAFTSMANSHIP

FN30 6.0 V2004 CPR / Fico, (owner occ clsz=1 G0K-150 K oltv < 80)

■ Seasoned loan's FICO

lending environment

35

30

25

..
C. 20 -
u

15

10

5
■ :FICO beoomes less important after a few years' seasoning
0
"'
9 "'.!..
0 0I
>,
~
0
a.'
..
aI
>
IO
aI
,c:
I,()
0
.!..
0I
>-
0I
a.
0I
> ~
0I
>-
co
0
....!.
<O
q
a.
<O
0
>
r--
aI
r--
a
.'..
r--
0I
>-
r--
0I
r--
0I
a.
r-
0I
>
a:>
0
'
co
~
aI
>-
0I
a.
0I
>
0
..!. 9>,
OJ
a
...!.
C:
...,
Ill Ill
:2i:
(\)
~
...,
:::I
(l)
(/)
0
z
(ti Ill
..., :2i: Ill
~
...,
:::I (l)
(/) z
0 (ti
:2i: ~
Ill ...,
:::I (l)
(/) z
0
C:
..., :a
(ti
(ti Ill
:!?
...,
:::I
(II
(/J
0
z
C:
..., :a
(\)
(ti (ti
:!?
QI
(/J
0
z
(\)
:i
<'O
~
...,
:,

Ill 01 ,ti 1

■ Higher FICO prepays faster for new mortgages

l--+-680-120 ---->I 720

MSCI •- 23
— GENERAL —
 MORTGAGE PREPAYMENT MODELING: SCIENCE AND CRAFTSMANSHIP

Example of modeling:
Assume ppm (pool, time) = f(X1, X2, X3,…. Xn) …

start by assuming separable risk factor: ppm= f1(x1)*f2(x2)…. Until (often) proven
incorrect…

estimating f1(x1) by “building cohort”, by bucketing loans/pools for groups of x1, but similar
x2, x3….
(this further assumes quasi linear property of x2, x3…. Average(f2(x2) f3(x3)…)= f2(ave(x2))*
f3(ave(x3))….

….. Checking overall fit after all Xn are fitted, adding extra variables to deal with non-linear and
interactive variables… this often does not lead to convergence …

• Time consuming and non-standard approaches

• Experience and step-by-step / regime-by-regime progress are valued

• Can new techniques of AI modeling provide the much needed disruption?

24
— GENERAL —
 FEED FORWARD NEURAL NETWORK

neural network model

Architecture* Mode~training Feedforward pass:compare
ac:tual outp11t with predicted
1 Select a set of training data: Actual out rut and dP.termine cost
Input layer inputs tog~thP.r with function based on the error
Output
correct/actual output calqtlnted

2 Pass trail'linq Al Model

data through
the network
to obtain MeP.ts
predidet.l Predicted
Output Criteria?
output
!:1
!'.1 = mllll(tl.1;! I 8 1,0)
!n
:!:'.,,- m111<(A
Values ot hidden layP.rs are ~enerated hy ar.tivation
futtctiu11s with previous layers as inputs, and weights and
biases as parameters
r·,-1+ 8 11, 0)
11 t
Tweak weiqt1Ls i:mu Ltiases
using gradient function

4
~Ye• ·I
Backward propagation: oµlfrnize weiqhls am.I uiases (par c1meter s)
until some stopping condition has been met by passing the error
•Exampfe of a Neural Network architecture Q Node signal through the network using gradien function

Network architecture:
Layers and nodes
Hyper-parameters
Batch size, number of nodes, learning rate, max-norm constraint, dropout rate
Ensemble techniques:
Bagging: minimum MSE of different realizations and neural networks
Boosting: Fine tune a neural network via changing a few hyper-parameters
MSCI •-
— GENERAL —
TRADITIONAL VS.. DEEP MACHINE LEARNING
Identify Set Compare Choose Optimize
Train models Deploy
problems benchmarks performance model model

Traditional learning algorithm Deep Learning

Pros Cons Pros Cons
Works better on smaller Hard to scale state-of-the-art for certain require large amount of data.
data domains, such as computer
vision and speech recognition.
Financially and Lack of variability Perform very well on image, Not suitable for classical
computationally cheap audio, and textual data, Easily machine learning problems.
updated with new data
Algorithms are easier to Labor intensive model Versatile architecture and low Computationally intensive to
interpret, have more maintenance overhead maintenance train, and they require much
theories to back them up more expertise to tune

LargeNN
~
C:
ro
E Medum NN
.g
(I)
a... Sma

Traditional ML algorith
Statisbcal Leaming ms I

MSCI DataSize
— GENERAL —
 NEURAL NETWORKS MODEL
Feed forward neural network (FNN)

the information moves in only forward direction from the input nodes to the output nodes. There are no
cycles or loops in the network.;

Deep FNN consists of tens of layers and thousands of nodes; the simplest kind of FNN is logistic model
X1
y(t) y(l) y(2) y(3)
W1
X2

w =
X3 out

wi Wo x(t) x(l) x(2) x(3)

Xo ~
Xi

Logistic Model Feedforward Neural Network Recurrent Neural Network

Recurrent Neural Network (RNN)

A class of neural networks exploit the historical input sequences. Such inputs could be text, speech, time
series, and anything else where the occurrence of an input in the sequence is dependent on the inputs
that appeared before it
Motivation: Not all problems can be converted into one with fixed length inputs and outputs, such as
text translation, speech recognition or time-series; predictions require a system to store and use context
information
The input at time t include both the attributes at t and the intermediate values containing history at t-1.

MSCI •-
— GENERAL —
BUILDING NEURAL NETWORK MODEL

Feature Build Performance

EDA selection model Evaluation

1, sanity check 1,information value 1,error tracking

1. link weights
2. data cleansing 2.correlation matrix 2.sensitivity
2. hyper-parameters
3. data transformation 3.domain knowledge 3. transparency tool (deeplift)

Deep neural network fitting

2003-2018 30yr agency MBS data (~25G data)

30+ input variables: pool attributes, macro-economic variables

To reduce complexity, we added incentive, 1 regime indicators, and 1 policy

indicator (HARP)

Cost function of RMS error of pool level prepayment

1 round of fitting can be completed in ~ 3 hours on a GPU machine

— GENERAL —
 MODEL DRIVERS
Independent variables
Derived Variables
WALA Weighted Average Loan Age
Incentive WAC - M ortgage Rate(t)
WAC Weighted Average Coupon
Rolling Incentive Average Incentive ( 20month)
CLNSZ Current Average Loan Size
Loan size dispersion (LNSZ_Q3-LNSZ_Q1)/AOL
OLTV Original Loan to Value
SATO Spread-at_origination = WAC - M ortgage Rate(0)
Refi% Percentage of Refinanced Loans by UPB
HPA 1: IssueM
House onth
Price <= Jun. 2009
Appreciation and factor date between
( HPI(t)/HPI(0)-1 ) M ar. 2009
SecHome% Percentage of Second Home Loans by UPB and Dec. 2011
M ultiFamily% Percentage of M uti Family Loans by UPB HARP-able 2: IssueM onth <= Jun. 2009 and factor date > Dec. 2011
Investor% Percentage of Investor Loans by UPB HARP-ed Refi% = 100 and OLTV > 80 and issueM onth > Jun. 2009
TPO% Percentage of Third party origination by UPB Underwritting standard 0: before 2008, 1: after 2008
AOL Original Average Loan Size Weight
LNSZ_Q4 M ax original loan size cBal Current Balance
LNSZ_Q3 M ax original Loan Size - 3rd Quartile Dependent Variable
LNSZ_Q1 M ax original Loan Size - 1st Quartile Prepayment speed Prepayment speed in SM M
Geo_CA% Percentage of California Loans by UPB
Geo_FL% Percentage of Florida Loans by UPB
Geo_TX% Percentage of Taxas Loans by UPB
Geo_NY% Percentage of New York Loans by UPB
Geo_NE% Percentage of New England Region Loans by UPB
Geo_NO% Percentage of North Region Loans by UPB
Geo_SO% Percentage of South region Loans by UPB
Geo_PC% Percentage of Pacific region Loans by UPB
Geo_AT% Percentage of Atlantic region Loans by UPB
Geo_NONUS% Percentage of non-US region Loans by UPB
Seasonality Calendar month

29
— GENERAL —
AGENCY 30YR UNIVERSE SPEEDS ERROR TRACKING
-...J
0
0-,
0
V,
0
.i:,.
CPR
0
w
0
I',.)
0
I-'
0
0

Dec-04

Dec-06

Dec-08

Dec-10

Dec-12

Dec-14
Aug-03

Aug-05

Aug-07

Aug-09

Aug-11

Aug-13

Aug-15
Apr-04

Apr-06

Apr-08

Apr-10

Apr-12

Apr-14
• l>

~
,-+

z
z
~
C
ri
• Training Data: 2003 – 2015 Dec. Random sample 10% pools.
n• Error tracking is generated using out-of-the-sample pools.
tn
~

-
30
— GENERAL —
 OUT-OF-SAMPLE FORECASTS

er::
c..
u

---- Actual --NNM ---Hmodel

• True out-of-time and out-of-sample test.

• Overall fitting is good in out-of-sample test
• Missed the refi wave in second half of 2016

MSCI •- 31
— GENERAL —
 MODEL RISK FACTORS
30 60
25 50
20
40
CPR

15

CPR
10 30
5 20
0 10
720 740 760 780 720 740 760 780
0
Pre-2008 Post-2008 -1.5 -1 -0.5 0 0.5 1
FICO SATO
Actual NNM
Actual NNM

30

-----------
35 25
30
20
25
CPR
20 15
CPR

15
10
10
5 5
0 0
50000 100000 150000 200000 250000 300000 350000 55 65 75 85 95 105
Current Loan Size CLTV

Actual NNM Actual NNM

MSCI •- 32
— GENERAL —
 MODEL BURNOUT

30 -~~-~ .........
25
---~"" ........
......
......
......
...... ......
.....
.....
....
o:::20 ....
....
c..
u
15
10
-----Actual
5
--NNM
0
0.4 0.6 0.8 1 1.2 1.4 1.6
Rolling Incentive

NNM and actual prepayment speeds against average incentive in prior 20 months

MSCI •- 33
— GENERAL —
 ~

MODEL POOL VARIABLES VS “HUMAN”MODEL

tn
n
-
NNM accurately captured state-level prepayment behaviors

NNM and Hmodel Error Tracking against State Variables

CPR
~ N W ~ ~ m ~ 00
0 0 0 0 0 0 0 0 0

Jun-09
Apr-10
n
)> Feb-11
Dec-11
Oct-12
n Aug-13
""Cl
;;o Jun-14
Apr-15
— GENERAL —

Feb-16
Dec-16
z
z Oct-17
s Oct-0

I Aug-0
Jun-OS
I Apr-06
3
0 Feb-07
a.
(1)
Dec-07
Oct-08
Aug-09
Jun-10
z
-< Apr-11
Feb-12
Dec-12
Oct-13
Aug-14
Jun-15
Apr-16
Feb-17
Dec-17
34
 MODEL POOL VARIABLES VS HUMAN MODEL
Ranking-Based Sample Error Tracking for Coupon 4s

45 45

40 I 40
I I
35 I 35 I
I I
, I
,I
,,
30 30
,
,, ,,
a:: 25 a:: 25
,,
Cl.
u 20

-- --
--
_, ,,'
Cl.
u 20

---- --
---- ----.....------- -
15 15
10 ,,
5
10
5 , ,, --
0-+-----~~-~----~~-~----~~- 0
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15

----Actual -NNM ----Actual -Hmodel

• Ranking based error tracking methodology provides a comprehensive measure

of model accuracy across all pool variables
• NNM performed better than Hmodel

MSCI •- 35
— GENERAL —
 MODEL POOL VARIABLES VS HUMAN MODEL
50
----Actual -NNM
40

1 4 7 1013 3 6 9 1215 2 5 8 1114 1 4 7 1013 3 6 9 1215 2 5 8 1114

3.5 4 3.5 4 3.5 4

Jun-13 Jun-16 Mar-18

50 ----Actual -Hmodel
40

o:::30
Q..
u 20

1 4 7 1013 3 6 9 1215 2 5 8 1114 1 4 7 1013 3 6 9 1215 2 5 8 1114

3.5 4 3.5 4 3.5 4

Jun-13 Jun-16 Mar-18

Sample ranking-based error tracking at different time point

MSCI •- 36
— GENERAL —
 MODEL HARP EFFECTIVENESS
Error Tracking against HARP effectiveness across CLTV Cohorts

60
55
' ' '"...-,
50 '
45
';x:::J
- ~ ~
c.::
u 40
Q. ,
35
'
30
25
20
60 80 9010 12 60 80 9010 12 60 80 9010 12

2012Q1 2012Q2 2012Q3 2012Q4

CLTV
----Actual NNM

NNM is able to pick up the general trend of HARP effectiveness but missed the
complexity of its revolution

MSCI •- Back
37
— GENERAL —
 MODEL SENSITIVITY
Model prepayment sensitivity to loan sizes and refinance Incentives
80 60 12
70
so 10
60

so 40 8

fu 40
a:
fu

,.~~~~,,,t.,.....
30 6 Cl.
30 u

20

10
l...... 20

10
4

2
0 ---t---.----.----.---.--.----.----.~~~~~~~--.----.---~
-100 -40 -10 10 30 60 100 140 200
0 ---t-----.-~~~~--.---~--.-----+ 0
Incentive (bps)
~ _c-.'4.-~'4.- r;:f- ~'4.- f::)'4.-_c-.'4.-r;:f-
"\ ""\)-~ '"1-o'"I-<-,,,_,o~\)-~o
~7Sk -100k-a-1SOk~200k

-300k-+-400k-SOOk -60 bps(LHS} -.--60 bps(RHS}

NNM captured the prepayment behavior for loan size

MSCI •- 38
— GENERAL —
 “MEDIA EFFECT”
FH 2011 3.5 vs 2010 4 comparisons, across TPO/Retail and Refi/Purchase combinations
Cohort Observation Range CPR WALA SATO CLTV CLNSZ Incentive FICO Avg.UPB(bn)
Purchase/Retail
FH 3.5 2011 Jul.12 - Dec. 12 16.1 13 -5 77 212258 52 770 2.91
FH 4 2010 Nov. 11 - Feb. 12 13.9 15 3 78 201901 45 767 6.26
Purchase/TPO
FH 3.5 2011 Jul.2012 - Dec. 12 21.9 12 -3 76 235847 50 770 4.04
FH 4 2010 Nov. 11 - Feb. 12 16.4 16 3 78 224734 45 765 8.66
Refi/Retail
FH 3.5 2011 Jul.12 - Dec. 12 29.2 12 -2 66 216270 54 771 7.31
FH 4 2010 Nov. 11 - Feb. 12 15.3 15 11 70 208962 52 766 30.89
Refi/TPO
FH 3.5 2011 Jul.12 - Dec. 12 46.1 12 64 269298 -8
46 773 9.58
FH 4 2010 Nov. 11 - Feb. 12 26.2 15 69 245496 2
44 767 23.02

~-

,...
,...

::a
c..
s
""'II
<:"I
w

'Tl

OQ
OQ
t1)

t1)
Q)

Q)
0
0
V,
V,
2011 3.5s and 2010 4s prepayment speeds
are compared across loan attributes, loan

V,
0
purpose and origination channel

V,
.i:,.
.i:,.
0
3.5s is much faster than 4s given similar

V,
w
loan attributes and incentive

w
0

Sep-11

Dec-12

Sep-16

Dec-17
Feb-12

Feb-17
Jan-10

Jan-15
Jul-12

Jul-17
Nov-10

Nov-15
Jun-10

Jun-15
Aug-14
Oct-13
May-13

May-18
Mar-14
Apr-11

Apr-16
n
tn
~

:::0
a..
m

m
~

C]Q

C]Q
"Tl
w
-<
39

:-,

r-t-

r-t-
QJ

QJ
X

-,
0

0
— GENERAL —
 ~

MODEL “MEDIA EFFECT”

tn
n
- 0 V,
f-'>
0
CPR
f-'>
V,
N
0
N
V,

2012 Vintage in 2012 refinance wave

Jul-12
~h Oct-12
Jan-13
VJ
Apr-13
' V,

-
)>
("')

C
(l)
Jul-13
Oct-13

Sep-12
z Dec-12
z Mar-13
~ .i:,.

Jun-13
Sep-13
Dec-13
I
3 Aug-12
0
Q. Nov-12
~
.i:,. Feb-13
V,
May-13 I ""> "'~
Aug-13
— GENERAL —

Nov-13

CPR
f-'> f-'> N N VJ
0 V, 0 V, 0 V, 0

--1
May-16
2015 Vintage in 2016 refinance wave
Sep-16
VJ Jan-17
' V,

-
)>
("')

C
(l)
May-17
Sep-17
Jan-18

Apr-16
Aug-16
z
z Dec-16
~ .i:,.
Apr-17
Aug-17
Dec-17
Ai;ir-18
I
3 Mar-16
0 Jul-16
Q.
~
.i:,.
V,
Nov-16
Mar-17
~-,
s.
Jul-17
~i-
Nov-17
Mar-18 ~, - ..
40
 ~

MODEL ERROR TRACKING

tn

2. Risk driver is missing, i.e., media

1. All attributes statistics are very

n

effect or regime change

except CPR.
close on July and August 2016
-
(1/2016-4/2018)
Out-of-sample
Out-of-time

(1/2003-12/2015)
Out-of-sample
In-time
!""""
20160201
20160301
20160401

20160501
0 •
...,

0
...,..

•
N

0
Nm

• H
3
§:
~
"'

~
8
"
I::::f 20040401
20040801
20041201
20050401
20050801
IffI IH
..... rvw.::..v,~"

11 3
§:
0
,l
a,

"

r;'
~
~
— GENERAL —

20160601 20051201
20160701 20060401
20060801
20160801 20061201
20160901 20070401
20161001 2oo 7 os01
20071201
20161101 20080401
20161201 20080801
20170101 20081201
20090401
20170201 20090801
20170301 20091201
20170401 20100401
20100801
20170501 20101201
20170601 20110401
20170701 20110801
20111201
20170801 20120401
20170901 20120801
20171001 20121201
20130401
20171101 20130801
20171201 20131201
20180101 20140401
20180201 20140801
20141201
20180301 201S0401
20180401 201S0801
20151201

I
~
Q.
~i;
!I I Ii
s: ►~
Q
;!; ;:;
-,:, -o
it
r,
&.
n
:::0 ;Q "0 "0
:,:, :,:,
41
 ~

MODEL ERROR TRACKING

tn
n
-
(11/2016-4/2018)
Out-of-sample
Out-of-time

(1/2003-10/2016)
Out-of-sample
In-time
;'~O ONAO'!OO -----,v
0,._,,AC"+COO
~ I ! oo~~~~E~>
"!!.
~ rl
"'6'
..✓• IL< s. 20030801 ~ I I I I I "I I i
,. o✓• q, i f M 20040101
6''Y,
rl q, vj / V I . • 20040601
o.,,_.o
rl :;,~
o.,,..,,
/ ~v j f 20041101
20050401
20050901
orl
rlo.,,~
— GENERAL —

20060201
->o.,,.
j~ 20060701
o.,, ~

\
rl 20061201
:..>4. 20070501
o.,,~
rl
20071001
->a
~o.,,..,,
~ 20080301
20080801
rl~ 20090101
o.,,_.o 20090601
rl :>~
o.,,..,, 20091101
) )
~ 20100401
o.,,~
rl
20100901
.>~
rl v.,, I
20110201
o.,,:.> 20110701
rl ,l~ 20111201
o.,,
~ 'I 20120501

~
rl ,l~
o.,,:.> 20121001
'Y, 20130301
rl ~
o✓, 20130801
d'o
~ :;,~
o✓•
d'orl
f 20140101
20140601
rl ~ 20141101
o✓,
d'o.,,. 20150401
~ ~ 20150901
o✓•
d'q,. 20160201
~
20160701
J

I I'
;:: ► ll I If
.:: >
~,.., [,.., i,.., [.,,
.,, .,,
:,0 :,0
.,, ,..,
:,0 :,0
42
 ~

MODEL ERROR TRACKING

tn
n
-
(11/2016-4/2018)
Out-of-sample
Out-of-time

2.

1. Better in the early stage

in training:
8/2016 – 10/2016 by 40 times
When Increase weights on

(1/2003-10/2016)
Out-of-sample
In-time
Sacrifice other period.
of out-of-time test

I :::o,
" 0.t_,
.,0
0
°"
"o,.,
"o,.,
.. • • " " • • • H I ::::fj . . . j j H
~
!i
i
~
'§
20040601
20041101
20050401
i ~
1!I~
•

~ 1'§
..-_,od 20050901
.> .,,., 20060201
— GENERAL —

0
"'>o.,,
0 20060101
<'o .,_," 20061201

~... 40

d 0-£>,.,
20070501
~~
20080301
.,,.,_, 20080801
.>
0 %,., 20090101
">o_,. 20090601
<'o
;,_,
".t 20091101
~ 20100401

d'
"o,.,_,

d
~
"

'b✓
20100901

rn~
~~
20111201
0
.,.,,., 20120501
~~ ~~
"'>,.,.,o 20130301
<'o " 20130801
"e.
0.to 20140101
d ;,
".tc9
0 20140601
d ?o✓ 20141101
".tc9
0 20150401
::,i
~ ~ 2M50901
"d'4; 20160201
0,., 20160701
0 1 • -1,(

I If
2;:: lll> fI I l l>
ll
&" &
;:;r,
-i::i ""Cl
nn&
"Cl ~
:,:i,::, :,::,;o
43
MODEL ERROR TRACKING
BudcetValue • Coupon •

Aclllal CPR ModelCPR

70 ..-------------------------------------

In-time 60+-------------------------------------
50-tt-------------------------------------
Out-of-sample 40 ------------------------------------------

(1/2003-10/2017) 30 +1-~1----------------,..----':'"'ll,,-r-----------
Values
--Actual CPR
20 t--'V--bl5H~t::----------l~Ht-r-#--¼-----,1i------¼----,,.----k-- -Model CPR

10 -l------¥~'1¥-!!ilic::,l-~+---------...:!l..,.~l!-....::l"'---\,,ey.
0 tt,Tmmmm"lffl'ITTTTTmTTmmmm....,'TfflTTTTTTffl"IT'mmmm'fflTTTTTTTmTTmmmm"lffl'ITTTTTfflTTfflTTmm....,TTTTTTTTTTmmmmmm'fflTTfflffmmm"f

Fac!O<Oale•

BudcetValue • Coupon •

AclllalCPR ModelCPR
12

Out-of-time
Out-of-sample 8

(10/2017-4/2018) 6+-------------------------------------
Values
--Actual CPR
-Model CPR
4

0+--------------~-----~------~------~------~
20171101 20171201 20180101 20180201 20180301 20180401

MSCI Fac!O<Oale•

— GENERAL —
44
 About MSCI

MSCI is a leading provider of critical decision support tools and services for the
global investment community. With over 45 years of expertise in research, data
and technology, we power better investment decisions by enabling clients to
understand and analyze key drivers of risk and return and confidently build
more effective portfolios. We create industry-leading research-enhanced
solutions that clients use to gain insight into and improve transparency across
the investment process. To learn more, please visit www.msci.com.

MSCII 45
 Contact us

AMERICAS EUROPE, MIDDLE EAST ASIA PACIFIC

& AFRICA

Americas +1 888 588 4567 * Cape Town + 27 21 673 0100 China North 10800 852 1032 *
Atlanta + 1 404 551 3212 Frankfurt + 49 69 133 859 00 China South 10800 152 1032 *
Boston + 1 617 532 0920 Geneva + 41 22 817 9777 Hong Kong + 852 2844 9333
Chicago + 1 312 675 0545 London + 44 20 7618 2222 Mumbai + 91 22 6784 9160
Monterrey + 52 81 1253 4020 Milan + 39 02 5849 0415 Seoul 00798 8521 3392 *
New York + 1 212 804 3901 Paris 0800 91 59 17 * Singapore 800 852 3749 *
San Francisco + 1 415 836 8800 Sydney + 61 2 9033 9333
São Paulo + 55 11 3706 1360 Taipei 008 0112 7513 *
Toronto + 1 416 628 1007 Thailand 0018 0015 6207 7181 *
Tokyo +81 3 5290 1555
• = toll free

msci.com
[email protected]

MSCII 46
 Notice & disclaimer
This document and all of the information contained in it, including without limitation all text, data, graphs, charts (collectively, the “Information”) is the property of MSCI Inc. or its subsidiaries (collectively, “MSCI”), or MSCI’s licensors, direct
or indirect suppliers or any third party involved in making or compiling any Information (collectively, with MSCI, the “Information Providers”) and is provided for informational purposes only. The Information may not be modified, reverse-
engineered, reproduced or redisseminated in whole or in part without prior written permission from MSCI.
The Information may not be used to create derivative works or to verify or correct other data or information. For example (but without limitation), the Information may not be used to create indexes, databases, risk models, analytics,
software, or in connection with the issuing, offering, sponsoring, managing or marketing of any securities, portfolios, financial products or other investment vehicles utilizing or based on, linked to, tracking or otherwise derived from the
Information or any other MSCI data, information, products or services.
The user of the Information assumes the entire risk of any use it may make or permit to be made of the Information. NONE OF THE INFORMATION PROVIDERS MAKES ANY EXPRESS OR IMPLIED WARRANTIES OR REPRESENTATIONS
WITH RESPECT TO THE INFORMATION (OR THE RESULTS TO BE OBTAINED BY THE USE THEREOF), AND TO THE MAXIMUM EXTENT PERMITTED BY APPLICABLE LAW, EACH INFORMATION PROVIDER EXPRESSLY DISCLAIMS ALL
IMPLIED WARRANTIES (INCLUDING, WITHOUT LIMITATION, ANY IMPLIED WARRANTIES OF ORIGINALITY, ACCURACY, TIMELINESS, NON-INFRINGEMENT, COMPLETENESS, MERCHANTABILITY AND FITNESS FOR A PARTICULAR
PURPOSE) WITH RESPECT TO ANY OF THE INFORMATION.
Without limiting any of the foregoing and to the maximum extent permitted by applicable law, in no event shall any Information Provider have any liability regarding any of the Information for any direct, indirect, special, punitive,
consequential (including lost profits) or any other damages even if notified of the possibility of such damages. The foregoing shall not exclude or limit any liability that may not by applicable law be excluded or limited, including without
limitation (as applicable), any liability for death or personal injury to the extent that such injury results from the negligence or willful default of itself, its servants, agents or sub-contractors.
Information containing any historical information, data or analysis should not be taken as an indication or guarantee of any future performance, analysis, forecast or prediction. Past performance does not guarantee future results.
The Information should not be relied on and is not a substitute for the skill, judgment and experience of the user, its management, employees, advisors and/or clients when making investment and other business decisions. All Information is
impersonal and not tailored to the needs of any person, entity or group of persons.
None of the Information constitutes an offer to sell (or a solicitation of an offer to buy), any security, financial product or other investment vehicle or any trading strategy.
It is not possible to invest directly in an index. Exposure to an asset class or trading strategy or other category represented by an index is only available through third party investable instruments (if any) based on that index. MSCI does not
issue, sponsor, endorse, market, offer, review or otherwise express any opinion regarding any fund, ETF, derivative or other security, investment, financial product or trading strategy that is based on, linked to or seeks to provide an
investment return related to the performance of any MSCI index (collectively, “Index Linked Investments”). MSCI makes no assurance that any Index Linked Investments will accurately track index performance or provide positive investment
returns. MSCI Inc. is not an investment adviser or fiduciary and MSCI makes no representation regarding the advisability of investing in any Index Linked Investments.
Index returns do not represent the results of actual trading of investible assets/securities. MSCI maintains and calculates indexes, but does not manage actual assets. Index returns do not reflect payment of any sales charges or fees an
investor may pay to purchase the securities underlying the index or Index Linked Investments. The imposition of these fees and charges would cause the performance of an Index Linked Investment to be different than the MSCI index
performance.
The Information may contain back tested data. Back-tested performance is not actual performance, but is hypothetical. There are frequently material differences between back tested performance results and actual results subsequently
achieved by any investment strategy.
Constituents of MSCI equity indexes are listed companies, which are included in or excluded from the indexes according to the application of the relevant index methodologies. Accordingly, constituents in MSCI equity indexes may include
MSCI Inc., clients of MSCI or suppliers to MSCI. Inclusion of a security within an MSCI index is not a recommendation by MSCI to buy, sell, or hold such security, nor is it considered to be investment advice.
Data and information produced by various affiliates of MSCI Inc., including MSCI ESG Research LLC and Barra LLC, may be used in calculating certain MSCI indexes. More information can be found in the relevant index methodologies on
www.msci.com.
MSCI receives compensation in connection with licensing its indexes to third parties. MSCI Inc.’s revenue includes fees based on assets in Index Linked Investments. Information can be found in MSCI Inc.’s company filings on the Investor
Relations section of www.msci.com.
MSCI ESG Research LLC is a Registered Investment Adviser under the Investment Advisers Act of 1940 and a subsidiary of MSCI Inc. Except with respect to any applicable products or services from MSCI ESG Research, neither MSCI nor
any of its products or services recommends, endorses, approves or otherwise expresses any opinion regarding any issuer, securities, financial products or instruments or trading strategies and MSCI’s products or services are not intended
to constitute investment advice or a recommendation to make (or refrain from making) any kind of investment decision and may not be relied on as such. Issuers mentioned or included in any MSCI ESG Research materials may include
MSCI Inc., clients of MSCI or suppliers to MSCI, and may also purchase research or other products or services from MSCI ESG Research. MSCI ESG Research materials, including materials utilized in any MSCI ESG Indexes or other products,
have not been submitted to, nor received approval from, the United States Securities and Exchange Commission or any other regulatory body.
Any use of or access to products, services or information of MSCI requires a license from MSCI. MSCI, Barra, RiskMetrics, IPD and other MSCI brands and product names are the trademarks, service marks, or registered trademarks of MSCI
or its subsidiaries in the United States and other jurisdictions. The Global Industry Classification Standard (GICS) was developed by and is the exclusive property of MSCI and Standard & Poor’s. “Global Industry Classification Standard
(GICS)” is a service mark of MSCI and Standard & Poor’s.

MSCII 47