Credit Risk –

Predictive Modelling
4EK614

28 April 2021
With You Today

Our services

Radek Lastovicka
Senior Manager in Credit Risk Team
Credit Risk Team
[email protected]

Risk Impairment
Parameters Loss
Jan Nusko
Senior Consultant in Credit Risk Team
[email protected]
AQR Regulatory

Our projects
Model Development Data Analysis Stress Testing
Model Validation Data Mining Impairment

Methodological Reviews LIC© Regulatory Reporting

Asset Quality Reviews Advanced Analytics Business Intelligence

Page 2 Credit Risk – Predictive Modelling

About This Seminar

Study materials
Course Structure
1. PowerPoint slides, provided after the course
Day 1: Credit Risk, Underwriting Process, Predictive Modelling
Day 2: Market Risk
Prerequisites
Classroom: Microsoft Teams
Time: 9:15 – 12:30 1. Basic understanding of statistical and mathematical concepts
2. Elementary knowledge of programming (Python, R, …)

Course Assessment

1. Case study – you can choose market risk and/or credit risk topics:
a) Credit Risk - Preparation of PD scorecard:
a) Prepare development sample from portfolio of mortgage loans
b) Model scorecard using logistic regression (or any technique you want!) and include assessment
b) Market Risk - TBA

2. Outputs – PPT presentation or PDF, summarizing the abovementioned outputs, and scripts used.

3. Output presentation – short (10-15 minute) presentation about results of this assessment.

Page 3 Credit Risk – Predictive Modelling

Agenda

1. Credit Risk & Banking W 09:15-09:50 Operative

2. Underwriting & Scoring W 09:50-10:25 Don’t hesitate to ask or comment at any point
Especially since it’s just virtual 
3. Predictive Modelling W 10:30-11:15
We recommend teams for case study
4. Scorecard Development W 11:15-11:50 Menti.com – 14 13 31 65

5. Model Assessment W 11:50-12:20

6. Q&A W 12:20-12:30

Page 4 Credit Risk – Predictive Modelling

Coursework

Goal Resources – please write an email to

[email protected]
• Your task is to build a PD scorecard using the • Mortgage_sample.csv: Modelling dataset
provided data. The goal is to create a model that with data about 50000 US mortgages
will predict a probability of default for each
mortgage.
• Mortagage_metadata.xlsx: Data dictionary
• The presentation contains an overview of a
proposed modelling process and some • Package suggestions:
considerations to consider when developing and • Python - scorecardpy
assessing the model.
• R - scorecard
• You will be assessed on the “good modelling
practice” you employ. Remember, the best model
is not necessarily the one with the highest
performance metric. Your goal should be to build
a scorecard with enough discriminatory power,
but the steps taken during the modelling process
are most important.

Page 5 Credit Risk – Predictive Modelling

Banks

Page 6 Credit Risk – Predictive Modelling

Balance sheet and off-balance sheet of a bank

Assets Liabilities
Cash Deposits from customers
Deposits at central bank Loans from other banks
Balance

Loans to customers Securities

sheet

Loans to other banks Hybrid instruments

Securities Other liabilities
Other assets Equity

Undrawn limits of credit lines Undrawn limits of credit lines

Off-balance

Loan commitments Guarantees received

sheet

Guarantees given Derivatives

Derivatives

Page 7 Credit Risk – Predictive Modelling

What is credit risk?

• The risk that a counterparty fails to meet a contractual obligation

Banking book Trading book

▪ Retail: mortgages, credit cards ▪ Counterparty credit risk (CCR): whenever a

▪ Corporate: Investment property financing, trade is settled in the future and/or is not
project financing, large corporate lending “delivery versus payment” (DvP), a firm takes
▪ Wholesale: Lending to banks & sovereigns on credit risk

Insurance Other

▪ Reinsurer default ▪ Intermediary: Default on commissions

▪ Corporate bond / ABS default / CDS receivable
▪ Derivative counterparties ▪ Accounts receivable: Non payment of invoice

Page 8 Credit Risk – Predictive Modelling

Components of credit risk

▪ Probability of Default: The likelihood the borrower will default on its obligation
PD either over the life of the obligation.

▪ Loss Given Default: Loss that lender would incur in the event of borrower’s
default. It is the exposure that cannot be recovered through bankruptcy
LGD proceedings, collateral recovery or some other form of settlement. Usually
expressed as a percentage of exposure at default.

▪ Exposure at Default: The exposure that the borrower would have at default.
EAD Takes into account both on-balance sheet (capital) and off-balance sheet
(unused lines, derivatives or repo transactions) exposures and payment
schedule.

Expected Credit Loss (ECL) = PD x LGD x EAD

Page 9 Credit Risk – Predictive Modelling

Credit risk agenda

► Risk management function reshaping roadmap ► Diagnostics on the effectiveness & efficiency of the
collections process
► Credit risk strategy and linkage to business strategy
► Development of a collections strategy, strategic and
► Risk appetite framework and statements tactical (cost-benefit) analysis of available
► Credit risk processes and segregation of duties outsourcing options

► Model governance framework (model request, design ► Design of a collections framework

implementation, validation) ► Support with collections technology requirements
► Stress testing framework analysis, selection and implementation of an
appropriate solution

Governance Collection services

Application process Performing portfolio Non-performing portfolio

Application scoring Rating models Provisioning LGD models

► Business model request ► Model design / validation / ► Design of impairment ► LGD estimates design and
specification internal audit reviews methodology in line with IFRS validation
► Application scorecard design ► Regulatory compliance ► Effective interest rate and ► LGD (scoring) models design
and validation ► PD estimation recognitions of fees and and validation
► Design and review of the Model usage for business commissions
► ► LGD data warehouse
application processes purposes ► Back-testing analyses specification
► Support with application ► Proprietary IT tools ► Collateral valuation scenarios
workflow technology

Page 10 Credit Risk – Predictive Modelling

Underwriting process
Underwriting process

• Underwriting (UW) process is the processing of credit application and making a decision about
the final approval or decline of the application.

• Generally the UW process can end up in several different states: approval, decline, cancelation
from client side, non-eligibility (for example the applicant is not meeting minimum age criteria,
etc.)

Product Personal/ Client Scoring and Decision /

parameters financial identification Overdebtedness Contract
selection information verification signature

Page 12 Credit Risk – Predictive Modelling

Underwriting process

• Client segment is a crucial parameter to the UW process and scoring

Entrepreneurs
Private individuals Small business Corporates
Freelancers
• Usually automated process • Usually automated process with • Partially automated process, but • Typically manual assessment on
• Scoring applications in order to assess possibly manual inputs mostly manual assessment yearly basis (rating process using
riskiness of newly issued loans/credits • Scoring applications in order to • Scoring applications for automated financial, qualitative and behavioral
• Scoring client behavior on monthly assess riskiness of newly issued products scoring)
basis on credit and deposit products loans/credits • Process for manual yearly rating • Sometimes not sufficient data to use
• Large data sets → statistical approach • Scoring client behavior on (typically financial scoring, statistical approach – especially in
• Need to verify income and over- monthly basis qualitative scoring and behavioral case of project financing
indebtedness • Large data sets → statistical scoring) • Industry dependent and seasonal
• Credit registers (BRKI, NRKI, Solus) approach • Sufficient data sets for statistical • Credit registers (CRÚ, Cribis,
• No need to verify income and approach Bisnode, etc.)
over-indebtedness • Credit registers (CRÚ, Cribis,
• Credit registers Bisnode, etc.)

Page 13 Credit Risk – Predictive Modelling

Underwriting process

• Underwriting process differs significantly for different products

Credit card, Overdraft

Mortgage Consumer loan and Revolving Investment loan

• Financing housing needs • Purpose or non-purpose • Credit limit that can be utilized, but • Typical financing for corporate and
• Subject to consumer protection • Subject to consumer protection it is not a must small business segments, but also for
• Requires real estate collateral and • Can have collaterals or guarantors, • Client can flexibly utilize whatever entrepreneurs
insurance but usually it doesn’t part of the limit he needs to • Processed manually
• Large financed amount • Automated, easy and fast UW • Grace period • Very high financed amount
• Typically longer maturity process • High interest rates • Based on business and financial plan
• More thorough and detailed UW • Higher interest rates • Typically no collaterals • Usually with collaterals and
process • Co-applicants possible, but not • Lower financed amount guarantees
• Partially manual assessment that frequent as for mortgages • Maturity is not specified (contract
• Loan to value condition • Medium financed amount terminates on request when fully
• Lower interest rates • Medium maturity repaid)
• Fixation periods • Medium risk • High risk
• Co-applicants possible • Credit cards come with plastic card

Page 14 Credit Risk – Predictive Modelling

Underwriting process

• First step in the process is the assessment of client general eligibility

• Is the client over 18 years old?
• Is the client eligible to sign contracts?
• Is the client on the international sanction list?
• Is the client a politically exposed person?
• Has the client a tax domicile in the same country?
• Does the client agree with all the legally required actions (credit bureau request, information protection principles,
general terms and conditions, pre-contractual information, etc.)?

• Second step is the assessment of client eligibility for the given product and channel
• Is the client below prescribed age when applying for a long term product such as mortgage?
• Does the client have eligible income for the particular product and process?
• Does the client have all prescribed documents (valid ID card and valid second ID document)?
• Is the collateral for the issued loan eligible and sufficient (LTV threshold)?

Page 15 Credit Risk – Predictive Modelling

Underwriting process

• There are several laws and directives that affect the underwriting process
Consumer needs to be protected from dishonest and
Law on consumer loan malicious practices including intentional over-
indebting, but also non-intentional over-indebting –
the responsibility of not over-indebting the client is
Consumer protection now on the borrower

Mortgage credit directive (MCD) Market and economy needs to be protected against
adverse economic impacts originating in the
financial system
Consumer credit directive (CCD)
Society needs to be protected against criminal acts
EBA guidelines and terrorism

Basel Capital Accord Consumer needs to be protected against loosing his

money deposited in a bank by irresponsible lending
and crediting banks clients
Anti-money laundering (AML)

Page 16 Credit Risk – Predictive Modelling

Underwriting process

•
• Client authentication •
Expiry date check – ID not expired
Check on validity in MPSV database
• Anti-fraud module • Issue date consistency check (based on linear regression below)
• Check on issue date – not week-end or public holiday
• Check on address at MěÚ or OÚ
• Control on ID manipulation (color histogram, fonts)
• Check on consistency of bar-code and ID number
• Consistency of sex and birth number (third digit)
• Birth date divisible by 11 after 1953
• Overall control number check
• Expiry date control number check
• Birth date control number check

Page 17 Credit Risk – Predictive Modelling

Underwriting process

• Internal blacklists on phone numbers, ID cards, IČO of employers, ready-made companies

• Frequency checks in on-line underwriting process (applications are tracked with respect to different identificators and
their combinations
• Device fingerprint (publicly available libraries)
Hardware: CPU architecture & device memory, GPU canvas, Audio stack
Software: User agent, OS version,
Storage: local storage, session storage
Display: color depth, screen size
Browser customizations: fonts, plug-ins, codecs, mime types, time zone, user language,
Miscellaneous: floating point calculations, callbacks / objects to DOM
• Phone number
• Account number
• ID card number
• E-mail address
• Birth number
• IP address

• Geolocation (via IP address and Google API) – can be used for anti-fraud as well as for scoring

• Checks on discrepancy between past applications with the same identifiers

Page 18 Credit Risk – Predictive Modelling

Underwriting process

• Individuals / Entrepreneurs:
• BRKI – Banking Register of Client Information
• Information about applications and loan contracts shared among the banks operating in Czech Republic. Generally only banks
can access it.
• Information is stored in BRKI during the existence of credit relationship and 4 years after it terminates. If the contract with the
bank has not been signed is this information in BRKI stored for one year.
• NRKI – Non-Banking Register of Client Information
• Information about applications and loan contracts shared among non-bank credit providers. Generally only those that
participate on the sharing can access it.
• SOLUS
• Information about applications and loan contracts shared among participating credit providers and some other companies.
Generally only those that participate on the sharing can access it. It contains both – register of negative as well as register of
positive information.
• In SOLUS participate also TELCO companies and utility providers.

• Companies / Entrepreneurs:
• CRÚ – Kreditní Registr Úvěrů
• Information about loan contracts of entrepreneurs and companies – compulsory register operated by Czech National Bank.

Page 19 Credit Risk – Predictive Modelling

Underwriting process

• Scoring is one of the tools to measure the creditworthiness of a business or person. It is the
result of scoring, where different scales are given different weight. This procedure results in
a credit score
Physical location,
work location,
Transactional
household location
profile,
behavior on
deposits

Account Invoice payment

statements, history, services,
application data, location track
psychoscoring

DATA
Text analytics,
Credit registers,
friends, posts,
social security,
activity, job
health insurance,
history
government

Relatives,
Device price, age
transactional networks
(suppliers, cost
and attractivity,
structure) level of user
experience

Page 20 Credit Risk – Predictive Modelling

Predictive modelling
Predictive Modelling - Model life-cycle

Model monitoring & 4 Model request

review 1 Business line
Management
Monitoring
Independent review
Internal audit Model
lifecycle
steps

Model implementation & Model development

usage 3 Management
IT & data 2 Risk
Loan approval
Capital requirement calculation

Page 22 Credit Risk – Predictive Modelling

Predictive Modelling - Goal

• Problem to be solved: single own house

university degree
29 years

targe
predictors t
i=1 i=2 i=3 i=4 …..

find a function f such that

𝑇 𝑓 𝑎𝑔𝑒𝑖 , 𝑠𝑡𝑎𝑡𝑢𝑠𝑖 , 𝑒𝑑𝑢𝑐𝑎𝑡𝑖𝑜𝑛𝑖 , ℎ𝑜𝑢𝑠𝑖𝑛𝑔𝑖 , … − I 𝑖 defaulting in 1 year from snapshot

is in some sense minimized
Can be numeric, ordinal,
Attains only values 0, or 1
nominal or even missing

the value f(.) we call probability of default (PD)

Page 23 Credit Risk – Predictive Modelling

Predictive Modelling - Discrimination

Page 24 Credit Risk – Predictive Modelling

Predictive Modelling - Workflow

• 1) Data exclusions
• 2) Missing values analysis
• 3) Outlier treatment
• 4) Variable transformation (feature engineering)
• 5) Univariate analysis
• 6) Correlation analysis
• 7) Modelling
• Selection of shortlist of variables
• Estimation of coefficients based

Page 25 Credit Risk – Predictive Modelling

Predictive Modelling – Sample definition

• Since we will be using a regressive

approach, we need to keep in mind that we
cannot have dependent observations.
• To avoid this, a cohort approach is used:
• Flexible cohort – fixed number of snapshots
after first observation
• Fixed cohort – fixed snapshot date (e.g. from
every September)
• For our target, we define a “performance”
window – usually 12 months
• No balancing needed ;)
• Unless we’re talking about LDP portfolios

Page 26 Credit Risk – Predictive Modelling

Predictive Modelling – Linear regression

• Problem to be solved: single own house

university degree
29 years

Historical data with already known target value

Name Age Status Education Housing Target
Adam 29 single high school rent 0 i=1 i=2 i=3 i=4 …..
Annie 27 single elementary with parents 1
Jane 31 single high school own house 0 Annie
John 30 married university mortgage 0

We choose linear function

𝑘

𝑓 𝑥Ԧ ≔ 𝛼 + ෍ 𝛽𝑗 𝑥𝑗
𝑗=1 Adam John Jane

Page 27 Credit Risk – Predictive Modelling

Predictive Modelling – Logistic regression

• Problem to be solved: single own house

university degree
29 years

Historical data with already known target value

Name Age Status Education Housing Target
Adam 29 single high school rent 0 i=1 i=2 i=3 i=4 …..
Annie 27 single elementary with parents 1
Jane 31 single high school own house 0 Annie
John 30 married university mortgage 0

We choose logistic function

1
𝑓 𝑥Ԧ ≔ 𝑘
1 + 𝑒 −𝛼−σ𝑗=1 𝛽𝑗𝑥𝑗 Adam John Jane

Page 28 Credit Risk – Predictive Modelling

Predictive Modelling - Logit

• We can choose other functions, but market standard is to use the logit link function

• Using linear function is not proper as it can give estimates above 1 or below 0, which is not
convenient for estimating probability of default

• Selection of the link function if it preserves the output between 0 and 1

• The reason for choosing logit function instead of others is mainly interpretational – the log-
odds ratio defined below is a linear combination of the predictors

𝑃𝐷
𝐿𝑜𝑔 − 𝑜𝑑𝑑𝑠 𝑟𝑎𝑡𝑖𝑜 = ln = 𝑓 −1 𝑃𝐷
1 − 𝑃𝐷

• By central limit theorem under very general conditions the log-odds ratio distribution
converges in distribution to a normal distribution

Page 29 Credit Risk – Predictive Modelling

Predictive Modelling – Prediction

• Let’s say we have processed our data (deduplication, formatting, primary keys, consistency
checks…)
• We could take advantage of models with some sort of elimination
• E.g. – Lasso regression
• Least absolute shrinkage and selection operator
• Performs both variable selection and regularization

• Is this a good model?

Page 30 Credit Risk – Predictive Modelling

Predictive Modelling - Scorecard

• Scorecard points (score)

• The motivation is to derive a scale such that:

• It’s a linear combination of log-odds ratio
• More score points means lower PD
• Double odds ratio corresponds to a prescribed number of score points 𝐴 :

• 𝐵 score points corresponds to a prescribed PD value 𝑥 :

Page 31 Credit Risk – Predictive Modelling

Predictive Modelling - Binning

• Another standardly used technique is binning of predictors and WoE transformation:

𝐺𝑂𝑂𝐷𝑆𝑖Τ𝐵𝐴𝐷𝑆𝑖
• Weight of evidence for i-th bin: 𝑊𝑜𝐸𝑖 : = ln 𝐺𝑂𝑂𝐷𝑆Τ𝐵𝐴𝐷𝑆

BIN GOODS BADS DR WoE

1 [-inf,33) 69 52 0,429752 -0,42156
2 [33,37) 63 45 0,416667 -0,36795
3 [37,40) 72 47 0,394958 -0,27790
4 [40,46) 172 89 0,340996 -0,04556
5 [46,48) 59 25 0,297619 0,15424
6 [48,51) 99 41 0,292857 0,17712
7 [51,58) 157 62 0,283105 0,22469
8 [58,inf) 93 25 0,211864 0,60930
9 MISSING 19 11 0,366667 -0,15787

• Why binning? solves leverage points, solves informative missings, solves non-
numerical (either ordinal or multinomial) variables, assesses robustness
• Why WoE transformation? normalizes predictors values, enables easy interpretation
(under reasonable conditions always attains negative and
positive values, zero value represents portfolio default rate)

Page 32 Credit Risk – Predictive Modelling

Predictive Modelling - WoE

• WoE is the new value of binned predictor

• Coefficient is the estimated parameter from
logistic regression corresponding to the
variable or to the absolute term (intercept)
• In case number in some bin is zero, we need
to compensate: 𝑊𝑜𝐸 = ln
𝐵𝐴𝐷𝑆 + 0.5 Τ 𝐺𝑂𝑂𝐷𝑆 + 0.5
𝑖 𝑖
𝐵𝐴𝐷𝑆Τ𝐺𝑂𝑂𝐷𝑆

• Missing category can be treated

• Scorecard points serve as a standardized
linear transformation of log-odds so that
certain criteria are met – it is motivated
mainly by interpretation
• Coefficients should be negative when using
WoE

Page 33 Credit Risk – Predictive Modelling

Model performance - GINI

• ROC (Receiver Operation Characteristics) curve, GINI

• Measuring discriminatory power – only ordering matters, not the actual score values
True positive rate if we sort the clients increasingly by the score value, the true positive rate can be calculated
for i-th observation as the number of observations with target=1 and index lower or equal to i
divided by the total number of observations with target=1
False positive rate if we sort the clients increasingly by the score value, the false positive rate can be calculated
for i-th observation as the number of observations with target=0 and index lower or equal to i
divided by the total number of observations with target=0

AUC (Area Under Curve) =

Client Event Score 1/3*1/5+2/3*1/5+3/3*3/5=
Annie 1 325 4/5=0.8
Paul 0 398
Lisa 1 415 GINI = 2*(AUC-0.5)=0.6
Jane 0 463
Jack 1 499 • GINI attains values between -1 and 1, but
Adam 0 520 relevant are only values between 0 and 1
John 0 611 • GINI=0 stands for theoretical random
Mary 0 672 model (no predictive power)
• GINI=1 stands for perfectly discriminating
model

Page 34 Credit Risk – Predictive Modelling

Model performance – Somers’ D

• Somers’ D is a measure of association between two variables X and Y calculated as follows:

𝑁𝐶 𝑋, 𝑌 − 𝑁𝐷 𝑋, 𝑌
𝑆𝑜𝑚𝑒𝑟𝑠 ′ 𝐷 𝑋, 𝑌 =
𝑁𝐶 𝑋, 𝑋 − 𝑁𝐷 𝑋, 𝑋

• 𝑁𝐶 𝑋, 𝑌 is the number of concordant pairs 𝑥𝑖 , 𝑦𝑖 and 𝑥𝑗 , 𝑦𝑗 , i.e. either 𝑥𝑖 < 𝑥𝑗 and 𝑦𝑖 < 𝑦𝑗 ,
or 𝑥𝑖 > 𝑥𝑗 and 𝑦𝑖 > 𝑦𝑗 and
• 𝑁𝐷 𝑋, 𝑌 is the number of dis-concordant pairs 𝑥𝑖 , 𝑦𝑖 and 𝑥𝑗 , 𝑦𝑗 , i.e. either 𝑥𝑖 < 𝑥𝑗 and 𝑦𝑖 >
𝑦𝑗 , or 𝑥𝑖 > 𝑥𝑗 and 𝑦𝑖 < 𝑦𝑗 .

• Somers’ D is more general measure than GINI as it can be used on other

than binary targets
• For binary target and no ties in the independent variable it equals GINI
• Clearly: 𝑁𝐷 𝑋, 𝑋 =0

Page 35 Credit Risk – Predictive Modelling

Model performance – Somers’ D

• Illustrative example (X = grade, Y = time spent studying):

Time spend studying
𝑁𝐶 𝑋, 𝑌 =a∙𝑑
Grades Minimal Extensive 𝑁𝐷 𝑋, 𝑌 =𝑏∙𝑐
𝑁𝐶 𝑋, 𝑋 = 𝑎+𝑏 ∙ 𝑐+𝑑
Bad 20 (a) 5 (b)
𝑁𝐷 𝑋, 𝑋 =0
Good 6 (c) 10 (d)

𝑁𝐶 𝑋, 𝑌 − 𝑁𝐷 𝑋, 𝑌
• Using the formula: 𝑆𝑜𝑚𝑒𝑟𝑠 ′ 𝐷 𝑋, 𝑌 =
𝑁𝐶 𝑋, 𝑋 − 𝑁𝐷 𝑋, 𝑋

200 − 30
𝑆𝑜𝑚𝑒𝑟𝑠 ′ 𝐷 𝑋, 𝑌 = = 0.4250 Somers’ D differentiates between
400 − 0
independent and dependent variable
vs.
200 − 30
𝑆𝑜𝑚𝑒𝑟𝑠 ′ 𝐷 𝑌, 𝑋 = = 0.4359
390 − 0

Page 36 Credit Risk – Predictive Modelling

Representativeness/Stability - PSI

• PSI (Population Stability Index) is a measure of difference between two discrete distributions
• It is typically used in order to assess representativity – i.e. assess whether distribution of a
binned variable differs in two different data samples which are typically from two different
time periods (threshold of 0.2 is frequently used)
𝑛
𝐴𝑐𝑡𝑢𝑎𝑙%𝑖
𝑃𝑆𝐼 = ෍ 𝐴𝑐𝑡𝑢𝑎𝑙%𝑖 − 𝐸𝑥𝑝𝑒𝑐𝑡𝑒𝑑%𝑖 ∗ ln
𝐸𝑥𝑝𝑒𝑐𝑡𝑒𝑑%𝑖
𝑖=1
where n is number of bins

Score bands Actual % Expected % Ac-Ex ln(Ac/Ex) Index

< 251 5% 8% -3% -0,470 0,014

251–290 6% 9% -3% -0,410 0,012

291–320 6% 10% -4% -0,510 0,020

321–350 8% 13% -5% -0,490 0,024

351–380 10% 12% -2% -0,180 0,004

381–410 12% 11% 1% 0,090 0,001

411–440 14% 10% 4% 0,340 0,013

441–470 14% 9% 5% 0,440 0,022

471–520 13% 9% 4% 0,370 0,015

520 < 9% 8% 1% 0,120 0,001

Population Stability Index (PSI) = 0,1269

Page 37 Credit Risk – Predictive Modelling