Scribd
Upload
5 views2 pages

Dapr2 Equation Sheet

The document outlines various regression models including simple, multiple, and interaction models, detailing their equations and components. It also discusses methods for calculating coefficients, model evaluation metrics like R² and AIC/BIC, and techniques for handling categorical predictors. Additionally, it covers logistic regression concepts, including probability, odds, and testing coefficients.

Uploaded by

Alex Jeffery
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
5 views2 pages

Dapr2 Equation Sheet

The document outlines various regression models including simple, multiple, and interaction models, detailing their equations and components. It also discusses methods for calculating coefficients, model evaluation metrics like R² and AIC/BIC, and techniques for handling categorical predictors. Additionally, it covers logistic regression concepts, including probability, odds, and testing coefficients.

Uploaded by

Alex Jeffery
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 2

General Model Equations NOTES

Simple regression model


𝑦𝑖 = 𝛽0 + 𝛽1 𝑥1𝑖 + 𝜖𝑖
where 𝜖 ∼ 𝑁 (0, 𝜎) independently
Multiple regression model
𝑦𝑖 = 𝛽0 + 𝛽1 𝑥1𝑖 + 𝛽2 𝑥2𝑖 + 𝜖𝑖
where 𝜖 ∼ 𝑁 (0, 𝜎) independently
Interaction model
𝑦𝑖 = 𝛽0 + 𝛽1 𝑥1𝑖 + 𝛽2 𝑥2𝑖 + 𝛽3 (𝑥1𝑖 ⋅ 𝑥2𝑖 ) + 𝜖𝑖
where 𝜖 ∼ 𝑁 (0, 𝜎)independently
Prediction Equation (2 predictor example)
𝑦𝑖̂ = 𝛽0 + 𝛽1 𝑥1𝑖 + 𝛽2 𝑥2𝑖
Sigma
𝑛
∑𝑖=1 (𝑦𝑖 −𝑦𝑖̂ )2
𝜎̂ = √ 𝑆𝑆𝑛−𝑘−1
𝑅𝑒𝑠𝑖𝑑𝑢𝑎𝑙
=√ 𝑛−𝑘−1

Calculating Coefficients
Intercept
𝛽0̂ = 𝑦 ̄ − 𝛽1̂ 𝑥̄
Slope
𝑆𝑃
𝛽1̂ = 𝑆𝑆𝑥𝑦
𝑥
Where 𝑆𝑃𝑥𝑦 = sum of cross-products:
𝑛
𝑆𝑃𝑥𝑦 = ∑𝑖=1 (𝑥𝑖 − 𝑥)(𝑦
̄ 𝑖 − 𝑦)̄
and 𝑆𝑆𝑥 = sums of squared deviations of 𝑥:
𝑛
𝑆𝑆𝑥 = ∑𝑖=1 (𝑥𝑖 − 𝑥)̄ 2
standardized coefficient
𝛽1∗̂ = 𝛽1̂ 𝑠𝑑
𝑠𝑑
𝑥
𝑦

SE( 𝛽1̂ )
𝑆𝐸(𝛽𝑗̂ ) = √ 𝑆𝑆 𝑅𝑒𝑠𝑖𝑑𝑢𝑎𝑙 /(𝑛−𝑘−1)
∑(𝑥 −𝑥 ̄ )2 (1−𝑅2 )
𝑖𝑗 𝑗 𝑥𝑗

𝑡-value for coefficients


𝛽𝑗̂ −0
𝑡 = 𝑆𝐸( 𝛽̂ )𝑗
follows a 𝑡-distribution with 𝑛 − 𝑘 − 1 degrees
of freedom.
confidence interval
𝛽𝑗̂ ± 𝑡∗ ⋅ 𝑆𝐸(𝛽𝑗̂ )
where 𝑡∗ denotes the critical value chosen
from t-distribution with 𝑛 − 𝑘 − 1 degrees of
freedom for a desired 𝛼 level of confidence.

Model evaluation & Comparison


Sums of squares
𝑆𝑆𝑇 𝑜𝑡𝑎𝑙 = 𝑆𝑆𝑀𝑜𝑑𝑒𝑙 + 𝑆𝑆𝑅𝑒𝑠𝑖𝑑𝑢𝑎𝑙
𝑛
𝑆𝑆𝑇 𝑜𝑡𝑎𝑙 = ∑𝑖=1 (𝑦𝑖 − 𝑦)̄ 2
𝑛
𝑆𝑆𝑅𝑒𝑠𝑖𝑑𝑢𝑎𝑙 = ∑𝑖=1 (𝑦𝑖 − 𝑦𝑖̂ )2
𝑛
𝑆𝑆𝑀𝑜𝑑𝑒𝑙 = ∑𝑖=1 (𝑦𝑖̂ − 𝑦)̄ 2
𝑅2 & Adjusted 𝑅2
𝑆𝑆𝑅𝑒𝑠𝑖𝑑𝑢𝑎𝑙
𝑅2 = 𝑆𝑆
𝑆𝑆𝑇 𝑜𝑡𝑎𝑙 = 1 − 𝑆𝑆𝑇 𝑜𝑡𝑎𝑙
𝑀𝑜𝑑𝑒𝑙

2
𝑅̂ 2 = 1 − (1−𝑅 )(𝑛−1)
𝑎𝑑𝑗𝑢𝑠𝑡𝑒𝑑 𝑛−𝑘−1

F
𝑀𝑆𝑀𝑜𝑑𝑒𝑙 𝑆𝑆𝑀𝑜𝑑𝑒𝑙 /𝑑𝑓𝑀𝑜𝑑𝑒𝑙
𝐹 (𝑑𝑓𝑚𝑜𝑑𝑒𝑙 , 𝑑𝑓𝑟𝑒𝑠𝑖𝑑𝑢𝑎𝑙 ) = 𝑀𝑆𝑅𝑒𝑠𝑖𝑑𝑢𝑎𝑙 = 𝑆𝑆𝑅𝑒𝑠𝑖𝑑𝑢𝑎𝑙 /𝑑𝑓𝑅𝑒𝑠𝑖𝑑𝑢𝑎𝑙

Incremental F
(𝑆𝑆𝑅𝑅 −𝑆𝑆𝑅𝐹 )/(𝑑𝑓𝑅 −𝑑𝑓𝐹 )
𝐹(𝑑𝑓𝑅 −𝑑𝑓𝐹 ),𝑑𝑓𝐹 = 𝑆𝑆𝑅𝐹 /𝑑𝑓𝐹

AIC & BIC


𝐴𝐼𝐶 = 𝑛 ln ( 𝑆𝑆𝑟𝑒𝑠𝑖𝑑𝑢𝑎𝑙
𝑛 ) + 2𝑘
𝐵𝐼𝐶 = 𝑛 ln ( 𝑆𝑆𝑟𝑒𝑠𝑖𝑑𝑢𝑎𝑙
𝑛 ) + 𝑘 ln(𝑛)
Categorical predictors NOTES
Dummy coding
𝑦𝑖𝑗 = 𝜇𝑖 + 𝜖𝑖𝑗
𝑦𝑖𝑗 = 𝛽0 + (𝜇
⏟ 𝑖 − 𝛽0 ) +𝜖𝑖𝑗
𝛽𝑖
𝜇𝑖 = 𝛽0 + 𝛽𝑖
Effects coding
𝑦𝑖𝑗 = 𝜇 + 𝛽𝑗 + 𝜖𝑖𝑗
𝛽𝑗 = 𝜇 𝑗 − 𝜇
𝜇1 = 𝛽0 + 𝛽1
For the dropped level:
𝜇𝑗 = 𝛽0 − (𝛽1 + 𝛽2 )
Testing Orthogonal comparisons
∑ 𝑐1𝑗 𝑐2𝑗 = 0

Probing Interactions
Simple slopes
𝑦 ̂ = (𝛽1 + 𝛽3 𝑧)𝑥 + (𝛽2 𝑧 + 𝛽0 )
Locating a crossing point
𝑥1 = −𝛽
𝛽
2
𝑥2 = −𝛽1
𝛽3
3

Model Diagnostics
Partial residuals
𝜖𝑖 + 𝐵𝑗 𝑋𝑖𝑗
Hat values
(𝑥𝑖 −𝑥)̄ 2
ℎ𝑖 = 𝑛1 + 𝑛
∑𝑖=1 (𝑥𝑖 −𝑥)̄ 2

Cooks Distance 2
𝐷𝑖 = (StandardizedResidual
𝑘+1
𝑖)
× ℎ𝑖
1−ℎ𝑖
2
Where (StandardizedResidual
𝑘+1
𝑖)
= Outlyingness
ℎ𝑖
and 1−ℎ = Leverage
𝑖
So 𝐷𝑖 = Outlyingness × Leverage
Variance Inflation Factor
1
𝑉 𝐼𝐹𝑗 = 1−𝑅 2
𝑗

Multiple Comparisons
Family-wise Error Rate
𝑃 (Making a Type I error in m tests) = 1 − (1 − 𝛼)𝑚
Bonferroni & Sidak
𝛼
𝛼𝐵𝑜𝑛𝑓𝑒𝑟𝑟𝑜𝑛𝑖 = 𝑚 𝑝𝐵𝑜𝑛𝑓𝑒𝑟𝑟𝑜𝑛𝑖 = 𝑝 ∗ 𝑚
1
𝛼𝑆𝑖𝑑𝑎𝑘 = 1 − (1 − 𝛼) 𝑚
Scheffe
√𝑟𝐹 (𝛼; 𝑟; 𝑑)

Logistic regression
Probability, odds, log odds
Probability = 𝑃 (𝑌𝑖 )
𝑃 (𝑌 =1) 𝑃 (𝑌 =1)
odds = 1−𝑃 (𝑌 =1) logodds = 𝑙𝑛 ( 1−𝑃 (𝑌 =1) )

Logistic Regression Model


𝑃 (𝑦𝑖 ) = 1+𝑒−(𝛽0 +𝛽1 𝑥11+𝛽2 𝑥2 ...+𝛽𝑘 𝑥𝑘 )
Or
𝑃 (𝑌 =1)
𝑙𝑛 ( 1−𝑃 (𝑌 =1) ) = 𝛽0 + 𝛽1 𝑥1 + 𝛽2 𝑥2 ... + 𝛽𝑘 𝑥𝑘

Deviance
𝑑𝑒𝑣𝑖𝑎𝑛𝑐𝑒 = −2 ∗ 𝑙𝑜𝑔𝑙𝑖𝑘𝑒𝑙𝑖ℎ𝑜𝑜𝑑
Testing logistic regression coefficients
𝑏
𝑧 = 𝑆𝐸(𝑏)

You might also like