Signi cance testing
of model parameters
GA RCH MODELS IN P YTH ON
Chelsea Yang
Data Science Instructor
Do I need this parameter?
Is it relevant
KISS: keep it simple stupid
Always prefer a parsimonious model
GARCH MODELS IN PYTHON
Hypothesis test
Null hypothesis (H0): a claim to be veri ed
H0: parameter value = 0
If H0 cannot be rejected, leave out the parameter
GARCH MODELS IN PYTHON
Statistical signi cance
Quantify having the observed results by chance
Common threshold: 5%
GARCH MODELS IN PYTHON
P-value
The odds of the observed results could have happened by chance
The lower the p-value, the more ridiculous the null hypothesis looks
Reject the null hypothesis if p-value < signi cance level
GARCH MODELS IN PYTHON
P-value example
print(gm_result.summary()) print(gm_result.pvalues)
GARCH MODELS IN PYTHON
T-statistic
T-statistic = estimated parameter / standard error
The absolute value of the t-statistic is a distance measure
If |t-statistic| > 2: keep the parameter in the GARCH model
GARCH MODELS IN PYTHON
T-statistic example
print(gm_result.summary()) print(gm_result.tvalues)
# Manual calculation
t = gm_result.std_err/gm_result.tvalues
GARCH MODELS IN PYTHON
Let's practice!
GA RCH MODELS IN P YTH ON
Validation of GARCH
model assumptions
GA RCH MODELS IN P YTH ON
Chelsea Yang
Data Science Instructor
Visual check
GARCH MODELS IN PYTHON
Autocorrelation
Describe the correlation of a variable with itself given a time lag
Existence of autocorrelation in the standardized residuals indicates the model may not be sound
To detect autocorrelation:
ACF plot
Ljung-Box
GARCH MODELS IN PYTHON
ACF plot
ACF: AutoCorrelation Function
ACF Plot: visual representation of the autocorrelation by lags
Red area in the plot indicates the con dence level (alpha = 5%)
GARCH MODELS IN PYTHON
ACF plot in Python
from statsmodels.graphics.tsaplots import plot_acf
plot_acf(my_data, alpha = 0.05)
GARCH MODELS IN PYTHON
Ljung-Box test
Test whether any of a group of autocorrelations of a time series are different from zero
H0: the data is independently distributed
P-value < 5%: the model is not sound
GARCH MODELS IN PYTHON
Ljung-Box test Python
# Import the Python module
from statsmodels.stats.diagnostic import acorr_ljungbox
# Perform the Ljung-Box test
lb_test = acorr_ljungbox(std_resid , lags = 10)
# Check p-values
print('P-values are: ', lb_test[1])
GARCH MODELS IN PYTHON
Let's practice!
GA RCH MODELS IN P YTH ON
Goodness of t
measures
GA RCH MODELS IN P YTH ON
Chelsea Yang
Data Science Instructor
Goodness of t
Can model do a good job explaining the data?
1. Maximum likelihood
2. Information criteria
GARCH MODELS IN PYTHON
Maximum likelihood
Maximize the probability of getting the data observed under the assumed model
Prefer models with larger likelihood values
GARCH MODELS IN PYTHON
Log-likelihood in Python
Typically used in log form: log-likelihood
print(gm_result.loglikelihood)
GARCH MODELS IN PYTHON
Over tting
Fit in-sample data well, but perform poorly on out-out-sample predictions
Usually due to the model is overly complex
GARCH MODELS IN PYTHON
Information criteria
Measure the trade-off between goodness of t and model complexity
Likelihood + penalty for model complexity
AIC: Akaike's Information Criterion
BIC: Bayesian Information Criterion
_Prefer models with the lower information criterion score _
GARCH MODELS IN PYTHON
AIC vs. BIC
Generally they agree with each other
BIC penalizes model complexity more severely
GARCH MODELS IN PYTHON
AIC/BIC in Python
print(gm_result.aic)
print(gm_result.bic)
GARCH MODELS IN PYTHON
Let's practice!
GA RCH MODELS IN P YTH ON
GARCH model
backtesting
GA RCH MODELS IN P YTH ON
Chelsea Yang
Data Science Instructor
Backtesting
An approach to evaluate model forecasting capability
Compare the model predictions with the actual historical data
GARCH MODELS IN PYTHON
In-sample vs. out-of-sample
In-sample: model tting
Out-of-sample: backtesting
GARCH MODELS IN PYTHON
MAE
Mean Absolute Error
GARCH MODELS IN PYTHON
MSE
Mean Squared Error
GARCH MODELS IN PYTHON
Calculate MAE, MSE in Python
from sklearn.metrics import mean_absolute_error, mean_squared_error
# Call function to calculate MAE
mae = mean_absolute_error(observation, forecast)
# Call function to calculate MSE
mse = mean_squared_error(observation, forecast)
GARCH MODELS IN PYTHON
Let's practice!
GA RCH MODELS IN P YTH ON
