Scribd
Upload
43 views93 pages

Forecasting: JY Le Boudec

This document discusses various techniques for forecasting including linear regression, avoiding overfitting, differencing data to make it stationary, ARMA models, and sparse ARMA models. It provides examples of fitting ARMA models which involve differencing data until it is stationary, examining the autocorrelation function to select model orders, maximizing the likelihood to estimate parameters, and computing forecasts and prediction intervals. Case studies on traffic data and electricity prices are presented to illustrate the techniques.

Uploaded by

Akash Jain
Copyright
© Attribution Non-Commercial (BY-NC)
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PPT, PDF, TXT or read online on Scribd
43 views93 pages

Forecasting: JY Le Boudec

This document discusses various techniques for forecasting including linear regression, avoiding overfitting, differencing data to make it stationary, ARMA models, and sparse ARMA models. It provides examples of fitting ARMA models which involve differencing data until it is stationary, examining the autocorrelation function to select model orders, maximizing the likelihood to estimate parameters, and computing forecasts and prediction intervals. Case studies on traffic data and electricity prices are presented to illustrate the techniques.

Uploaded by

Akash Jain
Copyright
© Attribution Non-Commercial (BY-NC)
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PPT, PDF, TXT or read online on Scribd
You are on page 1/ 93

Forecasting

JY Le Boudec

Contents
1. What is forecasting ? 2. Linear Regression 3. Avoiding Overfitting 4. Differencing 5. ARMA models 6. Sparse ARMA models 7. Case Studies

1. What is forecasting ?
Assume you have been able to define the nature of the load It remains to have an idea about its intensity It is impossible to forecast without error The good engineer should
Forecast what can be forecast
Give uncertainty intervals

The rest is outside our control

2. Linear Regression
Simple, for simple cases Based on extrapolating the explanatory variables

Estimation and Forecasting


In practice we estimate from y, , yt When computing the forecast, we pretend is known, and thus make an estimation error It is hoped that the estimation error is much less than the confidence interval for forecast In the case of linear regression, the theorem gives the global error exactly
In general, we wont have this luxury

10

We saw this already


A case where estimation error versus prediction uncertainty can be quantified

Prediction interval if model is known Prediction interval accounting for estimation (t = 100 observed points)
11

3. The Overfitting Problem


The best model is not necessarily the one that fits best

12

Prediction for the better model

This is the overfitting problem


13

How to avoid overfitting


Method 1: use of test data Method 2: information criterion

14

15

16

Best Model for Internet Data, polynomial of degree up to 2

17

d=1

18

Best Model for Internet Data, polynomial of degree up to 10

19

4. Differencing the Data

20

21

22

Point Predictions from Differenced Data

23

Background On Filters (Appendix B)


We need to understand how to use discrete filters.

Example: write the Matlab command for

24

25

A simple filter

Q: compute X back from Y

26

27

28

29

Impulse Response

30

31

32

A filter with stable inverse

33

How is this prediction done ?

This is all very intuitive

34

35

Prediction assuming differenced data is iid

36

Prediction Intervals
A prediction without prediction intervals is only a small part of the story
The financial crisis might have been avoided if investors had been aware of prediction intervals

37

38

39

Compare the Two

Linear Regression with 3 parameters + variance

Assuming differenced data is iid

40

41

5. Using ARMA Models


When the differenced data appears stationary but not iid

42

Test of iid-ness

43

44

ARMA Process

45

46

ARMA Processes are Gaussian (non iid)

47

48

49

ARIMA Process

50

Fitting an ARMA Process


Called the Box-Jenkins method Difference the data until stationary Examine ACF to get a feeling of order (p,q) Fit an ARMA model using maximum likelihood

51

Fitting an ARIMA Model


Apply Scientific Method 1. make stationary and normal (how ?) 2. bound orders p,q 3. fit an ARMA model to Yt - i.e. Yt - ARMA 4. compute residuals and verify white noise and normal

Fitting an ARMA model Pb is : given orders p,q given (x1, xn) (transformed data) compute the parameters of an ARMA (p,q) model that maximizes the likelihood
Q:What are the parameters ? A: the mean , the polynomial coefficients k and k , the noise variance 2
52

This is a non-linear optimization problem


Maximizing the likelihood is a non-linear optimization problems

Usually solved by iterative, heuristic algorithms, may converge to a local maximum may not converge

Some simple, non MLE, heuristics exist for AR or MA models


Ex: fit the AR model that has the same theoretical ACF as the sample ACF

Common practice is to bootstrap the optimization procedure by starting with a best guess
AR or MA fit, using heuristic above

53

Fitting ARMA Model is Same as Minimizing One-Step ahead prediction error

54

Best Model Order

55

Check the Residuals

56

Example

57

58

Forecasting with ARMA


Assume Yt is fitted to an ARMA process

The prediction problem is: given Y1=y1,,Yt=yt find the conditional distribution of Yt+h

We know it is normal, with a mean that depends on (y1,,yt) and a variance that depends only on the fitted parameters of the ARMA process There are many ways to compute this; it is readily done by Matlab

59

60

61

Forecasting Formulae for ARIMA


Y = original data X = differenced data, fitted to an ARMA model
1. 2. 3. Obtain point prediction for X using what we just saw Apply Proposition 6.4.1 to obtain point prediction for Y Apply formula for prediction interval

There are several other methods, but they may have numerical problems. See comments in the lecture notes after prop 6.5.2

62

63

Improve Confidence Interval If Residuals are not Gaussian (but appear to be iid)
Assume residuals are not gaussian but are iid How can we get confidence intervals ?

Bootstrap by sampling from residuals

64

65

With gaussian assumption

With bootstrap from residuals

66

6. Sparse ARMA Models


Problem: avoid many parameters when the degree of the A and C polynomials is high, as in the previous example Based on heuristics
Multiplicative ARIMA, constrained ARIMA Holt Winters

67

68

Holt Winters Model 1: EWMA

69

70

EWMA is OK when there is no trend and no periodicity

71

72

73

74

75

76

77

Constrained ARIMA

Sparse models give less accurate predictions but have much fewer parameters and are simple to fit.
(corrected or not)
78

7. Case Studies

79

80

81

82

83

84

85

86

87

h=1

88

89

h=2

90

91

log h=1

92

Conclusion
Forecasting is useful when savings matter; for example
Save money on server space rental Save energy

Capturing determinism is perhaps most important and easiest Prediction intervals are useful to avoid gross mistakes Re-scaling the data may help

vous de jouer.

93

You might also like