SCMS 3711

Business Analysis - Statistics

Chapter 8b

Forecast Error Measurements

(Chapter 8.2)

1
 Pause for Thought

We’ve learned a few of the basic

forecasting techniques.

How do we determine which

forecast to use?

2
 Errors
All forecasts have errors
Error is measured in various ways but all are
based on the difference between the actual
value and its forecasted value

Statisticians (mathematicians that deal with

sample data) call errors “residuals” (recall
the study of residuals in Linear Regression

3
 Calculating Error Measurements

Error measurements:
• Can only be calculated using actual data for
the time periods where actual data exists
• Can only be calculated where there is
forecasted values
• Cannot be calculated for the time periods
being forecasted beyond where there are
actual data points
4
 Forecast Accuracy
• Variation is expected, so state what
variations are expected and to what
degree
• Important so that the material, labor and
capital requirements are met when needed
• Variations can be one-time, periodic,
and/or ongoing
• Monitor forecasts for variations and
improve the technique as needed
5
 Forecast Accuracy
Figure 3.11 Conceptual representation of a Control Chart
and Error Measurements

6
Forecast Accuracy

7
Forecast Accuracy

8
 Forecast Errors
Different techniques allow for the
testing of alternative forecasting
techniques

Identify what types of weighting affect

the accuracy

Track the errors over time

9
 Error Analysis
Error = Actual – Forecast
Graph the errors to generate potential visual
clues
Goal: If we can identify the cause of errors and
correct for them, the forecast is better – not
perfect, but better
Potential issues:
– Must reflect on the causes of errors, not just use
some formula
– Past trends may not be current trends
10
 Interpretation of Error
• Bias (overestimate or underestimate)
• Variability in errors (MSE, MAD)
• Relative size of errors (MAPE)

11
 Error and Bias

The Average of Errors is called Bias

12
 Forecast Errors
MAD: Mean Absolute Deviation

All errors have the same weight

Bias does not use absolute value;

MAD does use absolute value

13
 Forecast Errors
MSE: Mean Squared Error

Errors are weighted based on their

squared values

Some textbooks use n,

This textbook uses n-1
14
 Forecast Errors
MAPE: Mean Absolute
Percentage Error

Errors are given weight based on their

relative error

15
Forecast Errors: Example 1

16
 Forecast Errors: Example 1
Using the previous slide’s data:

17
Forecast Errors: Example 2
Sales Forecast Error MAD MSE MAPE
Jan-20 4,885
Feb-20 4,904
Mar-20 5,104
Apr-20 5,082 5,022
May-20 5,190 5,071
Jun-20 5,223 5,149
Jul-20 5,003 5,199
Aug-20 5,960 5,088
Sep-20 6,620 5,599
First, we would determine
Oct-20 6,101 6,260
Nov-20 6,455 6,243
what method to use.
Dec-20 6,998 6,365
Jan-21 In this example, we are
Feb-21
Mar-21
using a 3 month moving
Apr-21 average
May-21
Jun-21
Jul-21
Aug-21
Sep-21
Oct-21
Nov-21
Dec-21

18
Forecast Errors: Example 2
Sales Forecast Error MAD MSE MAPE
Jan-20 4,885
Feb-20 4,904
Mar-20 5,104
Apr-20 5,082 5,022 60 60 3,588 1.179
May-20 5,190 5,071 119 119 14,209 2.297
Jun-20 5,223 5,149 74 74 5,476 1.417
Jul-20 5,003 5,199 (196) 196 38,416 3.918
Aug-20 5,960 5,088 872 872 760,907 14.636
Sep-20 6,620 5,599 1,021 1,021 1,042,033 15.420
Oct-20 6,101 6,260 (159) 159 25,376 2.611
Nov-20 6,455 6,243 212 212 45,114 3.290
Dec-20 6,998 6,365 633 633 400,309 9.041
Jan-21
Feb-21
Mar-21
Next, we calculate the
Apr-21 Error Measurements.
May-21 This is actually a part of
Jun-21
Jul-21
determining which
Aug-21 forecast method to use.
Sep-21
Oct-21
These are the
Nov-21
Dec-21 intermediate calculations.
371.8 291,928.5 5.98
19
Forecast Errors: Example 2
Sales Forecast Error MAD MSE MAPE
Jan-20 4,885
Feb-20 4,904
Mar-20 5,104
Apr-20 5,082 5,022 60 60 3,588 1.179
May-20 5,190 5,071 119 119 14,209 2.297
Jun-20 5,223 5,149 74 74 5,476 1.417
Jul-20 5,003 5,199 (196) 196 38,416 3.918
Aug-20 5,960 5,088 872 872 760,907 14.636
Sep-20 6,620 5,599 1,021 1,021 1,042,033 15.420
Oct-20 6,101 6,260 (159) 159 25,376 2.611
Nov-20 6,455 6,243 212 212 45,114 3.290
Dec-20 6,998 6,365 633 633 400,309 9.041
Jan-21
Feb-21
Mar-21
Apr-21
May-21
Jun-21
Jul-21 Here are the final Error
Aug-21 Measurement values.
Sep-21
Oct-21
Nov-21
Dec-21
371.8 291,928.5 5.98
20
 Forecast Errors: Example 3

Compare the error performance of these three

forecasting techniques using MAD, MSE
and MAPE:

• Naïve
• Two-period moving average, and
• Exponential smoothing with α = .10
for periods 3 through 11

21
Forecast Errors: Example 3

22
Forecast Errors: Example 3

23
Forecast Errors: Example 3
α = 0.10

24
 Common Mistake

There is a common error when calculating the Error

Measurements in Excel. The numerator of each
calculation does not divide by "n" or "n-1".

As each formula shows, first calculate the various

error terms (absolute value, the error squared, or
the relative error).

After those are calculated - only then do you add

them up and divide by "n" or "n-1".

25
 Error Measurements: Why?
Error measurements can only be calculated
when we have actual (historical) data,
as shown by the formula itself.

We determine forecast errors based on

historical data to select a technique, then
use that forecasting technique to generate a
forecast of future events assuming that this
technique will continue to provide a forecast
with the least error.
26
 Pause for Thought

We reviewed a few of the basic

techniques and introduced ideas for
more advanced techniques.

How does a company ever get

reasonable forecasts to be able to
plan the work to be done?
27
 Choosing a Technique
Factors: Cost (and time), Accuracy, and
Availability of data

Removing a pattern (mathematically) often

results in less variability of the
forecast errors

28
Forecasting Factors

29
Choosing a Technique

30
 Putting it All Together
We forecast to determine what could happen;
we measure errors to determine how
good or bad the forecast was
To chose which forecast to use:
1. Perform multiple forecasts on the same data
2. Calculate various error measurements for
each forecast
3. Select the forecast with the least overall error
across the various error measurements
31
 Putting it All Together:
Example
If we generate 7 forecasts for our data and
use all 4 error measurement tools
(Bias, MAD, MSE, and MAPE)
We will have 28 sets of error measurements:
4 for each of the 7 forecasts
The forecast to use is the one with the
smallest general error measurement,
which is a matter of interpretation vs.
strict mathematics
32
 The Best Forecast:
Occam’s Razor

The simplest
explanation is
considered to be the
better explanation

See ClubStreetPost.com
33
 Non-SCM Careers
• Econ / Fin / Acct / Mktg / Mgmt: Operations
use many types of forecasts that you don’t
typically perform to drive critical decisions
• BIT: Project estimating (forecasting) is
particularly treacherous because of the
variability of resources
• HR: These terms (like most) help segregate
candidates who know the terms vs. can
apply the concepts
34
 Technology

• Forecast management (techniques, error

measurement, etc.)
• Forecast tools
(SAS / JMP, SPSS, MatLab, R, etc.)

35
 Summary

• Qualitative and Quantitative methods

• Combining techniques
• Selecting a technique
• Forecast accuracy

36
 Learning

What did I learn?

What do I not understand?
What applies to my career?

37
Questions?

38