Scribd
Upload
89 views

Loss Fuction: by Fatema Khairunnasa Lecturer, Dept. of Statistics, Bsmrstu

Loss functions are used to quantify errors in statistical parameter estimation. Common loss functions include mean squared error and mean absolute error, which measure the average squared and absolute differences between estimated and true values. Loss functions are either symmetric, treating overpredictions and underpredictions of the same magnitude equally, or asymmetric, applying different penalties to different error directions. Examples of asymmetric loss functions include check loss and linex loss, which make the optimal prediction dependent on the loss function and forecast error distribution. Symmetric loss functions include mean squared error, mean absolute percentage error, and log hyperbolic cosine. The choice of loss function affects the construction of optimal forecasts.

Uploaded by

Sherena Khatun
Copyright
© © All Rights Reserved
Available Formats
Download as PDF, TXT or read online on Scribd
89 views

Loss Fuction: by Fatema Khairunnasa Lecturer, Dept. of Statistics, Bsmrstu

Loss functions are used to quantify errors in statistical parameter estimation. Common loss functions include mean squared error and mean absolute error, which measure the average squared and absolute differences between estimated and true values. Loss functions are either symmetric, treating overpredictions and underpredictions of the same magnitude equally, or asymmetric, applying different penalties to different error directions. Examples of asymmetric loss functions include check loss and linex loss, which make the optimal prediction dependent on the loss function and forecast error distribution. Symmetric loss functions include mean squared error, mean absolute percentage error, and log hyperbolic cosine. The choice of loss function affects the construction of optimal forecasts.

Uploaded by

Sherena Khatun
Copyright
© © All Rights Reserved
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 7

Loss Fuction

By
Fatema Khairunnasa
Lecturer, Dept. of Statistics,
BSMRSTU
Loss Function
• In statistics, typically a loss function is used for parameter estimation, and the event
in question is some function of the difference between estimated and true values for
an instance of data. The concept, as old as Laplace, was reintroduced in statistics
by Abraham Wald in the middle of the 20th century.
• In the theory of point estimation, a loss function quantifies the losses associated to
the errors committed while estimating a parameter. Often the expected value of the
loss, called statistical risk, is used to compare two or more estimators: in such
comparisons, the estimator having the least expected loss is usually deemed
preferable.
• Definition: Let be an unknown parameter and an estimate of ..The
estimation error is the difference,

• Example: The loss function is a function mapping estimation errors to the set of real
numbers.
 the absolute estimation error which coincides with the absolute value of
the error when the parameter is a scalar;
 the squared estimation error which coincides with the square of the
error when the parameter is a scalar,
 In both cases, the larger the estimation error is, the larger is the loss. The expected
value of the former is called mean absolute error (MAE), while the expectation of the
latter is known as mean squared error (MSE).
Lecture by Fatema Khairunnasa, Lecturer,
BSMRSTU
• Types of loss function
mean_squared_error, mean_absolute_error,
mean_absolute_percentage_error, mean_squared_logarithmic_error,
squared_hinge, Hinge, categorical_hinge, Logcosh,
categorical_crossentropy, sparse_categorical_crossentropy,
binary_crossentropy, kullback_leibler_divergence, Poisson,
cosine_proximity
• Symmetric vs. Asymmetric loss function: Symmetric
functions produce the same loss when underpredicting and
overpredicting of the same absolute error (i.e. amount of loss
same for the positive difference as well as negative diffrence).
However, an asymmetric loss function applies a different penalty
to the different directions of loss (i.e. for the positive difference
the amount of loss is totally different than that of negative
diffrence).

Lecture by Fatema Khairunnasa, Lecturer,


BSMRSTU
Some Symmetric Loss
functions
• The Mean Squared Error (MSE) is the familiar objective function in simple least-
squares regression, a convex function, emphasizing points distant from the bulk of
the data by squaring the error. MSE penalizes larger errors more than smaller error;
the function is steeper in the tails than in the middle.
• The Log Hyperbolic Cosine (LOGCOSH) or log(cosh(x)) is approximately 1/2(x^2) for
small x and abs(x) — log(2) for large x. Therefore, the LOGCOSH works much like the
MSE, but will be less affected by occasional wildly incorrect predictions and in this
regard is like the Mean Absolute Error (MAE) . The LOGCOSH and MAE are useful
when errors in the estimates for larger values do not need to be penalized more by
squaring like in the MSE.
• The Mean Squared Percentage Error (MSPE) is useful in problems where the relative
errors are more of interest (e.g., an error of 10 in 100 is more interesting than 10 in
100,000). However, the MSPE is biased towards lower predictions and is not suited
to problems where the data is skewed positive (e.g., streamflows that are always
>=0).
• There are other symmetric loss functions like:

Lecture by Fatema Khairunnasa, Lecturer,


BSMRSTU
When to use Asymmetric loss
function?
• The most prevalent loss function for the evaluation
of a forecast is the symmetric quadratic function.
Negative and positive forecast errors of the same
magnitude have the same loss. This functional
form is assumed because mathematically it is very
tractable, but from an economic point of view, it is
not very realistic. For a given information set and
under a quadratic loss, the optimal forecast is the
conditional mean of the variable under study. The
choice of the loss function is fundamental to the
construction of an optimal forecast. For
asymmetric loss functions, the optimal forecast can
be more complicated as it will depend not only on
the choice of the loss function but also on the
characteristics of the probability density function
of the forecast error (Granger
Lecture by Fatema 1999).
Khairunnasa, Lecturer,
BSMRSTU
Some Asymmetric loss
functions
There are some commonly used asymmetric loss functions. The
check loss function c (y, f ) ≡ [α – 1 (y < f )] · (y – f ), or c (e ) ≡ [α – 1 (
e < 0)] · e, makes the optimal predictor f the conditional quantile.
The check loss function is also known as the tick function or lil-lin
loss. The asymmetric quadratic loss c (e ) ≡ [α – 1(e < 0)] · e 2 can
also be considered. A value of α = 0.5 gives the symmetric squared
error loss.
A particularly interesting asymmetric loss is the linex function of
Hal Varian (1975), which takes the form
c 1(e, α) = exp (αet+1 1) – αe t + 1 – 1,
where α is a scalar that controls the aversion toward either
positive (α > 0) or negative (α < 0) forecast errors. The linex
function is differentiable. If α > 0, the linex is exponential for e > 0
and linear for e < 0. If α < 0, the linex is exponential for e < 0 and
linear for e > 0. To make the linex more flexible, it can be modified
to the double linex loss function, which is exponential for all
values of e (Granger 1999). When α = β, it becomes the symmetric
double linex loss function.
Lecture by Fatema Khairunnasa, Lecturer,
BSMRSTU
Lecture by Fatema Khairunnasa, Lecturer,
BSMRSTU

You might also like