Motivating Examples

Basic Probability Theory

Central Limit Theorem and Stable Distributions
Examples
Anomalous Behavior of Heavy-Tailed Random Variables
Concluding Remarks

Predicting Extreme Events in

Finance and Weather:
Use of Heavy-Tailed Distributions

M. Vidyasagar FRS

Cecil & Ida Green Chair

The University of Texas at Dallas
[email protected]
www.utdallas.edu/∼m.vidyasagar

IEEE Philadelphia Section, 28 April 2014

M. Vidyasagar FRS Predicting Extreme Events
 Motivating Examples
Basic Probability Theory
Central Limit Theorem and Stable Distributions
Examples
Anomalous Behavior of Heavy-Tailed Random Variables
Concluding Remarks

Outline
1 Motivating Examples
2 Basic Probability Theory
Probability Distribution and Density Functions
Two Standard Distributions
3 Central Limit Theorem and Stable Distributions
Central Limit Theorem
Stable Distributions
4 Examples
Using Common Sense
Using Stable Distributions
5 Anomalous Behavior of Heavy-Tailed Random Variables
6 Concluding Remarks
M. Vidyasagar FRS Predicting Extreme Events
 Motivating Examples
Basic Probability Theory
Central Limit Theorem and Stable Distributions
Examples
Anomalous Behavior of Heavy-Tailed Random Variables
Concluding Remarks

Outline
1 Motivating Examples
2 Basic Probability Theory
Probability Distribution and Density Functions
Two Standard Distributions
3 Central Limit Theorem and Stable Distributions
Central Limit Theorem
Stable Distributions
4 Examples
Using Common Sense
Using Stable Distributions
5 Anomalous Behavior of Heavy-Tailed Random Variables
6 Concluding Remarks
M. Vidyasagar FRS Predicting Extreme Events
 Motivating Examples
Basic Probability Theory
Central Limit Theorem and Stable Distributions
Examples
Anomalous Behavior of Heavy-Tailed Random Variables
Concluding Remarks

Motivation

There are many applications where underestimating the probability

of extreme events is catastrophic. Examples: Weather, finance,
insurance, Internet traffic.
Not all events are amenable to mathematical modeling, e.g. fire in
factory of your main supplier, political instability etc. Our focus is
on only those events that can be modeled and analyzed using the
methods of probability theory.
Why probability theory? Because it takes “guess work” and
arbitrariness out of the analysis.

M. Vidyasagar FRS Predicting Extreme Events

 Motivating Examples
Basic Probability Theory
Central Limit Theorem and Stable Distributions
Examples
Anomalous Behavior of Heavy-Tailed Random Variables
Concluding Remarks

Some Aspects of Predicting Extreme Events

How can one extrapolate far beyond the range of the observed
data, or with inadequate data, to get a realistic estimate of
the probability of extreme events?
Just how rare are ‘rare events’ ? (Answer: Not so rare as one
may suppose!)
What kinds of mathematics is needed to study these topics?

M. Vidyasagar FRS Predicting Extreme Events

 Motivating Examples
Basic Probability Theory
Central Limit Theorem and Stable Distributions
Examples
Anomalous Behavior of Heavy-Tailed Random Variables
Concluding Remarks

Specific Example: Predicting Extreme Rainfall

Challenge: You are asked to design a retaining wall that can

withstand excessive rainfall. The probability of failure (rain
overwhelming the retaining wall) cannot exceed 0.001, or 1 in
thousand.
Difficulty: You have data for only 2,000 days, out of which it did
not rain on 1,500 days!
With data for only 500 rainy days, how can you extrapolate to an
event that is likey to happen only once every 1,000 days?

M. Vidyasagar FRS Predicting Extreme Events

 Motivating Examples
Basic Probability Theory
Central Limit Theorem and Stable Distributions
Examples
Anomalous Behavior of Heavy-Tailed Random Variables
Concluding Remarks

Specific Example: Predicting Stock Prices

Challenge: You are asked to predict the “value at risk” of GE

stock tomorrow, that is, the cutoff threshold such that the
likelihood of the price exceeding that threshold is less than 0.05 –
the so-called 95% VaR.
Difficulty: No shortage of data! You have data stretching back for
years! But for predicting short-term fluctuations, all that data is
totally useless! At best you can use one month’s data (30 data
points) to make a prediction.
How can you make such a prediction using just 30 values?
Same logic applies also to predicting foreign exchange rates.

M. Vidyasagar FRS Predicting Extreme Events

 Motivating Examples
Basic Probability Theory
Central Limit Theorem and Stable Distributions
Examples
Anomalous Behavior of Heavy-Tailed Random Variables
Concluding Remarks

Extrapolating Far Beyond Observed Range

Approach: Choose a “candidate distribution” and adjust

parameters to get the best fit. Natural impulse: Use the Gaussian
distribution.
Theme of this talk:

In several application areas such as finance and weather, other

distributions, known as “heavy-tailed distributions,” fit the
data better.
With heavy-tailed distributions, extreme events are far less
rare and far more “bursty” than with Gaussian distributions.
Actual observations in finance and weather seem to justify the
use of heavy-tailed distributions.

M. Vidyasagar FRS Predicting Extreme Events

 Motivating Examples
Basic Probability Theory
Central Limit Theorem and Stable Distributions Probability Distribution and Density Functions
Examples Two Standard Distributions
Anomalous Behavior of Heavy-Tailed Random Variables
Concluding Remarks

Outline
1 Motivating Examples
2 Basic Probability Theory
Probability Distribution and Density Functions
Two Standard Distributions
3 Central Limit Theorem and Stable Distributions
Central Limit Theorem
Stable Distributions
4 Examples
Using Common Sense
Using Stable Distributions
5 Anomalous Behavior of Heavy-Tailed Random Variables
6 Concluding Remarks
M. Vidyasagar FRS Predicting Extreme Events
 Motivating Examples
Basic Probability Theory
Central Limit Theorem and Stable Distributions Probability Distribution and Density Functions
Examples Two Standard Distributions
Anomalous Behavior of Heavy-Tailed Random Variables
Concluding Remarks

Outline
1 Motivating Examples
2 Basic Probability Theory
Probability Distribution and Density Functions
Two Standard Distributions
3 Central Limit Theorem and Stable Distributions
Central Limit Theorem
Stable Distributions
4 Examples
Using Common Sense
Using Stable Distributions
5 Anomalous Behavior of Heavy-Tailed Random Variables
6 Concluding Remarks
M. Vidyasagar FRS Predicting Extreme Events
 Motivating Examples
Basic Probability Theory
Central Limit Theorem and Stable Distributions Probability Distribution and Density Functions
Examples Two Standard Distributions
Anomalous Behavior of Heavy-Tailed Random Variables
Concluding Remarks

Probability Distribution Functions on R

Suppose X is a real-valued random variable (examples: daily

rainfall, daily market return). The probability distribution
function (pdf) of X is denoted by ΦX and is defined as

ΦX (a) := Pr{X ≤ a}.

In words, ΦX (a) is the probability that X is less than or equal to a.

Clearly, if a < b then ΦX (a) ≤ ΦX (b). So the pdf is nondecreasing.
Moreover, ΦX (a) → 0 as a → −∞ and ΦX (a) → 1 as a → +∞.

M. Vidyasagar FRS Predicting Extreme Events

 Motivating Examples
Basic Probability Theory
Central Limit Theorem and Stable Distributions Probability Distribution and Density Functions
Examples Two Standard Distributions
Anomalous Behavior of Heavy-Tailed Random Variables
Concluding Remarks

Depiction of a Probability Distribution Function

ΦX (a)

a
−4 −2 −1 1.5 4

M. Vidyasagar FRS Predicting Extreme Events

 Motivating Examples
Basic Probability Theory
Central Limit Theorem and Stable Distributions Probability Distribution and Density Functions
Examples Two Standard Distributions
Anomalous Behavior of Heavy-Tailed Random Variables
Concluding Remarks

Complementary Distribution Function

In analyzing rare events, we often make use of the

complementary Distribution Function Φ̄X defined by

Φ̄X (a) = 1 − ΦX (a) = Pr{X > a}.

Properties of Φ̄X :

If a < b, then Φ̄X (a) ≥ Φ̄X (b); i.e., Φ̄X is non-increasing.

Φ̄X (a) → 1 as a → −∞, and Φ̄X (a) → 0 as a → +∞.

M. Vidyasagar FRS Predicting Extreme Events

 Motivating Examples
Basic Probability Theory
Central Limit Theorem and Stable Distributions Probability Distribution and Density Functions
Examples Two Standard Distributions
Anomalous Behavior of Heavy-Tailed Random Variables
Concluding Remarks

Depiction of Complementary Distribution Function

ΦX (a),Φ̄X (a)

a
−4 −2 −1 1.5 4

Φ̄ is used to analyze the “tail” of a r.v.

M. Vidyasagar FRS Predicting Extreme Events

 Motivating Examples
Basic Probability Theory
Central Limit Theorem and Stable Distributions Probability Distribution and Density Functions
Examples Two Standard Distributions
Anomalous Behavior of Heavy-Tailed Random Variables
Concluding Remarks

Probability Density Functions

If the distribution function Φ is continuously differentiable, then its

derivative φ : R → R+ is called the probability density function.
In this case we have
Z a
Φ(a) = φ(x)dx,
−∞
Z ∞
Φ̄(a) = φ(x)dx.
a

M. Vidyasagar FRS Predicting Extreme Events

 Motivating Examples
Basic Probability Theory
Central Limit Theorem and Stable Distributions Probability Distribution and Density Functions
Examples Two Standard Distributions
Anomalous Behavior of Heavy-Tailed Random Variables
Concluding Remarks

Mean and Variance

For a random variable X, we define its mean, variance, and

standard deviation as
Z ∞
E(X) = xφ(x)dx,
−∞
Z ∞
V (X) = (x − E(X))2 φ(x)dx,
−∞
p
σ(X) = V (X),
assuming the various integrals exist (not always guaranteed).

M. Vidyasagar FRS Predicting Extreme Events

 Motivating Examples
Basic Probability Theory
Central Limit Theorem and Stable Distributions Probability Distribution and Density Functions
Examples Two Standard Distributions
Anomalous Behavior of Heavy-Tailed Random Variables
Concluding Remarks

Outline
1 Motivating Examples
2 Basic Probability Theory
Probability Distribution and Density Functions
Two Standard Distributions
3 Central Limit Theorem and Stable Distributions
Central Limit Theorem
Stable Distributions
4 Examples
Using Common Sense
Using Stable Distributions
5 Anomalous Behavior of Heavy-Tailed Random Variables
6 Concluding Remarks
M. Vidyasagar FRS Predicting Extreme Events
 Motivating Examples
Basic Probability Theory
Central Limit Theorem and Stable Distributions Probability Distribution and Density Functions
Examples Two Standard Distributions
Anomalous Behavior of Heavy-Tailed Random Variables
Concluding Remarks

Normal (Gaussian) Density Function (Bell Curve)

The formula for the so-called ‘normal’ Gaussian density function is

1
φN (x) = √ exp(−x2 /2).
2π
M. Vidyasagar FRS Predicting Extreme Events
 Motivating Examples
Basic Probability Theory
Central Limit Theorem and Stable Distributions Probability Distribution and Density Functions
Examples Two Standard Distributions
Anomalous Behavior of Heavy-Tailed Random Variables
Concluding Remarks

Pareto Distribution

The Pareto distribution was invented by Vilfredo Pareto to model

the distribution of wealth in Italy. It is described by

φ(x) = 0, x ≤ 1, φ(x) = αx−(α+1) , x > 1,

where α > 0 is a parameter. Then

Φ(a) = 1 − a−α , Φ̄(a) = a−α , a ≥ 1.

We can of course “scale” and “translate” the distribution.

The Pareto distribution has finite mean only if α > 1, and has
finite variance only if α > 2.

M. Vidyasagar FRS Predicting Extreme Events

 Motivating Examples
Basic Probability Theory
Central Limit Theorem and Stable Distributions Probability Distribution and Density Functions
Examples Two Standard Distributions
Anomalous Behavior of Heavy-Tailed Random Variables
Concluding Remarks

Depiction of the Pareto Distribution

Note: Their k is our α.

M. Vidyasagar FRS Predicting Extreme Events
 Motivating Examples
Basic Probability Theory
Central Limit Theorem and Stable Distributions Probability Distribution and Density Functions
Examples Two Standard Distributions
Anomalous Behavior of Heavy-Tailed Random Variables
Concluding Remarks

Conditional PDFs and Densities: Motivation

Example: The life expectancy of an American male at birth is

75.38 years. So if a man is now 65 years old, is his life expectancy
10.38 years?
No! – it is a longer!
Reason: The life expectancy calculation of 75.38 years includes
those who fail to reach the age of 65. So the expectancy has to be
recomputed by taking this into account.
Example: The average rainfall in July in a certain city is 10.04
inches. It has already rained 8 inches by the middle of July. So
how much more rain can be expected in the second half of July?
Answer is not 2.04 inches!

M. Vidyasagar FRS Predicting Extreme Events

 Motivating Examples
Basic Probability Theory
Central Limit Theorem and Stable Distributions Probability Distribution and Density Functions
Examples Two Standard Distributions
Anomalous Behavior of Heavy-Tailed Random Variables
Concluding Remarks

Conditional PDF

Suppose X is a random variable with pdf ΦX (·) and density φX (·),

and b is some specified number. What are the pdf and density of
X once it is specified that X > b?

 1, if a ≤ b,
Φ̄X>b (a) = Φ̄X (a)
 , if a > b,
Φ̄X (b)

M. Vidyasagar FRS Predicting Extreme Events

 Motivating Examples
Basic Probability Theory
Central Limit Theorem and Stable Distributions Probability Distribution and Density Functions
Examples Two Standard Distributions
Anomalous Behavior of Heavy-Tailed Random Variables
Concluding Remarks

Self-Similarity of the Pareto Distribution

Recall that Φ̄(a) = a−α for all a ≥ 1. Consequently the Pareto

distribution has the property of “self-similarity.” For any real
number b, the conditional complementary PDF proportional to the
unconditional complementary PDF.
To paraphrase, if 10% of the people have 90% of the wealth, then
among those 10%, again 10% (i.e., 1% of the original population)
will have 90% of the wealth.

M. Vidyasagar FRS Predicting Extreme Events

 Motivating Examples
Basic Probability Theory
Central Limit Theorem and Stable Distributions Central Limit Theorem
Examples Stable Distributions
Anomalous Behavior of Heavy-Tailed Random Variables
Concluding Remarks

Outline
1 Motivating Examples
2 Basic Probability Theory
Probability Distribution and Density Functions
Two Standard Distributions
3 Central Limit Theorem and Stable Distributions
Central Limit Theorem
Stable Distributions
4 Examples
Using Common Sense
Using Stable Distributions
5 Anomalous Behavior of Heavy-Tailed Random Variables
6 Concluding Remarks
M. Vidyasagar FRS Predicting Extreme Events
 Motivating Examples
Basic Probability Theory
Central Limit Theorem and Stable Distributions Central Limit Theorem
Examples Stable Distributions
Anomalous Behavior of Heavy-Tailed Random Variables
Concluding Remarks

Outline
1 Motivating Examples
2 Basic Probability Theory
Probability Distribution and Density Functions
Two Standard Distributions
3 Central Limit Theorem and Stable Distributions
Central Limit Theorem
Stable Distributions
4 Examples
Using Common Sense
Using Stable Distributions
5 Anomalous Behavior of Heavy-Tailed Random Variables
6 Concluding Remarks
M. Vidyasagar FRS Predicting Extreme Events
 Motivating Examples
Basic Probability Theory
Central Limit Theorem and Stable Distributions Central Limit Theorem
Examples Stable Distributions
Anomalous Behavior of Heavy-Tailed Random Variables
Concluding Remarks

Why is the Gaussian Distribution So Pervasive?

Suppose X is a real-valued r.v. with finite variance (which also

implies that X has finite mean). Let µ, σ denote the mean
(expected value) and standard deviation of X, respectively.
Suppose we generate independent copies of X, call them {Xi }i≥1 ,
and then average them. Thus
l l
X 1X Sl
Sl = Xi , Al = Xi =
l l
i=1 i=1

denote the sum and average of the first l independent copies of x.

How do they behave as l → ∞?

M. Vidyasagar FRS Predicting Extreme Events

 Motivating Examples
Basic Probability Theory
Central Limit Theorem and Stable Distributions Central Limit Theorem
Examples Stable Distributions
Anomalous Behavior of Heavy-Tailed Random Variables
Concluding Remarks

The Central Limit Theorem

Theorem: As l → ∞, the distribution function of the centered

and scaled r.v.
Al − µ S − lµ
Yl = √ = l√
σ/ l lσ
converges to that of the normal random variable, that is, a
Gaussian r.v. with zero mean and standard deviation of one.
Note that the theorem is true even if X is a discrete-valued r.v.,
such as the payoff associated with a coin toss.

M. Vidyasagar FRS Predicting Extreme Events

 Motivating Examples
Basic Probability Theory
Central Limit Theorem and Stable Distributions Central Limit Theorem
Examples Stable Distributions
Anomalous Behavior of Heavy-Tailed Random Variables
Concluding Remarks

Illustrative Example

Suppose X assumes values in the discrete set {−2, −1, 0, 1, 2}

with distribution vector

φ = [ 0.15 0.20 0.25 0.20 0.20 ].

Then, for each integer l, the l-fold average Al also assumes values
in the interval [−2, 2]. While Al is also discrete-valued, the number
of possible values increases as l increases.
The next slides show the densities of 5-fold, 20-fold, and 100-fold
averages of independent copies of X.

M. Vidyasagar FRS Predicting Extreme Events

 Motivating Examples
Basic Probability Theory
Central Limit Theorem and Stable Distributions Central Limit Theorem
Examples Stable Distributions
Anomalous Behavior of Heavy-Tailed Random Variables
Concluding Remarks

Depictions of Densities

Blue curve is the original density, green is the 5-fold average, red is
the 20-fold average, and taupe is the 100-fold average.
M. Vidyasagar FRS Predicting Extreme Events
 Motivating Examples
Basic Probability Theory
Central Limit Theorem and Stable Distributions Central Limit Theorem
Examples Stable Distributions
Anomalous Behavior of Heavy-Tailed Random Variables
Concluding Remarks

Outline
1 Motivating Examples
2 Basic Probability Theory
Probability Distribution and Density Functions
Two Standard Distributions
3 Central Limit Theorem and Stable Distributions
Central Limit Theorem
Stable Distributions
4 Examples
Using Common Sense
Using Stable Distributions
5 Anomalous Behavior of Heavy-Tailed Random Variables
6 Concluding Remarks
M. Vidyasagar FRS Predicting Extreme Events
 Motivating Examples
Basic Probability Theory
Central Limit Theorem and Stable Distributions Central Limit Theorem
Examples Stable Distributions
Anomalous Behavior of Heavy-Tailed Random Variables
Concluding Remarks

Stable Distributions: Motivation

Suppose we generate independent copies of a r.v. X that may not

have finite variance (or even finite mean). As before we form the
quantities
l l
X 1X Sl
Sl = Xi , Al = Xi =
l l
i=1 i=1

By centering and scaling the partial sums, i.e.,

Sl − bl
Yl =
al
for suitable constants al and bl , can we get a “limit” r.v.?
Answer: The only possible limits are the stable distributions.

M. Vidyasagar FRS Predicting Extreme Events

 Motivating Examples
Basic Probability Theory
Central Limit Theorem and Stable Distributions Central Limit Theorem
Examples Stable Distributions
Anomalous Behavior of Heavy-Tailed Random Variables
Concluding Remarks

Some Basic Facts

Every stable distribution is either Gaussian, or “pseudo-Pareto.”

So if Φ is a stable CDF, and Φ is not Gaussian, then there is a
constant α ∈ (0, 2), called the “exponent,” such that

Φ̄(a) ∼ a−α .

Clearly, the smaller the value of α, the more slowly the

complementary CDF rolls off, and the more “heavy-tailed” is the
r.v.
Stable distributions give better fits to real-world data compared to
Gaussian distributions.

M. Vidyasagar FRS Predicting Extreme Events

 Motivating Examples
Basic Probability Theory
Central Limit Theorem and Stable Distributions Using Common Sense
Examples Using Stable Distributions
Anomalous Behavior of Heavy-Tailed Random Variables
Concluding Remarks

Outline
1 Motivating Examples
2 Basic Probability Theory
Probability Distribution and Density Functions
Two Standard Distributions
3 Central Limit Theorem and Stable Distributions
Central Limit Theorem
Stable Distributions
4 Examples
Using Common Sense
Using Stable Distributions
5 Anomalous Behavior of Heavy-Tailed Random Variables
6 Concluding Remarks
M. Vidyasagar FRS Predicting Extreme Events
 Motivating Examples
Basic Probability Theory
Central Limit Theorem and Stable Distributions Using Common Sense
Examples Using Stable Distributions
Anomalous Behavior of Heavy-Tailed Random Variables
Concluding Remarks

Outline
1 Motivating Examples
2 Basic Probability Theory
Probability Distribution and Density Functions
Two Standard Distributions
3 Central Limit Theorem and Stable Distributions
Central Limit Theorem
Stable Distributions
4 Examples
Using Common Sense
Using Stable Distributions
5 Anomalous Behavior of Heavy-Tailed Random Variables
6 Concluding Remarks
M. Vidyasagar FRS Predicting Extreme Events
 Motivating Examples
Basic Probability Theory
Central Limit Theorem and Stable Distributions Using Common Sense
Examples Using Stable Distributions
Anomalous Behavior of Heavy-Tailed Random Variables
Concluding Remarks

Rainfall Data for Houston

Source: http://www.ncdc.noaa.gov/cdo-web/datasets
A total of 2,039 days of rainfall data is available.
On most days there was no rain! Only 570 days on which it rained,
with a maximum of 2,002 (tenths of millimeters), or about 7.8
inches.
Objective: Find values of rainfall u such that Φ̄(u) = 0.1,
Φ̄(u) = 0.01, and Φ̄(u) = 0.001 (90%, 99% and 99.9% risk
values).

M. Vidyasagar FRS Predicting Extreme Events

 Motivating Examples
Basic Probability Theory
Central Limit Theorem and Stable Distributions Using Common Sense
Examples Using Stable Distributions
Anomalous Behavior of Heavy-Tailed Random Variables
Concluding Remarks

Rainfall Data at Houston: Medium Range

Rainfall plotted in log scale. For medium range,

Φ̄(u) ≈ c0 − c1 log u. So Φ̄(u) = 0 for u ≥ c0 /c1 .

M. Vidyasagar FRS Predicting Extreme Events

 Motivating Examples
Basic Probability Theory
Central Limit Theorem and Stable Distributions Using Common Sense
Examples Using Stable Distributions
Anomalous Behavior of Heavy-Tailed Random Variables
Concluding Remarks

Rainfall Data at Houston: Extreme Range

Plot of Φ̄(u) vs. u for Φ̄(u) ≤ 0.1.

M. Vidyasagar FRS Predicting Extreme Events
 Motivating Examples
Basic Probability Theory
Central Limit Theorem and Stable Distributions Using Common Sense
Examples Using Stable Distributions
Anomalous Behavior of Heavy-Tailed Random Variables
Concluding Remarks

Rainfall Data at Houston: Extreme Range (Cont’d)

Plot of log Φ̄(u) vs. log u for Φ̄(u) ≤ 0.1.

M. Vidyasagar FRS Predicting Extreme Events
 Motivating Examples
Basic Probability Theory
Central Limit Theorem and Stable Distributions Using Common Sense
Examples Using Stable Distributions
Anomalous Behavior of Heavy-Tailed Random Variables
Concluding Remarks

Rainfall Data at Houston: Extreme Range (Cont’d)

Least squares fit with log Φ̄(u) ≈ 11.4241 − 2.3376 × log(u).

M. Vidyasagar FRS Predicting Extreme Events
 Motivating Examples
Basic Probability Theory
Central Limit Theorem and Stable Distributions Using Common Sense
Examples Using Stable Distributions
Anomalous Behavior of Heavy-Tailed Random Variables
Concluding Remarks

Estimating the Tail of Houston Rainfall

Estimated tail fit is

log Φ̄(u) ≈ 11.4241 − 2.3376 log u, or Φ̄(u) ∼ u−2.3376 .

We can reconstruct the 99- and 99.9-percentile values of the

estimated Φ̄.
RHS = 0.01 if u = 950.5 (tenths of millimeters), or about 3.74
inches, and equals 0.001 if u = 2545.5 (tenths of millimeters), or
about 10 inches.
This value is more than the maximum observed rainfall of 2,002!

M. Vidyasagar FRS Predicting Extreme Events

 Motivating Examples
Basic Probability Theory
Central Limit Theorem and Stable Distributions Using Common Sense
Examples Using Stable Distributions
Anomalous Behavior of Heavy-Tailed Random Variables
Concluding Remarks

Outline
1 Motivating Examples
2 Basic Probability Theory
Probability Distribution and Density Functions
Two Standard Distributions
3 Central Limit Theorem and Stable Distributions
Central Limit Theorem
Stable Distributions
4 Examples
Using Common Sense
Using Stable Distributions
5 Anomalous Behavior of Heavy-Tailed Random Variables
6 Concluding Remarks
M. Vidyasagar FRS Predicting Extreme Events
 Motivating Examples
Basic Probability Theory
Central Limit Theorem and Stable Distributions Using Common Sense
Examples Using Stable Distributions
Anomalous Behavior of Heavy-Tailed Random Variables
Concluding Remarks

Daily Returns on Microsoft Shares CDF: Gaussian Fit

Data over 1,371 days. Actual in blue and fitted is in green. Note
mismatch between the empirical and fitted distribution function.
M. Vidyasagar FRS Predicting Extreme Events
 Motivating Examples
Basic Probability Theory
Central Limit Theorem and Stable Distributions Using Common Sense
Examples Using Stable Distributions
Anomalous Behavior of Heavy-Tailed Random Variables
Concluding Remarks

Microsoft Shares CDF: Stable Fit

The fitted distribution with α = 1.5911 matches the empirical

distribution very well.
M. Vidyasagar FRS Predicting Extreme Events
 Motivating Examples
Basic Probability Theory
Central Limit Theorem and Stable Distributions Using Common Sense
Examples Using Stable Distributions
Anomalous Behavior of Heavy-Tailed Random Variables
Concluding Remarks

VAR of GE Shares With Monthly Data

Problem: One month’s data for GE shares is available. Estimate

the 99% Value at Risk (VAR).
Difficulty: Even second highest value is 96.67-percentile!
Approach: Convert data into daily returns (as percentages), fit an
appropriate distribution, and choose 99-th percentile value.

M. Vidyasagar FRS Predicting Extreme Events

 Motivating Examples
Basic Probability Theory
Central Limit Theorem and Stable Distributions Using Common Sense
Examples Using Stable Distributions
Anomalous Behavior of Heavy-Tailed Random Variables
Concluding Remarks

Gaussian and Stable Fits

Blue is actual (sorted) data, red is Gaussian fit, green is stable fit
with α = 1.4558.
M. Vidyasagar FRS Predicting Extreme Events
 Motivating Examples
Basic Probability Theory
Central Limit Theorem and Stable Distributions Using Common Sense
Examples Using Stable Distributions
Anomalous Behavior of Heavy-Tailed Random Variables
Concluding Remarks

Estimated VAR

The maximum positive fluctuation over 30 days is 0.0266.

With the Gaussian fit, the 99-th percentile value is 0.0297.
With the stable fit, the 99-th percentile value is 0.0440!
So with stable fit, extreme fluctuations are more!

M. Vidyasagar FRS Predicting Extreme Events

 Motivating Examples
Basic Probability Theory
Central Limit Theorem and Stable Distributions
Examples
Anomalous Behavior of Heavy-Tailed Random Variables
Concluding Remarks

Outline
1 Motivating Examples
2 Basic Probability Theory
Probability Distribution and Density Functions
Two Standard Distributions
3 Central Limit Theorem and Stable Distributions
Central Limit Theorem
Stable Distributions
4 Examples
Using Common Sense
Using Stable Distributions
5 Anomalous Behavior of Heavy-Tailed Random Variables
6 Concluding Remarks
M. Vidyasagar FRS Predicting Extreme Events
 Motivating Examples
Basic Probability Theory
Central Limit Theorem and Stable Distributions
Examples
Anomalous Behavior of Heavy-Tailed Random Variables
Concluding Remarks

A Law of Large Numbers

Suppose X has finite mean µ, but not necessarily finite variance.

The law of large numbers states that if we define, as before
l l
X 1 1X
Sl := Xt , Al = Sl = Xt .
l l
t=1 t=1

Then Al converges in probability to the correct value µ. So if we

define the tail probability

δ(l, ) := Pr{Al ≥ µ + } = Pr{Sl ≥ l(µ + )},

then δ(l, ) → 0 as l → ∞, for each  > 0.

M. Vidyasagar FRS Predicting Extreme Events

 Motivating Examples
Basic Probability Theory
Central Limit Theorem and Stable Distributions
Examples
Anomalous Behavior of Heavy-Tailed Random Variables
Concluding Remarks

Behavior of Cumulative Sum vs. Maximum

We have already defined

δ(l, ) := Pr{Al ≥ µ + } = Pr{Sl ≥ l(µ + )},

Now define

γ(l, ) := Pr{max{X1 , . . . , Xl } ≥ l(µ + )}.

How do δ(l, ) and γ(l, ) compare?

If Xt are nonnegative, then

γ(l, ) ≤ δ(l, ), ∀l,  > 0.

A huge excursion in one r.v. causes the average to be larger than ,

but average can exceed  even through several small excursions in
each r.v.
M. Vidyasagar FRS Predicting Extreme Events
 Motivating Examples
Basic Probability Theory
Central Limit Theorem and Stable Distributions
Examples
Anomalous Behavior of Heavy-Tailed Random Variables
Concluding Remarks

A Key Theorem
Theorem: If Xt is nonnegative and has a Pareto-type tail with
exponent α < 2, then
γ(l, l )
γ(l, l ) ∼ δ(l, l ), i.e. lim = 1,
l→∞ δ(l, l )

for every sequence {l } ↓ 0 such that nΦ̄(l ) → 0 as l → ∞.

In contrast, if X has finite variance (corresponding to the exponent
α = 2), then every fixed  > 0,
γ(l, )
→ 0 as n → ∞.
δ(l, )
In words, if we average heavy-tailed r.v.s, then a large tail
excursion is just as likely to occur through a huge excursion of one
variable as through small excursions of several variables. But this is
not so for r.v.s with M.
finite variance. Predicting Extreme Events
Vidyasagar FRS
 Motivating Examples
Basic Probability Theory
Central Limit Theorem and Stable Distributions
Examples
Anomalous Behavior of Heavy-Tailed Random Variables
Concluding Remarks

An Interesting ‘Either-Or’ Situation

To recapitulate:

If the r.v. X has exponent α < 2, then large fluctuations of

the cumulative sum almost always happen only through a
huge excursion in just one sample.
If the r.v. X has finite variance, then large fluctuations of the
cumulative sum almost never happen only through a huge
excursion in just one sample. Instead, they almost always
happen through an accretion of several small fluctuations.

Notice that there is no in-between possibility.

Do observations of “real” quantities demonstrate the first type of
behavior? If so that is a strong case for modeling with heavy-tailed
distributions.
M. Vidyasagar FRS Predicting Extreme Events
 Motivating Examples
Basic Probability Theory
Central Limit Theorem and Stable Distributions
Examples
Anomalous Behavior of Heavy-Tailed Random Variables
Concluding Remarks

Observed Behavior of Asset Prices

• Increase in share price of Akamai from November 2001 until

now: 12.10, or a return of 1,110%.
⇒ Just three days account for 694% of the return!
• Between 1955 and 2004, S&P average moved up by a factor of
180.
⇒ If we remove ten largest movements (most of which were
negative), the increase is 350.
So real asset price movements do move in a few large bursts and
not smoothly!
Is this a justification for using heavy-tailed r.v.s in modeling?

M. Vidyasagar FRS Predicting Extreme Events

 Motivating Examples
Basic Probability Theory
Central Limit Theorem and Stable Distributions
Examples
Anomalous Behavior of Heavy-Tailed Random Variables
Concluding Remarks

Outline
1 Motivating Examples
2 Basic Probability Theory
Probability Distribution and Density Functions
Two Standard Distributions
3 Central Limit Theorem and Stable Distributions
Central Limit Theorem
Stable Distributions
4 Examples
Using Common Sense
Using Stable Distributions
5 Anomalous Behavior of Heavy-Tailed Random Variables
6 Concluding Remarks
M. Vidyasagar FRS Predicting Extreme Events
 Motivating Examples
Basic Probability Theory
Central Limit Theorem and Stable Distributions
Examples
Anomalous Behavior of Heavy-Tailed Random Variables
Concluding Remarks

Concluding Remarks

Many naturally observed quantities such as rainfall, foreign

exchange fluctations, Internet traffic etc. are better modeled
via heavy-tailed distributions than via Gaussian distributions.
Stable distributions with α < 2 have infinite variance! This is
not meaningful, but estimates even within the “visible range”
are meaningful.
When averaging heavy-tailed distributions, the cumulative
sums exhibit sudden bursts of movement – a characteristic
that has been observed in the price of several stocks for
example.

M. Vidyasagar FRS Predicting Extreme Events

 Motivating Examples
Basic Probability Theory
Central Limit Theorem and Stable Distributions
Examples
Anomalous Behavior of Heavy-Tailed Random Variables
Concluding Remarks

Next Steps

All that has been said is predicated on successive samples being

independent – an unrealistic assumption.
It is quite reasonable to assume fading memory – today’s stock
prices may be correlated with yesterday’s, but not with those of six
months ago.

M. Vidyasagar FRS Predicting Extreme Events

 Motivating Examples
Basic Probability Theory
Central Limit Theorem and Stable Distributions
Examples
Anomalous Behavior of Heavy-Tailed Random Variables
Concluding Remarks

Some Useful References (in Alphabetical Order)

Leo Breiman, Probability , SIAM, Philadelphia, 1992.

Paul Embrechts, Claudia Klüppelberg and Thomas Mikosch,
Modelling Extreme Events, Springer-Verlag, (Corrected Fourth
Printing) 2003.
B. V. Gnedenko and A. N. Kolmogorov, Limit Distributions of
Sums of Independent Random Variables, Addison-Wesley,
1954.
W. E. Leland, M. S. Taqqu, W. Willinger and D. V. Wilson,
“On the self-similar nature of ethernet traffic (extended
version)”, IEEE/ACM Trans. on Networking, 2(1), 1-15, 1994.

M. Vidyasagar FRS Predicting Extreme Events

 Motivating Examples
Basic Probability Theory
Central Limit Theorem and Stable Distributions
Examples
Anomalous Behavior of Heavy-Tailed Random Variables
Concluding Remarks

Some Useful References (in Alphabetical Order) (Cont’d)

Benoit B. Mandelbrot, “The variation of certain speculative

prices”, J. of Business, 36, 394-419, 1963.
Benoit Mandelbrot and Richard L. Hudson, The
(Mis)Behavior of Markets, Basic Books, New York, 2004.
John P. Nolan, “Modeling financial data with stable
distributions”, in book edited by S. T. Rachev.
S. T. Rachev (Editor), Handbook of Heavy Tailed
Distributions in Finance, Elsevier/North Holland, Amsterdam,
2003.
X. Zhu, J. Yu and J. Doyle, “Heavy tails, generalized coding,
and optimal web layout”, IEEE Infocom, April 2001.

M. Vidyasagar FRS Predicting Extreme Events

 Motivating Examples
Basic Probability Theory
Central Limit Theorem and Stable Distributions
Examples
Anomalous Behavior of Heavy-Tailed Random Variables
Concluding Remarks

Questions?

M. Vidyasagar FRS Predicting Extreme Events