Logistic Distribution

The logistic distribution is a continuous probability distribution that is often used in statistics and machine
learning. It is defined by two parameters: µ (the location parameter) and σ (the scale parameter).
The probability density function (PDF) of the logistic distribution is:

e−(x−µ)/σ
f (x; µ, σ) =
2
σ 1 + e−(x−µ)/σ

where:
• µ is the location parameter, which determines the center of the distribution.
• σ is the scale parameter, which controls the spread of the distribution.
The cumulative distribution function (CDF) of the logistic distribution is:

1
F (x; µ, σ) =
1 + e−(x−µ)/σ

This distribution is similar to the normal distribution but has heavier tails, which can make it useful for
modeling data with outliers or for binary classification problems in machine learning.
The range of x for the logistic distribution is (−∞, ∞). The logistic distribution is defined for all real numbers,
so x can take any value from negative infinity to positive infinity.
To derive the expectation (mean) and variance of the logistic distribution, we start with its probability density
function (PDF) and cumulative distribution function (CDF). For the logistic distribution with parameters µ
(location) and σ (scale), the PDF is:

e−(x−µ)/σ
f (x; µ, σ) =
2
σ 1 + e−(x−µ)/σ

and the CDF is:

1
F (x; µ, σ) =
1 + e−(x−µ)/σ

1. Expectation (Mean)
The expectation (mean) of the logistic distribution can be found directly from the PDF. However, to derive it
formally, let’s compute it using integration:
Z ∞
E[X] = xf (x; µ, σ) dx
−∞

Substitute the PDF into the integral:

∞
e−(x−µ)/σ
Z
E[X] = x
2 dx
−∞ σ 1 + e−(x−µ)/σ

To simplify this, use a change of variable. Let z = σ ,

x−µ
so x = µ + σz and dx = σdz. Substitute these into
the integral:

1
 ∞
e−z
Z
E[X] = (µ + σz) σ dz
−∞ σ(1 + e−z )2

∞
e−z
Z
E[X] = (µ + σz) dz
−∞ (1 + e−z )2

Separate the integral into two parts:

∞ ∞
e−z ze−z
Z Z
E[X] = µ dz + σ dz
−∞ (1 + e−z )2 −∞ (1 + e−z )2

The first integral is:

∞
e−z
Z
dz = 1
−∞ (1 + e−z )2

The second integral involves a standard result:

∞
ze−z
Z
dz = 0
−∞ (1 + e−z )2

Therefore:

E[X] = µ · 1 + σ · 0 = µ

2. Variance
To find the variance, we need E[X 2 ]:
Z ∞
E[X ] = 2
x2 f (x; µ, σ) dx
−∞

Substitute the PDF into the integral:

∞
e−(x−µ)/σ
Z
E[X ] =2
x2
2 dx
−∞ σ 1 + e−(x−µ)/σ

Use the same change of variable z = σ :

x−µ

∞
e−z
Z
E[X 2 ] = (µ + σz)2 dz
−∞ (1 + e−z )2

Expanding (µ + σz)2 :

(µ + σz)2 = µ2 + 2µσz + σ 2 z 2

∞
e−z
Z
E[X 2 ] = µ2 + 2µσz + σ 2 z 2


dz
−∞ (1 + e−z )2

Separate into three integrals:

2
 ∞ ∞ ∞
e−z ze−z z 2 e−z
Z Z Z
E[X 2 ] = µ2 dz + 2µσ dz + σ 2 dz
−∞ (1 + e−z )2 −∞ (1 + e−z )2 −∞ (1 + e−z )2

The first integral is:

∞
e−z
Z
dz = 1
−∞ (1 + e−z )2

The second integral:

∞
ze−z
Z
dz = 0
−∞ (1 + e−z )2

For the third integral, use the fact that:

∞
z 2 e−z π2
Z
dz =
−∞ (1 + e )
−z 2 3

Thus:

π2
E[X 2 ] = µ2 + σ 2 ·
3

Finally, the variance Var(X) is:

Var(X) = E[X 2 ] − (E[X])2

π2
Var(X) = µ2 + σ 2 · − µ2
3

π2
Var(X) = σ 2 ·
3
So, the expectation and variance of the logistic distribution are:
• Expectation (Mean): µ
2
• Variance: σ 2 · π3
To derive the cumulative distribution function (CDF) of the logistic distribution, we start with its probability
density function (PDF). The PDF of the logistic distribution with parameters µ (location) and σ (scale) is:

e−(x−µ)/σ
f (x; µ, σ) =
2
σ 1 + e−(x−µ)/σ

The CDF, F (x), is obtained by integrating the PDF from −∞ to x:

Z x
F (x; µ, σ) = f (t; µ, σ) dt
−∞

Substitute the PDF into the integral:

3
 x
e−(t−µ)/σ
Z
F (x; µ, σ) =
2 dt
−∞ σ 1 + e−(t−µ)/σ

To simplify this integral, use a change of variable. Let z = σ .

t−µ
Then, t = µ + σz and dt = σ dz. Substituting
these into the integral:

x−µ
e−z
Z σ
F (x; µ, σ) = dz
−∞ 1 + e−z

To simplify further, let’s evaluate the integral of the function:

z
e−u
Z
du
−∞ 1 + e−u

Rewrite the integrand:

e−u 1
=
1 + e−u 1 + eu

Thus:

z
1
Z
du
−∞ 1 + eu

Let v = 1 + eu . Then dv = eu du, so du = eu .

dv
Rewriting eu = v − 1:

1+ez 1+ez
z
1 1 dv
Z Z Z
dv
du = =
−∞ 1 + eu 1 vv−1 1 v(v − 1)

Using partial fraction decomposition:

1 1 1
= −
v(v − 1) 1−v v

Integrate:

1+ez 1+ez
1 1 v−1
Z  
− dv = ln
1 v−1 v v 1

Evaluating the bounds:

(1 + ez ) − 1 1−1 ez
ln − ln = ln
1 + ez 1 1 + ez

ez ez
 
= ln = ln = z − ln (1 + ez )
1 + ez 1 + ez

Rewriting this:

4
 x−µ
1 1
Z σ
du =
−∞ 1 + eu 1 + e−(x−µ)/σ

Therefore, the cumulative distribution function (CDF) of the logistic distribution is:

1
F (x; µ, σ) =
1+ e−(x−µ)/σ
This result shows that the logistic distribution has a CDF similar to the sigmoid function, which is often
used in logistic regression and neural networks.
To derive the moment-generating function (MGF) of the logistic distribution, we use its probability density
function (PDF). The MGF, MX (t), is defined as:

MX (t) = E[etX ]

where E denotes the expectation. For a random variable X with the logistic distribution, this is given by:
Z ∞
MX (t) = etx f (x; µ, σ) dx
−∞

where the PDF of the logistic distribution is:

e−(x−µ)/σ
f (x; µ, σ) =
2
σ 1 + e−(x−µ)/σ

Substituting the PDF into the integral:

∞
e−(x−µ)/σ
Z
MX (t) = etx
2 dx
−∞ σ 1 + e−(x−µ)/σ

Combine the exponents in the integrand:

∞
etx−(x−µ)/σ
Z
MX (t) =
2 dx
−∞ σ 1 + e−(x−µ)/σ

∞
etx−(x/σ−µ/σ)
Z
MX (t) =
2 dx
−∞ σ 1 + e−(x−µ)/σ

∞
ex(t−1/σ)+µ/σ
Z
MX (t) =
2 dx
−∞ σ 1 + e−(x−µ)/σ

To simplify this, let’s use a change of variables. Let z = σ ,

x−µ
which implies x = µ + σz and dx = σ dz.
Substitute these into the integral:

∞
e(µ+σz)(t−1/σ)+µ/σ
Z
MX (t) = 2 σ dz
−∞ σ (1 + e−z )

∞
eµ(t−1/σ)+σz(t−1/σ)+µ/σ
Z
MX (t) = 2 dz
−∞ (1 + e−z )

5
Combine the exponents:

∞
eσz(t−1/σ)
Z
MX (t) = e µ(t−1/σ)+µ/σ
2 dz
−∞ (1 + e−z )

Now, let’s simplify the integral. We know that:

∞
eσz(t−1/σ)
Z
2 dz
−∞ (1 + e−z )

This integral can be recognized as a known result in the literature and evaluates to:

∞
eσz(t−1/σ)
Z
πσ
dz =
−∞ (1 + e−z )
2 sin(π(t − 1/σ))

Thus:

πσ
MX (t) = eµ(t−1/σ)+µ/σ ·
sin(π(t − 1/σ))

πσeµt
MX (t) =
sin(π(t − 1/σ))

This formula provides the MGF of the logistic distribution.

To derive the raw moments of the logistic distribution, we need to find the n-th moment, which is given by:
Z ∞
E[X n ] = xn f (x; µ, σ) dx
−∞

where f (x; µ, σ) is the probability density function (PDF) of the logistic distribution:

e−(x−µ)/σ
f (x; µ, σ) =
2
σ 1 + e−(x−µ)/σ

Derivation of the n-th Raw Moment

1. Change of Variable:
To simplify the calculation, we use the change of variable z = σ .
x−µ
Hence, x = µ + σz and dx = σ dz.
Substitute these into the integral for the n-th moment:

∞
e−z
Z
E[X ] =
n
(µ + σz)n 2 σ dz
−∞ σ (1 + e−z )

∞
e−z
Z
E[X ] = n
(µ + σz)n dz
−∞ (1 + e−z )2

6
 2. Expand (µ + σz)n :
Using the binomial expansion, expand (µ + σz)n :

n  
X n
(µ + σz)n = µn−k (σz)k
k
k=0

Substituting this into the integral:

" n  
#
∞
e−z
Z X n n−k
E[X n ] = µ (σz)k dz
−∞ k (1 + e−z )2
k=0

n  
n n−k k ∞ k e−z
X Z
E[X n ] = µ σ z dz
k −∞ (1 + e−z )2
k=0

3. Evaluate the Integral:

The integral we need to evaluate is:

∞
e−z
Z
zk dz
−∞ (1 + e−z )2

This integral has known results:

• For k = 0, the integral is 1.
• For k = 1, the integral is 0.
• For k ≥ 2, the integral is given by:

∞
e−z (k − 1)!π
Z
zk dz =
−∞ (1 + e )
−z 2
sin(π k2 )

The results can be derived from properties of the Gamma function and specific integrals involving the
Beta function.
4. Combine Results:
Combining the results, the n-th raw moment of the logistic distribution is:

n  
X n (k − 1)!π
E[X n ] = µn−k σ k
k=0
k sin(π k2 )

Specifically, for integer n, this becomes:

• For n = 0: E[X 0 ] = 1
• For n = 1: E[X 1 ] = µ
2 2
• For n = 2: E[X 2 ] = µ2 + π 3σ
• For higher n, use the general formula with appropriate integrals.
This process allows us to compute raw moments for the logistic distribution, providing insights into the
distribution’s shape and characteristics.