Transformations

Motivation

X N10, 11 Y =
E(X) -Wo , 1)
i
,

X has a
cDFF , F(X)-W(0 1
• Suppose we transform a continuous random variable X
,

into another continuous random variable, Y = g(X).

If we know the PDF of X, how do we find the PDF of Y ?
• Linear transformation: Celsius to Fahrenheit
• Nonlinear: Human age = 16 log (Dog age) + 31
https://www.sciencemag.org/news/2019/11/
here-s-better-way-convert-dog-years-human-years-scientists-say

(3) X -U10, 1) , E(X) -N(0 1) ,

• Suppose we transform a random vector X = (X1 , . . . , Xn )
into another random vector Y = (Y1 , . . . , Yn ).
If we know the joint PDF of X,
how do we find the joint PDF of Y ?
• Rectangular to polar coordinates
• Suppose X1 , . . . , Xn are independent r.v.s
and we know their distributions.
How do we find the distribution of their sum or average?
 Change of variables in one dimension: Theorem
100X i e., Y = 100X
(1) g(X) =
,
.

12) g(X) = eY ,
i e, .

Y= eY
Suppose X is a continuous random variable with PDF f X .

Let Y = g(X), where the function g is differentiable and

either strictly increasing or strictly decreasing on the support of X.
• (If g doesn’t satisfy these assumptions, can’t use this theorem.
Instead, derive the CDF of Y. See Example 8.1.4.)

Assure g(X) is
strictly increasin
Q :
what is pay of Y ?
 g)
(ghcFexd)
Y = g(X) Fact :
,

f
T
X has a fxMX)
pdf
Fy(y) = P(Y < 7) *

= f(h(y1) h (Y)
=
P(g(X( = y)

,y))
+
y = g(x)
y = P(X =
g
g(y) =
98
%
fx(x)dX
Exx)y) 11

fy (3) = Fig =
+
fx (g (1) ·

(94y1/
=
Then the PDF of Y is
! !
y = g(x)
! dx !
f Y (y) = f X (x) · !! !!
dy

where x = g−1 (y) is the real number that solves y = g(x).

• dx/dy = 1 / (dy/dx) (by the inverse function theorem)
so we can compute whichever derivative is easier.

The support of Y is the set of all values y ∈ R such that

y = g(x) for some x in the support of X.
 Change of variables in one dimension: Intuition
Y = 100X i e ., ,
< .

=
! !
! dx !
f Y (y) = f X (x) · !! !!
dy
fy(y) = fxx) t Ex too
:

=
Suppose X is height in meters and Y = 100 X is height in cm.
•
Why doesn’t f (180) equal f (1.8) ?
filyl = For
Y X
Y
Example fx(X e: = , x> 0 ,
• f X (1.8) |dx| approximates the probability that too
=E
X is between 1.8 and (1.8 + dx), for a tiny increment dx
1180Y• = 182 1801 .
2
When we change units, we’re talking about the probability that
Y is between 180 and (180 + dy), where dy = 100 dx

II
• Thus, we want f Y (180) |dy| = f X (1.8) |dx|,
so f Y (180) = f X (1.8) / 100
• In other words, f X (x) measures probability per meter
while f Y (y) measures probability per centimeter
1 (1 :
8 X= 1 :
82) =
fx (1 8) :
:
0 02
.
 Example: Log-Normal PDF
Fact :

y ex X logy
llogy1
=
=
,

[y = g(x) ,
Let X ∼ N (0, 1) and Y =
x=
e X.
g
+
(y)]
(We say that Y has a Log-Normal distribution,
which means “the log of Y is Normal.”)
= ex
* = glogy
fix) = e- =
y
fy(21 =
fxx(y) =
fx(logy) ↑ ·
 -

Then the support of Y is (0, ∞), the positive real numbers:

• The support of X is R.
As x ranges from −∞ to ∞, y = ex ranges from 0 to ∞.

The PDF of Y is
! !
! dx ! 1 1
f Y (y) = f X (x) · !! !! = ϕ(x) · = ϕ(log y) ·
dy y y

=- Hig
for y > 0 (and zero otherwise).
 Change of variables in n dimensions
ABBA
=X nnnxn nxr non

Suppose X = (X1 , . . . , Xn ) is a continuous random vector

with joint PDF f X .

Let Y = g(X), where:

• g is an invertible (one-to-one) function
from the support of X to a subset of Rn
• The partial derivatives of g exist and are continuous

:
C We know the joint pdf
of X Ex (x,
. xn) ...,

what is fy 18 , , Yol ? ...

If Y = x A ,
YA+= X = X,
I
Then the joint PDF of Y is
! !
! ∂x !
f Y (y) = f X (x) · !! !!
∂y

where: fycey =
fi(yA+ )
• x = g−1 (y) is the vector that solves y = g(x)
/detAll
• |∂x/∂y| is the absolute value of the determinant of the
Jacobian matrix (see next slide), which we assume is never 0

Fact :

E =
(A-1T
det (B) = det (BT (
 The Jacobian matrix

The Jacobian matrix in the change-of-variables formula is

the n × n matrix
⎛ ∂x1 ∂x1 ∂x1 ⎞
∂y1 ∂y2 · · · ∂yn
= ⎝ ... .. ⎟
∂x ⎜
∂y . ⎠
∂xn ∂xn ∂xn
∂y1 ∂y2 · · · ∂yn

FactjA+ =x
=
:Y = (A+ )T
By the inverse function theorem and properties of determinants,
! ! ! !−1
! ∂x ! ! !
! ! = ! ∂y !
! ∂y ! ! ∂x !

if |∂y/∂x| &= 0.

So we can compute whichever of the two Jacobians is easier.

 Example: Bivariate Normal joint PDF

Suppose (Z, W) is Bivariate Normal with:

• Z ∼ N (0, 1)
• W ∼ N (0, 1)
• Corr (Z, W) = ρ where −1 < ρ < 1

Covr(z wi ,
=
z W)COV(z) ,

z vws
 *

ye
-

Ex, = 21

One way to generate (Z, W) with exactly that joint distribution

is to start with X and Y i.i.d. N (0, 1) and let:
• Z = X
(
• W = ρ X + τ Y where τ = 1 − ρ2
(This is derived in Example 7.5.10 but we can verify that it works.)

CoV1z ,W) = Cov/X , &X + TY)=PVarix)

Let’s use the change-of-variables formula =P
to derive the joint PDF of (Z, W) from the joint PDF of (X, Y).

EW = 0 , VarIW) = Var/PX + IY)

Var(W) 1 =
= Var(PX) + Varly/
= P2 Var(X) +irVar(Y)
 Example: Bivariate Normal joint PDF (2)

=
(1) z=4

w = px + ty w
-

Px)
Ew Ez
=
• By the change-of-variables formula in 2 dimensions,
-

! !
! ∂(x, y) !
&

f Z,W (z, w) = f X,Y (x, y) · !!

)
!
∂(z, w) !

where (x, y) solves z = x and w = ρ x + τ y.

(1) +x , 3(X , y) =
e
(1) fz w(z ,
, wl = fx
,
Y(z , Ew-fzlE
fzwiz ,i =
x= f(z) = z
=1
f(z) = 1
E = 0

y =

tw- =
-E
=E
He
• We can first compute the inverse Jacobian
⎛ ⎞
∂z ∂z ' (
∂(z, w) ⎜ ∂x ∂y
⎟ 1 0
= ⎝ ⎠ =
∂(x, y) ∂w ∂w ρ τ
∂x ∂y

which has determinant 1 · τ − 0 · ρ = τ .

• Therefore,

f Z,W (z, w) = f X,Y (x, y) · |τ |−1 = f X,Y (x, y) / τ

 Example: Bivariate Normal joint PDF (3)

• From the previous slide, f Z,W (z, w) = f X,Y (x, y) / τ

• Since X and Y are i.i.d. N (0, 1),

' (
1 1) *
f X,Y (x, y) = ϕ(x) · ϕ(y) = exp − x2 + y2
2π 2
• We need to express (x, y) in terms of (z, w):
If z = x and w = ρ x + τ y , then
x = z and y = (−ρ z + w) / τ

• After some algebra, we get

' (
1 1 ) *
f Z,W (z, w) = exp − 2 z2 + w2 − 2ρzw
2πτ 2τ
 Convolution sums

Suppose X and Y are independent random variables

and we know their distributions.

How do we find the distribution of T = X + Y ?

• Sometimes we’ve used story proofs or MGFs
• Now we’ll learn a more general (but more tedious) method
If X and Y are independent discrete r.v.s,
then the PMF of their sum T = X + Y is
+
P (T = t) = P (X = x) · P (Y = t − x)
x -
+
= P (Y = y) · P (X = t − y)
y -

(x , y) Set x+
y =

= P(X = x
,
Y =y) = [P(X x141
=
=

fiy = + x+y =+ 2)
 Convolution sums: Proof

• By LOTP,
+
P (X + Y = t) = P (X + Y = t | X = x) · P (X = x)
x

• P (X + Y = t | X = x) = P (Y = t − x | X = x)

• Since X and Y are independent,

P (Y = t − x | X = x) = P (Y = t − x)

• Putting it all together:

+
P (X + Y = t) = P (Y = t − x) · P (X = x)
x
• Similarly,
+
P (X + Y = t) = P (X = t − y) · P (Y = y)
y
 Convolution integrals

x(p(Y
P(x + y= + 1 =
[P(x = = tx)

If X and Y are independent continuous r.v.s,

then the PDF of their sum T = X + Y is

, ∞
= =P(Y =
y)P(X ty)
=

f T (t) = f X (x) · f Y (t − x) dx
,−∞
∞
= f Y (y) · f X (t − y) dy
−∞
 Example: Exponential convolution
pdf of fx(x) =
&Ne* X,
X :

pdf of Y :
Suppose X and Y are i.i.d. Expo(λ). -

X8
fy (7)
What’s the PDF of T = X + Y ?

• For t < 0, we have f T (t) = 0

• Intuitively clear since T is never negative
=

&Xe O
,
,

0 W
.
%20

• Or if we use the convolution integral,

the integrand f X (x) · f Y (t − x) is zero for all x

f +
ch =
S Fyct-XIdY
to g
f+ (+ ) = S. fx x) fy( + x)dx -

0
λe
21
• For t > 0:
• f T (t) =
-∞
−∞
f X (x) · f Y (t − x) dx
• The integrand is zero for x < 0 or x > t, so we get
- t −λx
· λe−λ(t−x) dx xX - x+ +1x
xe
-

xe .

• Now the integrand simplifies to λ2 e−λt ,

.
which is constant as we integrate with respect to x, so we get
, t
f T (t) = λ2 e−λt dx = λ2 t e−λt
0

x+ xX
x-exX
-
+
Acknowledgements: All materials of this class are based on
Winston Lin’s S&DS241 class in Fall 2019 with some
modifications.