Ignacio Cascos Fernández

Department of Statistics
Universidad Carlos III de Madrid

Random variables

Statistics — 2013–2014
A random variable assesses a real number to each possible outcome of a
random experiment.
Definición 1. Given a random experiment with associated sample space E
(and associated probability P ), a random variable X is a mapping from the
sample space E into R. Formally, it must satisfy the mathematical property
of being measurable.
That is, a random variable is a mapping
X : E −→ R .
For any set of real numbers B, its inverse image, given by X −1 (B) = {e ∈
E : X(e) ∈ B}, is an event associated with the random experiment.
Associated with a random variable X, it is possible to define a probability
whose sample space is R and the events are subsets of R. That is, the mapping
PX that associates each set of real numbers B with the value PX (B) =
−1


P X (B) is a probability. We will usually write P (X ∈ B) instead of
PX (B) and we will refer to it as the probability that the random variable X
lies in (or assumes a value in) B.

Discrete random variables, probability mass function. A discrete

random variable can assume a finite or denumerable set of different values.
If X is discrete and the set of values that it can assume is {x1 , x2 , . . .}, the
distribution of X is completely determined by its probability mass function.
The probability mass function is a mapping p : R −→ R such that p(x) ≥ 0
and satisfies p(x) = P (X = x) for all x ∈ R. Therefore, P for the values xi that
X can assume, it holds p(xi ) = P (X = xi ) > 0 and i p(xi ) = 1, meanwhile
if x ∈
/ {x1 , x2 , . . .}, then p(x) = 0. Given any set of real numbers B, it holds
X
P (X ∈ B) = p(xi ).
xi ∈B

1
Continuous random variables, density mass function. The set of
values that a continuous random variable can assume contains an interval of
real numbers. The distribution of a continuous random variable is determined
R +∞ mass function is a mapping f : R −→
by its density mass function. A density
R that satisfies f (x) ≥ 0 and −∞ f (x)dx = 1. If f is the density mass
function of X, then for any set of real numbers B, it holds
Z
P (X ∈ B) = f (x)dx,
B
Rb
specifically, given a, b ∈ R, we have P (a ≤ X ≤ b) = a f (x)dx.
Notice that the probability that a continuous random variable assumes
any fixed real value is 0. That is, if X is continuous and a ∈ R, then P (X =
a) = 0.

Cumulative distribution function of a random variable. A cumula-

tive distribution function (cdf) is a function F : R −→ R such that
F is nondecreasing;
F is right-continuous, for any x0 ∈ R, it holds lı́mx→x+0 F (x) = F (x0 );

lı́mx→−∞ F (x) = 0;
lı́mx→+∞ F (x) = 1.
The cumulative distribution function of the random variable X is defined
for every x ∈ R as F (x) = P (X ≤ x).
If X is discrete, the set of all possible values that it assumes is given by
{x1 , x2 , . . .}, and its probability mass function is p, then
X
F (x) = p(xi ) .
xi ≤x

The cdf of a discrete random variable is stepwise.

If X is continuous with density mass function f , then
Z x
F (x) = f (t)dt .
−∞

The cdf of a continuous random variable is continuous and its derivative is

the density mass function, that is, F 0 (x) = f (x) for all x ∈ R.

2
 Given a, b ∈ R with a < b, it is possible to express the probability that X
lies in any interval with a and b as extreme values in terms of the cdf of X.

P (a < X ≤ b) = P (X ≤ b) − P (X ≤ a) = F (b) − F (a);

P (a ≤ X ≤ b) = P (X ≤ b) − P (X < a) = F (b) − lı́m− F (x);
x→a
P (a < X < b) = P (X < b) − P (X ≤ a) = lı́m− F (x) − F (a);
x→b
P (a ≤ X < b) = P (X < b) − P (X < a) = lı́m− F (x) − lı́m− F (x).
x→b x→a

Mathematical expectation or mean. The expectation (or mean) of a

random variable X is an averaging of the values that it assumes.
If X is discrete, its expectation is given by
X
E[X] = xi p(xi ).
i

If X is continuous, its expectation is given by

Z +∞
E[X] = xf (x)dx.
−∞

The mean of a random variable (population) is alternatively denoted by

µ or µX to make explicit reference to the random variable.

Properties of the mean. Given two random variables X, Y and a, b ∈ R,

E[aX + b] = aE[X] + b;

E[X + Y ] = E[X] + E[Y ].

Given a function g : R −→ R and a random

P variable X, the expectation
of g(X) can be computed as E[g(X)] = i g(xi )p(xi ) if X is discrete, and
R +∞
E[g(X)] = −∞ g(x)f (x)dx if X is continuous.

Variance. The variance is a parameter that measures the scatter of the

random variable. It is the mean quadratic distance of the random variable to
its mean value. The variance of X is given by

var[X] = E (X − E[X])2 .
 

3
 Commonly, the variance of a random variable (population) is denoted by
2 2
σ or σX to make explicit reference to the random variable. The standard
deviation
p of a random variable is the (positive) square root of its variance,
var[X], it is denoted by σ or σX .

Properties of the variance. Given a random variable X and a, b ∈ R,

2
var[X] = E[X 2 ] − E[X] ;

var[b] = 0;

var[aX] = a2 var[X];

var[aX + b] = a2 var[X].

More characteristic features of a random variable.

Median: MeX satisfies F (MeX ) = 1/2 ;

Quantiles: Given p ∈ (0, 1), the quantile xp satisfies F (xp ) = p ;

k-th moment about the mean: µk = E[(X − µ)k ] ;

Skewness: Skew = µ3 /σ 3 ;

Kurtosis: Kurt = µ4 /σ 4 − 3 .

Transformations of random variables. Given a random variable X with

known distribution and a function g : R −→ R, it is possible to obtain the
distribution of Y = g(X). In general, we have

FY (y) = P (g(X) ≤ y) = P (X ∈ Ay ) ,

where Ay = {x : g(x) ≤ y}. It often happens that the sets Ay are simple to
describe (e.g. when g is monotonic), and then we can compute their proba-
bility just by using our knowledge of the distribution of X. In those cases,
it is highly recommendable to develop further the expression above. If we
are interested in the density mass function of Y , we can finally derive with
respect to y.

4
 If X is discrete, it holds
X
FY (y) = P (g(X) ≤ y) = pX (xi ) ;
g(xi )≤y
X
pY (y) = P (g(X) = y) = pX (xi ) .
g(xi )=y

If X is continuous and g is differentiable and injective, then the density

mass function of the random variable Y = g(X) is
dx
fY (y) = fX (x) .
dy

Random vectors
In many situations we are interested in more than one feature (variable)
associated with the same random experiment. A random vector is a mapping
from a sample space E into Rn . A bivariate random vector maps E into R2 ,
(X, Y ) : E −→ R2 .

Joint distribution of a random vector. The joint distribution of a ran-

dom vector describes the simultaneous behavior of all variables that build
the random vector.

Discrete random vectors. Given X and Y two discrete random variables

(on the same probability space), we define
joint probability
P mass
P function: p(x, y) = P (X = x, Y = y), satisfying,
p(x, y) ≥ 0 y x y p(x, y) = 1 .

P distribution function: F (x0 , y0 ) = P (X ≤ x0 , Y ≤

joint cumulative
P
y0 ) = x≤x0 y≤y0 p(x, y) .

Continuous random vectors. Given X and Y two continuous random

variables (on the same probability space), we define
joint
R +∞ R density mass function: f (x, y) satisfying f (x, y) ≥ 0 and
+∞
−∞ −∞
f (x, y)dxdy = 1. We can use it to compute probabilities,
Z bZ d
P (a ≤ X ≤ b, c ≤ Y ≤ d) = f (x, y)dydx .
a c

5
 joint cumulative distribution function:
Z x0 Z y0
F (x0 , y0 ) = P (X ≤ x0 , Y ≤ y0 ) = f (x, y)dydx .
−∞ −∞

We have further
∂ 2 F (x, y)
f (x, y) = .
∂x∂y

Marginal distributions. The distribution of each of the components of a

random vector alone is referred to as marginal distribution.

Discrete variables. Given X and Y two discrete random variables with

p(x, y) as joint probability mass function
(marginal) probability mass function of X:
X
pX (x) = P (X = x) = P (X = x, Y = y) .
y

(marginal) probability mass function of Y :

X
pY (y) = P (Y = y) = P (X = x, Y = y) .
x

Continuous variables. Given X and Y two continuous random variables

with f (x, y) as joint density mass function
(marginal) density mass function of X:
Z +∞
fX (x) = f (x, y)dy .
−∞

(marginal) density mass function of Y :

Z +∞
fY (y) = f (x, y)dx .
−∞

Conditional distributions. Distribution of one component given a con-

dition.

6
Discrete variables. Given X and Y two discrete random variables with
p(x, y) as joint probability mass function

probability mass function of Y given X = x0 (pX (x0 ) > 0):

P (X = x0 , Y = y) p(x0 , y)
p(y|x0 ) = P (Y = y|X = x0 ) = = .
P (X = x0 ) pX (x0 )

probability mass function of X given Y = y0 (pY (y0 ) > 0):

P (X = x, Y = y0 ) p(x, y0 )
p(x|y0 ) = P (X = x|Y = y0 ) = = .
P (Y = y0 ) pY (y0 )

Continuous variables. Given X and Y two continuous random variables

with joint density mass function f (x, y)

density mass function of Y given X = x0 (fX (x0 ) > 0):

f (x0 , y)
f (y|x0 ) = .
fX (x0 )

density mass function of X given Y = y0 (fY (y0 ) > 0):

f (x, y0 )
f (x|y0 ) = .
fY (y0 )

Independent random variables. Two random variables are independent

if the fact of knowing the value that one of them assumes does not provide
us with any information about the value that the other one might assume.
More specifically, two random variables X and Y defined on the same
probability space are independent if for all sets of real numbers B1 , B2 , it
holds

P (X ∈ B1 ) ∩ (Y ∈ B2 ) = P (X ∈ B1 )P (Y ∈ B2 ).
Equivalently, X and Y are independent if their joint cumulative distribution
function equals the product of the marginal cumulative distribution func-
tions, that is, F (x, y) = FX (x)FY (y) for all x, y.

7
 Discrete variables: X and Y are independent if for all x, y any of the
following conditions is fulfilled
p(y|x) = pY (y)
p(x|y) = pX (x)
p(x, y) = pX (x)pY (y) .

Continuous variables: X and Y are independent if for all x, y any of

the following conditions is fulfilled
f (y|x) = fY (y)
f (x|y) = fX (x)
f (x, y) = fX (x)fY (y) .

Property. The variance of the sum of two independent random variables

is the sum of their variances. That is, given X and Y independent,
var[X + Y ] = var[X] + var[Y ].

Characteristic features of a random vector A n-dimensional random

vector with components X1 , X2 , . . . , Xn is represented as a column vector,
 
X1
 X2 
X =  ..  .
 
 . 
Xn

Expectation. The mean vector of the random vector X is a column vector

each of whose entries is the expectation of an entry (component) of X,
 
E[X1 ]
 E[X2 ] 
µ = E[X] =  .
 
..
 . 
E[Xn ]
We can compute the expectation of a transformation of a bivariate random
vector (X, Y ) as:
 P P
x R y h(x, y)p(x, y) if X, Y discrete
E[h(X, Y )] = R +∞ +∞ .
−∞ −∞
h(x, y)f (x, y)dxdy if X, Y continuous

8
Covariance. The covariance is a measure of the linear relation between
two variables,
 
Cov(X, Y ) = E (X − E[X])(Y − E[Y ]) = E[XY ] − E[X]E[Y ] .

Properties of the covariance:

if X and Y are independent, Cov(X, Y ) = 0 (E[XY ] = E[X]E[Y ]).
Cov(X, Y ) = 0 does not guarantee that X and Y are independent.
Cov(aX + b, cY + d) = acCov(X, Y ).

Correlation. The correlation is (as well) a measure of the linear relation

between two variables,
Cov(X, Y )
ρ(X, Y ) = p .
Var[X]Var[Y ]
Properties of the correlation:
if X and Y are independent, ρ(X, Y ) = 0.
if ρ(X, Y ) = 0 they are said to be uncorrelated, but it does not guar-
antee that X and Y are independent.
−1 ≤ ρ(X, Y ) ≤ 1.
|ρ(X, aX + b)| = 1.

Covariance matrix. The covariance matrix of a n-dimensional random

vector X is a n × n square matrix given by
 
Var[X1 ] Cov(X1 , X2 ) · · · Cov(X1 , Xn )
 Cov(X1 , X2 ) Var[X2 ] · · · Cov(X2 , Xn ) 
MX = E (X−µ)(X−µ)t = 
 
.
 
.. .. ... ..
 . . . 
Cov(X1 , Xn ) Cov(X2 , Xn ) · · · Var[Xn ]

Variance of a linear combination.

Var[aX + bY ] = a2 Var[X] + b2 Var[Y ] + 2abCov(X, Y ) .