Random Variables (R.V.) : Probability Theory
Random Variables (R.V.) : Probability Theory
1
3. Random variables (r.v.)
Let (O, K, P) a probability field (or probability o-field).
Definition
A function X: OR with the property that: xeR, {eeOX(e)<x}e K is called a
random variable on this field.
Notations
- X(e) is the value of r.v. for event e;
- {eeOX(e)<x}={X<x};
Properties
Consider (O, K, P) a probability field (or probability o-field) and X, Y: OR two
random variables. Then
1) {eeOX(e)>Y(e)}e K;
2) {eeOX(e)sY(e)}e K;
3) {eeOX(e)=Y(e)}e K.
Definition
Consider (O, K, P) a probability field (or probability o-field) and X: OR a random
variable. The function F:RR given by:
F(x)=P({eeOX(e)<x})=P(X<x)
is named cumulative distribution function (c.d.f.) of X.
Properties
The cumulative distribution function F:RR, of a random variable X, has the following
properties:
1) 1 ) ( F lim =
x
x
and lim F( ) 0
x
x
= ;
2) F(x)e[0, 1] xeR;
3) If a<b P(asX<b)=F(b)-F(a)
4) F monotonically nondecreasing function: asbF(a) sF(b);
5) F is left continuous at any x
0
eR: ) ( F ) ( F lim
0
0
0
x x
x x
x x
=
<
Probability Theory
2
3.2 Discrete random variables. Operations with discrete random variables.
Consider (O, K, P) a probability field (or probability o-field) and X: OR a random
variable.
Definition
If the set X(O)={X(e)eeO} is at most countable (that is, a finite or countable set),
then X is a discrete random variable.
Remark
If X is a discrete random variable then X(O) is either finite - X(O)={x
1
, x
2
, , x
n
}- or
countable - X(O)={x
1
, x
2
, , x
n
,}.
Definition
The distribution of a discrete random variable is defined by the formula P(X=x
i
)=p
i
x
i
eX(O) and can be represented by the following distribution table:
X:
|
|
.
|
\
|
... ...
... ...
2 1
2 1
n
n
p p p
x x x
(or equivalently, X:
I e
|
|
.
|
\
|
i
i
i
p
x
with IcN).
Properties
Consider the distribution of a discrete random variable X, given by P(X=x
i
)=p
i
,
ieIcN. Then:
1) p
i
>0, ieI.
2)
eI
i
p
i
=1.
Proof
1) p
i
is a probability, so p
i
>0, ieI.
2) O= } ) X( {
I
i
i
x = e e
e
I
A
e i
i
and A
i
A
j
=u i=j, i, jeI. Then
1=P(O)=P(
I
A
e i
i
)=
eI
) P(A
i
i
=
eI
i
p
i
.
Example
The distribution table of the random variable X representing the face obtained when a
die is rolled is X:
|
|
|
.
|
\
|
6
1
6
1
6
1
6
1
6
1
6
1
6 5 4 3 2 1
.
Probability Theory
3
Remark
The c.d.f. of a discrete random variable having the distribution table X:
I e
|
|
.
|
\
|
i
i
i
p
x
with IcN can
be computed by the formula: F(x)=
< e } {
x x j i
i
j
p .
Definition
Let X, Y: OR two random variables defined on the same probability field (O, K, P). We
say that X, Y are independent iff
P((X<x)(Y<y)) =P(X<x) P(Y<y) (or P(X<x, Y<y)= P(X<x) P(Y<y))
Consider (O, K, P) a probability field (or probability o-field) and X, Y: OR two random
variables. The function Z: OR
2
defined by Z(e)=(X(e), Y(e)) eeO is called
two-dimensional (or bivariate) random variable (or random vector) and the random
variables X, Y are called marginal random variables (notation Z=(X, Y)).
If the random variables X and Y are discrete, then Z=(X, Y) is a discrete two-dimensional
random variable and the set Z(O) is at least countable.
The c.d.f. of Z is defined by:
F(x, y)=P(X<x, Y<y)=P({eeOX(e)<x, Y(e)<y}) (x, y) e R
2
.
Remark
Using the c.d.f. of random vector Z=(X, Y) we can write
X, Y independent iff F
Z
(x, y)=F
X
(x) F
Y
(y).
The distribution function of a discrete two-dimensional random variable Z=(X, Y).
Let X:
I e
|
|
.
|
\
|
i
i
i
p
x
, IcN and Y:
J i
j
j
q
y
e
|
|
.
|
\
|
, JcN the marginal random variables of
Z=(X, Y). We denote A
i
={eeOX(e)=x
i
}, B
j
={eeOY(e)=y
j
},
C
ij
=A
i
B
j
={eeOX(e)=x
i
, Y(e)=y
j
}, p
ij
=P(C
ij
) for every ieI, jeJ.
Remark
- The set (C
ij
)
(i,j)eIJ
is a partition of O.
- It can be proved that the probabilities p
ij
have the following properties:
- p
ij
>0, (i, j)eIJ;
-
eJ j
ij
p =p
i
, ieI and
eI i
ij
p =p
j
, jeJ;
e J I j i
ij
p
) , (
=1.
Probability Theory
4
As a consequence, the random variable Z=(X, Y) can be given by the matrix:
Z:
J I ) , (
) , (
e
|
|
.
|
\
|
j i
ij
j i
p
y x
.
In the case of finite marginal random variables (I=n, J=m), the random variable Z=(X, Y)
is suitable to be represented by the following table:
Y
X
y
1
y
2
y
j
y
m
P(X=x
i
)
x
1
x
2
p
11
...
.....
p
1j
p
1m
.
p
1
p
2
x
i
...
p
ij
p
i
x
n
p
n1
........ p
nj
.. p
nm
p
n
P(Y=y
j
) q
1
q
2
........
q
j
q
m
Remark
The discrete two-dimensional random variables X, Y are independent iff
p
ij
=p
i
q
j
, (i, j)eIJ.
Proposition 1 (operations with random variables)
Consider X : OR a random variables defined on the probability field (or probability o
field) (O, K, P). Then:
1) aX, aeR,
2) X,
3) X
k
, keN*,
4)
X
1
if 0eX(O),
are also random variables.
Probability Theory
5
Proof
1) If a=0 then xeR we have
{eeO(aX)(e)<x}=
< e > e e e
> e < e e e
0 a if , }
a
x
) ( X {
0 a if , }
a
x
) ( X {
K
K
.
If a=0 then {eeO(aX)(e)<x}=
< e
> e u
0 x if ,
0 x if ,
K
K
. Therefore, (aX) is a random variable.
2) For xeR we can write
{eeO X(e)<x}=
dx ) x ( f =1.
3) F(x)=f(x) provided that F(x) exists.
4) P(asX<b)=P(a<Xsb)=P(asXsb)=P(a<X<b)=
}
b
a
dx ) x ( f with a<b.
Exercise 1.
Consider a continuous random variable X with the p.d.f. of given by: f(x)=
e
otherwise , 0
b] [a, x if , k
.
Compute k and the c.d.f. F(x).
Exercise 2.
Consider a continuous random variable X with the p.d.f. of given by: f(x)=
e
otherwise , 0
3] [0, x if ,
2
kx
.
Compute k, c.d.f. F and P(1<X<2).
3.4. Numerical values associated to random variables.
Definition (expectation, mean value, expected value)
Consider X: OR a random variable on a probability field (or probability o field).
The expectation of r. v. X, denoted E(X), is defined by:
Probability Theory
7
- E(X)=
eI i
i i
p x if X:
I i
i
i
p
x
e
|
|
.
|
\
|
a discrete random variable;
- E(X)=
}
dx ) x ( xf if X: R e
|
|
.
|
\
|
x ,
) x ( f
x
is a continuous random variable.
Properties (expectation)
Consider X, Y: OR discrete random variables defined on the same probability field.
1) E(a)=a, aeR.
2) E(aX)=aE(X), aeR.
3) E(X+Y)=E(X)+E(Y).
4) E(XY)=E(X)E(Y) if X, Y are independent.
Proof
1) We have a:
|
|
.
|
\
|
1
a
, aeR, so E(a)=a1=a.
2) If X:
I i
i
i
p
x
e
|
|
.
|
\
|
a discrete random variable, then aX:
I i
i
i
p
ax
e
|
|
.
|
\
|
so
E(aX)=
eI i
i i
p ax =a
eI i
i i
p x =aE(X).
If X: R e
|
|
.
|
\
|
x ,
) x ( f
x
is a continuous random variable, then aX: R e
|
|
.
|
\
|
x ,
) x ( f
ax
so
E(aX)=
}
dx ) x ( axf =a
}
dx ) x ( xf =aE(X).
We prove the properties 3 and 4 only for discrete case.
3) Let X, Y: OR discrete random variables, X:
I i
i
i
p
x
e
|
|
.
|
\
|
, Y:
J j
j
j
q
y
e
|
|
.
|
\
|
, I, JcN and
X+Y:
J I ) j , i (
ij
j i
p
y x
e
|
|
.
|
\
| +
. Then we can write
E(X+Y)=
e
+
J I ) j , i (
ij j i
p ) y x ( =
e
+
J I ) j , i (
ij j ij i
p y p x =
e e I i J j
ij i
) p x ( +
e e J j I i
ij j
) p y ( =
=
eI i
i i
p x +
eI j
j j
q y =E(X)+E(Y).
Probability Theory
8
4) Let X, Y: OR independent discrete random variables, X:
I i
i
i
p
x
e
|
|
.
|
\
|
, Y:
J j
j
j
q
y
e
|
|
.
|
\
|
,
I, JcN and XY:
J I ) j , i (
j i
j i
q p
y x
e
|
|
.
|
\
|
. Therefore E(XY)=
e J I ) j , i (
j i j i
q p y x
=
e J I ) j , i (
j j i i
) q y )( p x ( =
eI i
i i
) p x (
eJ j
j j
) q y ( =E(X)E(Y).
Definition
The r
th
(initial) moment of a random variable X is defined by:
M
r
=E(X
r
)=
e
|
|
.
|
\
|
|
|
.
|
\
|
}
e
e
r.v. continuous , x ,
) (
: X if ) (
variable random discrete
p
x
: X if
i
i
R
x f
x
dx x f x
p x
r
I i
I i
i
r
i
, with reN*.
It is obvious that m
1
=E(X).
Another important typical value of a random variable is the variance, which measures the
dispersion of the values of X around its mean.
Definition (variance)
The variance of a random variable X: OR is defined by:
Var(X)=E[(X-E(X))
2
].
Remark
If E(X
2
)< then the variance of r. v. X can be computed as follows:
Var(X)=E(X
2
)-E
2
(X)
because Var(X)=E[(X-E(X))
2
]=E[X
2
-2XE(X)+E
2
(X)]=E(X
2
)-2E(X)E(X)+E(E
2
(X))=
=E(X
2
)-2E
2
(X)+E
2
(X)=E(X
2
)-E
2
(X).
Properties (variance)
Consider X, Y: OR, random variables defined on the same probability (o) field
1) Var(a)=0, aeR ;
2) Var(aX)=a
2
Var(X), aeR.
3) Var(XY)=Var(X)+Var(Y) if X, Y are independent random variables.
Probability Theory
9
Proof
1) Var(a)=E(a
2
)-E
2
(a)=a
2
-a
2
=0.
2) Var(aX)=E(a
2
X
2
)-E
2
(aX)=a
2
E(X
2
)-(aE(X))
2
=a
2
[E(X
2
)-E
2
(X)]=a
2
Var(X).
3) As X, Y are independent r. v. we have E(XY)=E(X)E(Y) (*).
Var(XY)=E[(XY)
2
]-E
2
(XY)= E(X
2
+Y
2
2XY)-(E(X) E(Y))
2
=
=E(X
2
)+E(Y
2
) 2E(XY)-(E
2
(X)+ E
2
(Y) 2E(XY))=
=E(X
2
)-E
2
(X)+E(Y
2
)-E
2
(Y) 2[E(XY)-E(X)E(Y)]= Var(X)+Var(Y) from (*).
Definition
Consider X: OR a random variable. The central moment of r
th
order of X is defined by:
r
=E[(X-E(X))
r
] with reN*.
The square root of Var(X) is called the standard deviation of X and we write: o
X
= ) X ( Var .
Remark
1)
1
=0;
2)
2
=Var(X).
Definition (central moment)
Consider X, Y:OR univariate random variables defined on the probability field (or
probability o - field) (O, K, P) and Z=(X, Y) bivariate (two-dimensional) random variable.
The (r, s) order central moment of Z is defined by:
r,s
=E[(X-E(X))
r
(Y-E(Y))
s
]
If Z=(X,Y) is discrete, X:
I i
i
i
p
x
e
|
|
.
|
\
|
, Y:
J j
j
j
q
y
e
|
|
.
|
\
|
, I, JcN then
r,s
=
e
J I ) j , i (
ij
s
j
r
i
p )) Y ( M y ( )) X ( M x ( where p
ij
=P(X=x
i
, Y=y
j
), (i, j)eIJ.
Particular cases
Let Z=(X, Y) bidimensional random variable then:
1)
2,0
=E[(X-E(X))
2
]=Var(X)
2)
0,2
=E[(Y-E(Y))
2
]=Var(Y)
3)
1,1
=E[(X-E(X))(Y-E(Y))]
not
= cov(X, Y) covariance or random variables X and Y.
Probability Theory
10
Properties (covariance)
1) cov(X, Y)=E(XY)-E(X)E(Y);
2) X, Y independent cov(X, Y)=0;
3) cov(X, Y)=cov(Y, X)
4) cov(X, X)=Var(X).
5) cov(X, a)=0 aeR
6) cov(aX, bY)= ab cov(X, Y)
7) cov(X+X, Y)=cov(X, Y)+cov(X, Y).
Consequence
Var(XY)=Var(X)+Var(Y)2cov(X, Y)
Definition (correlation coefficient)
Consider X, Y:OR random variables defined on the probability field (or probability o field)
(O, K, P). The correlation coefficient of random variables X and Y is defined by
(X, Y)=
Y X
Y) , X cov(
o o
=
) Y ( Var ) X ( Var
) Y ( M ) X ( M ) XY ( M
if o
X
, o
Y
=0.
Properties (correlation coefficient)
1) X, Y independent random variables (X, Y)=0;
2) (X, Y)e[-1, 1], X, Y;
3) If Y=aX+b (X, Y)=1;
4) If o
X
=0 then (X, X)=1 and (X, -X)=-1.
Exercise
Consider the discrete bidimensional random variable Z=(X, Y) given by the table:
Probability Theory
11
a)
Y
X
0 y p
i
x a
12
1
b
1
2
1
c
3
2
q
j
4
3
d
b)
Y
X
0 y p
i
x a
6
1
b
1
12
7
c
3
2
q
j
4
3
d
Find x, y a, b, c, d such as E(X)=
3
1
, E(Y)=
4
1
and for the obtained values, compute X, Y, the
cdf F
X
and F
(X, Y)
(0, 2), XY, cov(X, Y), (X, Y), o
X
, o
Y
.
Probability Theory
12
3.5 Several Classical Distributions
3.5.1 Binomial distribution
We say that a random variable X: OR follows a binomial distribution of parameters n and
p, and we write XeBi(n, p) if P(X=k)=
k
n
C p
k
(1-p)
n-k
, k= n , 0 or X:
|
|
.
|
\
|
k n k k
n
q p C
k
, k= n , 0 ,
q=1-p, pe[0, 1].
Remarks
1) p
k
=
k
n
C p
k
(1-p)
n-k
>0 k= n , 0 .
2)
=
n
0 k
k n k k
n
q p C =(q+p)
n
=1
Expectation: E(X)=np
Variance: Var(X)=npq
3.5.2 Poisson distribution
A random variable X: OR is said to be Poisson distributed with parameter >0
- we write XeP()) iff P(X=k)=e
-
! k
k
, keN , or X:
N e
|
|
|
.
|
\
|
k
k
! k
e
k
.
Remarks
1) p
k
=P(X=k)= e
-
! k
k
>0, keN.
2)
e
e
=
N N k
k
k
k
! k
e p = e
-
e
=1.
Expectation: E(X)=
XeP() E(X)=
>
0 k
k
! k
ke =e
-
>
1 k
k
! k
k = e
-
>
1 k
1 k
)! 1 k (
= e
-
>
0 k
k
! k
= e
-
e
=.
Variance: Var(X)=
Probability Theory
13
E(X
2
)=
>
0 k
k
2
! k
e k = e
-
>
1 k
k
)! 1 k (
k = e
-
>
1 k
1 k
)! 1 k (
k =
= e
-
(
>
1 k
1 k
)! 1 k (
) 1 k ( +
>
1 k
1 k
)! 1 k (
)= e
-
(
>
2 k
1 k
)! 2 k (
+ e
)= e
-
( e
+ e
)=(+1).
Hence Var(X)=E(X
2
)-E
2
(X)= (+1)-
2
=.
3.5.3 Hypergeometric distribution
A random variable X: OR has a hypergeometric distribution of parameters n, a and b (it is
written XeH(n, a, b)) iff P(X=k)=
k n-k
a b
n
a+b
C C
C
, ke
0
0, n where a, b, neN,
0
n min{ , n} a = and
ns a+b (or X:
0
k n-k
a b
n
a+b
k 0, n
k
C C
C
e
| |
|
|
|
\ .
).
Remarks
1) p
k
=P(X=k)=
k n-k
a b
n
a+b
C C
C
>0, ke
0
0, n .
2)
0 0 0
n n n
k n-k
a b k n-k n
k a b a+b
n n n
a+b a+b a+b
k=0 k=0 k=0
C C 1 1
p C C C 1
C C C
= = = =
.
Expectation: E(X)= n
a
a b +
Variance: Var(X)= n
1
a b a b n
a b a b a b
+
+ + +
.
3.5.4 Normal distribution
We say that a random variable X: OR follows a normal distribution of parameters m and o
- we write XeN(m, o) - if its p.d.f. has the form:
f(x)=
2
2
2
) m x (
e
2
1
o
t o
, meR, o>0.
Remarks
1) f(x)>0 xeR.
Probability Theory
14
2)
}
dx ) x ( f =
t o 2
1
}
dx e
2
2
2
) m x (
=
t o 2
1
}
o dt 2 e
2
t
=
t
1
}
dt e
2
t
=2
t
1
}
0
t
dt e
2
=
=2
t
1
2
t
=1 where (x-m)=to 2 .
Expectation: E(X)=m
E(X)=
}
dx ) x ( xf =
t o 2
1
}
dx xe
2
2
2
) m x (
t
2
m x
=
o
=
t o 2
1
}
o + o dt 2 e ) m t 2 (
2
t
=
=
t o 2
1
(o 2 )
2
}
dt te
2
t
+m
t o 2
1
}
dt e
2
t
=m
Variance: Var(X)=o
2
Var(X)=
}
dx ) x ( f ) m x (
2
=
}
t o
dx e
2
1
) m x (
2
2
2
) m x (
2
=
=
t o 2
1
}
dx e ) m x (
2
2
2
) m x (
2
t
2
m x
=
o
=
t o 2
1
}
o o dt 2 e t 2
2
t 2 2
=
t
o
2
}
dt )' e ( t
2
t
= =
t
o
2
(t
2
2
t
e
-
}
dt e
2
t
)=
t
o
2
2
}
0
t
dt e
2
=
t
o
2
2
2
t
=o
2
.
Standard deviation (o
X
=o)
The c.d.f. values of a normal distributed random variable are computed using Laplace
function defined by: | :RR, |(z)=
t 2
1
}
z
0
t
2
1
dt e
2
.
Properties
1) |(0)=0;
2) |(-z)=- |(z) (odd function);
3) ) z ( lim
z
|
=0.5;
4) ) z ( lim
z
|
=-0.5;
Probability Theory
15
Cumulative distribution function (F(z)=
2
1
+|(
o
m z
))
F(z)=
}
o
t o
z
2
) m x (
dx e
2
1 2
2
t
m x
=
o
=
t o 2
1
}
o
o
m z
t
2
1
dt e
2
=
=
t 2
1
}
0
t
2
1
dt e
2
+
t 2
1
}
o
m z
0
t
2
1
dt e
2
u
2
t
=
=
t 2
1
}
0
u
du ) 2 ( e
2
+|(
o
m z
)=
=
t
1
}
0
u
du e
2
+|(
o
m z
)=
2
1
+|(
o
m z
).
Remark
It is easy to prove that F(-z)=1-F(z), zeR.
The most important normal distribution is the standard normal N(0,1), with m=0 and o
2
=1:
- we write XeN(0, 1) - if its p.d.f. has the form: f(x)=
2
2
2
1
x
e
t
, xeR
3.5.5 Gamma distribution
We say that a random variable X: OR follows a gamma distribution of parameters a and b
- we write Xe(a, b) - if its p.d.f. has the form:
f(x)=
1
1
, x>0
( )
0 , x 0
x
a
b
a
x e
a b
, a, b>0.
Remarks
1) f(x)>0 xeR.
2)
}
dx ) x ( f =
a
b ) a (
1
I
}
0
b
x
1 a
dx e x
t
b
x
=
=
a
b ) a (
1
I
}
0
t 1 a
bdt e ) bt ( =
a
b ) a (
1
I
}
0
t 1 a a
dt e t b =
=
a
b ) a (
1
I
b
a
I(a)=1
Expectation: E(X)=ab
Probability Theory
16
E(X)=
}
dx ) x ( xf =
a
b ) a (
1
I
}
0
b
x
a
dx e x
t
b
x
=
=
a
b ) a (
1
I
}
0
t a
bdt e ) bt ( =
) a (
b
I
}
0
t a
dt e t =
=
) a (
b
I
I(a+1)=
) a (
b
I
aI(a)=ab.
Variance: Var(X)=ab
2
E(X
2
)=
a
b ) a (
1
I
}
+
0
b
x
1 a
dx e x
t
b
x
=
=
a
b ) a (
1
I
}
+
0
t 1 a
bdt e ) bt ( =
) a (
b
2
I
}
+
0
t 1 a
dt e t =
=
) a (
b
2
I
I(a+2)=
) a (
b
2
I
a(a+1) I(a)=ab
2
(a+1)
Var(X)= ab
2
(a+1)-a
2
b
2
=ab
2
.
3.5.6 Exponential distribution
We say that a random variable X: OR follows a negative exponential distribution of
parameter >0 - we write XeExp() - if its p.d.f. has the form: f(x)=
s
>
0 x , 0
0 x , e
x
.
Remark
XeExp()XeI(1,
1
) so E(X)=
1
and Var(X)=
2
1
.
3.5.7 Beta distribution
We say that a random variable X: OR follows a Beta distribution of parameters a and b
and we write Xe(a, b), if its pdf is:
f(x)=
, a, b>0.
Show that the expectation and variance of X are E(X)=
b a
a
+
and Var(X)= .
) 1 ( ) (
2
+ + + b a b a
ab
Probability Theory
17
3.5.8 Chi-square distribution
We say that a random variable X: OR follows a chi-square distribution of parameters n
and o - we write Xe_
2
(n, o) iff Xe(
2
n
, 2o
2
) . Then the p.d.f. of X is:
f(x)=
2
1
2 2
2
2
1
, x>0
( )(2 )
2
0 , x 0
x n
n
x e
n
o
I o
+ , xeR, neN\{0,1,2}.
It can be proved that E(X)=0 and Var(X)=
2 n
n