Probability

[email protected]
Office 2-B14 (Edificio Politécnico)

http://areaestadistica.uclm.es
http://areaestadistica.uclm.es/oed

Probability–[email protected]
Outline

1. Basic properties.

2. Random Variables.

3. Remarkable distributions.

Probability–[email protected]
Introduction

I 17th century: Pascal and Fermat... correspondence

(but 16th century: Cardano... probabilities of rolling a die)

I 18th century: Bernoulli applied infinitesimal calculus, De

Moivre introduced normal distribution and Central Limit
Theorem

I ... Laplace, Gauss, Newton...

I 20th century: Kolmogorov established the probability axioms

Probability–[email protected]
Concepts

I Experiment: unknown outcome beforehand (but we know

possible outcomes).

I Random phenomena vs deterministic.

Tossing a coin, quality control, atmospheric temperature ...

I Sample space, Ω.
For a die, Ω = {1, 2, 3, 4, 5, 6}.

I Event, subset from Ω.

Probability–[email protected]
Events

I Elementary event, w .

I Impossible event, φ.

I Certain event, Ω.

I Complementary event, AC .

I Union event, A ∪ B.

I Intersection event, A ∩ B = AB.

I Mutually exclusive event, AB = φ.

Probability–[email protected]
Definition

I Laplace rule (elementary event equally likely)

favourable events
p=
possible events
I Frequentism (Empirical probability)

fn n→∞
pn = −→ p
n
I Subjectivism (Bayesian)... Bets

Probability–[email protected]
Properties

1. P(A) ≥ 0, ∀A ⊂ Ω
2. P(Ω) = 1
⊂ Ω mutually exclusive 2 by 2
3. A1 , A2 , A3 , . . .P
P(∪i=1 Ai ) = ∞
∞
i=1 P(Ai )

4. P(AC ) = 1 − P(A) ⇒ P(φ) = 0

5. 0 ≤ P(A) ≤ 1, ∀A ⊂ Ω
6. P(A ∪ B) = P(A) + P(B) − P(AB)

Exerc.: Probability of at least an even number in 2 tosses of a die?

Probability–[email protected]
Conditional probability

P(AB)
Additional information: B occurred, P(B) > 0, P(A|B) =
P(B)

Exerc.: For a die P(0 40 |0 even0 )?

Probability–[email protected]
Total Probability Theorem

I Product Rule: If P(A1 . . . An−1 ) > 0 then

P(A1 . . . An ) = P(A1)P(A2|A1) . . . P(An |A1 . . . An−1 )

I Total Probability Theorem:

Let A1 , A2 , . . . , An be a partition of Ω, with P(Ai ) > 0

P(B) = P(B|A1 )P(A1 ) + . . . + P(B|An )P(An )

Probability–[email protected]
Bayes Theorem

P(B|Ak )P(Ak )
P(Ak |B) = Pn
i=1 P(B|Ai )P(Ai )

Independent events:
P(A|B) = P(A) ⇔ P(B|A) = P(B) ⇔ P(AB) = P(A)P(B)

Probability–[email protected]
Example

2 boxes with 100 and 30 balls respectively.

In the first there are 60 red balls and 10 in the second.
1. A ball is chosen, probability of being red?

2. If the selected ball is red, probability that it comes from the

first box?

Probability–[email protected]
Random variables, concepts

Random variable (r.v.): X : Ω → R

w → X (w )
Probability distribution: P(X ∈ A) = P({w ∈ Ω|X (w ) ∈ A})

Examples:
X Distribution P
0(non defective) or 1(defective) Bernoulli P(X=1)=p
Number of defectives in n items Binomial P(X=k)=...
Number from a die rolled Uniform P(even)=3/6
Temperature at 9am in a place P(T> 50o C)≈ 0

Probability–[email protected]
Cumulative distribution function, cdf

F (x) = P(X ≤ x)
Properties:
1. 0 ≤ F (x) ≤ 1
2. ‘Increasing’ (non decreasing)
3. limx→−∞ F (x) = 0, limx→+∞ F (x) = 1
4. P(X > x) = 1 − F (x)
5. P(a < X ≤ b) = F (b) − F (a)
6. Right-continuous

Probability–[email protected]
Probability distribution function, pdf

X discrete:
f (x) = P(X = x)
X (absolutely) continuous:
Z b
∃f (x)|P(a < X < b) = f (x)dx
a

Properties:
1. f (x) ≥ 0, ∀x
P R∞
2. (disc) x f (x) = 1 = −∞ f (x)dx (cont)
3. X continuous, P(X = x) = 0, ∀x ⇒ P(X < a) = P(X ≤ a)
dF (x)
4. X continuous, f (x) =
dx

Probability–[email protected]
Examples/Exercises

I cdf and pdf for the distribution

of the outcomes of a die, D

I cdf and pdf for the uniform

continuous distribution, U[0, 1]

I P(1/3 ≤ U ≤ 2/3)?

cx ,0 ≤ x ≤ 4
I f (x) =
0 , otherwise
I c?
I P(1 ≤ X ≤ 2)?
I P(X ≥ 2)?

Probability–[email protected]
Bivariate distributions

(X,Y) random vector

I Both discrete: f (x, y ) = P(X = x, Y = y )

X /Y y1 y2 ... 
x1 p11 p12 . . . p1·  MarginalP
 distribution of X:
x2 p21 p22 . . . p2· f1 (xi ) = yj f (xi , yj )
.. .. .. ..  | {z } | {z }
. . . . pi· pij

p·1 p·2 ... 1
P
Marginal distrib. of Y: p·j = f2 (yj ) = xi f (xi , yj )

RR
I Both continuous: P((X , Y ) ∈R R) = R f (x, y )dxdy
∞
Marginal distrib.: f1 (x) = −∞ f (x, y )dy ...

Probability–[email protected]
Examples/Exercises

X /Y 1 2 3 4
1 0.1 0 0.1 0 0.2
2 0.3 0 0.1 0.2 0.6
3 0 0.2 ? 0 ?
0.4 0.2 ? 0.2 ?

I P(X ≥ 2, Y ≥ 2) = ...

I f1 (1) = P(X = 1) = ...

I P(X = 2|Y = 4) conditional!

Probability–[email protected]
Conditional distributions

f (x, y )
I If f2 (y ) 6= 0 ⇒ f12 (x|y ) =
f2 (y )
f (x, y )
I If f1 (x) 6= 0 ⇒ f21 (y |x) =
f1 (x)
I Discrete case:
P(X = x, Y = y ) f (x, y )
f12 (x|y ) = P(X = x|Y = y ) = =
P(Y = y ) f2 (y )
I X and Y independent variables ⇐⇒ f (x, y ) = f1 (x)f2 (y ), ∀x, y

I Ex.(cont.):
I f12 (2|4) = ...
I f12 (3|3) = ...
I f21 (4|1) = ...
I Are X and Y independent variables?

Probability–[email protected]
Moments

Mathematical Expectation:
 P
xf (x) Discrete case
µ = E (X ) = R ∞x
−∞ xf (x)dx Continuous case

Variance and Standard Deviation:

σ 2 = Var (X ) = E [(X − µ)2 ] = E (X 2 ) − E 2 (X ) > 0

√
σ = σ2
Covariance and Correlation:

σXY = Cov (X , Y ) = E [(X −µx )(Y −µy )] = E (XY )−E (X )E (Y ) if exists

σXY
ρXY = Corr (X , Y ) = ∈ [−1, 1]
σX σY

Probability–[email protected]
Moments Properties

I Expectation:
1. does not exist always (divergent sum or integral)
R∞
2. E (r (X )] = −∞ r (x)f (x)dx

3. Linear: E (aX + b) = aE (X ) + b

4. If P(a ≤ X ≤ b) = 1 ⇒ a ≤ E (X ) ≤ b

5. E (X1 + . . . + Xn ) = E (X1 ) + . . . + E (Xn )

6. X1 , . . . , Xn indep.vars. ⇒ E (X1 . . . Xn ) = E (X1 ) . . . E (Xn )

Probability–[email protected]
Moments Properties

I Variance
1. P(X = c) = 1 ⇐⇒ Var (X ) = 0

2. Var (a + bX ) = b 2 Var (X )

3. X1 , . . . , Xn indep.vars.
⇒ Var (X1 + . . . + Xn ) = Var (X1 ) + . . . + Var (Xn )

Probability–[email protected]
Moments Properties

I Covariance and correlation

1. X and Y independent variables ⇒ ρXY = 0
:

2. Var (aX + bY + c) = a2 Var (X ) + b 2 Var (Y ) + 2abCov (X , Y )

P P P
3. Bilinear: Cov ( i ai Xi , j bj Yj ) = ij ai bj Cov (Xi , Yj )

Probability–[email protected]
Exercises


 0.1 ,x = −2
0.4 ,x =0

I X with pdf f (x) =

 0.3 ,x =1
0.2 ,x =4

I E (X ) = 0.9 = ...
I Var (X ) = ...
I Var (4X − 7) = ...

2x ,0 < x < 1
I X with pdf f (x) =
0 , otherwise
I E (X ) = 2/3 = ...
I Var (X ) = ...

I X1 , X2 , X3 independent variables, E (Xi ) = 0 and E (Xi2 ) = 1,

E [X12 (X2 − 4X32 )] = 17 = ...

Probability–[email protected]
Inequalities

I Markov’s inequality:
E (X )
If P(X ≥ 0) = 1 and E (X ) exits ⇒ ∀t > 0, P(X ≥ t) ≤
t

I Chebyshev’s inequality:
Var (X )
If Var (X ) exits ⇒ ∀t > 0, P(|X − E (X )| ≥ t) ≤
t2
I Application: Sample size computations.
X1 , . . . , Xn independent and identically distributed random sample
X1 + . . . + Xn σ2
X = ... ⇒ P(|X − µ| ≥ t) ≤ 2
n nt

Probability–[email protected]
Remarkable discrete distributions
Name Notation pdf E (X ) = µ
Values f (xi ) Var (X ) = σ 2
1 Pn
Uniform U[{x1 , x2 , . . . , xn }] 1/n xi
n i=1
1 Pn
x1 , x2 , . . . , xn x 2 − µ2
n i=1 i
Rolling a die ...
Bernoulli b(p) f (1) = p p
0,1 pq
Tossing a coin, Quality
control (defective, non)...
n x n−x
Binomial B(n, p) x p q np
0,1,. . .,n npq
n independent Bernoulli’s experiments
e −λ λx
Poisson P(λ) λ
x!
0,1,. . . λ
Daily phone calls, N of defectives in 1 m2 ...
o

Probability–[email protected]
Relationships

I b(p) ∼ B(1, p)

I P(λ)
I X ∼ P(λ) ⇒ aX ∼ P(aλ)
I X1 , . . . , Xk independent with P(λi )
⇒ X1 + . . . + Xk ∼ P(λ1 + . . . + λk )

I B(n, p) ∼ P(np)
if n is sufficiently large and p is sufficiently small
(Rule of thumb: n ≥ 100 and np ≤ 20)

Probability–[email protected]
Remarkable continuous distributions
Name Notation pdf µ(mean)
Values f (xi ) σ 2 (variance)
1
Uniform U[a, b] (a + b)/2
b−a
x ∈ [a, b] (b − a)2 /12
1 1 (x − µ)2
Normal N(µ, σ 2 ) √ exp{− } µ
2πσ 2 2 σ2
(Gaussian) x ∈R σ2
βα
Gamma γ(α, β) x α−1 e βx α/β
Γ(α)
x >0 α/β 2
Time to failure...
Exponential Exp(β) ∼ γ(1, β)
Γ(α + β) α−1 α
Beta B(α, β) x (1 − x)β−1
Γ(α)Γ(β) α+β
αβ
x ∈ [0, 1] 2
(α + β) (α + β + 1)
Probability–[email protected]
Standard normal distribution

I X ∼ N(µ, σ 2 ) ⇒ aX + b ∼ N(aµ + b, a2 σ 2 )
In particular,
X −µ
∼ Z ∼ N(0, 1)
σ
1 2
Z has pdf f (z) = √ e −z /2 and cdf F (z) (tables).
2π

I X1 , . . . , Xn independent
⇒ X1 + . . . + Xn ∼ N(µ1 + . . . + µn , σ12 + . . . + σ22 ).
In particular,
X ∼ N(µ, σ 2 /n)
.

Probability–[email protected]
Montecarlo simulations

Let F the cdf of any continuous r.v. X and Y = F (X ) ∈ [0, 1].

cdf of Y?
∗
FY (y ) = P(Y ≤ y ) = P(X ≤ x0 ) = F (x0 ) = y
* ∃x0 |F (x0 ) = y

fY (y ) = FY0 (y ) = 1 ⇒ Y ∼ U[0, 1]!!

Generating values of Y ∼ U[0, 1] and turning with FX−1 (y ),

values of X are generated.

Probability–[email protected]
Central Limit Theorem

X1 , . . . , Xn random sample with µ and σ 2

X −µ L
√ −→ N(0, 1)
σ/ n n→∞

i.e.
X ∼ N(µ, σ 2 /n)

Example: Random sample of 900 items, with 50% chance to come

from A provider. What is the probability that more than 495 items
come from A provider?

Probability–[email protected]
Sample distributions

1. Chi-squared (Pearson): Z1 , . . . , Zn r.s. from a N(0,1)

X = Z12 + . . . + Zn2 ∼ χ2n

I χ2n ∼ γ(α = n/2, β = 1/2)

I Fisher Theorem: X1 , . . . , Xn r.s. from a Nµ, σ 2 ), then
i) X ∼ N(µ, σ 2 /n)
ii) (n − 1)Sc2 /σ 2 ∼ χ2n−1
iii) X and Sc2 are independent r.v.

Probability–[email protected]
Sample distributions (cont)

2. Student’s t: Z ∼ N(0, 1), Y ∼ χ2n independent

Z
T =p ∼ tn
Y /n

n
I E (T ) = 0(n > 1) and Var (T ) = (n > 2)
n−2
I Theorem: X1 , . . . , Xn r.s. from a Nµ, σ 2 ),

X −µ
T = √ ∼ tn−1
Sc / n

Probability–[email protected]
Sample distributions (cont)

3. Snedecor’s F: Y1 ∼ χ2m and Y2 ∼ χ2n independent,

Y1 /m
X = ∼ Fm,n
Y2 /n

2
n>2 n n>4 n (2m + 2n − 4
I E (X ) = and Var (X ) =
n−2 m(n − 2)2 (n − 4)
1
I X ∼ Fm,n ⇒ ∼ Fn,m
X
2
I X ∼ tn ⇒ X ∼ F1,n

Probability–[email protected]