Econ6034-Econometrics and Business

Statistics
Topic 3: Probability and Random Variables

1 / 63
Probability
(Ref: Keller Chapter 6-1, 6-2, 6-3)

2 / 63
Terminology

I A random experiment is a process that results in a number of

possible outcomes, none of which can be predicted with
certainty.

I Examples
Rolling a die: Possible outcomes are 1, 2, 3, 4, 5, or 6;
Tossing a coin: Possible outcomes are heads or tails.

3 / 63
Terminology

I The sample space of a random experiment is a list of all

possible outcomes.
I Outcomes must be mutually exclusive and exhaustive.
Mutually exclusive: No two outcomes can both occur at the
same time on any one trial.
Exhaustive: All possible outcomes must be included in the
sample space.
I For example, when rolling a die, the sample space is:
S=1, 2, 3, 4, 5, 6.

4 / 63
Terminology

I A simple event is an individual outcome in a sample space.

When rolling a die, a simple event is the event that number 1
comes up.

I An event is a collection of one or more simple (individual)

events.
When rolling a die, event A is the event that an odd number
comes up. Then A=1, 3, 5.

5 / 63
Two Important Rules

I In general:
The sample space, S = {E1 , E2 , ..., En } where there are n
possible outcomes; and,

The probability of event Ei occurring on a single trial is written

as P (Ei).

I Two Important Rules:

1. 0 ≤ P (Ei ) ≤ 1 for all i

n
X
2. P (Ei ) = 1
i=1

6 / 63
Joint, Marginal and Conditional Probability

I We study methods to determine probabilities of events that

result from combining other events in various ways.

I There are several types of combinations and relationships

between events:

Complement event
Intersection of events
Union of events
Mutually exclusive events
Dependent and independent events

7 / 63
Probability of an Event

I The complement of event A is the set of all outcomes that are

‘not in A0 .

¯
I P (Ac ) = P (A)=P(A complement)=P(A does not occur)

8 / 63
Probability of Combined Events

Consider two events, A and B.

I Union of Events
P(A or B)=P(AUB)=P(A
union with B)
=P(A occurs, or B occurs,
or both occur)

9 / 63
Probability of Combined Events
Consider two events, A and B.
I Union of Events Figure: Union of events
P(A or B)=P(A ∪ B)=P(A
union with B)
=P(A occurs, or B occurs, or
both occur)
I Intersection of Events
P(A and B) =P(A ∩ B) = P(A
intersection with B)
= P(A and B both occur)
I Dependent Events Figure: Intersection of events
P(A|B)=P(A occurs given that
B has occurred)

10 / 63
Probability of Mutually Exclusive Events

I If (A and B)=∅ (the null or empty set)

I A and B are mutually exclusive, then P(A and B)=0.

Example: if A is the event that we roll a 1, and B is the event

that we roll a 2, then A and B together cannot happen – they
are mutually exclusive.
I P(A and B) = 0

11 / 63
Joint, marginal and conditional probability
Example

Suppose we are investigating the relationship between how well the

mutual fund performs and where the fund managers earns his or her
MBA. Let events be known as follows:

A1=Fund manager graduated from a top-20 MBA program

A2=Fund manager did not graduate from a top-20 MBA program
B1=Fund outperforms the market
B2=Fund does not outperform the market

12 / 63
 Probabilities Mutual Fund outperforms Market Mutual Fund Does not outperform Market
Top-20 MBA 0.11 0.29
Not Top-20 MBA 0.06 0.54

E.g. P( Mutual fund outperforms AND top-20 MBA)

= P(A1 AND B1)= 0.11

E.g. P( Mutual fund not outperform AND top-20)

=P(A1 AND B2)=0.29

13 / 63
Marginal Probabilities

Probabilities Mutual Fund outperforms Market Mutual Fund Does not outperform Market Total
Top-20 MBA P (A1 and B1 )=0.11 P (A1 and B2 )=0.29 P (A1 ) = 0.40
Not Top-20 MBA P (A2 and B1 )=0.06 P (A2 and B2 )=0.54 P (A2 ) = 0.60
Total P (B1 ) = 0.17 P (B2 ) = 0.83 1.00

The Marginal Probability of A1 is the probability of event A1

occurring regardless of B1 and B2 .

Marginal probabilities: P (A1 ), P (A2 ), P (B1 ), P (B2 )

P (A1 ) = P (A1 and B1 ) + P (A1 and B2 ) = 0.11 + 0.29 = 0.40
P (A2 ) = P (A2 and B1 ) + P (A2 and B2 ) = 0.06 + 0.54 = 0.60
P (B1 ) = P (B1 and A1 ) + P (B1 and A2 ) = 0.11 + 0.06 = 0.17
P (B2 ) = P (B2 and A1 ) + P (B2 and A2 ) = 0.29 + 0.54 = 0.83

14 / 63
Conditional Probability

I Conditional probability: the probability of one event given the

occurrence of another event.

I Bayes’ Theorem:

P(A and B)
P(A|B) =
P(B)

15 / 63
Conditional Probability Example

I What is the probability that a mutual fund with a manager who

graduated from a top-20 MBA program outperforms market?

P (A1 and B1 ) 0.11

P (B1 |A1 ) = = = 0.275
P (A1 ) 0.4

I For a randomly selected mutual fund which underperforms the

market, What is the probability that a graduate of a top- 20
MBA program manage it?

P (A1 and B2 ) 0.29

P (A1 |B2 ) = = = 0.3494
P (B2 ) 0.83

16 / 63
Independence

I Two events are independent if

P(A|B) = P(A) or P(B|A) = P(B)

In other words, two events are independent if the probability of

one event is not affected by the occurrence of the other event.

Example:
A1=Fund manager graduated from a top-20 MBA program
B2=Fund does not outperform the market

P (A1 and B2 ) 0.29

P (A1 |B2 ) = = = 0.3494
P (B2 ) 0.83

P (A1 ) = P (A1 and B1 ) + P (A1 and B2 ) = 0.11 + 0.29 = 0.40

17 / 63
Probability Rules

Three rules that enable us to calculate the probability of more

complex events from the probability of simpler events:

1. The Complement Rule

2. The Multiplication Rule
3. The Addition Rule

18 / 63
Complement Rule

Given an event A and its complement Ā,

P (Ā) = 1 − P (A).

Interpretation: either A happens, or it doesn’t.

19 / 63
Multiplication Rule

I The joint probability of two independent events are

P (A and B) = P (A) × P (B)

I The joint probability of two events when they are not

independent:

P (A and B) = P (B|A) × P (A) = P (A|B) × P (B)

Examples:
What is the probability of obtaining a Head on both of two
consecutive tosses of a coin?
1 1 1
P (H ∩ H) = P (H) × P (H) = × =
2 2 4
What is the probability of obtaining a double-six when rolling
two dice?

20 / 63
Multiplication Rule (not independent events)

An econometric course has 7 male and 3 female students. The

lecturer wants to select two students at random to be his research
assistants. What is the probability that the two students chosen are
female?

Solution: Let
A = the first students chosen is female
B = the second student chosen is also female.
We want to find the joint probability:

P (A and B) = P (B|A) × P (A)

21 / 63
cont...

I The probability that the first student chosen is female is

3
P (A) =
10

I What is the probability that the second student chosen is also

female?
Given that the first female student has been chosen, there are
only 2 female students left and there are only 9 students left in
the class.

Therefore, P (B|A) = 2/9.

Thus, the joint probability is:

P (A and B) = P (B|A) × P (A) = (2/9)(3/10) = 0.067

22 / 63
Addition Rule

I Addition Rule

P(A or B) = P(A) + P(B) − P(A and B)

I The probability of union of two mutually exclusive events is:

P (A or B) = P (A) + P (B)

23 / 63
Addition Rule

Example

In Sydney, 22% of the households subscribe to the Sydney Morning

Herald and 35% subscribe to the Australian. A survey reveals that
10% of all households subscribe to both newspapers. What is the
probability that a randomly selected household subscribe to at least
one of the newspapers?

let A= subscribes to the Sydney Morning Herald

B= subscribes to the Australia

Solution:
P (A or B) = P (A) + P (B) − P (A and B) = 0.22 + 0.35 − 0.1 =
0.47.

24 / 63
Random Variables

(Ref: Keller Chapter 7-1, 7-2, 7-3)

25 / 63
Random Variables

Definition: a random variable is a function that assigns a numeric

value to each outcome of an experiment.

Example: Imagine tossing three unbiased coins, once at a time.

I 8 equally likely outcomes.

S={HHH, HHT, HTH, HTT, THH, THT, TTH, TTT}

Let X = number of heads that occur.

X can take values 0, 1, 2, 3.
Actual value of X depends on chance – call it a random variable
(r.v.)

26 / 63
Notation

Denote random variables (X, Y, etc) in upper case.

Denote actual realised values (x, y etc) in lower case.

Example:
I X is the random variable that can take values 0, 1, 2, 3.
I Suppose we actually perform the experiment and find the
pattern HTT. Then x=1.

27 / 63
Two Types of Random Variables

I Discrete Random Variable

has a countable number of possible values
E.g. number of accidents per day in Sydney, etc.

I Continuous Random Variable

has an infinite number of possible values (number of elements
in sample space is infinite as a result of continuous variation)
E.g. temperature, height of a person, wage, etc.

28 / 63
Probability Distributions

A probability distribution is a table, formula, or graph that describes

the values of a random variable and the probability associated with
these values.

I Discrete Probability Distribution

I Continuous Probability Distribution

29 / 63
Discrete probability distributions

Definition: A table, formula or graph listing all possible values that

a discrete r.v. can take, together with the associated probabilities.
E.g. toss three coins example

X = number of heads that occur

X 0 1 2 3
P (X = x) 1/8 3/8 3/8 1/8

30 / 63
Probability Notation:

I The probability that r.v. X takes the value xi is:

P (Xi ) = P (X = xi ).

I Examples

p(0) = P (X = 0) = P (no heads = P (T T T ) = 1/8

p(1) = P (X = 1) = P (1 head = P (HT T, T HT, T T H) =

3/8

p(2) = P (X = 2) = P (2 heads = P (HHT, HT H, T HH) =

3/8

p(3) = P (X = 3) = P (3 heads = P (HHH) = 1/8

31 / 63
Discrete Probability Distributions

Properties/ Rules

I If a r.v. X can take values xi , then

1. 0 ≤ p(xi ) ≤ 1 for all i

X
2. p(xi ) = 1
xi

Example
X
p(xi ) = p(0) + p(1) + p(2) + p(3)
= P (X = 0) + P (X = 1) + P (X = 2) + P (X = 3)
= 1/8 + 3/8 + 3/8 + 1/8 = 1

32 / 63
cont...

I What is the probability of at most one head?

P (X ≤ 1) = P (X = 0 or X = 1)
= p(0) + p(1) = 1/8 + 3/8 = 1/2

I What is the probability of at least one head?

P (X ≥ 1) = p(1) + p(2) + p(3) = 3/8 + 3/8 + 1/8 = 7/8, or

P (X ≥ 1) = 1 − P (X = 0) = 1 − 1/8 = 7/8

33 / 63
Population/Probability Distribution

I The discrete probability distribution represents a population.

I Since we have populations, we can describe them by computing

various parameters.

E.g. the population mean and population variance.

34 / 63
Population Mean

I The population mean (expected value) of a discrete random

variable X, denoted by E(X)or µ, is:
X
µ = E(X) = xi p(xi )
xi
I The weighted average of all of its values (xi ).
I The weights are the probabilities p(xi ).

35 / 63
Example

Let’s go back to the coin tossing example

x 0 1 2 3
P (X = x) 1/8 3/8 3/8 1/8

X
µ = E(X) = xi p(xi )
xi
1 3 3 1
=0× +1× +2× +3×
8 8 8 8
= 1.5

36 / 63
Population Variance

I The Population Variance of a discrete r.v X, denoted by V (X)

or σ 2 is:

the weighted average of the squared deviations from the mean

measures spread/dispersion of distribution

h i
σ 2 = V (X) = E (X − µ)2
X
= (xi − µ)2 p(xi )
xi
X
= x2i p(xi ) − µ2 (“short-cut”formulation)
xi

37 / 63
Example

Tossing three coins – again...

x 0 1 2 3
P (X = x) 1/8 3/8 3/8 1/8

X
σ 2 = V (X) = x2i p(xi ) − µ2
xi
1 3 3 1
= 02 × + 12 × + 22 × + 32 × − 1.52
8 8 8 8
= 0.75

√
σ = SD(X) = 0.75 = 0.866(3dp)

38 / 63
Laws of Expected Value

I If X and Y are random variables, and c is any constant, then

the following hold:

E(c) = c
E(cX) = cE(X)
E(X − Y ) = E(X) − E(Y )
E(X + Y ) = E(X) + E(Y )
E(XY ) = E(X) × E(Y ) only if X and Y are independent

39 / 63
Examples

1. Let Z = 2X − 7. If E(X) = 3, then

E(Z) = E(2X − 7)
= 2 ∗ E(X) − 7 = −1

2. Let Z = 3X + 2Y − 2XY + 3, with E(X) = 3, E(Y ) = 5, X

and Y independent

E(Z) = E(3X + 2Y − 2XY + 3)

= 3E(X) + 2E(Y ) − 2E(X) ∗ E(Y ) + 3
=3∗3+2∗5−2∗3∗5+3
= 9 + 10 − 30 + 3 = −8.

40 / 63
Laws of Variance

I If X and Y are r.v.s and c is a constant,

V (c) = 0
V (cX) = c2 V (X)
X(X + c) = V (X)
V (X + Y ) = V (X) + V (Y ) if X and Y are independent
V (X − Y ) = V (X) + V (Y ) if X and Y are independent

41 / 63
Example

1. If V (Z) = 6 and Y = 7 ∗ Z − 3, then

V (Y ) = V (7 ∗ Z − 3)
= V (7 ∗ Z)
= 49 ∗ V (Z)
= 49 ∗ 6
= 294

42 / 63
Example

2. If Z = 3X − 2Y , and X and Y are independent with V (X) = 2

and V (Y ) = 1, then

V (Z) = V (3X − 2Y )
= V (3X) + V (2Y )
= 9 ∗ V (X) + 4 ∗ V (Y )
=9∗2+4∗1
= 22

43 / 63
Bivariate Distributions

I Distribution of a single variable – univariate

I Distribution of two variables together – bivariate
I Bivariate probability distributions are also called joint
probability.
I So, if X and Y are discrete random variables, then we say
p(x, y) = P (X = x and Y = y)
is the joint probability that X = x and Y = y.

44 / 63
Independence of Random Variables

If the random variables X and Y are independent, then

P (X = x and Y = y) = P (X = x) × P (Y = y)
or
p(x, y) = p(x) × p(y)

45 / 63
The sum of two random variables

Consider two real estate agents, A and B.

X = the number of houses sold by A in a week

Y = the number of houses sold by B in a week

The bivariate distribution of X and Y is:

X
Y
0 1 2 p(y)
0 0.12 0.42 0.06 0.60
1 0.21 0.06 0.03 0.30
2 0.07 0.02 0.01 0.10
p(x) 0.40 0.50 0.10 1

46 / 63
I Marginal Distribution of X

x 0 1 2
P (x) 0.4 0.5 0.1

I Marginal distribution of Y below.

y 0 1 2
P (y) 0.6 0.3 0.1

I In general, marginal distribution

X of X is given by
p(xi ) = P (X = xi ) = p(xi , yj )
yj
I Test for independence:
P (X = 1 and Y = 2) = 0.02 6= P (X = 1) × P (Y =
2)=0.5×0.1=0.05
X and Y are not independent.
47 / 63
I Describing the bivariate distribution
We can describe the mean and variance of each variable in a
bivariate distribution.

I We can show that (check these at home!)

E(X) = 0.7
V (X) = 0.41
E(Y ) = 0.5
V (Y ) = 0.45

Hint: Same formula as for univariate distributions.

48 / 63
Suppose the interest is in X+Y

That is, the total number of houses A and B sell in a week.

I Possible values of X+Y: 0, 1, 2, 3, 4.

I Want to know P(X+Y=2)

I Sum of all joint probabilities for which x+y=2.

P (X + Y = 2) = p(0, 2) + p(1, 1) + p(2, 0)

= 0.07 + 0.06 + 0.06
= 0.19

49 / 63
I We repeat the calculation for X+Y= 0,1,3 and 4:

x+y 0 1 2 3 4
p(x + y) 0.12 0.63 0.19 0.05 0.01

I Can evaluate mean and variance of (X+Y)

E(X + Y ) = 1.2
V (X + Y ) = 0.56

(check these at home!)

Hint: Same formula as for univariate distributions.

50 / 63
Covariance

I The covariance of two discrete variables (X and Y), denoted by

cov(X,Y) or σxy is:

σx,y = Cov(X, Y )
= E [(X − µx ) (Y − µy )]
XX
= (xi − µx ) (yj − µy ) p (xi , yj )
y x
XX
= xi yj p (xi , yj ) − µx µy
y x

51 / 63
Correlation coefficient

I The coefficient of correlation is calculated in the way as

described earlier...

Cov(X, Y )
ρxy =
σx σy

−1 ≤ ρ ≤ 1

52 / 63
Example

I Let X be the number of heads.

I Let Y be the number of changes of sequence, i.e. the number
of times we change from H → T or T → H.

HHH: x=3, y=0 TTT: x=0, y=0

HHT: x=2, y=1 TTH: x=1, y=1
HTH: x=2, y=2 THT: x=1, y=2
THH: x=2, y=1 HTT: x=1, y=1

53 / 63
Outcomes

Outcomes (S) x y
HHH 3 0
HHT 2 1
HTH 2 2
THH 2 1
TTH 1 1
THT 1 2
HTT 1 1
TTT 0 0
54 / 63
Joint and marginal probabilities

3
µx = ; µy = 1
Check yourself at home: 2
3 1
σx2 = ; σy2 =
4 2
55 / 63
Calculate covariance

I Covariance:
XX
σx,y = xi yj p(xi , yj ) − µx µy
2 1 2 1 3
=1×1× +1×2× +2×1× +2×2× − ×1
8 8 8 8 2
=0

σxy
I The correlation coefficient ρxy = =0
σx σy

I X and Y are uncorrelated.

56 / 63
Linear Combinations of Random Variables

I Consider Z = aX + bY , where a and b are constants, and

X and Y are random variables.

E(aX + bY) = aE(X) + bE(Y)

V(aX + bY) = a2 V(X) + b2 V(Y) + 2ab Cov(X, Y)
= a2 V(X) + b2 V(Y) + 2abρσx σy

I When a=1 and b=1 (sum of two r.v.s)

E(X + Y ) = E(X) + E(Y )

V (X + Y ) = V (X) + V (Y ) + 2Cov(X, Y )

57 / 63
Example

Example: portfolio diversification and asset allocation

I In Finance, use variance/standard deviation to assess risk of an

investment.

I Analysts reduce risk by diversifying their investments – that is,

combining investments where the correlation is small.

58 / 63
Example cont...
An investor forms a portfolio by putting 25% of his money in Stock
A and 75% in stock B, with parameters below.

Expected Return Standard Deviation

Stock A 8% 12%
Stock B 15% 22%

I The expected returns of stock A and Stock B:

E(RA ) = 0.08 and E(RB ) = 0.15

I The expected return of the portfolio:

E(Rp ) = E(0.25 ∗ RA + 0.75RB )

= 0.25 ∗ E(RA ) + 0.75E(RB )
= 0.25 ∗ 0.08 + 0.75 ∗ 0.15
= 0.1325
Expected return of the portfolio is 13.25%.
59 / 63
I Determine the variance of the return if ρ=1
I 2 stock returns are perfectly positively correlated.

V(aX + bY) = a2 V(X) + b2 V(Y) + 2abρσx σy

V (Rp ) = V (0.25RA + 0.75RB )

= 0.252 σA
2
+ 0.752 σB
2
+ 2 × 0.25 × 0.75ρσA σB
= 0.252 × 0.122 + 0.752 × 0.222
+ 2 × 0.25 × 0.75 × ρ × 0.12 × 0.22
= 0.0281 + 0.0099 × 1
= 0.0380 (to 4 dp)

60 / 63
I Determine the variance of the return if ρ=0

I 2 stock returns are perfectly uncorrelated.

V (Rp ) = 0.252 σA
2
+ 0.752 σB
2
+ 2 × 0.25 × 0.75ρσA σB
= 0.252 × 0.122 + 0.752 × 0.222
+ 2 × 0.25 × 0.75 × ρ × 0.12 × 0.22
= 0.0281 + 0.0099 × 0
= 0.0281 (to 4 dp)

61 / 63
I The standard deviation of the portfolio
√
If ρ = 1, SD(Rp ) = 0.0380 = 0.1949
√
If ρ = 0, SD(Rp ) = 0.0282 = 0.1676

I The risk of the portfolio is lower when these two stocks are not
correlated.

62 / 63
Summary

I Probabilities
Terminologies: random experiment, sample space, events
Joint, marginal and conditional probabilities
Probability Rules
I Random Variables;
I Probability distributions:
Expected values, variances, standard deviations
Laws of expected values
Laws of variances
I Bivariate distributions:
Marginal distributions
Independence
Sum of two random variables
Covariance and Correlation
Linear Combinations of Variables

63 / 63