Lecture 5

BUSINESS STATISTICS PROBABILITY AND

Advanced Educational Program RANDOM VARIABLES

Reading materials:
Chap 6,7,8 (Keller)

1 2

Why do we need to study probability and

probability distribution?

3 4

Why do we need to study probability and

probability distribution?
Terminology (1)
• A random experiment is a process that results
• Prob is a crucial component to obtain information
in a number of possible outcomes. None of
about pops from samples
which can be predicted with certainty.
• Prob provides the link between pops and samples.
• Eg: • Eg:
– From sample means => infer pop means – Roll a dice: outcomes 1, 2, 3, 4, 5, 6.
– From a known pop => measure the likelihood of obtain a – Flip a coin: outcomes Heads, Tails
particular event or sample. – Take an exam: pass or fail
– Take a sample from a population

5 6
 Terminology (3)
Terminology (2)
— The sample space of a random experiment is a
list of ALL possible outcomes
— Outcomes must be mutually exclusive and
exhaustive.
◦ Mutually exclusive: No two outcomes can both
occur on any one trial
◦ Exhaustive: All possible outcomes must be included
— E.g. roll a dice: sample space:
S={1, 2, 3, 4, 5, 6}.

7 8

Example Terminology: continued

• An event is a collection of one or more simple
(individual) outcomes or events.
• E.g. roll a dice: event A = odd number comes
up.
• In general, use sample space S={E1, E2,…, En}
where there are n possible outcomes.
• Probability of an event Ei occurring on a single
trial is written as P(Ei)
• Use either fractions, decimals, or percentages
to indicate the values of probability.
9 10

How do we measure probability Probabilities: classical method

(prob.) Number of favorable outcomes
Probability of an event=
Total number of outcomes
• Classical • For the sample space S: P(S)=1
• Frequentist • E.g. roll a dice: sample space:
S={1, 2, 3, 4, 5, 6}.
• Subjective/Bayesian
Example of events:
Ø Obtain the number ‘1’
Ø Obtain an odd number
Ø Obtain a number larger than 6
Ø Obtain a number smaller than 7

11 12
 Probabilities of Combined Events
Two rules about probabilities
• The probability assigned to each simple event
Ei must satisfy:

1. 0 ≤ P ( Ei ) ≤ 1 for all i
n
2. ∑ P(E ) =1 i
i =1

13 14

In formulas Joint Probabilities (1)

• Eg: mutual funds(Textbook, p.181)
• Consider two events, A and B.
ØP(A or B) = P(A U B) = P(A union with B)
= P(A occurs, or B occurs, or both occur)

ØP(A and B) = P(A ∩ B) = P(A intersection with

B)
= P(A and B both occur)

ØP(Ā)=P(Ac)= P(A complement) = P(A does not

occur)

ØP(A|B)=P(A occurs given that B has occurred)

15 16

Joint Probabilities (2)

• Eg: mutual funds: Why do some mutual fund managers
Marginal Probabilities (1)
perform better than the others? Do these successful
managers earn their MBA in top program?
B2 = Mutual fund Probabilities B1 B2
B1 = Mutual Fund
Probabilities does not outperform
outperforms market A1 0.11 0.29
market
A1=Top-20 MBA program 0.11 0.29 A2 0.06 0.54
A2 = Not top-20 MBA
0.06 0.54
program
• Joint probabilities = P(A ∩ B)
• Marginal probabilities:
– Computed by adding across rows or down columns
P(Mutual fund outperforms AND top-20 MBA)=0.11
– Named because they are calculated in the margins of
P(Mutual fund outperforms AND not top-20)=0.06
the table
P(Mutual fund not outperform AND top-20)=0.29
P(Mutual fund not outperform AND not top-20)=0.54
17 18
 Marginal Probabilities (2) Conditional probability
• Conditional probability that A occurs, given that B
Probabilities B1 B2 Totals has occurred:
A1 0.11 0.29 0.40
P( A and B)
P ( A | B) =
A2 0.06 0.54 0.60 P( B)
Totals 0.17 0.83 1.00 • Want to see whether a fund managed by a graduate
of a top-20 MBA program outperform the market
P(A1)=P(A1 and B1)+P(A1 and B2)=0.11+0.29=0.40
P(A2)=P(A2 and B1)+P(A2 and B2)=0.06+0.54=0.60
P(B1)=P(B1 and A1)+P(B1 and A2)=0.11+0.06=0.17
P(B2)=P(B2 and A1)+P(B2 and A2)=0.29+0.54=0.83
19 20

Some rules of probability Additive rules

• If the events are mutually exclusive, then:
• Additive rule P(A or B)= P(A)+P(B)
• Multiplicative rule • If the events are union, then:
P( A or B) = P( A) + P( B) − P( A and B)
• If the events are complement: A and its
complement, Ā,
so P(A)+P(Ā)=1;
therefore P(Ā)=1-P(A)

21 22

Additive rules: example Additive rules: example

• 50% of families in a certain city subscribe to the • Therefore
morning newspaper, 65% subscribe to the P(at least one paper) = P(Morning or Afternoon)
afternoon newspaper, and 30% of the families
= P(Morning) + P(Afternoon) − P(Morning and
subscribe to both newspapers. What proportion
Afternoon)
of families subscribe to at least one newspaper?
= 0.5 + 0.65 − 0.3
• We are told that:
= 0.85.
- P(Morning) = 0.5
So 85% of the city subscribe to at least one of the
- P(Afternoon) = 0.65 and
newspapers.
- P(Morning and Afternoon) = 0.3
23 24
 Multiplicative rules Independence
• If the events are independent, then: — Two events are independent if
P(A and B) = P(A)*P(B) P(A|B)=P(A) or P(B|A)=P(B)
• If events are conditional:
P( A and B) Note: If A and B are independent, then
P ( A | B) = => P( A and B) = P ( A | B ) P( B) = P ( B | A) P( A)
P( B) P(A and B) = P(A)*P(B) Note: only if indep!
Then P(A|B) = [P(A and B)]/P(B)
=[P(A)*P(B)] /P(B)
=P(A)
25 26

Activity 1 Random Variables

• Check whether the event that manager • Imagine tossing three unbiased coins.
graduated from a top-20 MBA program is S= {HHH, HHT, HTH, THH, HTT, THT, TTH, TTT)
independent from the event that the fund • 8 equally likely outcomes.
• Let X = number of heads that occur.
outperforms the market.
• X can take values 0, 1, 2, 3.
• Actual value of X depends on chance => call it a random
variable (r.v.)
• A random variable is defined as the numerical outcomes of
a random experiment.

27 28

Notation Discrete vs continuous R.V.s

• Denote random variables (X, Y, ...) in upper • A discrete random variable has a countable
case number of possible values, e.g. number of
• Denote actual realised values (x, y,...) in lower heads, number of sales etc.
case • A continuous random variable has an infinite
number of possible values – number of
elements in sample space is infinite as a result
of continuous variation e.g. height, weight etc.

29 30
 Discrete probability distributions More on discrete prob. distribution
— Definition: A table or formula listing all • If x is the value taken by a r.v. X, then
possible values that a discrete r.v. can take, p(x)=P(X=x)= sum of all the probabilities
together with the associated probabilities. associated with the simple events for which
— E.g. for our toss three coins example: X=x.
• If a r.v. X can take values xi, then
x 0 1 2 3
P(X=x) 1/8 3/8 3/8 1/8 1. 0 ≤ p ( xi ) ≤ 1 for all xi
2. ∑ p(x ) =1 i
(Check the probability in the table) xi

31 32

Activity 2 Describing the Probability Distribution

o What is the probability of at most one head? • Expected value or mean of a discrete random
o What is the probability of at least one head? variable, X, which takes on values x with
probability p(xi) is:

µ = E ( X ) = ∑ xi × p( xi )
all xi

33 34

Mean of the sample and population

Back to the coin tossing

x 0 1 2 3
P(X=x) 1/8 3/8 3/8 1/8

µ = E ( X ) = ∑ xi × p( xi )
all xi

= 0* + 1* 83 + 2* 83 + 3* 81
1
8

= 128 = 1.5
35 36
 Rules for Expectations Variance
If X and Y are random variables, and c is any • Measures spread/dispersion of distribution
constant, then the following hold: • Let X be a discrete random variable with
— E(c)=c values xi that occur with probability p(xi), and
— E(cX)=cE(X) E(X) = µ.
— E(X-Y)=E(X)-E(Y) • The variance of X is defined as
— E(X+Y)=E(X)+E(Y) 2
— E(XY)=E(X)*E(Y) only if X and Y are
σ 2 = E ⎡( X − µ ) ⎤
⎣ ⎦
independent 2
= ∑ ⎡( xi − µ ) × p ( xi )⎤
all x
⎣ ⎦
i
37 38

Variance continued Tossing three coins – again…

x 0 1 2 3
2
σ 2 = E ⎡( X − µ ) ⎤ P(X=x) 1/8 3/8 3/8 1/8
⎣ ⎦
2 2
= E ⎡⎣ X ⎤⎦ − µ V ( X ) = ∑ ⎡⎣ xi2 × p ( xi )⎤⎦ −µ 2
all xi
= ∑ ⎡⎣ xi2 × p ( xi )⎤⎦ −µ 2
all xi = ⎡⎣02 × 18 + 12 × 83 + 22 × 83 + 32 × 81 ⎤⎦ − 1.52
= 0.75
Std Dev ( X ) = 0.75 = 0.866 (to 3dp)
39 40

Laws for Variances More on discrete prob. distribution

• If X and Y are r.v.s and c is a constant

1. V(c)=0
2. V(cX)=c²V(X)
3. V(X+c)=V(X)
4. V(X+Y)=V(X)+V(Y) if X and Y are
independent
5. V(X-Y)=V(X)+V(Y) if X and Y are
independent
41 42
 A special distribution The Binomial distribution
• A bernoulli process has the following • If a fixed number, n, of bernoulli trials are
properties: undertaken, the random variable representing the
number of successes in the n trials has a binomial
– There are two possible outcomes, which we call
distribution.
success and failure.
• Key factors:
– The probability of a success is p, the probability of
– Fixed number of trials, n.
failure is (1-p).
– Each trial results in success (with probability p) or failure
– The trials are independent – that is, the result of (with probability (1-p)).
one trial does not affect the result of any other – Same probability of success in each trial, i.e. trials are
trials. independent

Simple examples Binomial Random Variable

• Is the number of successes in n trials.
• Flip a coin 10 times, X = number of heads.
• Possible values : 0, 1, 2, …, n
– X is a binomial random variable, n=10, p=0.5
• Discrete random variable
• Do a survey of 1000 people, X = number who
think current PM is doing a good job. • Formula to calculate probabilities:
– X is a binomial random variable, n=1000, p=? n! n− x
P ( X = x) = p x (1 − p )
• Use notation X~Bin(n,p) x !( n − x )!

Note: n ! = n(n − 1)(n − 2)...(2)(1)

Bivariate Distributions Example

• Toss three coins.
• Distribution of a single variable – univariate • Let X be the number of heads.
• Distribution of two variables together – bivariate • Let Y be the number of changes of sequence,
• So, if X and Y are discrete random variables, then i.e. the number of times we change from H
we say p(x,y) = P(X=x and Y=y) is the joint →T or T→H.
probability that X=x and Y=y. – HHH: x=3, y=0 TTT: x=0, y=0
– HHT: x=2, y=1 TTH: x=1, y=1
– HTH: x=2, y=2 THT: x=1, y=2
– THH: x=2, y=1 HTT: x=1, y=1

47 48
 Example continued Bivariate probability distribution

Outcome (S) x y y
HHH 3 0
0 1 2 px(x)
HHT 2 1
HTH 2 2 0 1/8 0 0 1/8
THH 2 1 1 0 2/8 1/8 3/8
TTH 1 1 x
2 0 2/8 1/8 3/8
THT 1 2
3 1/8 0 0 1/8
HTT 1 1
TTT 0 0 py(y) 2/8 4/8 2/8 1

49 50

Independence of Random Variables Covariance

• If the random variables X and Y are • Consider the r.v.s X and Y with joint pdf p(x,y);
independent, then x=x1,…,xm; y=y1,…,yn.
P(X=x and Y=y) = P(X=x) . P(Y=y) • If E(X) = µx and E(Y)= µy, then the covariance
p(x,y) = px(x) . py(y) between X and Y is given by
• In previous example, X and Y are clearly not
independent: σ xy = cov ( X , Y ) = E ⎡⎣( X − µ x ) (Y − µ y )⎤⎦
p(0, 0) = 1/8
m n
px(0) . py(0) = 1/8 * 2/8 = 1/32 = ∑∑ xi y j p ( xi , y j ) − µ x .µ y
p(0, 0) ≠ px(0) . py(0) i =1 j =i

51 52

Correlation coefficient Return to 3 coins tossed

• Associated with covariance. • X = number of heads, Y = number of sequence
changes. Verify that X and Y are independent.
cov( x, y) • Check for yourself:
ρ= ; −1 ≤ ρ ≤ 1
σ xσ y 3
µx = , µ y = 1
2
3 1
σ x2 = , σ y2 =
4 2

53 54
The sum of two random variables Bivariate distribution of X and Y
o Consider two real estate agents.
X
ØX = the number of houses sold by Vinhomes in a Y
week 0 1 2 py(y)

ØY = the number of houses sold by THM in a 0 0.12 0.42 0.06 0.60

week 1 0.21 0.06 0.03 0.30
o Bivariate distribution of X and Y shown on 2 0.07 0.02 0.01 0.10
next slide px(x) 0.40 0.50 0.10 1

55 56

We can show (check these at home!) Suppose interest is in X+Y

• E(X)=0.7 — That is, the total number of houses Vinhomes

and THM sell in a week.
• V(X)=0.41
— Possible values of X+Y: 0, 1, 2, 3, 4.
• E(Y)=0.5
— Then, P(X+Y=2) = sum of all joint
• V(Y)=0.45 probabilities for which x+y=2;
— That is P(X+Y=2) = p(0,2) + p(1,1) + p(2,0)
= 0.07 + 0.06 + 0.06
=0.19

57 58

Law of expected value and variance of the sum

Repeat this for 0, 1, 2, 3, 4… of two variables

x+y 0 1 2 3 4 If a and b are constants, and A and Y are

p(x+y) 0.12 0.63 0.19 0.05 0.01 random variables, then
E (aX + bY ) = aE ( X ) + bE (Y )
— Can evaluate mean and variance of (X+Y) V (aX + bY ) = a 2V ( X ) + b 2V (Y ) + 2ab cov( X , Y )
E(X+Y) = 1.2
V(X+Y) = 0.56
(check these at home!)

59 60
 Application of this – portfolio
Continuous probability distribution
diversification and asset allocation

• See Keller
• Remember: discrete data has a limited (finite)
– Pages 237-242 (9th edition) number of possible values è discrete
• In Finance, use variance and standard deviation to assess probability distributions can be put in tables
risk of an investment.
• Analysts reduce risk by diversifying their investments – • Continuous data have an infinite number of
that is, combining investments where the correlation is possible values è we use a smooth function,
small.
f(x) to describe the probabilities

61 62

About the function Notes about continuous pdfs

• f(x) must satisfy the following: 1) P(a<X<b) = area under the curve between a
1. f(x)≥0 for all x, that is, it must be non- and b b

negative. Evaluate as P ( a < X < b ) = ∫ f ( x)dx

a
2. The total area underneath the curve
representing f(x) = 1.

63 64

Notes about continuous pdfs Notes about continuous pdfs

2) For a continuous pdf, the probability that X 3) A continuous random variable has a mean and
will take any specific value is zero. a variance!
Let a èb – see that area è 0. The mean measures the location of the
distribution, the variance measures the spread
of the distribution.

65 66
 The Normal Distribution Notes about the Normal Distribution
• Bell-shaped, symmetric about µ, reaches highest
point at x=µ, tends to zero as x→±∞. 1. E(X) = µ; V(X) = σ².
2. Area under curve = 1
3. Different means – shift curve up and down x-
axis
4. Different variances – curve becomes more
peaked
5. Shorthand notation: X~N(µ, σ²).

67 68

Different means Different variances

0.5

σ=1
σ=0.5
f(x)

σ=2
f(x)

0
0
-4 -2 0 2 4
x x

69 70

Probabilities from the Normal Distribution (1) OR we require (2)

• Generally, we require probabilities P(X<a) • P(a<X<b)

71 72
 So, need to find the area under the curve…(3) Tabulated values

• That is, need to integrate as follows: • Tables made to provide probabilities.

• However, obviously, different values needed for each
b b 1 ⎛ x−µ ⎞
2 different µ and σ² - infinite possible values, so impossible to
1 − ⎜
2⎝ σ ⎠
⎟ have all the tables needed!
Area = ∫ f ( x )dx = ∫ e dx. • So we select one particular normal distribution – µ=0, σ² =1 –
a a σ 2π call this the Standard Normal Distribution, and tabulate all the
probabilities for it.
• Not easy to do! • Call a r.v. from this a Standard Normal r.v., use notation
Z~N(0,1)
• Now we just need a way to convert any other normal
distribution to the standard normal – then we can use the
existing tables
73 74

Standardising Standardising …
• The process of converting any Normal random
variable to a Standard Normal Random Variable.
• If X~N(µ,σ²), then use the linear transformation
below:

X −µ
Z= ~ N (0,1)
σ

75 76

Standardising (cont.) Rules to find probabilities normal tables

• So, for ANY random variable that comes from a • Symmetry

normal distribution, if we subtract the mean and Ø P(Z<-a) = P(Z>a)
divide by the standard deviation, we get a r.v.~N(0,1).
Ø P(Z>a) = 1 – P(Z<a)
• See the Z-table in Appendix B-8. This Table provides
P(Z<z) for various values of z • P(a<Z<b) = P(Z<b) – P(Z<a)
• Others give P(Z>z) for various values of z. • Total area under curve is 1, total area under each
half of curve is 0.5, i.e. P(Z<0)=P(Z>0)=0.5
• Draw the curve, shade the area, break it up into
areas you can find (differences or sums)

77 78
 Examples using tables (1) Examples using tables (2)
1) P(Z<1.5) 2) P(Z>1)

79 80

Examples using tables (3) Examples using tables (4)

3) P(Z<-1) 4) P(1<Z<1.5)

81 82

In general

• Given X~N(µ,σ²), suppose we require P(X<a).

X −µ
Know that Z = ~ N (0,1).
σ
⎛ X −µ a−µ ⎞
So, P ( X < a ) = P ⎜ <
⎝ σ σ ⎟⎠
⎛ a−µ ⎞
= P⎜Z < where Z ~ N (0,1).
⎝ σ ⎟⎠
83