1 The Probability Model

1.1 Models: Deterministic and Probabilistic

Central to the solution of almost all engineering design problems

is a mathematical model of the target system

• Model gives an approximate representation of the physical

system – must explain and/or characterise all relevant aspects
of the system
• Complexity depends on the detail of representation required
• Model gives mathematical relations amongst various
measurable properties of the system
• Model is used to predict behaviour of physical system without
the need to actually build it
• Model also used to study effects of changing the values of the
various parameters of the physical system, e.g. study the
sensitivity of the design to uncertainties in the actual parameter
values

(a) Deterministic Models

• Applicable when conditions under which experiment was carried
out determine outcomes of the experiment, e.g. circuit theory

(b) Probabilistic Models

• Applicable when outcomes of the experiment are non-
repeatable (unpredictable) even when experiments are
conducted under seemingly identical conditions, e.g. coin
tossing
• Probabilistic models give an indication of the range of possible
outcomes of the experiment and their associated probability of
occurrence
Dr Yue Rong (2008–2011), Dr Y Leung (2006, 2007) 1-1
Curtin University of Technology, Australia
1.2 Approaches to the Definition of Probability

(a) Classical Definition- Laplace (1749-1827)

• Let A be an event. Classically, the probability of A is

defined by
NA
P [ A] = (1)
N
where N is number of possible outcomes, N A is number of
outcomes favourable to A , and all outcomes assumed equally
likely

Example (Fair Dice): P [{even faces}] = 3 = 0.5

6

• Criticisms
1. “Equally likely” actually means “equally probable”. Thus
definition makes use of the concept it is trying to define
2. Outcomes must be equally likely
3. Has problems with experiments where N → ∞
4. Must be careful with counting technique to find N and N A

Example (Two fair dice) Determine P [{sum = 7}]

Method 1 Possible outcomes are 2, 3, …, 12
∴ P [{sum = 7}] = 1 11
Method 2 Possible outcomes are (1,1), …, (1,6), (2,2), … (2,6), …, (6,6)
Favourable outcomes are (1,6), (2,5) and (3,4)
∴ P [{sum = 7}] = 3 21 = 1 7
Method 3 Possible outcomes are (1,1) (1,2) ⋯ (1,6)
(2,1) (2,2) ⋯ (2,6)
⋮ ⋮ ⋱ ⋮
(6,1) (6,2) ⋯ (6,6)
Favourable outcomes are (1,6), (6,1), (2,5), (5,2), (3,4) and (4,3)
∴ P [{sum = 7}] = 6 36 = 1 6

See www.cut-the-knot.org/bertrand.shtml for a less

straightforward example
Dr Yue Rong (2008–2011), Dr Y Leung (2006, 2007) 1-2
Curtin University of Technology, Australia
(b) Relative Frequency Definition

• Let nA be number of occurrences of the event A in n

independent trials of an experiment. Relative frequency of A
defined by
nA
fn ( A) = (2)
n

• Relative frequency definition of the probability of A given by

P [ A] = lim fn ( A) (3)
n →∞

• Assumes statistical regularity, i.e. fn ( A) will exhibit a certain

stability about some value as n → ∞

Example (Fair Dice)

In 1000 rolls of a dice, number of times even faces are
observed is 502.
∴ P {even faces} ≈ 502 = 0.502
1000

• Critique
1. Based on observations, ∴ intuitively satisfying
2. May need a large nA and n
3. Mode of convergence of nA n as n → ∞ not clear
4. Requires experiments to be repeated under identical
conditions

Dr Yue Rong (2008–2011), Dr Y Leung (2006, 2007) 1-3

Curtin University of Technology, Australia
(c) The Axiomatic Approach

• Developed by Kolmogorov (1933); allows deductive reasoning

• Preliminary definitions:

An experiment consists of a procedure and observations

An outcome of an experiment is any of its possible
observations

The sample space of an experiment is the finest-grain,
mutually exclusive, collectively exhaustive set of all possible
outcomes

An event is a set containing a number of the outcomes
grouped together according to some selection rule

An event space, or partition, is a collectively exhaustive,
mutually exclusive set of events

Finest-grain means all possible distinguishable outcomes are
identified separately

Mutually exclusive means if one outcome (resp. event)
occurs, then no other outcome (resp. event) also occurs

Collectively exhaustive means every outcome is in the
sample, or event, space

Example (Dice)
Sample space
An event
O = {odd rolls}

Another event
E = {even rolls}
An outcome
{6}
Yet another event
L = {low rolls}
{O, E} forms an event space
Dr Yue Rong (2008–2011), Dr Y Leung (2006, 2007) 1-4
Curtin University of Technology, Australia
• Sample spaces can be discrete or continuous
Discrete sample spaces can be finite or countably infinite

Examples
(a) Continuous sample space: time for a hard disk to crash
(b) Finite discrete sample space: dice, coin, lotto
(c) Countably infinite discrete sample space: number of coin
tosses to get a head

• Axiomatic approach requires:

1. the experiment E to be defined

2. the sample space S, consisting of all possible outcomes

{s } of E, to be identified

3. a set F of subsets of S , called events, to be constructed

Event A in F is said to have occurred when an outcome in
A is outcome of E. Thus S is the certain event, ∅ the
impossible or null event, and the event {si } consisting of
a single element si an elementary event

4. a rule that assigns to each event A ∈ F a number, P [ A] ,

called the probability of A , to be specified. This rule,
called the probability law of E , must satisfy following
axioms:
(i) P [ A] ≥ 0 (4)
(ii) P [S ] = 1 (5)
(iii) For any countable collection of mutually exclusive
events A1, A2, … 1

P [ A1 ∪ A2 ∪ ⋯] = P [ A1] + P [ A2 ] + … (6)
1
Many textbooks, including Kolmogorov’s original manuscript, state the third axiom as follows and derive
(6) as a theorem
If A1 and A2 are mutually exclusive events, then P [ A1 ∪ A2 ] = P [ A1] + P [ A2 ]
Irrespective of the choice, the set of axioms and theorems shown in the next page are logically consistent
Dr Yue Rong (2008–2011), Dr Y Leung (2006, 2007) 1-5
Curtin University of Technology, Australia
• Formally, F is required to be a σ -algebra. That is, F must
include ∅ and be closed under complementation and
countable unions, i.e.
(i) A∈F ⇒ AC ∈ F (7)

and (ii) Ai ∈ F , i = 1, 2, …, ⇒ ∪ Ai ∈F (8)

i
Above requirement forces us to assign probabilities to all events
that may be of interest

• The triple (S, F , P ) describing E is called the probability

space
of E . It also constitutes a complete probability model for E
In particular, to construct a probability model for E, we must first
identify all events of possible interest, then assign to them
probabilities in a manner consistent with the “3 axioms” (4) − (6)

Example (Dice)
Suppose O = {odd rolls} and E = {even rolls} . Intuitively,
P [O ] = P [E ] = 21 . But P [O ] = 31 and P [E ] = 32 also ok.

Dr Yue Rong (2008–2011), Dr Y Leung (2006, 2007) 1-6

Curtin University of Technology, Australia
• It can be deduced from the 3 axioms (4) − (6) that

(i) P [ AC ] = 1 − P [ A] (9)

(ii) P [ A] ≤ 1 (10)

(iii) P [∅] = 0 (11)

(iv) If A1, A2, …, Am are pairwise mutually exclusive, i.e.
Ai ∩ A j = 0 for i ≠ j , then

m  m
P  ∪ Ai  = ∑ P [ Ai ] (12)
 i =1  i =1

(v) For any two not necessarily disjoint events A and B

P [ A ∪ B ] = P [ A] + P [B ] − P [ A ∩ B ] (13)

Above result can be generalised as follows

P [ A ∪ B ∪ C ] = P [ A] + P [B ] + P [C ]
= − P [ A ∩ B ] − P [ A ∩ C ] − P [B ∩ C ]
P[A ∪ B ∪ C] = + P[A ∩ B ∩ C] (14)
and more generally
m  m
P  ∪ Ai  = ∑ P [ Ai ] − ∑ P [ Ai ∩ A j ] + ⋯
 i =1  i =1 i<j

+ (−1)m +1P [ A1 ∩ ⋯ ∩ Am ] (15)

(vi) A ⊂ B ⇒ P [ A] ≤ P [ B ] (16)

(vii) For any event A and event space {B1, B2, …, Bm }

m
P [ A] = ∑ P[ A ∩ Bi ] (17)
i =1

Dr Yue Rong (2008–2011), Dr Y Leung (2006, 2007) 1-7

Curtin University of Technology, Australia
1.3 Conditional Probability

(a) Definition

• Given two events A and B . The conditional probability of

A , given B has occurred, is defined by

P [ A ∩ B]
P [ A | B] = , P [B ] > 0 (18)
P [B ]

A
A∩ B
B

B has occurred ⇔ s ∈ B
A occurs only if s ∈ A ∩ B
Definition renormalises P [ A ∩ B ]

• P [ A | B ] defined only for P [B ] > 0 . P [B ] = 0 means B almost

certain not to have occurred. Therefore, illogical to consider
probability of A , given B has occurred

• P [ A | B ] satisfies axioms of probability – can be proved from the

definition (18)

• Frequency interpretation of (18)

P [ A | B ] is relative frequency of event A ∩ B in experiments
where B has occurred
Suppose experiment done n times, during which B occurred
nB times, and A ∩ B occurred nA∩B times. Then

nA∩B n n P [ A ∩ B]
P [ A | B] ≅ = A ∩B ≅
nB nB n P [B ]

Dr Yue Rong (2008–2011), Dr Y Leung (2006, 2007) 1-8

Curtin University of Technology, Australia
• For convenience, we often write P [ A ∩ B ] as P [ AB ] . Thus, (18)
can also be expressed as follows
P [ AB ]
P [ A | B] = , P [B ] > 0 (19)
P [B ]

Work Example 1 (Fair Dice)

Define L = {1, 2}, E = {2, 4, 6}, B = {4, 5, 6}
Determine (a) P [L | E ]
(b) P [L | B ]

P [LE ] 1/ 6
Solution: (a) P [LE ] = P [{2}] = 1/ 6 , P [L | E ] = = = 1/ 3
P [E ] 1/ 2
P [LB ]
(b) P [LB ] = P [∅] = 0 , P [L | B ] = =0
P [B ]

Work Example 2 (Binary Communication System)

Transmitter sends '1' with probability p and '0' with probability
1− p .
Receivers detects transmitted symbols with probability of error ε .
Define: Ti to be the event {symbol ' i ' transmitted}, i = 0, 1
Ri to be the event {symbol ' i ' decoded}, i = 0, 1
Determine (a) P [R0T0 ]
(b) P [R1T0 ]
(c) P [R0T1]
(d) P [R1T1]

Dr Yue Rong (2008–2011), Dr Y Leung (2006, 2007) 1-9

Curtin University of Technology, Australia
Solution: (a) P [R0T0 ] = P [T0 ]P [R0 | T0 ] = (1− p )(1− ε )

(b) P [R1T0 ] = P [T0 ]P [R1 | T0 ] = (1− p )ε

(c) P [R0T1] = P [T1]P [R0 | T1] = pε

(d) P [R1T1] = P [T1]P [R1 | T1] = p(1− ε )

Dr Yue Rong (2008–2011), Dr Y Leung (2006, 2007) 1-10

Curtin University of Technology, Australia
(b) Law of Total Probability

• Suppose {B1, B2, …, Bm } is an event space of S and A is an

event in S . Clearly
A = A ∩ S = A ∩ (B1 ∪ ⋯ ∪ Bm ) = ( A ∩ B1) ∪ ⋯ ∪ ( A ∩ Bm )
Thus
P [ A] = P [ AB1] + ⋯ + P [ ABm ]
= P [ A | B1] ⋅ P [B1] + ⋯ + P [ A | Bm ] ⋅ P [Bm ]

m
or P [ A] = ∑ P [ A | Bi ] ⋅ P [Bi ] (20)
i =1

Work Example 3 (Binary Communication System)

Transmitter sends '1' with probability p and '0' with probability
1− p .
Receivers detects transmitted symbols with probability of error ε .
Define: Ti to be the event {symbol ' i ' transmitted}, i = 0, 1
Ri to be the event {symbol ' i ' decoded}, i = 0, 1

Determine (a) P [R0 ] (b) P [R1]

Solution: With the help of the butterfly diagram, we have
(a) P [R0 ] = P [R0 | T0 ]P [T0 ] + P [R0 | T1]P [T1]
= (1− ε )(1− p ) + εp = 1− ε − p + 2εp
(b) P [R1] = P [R1 | T0 ]P [T0 ] + P [R1 | T1]P [T1]
= (1− ε)p + ε(1− p ) = ε + p − 2εp
Verification: P [R0 ] + P [R1] = 1

Dr Yue Rong (2008–2011), Dr Y Leung (2006, 2007) 1-11

Curtin University of Technology, Australia
Work Example 4 (Testing of Memory Chips)
Manufacturer produces “good” chips with failure rate α and “bad”
chips with failure rate 1000α , i.e., probability that a “good” chip will
work after t seconds is e−αt and probability that a “bad” chip will
work after t seconds is e−1000αt
Suppose fraction of “good” chips is 1− p and fraction of “bad”
chips is p .
Find probability that a randomly chosen chip will still be working
after t seconds of testing.
Solution: (hint: we can use the butterfly diagram, where T0 is
“good”, T1 is “bad”, R0 is “working”, and R1 is failure)
P [Work ] = P [Work | Good ]P [Good ] + P [Work | Bad ]P [Bad ]
= e−αt (1− p ) + e−1000αt p

(c) Bayes’ Rule

• Since P [ AB ] = P [BA] , it follows from (18) that

P [B | A ] ⋅ P [ A ] = P [ A | B ] ⋅ P [B ]

P [ A | B ] ⋅ P [B ]
or P [B | A ] = (21)
P [ A]

• Generalising, suppose {B1, B2, …, Bm } is an event space of S .

Then from (20) and (21)
P [ A Bi ] ⋅ P [Bi ] P [ A Bi ] ⋅ P [Bi ]
P [Bi A] = = m
(22)
P [ A]
∑ P [ A B j ] ⋅ P [B j ]
j =1

Bayes’ Rule gives a posteriori probability of Bi given A has

occurred, in terms of the a priori probabilities of B1, B2, …, Bm
“Reasoning about causes when we observe effects”

Dr Yue Rong (2008–2011), Dr Y Leung (2006, 2007) 1-12

Curtin University of Technology, Australia
Work Example 5 (Binary Communication System)
Transmitter sends '1' with probability p and '0' with probability
1− p .
Receivers detects transmitted symbols with probability of error ε .
Define: Ti to be the event {symbol ' i ' transmitted}, i = 0, 1
Ri to be the event {symbol ' i ' decoded}, i = 0, 1
Determine, in terms of p and ε
(a) P [T0 | R0 ] ; (b) P [T1 | R0 ] ; (c) P [T0 | R1] ; (d) P [T1 | R1]

Interpret the above results for p = 1 and p = 0 .

2

Solution: From Work Example 3, we have

P [R0 ] = 1− ε − p + 2ε p , P [R1] = ε + p − 2ε p
Using Bayes’ rule,
P [T0R0 ] P [R0 | T0 ]P [T0 ] (1− p )(1− ε )
(a) P [T0 | R0 ] = = =
P [R0 ] P [R0 ] 1− ε − p + 2ε p

P [T1R0 ] P [R0 | T1]P [T1] εp

(b) P [T1 | R0 ] = = =
P [R0 ] P [R0 ] 1− ε − p + 2ε p

P [T0R1] P [R1 | T0 ]P [T0 ] (1− p )ε

(c) P [T0 | R1] = = =
P [R1] P [R1] ε + p − 2εp
P [T1R1] P [R1 | T1]P [T1] p(1− ε )
(d) P [T1 | R1] = = =
P [R1] P [R1] ε + p − 2εp
For p = 1/ 2 , P [T0 | R0 ] = 1− ε , P [T1 | R0 ] = ε , P [T0 | R1] = ε

P [T1 | R1] = 1− ε . When the a-priori probability of T0 and T1 are

equal, if ε < 1/ 2 , then T0 is more likely than T1 when R0 is
received. Also, T1 is more likely than T0 when R1 is received.

For p = 0 , P [T0 | R0 ] = P [T0 | R1] = 1, P [T1 | R0 ] = P [T1 | R1] = 0 .

T0 is more likely than T1 no matter what is received.

Dr Yue Rong (2008–2011), Dr Y Leung (2006, 2007) 1-13
Curtin University of Technology, Australia
Work Example 6 (Testing of Memory Chips)
Manufacturer produces “good” chips with failure rate α and “bad”
chips with failure rate 1000α .
Suppose fraction of “good” chips is 1− p and fraction of “bad”
chips is p .
To test for “good” chips, we test each device for t seconds. If the
chip still works after t seconds, we deem the chip “good”.
Find t such that the probability that chip is indeed good, given it
has passed the test, is 0.99.

Solution: From Work Example 4, we have

P [Work ] = e−αt (1− p ) + e−1000αt p

P [GW ] P [W | G ]P [G ]
P [G | W ] = =
P [W ] P [W ]

e−αt (1− p )
= −αt −1000αt
= 0.99
e (1− p ) + e p

1  99 p 
t= ln  
999α 1− p 

W—Work, G—Good.

Dr Yue Rong (2008–2011), Dr Y Leung (2006, 2007) 1-14

Curtin University of Technology, Australia
1.4 Independent Events

(a) Pair of Events

• Events A and B are said to be independent if

P [ AB ] = P [ A] ⋅ P [B ] (23)

• Thus, if A and B are independent, then

P [ A B ] = P [ A] and P [B A ] = P [B ] (24)

• A , B mutually exclusive ⇔
/ A , B independent

Example (Fair Dice)

(a) A = {1, 2, 3} , B = {4, 5, 6}
(i) A∩B =∅
(ii) P [ AB ] = P [∅] = 0 ; P [ A] ⋅ P [B ] = 21 × 21 = 1
4
∴ A, B mutually exclusive, not independent

(b) A = {1, 2, 3} , B = {1}

(i) A ∩ B = {1}

(ii) P [ AB ] = P [{1}] = 61 ; P [ A] ⋅ P [B ] = 21 × 61 = 12
1

∴ A, B not mutually exclusive, not independent

(c) A = {1, 2} , B = {2, 4, 6}

(i) A ∩ B = {2}
(ii) P [ AB ] = P [{2}] = 61 ; P [ A] ⋅ P [B ] = 31 × 21 = 61
∴ A, B not mutually exclusive, independent

Dr Yue Rong (2008–2011), Dr Y Leung (2006, 2007) 1-15

Curtin University of Technology, Australia
 (d) A = ∅, B = {2, 4, 6}
(i) A∩B =∅
(ii) P [ AB ] = P [∅] = 0 ; P [ A] ⋅ P [B ] = 0 × 21 = 0
∴ A, B mutually exclusive, independent

Example (Unfair Dice)

Suppose dice now not fair with p1 = 0.5
6
, p3 = 1.5
6
and
p2 = p4 = p5 = p6 = 61

Repeating part (c) above, we have

A = {1, 2} , B = {2, 4, 6}
(i) A ∩ B = {2}

(ii) P [ AB ] = P [{2}] = 61 ;
P [ A] ⋅ P [B ] = ( 0.5
6
+ 61 ) × ( 61 + 61 + 61 ) = 0.75
6

∴ A, B not mutually exclusive as before, but now not independent

• Independence of events often not intuitively obvious – must do

calculations to verify

Dr Yue Rong (2008–2011), Dr Y Leung (2006, 2007) 1-16

Curtin University of Technology, Australia
(b) Triplet of Events

• Events A1, A2, A3 are independent if

(i) A1, A2 are independent

(ii) A2, A3 are independent

(iii) A1, A3 are independent

and (iv) P [ A1 A2 A3 ] = P [ A1] ⋅ P [ A2 ] ⋅ P [ A3 ] (25)

• Note: (i) and (ii) and (iii) ⇒

/ (iv)
Conversely, (iv) ⇒
/ (i) or (ii) or (iii)

Work Example 8
Suppose S = {1, 2, 3, 4} with P [s ] = 1 for all s ∈ S .
4

(a) Consider the three events

A1 = {1, 2} , A2 = {2, 3} , A3 = {3, 1}

Clearly, P [ A1] = P [ A2 ] = P [ A3 ] = 21 .

(i) Are A1, A2 , A3 pairwise independent?

(ii) Are they independent?

(b) Consider next the three events

A1 = {1, 3, 4} , A2 = {2, 3, 4} , A3 = ∅

Clearly, P [ A1] = P [ A2 ] = 34 and P [ A3 ] = 0 .

(i) Does P [ A1 A2 ] = P [ A1] ⋅ P [ A2 ] ?

(ii) Does P [ A1 A2 A3 ] = P [ A1] ⋅ P [ A2 ] ⋅ P [ A3 ] ?

Dr Yue Rong (2008–2011), Dr Y Leung (2006, 2007) 1-17

Curtin University of Technology, Australia
Solution:
(a) (i) P [ A1A2 ] = P [{2}] = 41 , P [ A1]P [ A2 ] = 1
4

P [ A1A3 ] = P [{1}] = 41 , P [ A1]P [ A3 ] = 1

4

P [ A2 A3 ] = P [{3}] = 41 , P [ A2 ]P [ A3 ] = 1
4

So, A1, A2 , A3 are pairwise independent.

(ii) P [ A1A2 A3 ] = P [∅] = 0 ≠ P [ A1]P [ A2 ]P [ A3 ]

So, A1, A2 , A3 are not independent.

(b) (i) P [ A1A2 ] = P [{3,4}] = 21 , P [ A1]P [ A2 ] = 34 × 34 = 16

9

P [ A1A2 ] ≠ P [ A1]P [ A2 ]

(ii) P [ A1A2 A3 ] = P [∅] = 0 = P [ A1]P [ A2 ]P [ A3 ]

• In other words, to check A1, A2 , A3 are independent, it is

necessary to test all 4 conditions listed in (25)

Dr Yue Rong (2008–2011), Dr Y Leung (2006, 2007) 1-18

Curtin University of Technology, Australia
(c) n Events

• In general, A1, A2 , …, An are independent if, for k = 2, …, n

P [ Ai1 ⋯ Ai k ] = P [ Ai1 ] ⋯ P [ Ai k ] ,
1 ≤ i1 < i 2 < ⋯ < i k ≤ n (26)

Example (4 Events)
Suppose n = 4 such that k = 2, 3, 4
(i) For k = 2 , we check:
P [ A1 A2 ] = P [ A1] P [ A2 ] ;

P [ A1 A3 ] = P [ A1] P [ A3 ] ;
P [ A1 A4 ] = P [ A1] P [ A4 ] ;
P [ A2 A3 ] = P [ A2 ] P [ A3 ] ;

P [ A2 A4 ] = P [ A2 ] P [ A4 ] ;

P [ A3 A4 ] = P [ A3 ] P [ A4 ]
(ii) For k = 3 , we check:
P [ A1 A2 A3 ] = P [ A1] P [ A2 ] P [ A3 ] ;

P [ A1 A2 A4 ] = P [ A1] P [ A2 ] P [ A4 ] ;

P [ A1 A3 A4 ] = P [ A1] P [ A3 ] P [ A4 ] ;

P [ A2 A3 A4 ] = P [ A2 ] P [ A3 ] P [ A4 ]
(iii) For k = 4 , we check:
P [ A1 A2 A3 A4 ] = P [ A1] P [ A2 ] P [ A3 ] P [ A4 ]

Dr Yue Rong (2008–2011), Dr Y Leung (2006, 2007) 1-19

Curtin University of Technology, Australia
1.5 Sequential Experiments

(a) Definitions and Notations

• A sequential experiment E consists of a sequence of simpler

subexperiments E1, E2, …, En , each with sample space
S1, S2, …, Sn

• Sample space S of E given by the Cartesian product

S = S1 × S2 × ⋯ × Sn (27)

• An outcome s of E is represented by the n-tuple

s = (s1, s2, …, sn ) (28)
where si ∈ Si , i = 1, …, n
• Suppose Ai is an event of Ei , i = 1, …, n . Probability of the
sequence of events A1, A2, …, An , given by the repeated
application of (18)

P [ An An−1 ⋯ A1] = P [ An An−1 ⋯ A1] ⋅ P [ An−1 An−2 ⋯ A1]

⋯ P [ A2 A1] ⋅ P [ A1] (29)

Dr Yue Rong (2008–2011), Dr Y Leung (2006, 2007) 1-20

Curtin University of Technology, Australia
(b) Tree Diagrams

• Tree diagrams give a handy visualisation of the sequence of

sub-experiments

• Each node represents sequence of events that has occurred up

to that point
Branches from a node describe all events that may arise in the
subsequent subexperiment. These events form an event space
Branches are labelled with conditional probabilities of the
events in the subsequent subexperiment

• Probability of observing a certain sequence of events, i.e.,

getting a certain node, is given by product of conditional
probabilities along the branches from the root to the node. Thus
P [ A35 A23 A12 ] = P [ A35 A12 A23 ] ⋅ P [ A23 A12 ] ⋅ P [ A12 ]

Above concurs with (29)

• Strictly P [ A35 A12 A23 ] should read P [ A12 A23 A35 A12 A23 ] , but as
can be shown from (18)

P [ A12 A23 A35 A12 A23 ] = P [ A35 A12 A23 ]

In any case, above result should be intuitively obvious

Dr Yue Rong (2008–2011), Dr Y Leung (2006, 2007) 1-21
Curtin University of Technology, Australia
Work Example 9
Have two coins. Coin 1 is biased with P [H ] = 3 and P [T ] = 41 .
4
Coin 2 is fair with P [H ] = P [T ] = 21 . Experiment consists of E1 :
randomly choosing a coin, and then E2 : flip it.
Determine (a) P [C1 | H ]
(b) P [C1 | T ]

Solution:
(a) P [H ] = P [C1H ] + P [C2H ] = P [H | C1]P [C1] + P [H | C2 ]P [C2 ]

= 34 × 21 + 21 × 21 = 85

P [H | C1]P [C1] 34 × 21 3
P [C1 | H ] = = 5 =5
P [H ] 8

(b) P [T ] = P [C1T ] + P [C2T ] = P [T | C1]P [C1] + P [T | C2 ]P [C2 ]

= 41 × 21 + 21 × 21 = 83

P [T | C1]P [C1] 41 × 21 1
P [C1 | T ] = = 3 =3
P [T ] 8

Dr Yue Rong (2008–2011), Dr Y Leung (2006, 2007) 1-22

Curtin University of Technology, Australia
Work Example 10 (Binary Communication System)
(Work Example 5 revisited)
Transmitter sends '1' with probability p and '0' with probability
1− p .
Receivers detects transmitted symbols with probability of error ε .
Define: Ti to be the event {symbol ' i ' transmitted}, i = 0, 1
Ri to be the event {symbol ' i ' decoded}, i = 0, 1
Determine, in terms of p and ε
(a) P [T0 | R0 ] (c) P [T0 | R1]

(b) P [T1 | R0 ] (d) P [T1 | R1]

Solution: Tree diagram

Dr Yue Rong (2008–2011), Dr Y Leung (2006, 2007) 1-23

Curtin University of Technology, Australia
(c) Independent Trials

(i) Binomial Probability Law

• A Bernoulli trial consists of performing an experiment E once

and noting whether a certain event A has occurred
Its sample space is thus binary with outcomes {success, failure}

• Probability of A occurring n1 times in n independent

repetitions (Bernoulli trials) of E is given by the binomial
probability law
n
P [Sn0 , n1 ] =   pn1 (1− p )n0 , n1 = 0, 1, …, n (30)
n1

where n0 = n − n1 is number of failures, p is probability of

success, i.e. p = P [ A] , and
 
 n  = n!
(31)

n1 n1! (n − n1)!

• It can be shown that

(n − n1)p
P [Sn0 −1, n1+1] = P [Sn0 , n1 ] (32)
(n1 + 1) (1− p )

• If n0 and n1 are large, following approximations can be useful

De Moivre-Laplace approximation
( n1 − np )2
 n  n 1 −
  p 1 (1− p )n−n1 ≈ e 2np(1− p )
(33)
n1 2πnp(1− p )

Poisson approximation (p small)

  n1
 n  pn1 (1− p )n−n1 ≈ (np ) e−np (34)
n1 n1!

Dr Yue Rong (2008–2011), Dr Y Leung (2006, 2007) 1-24

Curtin University of Technology, Australia
(ii) Multinomial Probability Law

• Generalisation of the binomial probability law

• Suppose { A1, A2, …, Am } is an event space of S , the sample
space of E , with P [ Ai ] = pi

• Probability of gaining ni successes of Ai , i = 1, …, m , in n =

n1 + ⋯ + nm independent trials of E , is given by the
multinomial probability law
n!
P [Sn1, …, nm ] = p1n1 p2n2 ⋯ pm
nm
(35)
n1! n2 ! ⋯ nm !

(iii) Geometric Probability Law

• Suppose independent Bernoulli trials of E are repeated until

event A occurs. Probability that m trials are required is given
by the geometric probability law
p(m ) = P [ AC (1) ∩ AC (2) ∩ ⋯ ∩ AC (m − 1) ∩ A(m )]
p(m ) = (1− p )m−1 p (36)
where A(i ) denotes the event “success in i th trial”, and
p = P [ A]

• It can be verified that

∞
∑ p(m) = 1 (37)
m=1

∞
and P [{m > K }] = ∑ p(m ) = (1− p )K (38)
m=K +1

Dr Yue Rong (2008–2011), Dr Y Leung (2006, 2007) 1-25

Curtin University of Technology, Australia
(iv) Markov Chains

• Suppose subexperiments of a sequential experiment E are

not independent but satisfy following property
P [ Ai Ai −1 ⋯ A0 ] = P [ Ai Ai −1] (39)
i.e. probability of event Ai occurring in i th subexperiment,
given knowledge of outcomes of all previous subexperiments,
depends only on outcome of the most recent subexperiment
E is called a Markov chain

• Thus
P [ An ⋯ A0 ] = P [ An An−1] ⋅ P [ An−1 An−2 ] ⋯ P [ A1 A0 ] ⋅ P [ A0 ] (40)
and E is characterised by the set of transitional probabilities
P [ Ai Ai −1]

Work Example 11
A communication system sends a bit 3 times. Receiver detects bits
with probability of error ε = 10−3 . Receiver detects 3 bits and
makes a majority vote. Thus it can tolerate 1 error but not 2 or 3
errors. Suppose each transmission is an independent Bernoulli trial.
Determine probability that the receiver will make an error.
Solution:
P [E ] = P [2bits ] + P [3bits ]
3 2 3 3
( )( )
=   10−3 1− 10−3 +   10−3
2 3
( ) ≈ 3 ×10−6

Work Example 12
A target area is partitioned into 3 areas A1, A2 , A3 . Suppose
probabilities of hitting these areas are P [ A1] = 0.2 , P [ A2 ] = 0.3
and P [ A3 ] = 0.5 . Determine probability that in 9 shots, 2 shots will
hit A1, 3 shots will hit A2 , and 4 shots will hit A3 .

Dr Yue Rong (2008–2011), Dr Y Leung (2006, 2007) 1-26

Curtin University of Technology, Australia
Solution:
9! 2 3 4
P [2,3,4] = (0.2) (0.3) (0.5) = 0.0851
2!3!4!

Work Example 13
In a communication system, the receiver can detect (any) errors
and will request a retransmission. Suppose probability of error in a
transmission is ε = 0.1 . Find probability that more than 2
transmissions are required.

Solution:
P [T > 2] = 1− P [T = 1] − P [T = 2] = 1− (1− ε ) − ε(1− ε ) = ε 2 = 0.01
2
Or P [T > 2] = 1− (1− ε) = ε2 = 0.01

1.6 Reliability Problems

• Engineering systems often consist of a number of components

working together
Question: Knowing failure probabilities of each component,
what is probability overall system will fail?
Clearly, answer depends on interconnection of components

• Two basic configurations: components in series

components in parallel

• Denote by W the event the overall system is functioning, and

by
Wi the event the i th component is functioning
Assume W1, W2, … are independent

Dr Yue Rong (2008–2011), Dr Y Leung (2006, 2007) 1-27

Curtin University of Technology, Australia
• Components in series – system fails if any component fails

W1 W2 W3

P [W ] = P [W1W2W3 ] = P [W1] ⋅ P [W2 ] ⋅ P [W3 ] (41)

• Components in parallel – systems fails only if all components

fail

W1

W2

W3

P [W C ] = P [W1CW2CW3C ] = P [W1C ] ⋅ P [W2C ] ⋅ P [W3C ]

or P [W ] = 1 − {1− P [W1]}{1− P [W2 ]} {1− P [W3 ]} (42)

• Note: above 2 figures do not necessarily correspond to the

actual physical interconnection of the components

• Reliability of systems with more complicated interconnection of

components can be determined by reducing successively the
interconnection to the simpler forms shown above

Dr Yue Rong (2008–2011), Dr Y Leung (2006, 2007) 1-28

Curtin University of Technology, Australia
 Example

W1 W2 W4

W3 W3

P [W4 ] = P [W1] ⋅ P [W2 ]

P [W ] = 1 − {1− P [W4 ]} {1− P [W3 ]}

= 1 − {1− P [W1] ⋅ P [W2 ]} {1− P [W3 ]}
= P [W1] ⋅ P [W2 ] + P [W3 ] − P [W1] ⋅ P [W2 ] ⋅ P [W3 ]

Dr Yue Rong (2008–2011), Dr Y Leung (2006, 2007) 1-29

Curtin University of Technology, Australia