Random Variables

Chapter 1 Part 1

Sample Space and Events

Experiments
Any process of trial and observation
Random experiments : an experiment whose outcome is uncertain

Sample space
The collection of possible elementary outcomes
Sample points : the elementary outcomes of an experiment denoted by wi, i=1,2,

S w1 , w2 ,, wn
Sample space

Sample points

Event
any one of a number of possible outcomes of an experiment
a subset of the sample space

Example

Elementary outcomes

if we toss a die, the sample space is S 1,2,3,4,5,6

the outcome of the toss of a die is an even number E 2,4,6
three coin-tossing experiment S HHH , HHT , HTH , HTT , THH , THT , TTH , TTT
the event one head and two tails E HTT , THT , TTH
2

Sample Space and Events

In a single coin toss experiment,
sample space S H , T
the event that a head appears on the toss E H
the event that a tail appears on the toss E T

If we toss a coin twice

sample space
the event that a head appears on the 2nd toss

If we toss a die twice

sample space
the event that the sum of the two tosses is 8

If we measure the lifetime of an electronic component

Sample space
the event that the lifetime is not more that 7 hours
3

Sample Space and Events

1.2 Sample Space and Events algebra of events
union of events A and B

C A B

intersection of events A and B

D A B

Union of events A and B : the event that consists of all sample points that are
either in A or in B or in both A and B
Intersection of events A and B : the event that consists of all sample points that
are in both A and B
Mutually exclusive : if their intersection contains no sample point.
Difference of events A and B : the event that all sample points are in A but not in
B. C A B
4

Definition of Probability
Three different kinds of definitions for Probability
axiomatic, relative-frequency, classical definitions

1.3.1 Axiomatic Definition

For each event defined on a sample space S, we shall assign a nonnegative
number
Probability is a function : It is a function of the events defined
P(A) : The probability of event A

Axiom 1

0 P( A) 1

; work with nonnegative numbers

Axiom 2

P(S ) 1

; sample space itself is an event,

it should have the highest probability

N N
PU An P( An )
if Am An
n 1 n 1
for all m n = 1, 2, , N with N possibly infinite
; the probability of the event equal to the union of any number of
mutually exclusive events is equal to the sum of the individual event probabilities.

Axiom 3

Definition of Probability
Definition of probability Example

unbiased

Obtaining a number x by spinning the pointer on a fair wheel of chance that is labeled from 1
to 100 points.
Sample space
S {x | 0 x 100 }
The probability of the pointer falling between any two numbers x2 x1

Consider events

P( x1 x x2 ) ( x2 x1 ) / 100

Axiom 1

0 P( A) 1 A {x1 x x2 }

Axiom 2

P(S ) 1

Axiom 3

Break the wheels periphery into N continuous segments, n=1,2,N with x 0=0

for any

; for all x1, x2

x2 100 and x1 0

An {xn1 x xn }, xn (n)100 / N

P( An ) 1 / N

P An P( S ) 1
n 1
N
N
1
P( An ) 1

n 1
n 1 N
6

Definition of Probability
1.3.2 Relative frequency definition
Probability as a relative frequency
Flip a coin : heads show up nA times out of the n flips
Probability of the event heads
n
P( A) lim A
n n
Statistical regularity : relative frequencies approach a fixed value (a probability) as n
becomes large.

1.3.3 Classical Definition

Probability as a classical definition
P( A)

NA
N

This probability is determined a priori without actually performing the experiment.

Definition of Probability

Example Tossing two dice

Figure 1.1 Sample Space for Example 1.1
A1

Second Die

(1,6)

(2,6)

(3,6)

(4,6)

(5,6)

(6,6)

(1,5)

(2,5)

(3,5)

(4,5)

(5,5)

(6,5)

(1,4)

(2,4)

(3,4)

(4,4)

(5,4)

(6,4)

(1,3)

(2,3)

(3,3)

(4,3)

(5,3)

(6,3)

(1,2)

(2,2)

(3,2)

(4,2)

(5,2)

(6,2)

(1,1)

(2,1)

(3,1)

(4,1)

(5,1)

(6,1)

3
2
1

A2

- Sample space : 62=36 points

- For each possible outcome,
a sum having values from 2 to 12

A1 {sum 7}, A2 {sum 11}

B {sum 7}or{sum 11}
C {2 nd die 1st die},
D {both dice even}

First Die

1 1
P( A1 ) 6 ,
36 6
P(C ) ,

1 1
P( A2 ) 2
36 18

P( B) P( A1 ) P( A2 )

2
,
9

P( D)

Definition of Probability

Summary - Mathematical model of Experiments

A real experiment is defined mathematically by three thing
1. Assignment of a sample space
2. Definition of events of interest
3. Making probability assignment to the events such that the axioms are satisfied

Generally, it is not easy to construct correct mathematical model

A die worn out

Definition of Probability

Exercise conditional probability

Example
80 resistors in a box : 10W -18, 22W -12, 27W -33, 47W -17, draw out one resistor,
equally likely
P(draw10) 18 / 80 P(draw 22) 12 / 80
P(draw 27) 33 / 80 P(draw 47) 17 / 80

Suppose a 22W is drawn and not replaced. What are not the probabilities of drawing a
resistor of any one of four values?

P(draw10 | 22) 18 / 79
P(draw 22 | 22) 11 / 79
P(draw 27 | 22) 33 / 79
P(draw 47 | 22) 17 / 79

The concept of conditional probability is needed.

10

Applications of Probability

# Homework for reading : 1.4 Application of Probability

1.4.1 Reliability Engineering
1.4.2 Quality Control (QC)
1.4.3 Channel Noise
1.4.4 System simulation
random # of generation that can be used to represent events such as
arrival of customers at a bank in the sytem being modeled
Our main work will be focused on the area

11

Elementary Set Theory

Definitions
Set : a collection of objects A (capital letter)
Objects : Elements of the set a (small letter)
a A
If a is an element of set A :
If a is not an element of set A :
a A
Methods for specifying a set

Ex) -

a
a
a
a

set
set
set
set

of
of
of
of

voltages
airplanes
chairs
sets

Tabular method
Ex) {6, 7, 8, 9}
Rule method
Ex) {integers between 5 and 10}, {i | 5 < i < 10, i an integer}

Set
Countable, uncountable
Finite, infinite
Null set(=empty) :
a subset of all other sets
countably infinite set

12

Set Definitions

Definitions
A is a subset of B
If every element of a set A is also an element in another set B, A is said to be
contained in B.
A B

A is a proper subset of B
If at least one element exists in B which is not in A

A B

Two sets, A and B are called disjoint or mutually exclusive if they have no
common elements

13

Set Definitions

Example

A {1,3,5,7}
B {1,2,3, }
C {0.5 c 8.5}

D {0.0}
E {2,4,6,8,10,12,14}
F {5.0 f 12.0}

A : Tabularly specified, countable, and finite

B : Tabularly specified, countable, and infinite
C : Rule-specified, uncountable, and infinite
D and E : Countably finite
F : Uncountably infinite
D is the null set?
A is contained in B, C, and F
C F , D F and E B
B and F are not subsets of any of the other sets or of each other
A, D, and E are mutually exclusive of each other
14

Set Definitions
Universal set
The largest set or all-encompassing set of objects under discussion in a given
situation

Power set
Power set of A : the set of all subsets of a set A, s(A)
Example
A = {a,b} s(A) = {{a}, {b}, {a,b}, }

Cardinality
Cardinality of A : the number of members of a set A, |A|.
Example
A = {a,b} |A| = 2
|s(A)| = 2n when |A| = n

15

Set Definitions
Rolling a die (Example 1.1-2)
S={1,2,3,4,5,6}
A person wins if the number comes up odd : A={1,3,5}
Another person wins if the number shows four or less : B={1,2,3,4}
Both A and B are subset of S

For any universal set with N elements, there are 2N possible subsets of S
Example : Token
S = {T, H}

{}, {T}, {H}, {T,H}

Example : Tossing a token twice

S = {TT, HT, TH, HH}

24=16 number of subsets exist

Example : Rolling a die

S = {1, 2, 3, 4, 5, 6}

26=64 number of subsets exist

16

Set Definitions Problem Solving

Problems
Specify the following sets by the rule method.
A={1,2,3}
->
A={k | 0 < k < 4}
B={8,10,12,14} -> B={k | 6 < k <16, k/2 }
C={1,3,5,7,}
->
C={2k-1 | k is the positive integer}

State every possible subset of the set of letters {a,b,c,d}

{}, {a}, {b}, {c}, {d}, {a,b}, {a,c}, {a,d}, {b,c}, {b,d}, {c,d}, {a,b,c}, {a,b,d}, {a,c,d}.
{b,c,d}, {a,b,c,d} -> Total 16 number of subsets

A random noise voltage at a given time may have any value from -10 to 10V.
(a) What is the universal set describing noise voltage?
-> S={s R | -10s10}
(b) Find a set to describe the voltages available from a half-rectifier for positive voltages that
has a linear output-input characteristic.
-> V={s R | 0s10}
(c) Repeat parts (a) and (b) if a DC voltage of -3V is added to the random noise.
-> S={ s R | -13s7}, V={s R|0s7}
17

Set Operations
Venn Diagram

Universal set

C is disjoint from both A and B

B is a subset of A

Equality : A=B
Two sets are equal if all elements in A are present in B and all elements in B are in A
That is, if

A B and B A

A B

Difference : A-B
The difference of two sets A and B is the set containing all elements of A that are not
present in B
Example

A {0.6 a 1.6}, B {1.0 b 2.5}

A B {0.6 a 1.0}, B A {1.6 b 2.5}

A B B A
18

Set Operations

Union and intersection

Union (Sum) : C A B
The union (call it C) of two sets A and B
The set of all elements of A or B or both

Intersection (Product) : D A B
The intersection (call it D) of two sets A or B
The set of all elements common to both A and B
For mutually exclusive (M.E.) sets A and B,

A B

The union and intersection of N sets An, n=1,2,,N

C A1 A2

AN

An ,

D A1 A2

n 1

AN

An
n 1

Complement
The complement of the set A is the set of all elements not in A
ASA

S , S , A A S , and A A
19

Set Operations

Example
S {1 integers 12}
A {1,3,5,12}
B {2,6,7,8,9,10,11}
C {1,3,4,6,7,8}
Union (Sum) and Intersection (Product)

A B {1,2,3,5,6,7,8,9,10,11,12}
A C {1,3,4,5,6,7,8,12}
B C {1,2,3,4,6,7,8,9,10,11}

A B
A C {1,3}
B C {6,7,8}

Complement
A {2,4,6,7,8,9,10,11}
B {1,3,4,5,12}
C {2,5,9,10,11,12}
20

Set Operations

Duality Principle
If in an identity we replace unions by intersections, intersections by unions,
A ( B C ) ( A B) ( A C )
A ( B C ) ( A B) ( A C )

Example

A {1,2,4,6}
B {2,6,8,10}
C {3 c 4}

B C {2, 3 c 4, 6,8,10}
A B {2,6}
A C {4}

A ( B C ) {2,4,6}
( A B) ( A C ) {2,4,6}

A ( B C) ( A B) ( A C)
21

Set Operations

Algebra of Sets
Commutative law
A B B A
A B B A

Distributive law
A ( B C ) ( A B) ( A C )
A ( B C ) ( A B) ( A C )

Associative law
( A B) C A ( B C ) A B C
( A B) C A ( B C ) A B C

22

Set Operations

De Morgans law
The complement of a union (intersection) of two sets A and B equals the
intersection (union) of the complements A and B
( A B) A B
( A B) A B

Example
S {2 s 24}
A {2 a 16}, B {5 b 22}
C A B

A B {5 c 16}

C A B {2 c 5, 16 c 24}

A S A {16 a 24},
B S B {2 a 5, 22 a 24}
C A B {2 c 5, 16 c 24}

( A B) A B
23

Set Operations Problem Solving

Problems
Show that C A if C B and B A.
Explain it by using Ven diagram

Two sets are given by A={-6, -4, -0.5, 0, 1.6, 8} and B={-0.5,0,1,2,4}. Find:
(a) A-B ->
(b) B-A ->
(c) AB ->
(d) AB->

{-6, -4, 1.6, 8}

{1, 2, 4}
{-6, -4, -0.5, 0, 1, 1.6, 2, 4, 8}
{-0.5, 0}

1.2-4. Using Venn diagrams for three sets A,B,C, shade the areas corresponding to
the sets:
(a) (AB)-C
(b) A-B
(c) C-(AB)

24

Set Operations Problem Solving

Problems
Sketch a Venn diagram for three events where AB0, BC0, CA0, but ABC=0.
Show the equations of Venn diagrams

Sets A={1s14}, B={3,6,14}, and C={2<s9} are defined on a sample space S. State if
each of the following conditions is true or false.
(a) CB False
(b) CA True
(c) BC=0 False
(d) CUB=S False
(e) S

(f) A S
(g) C A B

25

Properties of Probability
Properties of Probability
1. The probability of the complement of A is one minus the probability of A.
P( A ) 1 P( A)

2. The null event has probability zero. P() 0

3. If A is a subset of B, the probability of A is at most the probability of B.
A B P( A) P( B)

4. P(A)1 the probability of event A is at most 1.

5. N N
PU An P( An )
n 1 n 1

if

Am An

6. A ( A B) ( A B ) P( A) P( A B) P( A B )
7. P( A B) P( A) P( B) P( A B) Joint Probability P( A B)
8. Generalization of Property 7
P( A B) P( A) P( B) P( A B) P( A) P( B)
The probability of the union of two events never exceeds the sum of the event probabilities.

26