Statistics University of Wisconsin Madison – Chelsey Green

[email protected]

NOTES 3: PROBABILITY AND SIMULATION

THE RELATIONSHIP BETWEEN INFERENTIAL STATISTICS AND PROBABILITY [AND SIMULATION]
In inferential statistics, we look at the information from our sample and would like to make a good guess
about what is true in the population from which the sample was drawn. In order to do that, we need
probability (and/or simulation) to understand the variety of sample characteristics we could find when
drawing from a known population.

Take the figure at right:

Inferential statistics asks “What models of the population (pail contents) make the
observed sample (what is in my hand) plausible?” and

Probability asks “Given we know or assume information about the population (in the
pail), what is likely to occur in our sample (in your hand)?”

PROBABILITY TERMINOLOGY

Video Screen Tension Example: A video terminal manufacturer must control the
tension on the mesh of fine wires that lies behind the surface of the viewing screen.
The tension is measured by an electrical device with output readings in millivolts
(mV). The tension readings for the 20 screens produced in the last hour are given
below in R code.

tension<-c(269.5, 297.0, 269.6, 283.3, 304.8, 280.4,

233.5, 257.4, 317.5, 327.4, 264.7, 307.7, 310.0, 343.3,
328.1, 342.6, 338.8, 340.1, 374.6, 336.1)

hist(tension, labels=TRUE, main="Video Screen

Tensions", breaks=seq(from=220,to=380,by=20),
ylim=c(0,7), xaxp=c(220, 380, 8),
xlab="Tension (mV)")

Video Screen Tension Example a: Too much tension can lead to mesh tears. If a screen is chosen from the
20 produced, what is the chance/probability that it has screen tension above 320 mV? You can assume
that any screen is equally likely to be chosen.
Statistics University of Wisconsin Madison – Chelsey Green
[email protected]
A random process is any process that generates information in which the result has some amount of
random chance involved
E.g 1: flip coin and record what side (H or T) is face up
E.g 2: sample three students and calculate the average height
E.g. 3: choose 1 of 55 cars at random and record MPG

An [elementary] outcome is a distinct result of a random process.

E.g.1: H lands up when flipping coin;
E.g.2 :average weight of 3 students was 145.7 lbs.,
E.g.3: the 1 car chosen had MPG of 40

A sample space S is the set of all possible outcomes of a random process. The sum of all [elementary]
outcomes’ probabilities in the sample space is 1.
E.g.1: S={H, T} is the sample space for face that shows up on a flip of a coin ,
E.g. 2: S={145.7, 143.3, 151.2, all possible average heights for 3 students}, is the sample space for
average height of 3 randomly chosen students
E.g. 3: S={58, 56, 55, 52…[51 remaining possible MPG]} is the sample space for the MPG of 1 car
chosen at random

An event E is a collection of outcomes (a subset of the sample space). An event is said to have occurred if
at least one of the outcomes in the collection occurs.
E.g.1: When rolling a fair 6-sided die and recording the side that lands up, the sample space
S={1,2,3,4,5,6}. We would say the event “even side landed up” occurred if a 2,4, or 6 side landed up.
E.g. 2: We say a MPG over 50 occurred if a car with MPG {58,56,55,52, 52} was chosen.

The Probability of an Event has a few different interpretations that can be useful/applicable:
1. classical interpretation: if all outcomes are equally likely and enumerable, the probability of an
event E is computed by taking the ratio of the number of outcomes 𝑁𝑒 favorable to the event E to the
𝑁
total number of possible outcomes, 𝑁. 𝑃(𝑒𝑣𝑒𝑛𝑡 𝐸) = 𝑁𝑒.
E.g.: There are 13 cards in the suit of Hearts in a deck of 52 cards. Assuming each card is
13
equally likely to be picked, 𝑃(𝑐𝑎𝑟𝑑 𝑐ℎ𝑜𝑠𝑒𝑛 𝑖𝑠 𝑎 ℎ𝑒𝑎𝑟𝑡) = 52 = 0.25.
2. relative frequency interpretation: the proportion of times the event is expected to occur in
many repeated trials of an experiment.
E.g.: In a simulation of many replications of an experiment, an outcome of interest occurred
25% of the time, thus we would estimate the probability of the outcome as 0.25.
3. personal/subjective interpretation: a one-shot statement of belief about the likelihood of an
event based on personal experience. Probability is a degree of belief that the event will occur.
E.g.: Expert offers probability that legislation will be passed
Statistics University of Wisconsin Madison – Chelsey Green
[email protected]
Breast Cancer Example: Many women choose to have annual mammograms to screen for breast cancer after
age 40. A two-way frequency table summarizes the Breast Breast Total
most recent mammogram findings and breast cancer Cancer: Yes Cancer: No
diagnosis for 10,000 (hypothetical) women over 40 Mammogram: 97 594 691
from a region of interest. Positive
Mammogram: 3 9306 9309
Negative
Breast Cancer Example a: Consider the random Total 100 9900 10,000
process of choosing a woman from this sample of
10,000 women and recording the outcome of whether they had breast cancer or not. We would say the
probability of the event: patient with breast cancer is 100/10000=0.01 using the classical interpretation of
probability. We would also say the probability of the event: patient has a positive mammogram is
691/10000=0.0691. Since this is a large representative sample of women from the region, 0.01 and 0.0691 may
be used as an empirical estimate for the probability of breast cancer and probability of positive mammogram,
respectively, in the entire region.

Video Screen Tension Example: Use the histogram that gives the mesh
tension for the 20 screens produced in the last hour to answer the
following questions (using the convention that values on borderline are
included in the lower bin).
Video Screen Tension Example b: What is the probability that the 1
screen chosen has tension 320 or below?

Video Screen Tension Example c: What is the probability that the 1 screen chosen has tension outside of
the target tension range (above 360 or less than or equal to 240)?

Video Screen Tension Example d: What is the probability that the 1 screen chosen has tension above 360
or above 320?

The union of two events A and B, or “A or B” is denoted by 𝐴 ∪ 𝐵. “A or B” is the

set of outcomes that belong to A, to B, or to both [in the intersection].
E.g.1 : see Figure 1 at right.
E.g.2: Event A: MPG of car is above 50. All cars with MPG > 50 would be in the
A circle. Event B: Cost of Car is <$60,000. All cars with cost <$60,000 would be in the B
circle. “A or B”: MPG of car is above 50 or cost of Car is <$60,000. All cars with
MPG>50 or cost <$60,000 would fall into one of the circles (or the overlap if some cars Figure 1
have both properties)
Statistics University of Wisconsin Madison – Chelsey Green
[email protected]
The intersection of two events A and B or “A and B” is denoted by 𝐴 ∩ 𝐵. “A and B” is the set of outcomes
that belong to both A and B.
E.g.1 : see Figure 1 at right.
E.g.2: Event A: MPG of car is above 50. Event B: Cost of Car is <$60,000. “A and B”: MPG of car is
above 50 and cost of Car is <$60,000. All cars with MPG>50 and cost <$60,000 would fall into the
intersection/overlap of the two circles.

Events A and B are mutually exclusive if they have no outcomes in common. The
intersection of mutually exclusive events is empty.
E.g.1: see Figure 2 at right.
E.g.2: Drawing an Ace and drawing a King in a standard deck are mutually
exclusive events because each card can only be one or the other (not both)
Figure 2

The complement of an event A or “not A” is denoted by 𝐴𝑐 .“Not A” is the set of all outcomes that are not
in A. The occurrence of 𝐴𝑐 means that A does not occur.
E.g.1: see Figure 3 at right.
E.g. 2: Event A: face up on dice roll is 1, Event 𝐴𝑐 : face up on dice roll is 2,3,4,5, or 6
E.g. 3: Event A: MPG of car is greater than 50, Event 𝐴𝑐 : MGP of car is 50 or below
Figure 3

Probability Addition Law: 𝑃(𝐴 𝑜𝑟 𝐵) = 𝑃(𝐴 ∪ 𝐵) = 𝑃(𝐴) + 𝑃(𝐵) − 𝑃(𝐴 ∩ 𝐵): The probability of A or
B is the probability of A plus the probability of B minus the intersection of A and B.
*Notice, if A and B are mutually exclusive, there will be no overlap and P(A or B)=P(A)+P(B)

Probability Law of Complement : 𝑃(𝐴𝑐 ) = 1 − 𝑃(𝐴)

E.g. 1: P(face up on 1 roll of die is 1)=1/6 so P(face up on 1 roll of die is not a 1)=5/6
E.g. 2: P(MPG of car is 50 or below)=1-P(MPG of car is above 50).

Breast Cancer Example: Many women choose to have Breast Breast Total
annual mammograms to screen for breast cancer after age Cancer: Yes Cancer: No
40. A two-way frequency table summarizes the most Mammogram: 97 594 691
recent mammogram findings and breast cancer diagnosis Positive
for 10,000 (hypothetical) women over 40 from a region of Mammogram: 3 9306 9309
interest. Negative
Total 100 9900 10,000
Breast Cancer Example b: If we sampled a patient at
random from the 10,000, the probability of choosing a patient that has breast cancer or has a positive
mammogram is:
Let’s denote A as having breast cancer, B as having positive mammogram. Then 𝐴 ∩ 𝐵 is having both breast
cancer and a positive mammogram, 𝐴 ∪ 𝐵 is having breast cancer or positive mammogram.
P(𝐴 ∪ 𝐵) = 𝑃(𝐴) + 𝑃(𝐵) − 𝑃(𝐴 ∩ 𝐵) =100/10000+691/10000-97/10000=0.0694
[P(Yes Breast Cancer or + Mammogram)= P(Yes Breast Cancer)+P(+ Mammogram)-P(Yes Breast Cancer and
+Mammogram)=100/10000+691/10000-97/10000=0.0694 ]
Statistics University of Wisconsin Madison – Chelsey Green
[email protected]
Breast Cancer Example c: Are the events Patient has Breast Cancer and Patient receives Negative
Mammogram mutually exclusive in this set of 10,000 women?
No. From the table we see that there are 3 patients who were diagnosed with breast cancer and also had
a negative mammogram.

Video Screen Tension Example: Use the histogram that gives the mesh tension for
the 20 screens produced in the last hour to answer the following questions
(using the convention that values on borderline are included in the lower bin).
Consider selecting each screen [tension] with the same liklihood.

Video Screen Tension Example e: Suppose your colleague removed one of the
screens with tension over 320 and hasn’t returned it yet. What is the probability
that you will choose a screen with tension over 320 from the remaining screens ?

Video Screen Tension Example f: Suppose that the both the screen chosen by your colleague and the one
chosen by you are given to the same customer. What is the probability that both screens will have tension
over 320 mV?

Video Screen Tension Example g: Suppose that the both the screen chosen by your colleague and the one
chosen by you are given to the same customer. What is the probability that exactly one of the screens will
have tension over 320 mV?

Video Screen Tension Example h: Suppose that the both the screen chosen by your colleague and the one
chosen by you are given to the same customer. What is the probability that neither of the screens have
mesh tensions over 320 mV?
Statistics University of Wisconsin Madison – Chelsey Green
[email protected]

Conditional Probability of A given B. This is the revised probability of A given that B has occurred and it is
𝑃(𝐴 ∩𝐵) 𝑃("𝐴 𝑎𝑛𝑑 𝐵")
donated: 𝑃(𝐴 |𝐵). It is calculated: 𝑃(𝐴 |𝐵) = = . Note, “|” is read “given” and 𝑃(𝐵) ≠
𝑃(𝐵) 𝑃(𝐵)
0.

Multiplication Law of Probability : 𝑷(𝑨 𝒂𝒏𝒅 𝑩) = 𝑃(𝐴 ∩ 𝐵) = 𝑃(𝐵) ∗ 𝑃(𝐴|𝐵). The probability of both A
and B occurring is the product of the probability of B times the probability of A conditioned on B
occurring.

Breast Breast Total

Breast Cancer Example: Many women choose to have Cancer: Yes Cancer: No
annual mammograms to screen for breast cancer after Mammogram: 97 594 691
age 40. A two-way frequency table summarizes the most Positive
recent mammogram findings and breast cancer diagnosis Mammogram: 3 9306 9309
for 10,000 (hypothetical) women over 40 from a region of Negative
interest. Total 100 9900 10,000

Breast Cancer Example d: Consider sampling from the 10,000 patients summarized in the table. What is
the probability that a patient actually has breast cancer given they had a positive mammogram?

We compute this conditional probability:

97
𝑃(𝑌𝑒𝑠 𝐵𝑟𝑒𝑎𝑠𝑡 𝐶𝑎𝑛𝑐𝑒𝑟 ∩ +𝑀𝑎𝑚𝑚𝑜𝑔𝑟𝑎𝑚) 10000 97
𝑃(𝑌𝑒𝑠 𝐵𝑟𝑒𝑎𝑠𝑡 𝐶𝑎𝑛𝑐𝑒𝑟| + 𝑀𝑎𝑚𝑚𝑜𝑔𝑟𝑎𝑚) = = 691 = 691 = 0.1404.
𝑃(+𝑀𝑎𝑚𝑚𝑜𝑔𝑟𝑎𝑚)
10000

Breast Cancer Example e: Consider sampling from the 10,000 patients summarized in the table. What is the
probability that the mammogram was positive given that the patient actually had breast cancer?

We compute this conditional probability:

97
𝑃(𝑌𝑒𝑠 𝐵𝑟𝑒𝑎𝑠𝑡 𝐶𝑎𝑛𝑐𝑒𝑟 ∩ +𝑀𝑎𝑚𝑚𝑜𝑔𝑟𝑎𝑚) 10000 97
𝑃(+ 𝑀𝑎𝑚𝑚𝑜𝑔𝑟𝑎𝑚|𝑌𝑒𝑠 𝐵𝑟𝑒𝑎𝑠𝑡 𝐶𝑎𝑛𝑐𝑒𝑟) = = 100 = = 0.97.
𝑃(𝑌𝑒𝑠 𝐵𝑟𝑒𝑎𝑠𝑡 𝐶𝑎𝑛𝑐𝑒𝑟) 100
10000
This value is referred to as the True Positive Rate (or Sensitivity) of the diagnostic test.

Breast Cancer Example f: What is the probability that a randomly chosen patient had breast cancer and
received a positive mammogram?

We can read this value directly off of the frequency table:

97
𝑃(𝑌𝑒𝑠 𝐵𝑟𝑒𝑎𝑠𝑡 𝐶𝑎𝑛𝑐𝑒𝑟 ∩ +𝑀𝑎𝑚𝑚𝑜𝑔𝑟𝑎𝑚) = 10000 = 0.0097.
We could also use our Multiplication Law of Probability: 𝑃(𝑌𝑒𝑠 𝐵𝑟𝑒𝑎𝑠𝑡 𝐶𝑎𝑛𝑐𝑒𝑟 𝑎𝑛𝑑 + 𝑀𝑎𝑚𝑚𝑜𝑔𝑟𝑎𝑚) =
691 97 97
𝑃(+𝑀𝑎𝑚𝑚𝑜𝑔𝑟𝑎𝑚) ∗ 𝑃(𝑌𝑒𝑠 𝐵𝑟𝑒𝑎𝑠𝑡 𝐶𝑎𝑛𝑐𝑒𝑟 |+𝑀𝑎𝑚𝑚𝑜𝑔𝑟𝑎𝑚) = ∗( )= = .0097
10000 691 10000

Breast Cancer Example g: What is the probability of choosing two patients that do not have Breast Cancer
when taking a random sample of two people (without replacement)?

We can again use our Multiplication Law of Probability:

Statistics University of Wisconsin Madison – Chelsey Green
[email protected]
P(1st person: No BC And 2nd Person: No BC)=
P(1st person: No BC)*P(2nd person: No BC|1st person: No BC)=9900/10000*9899/9999=0.980099 which
tells us choosing two patients from this group without replacement that do not have Breast Cancer is a
very likely event

Independence: Two events A and B are independent if the occurrence of one does not change the
probability of the other. 𝑷(𝑨 |𝑩) = 𝑷(𝑨) equivalently 𝑷(𝑩|𝑨) = 𝑷(𝑩)
If A and B are independent, the Multiplication Law of Probability becomes:
𝑷(𝑨 𝒂𝒏𝒅 𝑩) = 𝑷(𝑨 ∩ 𝑩) = 𝑷(𝑩) ∗ 𝑷(𝑨)
This rule can be extended to the case where there are more than two independent events. If A, B,
C, … are independent events, then
𝑷(𝑨 𝒂𝒏𝒅 𝑩 𝒂𝒏𝒅 𝑪 𝒂𝒏𝒅 … ) = 𝑷(𝑨) ∗ 𝑷(𝑩) ∗ 𝑷(𝑪) ∗ …

E.g.1: The event: A: 1st roll of a fair 6-sided die is a 6 and event: B: 2nd roll of the fair 6-sided die is a
6 are independent. P(B: 2nd roll of die is a 6)=1/6 and so is: P(B|A). What happens on the first roll
does not affect the probability of subsequent rolls. The probability of A and B, rolling a 6 on both
rolls of a fair 6-sided die is P(A and B)=1/6*1/6=1/36.

Breast Cancer Example g: Show that the event: + Mammogram is not independent from the event: Yes
Breast Cancer diagnosis in the table of 10000 results.

97 100
𝑃(𝑌𝑒𝑠 𝐵𝑟𝑒𝑎𝑠𝑡 𝐶𝑎𝑛𝑐𝑒𝑟 |+𝑀𝑎𝑚𝑚𝑜𝑔𝑟𝑎𝑚) = 691 = 0.1403763 ≠ 𝑃(𝑌𝑒𝑠 𝐵𝑟𝑒𝑎𝑠𝑡 𝐶𝑎𝑛𝑐𝑒𝑟) = 10000 = 0.01. The
probability of having a Breast Cancer diagnosis is much higher for those patients with a positive
mammogram.
97 691
Similarly, 𝑃(+ 𝑀𝑎𝑚𝑚𝑜𝑔𝑟𝑎𝑚|𝑌𝑒𝑠 𝐵𝑟𝑒𝑎𝑠𝑡 𝐶𝑎𝑛𝑐𝑒𝑟) = 100 = 0.97 ≠ 𝑃(+ 𝑀𝑎𝑚𝑚𝑜𝑔𝑟𝑎𝑚) = 10000 = 0.0691. The
probability of having a positive mammogram is much higher for those patients with a breast cancer
diagnosis.

We can also show that the Multiplication Law of Probability for independent A and B does not work
𝑃(𝑌𝑒𝑠 𝐵𝑟𝑒𝑎𝑠𝑡 𝐶𝑎𝑛𝑐𝑒𝑟 𝐴𝑁𝐷 + 𝑀𝑎𝑚𝑚𝑜𝑔𝑟𝑎𝑚) = 0.0097 ≠
100 691
𝑃(𝑌𝑒𝑠 𝐵𝐶) ∗ 𝑃(+𝑀𝑎𝑚𝑚𝑜𝑔𝑟𝑎𝑚) = ∗ = 0.00691
1000 10000

Video Screen Tension Example i: Compute the probability of a sample of two

customers both receiving a screen with tension > 320 if the distribution of tensions
always looks like that given at the right (“Sampling with replacement”). Compare
this probability to that computed when sampling without replacement. In which
sampling scenario do we have independence?
Statistics University of Wisconsin Madison – Chelsey Green
[email protected]
Video Screen Tension Example j: Confirm your computations for Screen Tension examples a-i using the
simulations in R included in the course notes .Rmd file.

Some R Code that we will be using in our simulations:

Function that creates a vector by repeating an input a specified number of times

rep(x, times=)

For loop : code to structure repeating a process for a specified number of iterations
for (i in 1:manytimes) {
process to be repeated manytimes
[i]<-value
}

Take a sample of the specified size from the elements of x using either with or without replacement.
sample(x, size, replace = FALSE, prob = NULL)

Manufacturing Machinery Example: A Perfect Good Fair Damaged Total

manufacturer of grommets has facilities in four
cities. Each facility has many pieces of identical Gimery 6 15 8 1 30
equipment of varying status. A survey of Murgh 19 5 1 5 30
equipment status is made. If a machine is Olisford 2 1 4 7 14
sampled at random from these 88, what is the Uchoset 2 6 5 1 14
Totaal 29 27 18 14 88
probability…

a. …that it is from Gimery or Murgh? d. …that it is in fair condition give that it is from
Gimery?

b. …that it is in Good quality or from Uchoset? e. …that it is from Gimery given that it is in fair
condition?

c. …that it is a damaged machine from Olisford?

Statistics University of Wisconsin Madison – Chelsey Green
[email protected]

Manufacturing Machinery Example f. Test whether the events “the machine is from Murgh” and “the
machine is perfect” are independent events in this set of 88 pieces of equipment..

Circuit Example: A system is built with three components working in parallel. As long as one component is
working, the system will continue to function. It is thought these type of components have a failure rate of
33.33% in 1 year.
I will use the following assumptions in my calculations:
(1) The 33.33% failure rate means a 0.3333 probability of failure for each component within 1
year.
(2) The success/failure of each component is independent from the others.

Circuit Example a .What is the probability that none of the three components fail within the year?

Circuit Example b. What is the probability that only one of the components fail under the assumptions?

Circuit Example c. If two of them fail within the year, is that strong evidence to suggest that the failure
rate of all components is higher than 33.33%?

Circuit Example d: What is the probability that the system still functions (at least one of the components
works)?