Chapter 4.3, 4.2 and 4.

3 – Fundamental Concepts of Probability

Probability

- A measure of the likelihood that an event occurs

- The value of probability is between 0 and 1
- 0 = impossible event
- 1 = definite event
- P(A) = 0, event A can never happen
- P(A) = 1, event A always happens
- P(A) = 0.5, event A is equally likely to occur or not to occur

Experiment

- A process that leads to one of several possible outcomes

- A planned operation

Sample Space (S)

- All possible outcomes of an experiment

- Must contain all possible outcomes from the experiment. Eg sample space for
flipping a coin 3 times: heads (H), tails (T) and all possible combinations, S =
{HHH, HHT, HTH, HTT ,THH, TTH, THT, TTT}

Event

- A subset of the sample space. Eg one possible outcome

- Eg sample space (S) for rolling a die: S = { 1, 2, 3, 4, 5, 6}
- Events: rolling an even or odd number: P(A) = { 2, 4 , 6}, P(B) = { 1, 3, 5 }

Independent Events

- Two events are independent if one occurring does not affect the chance of the other
occurring
- Two events are independent if one of the following is true:
 - P( A|B ) = P(A)
- P( B|A ) = P(B)
- P( A ∩ B ) = P(A) P(B)

Dependent Events

- Two events are dependent if one occurring affects the probability of the other event
occurring
- Two events are dependent if one of the following is true:
- P( A|B ) ≠ P(A)
- P( B|A ) ≠ P(B)
- P( A ∩ B ) ≠ P(A) P(B)

Exhaustive Events

- When all possible outcomes of an experiment are included in the events

- ie when sample space = events

Mutually Exclusive Events

- Events that do not share any common outcomes for an experiment

- Eg when flipping a coin for heads or tails you can’t get both in one event
- P(A ∩ B) = 0

Conditional Probability (P(A | B))

- The probability of an event occurring given that another event has already happened.

Subjective probability

- Probability based on personal and subjective judgement/feelings

Empirical probability

- Estimated probability of future outcomes based on the frequency of past outcomes

- Eg if you flip a coin and get heads 3 times then P(H) = 100% while P(T) = 0%

Classical probability
 - The probability of something happening assuming all possible outcomes are equally
likely
- Logical analysis rather than on observation or personal judgment
- Eg P(3) when rolling a die = 1/6

Empirical and classical probabilities are often grouped as objective probabilities

Law of large numbers

- If the same experiment or study is repeated independently a large number of times, the
average of all results must be close to the expected value

The Union of two events (A U B)

- All events in A and B

The Intersection of two events (A∩B)

- Events that overlap in A and B

The Complement of an event (AC)

- The probability that the event does not occur

- Probability of an event + its complement = 1

- An event and its complement are mutually exclusive and exhaustive.

- Either the event or its complement will happen.

- BOTH CANNOT HAPPEN

The sum of probabilities for a list of mutually exclusive and exhaustive events = 1

- P(S) = 1

-------------------------------------------------------------------------------------------------------------------

Probability rules

The Complement Rule

- P(AC) = 1 – P(A)

The Addition Rule (Union)

- P(A U B) = P(A) + P(B) – P(A ∩ B)

 - If A and B are mutually exclusive, P(A ∩ B) = 0, therefore P(A U B) = P(A) + P(B)

The Multiplication Rule (Intersection)

- If A and B are dependent events, P( A∩B ) = P( A|B ) P(B) or P( A∩B ) = P( B|A ) P(A)
- If A and B are independent events, P( A∩B ) = P(A) P(B)

Conditional Probability

- The probability of A occurring given B

P( A ∩ B)
- P ( A|B )= P( B)

- The probability of B occurring given A

P( A ∩ B)
- P ( B| A )= P ( A )

- A and B are independent if P( A|B ) = P(A) or P( B|A ) = P(B)

-------------------------------------------------------------------------------------------------------------------

Probability and Contingency Tables

 - Contingency table useful when examining the relationship between two categorical
variables
- It shows the frequencies for two categorical variables, x and y
- Each cell is a mutually exclusive combination of x and y values
- We can estimate empirical probability by finding the relative frequency of occurrence for
each event. ie relative frequency = probability

Be able to calculate the marginal probabilities in a Contingency Table

Be able to calculate joint probabilities in a Contingency Tables

Be able to calculate the probability of the union of two events using a Contingent Table

Be able to calculate the probability of the intersection of two events using a Contingent Table

Be able to calculate the probability of conditional events using a Contingent Table

Identify if two events are independent or dependent using a Contingency Table

-------------------------------------------------------------------------------------------------------------------

Chapter 6.2 – Normal Distribution

Characteristics of a normal distribution

- Bell-shaped (symmetrical) around the mean

- Mirror image
- Mean, median and mode are all equal and occur at the peak of the curve
- Mean always = 0
- Standard Deviation determines the width of the curve
- Standard deviation always = 1
- X has an infinite range (-∞ < X < ∞)
 - Asymptotic – the tails approach the x-axis but never touch

Standardizing a normal random variable

Use Z – score

- z-score = the number of standard deviations a random variable is from its mean
- This transforms the mean and standard deviation of the normal random variable to 0 and
1 respectively

- Z – score = ( x – mean ) / standard deviation

- Also gives Z value to use in Z – tables

Can rearrange formula to find x, mean and standard deviation using the closest Z – score if
needed

- Eg pass mark on an exam to be in top 10%

- Find probability closest to 0.9 on Z – table and use the corresponding Z – score in
formula

Z – table

- Aka standard normal table

- Gives the cumulative probabilities to the left of the Z – score
- Gives area under the curve
- If Z – score is positive look at the positive side of Z – table
- If Z – score is negative look at the negative side of Z – table.

To find the probability of x between two events, find Z–score of both events, and subtract them
-------------------------------------------------------------------------------------------------------------------

Chapter 7.1, 7.2, 7,3 – Sampling Distribution

Non – probability sampling

- The selection of a sample from a population based on subjective or non-random methods

Probability sampling

- The selection of a sample from a population based on random selection or chance

- Low sample bias = more accurate data and statistical inferences
- Necessary when making statistical inferences of a population
- Statistical inference provides estimates of population characteristics based on
sample information
- Specifically population mean and standard deviation

Sample bias

- The tendency of a sample statistic to systematically overestimate or underestimate a

population parameter
- These samples do not represent the population

Simple Random sampling (Probability sampling)

- When all observations have the equal probability of being chosen

- Reduces bias

Stratified Random sampling (Probability sampling)

- Divide population into groups, called strata, based on some characteristic

- Then select a simple random sample from each group or strata

- Divided by location, age, income, etc

Cluster sampling (Probability sampling)

- Geographic or spatial characteristic divides target population into random clusters prior
to randomly sampling

- Eg geographic divisions: neighborhood, schools, hospitals

- Low cost

-------------------------------------------------------------------------------------------------------------------

Sample mean

- An unbiased estimator of population mean

- The random variable of interest, x̅
- Each new sample from the population = new sample mean
- Aka the estimator or point estimator
- Estimator – a stat used to estimate a population parameter
- Point estimator – a function of the random sample used to make inferences about
the value of an unknown population parameter

Standard error (se)

- Estimates how well sample data represents the whole population

- se( x̅ ) = 𝜎 / √𝑛
- se = standard error
- x̅ = sample mean
- σ = population standard deviation
- n = total sample size

- High standard error – sample does not closely represent population

- Increasing sample size decrease standard error

- Standard error is an inferential statistic

Central Limit Theorem

 - The mean of a large number of independent observations from the same underlying
distribution has an approximate normal distribution
- Sample size (n) ≥ 30 assume a normal distribution
- Sample distribution is not normal if the population is not normal or the sample size < 30

Sampling distribution

- Distribution of sample mean from simple random sampling of a population

Expected value (E)

- E( x̅ ) = E( x ) or µ

Standardizing a normal distribution

- Z = ( x̅ - µ ) / ( se ) ≡ Z = ( x̅ - µ ) / ( 𝜎 / √𝑛 )

-------------------------------------------------------------------------------------------------------------------

Sample proportion ( p̅ ) ( P̅ is population proportion)

- p̅ = x / n
- Sampling Distribution of p̅ is the probability distribution of all possible values of p̅
- E( p̅ ) = the population proportion ( P̅ )
- se( p̅ ) = √ p̅ (1 – p̅ ) / n
- se( p̅ ) ≡ the standard deviation of p̅
- CLT – distribution of p̅ is ≈ normal if and only if both ( n p̅ ) ≥ 5 AND n ( 1 – p̅ ) ≥ 5

Standardizing proportion

- Z = ( p – p̅ ) / √ se ( p̅ ) ≡ Z = ( p – p̅ ) / √ p̅ ( 1 – p̅ ) / n
- p ≡ proportion of interest