(EE3006/3006A)

Unit 9. Probability Distributions

1. Basics of Probability & Statistics
• Sample space
• Events
• Probability of an event
• Random variables and distributions
• Mathematic expectation
2. Discrete Probability Distributions
• Uniform distributions
• Binomial distributions
• Poisson distributions
3. Continuous Probability Distributions
• Uniform distributions
• Normal distributions
1

(EE3006/3006A)

1. Basics of Probability & Statistics

 Why study probability and statistics?
 We had studied many methods to deal with deterministic problems and
systems. A question remained to answer is: how to handle and analyze
the problems with randomness and uncertainty?
 How to measure the randomness and uncertainty: probability
(e.g. risk assessment of the trade in financial markets;
reliability of automobiles and consumer electronics)

 How to interpret and analyze data (of random events) collected by

experiments: statistics.

1
 (EE3006/3006A)

 Probability or statistics

(Probability is deductive: general -> particular)

(Statistics is inductive: particular -> general)

3

(EE3006/3006A)

1.1 Sample Space

• The set of all possible outcomes of a random experiment is called the
sample space, which is usually represented by the symbol S;
• Each outcome in a sample space is called an element, or a member of
the sample space, or simply a sample point;

=> How to count all possible arrangements or orders of a group of

sample points or objects?


n
n!  k 1
k

2
 (EE3006/3006A)

Theorem 1: If an operation can be performed in n1 ways and if for each

of these ways a second operation can be performed in n2 ways, the
two operations can then be performed together in n1n2 ways.

Theorem 2: The number of permutations of n objects is n!.

Example: Consider the three letters a, b, c. The possible permutations are
{abc, acb, bca, cab, cba}. The number of permutations can be calculated as
3! =3×2×1=6

Theorem 3: The number of permutations of n objects taken r at a time is:

nPr=n!/(n-r)!

Example: Three awards will be given for a class of 25 students in the department of
electrical engineering each year. If each student can receive at most one award,
how many possible selection are there?
25!
P 
25 3  25  24  23  13,800.
(25  3)!
5

(EE3006/3006A)

Theorem 4: The number of distinct permutations of n things which n1

are of one kind, n2 of the second kind, …nk of the k-th kind is:
n!/(n1! n2! ··· nk!)
Example: In a college football training session, the defensive coordinator needs to
have 10 players standing in a row. Among these 10 players, there are 1 freshman,
2 sophomore, 4 juniors and 3 seniors, respectively. How many different ways can
they be arranged in a row if only their class level will be distinguished?
10!
 12,600.
1!2!4!3!

Theorem 5: The number of combination of n objects taken r at a time is:

 n n!
 
 r  ( n  r )! r !
Example: The number of ways of selecting 3 cartridges from 10 can be calculated as
10  10!
3    120
  (10  3)!3!
6

3
 (EE3006/3006A)

1.2 Events
• An event is a set of outcomes, i.e., a subset of a sample space.
• An event may be a subset that includes the entire sample space S, or a
subset of S called the null set and denoted by the symbol ϕ, which
contains no element at all.
• The complement of an event A with respect to S is the subset of all
elements of S that are not in A. We denote the complement of A by the
symbol A.

Example: Consider the sample space S={book, catalyst, cigarette, precipitate,

engineer, rivet}. Let A={catalyst, rivet, book, cigarette}. Then
A ={precipitate, engineer}

(EE3006/3006A)

• The intersection of two events A and B, denoted by the symbol A ∩ B,

is the event containing all elements that are common to A and B.

Example: Let M={a, e, I, o, u} and N={r, s, t}; then it follows that

M ∩ N= ϕ.
That is, M and N are mutually exclusive and have no element in common.

 The union of the two events A and B, denoted by the symbol A∪B, is
the event containing all the elements that belong to A or B or both.

Example: Let A = {a, b, c} and B = {b, c, d, e}; then

A ∪ B = {a, b, c, d, e}.

4
 (EE3006/3006A)

• The relationship between events and the corresponding sample

space can be illustrated graphically by Venn diagrams.

A ∩ B = regions 1 and 2;
B ∩ C = regions 1 and 3;
A ∪ C = regions 1, 2, 3, 4,
5, and 7;
B ∩ A = regions 7 and 4;
A ∩ B ∩ C = region 1;
(A ∪ B) ∩ C= regions 4, 1,
and 3;
Events represented by various regions

(EE3006/3006A)

1.3 Probability of an Event

The probability of an event A is the sum of the chances (or weights) of
all sample points in A with respect to the sample space S. If it is denoted
by P(A), one has
0 ≤ P(A) ≤ 1, P(ϕ) = 0, and P(S) = 1.
If A1, A2, A3, . . . is a sequence of mutually exclusive events, then
P(A1 ∪ A2 ∪ A3 ∪ ·· ·) = P(A1) + P(A2) + P(A3) + · · · .

Theorem 1: If an experiment can result in any one of M different equally

likely outcomes, and if exactly n of these outcomes correspond to event
A, then the probability of event A is
P(A) =n/M .

10

5
 (EE3006/3006A)

 Additive rules
Theorem 1: If A and B are any two events, then

Corollary 1: If A and B are mutually exclusive, then

Corollary 2: If A1, A2, …, An are partitions of a sample space S, then

* Example :
John was interviewed at two companies he likes, he assesses that his probability
of getting an offer from company A is 0.8, and the probability of getting an offer
from company B is 0.6. He believes that the probability he will get offers from
both companies is 0.5. What is the probability that he will get at least one offer?

11

(EE3006/3006A)

 Conditional probability
The probability of an event B given that the event A has occurred is called
a conditional probability and is denoted by P(B|A) (reads as “probability
of B given A” or “probability of B under the condition A”)

 Multiplicative rules
Theorem 1: If the events A and B can both occur in an experiment, then

Corollary 1: Two events A and B are independent if and only if

12

6
 (EE3006/3006A)

1.4 Random Variables and Distributions

=> A random variable is a function that associates a real number with
each element in the sample space.

• One can use a capital letter, say X, to denote a random variable and
its corresponding small letter, x in this case, for one of its values.
• Random variables (RV) can be classified into:

Discrete RV : its set of possible outcomes is countable.

Continuous RV : its possible outcomes take on a

continuous scale.

13

(EE3006/3006A)

 Probability distributions
=> The mathematical function describing the possible values of a random
variable and their associated probabilities is known as a
probability distribution.

i. If examples are found in experiments

whose sample space is encoded by a
discrete random variable, the
distribution is a probability mass
function.

ii. If experiments are with sample spaces

encoded by a continuous random
variable, the distribution is a
probability density function.

14

7
 (EE3006/3006A)

• Cumulative distribution
 Cumulative distribution describes the probability that a real-valued
random variable X with a given probability distribution will be found
at a value less than or equal to x.
 Intuitively, it is the "area so far" function of the probability distribution.

The cumulative distribution F (x)

where X is the random variable with the probability (mass or density)

distribution f (x).

15

(EE3006/3006A)

1.5 Mathematical Expectation

 Mathematical expectation is the summation (or integration) of the product of the
value of a random variable and its probability (or its probability density) over all
values of the random variable.
 It is a generalization of weighted average and can be used to calculate the
characteristics, such as mean and variance, of a random variable.
 Mean of a random variable
The mean of a random variable is the weighted average of all possible values
that this random variable can take on.

16

8
 (EE3006/3006A)

Example 1:
A batch containing 7 components is sampled by a quality inspector;
the batch contains 4 good components and 3 defective components.
A sample of 3 is taken by the inspector. Find the expected value of
the number of good components in this sample.

17

(EE3006/3006A)

18

9
 (EE3006/3006A)

 Variance ( 2) and Standard deviation ( )

Let X be a random variable with probability distribution f (x) and mean μ.

(the expectation of the squared deviation of X from its mean)

The positive square root of the variance, i.e., σ, is called the standard
deviation of X.

19

(EE3006/3006A)

* Example 2:
Let the random variable X represent the number of automobiles that are
used for official business purposes on any given workday. The
probability distribution for company A is

and that for company B is

Calculate the variances of the probability distribution for company A

and company B, respectively.

20

10
 (EE3006/3006A)

Solution:

=E(X)=10.3+20.4+30.3=2.0

σA2

B=00.2+10.1+20.3+30.3+40.1=2.0,

σB2

21

(EE3006/3006A)

For the continuous case , replace summations by integrations.

22

11
 (EE3006/3006A)

Example 3:

23

(EE3006/3006A)

2. Discrete Probability Distributions

 Discrete probability distribution is a function that gives the
probability of a random variable that can have discrete values as
outcomes.

Some typical discrete probability distributions:

• Uniform distributions
• Binomial distributions
• Poisson distributions
24

12
 (EE3006/3006A)

2.1. Uniform Distributions

 The discrete uniform distribution is a probability distribution
whereby a finite number of values are equally likely to be observed;
every one of k values has equal probability 1/k.

25

(EE3006/3006A)

Example 4:
Let X represent the number that may appear on top when we toss a
dice, find the mean and variance of X.

1 2 3 4 5 6

26

13
 (EE3006/3006A)

2.2. Binomial Distributions

• Bernoulli trial: an experiment has only two possible outcomes that are
typically labeled as success or failure and assigned with the values of 1 and 0.
• Typically, the number of successes in n Bernoulli trials is called a binomial
random variable.
• Binomial distribution is the probability distribution of a binomial random
variable. If the experiment consists of n repeated trials and its probability of
success is p, its probability distribution of getting exactly x successes is

 Where does the name “Binomial” come from?

27

(EE3006/3006A)

28

14
 (EE3006/3006A)

* Example 5:
It is conjectured that an impurity exists in 30% of all drinking wells in a certain
rural community. In order to gain some insight into the true extent of the
problem, it is determined that some testing is necessary. It is too expensive to
test all of the wells in the area, so 10 are randomly selected for testing.
(a) Using the binomial distribution, what is the probability that exactly 3 wells
have the impurity, assuming that the conjecture is correct?
(b) What is the probability that more than 3 wells are impure?

29

(EE3006/3006A)

2.3 Poisson Distributions

 Poisson process
• The probability that a single outcome will occur during a very short time
interval (or in a small region) is proportional to the length of the time
interval (or the size of the region) and does not depend on the number of
outcomes occurring outside this time interval (or region). In this way, it
is sometimes called a purely or completely random process.
• The number of outcomes occurring in one time interval (or specified
region) is independent of the number that occurs in any other disjoint
time interval (or region of space). In this way we say that the Poisson
process has no memory.

30

15
 (EE3006/3006A)

 Poisson distributions

Theorem: Both the mean and the variance of the Poisson

distribution p(x; λt) are λt (i.e., 𝜇 = 𝜎 =λt ).
The proof is in Appendix A26 of Ref.1.

31

(EE3006/3006A)

=> The form of the Poisson distribution becomes more and more symmetric,
even bell-shaped, as the mean grows large.

32

16
 (EE3006/3006A)

* Example 6:
During a laboratory experiment, the average number of radioactive
particles passing through a counter in 1 millisecond is 4. What is the
probability that 6 particles enter the counter in 2 milliseconds?

Solution:
Using the Poisson distribution with x=6 and λt=4×2=8,
we have

p(6; 8)=e-886/6! = 0.122

So, the probability is 12.2%.

33

(EE3006/3006A)

3. Continuous Probability Distribution

• A continuous random variable has a probability of zero of assuming
exactly any of its values, and its probability is calculated for various
intervals;
• The probability distribution of a continuous random variable is
described by probability density function (PDF).

f (x)

ab

34

17
 (EE3006/3006A)

3.1. Continuous Uniform Distribution

The continuous uniform distribution is a family of probability distributions
such that all intervals of the same length on the distribution's support are
equally probable for each member of the family.

35

(EE3006/3006A)

Example 7:
Suppose that a large conference room can be reserved for no more than 4
hours. Both long and short conferences occur quite often. In fact, it can be
assumed that the length X of a conference has a uniform distribution on the
interval [0, 4].
(a) What is the probability density function?
(b) What is the probability that any given conference lasts at least 3 hours?

36

18
 (EE3006/3006A)

3.2. Normal Distribution

=> Normal distributions are very important in probability and statistic analysis
and are often used in the natural and social sciences for real-valued random
variables whose distributions are not known.

.
σ
f (x)
x
µ

 Once μ and σ are specified, the normal curve is completely determined.

37

(EE3006/3006A)

Properties:
i. The curve is symmetric about a
vertical axis through the mean μ,
and its maximum occurs at x = μ.
ii. The curve has its points of
Normal curves with μ1 = μ2 and σ1 < σ2.
inflection at x = μ ± σ; it is
concave downward if μ −σ <x <
μ+ σ and is concave upward
otherwise.
iii. The total area under the curve
and above the horizontal axis is
2 2 equal to 1.
Normal curves with μ1 < μ2 and σ1 < σ2.

38

19
 (EE3006/3006A)

 Dark blue area is less than one standard deviation away from the mean.
For the normal distribution, this accounts for about 68%, while two
standard deviations from the mean (medium and dark blue) account for
about 95%, and three standard deviations (light, medium, and dark
blue) account for about 99.7%.

39

(EE3006/3006A)

40

20
 (EE3006/3006A)

• Areas under the normal curve can be read

from Stand Normal Distribution Table
z

41

(EE3006/3006A)

• Areas under the normal curve can be read

from Stand Normal Distribution Table
z

42

21
 (EE3006/3006A)

Example 8:
Gauges are used to reject all components for which a certain
dimension is not within the specification 1.50 ± d. It is known that
this measurement is normally distributed with standard deviation 0.2.
Determine the value d such that the specifications “cover” 95% of the
measurements.

z
Therefore
x1=z1× + µ
=-1.96 ×0.2+1.5=1.108.
from which we obtain
? ?
d=1.5-1.108 = 0.392

43

(EE3006/3006A)

Classwork 9:
Q1. Prove that the mean and variance of the uniform distribution between interval
[a, b] are

Q2. In an industrial process, the diameter of a ball bearing is an important

specification of component parts. The buyer sets specifications on the diameter
to be 3.0+0.01 cm. The implication is that no part falling outside these
specifications will be accepted. It is known that in the process the diameter of a
ball bearing has a normal distribution with the mean of 3.0 and the standard
deviation of 0.006. On average, how many manufactured ball bearings (in
terms of percentage) will be scrapped?

44

22
 (EE3006/3006A)

Reference:
R. E. Walpole, R. H. Myers, S. L. Myers, K. Ye
Probability & Statistics for Engineers & Scientists
Prentics Hall, Inc., 2002

3. Random Variables and Probability Distributions (pp.63-69)

4. Mathematical Expectation (pp. 88 – 110)
5. Some Discrete Probability Distributions (pp. 115 – 135)
6. Some Continuous Probability Distributions (pp.142-158)

45

23