Probability and Statistic

Probability and
Statistic
ISTA, University of 20 August 1955 -Skikda

DR. MECHHOUD EL-ARKAM

April 2020

2019/2020
 Table des
matières

Objectifs 5

I - Permitations and Combinations 7

A. Operations..................................................................................................7

B. Arrangement & Combination........................................................................12

C. Probability.................................................................................................14

D. Basic rules of combining probabilities...........................................................16

E. Series-Parallel systems...............................................................................19

II - Statistic 23

A. Central Location.........................................................................................23

B. Variability or Spread of the Data..................................................................25

C. Quartiles, Deciles, Percentiles, and Quantiles.................................................27

III - Probability Distributions of Discrete Variables 29

A. Probability Functions and Distribution Functions.............................................29

B. Expectation and Variance............................................................................30

C. Binomial Distribution..................................................................................32

D. Poisson Distribution....................................................................................33

IV - Probability Distributions of Continuous Variables 37

A. Probability from the Probability Density Function............................................37

B. Expected Value and Variance.......................................................................38

C. Exponential Distribution..............................................................................40

D. The Normal Distribution..............................................................................40

E. LOGNORMAL DISTRIBUTION........................................................................44

Bibliographie 47

3
Objectifs

The ojective of this course is to get the basic knowledge that will
allow students to understand random phenomena and statistical
methods related to the field of risk management.

5
Permitations and
I-

I
Combinations

Operations 7
Arrangement & Combination 12
Probability 14
Basic rules of combining probabilities 16
Series-Parallel systems 19

A. Operations

The result of an operation is called an “outcome”.

For example, if we throw a die one possible outcome is 5.
If we throw a die there are 6 possible outcomes: 1, 2, 3, 4, 5 or 6.

Fundamental Principle of Counting 1

Suppose one operation has m possible outcomes and that a second operation has n
outcome. The number of possible outcomes when performing the first operation
followed by the second operation is : m×n
Performing one operation and another operation means we multiply the number of
possible outcomes.

Remarque : Note
We assume that the outcome of one operation does not affect the number of
possible outcomes of the other operation.
The fundamental principal 1 of counting can be extended to three or more
operations.

7
 Permitations and Combinations

Exemple : Example
• If a die thrown and a coin is tossed, how many different outcomes are possible?
• Write out all the possible outcomes.
1. There are 6 possible outcomes for a die: 1, 2, 3, 4, 5 or 6.
2. There are 2 possible outcomes for a coin: H or T.
Hence, the number of different outcomes=6 ×2 = 12.
(1,H) ; (2,H) ;(3,H) ;(4,H) ;(5,H) ;(6,H) ; (1,T) ;(2,T) ;(3,T) ;(4,T) ;(5,T) ;(6,T)

Fundamental Principle of Counting 2

Suppose one operation has m possible outcomes and that a second operation has n
outcomes. Then the number of possible outcomes of the first operation or the
second operation is given by: m+n
Performing one operation or another operation means we add the number of
possible outcomes.

Remarque : Note
We assume it is not possible for both operations to occur. In other words, there is
no overlap of the two operations.
The fundamental principal 2 can be extended to three or more operations, as long
as none of the operations overlap.

Exemple : Example
A bag contains nine discs, numbered from 1 to 9. A disc is drawn from the bag. If
the number is even, then a coin is tossed. If the number is odd, then a die is
thrown.
How many outcomes are possible?
Break the experiment into two different experiments and work out the number of
outcomes separately. Then add these results.

8
 Permitations and Combinations

Repetition of an Event
If one event with n outcomes occurs r times with repetition allowed, then the
number of ordered arrangements is
r
n

Exemple : Example
What is the number of arrangements if a die is rolled :
a. 2 times ?
b. 3 times ?
c. r times ?
a. The number of arrangement if a die is rolled 2 times is : 6×6=6 2=36
b. The number of arrangement if a die is rolled 3 times is : 6×6×6=63=216
c. The number of arrangement if a die is rolled r times is : 6×6×.....=6 r

Factorial Representation
If we have n different objects to arrange, then : The total number of
arrangements=n !.

n!=n×( n−1)×( n−2).....3×2×1

Exemple : Example
• Consider the three letters P, Q and R. If these letters are written down in a row,
there are six different possible arrangements:
.
There is a choice of 3 letters for the first place, then there is a choice of 2 letters for
the second place and there is only 1 choice for the third place.
Thus the three operations can be performed in : 3×2×1=3!=6
• In how many ways can 6 people be arranged in a row?
6×5×4×3×2×1=6!=720

9
 Permitations and Combinations

B. Arrangement & Combination

Arrangement or Permutation
Distinctly ordered sets are called arrangements or permutations.
The number of permutations of n objects taken r at a time is given by:
n
n!
P=
r ( n−r)!

Where: n : number of objects

r :number of positions

Exemple : Example
How many arrangements can be made of the letters P, Q, R, S, and T, taking two
letters at a time, if no letter can be repeated?
5
5! 5!
P= = =20
2 (5−2)! 3!

Arrangements with Repetitions

If we have n elements of which x are alike of one kind, y are alike of another kind,
z are alike of another kind,............ then the number of ordered selections or
permutations is given by:

n!
x!× y!×z!

Exemple : Example
A machinist produces 22 items during a shift. Three of the 22 items are defective
and the rest are not defective. In how many different orders can the 22 items be
arranged if all the defective items are considered identical and all the non-defective
items are identical of a different class?
The number of ways of arranging 3 defective items and 19 non-defective items is:
22!
=1540
19!×3!

Combinations (Selections):
A combination is a selection of a number of objects in any order.
In making a selection of a number of objects from a given set, only the contents of
the group selected are important, not the order in which the items are selected.
For example, AB and BA represent the same selection. However, AB and BA
represent different arrangements.
The number of different combinations (i.e. unordered sets) of r objects from n
distinct objects is represented by:

number of permutations
number of combination=
arrangements of r objects

10
 Permitations and Combinations

And is denoted by:

n
n P
n!
C= = r
r ( n−r )!×r! r!

Exemple : Example 1
How many ways can a basketball team of 5 players be chosen from 8 players?
8
8!
C =( 8)= =56
5 5 (8−5!)×5!

Exemple : Example 2
There are 5 women and 4 men in a club. A team of four has to be chosen. How
many different teams can be chosen if there must be either exactly one woman or
exactly two women on the team?
We have and . A team must consist of 4 people.
and or and
5! 4! 5! 4!
( 5)×( 4 )+( 5 )×( 4)= × + × =5×4+10×6=80
1 3 2 2 (5−1!)×1! ( 4−3)!×3! 2!×(5−2)! ( 4−2)!×2!

C. Probability

Probability involves the study of the laws of chance. It is a measure of the chance,
or likelihood, of something happening.

Définition : Definition
The measure of the probability of an event, E, is given by:

number of successful outcomes

Pr [ E ]=
number of possible outcomes

The probability of an event is a number between 0 and 1, including 0 and 1.

0⩽Pr [ E ]⩽1

The value of Pr(E) can be given as a fraction, decimal or percentage.

Remarque : Note
means that an event is impossible.
means that an event is certain.

11
 Permitations and Combinations

Probability of an Event not happening

If E is any event, then ‘not E' is the event that E does not occur. We have the
following relationship between the probabilities of E and not E:

Pr [ E ]+ Pr [not E ]=1

Pr [not E]=1−Pr [E ]

Exemple : Example
A bag contains 8 red, 3 blue and 13 yellow discs. A disc is selected at random from
the bag. What is the probability that the disc selected is:
1) Red,
2) blue,
3) yellow,
4) not yellow?
There are 8+3+13=24 discs in the bag.

number of reddisc 8 1
Pr ( reddisc)= = =
total number of discs 24 3

number of bleu discs 3 1

Pr (bleu disc)= = =
total number of discs 24 8

number of yellowdisc 13
Pr ( yellowdisc )= =
total number of discs 24

13 11
Pr ( yellowdiscs )+ Pr (not yellowdiscs)=1 ⇒ Pr ( not yellowdiscs )=1−Pr ( yellowdiscs)=1− =
24 24

Conditional probability
With conditional probability we are given some prior knowledge, or some extra
condition, about the outcome. This usually reduces the size of the sample space.

Exemple : Example
In a class, there are 21 boys and 15 girls. Three boys wear glasses and five girls
wear glasses. A pupil is picked at random from the class.
1) What is the probability that the pupil is a boy
2) What is the probability that the pupil wears glasses?
3) What is the probability that the pupil is a boy who wears glasses?
A girl is picked at random from the class.
4) What is the probability that she wears glasses?
A pupil wearing glasses is picked at random from the class
5) What is the probability that is a boy?

12
 Permitations and Combinations

It is good practice to represent the information in a table (including the totals for
each column and row).

Boy Girl Total

Does not wear 18 10 28

glasses

Wears glasses 3 5 8

Total 21 15 36

There are 21+15=36 pupils in the class.

number of boys 21 7
Pr ( Boy)= = =
number of pupils inthe class 36 12

number of pupil who wear glasses 8 2

Pr ( pupil wears glasses)= = =
number of pupil inclass 36 9

number of boys who wear glasse 3 1

Pr (Boy who wear glasses)= = =
number of pupil inthe class 36 12

The next two questions require the use of conditional probability, where the size of
the sample space has been reduced.
4) We are certain that the pupil picked is a girl. There are 15 girls in the class. 5 of
these wear glasses

number of girls inthe class who wear glases 5 1

Pr ⁡( when a girl is picked she wears glasses)= = =
number of girls inthe class 15 3

5) We are certain that the pupil picked wears glasses. There are 8 pupils who wear
glasses. 3 of them are boys.

N of B who wear glasses 3

Pr ⁡( when a pupil who wears glasses is picked , the pupil is boy)= =
N of P who wear glasses 8

Attention : NB :
The value of probability of event A occurring given that condition B has occurred
can be deducted from an analysis of the Venn diagram. From the Venn diagram it
can be concluded that the following expression holds:

Pr ( A∩B) Pr (A∩ B)
Pr ( A∣B)= ; Pr (B∣A)=
Pr ( B) Pr ( A)

13
 Permitations and Combinations

D. Basic rules of combining probabilities

Addition rule
This can be divided into two parts, depending upon whether there is overlap
between the events being combined.
• If the events are mutually exclusive, there is no overlap: if one event occurs,
other events cannot occur. In that case the probability of occurrence of one or
another of more than one event is the sum of the probabilities of the separate
events.
• If the events are not mutually exclusive, there can be overlap between them. This
can be visualized using a Venn diagram. The probability of overlap must be
subtracted from the sum of probabilities of the separate events (i.e., we must not
count the same area on the Venn Diagram twice).

Venn diagram

Pr ( A∪B)=Pr ( A)+ Pr ( B)−Pr (A∩B)

Exemple : Example 1
A sample of four electronic components is taken from the output of a production
line. The probabilities of the various outcomes are calculated to be:
Pr [0 defectives] = 0.6561, Pr [1 defective] = 0.2916, Pr [2 defectives] =
0.0486, Pr [3 defectives] =0.0036, Pr [4 defectives] = 0.0001. What is the
probability of at least one defective?
It would be perfectly correct to calculate as follows:
Pr [at least one defective] = Pr [1 defective] + Pr [2 defectives] +Pr [3
defectives] + Pr [4 defectives]= 0.2916 + 0.0486 + 0.0036 + 0.0001 =
0.3439.
but it is easier to calculate instead:
Pr (at least 1 defective)=1−Pr (0 defective)=1−0,6561=0,344

Exemple : Example 2

Location in sputtering tool

Contaminati Center Edge Total

on

Low 514 68 582

High 112 246 358

Total 626 314 940

14
 Permitations and Combinations

Tableau 1 Waffers in Semiconductor manufacturing classified by contamination and

location

The above table lists the history of 940 wafers in a semiconductor manufacturing
process. Suppose one wafer is selected at random. Let H denote the event that the
wafer contains high levels of contamination. Then, Pr(H)=358/940.
Let C denote the event that the wafer is in the center of a sputtering tool. Then,
Pr(C)=626/940.
Also, Pr(H∩C) is the probability that the wafer is from the center of the sputtering
tool and contains high levels of contamination. Therefore,Pr(H∩C)=112/940
The event (H ∪C) is the event that a wafer is from the center of the sputtering tool
or contains high levels of contamination (or both). From the table,
Pr(H ∪ C)=872/940. An alternative calculation of Pr(H∪C) can be obtained as
follows. The 112 wafers that comprise the event (H∩C) are included once in the
calculation of Pr(H) and again in the calculation of P(C). Therefore, Pr(H ∪C) can
be found to be:

872
Pr ( H ∪C )=Pr ( H )+ Pr (C )−Pr ( H ∩C)=
940

Multiplication rule
The simultaneous occurrence of two events A and B is the occurrence of both
events A and B. In terms of a Venn diagram, this occurrence is represented by the
operation of intersection as depicted in figure 3.6. Mathematically it is expressed as
follows:

( A∩B) ,( A∧B) ,( AB)

The general multiplication rule for two events:

Pr ( A∩B)=Pr ( A)× Pr ( B∣A)=Pr (B)× Pr (A∣B)

In this rule there are two cases to consider, the first is when the two events are
independent and the second is when the two events are dependent.
1)Events are independent:If A and B are independent, then the following applies:
Pr(A│B)=Pr(A)and Pr(B│A)=Pr(B)
the multiplication rule for two independent events: Pr(A∩B)=Pr(A)×Pr(B)
If there are m independent events, the above equation can extended to give
m
Pr ( A1 ∩A2∩.....∩Am )=∏ Pr ( Ai )
i =1

2) Events are dependent: If the two events are dependent the generally valid
general equation of the multiplication , can still be used but cannot be simplified as
in the case of independent events.

Exemple : Example
Continuing with the semiconductor manufacturing example, assume the following
probabilities for product failure subject to levels of contamination in manufacturing:

15
 Permitations and Combinations

Probability of Level of
failure contamination

0.10 High

0.01 Medium

0.001 Low

In a particular production run, 20% of the chips are subjected to high levels of
contamination, 30% to medium levels of contamination, and 50% to low levels of
contamination. What is the probability that a product using one of these chips fails?
Let H denote the event that a chip is exposed to high levels of contamination M
denote the event that a chip is exposed to medium levels of contamination L denote
the event that a chip is exposed to low levels of contamination Then,
Pr(F)=Pr(F│H) Pr(H)+Pr(F│M) Pr(M)+Pr(F│L) Pr(L)
Pr(F)=0.10(0.20)+0.01(0.30)+0.001(0.50)=0.0235

E. Series-Parallel systems

Series structure of components

The arrangement depicted in following figure represents a system whose subsystem
or components form a series network. If anyone of the subsystem or component
fails, the series system experiences an overall system failure.

Series structure of components

Ā = Component A is down
A = Component A is up.

16
 Permitations and Combinations

If the series system component failures are independent, then the probability that
the system is up can be calculated with the following formula:
m
Pr ⁡( System is up)=Pr ⁡(A1 up ∩ A2 up ∩… ∩ Am up)=Pr ⁡( A1). Pr ⁡(A2 )… Pr ⁡( Am )=∏ Pr (Ai )
i=1

The probability that the system will not perform the required function can be
calculated with the formula:
m
Pr (systemis down)=1−∏ [1− Pr (Ai )]
i=1

Parallel configuration of components

The parallel configuration of components is shown in following figure. This system
will fail if and only if all the units in the system malfunction (one out of m). The
model is based on the assumption that all the components are in the same mode of
operation. In addition it is assumed that the component failures are independent.
The parallel system experiences an overall system failure if all the subsystems or
components have been failed. In terms of probabilities, this can be expressed as
follows:
m
Pr (sys is down)=∏ Pr ( Āi)
i=1

m
Pr (sys is up)=1−∏ [1−Pr ( Ai)]
i=1

Parallel structure

Exemple : Example
The following circuit operates only if there is a path of functional devices from left
to right. The probability that each device functions is shown on the graph. Assume
that devices fail independently. What is the probability that the circuit operates?

17
 Permitations and Combinations

Pr ( A∨B∨C )=1−(1−09)3=1−0,13=0,999

Pr ( D∨E)=1−(1−0,95) 2=1−0,052 =0,9975

Pr (global )=Pr ( A∨B∨C)×Pr (D∨E)× Pr (F )=0,987

18
 Statistic
II -

II

Central Location 23
Variability or Spread of the Data 25
Quartiles, Deciles, Percentiles, and Quantiles 27

Original goal of statistics was to collect data about population based on population
samples. By population we mean a group of all existing components available for
observation during statistical research.

A. Central Location

Arithmetic Mean
fi fi
n n
Of these “averages,” the most common and familiar is the arithmetic
∑ fi ∑ fi
i =1 i =1
mean, defined by:
N
1
x̄=μ=
N
∑ xi
i=1

If some results occur more than once.

N

∑ xi f i
x̄=μ= i =1
∑ fi
fi
n
Where: is the frequency and is the relative frequency of
∑ fi
i =1

Exemple : Example
To illustrate, suppose we toss two coins 15 times. The possible number of heads on
each toss is 0, 1, or 2. Suppose we find no heads 3 times, one head 7 times, and

19
 Statistic

two heads 5 times. Then the mean number of heads per trial using the second
equation is

0×3+1×7+ 2×5 17
x̄= = =1,13
3+ 7+ 5 15

Median
Another representative quantity, quite different from a mean, is the median. If all
the items with which we are concerned are sorted in order of increasing magnitude
(size), from the smallest to the largest, then the median is the middle item.

Exemple : Example
• Consider the five items: 12, 13, 21, 27, 31. Then 21 is the median.
21+27
• Consider the six items: 12, 13, 21, 27, 31, 33. The median is =24 .
2
The median is the value exceeded by 50% of the observations

Mode
Is defined as a variant that occurs most frequently.

Exemple : Example
An observational study has been undertaken on the use of an intersection. The
collected data are in the table below. The data is made up of colors of cars that
pass through the intersection. Analyze the data and determine the mode.

red blue red green

blue red red white

green green blue red

From the table it is obvious that the collected colors are qualitative (lexical)
variables, and because there is no order or comparison between them, we can say
they are nominal variables.
For better description we create a frequency table and we determine the mode.

Frequency table

Colors of passing Absoulate frequency Relative frequency

cars
fi fi
n

∑ fi
i =1

red 5 5/12=0.42

blue 3 3/12=0.25

white 1 1/12=0.08

green 3 3/12=0.25

Total 12 1.00

20
 Statistic

Mode = red (i.e. in our sample most cars were red)

B. Variability or Spread of the Data

Mean Deviation from the Mean

The mean deviation from the mean, defined as:
N
( xi − x̄ )
∑ N
i=1

Mean Absolute Deviation from the Mean

However, the mean absolute deviation from the mean, defined as:
N
|( x i− x̄ )|
∑ N
i=1

Exemple : Example
For Groups A, B, and C the mean absolute deviation is as follows, is used frequently
by engineers to show the variability of their data, although it is usually not the best
choice. Its advantage is that it is simpler to calculate than the main alternative, the
standard deviation, which will be discussed below.

Group A : 2, 3, 4, 8

Group B : 1, 2, 4, 10

Group C : 0, 1, 5, 11

(2,25+1,25+0,25+3,75) 7,5
Group A : = =1,875
4 4
(3,25+2,25+ 0,25+5,75) 11,5
Group B : = =2,875
4 4
(4,25+3,25+0,25+6,75) 15
Group C : = =3,75
4 4

Attention : NB :
Its disadvantage is that it is not simply related to the parameters of theoretical
distributions. For that reason its routine use is not recommended.

21
 Statistic

Variance
The variance is one of the most important descriptions of variability for engineers.
It is defined as :
N

∑ ( x i −μ )2
σ 2= i =1
N

Notice that variance is defined in terms of the population mean, μ

Standard Deviation
The standard deviation is extremely important. It is defined as the square root of
the variance:

√
N

∑ ( x i−μ)2
i=1
σ=
N

Estimation of Variance and Standard Deviation from a Sample

The estimate of variance obtained using the sample mean in place of the population
mean can be made unbiased by multiplying by the factor .
N

∑ ( x i− x̄)2
S 2= i=1
N −1

S 2 is the variance estimated from a sample or Sample variance

The standard deviation is always the square root of the corresponding variance, so
S is called the sample standard deviation.

√
N

∑ ( xi − x̄ )2
i=1
S=
N −1

Exemple : Example
The numbers were as follows:

Group A : 2, 3, 4, 8

Group B : 1, 2, 4, 10

Group C : 0, 1, 5, 11

Find the sample variance and the sample standard deviation of each group of
numbers.
Since the mean of Group A (and also of the other groups) is 4.25

22
 Statistic

2 [(2−4,25)2+(3−4,25)2+( 4−4,25)2 +(8−4,25)2 ]

Group A : S = =6,917
( 4−1)
S = √6,917=2,63
2 2 2 2
2[(1−4,25) +(2−4,25) +(4−4,25) +(10−4,25) ]
Group B : S = =16,25
(4−1)
S = √ 16,25=4,031
2 2 2 2
2 [(0−4,25) +(1−4,25) +(5−4,25) +(11−4,25) ]
Group C : S = =44,015
( 4−1)
S = √ 44,015=6,634

Coefficient of Variation
A dimensionless quantity, the coefficient of variation is the ratio between the
standard deviation and the mean for the same set of data, expressed as a

percentage. This can be either

σ ; s whichever is appropriate, multiplied by
μ x̄
100%.

C. Quartiles, Deciles, Percentiles, and Quantiles

Quartiles, deciles, and percentiles divide a frequency distribution into a number of

parts containing equal frequencies. The items are first put into order of increasing
magnitude. Quartiles divide the range of values into four parts, each containing one
quarter of the values. Again, if an item comes exactly on a dividing line, half of it is
counted in the group above and half is counted below. Similarly, deciles divide into
ten parts, each containing one tenth of the total frequency, and percentiles divide
into a hundred parts, each containing one hundredth of the total frequency. If we
think again about the median, it is the second or middle quartile, the fifth decile,
and the fiftieth percentile. If a quartile, decile, or percentile falls between two items
in order of size, for our purposes the value halfway between the two items will be
used. Other conventions are also common, but the effect of different choices is
usually not important. Remember that we are dealing with a quantity which varies
randomly, so another sample would likely show a different quartile or decile or
percentile.

Exemple : Example
Consider the sample consisting of the following nine results: 2.3, 7.2, 3.7, 4.6, 5.0,
7.0, 3.7, 4.9, 4.2.
a) Find the median of this set of results by two different methods.
b) Find the lower quartile.
c) Find the upper quartile.
d) Estimate the probability that an item, from the population from which this
sample came, would be less than 4.9.
e) Estimate the probability that an item from that population would be less than
3.7.
Answer: The first step is to sort the data in order of increasing magnitude, giving
the following table:

23
 Statistic

i 1 2 3 4 5 6 7 8 9

x(i) 2.3 3.7 3.7 4.2 4.6 4.9 5 7 7.2

a) The basic definition of the median as the middle item after sorting in order of
increasing magnitude gives x(5) = 4.6 Or Putting (i-0.5)/9=0.5
⟹i=9×0.5+0.5=5, so again the median is x(5)=4.6.
b) The lower quartile is obtained by putting (i-
0.5)/9=0.25⟹i=9×0.25+0.5=2.75. Since this is a fraction, the lower quartile is
(x(2)+x(3))/2=(3.7+3.7)/2=3.7
c) The upper quartile is obtained by putting (i-
0.5)/9=0.75⟹i=9×0.75+0.5=7.25. Since this is a fraction, the lower quartile is
(x(7)+x(8))/2=(5+7)/2=6
d) Probabilities of values smaller than the various items can be estimated as the
corresponding fractions. 4.9 is the 6th item of the 9 items in order of increasing
(6-0.5)/9=0.61. Then the probability that an item, from the population from
which this sample came, would be less than 4.9 is estimated to be 0.61.
e) 3.7 is the item of order both 2 and 3, so we have two estimates of the
probability that an item from the same population would be less than 3.7. These
are (2-0.5)/9 and (3-0.5)/9, or 0.17 and 0.28.

24
Probability
III -

III
Distributions of
Discrete Variables

Probability Functions and Distribution Functions 29

Expectation and Variance 30
Binomial Distribution 32
Poisson Distribution 33

A. Probability Functions and Distribution Functions

Probability Functions
Say the possible values of a discrete random variable, X, are: x 0, x 1, x 2,. .. , x k , and
the corresponding probabilities are p (x 0 ), p ( x 1) , p( x 2) ,... , p( x k ) . Then for any
k
choice of i, p (x i )≥0 ,and ∑ p( xi )=1 , where k is the maximum possible value of i.
i=0
Then p (x i ) is a probability function, also called a probability mass function. An
alternative notation is that the probability function of X is written Pr [ X =x i ] , and
x i are related by an algebraic function, but in other cases the relation is shown in
the form of a table or by isolated spikes on a bar graph as shown in the following
figure.

25
 Probability Distributions of Discrete Variables

Cumulative Distribution Functions

Cumulative probabilities, Pr [X ≤ x], where X still represents the random variable
and x now represents an upper limit, are found by adding individual probabilities.
k
Pr [ X ⩽x ]= ∑ p( xi )
x i ⩽x

where is an individual probability function.

Exemple : Example
For example, if can be only zero or a positive integer,
Pr [X ≤ 3] = p(0) + p(1) + p(2) + p(3)
we have: p(3) = Pr [X ≤ 3] – Pr [X ≤ 2].
In general, p ( x i )=Pr [ X ⩽x i ]− Pr [ X ⩽ xi −1]

B. Expectation and Variance

Expectation of a Random Variable

The mathematical expectation or expected value of a random variable is an
arithmetic mean that we can expect to closely approximate the mean result from a
very long series of trials, if a particular probability function is followed. The
expectation of a random variable X is denoted by E(X) or μ.
The probability of a particular result would be given to a good approximation by the
relative frequency of that result from an extremely large number of trials:
f (x i )
Pr [ x i ]=
∑ f ( xi )
all i

If the number of trials became infinite, this relation would become exact.

26
 Probability Distributions of Discrete Variables
N
f (x i )
We also have : x̄=∑ [ ]
j =1 ∑ f ( x i)
all i
The factor within square brackets in the above equation is the relative frequency for
factor j. Then for an infinite number of trials we have,
E ( X )=μ X =∑ ( xi ) Pr [ x i ]
all i

Exemple : Example
The relation for the expected value can be illustrated for the random variable, R,
which was shown in the followin table and figures. It is the number of heads
obtained on tossing five fair coins.

r : number of Probability
heads Pr

0 1/32
1 5/32
2 10/32
3 10/32
4 5/32
5 1/32

Total 1

1 5 10 10 5 1
μ R= E ( R)=0×( )+1×( )+2×( )+3×( )+ 4×( )+ 5×( )=2,5
32 32 32 32 32 32

Variance of a Discrete Random Variable

The variance was defined for the frequency distribution of a population by:
N 2
( x −μ )
∑ iN
i=1

2
that is the mean value of (x i−μ) . Since the quantity corresponding to the mean

27
 Probability Distributions of Discrete Variables

for a probability distribution is the expectation, the variance of a discrete random

variable must be : σ X = E (x−μ x ) =
2 2
∑ ( x i−μ x )2 Pr ( xi )
all i
An alternative form, like the one found in equation of E(X), is faster to calculate. It
is obtained as follows:
2 2 2 2
E [( X −μ X ) ]=E [( X −2μ X X +μ X )]= E [ X ]−2μ X E [ X ]+μ X
But E [ X ]=μ X . Then
2 2 2 2 2 2 2 2
σ X = E [( X −μ X ) ]=E [( X −2 μ X X +μ X )]=E [ X ]−2μ X +μ X = E [ X ]−μ X

E ( X )=∑ x i Pr ( x i )
2 2

all i

The standard deviation is always simply the square root of the corresponding
variance. Then :

σ=√ E [( X −μ X )2 ]=√ E ( X 2 )−[E ( X )] 2

Exemple : Example
Let us continue with the previous illustration for the random variable, R, given by
the number of heads obtained on tossing five fair coins.
1 5 10 10 5 1
E (R2 )=∑ r 2i Pr (r i)=02 ×( )+12×( )+2 2×( )+3 2×( )+42×( )+52×( )=7,5
all i 32 32 32 32 32 32

From the previous calculation, E [ R]=μ R =2,5

2 2 2
Then σ R= E [ R ]−μ R=7,5−( 2,5) =1,25 and
2
σ R= √ 1,25=1,118

C. Binomial Distribution

The most natural random variable to define on the sample space of Bernoulli trials
is the number of successes. Such a random variable is called a binomial random
variable. If X is the number of successes in n Bernoulli trials where the probability
of success at each trial is p, then we represent the distribution of X by the short-
hand notation: X→B(n,p)
Where: B indicates that X has a binomial distribution and n and p are the
parameters determining which particular distribution from the binomial family
applies to X. The probability distribution of a binomial random variable can be
expressed algebraically as:

Pr [ R=r ]= nC r pr q(n−r)

Where:

Exemple : Example
On the basis of past experience, the probability that a certain electrical component
will be satisfactory is 0.98. The components are sampled item by item from
continuous production. In a sample of five components, what are the probabilities
of finding (a) zero, (b) exactly one, (c) exactly two, (d) two or more defectives?
Answer
The requirements of the binomial distribution are met.

28
 Probability Distributions of Discrete Variables

n = 5, p = 0.02, q = 0.98, where q is taken to be the probability that an item will

be satisfactory, and so p is the probability that an item will be defective.
a) Pr [ X =0]=(0,98)5=0,904
5 1 4
b) Pr [ X =1]= C 1×0,02 ×0,98 =0,092
5 2 3
c) Pr [ X =2]= C 2×0,02 ×0,98 =0,038
d) Pr[X≥2]= 1- Pr[X=0]-Pr[X=1]=1-0.9039-0.0922=0.0038

Expected Mean and Standard Deviation

For any discrete random variable, the expected mean is :
E (R)=μ=∑ (number of sucsses)( Probability of that number of sucsses) for all possible results.
For the binomial distribution :
n n

∑ (r ) Pr [ R=r ]=∑ (r )C nr q(n−r ) p r

r =0 r=0

If the algebra is followed through, the result is: μ=np

Thus, the mean value of the binomial distribution is the product of the number of
trials and the probability of “success” in a single trial. This seems to be intuitively
correct.

σ 2=npq ; σ=√ npq

Exemple : Example
A company is considering drilling 4 oil wells. The probability of success for each well
is 0.40, independent of the results for any other well. The cost of each well is
$200,000. Each well that is successful will be worth $600,000.
a) What is the expected number of successes?
b) What is the standard deviation of the number of successes?
Answer
Binomial distribution with n = 4, p = 0.4, q = 0.6.
Then the expected number of successes is np = (4)(0.400) =1.60.
The standard deviation of the number of successes is √ σ2 =√ 4×0,4×0,6=0,980

D. Poisson Distribution

The Poisson distribution has a single parameter and therefore we denote this
random variable by the symbolic notation, X→P(λt)
The probability of exactly r occurrences in a fixed interval of time or space under
particular conditions is given by:

(λ t ) r e−λ t
Pr [ R=r ]=
r!

where t (in units of time, length, area or volume) is an interval of time or space in
which the events occur, and λ is the mean rate of occurrence per unit time or space
(so that the product λt is dimensionless).

29
 Probability Distributions of Discrete Variables

Once one of these probabilities is:

λt
Pr [ R=r +1]=( ) Pr=[ R=r ]
r +1

Mean and Variance for the Poisson Distribution

Since the Poisson distribution is discrete, the mean and variance can be found from
the previous general relation. the mean or expectation of the number of
occurrences according to the Poisson Probability Distribution is: μ=λt
Therefore an alternative form of the probability function for the Poisson distribution
is:
r −μ
μ e
Pr [ R=r ]=
r!

Thus, the variance of the number of occurrences for the Poisson distribution is
equal to the mean number of occurrences, μ.
2
σ =μ=λ t

30
 Probability Distributions of Discrete Variables

Exemple : Example
The number of meteors found by a radar system in any 30-second interval under
specified conditions averages 1.81. Assume the meteors appear randomly and
independently.
a) What is the probability that no meteors are found in a one-minute interval?
b) What is the probability of observing at least five but not more than eight meteors
in two minutes of observation?
Answer
a) λ = (1.81) / (0.50 minute) = 3.62 / minute.
For a one-minute interval, μ = λt = 3.62.
Pr [none in one minute] =exp(–λt)= exp(–3.62)= 0.0268.
b) b) For two minutes, μ = λt = (3.62)(2) = 7.24.
(λ t ) r e−λ t
Pr [ R=r ]=
r!
Then:
(7,24)5 e −7,24
Pr [ R=5]= =0,1189
5!
From the equation
Pr[R=r+1]=(λt/(r+1))Pr[R=r]
So
Pr[R=6]=(7.24/6)(0.1189)=0.1435,
Pr[R=7]=(7.24/7)(0.1435)=0.1484,
Pr[R=8]=(7.24/8)(0.1484)=0.1343,
Then Pr [at least five but not more than eight meteors in two minutes] = Pr
[5 or 6 or 7 or 8 meteors in two minutes]
Pr [5 or 6 or 7 or 8 meteors in two minutes]=
0.1189+0.1435+0.1484+0.1343 = 0.545

31
Probability
IV -

IV
Distributions of
Continuous
Variables

Probability from the Probability Density Function 37

Expected Value and Variance 38
Exponential Distribution 40
The Normal Distribution 40
LOGNORMAL DISTRIBUTION 44

A. Probability from the Probability Density Function

Basic Relationships
The probability that a continuous random variable will be between limits a and b is
given by an integral, or the area under a curve.
b

Pr [a < X < b]=∫ f ( x) dx

a

f(x): Probability density function.

X: Continuous random variable.

33
 Probability Distributions of Continuous Variables

The cumulative distribution function for a continuous random variable is given by

the integral of the probability density function between x = –∞ and x= x1, where x 1
is a limiting value.
x1

Pr [ X ⩽b]=F ( x1 )= ∫ f ( x ) dx
−∞

+∞
F (∞)=∫ f ( x)dx=1
−∞

Since any probability must be between 0 and 1, as we have seen previously, the
probability density function must always be positive or zero, but not negative.
f(x)≥0

34
 Probability Distributions of Continuous Variables

Exemple : Example
A probability density function is given by:

{ }
0 for x <0
3
f (x )= × x 2 for 0< x <2
8
0 for x >2

+∞ 0 2 +∞
3 3 1
F ( ∞)=∫ f ( x) dx= ∫ 0 dx +∫ ×x + ∫ 0 dx= × ×2 =1
2 3

−∞ −∞ 0 8 2 8 3

B. Expected Value and Variance

For a continuous random variable this becomes the corresponding integral involving
the probability density function:
+∞
μ=E (x )=∫ x f ( x)dx
−∞

We saw also that the variance of a discrete random variable is the expectation of
(x−μ)2 . This carries over to a continuous random variable and becomes:
+∞
σ x =E [( x−μ) ]= ∫ ( x−μ) f ( x)dx
2 2 2
−∞

The alternative form given by :

2 2 2
σ x =E ( X )−μ x

still holds and is generally faster for calculations. For continuous random variables
+∞
E (x )= ∫ x f ( x) dx
2 2
−∞

35
 Probability Distributions of Continuous Variables

Exemple : Example
The random variable has the probability density function given by:

{ }
0 for x <0
3
f (x )= × x 2 for 0< x <2
8
0 for x >2

a) Find the probability that X is between 1 and 2.

b) Find the cumulative distribution function of X.
c) Find the expected value of X.
d) Find the variance and standard deviation of X.
Answer
2 2
3 7
a) Pr [ 1< X < 2]= ∫ f ( x) dx= ×∫ x dx=
8 1
2
8
1

{ }
x1

x 1<0, F (x 1 )=∫ 0 dx=0

−∞
x1 0 x1

Pr [ X ⩽x 1 ]=F (x 1)=∫ 3 1
b) −∞
0< x 1< 2, F ( x1 )= ∫ 0 dx +∫ ( )×x 2= x 31
−∞ 0 8 8
0 2 x1
3
x1 > 2 , F (x 1)= ∫ 0 dx+∫ ( )×x +∫ 0 dx=1
2

−∞ 0 8 2

+∞ 0 x1 x1 2
3 3 3
c) μ=E ( x )=∫ x f ( x)dx= ∫ x ×0 dx +∫ ( 8 ×x )×x dx +∫ x 0 dx=∫ ( 8 x )dx =1,5
2

−∞ −∞ 0 2 0

+∞ 0 x1 x1 2
3 3 4
d) E ( x )= ∫ x f ( x) dx =∫ x ×0 dx +∫ ( 8 × x )× x dx+∫ x 0 dx=∫ ( 8 x ) dx=2,4
2 2 2 2 2 2

−∞ −∞ 0 2 0

2 2 2 2
Then σ = E [ X ]−μ =2,4−(1,5) =0,150
X X

And σ X = √ 0,150=0,387 .

C. Exponential Distribution

The exponential distribution has the following probability density function:

f (x )=λ e−λ x ; for x⩾0

f ( x)=0 ; for x <0

where is a constant closely related to the mean and standard deviation.

For the cumulative distribution function for the exponential distribution is found
easily by integration:
x1

F ( x 1)= Pr [ 0< X < x1 ]=∫ λ e

−λ x −λ x 1
dx=1−e
0

The mean of the exponential distribution is the reciprocal of the rate parameter.
The result can be obtained through integrating the expected value integral by parts.

36
 Probability Distributions of Continuous Variables

+∞
1
E ( X )=∫ λ t e−λ x dx=
0
λ

The variance of the exponential distribution is obtained from evaluating the

following integral again through integration by parts.

2 2 2 1
σ =E ( X )−[ E ( X )] = 2
λ

The variance equals the square of the mean and therefore the mean equals the
standard deviation for an exponential distribution.

D. The Normal Distribution

The probability density function for the normal distribution is given by:
2
(x−μ)
−
1 2σ
2

f (x )= e
σ √2 π

where μ is the mean of the theoretical distribution, σ is the standard deviation, and
π = 3.14159 ... This density function extends from –∞ to +∞. Its shape is shown in
the following figure.
the mean and variance of X are: μ and σ2 .

Let z=(x-μ)/σ , dz=(1/σ) dx ⟹dx=σdz

z2 z
2
z2 z
2

1 − 1 −2
Pr [ x 1 < X < x 2 ]=∫ e 2 σ dz=∫ e dz
1z σ √2 π z √2 π 1

Where:
x 1−μ x 2−μ
z 1= σ , z 2= σ

37
 Probability Distributions of Continuous Variables

The standard normal cumulative distribution function, Φ(z), as a function of z, is

defined as follows:
z2 z
2

1 −2
Φ( z 1)= Pr [−∞< Z < z 1 ]=Pr [Z < z 1 ]=∫ e dz
−∞ √2 π

Using Tables for the Normal Distribution

Table A1 gives values of the cumulative normal probability as a function of z, the
number of standard deviations from the mean.

38
 Probability Distributions of Continuous Variables

Exemple : Example
What is the probability that Z for a normal probability distribution is between –0.76
and +0.76?
Pr [–0.76 < Z < +0.76] corresponds to the middle area cross-hatched in Figure.
The calculation of probabilities is as follows:
Pr [–0.76 < Z < +0.76] = Pr[Z > 0.76] – Pr [Z > – 0.76]
Pr [–0.76 < Z < +0.76]= Φ(0.76) – Φ(–0.76)
Pr [–0.76 < Z < +0.76]= 0.7764 – 0.2236
Pr [–0.76 < Z < +0.76]= 0.5528

39
 Probability Distributions of Continuous Variables

40
 Probability Distributions of Continuous Variables

E. LOGNORMAL DISTRIBUTION
2
Let X be N (μ X ,σ X ) . The random variable Y as determined from Equation is
X
Y =e said to have a lognor mal distribution. The pdf of Y is easy to determine

{ }
1
− (ln( y−μ)2)
1 2 σ2x
f ( y)= y σ √ 2 π e for y≥0
x
0 elsewhere

41
 Probability Distributions of Continuous Variables

Now, writing σ X =σ(ln Y ) , the pdf of Y can be written in the form :

{ }
1 y
− ( )
1 2(σ lny)2 θY
f ( y)= y σ √ 2 π e for y≥0
(ln y)
0 elsewhere

The mean and standard deviation of Y can be found either through direct
integration by using f Y ( y) . In terms of Y and ln Y , they take the forms:

{ }
2
σln Y
( )
2
μY =θY e
2
2 2 σln Y
σ =μ Y (e
Y −1)

42
Bibliographie

[[1]] W.J. DECOURSEY.Statistics ⁡and⁡Probability⁡for⁡Engineering⁡Applications⁡With⁡Microsoft®⁡Excel.

2003, Elsevier Science (USA).
[[2]] WENDY L. MARTINEZ, ANGEL R. MARTINEZ Computational⁡statistics⁡Handbook⁡with⁡MATLAB. 2002
Chapman & Hall/ CRC
[[3]] NICK T. THOMOPOULOS, ARVID C. JOHNSON, Tables⁡And⁡Characteristics⁡of⁡the⁡Standardized⁡Lognormal
Distribution
[[4]] T.T. SOONG.⁡ FUNDAMENTALS⁡OF⁡PROBABILITY⁡AND⁡STATISTICS⁡FOR⁡ENGINEERS. 2004.
John Wiley & Sons Ltd
[[5]] RADIM BRIŠ. PROBABILITY⁡AND⁡STATISTICS⁡FOR⁡ENGINEERS. 2011 VŠB - Technical University
of Ostrava Faculty of Electrical Engineering and Computer Science Department of Applied Mathematics
[[6]] DOUGLAS C. MONTGOMERY, GEORGE C. RUNGER,⁡Applied⁡Statistics⁡and⁡Probability⁡for⁡Engineers. 2002.
John Wiley & Sons.

43