ENG 023

Metrology and Calibration

Topic 3
Introduction to
Probability

Sayfa 1
 Content
PART I
BASIC SET THEORY

PART II
BASIC PROBABILITY

PART III
RANDOM VARIABLES

PART IV
SPECIAL DISTRIBUTION FUNCTIONS

Sayfa 2
 PART I

BASIC SET THEORY

Sayfa 3
Set Theory
In mathematics, a well defined collection of objects
is called the set.

Examples:

A = {1,2,3,4} => finite set

M = {apple, banana, orange} => finite set
R = {x | x is a river on Earth} => finite set
N = {0,1, 2, 3, 4, ...} => infinite set
P = {2, 4, 8, ...} => infinite set
K = {x | 2<x<5, x is a real} => infinite set

Sayfa 4
Notation:

p A p is elements of A

A B A is subset of B

U Universal set

 Empty set

For any set A

  AU
Sayfa 5
Set Operations: Venn Diagram

Intersection
A B

Union
A B

Difference
B A

Complement C
A
Sayfa 6
Example
Let
A ={1, 2, 3, 4}, B = {3, 4, 5, 6} and U={1, 2, 3, 4, 5, 6, 7,8 …}

A U B = {1, 2, 3, 4, 5, 6}

A n B = {3, 4}

A \ B = {1, 2}

Ac = {5, 6, 7, 8, ...}

Sayfa 7
 PART II

BASIC
PROBABILITY

Sayfa 8
Probability (=Olasılık)
Historically, the probability theory began with study of
games of chance, such as roulette and cards.

The probability is the study of

random or non-deterministic experiments

If a coin is tossed in the air, then

it is certain that the coin will come down

but
it is not certain that a head will appear.

Sayfa 9
Relative Frequency (=Göreli Sıklık)

Suppose we repeat an experiment of tossing a die.

Let
s be the number of times a “six” appears
n be the number of tosses

Then the ratio s/n becomes stable in the long run:

s
f  f approaches
a limit
n as n -> ∞

This stability is the basis of probability theory!

Sayfa 10
Example
Here is the result obtained from a computer
simulation for tossing of a coin and observing
frequency of head!

n s f = s/n
------------- -------- ---------
10 4 0.4000000
100 41 0.4100000
1,000 476 0.4760000
10,000 5059 0.5059000
The result
100,000 49942 0.4994200
approaches
1,000,000 500351 0.5003510
a limit
10,000,000 4998906 0.4998906
as n -> ∞
100,000,000 50006417 0.5000641
1,000,000,000 500000839 0.5000084
Sayfa 11
Example
Here is the result obtained from a computer
simulation for tossing of a die and observing
frequency of six!

n s f = s/n
------------- -------- ---------
10 3 0.3000000
100 19 0.1900000
1,000 186 0.1860000
10,000 1659 0.1659000
The result
100,000 16748 0.1674800
approaches
1,000,000 166705 0.1667050
a limit
10,000,000 1667210 0.1667210
as n -> ∞
100,000,000 16666290 0.1666629
1,000,000,000 166666653 0.1666666
Sayfa 12
Probability Theory
The probability p of an event A is defined as follows:
If A occurs in s ways out of a total n equally likely ways
then
s
p  P( A) 
n

* Tossing a coin: Head occurs 1 way out of 2 => p = 1/2

* Tossing a die: Six occurs 1 way out of 6 => p = 1/6
* Tossing a die: Even number occurs 3 ways out of 6 => p = 3/6

Probability is the measure of

how likely an event is
Sayfa 13
Example
A glass jar contains 6 red, 5 green, 8 blue and
3 yellow marbles. If a single marble is chosen at
random from the jar, what is the probability of choosing
a red marble? a green marble? a blue marble? a yellow
marble?

Sayfa 14
Sample Space (=Örneklem Uzayı)

The set S of all possible outcomes of some given

experiment is called “sample space”

* Tossing a coin: S = {H, T}

* Tossing two coins: S = {HH, HT, TH, TT}

Sayfa 15
* Tossing a die:
S = {1, 2, 3, 4, 5, 6}

* For two dice the outcomes are

S = {11,12,13,14,15,16,
21,22,23,24,25,26,
31,32,33,34,35,36,
41,42,43,44,45,46,
51,52,53,54,55,56,
61,62,63,64,65,66}

Sayfa 16
Selecting a card randomly from a shuffle pack of playnig
cards, the possible outcomes are:

Sayfa 17
Axioms of Probability
Let S be sample space and A and B are two events.

A1. 0  P( A)  1

A2. P( S )  1

A3. If A and B are mutually exclusive events (ayrık olaylar)

P( A  B)  P( A)  P( B)

Sayfa 18
Theorems of Probability
T1. P ( )  0 (probability of impossible event is zero)
T2. If Ac is the complement of A, then P( Ac )  1  P( A)

T3. If A and B are any two events:

P( A  B)  P( A)  P( A  B)

T4. If A and B are any two events:

P( A  B)  P( A)  P( B)  P( A  B)

Sayfa 19
Example
A point Q is selected randomly
in a square whose side is 2 cm.
A circle is drawn tangent to the
edges of the square.
Find the probability of the point
being inside the circle.

Sayfa 20
 PART III

RANDOM
VARIABLES

Sayfa 21
Random Variable (=Raslantı Değişkeni)
 A Random Variable (RV) is a set of values that occur
randomly and have associated probabilities.

 RV can be discrete (kesikli) or continues (sürekli).

 Probability mass function (pmf)

describes the distribution of discrete probabilities

 Probability distribution function (pdf)

describes the distribution of continues probabilities.

Sayfa 22
Discrete RV Examples:
pmf

Tossing a coin: f(x)

X = {H,T}
f(x) = {1/2, 1/2}
0.5

x
H T

Tossing a die: f(x)

X = {1,2,3,4,5,6}
f(x) = {1/6,1/6,1/6,1/6, 1/6, 1/6}
0.167

x
1 2 3 4 5 6

Sayfa 23
Discrete RV Examples:
pmf

f(x)
0.4
X = {1,2,3,4}
f(x) = {0.3, 0.4, 0.1, 0.2} 0.2
x
1 2 3 4

Sayfa 24
Continues RV Examples:
pdf

Random angle f(x)

in the range [0, 2π]

X = [0, 2π] 1/2π
f(x) = 1/2π
x
0 2π

X = [0, 2]
f(x) = 3x2/8

Sayfa 25
Properties of Random Variables

Discreate RV Continues RV

f ( xi )  0 f ( x)  0


 f (x )  1
i
i  f ( x)dx  1


b b

 f ( x )  P ( a  x  b)
ia
i  f ( x)dx  P(a  x  b)
a

Sayfa 26
Expectation Values
Expectation Value or Mean Value of RV is denoted by:

E[X] or <X> or x

and defined by:

Discrete RV: E[ X ]   xi f ( xi )
i


Continues RV: E[ X ]   x f ( x)dx


Sayfa 27
Mean of the Squares
It is defined as:

Discrete RV: E[ X ]   x f ( xi )
2 2
i
i


E[ X ]  x
2 2
Continues RV: f ( x)dx


Note that (RMS: Root Mean Square)

RMS  E[ X 2 ]
Sayfa 28
Variance
Variance is defined as:

Discrete RV:  2   ( xi  x ) 2 f ( xi )
i

Continues RV:  2   ( x  x ) 2 f ( x)dx


One can also prove that

 2  E[ X 2]  ( E[ X ]) 2
 X    X 
2 2

Sayfa 29
Standard Deviation
Square root of variance is called standard deviation:

  2

  E[ X ]  ( E[ X ])
2 2

  X 2    X 2

Sayfa 30
Example
X = {1, 2, 3, 4, 5, 6}
f(x) = {1/6, 1/6, 1/6, 1/6, 1/6, 1/6}
Find
(a) Expectation value of X, (b) RMS and (c) standard deviation

Sayfa 31
Example
The following table gives the probability distribution of, X,
the number of telephones in a randomly selected home in a
certain community.

x: 0 1 2 3 4
f(x): 0.021 0.412 0.283 0.188 0.096

One home is selected randomly.

The probability that it will have:
(a) no telephone is 0.021
(b) fewer than two telephones is 0.433
(c) at least three telephones is 0.284
(d) one or two telephones is 0.695
Sayfa 32
Example f(x)
Consider the uniform pdf.
X = [0,1] 1
f(x) = 1
x
Find 0 1

(a) mean, (b) RMS and (c) standard deviation

Sayfa 33
Example f(x)
Consider the triangular pdf.
X = [0,2] 1
f(x) = kx
x
Find 0 2

(a) the value k (b) mean and (c) standard deviation

Sayfa 34
 PART IV

SPECIAL
DISTRIBUTION
FUNCTIONS

Sayfa 35
Binomial Distribution Function
The binomial distribution function specifies the number of times
(k) that an event occurs in n independent trials where p is the
probability of the event occurring in a single trial.

where

Sayfa 36
Example: A coin is tossed 3 times.
If you call heads a success, draw pmf for k = 0, 1, 2, 3.

Sayfa 37
Example: A coin is tossed 6 times.

The probability of getting exactly four heads:

The probability of getting at least four heads:

Sayfa 38
Poisson Distribution Function
In the binomial equation, if the probability p is so small then the
distribution of events can be approximated by the Poisson
distribution.

lim Binomial Distribution = Poisson Distribution

p->0

Sayfa 39
Example: Birthday problem
Probability of one person to have birthday in any day is
1/365 = 0.00274. Calculate the probability that 4 people
share a birthday in a group of 1000 people.
----

Mean: λ = 1000*0.00274 = 2.734

Probability: p = exp(-2.734)*2.7344/4! = 0.151

Sayfa 40
Example
Suppose 1% of items made by a factory are defective.
Find the probability that 6 defective items in a sample of
300 items.

Sayfa 41
The Gaussian or Normal Distribution Function
In Statistics, if the number of events is very large (n>20),
then the Gaussian (normal) distribution function may be
used to describe nearly all events.

The Gaussian distribution is a continuous

Random Variable of the form:

Sayfa 42
Sayfa 43
Properties of Gaussian Function


p( x)  0
 p( x)dx  1


E[ X ]   xp( x)dx  

b

  p( x)dx  P(a  x  b)
 (x  ) p( x)dx  
2 2 a



Sayfa 44
Standard Normal Curve
The normal distribution function for

µ = 0 and σ = 1

is called the standard normal distribution function.

1  x2 / 2
f ( x)  e  0.4 exp(  x / 2)
2

2

Sayfa 45
Sayfa 46
Area Under the Curve
Total area under the standard normal curve is 1.
1  x2 / 2
f ( x)  e
2


 f ( x)dx  1


Sayfa 47
Area under the standard normal curve between [-1, 1] is:

1
1  x2 / 2

1 2
e dx  0.6827

This corresponds
+- 1 sigma

Sayfa 48
Area under the standard normal curve between [-2, 2] is:

2
1  x2 / 2

2 2
e dx  0.9545

This corresponds
+- 2 sigma

Sayfa 49
Area under the standard normal curve between [-3, 3] is:

3
1  x2 / 2

3 2
e dx  0.9973

This corresponds
+- 3 sigma

Sayfa 50
Area under the standard normal curve between [a, b] is:

b
1  x2 / 2

a 2
e dx  (b)  (a)

The values of the

function phi(x)
can be taken from
a table or from the
figure on next page.

Sayfa 51
Sayfa 52
Example:
2.3
1
e
 x2 / 2
dx  (2.3)  (1.2)  0.99  0.12  0.87
2 1.2

From CDF figure

(previous page)

 ( 2.3)  0.99
(1.2)  0.12

Sayfa 53
Alternatively one can use Octave’s normcdf(x) function:

2.3
1  x2 / 2

1.2 2
e dx  (2.3)  (1.2)  0.8742

octave:> normcdf(2.3)
ans = 0.9893
octave:> normcdf(-1.2)
ans = 0.1151
octave:> normcdf(2.3)-normcdf(-1.2)
ans = 0.8742

Sayfa 54
Example: Evaluate the integral
1.4
1
e
 x2 / 2
dx
2 0

Sayfa 55
Example: Evaluate the integral
1.1
1
e
 x2 / 2
dx
2 

Sayfa 56
Example: Evaluate the integral
0.5
1
òe
- x2 /2
dx
2p -0.5

Sayfa 57
Example: Let X be a random variable with the standard
normal distribution. Find:
(a) P(0<X<1.4)
(b) P(X<-1.1)
(c) P(|X|<0.5)
(d) P(X>3)
(e) P(X>5)

Sayfa 58
 If a random varibale X is normally distributed, we compute
the probability that X lies between a and b as follows.

1. First we change a and b into standard units:

a b
a*  b* 
 
2. Then we compute the probability from

P(a  X  b)  P(a*  X *  b*)

= area under the standard normal curve between a and b

Sayfa 59
Example: Suppose the temperature during June is normally
distributed with mean 20 oC and standard deviation 3.33 oC.
Find the probability p that the temperature is between 21.11 oC
and 26.66 oC (Answer: 0.3479)

Sayfa 60
Example: Mean weight of 500 male students at a certain
university is 72 kg and the standard deviation is 5 kg.
Assuming that the weights are normally distributed, find how
many students weigh:
(a) between 66 and 75 kg (Answer: 305)
(b) more than 80 kg (Answer: 27)

Sayfa 61
Example Normal Distributions
 Here we will examine some interesting real data whose
values are distributed normally.

 For each example, histogram of the data is fitted to a

Gaussian Function indicated by a blue line.

Sayfa 62
Annual Rainfall (1960-2012)
Mean : <x> = 651.10 mm
Std. Dev. : σ = 74.35 mm

Data: http://www.mgm.gov.tr
Sayfa 63
Air temperature in Istanbul for the last 105 years.
Mean temperature: <x> = 21.97 oC
Std. Dev. : σ = 1.12 oC

Data: http://data.giss.nasa.gov/tmp/gistemp/STATIONS/tmp_649170620000_14_0/station.txt
Sayfa 64
Background Radiation in Gaziantep (2013)
Mean : <x> = 101.3 counts / sec
Std. Dev. : σ = 2.5 counts / sec

Data is obtained by: Research Assistant Sadık Zuhur (University of Gazaintep)

Sayfa 65
Questions
1. Let
U = {a,b,c,d,e,f,g}, A = {a,b,c,d,e}, B = {a,c,e,g}, C = {b,e,f,g}
Find AnB, AUB, C\B, (A\Bc)c and Cc n A.

Sayfa 66
2. Which of the following is a random experiment?
(a) Tossing a coin.
(b) Rolling a single 6-sided die.
(c) Choosing a marble from a jar.
(d) All of the above.
3. Which of the following is an outcome?
(a) Rolling a pair of dice.
(b) Landing on red.
(c) Choosing 2 marbles from a jar.
(d) None of the above.
4. Which of the following experiments does NOT have equally
likely outcomes?
(a) Choose a number at random from 1 to 7.
(b) Toss a coin.
(c) Choose a letter at random from the word SCHOOL.
(d) None of the above.
Sayfa 67
5. What is the probability of choosing a vowel from the English
alphabet?
6. A number from 1 to 11 is chosen at random. What is the
probability of choosing an odd number?
7. What is the probability of choosing a king from a standard
deck of playing cards?
8. What is the probability of choosing the letter i from the word
probability?
9. What is the probability of choosing a jack or a queen from a
standard deck of 52 playing cards?
10. What is the sample space for choosing a letter from the
word mathematics?
11. What is the sample space for choosing a prime number less
than 15 at random?
12. What is the probability that a single throw of a die will result
in either 2 or 5?
Sayfa 68
13. A fair coin is tossed 5 times. Find the probability of getting
(a) exactly four heads (b) at least two heads (c) no heads.

14. A fair dice is tossed 7 times. Find the probability of getting:

(a) exactly four ONEs (b) at least four ONEs (c) no ONEs

15. A woman has 8 children, the probability of each child being

female is 50%. What is the probability of being
(a) 4 children female (b) all children female.

Sayfa 69
16. A communication system contains 6 stations. Independent
probability of each station being functional is 90%. If the
system requires at least 4 stations to be functional what is
the probability that the communication system is functional.
(Answer: 0.9842)

17. Suppose 2% of the people on the average are left-handed.

Find the probability of 3 or more left-handed among 100
people.
(Answer: 0.325)

Sayfa 70
18. Given the random variable and corresponding probability
mass function (pmf)

X = {1, 2, 3, 4, 5}
f(x) = {0.1, 0.3, 0.4, 0.1, 0.1}

Calculate
(a) ∑ f(xi)
(b) P(1<x<5)
(c) E[X]
(d) E[X2]
(e) RMS
(f) Variance
(g) standard deviation

Sayfa 71
19. Given the probability density function f(x) = 3x2/8.

Calculate
(a) Mean
(b) RMS
(c) P(x>0.5)

20. Given the pdf f(x) = N sin(x) for 0<x<π.

(a) Compute the normalization constant N.
(b) Compute the expectation value and variance.

21. Given the pdf f(x) = k exp(-0.1x) for 0 < x < inf.
(a) Compute the normalization constant k.
(b) Find P(x>2).

Sayfa 72
22. Evaluate the following integrals:
2
1

z2 / 2
e dz
2 1

2
1

( z 1) 2 / 8
e dz
2 2 1

Sayfa 73
23. Suppose the diameters, D, of screws
manufactured by a company are normally distributed with
mean 0.25 cm and standard deviation 0.02 cm. A screw is
considered defective if its diameter D < 0.22 cm.
A sample of 250 screws are selected randomly.
Estimate the number of defective screws in this sample.

Sayfa 74