Probability and Statistics (MATH F113)

Descriptive Statistics

Himadri Mukherjee

Department of Mathematics
BITS PILANI K K Birla Goa Campus, Goa

December 26, 2022

 Introduction

Descriptive Statistics is the study of a data through its various

parameters, such as mean, median, standard deviation, co-relation,
frequency. Such parameters are used to understand the data and
its distribution.

Inferential Statistics uses several estimation techniques,

hypothesis testing to make inferences about the data set.
 Notations

• The term population represents items of interest.

• Sample or Random Sample is a portion of the population.
• A Parameter is a summary measure of the population.
• Statistic is a summary measure of the sample.
 Sample

A Sample is a part of the population, that is ”representative” of

the population in some way. A random sample when every
subset of the population has an equal chance of being selected.
Definition
A set of observations X1 , X2 , X3 , . . . Xn constitutes a random
sample of size n if,
• Xi are independent.
• Xi have the same distribution as X.
 Statistic

A statistic is a random variable whose numerical values can be

determined from a random sample. So it is a function of a random
sample.

n P
i Xi
X
Example. max{Xi }, min{Xi }, Xi , .
n
i=1
 Sample mean

Definition
Let X1 , X2 , X3 , . . . Xn be random samples from the distribution of
X.The statistic
Xn
Xi
i=1
X=
n
is called the sample mean of the population.
 Sample variance

Definition
Let X1 , X2 , X3 , . . . Xn be random samples from the distribution of
X. The statistic,
Xn
(Xi − X)2
i=1
S2 =
n−1
√
is called the sample variance, and S = S 2 is called the sample
standard deviation.
 Sample variance continued

The sample variance of a sample can be calculated using the

following formulas as well,
n
X 2
Xi2 − nX
i=1
S2 =
n−1
or equivalently,
n
X Xn
n Xi2 − ( Xi )2
i=1 i=1
S2 =
n(n − 1)
 Law of large numbers

Theorem
If a random sample is taken from a population having mean µ and
standard deviation σ, then X is a random variable with mean µ
2
and variance σn .

The standard deviation √σ , of X is also called the standard error

n
of the mean.
 Sampling

Theorem
if a random sample is taken from a population having mean µ and
variance σ 2 then E(S 2 ) = σ 2 .
Proof
 The central limit theorem

Theorem
If X is the mean of a sample of size n taken from a population of
mean µ and variance σ 2 , then for n large, X is normal with mean
2
µ and variance σn .

Corollary
X−µ
√
σ/ n
follows a standard normal distribution.
Explanation
 Example 1

Example 1. The mean lifetime of a tool is estimated to be of 41.5

hours with a standard deviation of 2.5 hours. What is the
probability that a sample of 50 of such tools will have a mean
between 40.5 and 42 hours?
Continued
 Example 2.

Example 2. Suppose a mathematician has 100 light bubs, each

having longevity following an exponential distribution of 5 months.
If the mathematician uses the bulbs for his study desk, and
replaces one immediately after one expires. What is the probability
that there will be a light bulb working after 45 years?
Continued
 CLT for sum

Theorem
If n samples Xi are drawn from a population with meanP µ, and
standard deviation σ, then for large enough n, the sum ni=1 Xi
follows a normal distribution with mean nµ and standard deviation
√
nσ.
 Normal approximation to Binomial and Poisson

Theorem
• For a large enough n, a random variable X following binomial
distribution
p B(n, p) can be approximated by
N (np, np(1 − p)).
• for a large enough n, a Poisson random variable X, with
√
Poisson parameter, nλ, can be approximated by N (nλ, nλ).
Continued
 Estimation

Estimation: the procedure of estimating a population parameter

using the sample is called estimation. There are two kinds of
estimation, point estimation, interval estimation.
1 Estimation of a population parameter by a single real number
is called a point estimation.
2 A point estimator of a population parameter θ, will be
denoted by θ̂
 Example 4.

Example 4. Let us look at this data of lifetime of a gadget in

months,
24.46,25.61,26.25,26.42,26.66,27.15,27.31,27.54,27.74,27.94,27.98,
28.04, 28.28, 28.49, 28.50, 28.87, 29.11, 29.13, 29.50, 30.88,31.20
 Unbiased estimator

Definition
An unbiased estimator of a parameter θ, is θ̂ such that E(θ̂) = θ.

• S 2 is an unbiased estimator of the variance.

• The sample mean X is an unbiased estimator of the
population mean.
• S is not an unbiased estimator of the variance.
 Unbiased estimator continued

The bias of an estimator is the difference E(θ̂) − θ. The mean

square error of an estimator is E(θ̂ − θ)2 )
 Example 4.
X
Example 4. For a binomial random variable X ∼ B(n, p), p̂ = n
is an unbiased estimator of the parameter p.
 Example 5.

Example 5. Let X1 , X2 , X3 , . . . Xn be samples from a population

with mean µ and standard deviation σ. Then the following is an
unbiased estimator of the variance σ 2 .
(Xi − X)2
P
2
S =
n−1
 Example 6.
√
X+p√ n
Example 6. If X ∼ B(n, p) then n+ n
is an unbiased estimator
for p.
 Estimation methods

The method of moments let X1 , X2 , X3 . . . Xn be a random

sample from a population following density function f (x). Using
the moments Mk = E(X k ) and equating it with the sample
Xik
P
moments n we solve for the parameters.
 Example 7.

Example 7. Let X1 , X2 , X3 , . . . Xn be a random sample from a

gamma distribution with parameter α, β. Using the method of the
moments find estimators for the parameters α and β.
 Example 7 continued

Let us have this following data following a gamma distribution

Γ(α, β). Find estimators for the parameters.

152,115,109, 94, 88, 137, 152, 77, 160, 165, 125, 40, 128, 123,
136, 101, 62, 153, 83, 69
 The maximum likelihood estimator

Definition
( discovered by Fisher) Let X1 , X2 , X3 . . . Xn have a joint
distribution f (x1 , x2 , x3 , . . . , θ1 , θ2 , θ3 . . . θm ), where the
parameters θ1 , θ2 , θ3 , . . . θm are unknown, x1 , x2 , x3 . . . xn are
observations, expressing the likelihood of these observations as
functions of θ1 , θ2 , θ3 θm . This is called the likelihood function.
The values of θi that maximizes the likelihood function are called
the maximum likelihood estimators or simply MLE.
 Example 8.

Example 8. Find the maximum likelihood estimator where the

population X ∼ B(n, p), for the parameter p.
Continued
 Example 9.

Example 9. Let X1 , X2 , X3 . . . Xn follow a Poisson distribution

P (λ). Find the maximum likelihood estimator for λ.
 Example 10.

Example 10. Let X1 , X2 , X3 , Xn be a random sample from an

exponentially distributed population with parameter λ, find the
MLE for the parameter.
Example 10. continued
 Example 11.

Example 11. Let X1 , X2 , X3 , Xn be a random sample from a

normal distribution N (µ, σ), find maximum likelihood estimators
for the mean and the variance.
Continued
Continued
 The invariance principle

Theorem
Let θˆ1 , θˆ2 , θˆ3 . . . θˆn be the MLE for the parameters θ1 , θ2 , θ3 , . . . θn ,
let h(θ1 , θ2 , θ3 , . . . θn ) be any function. The MLE of the function
h(θ1 , θ2 , θ3 , . . . θn ) is h(θˆ1 , θˆ2 ,θˆ3 , . . . θˆn )
 Example 12.

Example 12. Find the MLE for the standard deviation in a

Normal population.
 Example 13.

Example 13. Use MLE to find an estimator for the parameter a

for a random sample following uniform distribution over (0, a).
 Example 14.

Example 14. Using MLE find an estimator for a for a random

sample following the distribution f (x) = (a + 1)xa where
0 < x < 1.
Continued
 Example 15.

Example 15. Find the maximum likelihood estimator for the

parameter a, in the distribution,
(
3a3 x−4 if x ≥ a
f (x) =
0 otherwise
Continued
 Example 16.

Example 16. Let X be a random variable that follows the

following distribution, find the MLE for the fourth moment of X.
(
cxa if 0 < x < 1
f (x) =
0 otherwise
Continued
 Example 17.

Example 17. The shear strength of each of 10 text spot welds is

determined giving the following data
292,376,401,367,389, 362, 409, 415, 358, 375.
• Assuming that the shear strength is normally distributed,
estimate the true average shear strength and the standard
deviation of shear strength using the method of maximum
likelihood estimator.
• Estimate the value below which 95 percent of all welds will
have their strength.
• Suppose we decide to examine another spot weld, let X be
the strength of that spot weld. Use the given data to obtain
the MLE of P (X < 400).
Continued
Continued
 Example 18.

Example 18.