Scribd
Upload
68 views16 pages

Mathematical Statistics (MA212M) : Lecture Slides

This document provides an overview of statistical inference and introduces key concepts like point estimation, statistics, and the method of moment estimator. It explains that the goal of statistical inference is to estimate unknown population parameters based on a random sample. A point estimator is a statistic used to estimate an unknown parameter, and the method of moment estimator equates sample moments to the theoretical moments of the population to obtain parameter estimates. Several examples are provided to illustrate how to derive method of moment estimators for different probability distributions.

Uploaded by

akshay
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
68 views16 pages

Mathematical Statistics (MA212M) : Lecture Slides

This document provides an overview of statistical inference and introduces key concepts like point estimation, statistics, and the method of moment estimator. It explains that the goal of statistical inference is to estimate unknown population parameters based on a random sample. A point estimator is a statistic used to estimate an unknown parameter, and the method of moment estimator equates sample moments to the theoretical moments of the population to obtain parameter estimates. Several examples are provided to illustrate how to derive method of moment estimators for different probability distributions.

Uploaded by

akshay
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 16

Mathematical Statistics (MA212M)

Lecture Slides
Lecture 23
Statistics
Introduction to Statistics

All most all field of study uses it.


News papers
Sports
Science and Technology
Social Sciences . . .
Aim and Issues

Our aim is to find information regarding some numerical


characteristic of a collection of items/persons/product. This
collection is called population.
For example, I want to know the average height of Indian.
Most of the time it is not possible to enumerate each and every
individual in the population. For example, it is very costly (in
terms of money and time) to go to every Indian and note down
their height to calculate the average.
In many other cases, it is not possible to enumerate each and
every item in the population. For example, suppose that we want
to find the average lifetime of an electronic item manufactured
by a company. Here we cannot put all the items manufactured
by the company on test and wait for failure of all the items.
Approach

Take a subset of the population based on which we try to find


out the value of the numerical characteristic. Obviously it will
not be exact, hence it is an estimate. This subset is called a
sample.
We should choose the sample such that it will be a good
representative of the population. Otherwise the estimate may
not be close (in some sense) to the original value, which we do
not know.
There are different ways of selecting sample from a population.
We will not discuss this issue here.
We will consider one such sample which is called random sample
(definition will be given).
Statistical Modeling

As the value of numerical characteristic may vary form item to


item of the population, we will model it with a random variable
and the uncertainty using a probability distribution.
Let X be a random variable (either DRV or CRV) which denotes
the numerical characteristic of the population.
There are two possibility:
1 X has a CDF F with known functional form except perhaps
some parameters. Here our aim is to find the (educated) guess
value of the parameters. This is known as parametric
inference.
2 X has a CDF F who’s functional form is unknown. This is
known as nonparametric inference.
In this course we will mainly consider parametric inference.
Parametric Inference

We will assume that the class of distribution is indexed by


parameter θ. It could be scaler or vector valued. The parameter
θ is fixed but unknown. We also assume that the possible values
of θ belongs to a set Θ. Θ is called parametric space.
Example 100: X ∼ N(µ, σ 2 ). Here µ ∈ R and σ > 0. We can
take θ = (µ, σ) and Θ = R × R+ .
Example 101: Let us assume that X denotes the height of a
large community of people and we try to model it using a normal
distribution. It is passible to assume that µ > 0. Hence Θ would
be R+ × R+ .
Aim: To decide which member of the family {Fθ : θ ∈ Θ} can
represent the CDF of X , which is equivalent to decide the value
of θ in Θ.
Random Sample

Def: The RVs X1 , X2 , . . . , Xn is said to be a random sample (RS) of


size n from the population F if X1 , X2 , . . . , Xn are independently and
identically distributed with marginal CDF F .
Remark: F may (and for inference will) depend on unknown
parameters θ.
Remark: How to model a data using RS?
Xi denotes the ith observation.
We need to make sure that the observation are taken such a way
that the value of one does not effect the others.
And hence each Xi is an observation on the same variable and
each Xi has same CDF F .
Random Sample (Cont.)

Remark: Equivalently, we will say that X1 , . . . , Xn is a RS from a


PMF/PDF f .
Remark: Joint CDF of a RS X1 , . . . , Xn is
n
Y
FX (x1 , . . . , xn ) = F (xi ).
i=1

Remark: Joint PMF/PDF of a RS X1 , . . . , Xn is


n
Y
fX (x1 , . . . , xn ) = f (xi ).
i=1
Concepts That Will Be Covered

Statistical Inference

Estimation Testing of Hypothesis


Finding guess for parameters Check if a statement about parameter is true or false

Point Estimation Interval Estimation


Finding a guess value Finding a interval in which the parameter will lie with high chance
Statistic
Def: Let X1 , . . . , Xn be a RS. Let T (x1 , . . . , xn ) be a function
having domain that includes the sample space of X1 , X2 , . . . , Xn .
Then the RV Y = T (X1 , . . . , Xn ) is called a statistic if it is not a
function of unknown parameters.
Remark: Note that our aim is to find a guess value of unknown
parameters based on a RS. Hence we are considering a function of
RS.
Remark: If the function involve any unknown parameters, we will not
be able to compute the value of the function given a realization of a
RS. Hence the function that involves unknown parameters is of no
use in this respect.
Example 102: Let X1 , . . . , Xn be a RS from a population
Pn with PDF
1
fµ, σ and µ and σ are both unknown. Then X = n i=1 Xi ,
s 2 = n1 ni=1 (Xi − X )2 are examples of statistic. However X σ−µ is not
P
a statistic.
Point Estimator

Def: In the context of estimation, a statistic is called a point


estimator.
Remark: In the above definition, we do not mention about the
parameter that is to be estimated.
Remark: No mention of the range of the estimator.
Remark: However, in practice we need to take care of the parameter
to be estimated and its’ range. For example, to estimate population
variance, we should not use an estimator that can be negative.
Remark: Estimator is a function of RS, and estimate is the realized
value of an estimator.
Findind Point Estimator

There are several methods to find an estimator.


We will consider three of them:
Method of moment estimator
Maximum likelihood estimator
Least square estimator
Method of Moment Estimator (MME)

Introduced by Karl Pearson in the year 1902.


The method is as follows:
1 Suppose that we have a RS of size n form a popupation with
PMF/PDF f (x; θ), where θ = (θ1 , . . . , θk ) is the unknown
parameter.
2 Calculate first k (no. of unknown parameters) moments
µ01 , . . . , µ0k of f (x; θ).
3 Calculate first k sample moments m10 , . . . , mk0 . Here mr0 is
n
define by mr0 = n i=1 Xir .
1 P
4 Equate µ0r = mr0 for r = 1, 2, . . . , k.
5 Solve the system of k equations (if they are consistent) for θi ’s.
The solutions are the MMEs of the unknown parameters.
Examples

i.i.d.
Example 103: Let X1 , X2 , . . . , Xn ∼ Bernoulli(θ), θ ∈ [0, 1] = Θ.
Here we have one parameter θ, with k = 1. E (X1 ) = θ. Hence we
get the MME of θ is θb = X .
i.i.d.
Example 104: Let X1 , X2 , . . . , Xn ∼ N(µ, σ 2 ),
θ = (µ, σ 2 ) ∈ R × R+ = Θ.
Here k = 2, E (X ) = µ, and E (X 2 ) = σ 2 + µ2 . Hence we get the
MMEs of µ and σ 2 are µb = X and σb2 = n1 ni=1 (Xi − X )2 ,
P
respectively.
Examples

i.i.d.
Example 105: Let X1 , X2 , . . . , Xn ∼ N(0, σ 2 ), σ > 0.
Here k = 1. However, as E (X ) = 0, equating E (X ) = X does not
provide any solution (in consistent). Alternatively we can find
E (X 2 ) = σ 2 and equate to m20 to obtain MME of σ 2 as
σb2 = n1 ni=1 Xi2 .
P

i.i.d.
Example 106: Let X1 , X2 , . . . , Xn ∼ N(θ, θ2 ), θ > 0.
Here k = 1. E (X ) = θ. Equating E (X ) = X , we get MME of θ is
θb = X . However, this may not be a meanful estimator as X can be
negative with positive probability, while θ > 0.
Remark: Previous two examples show that there are some degree of
arbitrariness in this method.

You might also like