CHAPTER

ESTIMATION

population may be unknown

Many times we deal with population of various types. The
are unknown. We always want to know the
or known where some of the parameters
parameters. But to deal with he
characteristies of the population through its diflerent

population may be dificult or impossible. In that case, we take sample fromn the
total

population todraw inference for the population. Statistical inference theory gives us the

tool to deal with population through sample. Statistical inference theory may
necessary

be divided into two major areas, such as estimation and testing of hypotheses.

In estimation, we estimateor predict the value of a parameter through sample. In testing

decision
of hypotheses, we do not estimate a parameter, but we try to take a correct
about a pre-stated hypothesis relating one or more parameters so far as acceptance or

rejection is concerned.

In this chapter we shall discuss about estimation. Testing of hypotheses shall be

discussed in the next chapter.

9.1 Estimator and Estimate

The statistic used to predict the parameter is known as estimator and the value taken by
the estimator is called estimate.

but the
For example,

population.

The
of
mean

statistic

X,,X,,...,x,
We
T
u
let

is

chose
X be a random variable known to
not known. We draw n nunmber

is called

is called
a statistic TUX,
the estimator of
an estimate of
.
.
have nornal
of sample
...,X) to estimate the unknown parameter
X,,
X,
The numerical value of T after realization
So, the estimator as
distribution

X,.... X,from the

a statistic ,
N(,

consists
.
),.

of

a set of rules to calculate an estimate.

Estimation is of two types as

1. Point estimation: Inpoint estimation, a singlevalue is calculated to estimate the

population parameter based on sample data. The statistie used to estimate is called
point estimator.

293
Many times we deal with population
of various The population may be
types.
orknown where some of the unknown
parameters are unknown. We always want to know
characteristics of the population the
through its different parameters. But to
deal with the
total population may be difficult or impossible. In that case, we take sampe from the
population to draw inference for the population.
Statistical inference theory gives
necessary tool to deal with us the
population through sample. Statistical
inference theory may
be divided into twO major
areas, such as estimation and
testing of hypotheses.
In estimation, we
estimate or predict the value of aparameter
through sample. In testing
of hypotheses, we
do not estimate a parameter, but we try to take a
correct decision
about a pre-stated hypothesis relating one or
more parameters so far as acceptance or
rejection is concermed.

In this chapter we shall discuss about estimation. Testing of hypotheses shall be

discussed in the next chapter.

9.1 Estimator and Estimate

The statistic used to predict the parameter is known as estimator and the value taken by
the estimator is called estimate.

but the
For

population.

The
example, let
mean u

statistic
We
T
is
X be a random variable

.
known to N(u ,),
have normal distribution
not known. We draw n number of sample X,, X,...,X, from the
chose a statistic T(X,, X,,....,X) to estimate the unknown parameter
is called the estimator

of X, X,, ...., X, is called an estimate of

of The numerical value
So, the estimator as a
ofT afier realization
statistic ,consists
.
of
a set of rules to calculate an estimate.

Estimation is of two types as

1. Point estimation: In point estimation, a single value is calculated to estimate the
population parameter based on sample data. The statistic used to estimate is called
point estimator.

293
2 Interval estimation: In
within which the parameter
intervalestimation,two points are estimated

is expected to lie with certain degree

to

of
form an
interval

so obtained is called confidence interval. confidence.The

interval

9.2 Point Estimaüon

In many problems, there may exist several statistics as estimators which can be lusod.

point estimates of a population parameter. To know the best out of several estimatore
we need to know about their characteristics based on their sampling distributions and

also we need to know the methods of finding the estimates.

Error of estimation: As estimation of parameter is

done using few samples, the
estimated value and the true value of the parameter
may not be same. The difference
between the estimate and the true value of the parameter is
called the eror of estiunation.

Margin of error: Considering the sample size to be large, the sampling

distribution of
the estimatorcan be approximated by a normal
as per central
As per normal distribution,

between the point estimator and true value

distribution

95% of observations wil lie

of the parameter
within
will
± (1.96)o. The
be
limit theorem.

difference

less than 1.96 times

the standard error and greater than -1.96 times standard error. This
quantity is called
the 95% margin of error. This margin of error provides an upper and lower
bound for
the error of estimation.

For estimation of population mean using a large sample (N> 30), the standard error is

S
SE = where s is the sample standard deviation

The 95% margin of error is estimated as

1.96|

For population proportion of a binomial population, if pis

estimating
the sample
proportion for large sample, then the standard error is

SE = where q=l-p
n

Similarly, we can have other percentage margin of error.

Example 9.1

It is required to know the proportionof people interested to purchase one particular brand
of air conditioner. For the purpose, a random sample of 200 people is taken and it 1s
found that 30% of them show their interest for that particular brand of air conditioner.
Find the 95% margin of erTor for the estimate and what conclusion can be made about
populationproportion.
 ESTIMATION 295
CHAPTER9:

9Solution

p be the sample proportion which approximate the population proportion p.

Let

p= 0.3

(O3)(0.7) =0.0635
Margin of error =1.
200

we can conclude that the estimate of 0.3 is within 0.0635 of the true value of
So.

population proportion. Hence, we conclude that the lower bound of p is 0.3 - 0.0635
= 0.2365 and the upper bound is 0.3 + 0.0635 = 0.3635.

9.2.1. Characteristics of estimators

The followings four characteristics are mainly used to determine a good estimator.

(ay Unbiasedness
Consistency

(c) Effñciency

(d) Sufficiency

from
Unbiased estimator Let X,, Xy,...,X, be n samples drawn
a population
(a)

having parameter 0. The statistic

TX, X,....,X) is an unbiased estimator of 0, if E(T) =0

.
Example 9.2

X, X...., X, are n samples drawn from a population having mean u and variance
If

o,then show that

s² =(x-X
n-li=
is an unbiased estimator of

9 Solution

Let X be the sample mean. So ; =- -and for each X, o =o

n-lel

7-2(X, -u) -(-))

-Ž(X,--2-MXX,
n-li=l
-)-(R-')
 A TEXTBOOK ON PROBABILITYAND
STISTE

296
CHAPTER9:

n-1

n-lel
Let
1

Hence,
E(S') = EX,-9-E(R-)} So, Z

ng
n-1

Since, expectation of

population variance

Example
If T is
9.3
. S

an unbiased estimator of parameter

is the population

0,then show
variance; S

that T²
isan

is a
unbiased estimator

biased estimator of
of

O.

Solution So.

Since T is an unbiased estimator of 0, we have E(T) = dnu

()
Var() =E(T) -(E(T)² = E(T) -e2 TE

As Var(T)>0, E(T) > 02 Out

estim

Hence, T² is a biased estimator of 02.

Consistent estimator med

the
The statistic T(X,, X, ...., X)of a random sample of size n, is said to be a consisteu
estimator of parameter 0, if T converges to in probability. In other words we sa

lim P(T
I-o -d<e)=1 of
of
Example 9,4
Show that in a random sample of mean
is

a gonsistent
size n from a normal
population, thesample E
estimator of
population mean. L
Solution

Let X, X, be a sample of size n drawn from a normal

and variance o. population
 n j=l n

X- n(-)
Let Z=

So. Z becomes a standard normal variate.

lim
n P -<0= lin no0
P(Z|<

evn

=lim

2 dz

V27=1
2
So, sample mean X is a consistent estimator of population mean u.

(c) Eficient estimator

There may be many statistics which can be the estimators of a population parameter.
these estimators,the estimator having smallest variance is called the efficient
Out of all

estimator.

For example, in sampling from a normal population, both sampe mean and sample
estimator as
median are estimators of population mean. But sample mean is an efficient
the variance of sample mean is less in comparison to the
variance of sample median.

(d) Sufficient estimator

Let the T(X,, X,,....,X) based on the samples

statistic X, X,....X, be an estimator
of the parameter 0. Then T is a sufficient estimator for 0 if the conditional distribution

O1 A,Xy..,X, for given T, is independent of parameter 0.

A sufficient estimator contains all information in the sample regarding the parameter.

Example 9.5
Let X, X,....,X, be a random sample of size n drawn from a Bernoulli population with

Parameter p. Show that T = X, + X, + ... +X, is a sufficient for p.

 P(X, +X, t...+X, =k)
p'at-k

(as under the condition X, +X, +...+X, =k,exactly k number of X,'s will be 1)

which is independent of p.

Sothe statistic T is suffhcient for p.iba at onia o loaD

.2Methods of estimation
There are different methods of estimation such as method of minimum variance, method

ofmoments, method ofminimum chi-square. But we will discuss only one method which

is method of maximum likelihood estimation as it s frequently used in many practical

problems.

Method of Maximum Likelihood Estimation

Let X,X,,...X, be independent random samples, drawn from a population f(x; 9)

where is the parameter. Then for the sample values x,, x,....*, the joint probability
function is called the likelihood function L.

L(0) =f( y0)

=f(r,9) fi, 0).fa, e) (as the samples are independent)
 A TEXTBOOK ON PROBABILITY AND STATISTICe

298

9Solution
we have
population,
X,is a random sample from Bernouli
As X, X,...
with probability

with probability q =1-p

parameters n and p.
distributed with
Further, T= X + X, t ... + X, is binomially

Hene, P(T

The
)
= =
conditional distribution
of X, =X, X,= Xy, .., X,=x, for given T= kis givenby
=
P(X, %,X, =X..,X, =x,T= k)
P(X = x,X, =x...X,= %, X+
X, +...+X, =k)
P(X,+X, t...+X, =k)

(as under the condition X + X, +...+ X, =k,exactly k number of X's will be l)

which is independent of p.

So the statistic T is sufficient for p.

L39.2.2 Methods of estimation

There are different methods of estimation such as method of minimum variance, method

moments, method of minimum chi-square. But we will discuss only one method which
of

ismethod of maximum likelihood estimation as it s frequently used in many practical

problems.

of Maximum Likelihood Estimation

.Method
Let X,X,, ..X be independent random samples, drawn from a population f(x; 8),

where 0 is the parameter. Then forthe sample values x,, y.. the joint probability

function is called the likelihood function L.

L(0) =f zy..*;8)
=fo,9) fr,, 0).f(, 0) (as the samples are independert)
 the unknown parameter 0, there may exit more than estimators. So,
In estimating

to maximum likelihood estimation, we find that estimator for which the likelihood

Aunction L(0) becomes maximum. Then, is called the maximum likelihood estimator

of 8.

If maximizes L(0), then for = LO)=0 and <0 for 6=0

Sometimes, it is easier to maximize log L(0) instead of L(0). If 0 maximizes L(0),

then also maximizes log L(0, as L(0) is always positive being the product of probability

functions and log L(0) is a non-decreasing function of L(0).

Example 9.6
Find maximum likelihood estimate of mean of Poisson distribution.

9Solution

If X follows Poisson

Let X, Xy
distribution with mean

be independent random samples of

,) then f(x; = x!

size n is drawn fromn the

population.

L)= x! x!
x, log2-log(x, !)]
So, logL()=[-1+x,log 1-log(x )]+...+[-l+

=-nl+(x +...+x,)log L(2)-[log(x !) +...+log(x,!)]

dlogL(2)-=-n+

logL(2) t..+ *,

dlog L(2)
-0-itt=,=
and
a logL(2) c0
<0

X the maximum likelihood estimator of mean of Poisson

Hence, sample mean is

distribution.
 A TEXTBOOK ON PROBABILITYAND
300 STATISTICS

Example 9.7
N(4, o), find the maximum
sampling from normal
population
In random likelihot

estimator for

() when o u is known (ii) g² when u is known

9 Solution
of size n be drawn fromthe
Xy.X,t be independent random
samples population,
Let X,
N(4 ,o2).

fain,d)=e
n

So,

) o-log(2r)--'
1

ett

--
logL=-log i=l

J logL
(as o is known)

logL
i=l
n

a log L

Gi)
So, sample mean

logL=-logo--log(2r)-
X is the maximum

2o
likelihood estimate of .
dlogL n
(as isknown)

OlogL =0-"
26 t-u=0 2o

njl

It can be shown that

the second order
derivativeis negative.
 Sectively.

g oróer as
u) oy y

monsfaeer

iielet
eAg
CHAPTER 9: ESTIMATION 301

So. G=X(,- is the maximum likelihood estimator of populationvariance o.

This example shows that maximum likelihood estimator need not be unbiased.

Exafnple 9.8
Let X,Xq.... .,be independent random samples of size n be drawn from the uniform
distribution, whose density function is given by

for
f(x,) =1
2
=0 otherwise

Find the maximum likelihood estimator of 0.

Solution
Let the realization of X, X,.., be x, Xz ..*respectively.

Here, L =1 for
2
=0 otherwise

Let the values x,xg....„*,are rearranged in ascending order as ),2y * o

Then
2

Thus L attains maximum if

1
and xn)
,so

8s X) tand 2n)

So, anyvahue in [x)-tlisa maximum likelibood estimator of 0.

This example shows that the maximum likelihood estimator need not be unique

9.3 Interval Estimation.

Inpoint estimation, we find a particular value as an estimate of the population parameter.

But, in many cases, it is unlikely to estimate the population parameter exactly. When a
manufacturer gives guaantee of his product, he always use point estimation. But, we
observe that the product does not work exactly up to the same guaranteed time. Some
 A TEXTBOOK ON PROBABILITY AND
302 DSTATISTICs

time and sometimes works after

that.So,in
the guaranteed

it
time it fails before
many
it is preferable to determine an interval within which we expect to
situations,
findthe
certain confidence. Such an interval is called
value the parameter with interval e

of
estimate
and the process

An interval

Theend points
of

estimate
finding

6, and
such

of a population
interval estimate

o, are estimated by an
parameter is
is called

an interval

interval
interval

estimator
of

O
estimation.

the
fom
which
,<0<,,
is a
statistic
following some probability distribution.

9.3.1 Confdence interval and degree of confidence

We know that a statistic is a random variable which takes different values for different
samples. So, the interval estimator ,as a statistic, takes different values. Hence, the end
points of the interval estimate
0,,9,, take different values corresponding to the random
variables ,,8,.
From the sampling distribution of 0,we can determine O, and ,,
such that
P(O, <0<O,,)=1-a, where 0 <a<l
Then we have a probability
of1-a of selecting a random sample that
an interval (0,,0,) containing 0.
will produce

We
(1-a) is
call the interval
(0,,0,) as 100(1 - a)% confidence interval. The
called degee ofconfidence quantity
or confidence co-efficient.
,,0,,are called lower and upper
confidence limit
When degree of confidence respectively.
increases, the
always prefer small confidence interval
confidence interval with
becomes wider. Butwe
high degree of confidence.
For example, the
90% confidence interval
is more (6,7) years for
acceptable rather than 99% average life of a battery
confidence
In the following interval (2,9) years.
sections,
procedures for finding
parameter from a single confidence interval for
sample and population
are discussed. comparisonof different
parametersfrom two
samples
9.3.2 Confidence
interval for Mean
Here two cases will
arise
known. depending on whether
population
variance is known or no
Case 1:
Population variance o'is
We known.
consider the
sampling is drawn from
sample mean X is
normal
normally population. So, the
o distributed with distribution
of
deviation mean
where n is the (population mean) and standard
sample size. Let the degree of
confidence be 1- So the
a.
 303
9: ESTIMATION
CHAPTER

that the mean

will lie in confidenceinterval is 1 -a.So, the probability of

probability interval is
outside the confidence
mean lying

the confidence interval, we use Z statistic, where Z= X-!.zis

Eor estimating
N(0, 1).
nomally distributed

Let z be the z-value, where the arcabetween the standard normal probability curve

and z-axis to the right ofzai2 iS /2 and to the left of -Zoi) is also a/2.

Hence, P-Zal2 <Z< Zal2)=l-a

or,

or,

Hence, the confidence interval for population mean for (1 -a) degree of confidence is

where, is the computed value of X

sample size should be 30 or more
If sample isdrawn from non-normal population,
Then
so that distribution of X shall be
approximately normal by central limit theorem.
as discussed before.
using Z statistic, we can find the confidence interval

Case 2: population variance o² is unknown

T to find the confidence interval.
In this case, instead of Z statistic we use statistic

The statistic T= follows -distribution with (n - 1) degrees of freedom.

SINn
100(1 - a)% confidence interval for o' is
Applying the same procedure as above, the

,I+lan

X and s the computed value of sample standard

Where, r is the computed value of is

deviation with (n - 1) degrees of freedom in the -distribution curve

S. aiy is the t-value
the right of Lo) and an area of a/2 lies to the left of-to':
Where an area of a/2 lies to

LExample 9.9
Ihe authority of Electricity Board wants to know the average consumption of electricity

Dy the houses in a particular city, In a sample of 80 houses, it is found that the average
 A
TEXTBOOK ON PROBABILITY
304 AND
STATISTICS

consumption

in the city is
is 260 units.
40 units, find
If

a
the standard

95% confidence
deviation of electricity

interval for the actual

consumption offall : houses
average
9Solution
consumption,

The data given in the problem are, =260, G =40, 1- c=0.95 or a=0.05
As the population standard deviation is known, Z statistic shall be used to find h.
confidence interval and the 95% confidence interval is

T-a2I-Za2
Vn Vn

From the normal distribution table, z, = 1.96. So the confidence interval is

40 40
260-1.96 260+; 268.77)
V80

Example 9.10
The Principal of a college wants know the average time per day a student
to
spends
in the college library. In a sample of 30 students, it is found that the
average time
5 hours with a standard deviation of 1.2 hours. Find a 99%
is

confidence interval for the

average time.

Solution
The
As
data given in
the population
the problem
variance
are,

is
= 5,

unknown,
s =
we
1.2, n
can use
=30
T statistic to determine the
confidence interval and the 99% confidence interval is

S S

where, degrees of freedom of t-distribution is n -1=29

Here, (1 -a) =0.99. So, a =0.01
From t-distribution table, for 29 degrees of freedom, ton lons= 2.756
Hence, the confidence interval is

1.2
2
|5-2.756V30 ,S-2756-
(44,s6)

9.3.3 Confidence interval for Proportion

In many situations, we want the proportion of occurrences of one particular event (or
success) out of n possible occurrences. If the random variable X denotes the number