Statistics and Their Distributions

Statistical Methods in Economics-II

Lesson: Statistics and Their Distributions
Lesson Developer: Chandra Goswami
College/Department: Department of Economics,
Dyal Singh College(M), University of Delhi

Institute of Lifelong Learning, University of Delhi 1

 Statistics and Their Distributions

TABLE OF CONTENTS
Section No. and Heading Page No.

Learning Objectives 2

1 Sample Statistic 2
1.1 Sampling With and Without Replacement 3
1.2 Sample Statistic and Sampling Distribution 4
2 Sampling Techniques 5
2.1 Simple Random Sampling 5
2.2 Stratified Random Sampling 6
2.3 Two- Stage or Multi-Stage Random Sampling 6
2.4 Systematic Random Sampling 7
2.5 Purposive Sampling 7
2.6 Cluster Sampling 7
3 Sampling and Non-sampling Errors 8
4 Deriving a Sampling Distribution 9
5 Analytical Methods for Deriving a Sampling Distribution 10
5.1 Using Probability Rules 10
5.2 Simulation Experiments 11
6 Distribution of the Sample Mean 12
7 Distribution of Sample Means when Population is Normally Distributed 14
8 Central Limit Theorem 16
9 Distribution of Sample Means when the Population is Non-Normal 17
10 Distribution of the Sum and Difference of Sample Means 18

Practice Questions 20

References:
1. Jay L. Devore, Probability and Statistics for Engineering and the Sciences,
8th edition, Cengage Learning
2. Irwin Miller and Marylees Miller, Mathematical Statistics, Seventh Edition,
Pearson.

Institute of Lifelong Learning, University of Delhi 2

 Statistics and Their Distributions

3. A.L.Nagar and R.K.Das, Basic Statistics, Second Edition, Oxford University

Press

Content developer: Chandra Goswami, Dyal Singh College (M)

STATISTICS AND THEIR DISTRIBUTIONS

Learning Objectives:
In this chapter you will learn what is a sample statistic and its sampling
distribution. You will learn how to derive the probability distribution of a sample
statistic and the three alternative methods that can be used for this purpose. The
first method is based on selecting samples from the population. You will learn
about different methods for selecting a representative sample and the difference
between sampling and non-sampling errors. You will study in depth about the
probability distribution of sample means. You will learn about the significance of
the Central Limit Theorem in this context. The chapter ends with the study of
distribution of combinations of two sample means. The chapter is followed by
practice questions so that you can test your understanding of the chapter
contents.

Chapter Outline
1. Sample Statistic
2. Sampling Techniques
3. Sampling and Non-sampling Errors
4. Deriving a Sampling Distribution
5. Analytical Methods for Deriving a Sampling Distribution
6. Distribution of the Sample Mean
7. Distribution of Sample Means when Population is Normally Distributed
8. Central Limit Theorem
9. Distribution of Sample Means when the Population is Non-Normal
10. Distribution of the Sum and Difference of Sample Means

1 SAMPLE STATISTIC
Measures which describe some characteristics of the population are known as
parameters. Examples of population parameters are population mean μ,
~ , etc.
population standard deviation σ, population ratio p, population median 

Institute of Lifelong Learning, University of Delhi 3

 Statistics and Their Distributions

These are constants for a population and remain unknown in the absence of
complete population census data.

In many situations, a population census may be impractical and very costly in

terms of time and money. Then we select a representative sample from the
population and use sample data to make inferences about the unknown
population parameters.

1.1 Sampling With and Without Replacement

A sample is said to be with replacement if the process of sampling is such that
after each unit is drawn from the population its value is observed and then the
unit is returned to the population. This means that the population remains
unchanged before every subsequent draw and each population unit has equal
probability of being selected on every draw.

Suppose the population to be sampled is of size N. If a sample of size n is to be

selected from the population, then each of the sample elements can be selected
in N different ways. So the first sample unit can be drawn in N different ways, the
second sample unit can also be selected in N different ways and so on for the n
elements of the sample. In sampling with replacement, since each element of the
population has the same probability of being selected on each draw, the total
number of possible samples is Nn. Now each of the Nn possible samples is equally
1
likely. Hence, the probability of selection of any one sample is . Theoretically,
Nn
an infinitely large number of samples can be drawn from the population if
sampling is with replacement as the population is not exhausted.

A sample is said to be without replacement if the process of sampling is such

that the population units selected for the sample are not returned to the
population. As each unit is selected it is withdrawn from the population so that
the size and composition of the population changes after each draw. An item once
selected cannot be repeated in subsequent draws. The probability of any
population unit being selected in the sample is thus not independent of the results
of previous draws. This is particularly the case of sampling without replacement
from a finite population where the population size is not very large compared to
the sample size.

Institute of Lifelong Learning, University of Delhi 4

 Statistics and Their Distributions

From a population of size N, the first unit can be drawn in N ways, the second
unit in N-1 ways and so on, when sampling is without replacement. Since the
order in which the sample units are selected is not relevant, the total number of

N N!
possible samples of size n from a population of size N is   
 n  n!N  n !

When the population from which the sample is drawn is very large in relation to
sample size, ie, when n < 0.05N, then for all practical purposes we can consider
the population to be infinitely large. If population is infinitely large then number
of samples that can be drawn from the population is also infinitely large,
irrespective of whether sampling is with or without replacement. It is only when
population is finite, sample size n > 0.05N, and sampling is without replacement

N
that the total number of samples will be   and probability of selection of any

n
one of the equally likely samples is 1 .
N
 
n

1.2 Sample Statistic and Sampling Distribution

A sample statistic or estimator is a function of sample values. It is a procedure
by which sample values are combined. The sample statistic is used to estimate
the population parameter. The numerical value of the sample statistic or
estimator of a given sample gives us the estimate of the value of the population
parameter. This numerical value of the sample statistic is called the estimate.

Several different functions of sample values could be used to obtain the estimate
of the parameter value. Examples of estimators for population mean μ are the
~
sample mean X , the trimmed mean X tr , the median X , or some weighted
average of sample values.

Consider selecting two different samples of the same size n (x1, x2,……xn) from
the same population. It is very unlikely that all the sample values of the first
sample will be repeated in the second sample. Since a sample is only a small
subset of the population and a large number of samples of the same size can be
drawn from the same population, the sample values will be likely to differ from
one sample to the other sample.

Institute of Lifelong Learning, University of Delhi 5

 Statistics and Their Distributions

Before we obtain sample data, there is uncertainty about the value of each Xi,
since each sample element can be any one of the population units. Because of
this uncertainty, each observation is a random variable Xi before the data
becomes available.

Since sample observations are random variables, the value of any function of the

sample observations (eg, sample mean X , sample variance S2, etc) is also a
random variable which varies from sample to sample. There is uncertainty about

the value of X , the value of S2, and so on prior to obtaining the sample
observations. The value of the sample statistic will depend on which sample was
selected and the parameter estimate would differ accordingly.

Hence a statistic or estimator is a random variable and has a probability

distribution. Suppose the process of drawing samples of size n from a given
population could be repeated indefinitely and we calculated the value of the
statistic for each of these samples. We would then generate a distribution of all
possible values of the statistic, ie its probability distribution. This theoretical
probability distribution of a statistic is referred to as its sampling distribution.
The sampling distribution describes how the statistic varies in value across all
possible samples that might be selected from a population with the respective
probabilities. Standard deviation of the sampling distribution is called the
standard error of the statistic.

The sampling distribution depends on the population distribution (eg. normal,

uniform etc), the sample size n and the method of sampling.

2 SAMPLING TECHNIQUES
Samples selected from a population must be representative of the population. If a
sample is unrepresentative of the population and the sample statistic is used to
estimate the corresponding parameter value, then this will result in an inaccurate
estimate. If the selected sample contains a disproportionately large number of
units from one end of the population distribution then the sample statistic will
provide an underestimate or overestimate of the parameter value. For example, if
the sample observations are atypically larger than most of the population values,
then the sample mean will be an overestimate of the population mean.

The technique used in the collection of sample data should be such that it
minimizes the possibility of such errors. There exist a number of alternative

Institute of Lifelong Learning, University of Delhi 6

 Statistics and Their Distributions

sampling methods for obtaining representative samples. The choice of sampling

technique adopted for a particular study will depend on the purpose of the study
and the kind of population being sampled.

2.1 Simple Random Sampling

Simple random sampling method is often used to select samples from the
population. The popularity of this sampling method is due to the fact that
sampling distribution of any statistic can be more easily obtained for simple
random samples than for samples obtained by using any other sampling method.
Sample units are selected by lottery method or by using random number tables.

Simple random sampling ensures that each unit of the population gets an equal
chance of being selected in the sample. Since several different samples can be
selected from any population, this method ensures that each sample of the same
size has the same probability of being selected. This method is useful for
homogeneous populations where there are no extreme values. In case of
homogeneous populations atypical observations are unlikely in the selected
sample and the estimate is unlikely to be biased.

The random variables X1, X2,…….Xn are said to form a simple random sample of
size n if the following two conditions are satisfied:
1. The Xi’s are independent random variables
2. Every Xi has the same probability distribution.
In a simple random sample (SRS), each unit of the sample is then said to be
independently and identically distributed (iid).

If sampling is with replacement from a finite population or from an infinitely large

population (with or without replacement) then these conditions are satisfied
exactly. If sampling is without replacement from a finite population then these
conditions will be approximately satisfied provided the sample is small relative to
the population size. In practice, we can proceed as if Xi’s form a random sample if
at most 5% of the population is sampled.

2.2 Stratified Random Sampling

Stratified random sampling is suitable when the population is heterogeneous but
can be subdivided into homogeneous subgroups or strata. Random sampling
method is used to select units from each subgroup or stratum in such a way that
pattern of distribution of units from different subgroups in the sample is the same

Institute of Lifelong Learning, University of Delhi 7

 Statistics and Their Distributions

as that found in the population. The proportions in the sample from each
subgroup conform to the proportions in the population. However, more
information is required about the population in this method than in SRS.

2.3 Two- Stage or Multi-Stage Random Sampling

This method is useful when the population is infinitely large. For example, this
method is very useful in crop-cutting experiments where the objective is to
estimate the average yield of crops. All the districts where the crop is grown are
listed and some districts are selected at random. Then some villages from the
selected districts are chosen at random. From the selected villages some fields or
plots are selected at random. The average yield of the crop in these fields is then
examined. In this example the experiment is conducted in three stages (district
level, village level, and field level). At each stage random sampling method is
used to go to the next stage. So the study involves three stage random sampling.

2.4 Systematic Random Sampling

This is another example of how random sampling method is used in a specific
manner. The population is first listed or ordered in a random fashion. If it is
decided to select every kth unit from the population then the first sample unit is
selected at random from the first k units of the population. Then every k th unit
after that is chosen in a systematic manner. Thus if the first unit is the jth
population unit, the second unit will be the (j+k) th unit, followed by every kth unit
after that. Hence the first unit is selected at random from a randomly arranged
population, followed by systematically selecting every kth unit after that. The
method requires that that there is no pattern in the initial ordering of the
population to prevent bias in the selection of sample units.

This method is often used in estimating the timber available in a forest. A tree is
selected at random and then a direction is selected at random. Every i th tree in
the selected direction, starting from the first tree, is then examined.

2.5 Purposive Sampling

The population is first restricted according to the purpose of the study. The
sample units are then drawn using simple random sampling from this specified
subset of the population. For example, if the purpose of the study is to study the
effects of inflation on the nutrition levels of the poor, then we restrict the
population to only those families which can be classified as poor. Then a random
sample of poor families is selected from this subset of the population.

Institute of Lifelong Learning, University of Delhi 8

 Statistics and Their Distributions

2.6 Cluster Sampling

The population is divided into clusters or groups. A random sample of clusters is
then chosen. All the observations in the selected clusters are included in the
sample. If the purpose of study is to examine crop yield, the villages may be each
considered to be a cluster of farms. A random sample of these clusters is
selected. All the farms in the selected clusters make up the sample units. The
crop yield of all the farms belonging to the selected clusters is then studied. This
method may however lead to biased results if an unusually large (or small)
proportion of farms in the selected sample use correct fertilizers and irrigation to
increase crop yields.

We see from the listed sampling techniques that there is no unique method for
obtaining a representative sample from a given population. Other sampling
methods exist which combine features of more than one of the above methods.
One such example is Stratified Cluster Sampling. The method adopted when
selecting a sample will depend on the nature of the population, purpose of study,
along with time and expenditure constraints.

3 SAMPLING AND NON-SAMPLING ERRORS

When there is a population census all population units are considered in
calculation of the parameter value. In a sample survey. since only a small subset
of the population is studied, the value of the sample statistic used to infer about
the parameter may differ from the value obtained from census data. The results
will also vary from sample to sample even though the samples are all of the same
size and are highly representative of the population. The difference between the
true parameter value and the value of sample statistic is called the sampling
error. Variation of the value of the statistic from different samples is known as
sampling fluctuation.

It is a matter of pure chance which sample is selected. Hence sampling errors are
due to chance factors. Sampling errors are observed only in a sample survey. It is
completely absent in the census method. Factors which contribute to sampling
errors are:
1. Heterogeneity or variability of the population.
2. Bias in the estimation method if incorrect formula used for the statistic
3. Sometimes, in a properly selected sample, some of the sample units
cannot be observed and these are substituted by other units and the

Institute of Lifelong Learning, University of Delhi 9

 Statistics and Their Distributions

characteristics of the substituted units differ from the originally selected

sample units.
4. Failure to observe the data from all the selected sample units, particularly
seen when there are instances of non-response.

Non-sampling errors can occur at any stage of the planning, collection,

processing and analysis of data. These errors can be present in both census and
sample studies. Some of the factors that give rise to non-sampling errors are:
1. Faulty definition of the population or sample units
2. Personal bias of the investigator
3. Inadequately trained investigators
4. Improper coverage so that units that should have been covered are
excluded. Or, if units are included that should have been excluded.
5. Errors in compilation of data when wrong entries are made or due to
wrong calculations.
Non-sampling errors can be reduced with proper planning, training and care at
every stage of the process from collection of data to final calculations.

4 DERIVING A SAMPLING DISTRIBUTION

Suppose random samples of size 2 are selected with replacement from the
following population: 2, 6, 10
The population mean and variance are µ = 6 and σ2 = 10.667

The total number of likely samples is 32 = 9. Following is the list of samples with

 x 
n n

x
2
i i x
their respective means and variances, where x= i 1
and s2  i 1

n n 1

(x1,x2): (2,2) (2,6) (2,10) (6,2) (6,6) (6,10) (10,2) (10,6) (10,10)
x: 2 4 6 4 6 8 6 8 10

s2 : 0 8 32 8 0 8 32 8 0

Sampling distribution of x :
x : 2 4 6 8 10
P( x ) : 1/9 2/9 3/9 2/9 1/9

X = 2(1/9) + 4(2/9) + 6(3/9) + 8(2/9) + 10(1/9) =(2+8+18+16+10)/9 = 6

Institute of Lifelong Learning, University of Delhi 10

 Statistics and Their Distributions

 1 3 2 1 
Var( x ) = 4   16   36   64   100   62 
2 372 10.667
   36  5.33 
 9 9 9 9  9  9 2

Thus  X = µ and  x2 = σ2/n

We can similarly derive the distribution of sample variances and obtain the mean
and variance of the sampling distribution.

2
Sampling distribution of s :

s2 : 0 8 32
2
P( s ) : 3/9 4/9 2/9

3 4 2 96

E( s )= 0   8   32    10.667 , so that E( s )=σ2
2 2

9 9 9 9

  3  2   96 
2
4
V( s )= 0   64   1024   
2304 9216
    142.22
2

 9 9  9   9  9 81

Above example illustrates how sampling distribution of a statistic is obtained by

first listing all the equally likely probable samples. We can, however, obtain the
sampling distribution of a statistic by analytical methods, without actually drawing
all possible samples of the same size from the population. In our example the
population was very small and sample size large relative to the population so that
the number of possible samples was limited. In practice this would not be the
case, and deriving the sampling distribution by repeated sampling would be
almost impossible.

5 ANALYTICAL METHODS FOR DERIVING SAMPLING DISTRIBUTIONS

The sampling distribution can be obtained either with the help of calculations
based on probability rules or by carrying out a simulation experiment.

5.1 Using Probability Rules

Probability rules can be used to obtain the distribution of a statistic provided that
it is a “fairly simple” function of the Xis and either there are relatively few
different X values in the population or else the population distribution has a “nice”
form.

If the form of the population distribution is known then the probability of selection
of a sample unit will be the probability of its occurrence in the population. Using
this information, the probability of selection of a particular sample is the joint

Institute of Lifelong Learning, University of Delhi 11

 Statistics and Their Distributions

probability of all the sample units. If the sample units are assumed to be
independent, then the probability of selection of a particular sample is the product
of the probabilities of the sample units. This follows from the assumption that the
sample units are independent. The probability associated with a particular sample
is also the probability of its mean and its variance. The probability of a sample
mean is the same as the probability of selecting the sample for which the mean is
computed. Similarly, probability of a sample variance is equal to the probability of
selecting the sample for which the variance is computed.

In the above example probability of selection of any one of the 3 populations

units (2, 6 or 10) is 1/3 as each unit is equally likely. Provided the sample unit is
replaced in the population before the second unit is drawn, the population
remains unchanged, and the probability of selection of the second sample unit
remains unchanged at 1/3. Then the probability of selection of any sample of size
2 from the population is the joint probability (1/3 )( 1/3 ) = 1/9. This can be
verified from the fact that total number of possible samples is Nn = 32= 9 so that
probability of selection of any one of the equally likely samples is 1/9.

The value of mean and variance of each of the 9 samples is equally likely with
probability 1/9. Now we have three samples (2,10), (6,6), and (10,2) which have
same mean value 6. The probability of obtaining a mean value of 6 is the same as
the probability of selecting any one of the three samples: (2,10) or (6,6) or
(10,2). The sum of the probabilities is 3/9, which is the probability of x =6.

Thus the sampling distribution of means and variances can be derived from the
probability of selecting a sample that results in specific values of sample mean
and variance.

5.2 Simulation Experiments

Simulation experiments are performed to obtain the sampling distribution of a
statistic usually when derivation via probability rules is too difficult or complicated
to be carried out.

The following characteristics of an experiment must be specified at the start of

the experiment:
1. The statistic of interest. ( x , s, x tr etc)
2. The population distribution and its parameters. [N(μ,σ2); U(α,β) etc]
3. The sample size (n)

Institute of Lifelong Learning, University of Delhi 12

 Statistics and Their Distributions

4. The number of replications or trials of the experiment (k)

Then use a computer to obtain k different random samples, each of size n, from
the population distribution.

For each such sample, calculate the value of the statistic. From k replications we
get k samples and k calculated values of the statistic. Now construct a histogram
for the k values. The histogram gives the approximate sampling distribution of
the statistic.

The larger the number of replications (k), the better will be the approximation of
the sampling distribution. In practice, k=500 or 1000 is usually enough. Actual
sampling distribution is obtained when k→∞.

6 DISTRIBUTION OF THE SAMPLE MEAN

The sample mean X is useful as it can be used to draw conclusions about the
population mean μ. Some of the most frequently used inferential procedures are

based on the properties of the sampling distribution of X .

Let X1, X2,….Xn be a random sample from a population with mean μ and standard
deviation σ. Since this is a random sample, the Xi’s are independently and
identically distributed.

Since the sample units are drawn at random from the same parent population
distribution, with mean μ and variance σ2, each observation Xi is independently
and identically distributed with mean μ and variance σ2.

Since any unit of the population could have been selected and any unit could take
any of the population values with respective probabilities, hence E(Xi) = µ and
V(Xi) = σ2. This is because any population unit could be selected in the sample.
So every sample unit could be any one of the population values. Given the
population has a distribution with different populations values associated with
some probability, every sample unit has the same probability distribution as the
population.

n
X = X
i 1
i n is a linear combination of n independent random variables Xi, each

having equal weight of 1/n. Then,

Institute of Lifelong Learning, University of Delhi 13

 Statistics and Their Distributions

n n
E( X ) = E   Xi n  = 1  E  X i  = 1
. n.µ = µ  =µ
i 1 n i 1 n X

And,
n n
2 
V( X ) = V   Xi n = 1
 V  X i  =
1
.n. σ2
=  =
i 1 n2 i 1 n2 n X
n
Thus, the mean of sampling distribution is independent of sample size but the

standard deviation of X (or the standard error of the mean),  , is inversely

X

related to the sample size .

Fig 1 Population distribution and distribution of sample means

As the sample size increases we obtain more information from the sample and we
can expect the value of sample mean to be closer to the population mean value.
As the sample size increases the sampling distribution becomes narrower and the
sample means become clustered closer to μ. In the limit as sample size increases
indefinitely the sampling distribution collapses to a single point. Each and every

sample mean will be equal to the population mean, ie, X = → 0 as n →∞
n
As long as n>1,    X . The reason being that for each sample, the sample
X

mean ( X ) averages out the variability of the observations within the sample.

The sample mean is a central value for each sample. Although the value of the
sample mean is affected by all sample values, by its very nature the mean value
must lie somewhere in the middle of the range of sample values. This is true for
each and every possible sample drawn from the population. Thus the variability in
values of sample means must be less than the variability in the population values

Institute of Lifelong Learning, University of Delhi 14

 Statistics and Their Distributions

where all the population units have to be considered in calculation of population

variance.

Exercise 1 For random samples from an infinite population, what happens to

the standard error of the mean if sample size is
(a) increased from 25 to 100;
(b) decreased from 400 to 100?

Solution Since X = increasing n reduces standard error and
n
decreasing n increases the standard error
   
(a) = and = . Therefore  is halved
25 5 100 10 X

when sample size is four times the previous size

   
(b) = and = . Therefore  is doubled
400 20 100 10 X

when sample size is one-fourth the previous size

7 DISTRIBUTION OF SAMPLE MEANS WHEN POPULATION IS

NORMALLY DISTRIBUTED
Let the population be infinitely large or it is finite and sampling is with
replacement. Suppose the population is normally distributed with mean μ and
variance σ2. Then each of the observations Xi of the random sample of size n will
also be normally distributed with mean μ and variance σ2. The sample mean X is
a linear combination of n independent, normal variables, each having an equal
1
weight of , so that X also has a normal distribution with mean µ and variance
n
σ2/n.

Since X =  n , the standard deviation of the sampling distribution of means

is inversely related to sample size n. As n increases,  X decreases. For larger

samples, the distribution of sample means will be clustered closer to µ. For

smaller sample sizes the sampling distribution is more spread out. See figure 2.

Institute of Lifelong Learning, University of Delhi 15

 Statistics and Their Distributions

Fig 2 Distributions of sample means where n1< n2 < n3

If sampling is without replacement from a finite population, then we must apply
the finite population correction factor. The sample means would be normally

2 N n
distributed with mean µ and variance  . However, if the sample
n N 1
N n  2 when
→ 1 and  x →
2
size is very small relative to population size,
N 1 n
n<0.05N

If population is normally distributed then sampling distribution of means

will also be normally distributed, irrespective of sample size, ie, for any n

Exercise 2 Suppose annual percentage salary increases of CEO’s of all midsize

corporations are normally distributed with μ=12.2% and σ=3.6%.
A random sample of 9 observations is obtained from this
population and the sample mean computed. What is the probability
that X will be less than 10%?
Solution Since population is normally distributed the sampling distribution of

X will also be normal with  X = µ=12.2 and  X =3.6/√9 =1.2

 10  12.2 
P( X <10) = P  Z     1.83 = 0.0336
 1.2 

Exercise 3 Marks obtained by 3000 students in an admission test are

normally distributed with μ=68 and σ=3. If 80 random samples of
25 students each are obtained, what would be  X and  X if
sampling is done (a) with replacement? (b) without replacement?
In how many samples would you find X >69?
Solution Whether sampling is with or without replacement,  X = μ=68

Institute of Lifelong Learning, University of Delhi 16

 Statistics and Their Distributions

3
If sampling (a) with replacement then X   0.6 and
25

3 3000  25
(b) without replacement then X   0.6
25 3000  1
[We obtain similar results for (a) and (b) since n = 0.0083N]

 69  68 
P( X >69)= 1- P( X <69)= 1- P  Z    1  1.67 
 0.6 
= 1-0.9525 = 0.0475
Number of samples with X >69 = (80)(0.0475)=3.8 4

8 CENTRAL LIMIT THEOREM

The Central Limit Theorem (CLT) is one of the most important theorems in
Statistics. It often forms the basis for estimation of population mean and tests of
means. The central limit theorem applies only to the sample mean and to no
other sample statistic.

The central limit theorem states that when an infinite number of successive
random samples of the same size, n, are taken from a given population with
mean μ and variance σ2, the distribution of sample means X will be

approximately normally distributed with mean μ and standard deviation ,
n
provided n is sufficiently large, irrespective of the shape of the population
distribution.

The larger the value of n, better is the approximation. Even when the population
distribution is highly non-normal, averaging of sample values while computing
X produces a distribution more bell-shaped than the population itself. If n is
large, a suitable normal curve will approximate the actual distribution of X . That is
why sampling distribution of X is said to be asymptotically normal. This is
illustrated in figure 2.

Institute of Lifelong Learning, University of Delhi 17

 Statistics and Their Distributions

Fig 3 The Central Limit Theorem; n1 < n2

The red curve in figure 2 is the positively skewed population distribution. The
green and blue curves depict two distributions of sample means for different
sample sizes where n1 < n2. The distribution of X with sample size n1 is less
skewed than the distribution of the rv X. As sample size is increased suitably to
n2, the distribution of X is approximately normal.

How large must be sample size will depend on how much is the departure of the
shape of the population being sampled from a normal distribution. In many cases
the sampling distribution quickly approaches a normal distribution, as in case the
population has a uniform distribution where sample size of 12 is sufficient. In
some other cases sample size of 60 or more may be required. There is no hard
and fast rule about the sample size required for the sampling distribution of
means to be normally distributed. In practice, quite satisfactory approximations
can be obtained for n > 30, provided N > 2n where N is population size.

The central limit theorem is applicable whether population variable is discrete or

continuous.

CLT plays an important role in estimation and tests of hypotheses about the mean
as the probability distribution of the population being sampled is often not known.
The central limit theorem enables us to use the normal distribution as an
approximation of the distribution of sample means.

9 DISTRIBUTION OF SAMPLE MEANS WHEN POPULATION

DISTRIBUTION IS NON-NORMAL
The Central Limit Theorem extends the results for the normally distributed
population to the general case of non-normal populations. If the population

Institute of Lifelong Learning, University of Delhi 18

 Statistics and Their Distributions

distribution is not known or if the population distribution is non-normal and

sample size is sufficiently large, the distribution of sample means is
approximately normal.

If population standard deviation σ is known then sampling distribution of means

will be approximately normal with mean μ and variance σ 2/n for n > 30. If σ is
unknown and is estimated by sample standard deviation S, then a larger sample
size is required. The distribution of X will then be approximately normal with
mean μ and variance S2/n where n ≥ 40.

Therefore, if the population being sampled is not known or it is known to be not

normally distributed, the CLT provides a short-cut. When the sample size is large,
we can treat the distribution of X to be approximately normal.

If population distribution is not normal then sampling distribution

of means will be approximately normal if n> 30 when σ2 is known,
and, if n > 40 when σ2 is not known and is estimated by sample S2

Exercise 4 A random sample of size n=100 is taken from an infinite population

with the mean μ=75 and σ2=256. With what probability can we
assert that the value of X will fall between 67 and 83?
Solution The form of the population is unknown but σ is known and n>30.
According to the central limit theorem the sampling distribution of
X will be approximately normal with E( X )=75 and  X = 1.6

Then P(67< X <83)  P  67  75  Z  83  75  = 5   5 =1

 1.6 1.6 

Exercise 5 A random sample of size n=200 is taken from a uniform population

with α=24 and β=48. What is the probability the mean of the
sample will be less than 35?
 
Solution Mean of the continuous uniform distribution is  36 and
2


   2  6.928 . Based on the CLT, X will be approximately
12
normal with E( X )=36 and  =0.49.
X

P( X <35)  P Z  35  36    2.04 =0.0207

 0.49 

Institute of Lifelong Learning, University of Delhi 19

 Statistics and Their Distributions

10 DISTRIBUTION OF THE SUM AND DIFFERENCE OF SAMPLE MEANS

Let X 1 be the mean of a random sample of size n1 drawn from a population with

mean μ1 and variance  12 where the population is infinite or sampling is with

replacement from a finite population.

Let X 2 be the mean of a random sample of size n2 drawn from a population with

mean μ2 and variance  22 where the population is infinite or sampling is with

replacement from a finite population.

If the samples are independent then we know from rules of mathematical

expectation that
E( X 1 + X 2 ) = E( X 1 )+E( X 2 ) = μ1 + μ2 and E( X 1 + X 2 ) = E( X 1 )+E( X 2 ) = μ1 + μ2

V( X 1 + X 2 ) = V( X 1 )+V( X 2 ) =  1 +  2 and V( X 1 - X 2 ) = V( X 1 )+V( X 2 ) =  1 +  2

2 2 2 2

n1 n2 n1 n2

If the populations are normally distributed then ( X 1 + X 2 ) will also be normally

distributed with mean μ1 + μ2 and standard deviation  12  22 irrespective of the


n1 n2

sample sizes.

If populations are not known to be normal and sample sizes are sufficiently large
(n1 > 30, n2 > 30) then by CLT the distribution of ( X 1 + X 2 ) will be approximately

normal with mean μ1 + μ2 and standard deviation  12  22 .


n1 n2

If population variances  12 and  22 are not known and are estimated by sample

variances S i2 and S 2 respectively, then by CLT, if n1 > 40, n2 > 40, the
2

distribution of ( X 1 + X 2 ) will be approximately normal with mean μ1 + μ2 and

standard deviation S 12 S 22

n1 n 2

If the populations are finite of size N1 and N2, and the samples from the two
populations are drawn without replacement, then the finite population correction
factor must be applied to the variance. If the population variances are known and

Institute of Lifelong Learning, University of Delhi 20

 Statistics and Their Distributions

the populations are normal then the sampling distribution of ( X 1 + X 2 ) will be

normal with mean μ1 + μ2 and standard deviation  12  N 1  n1   22  N 2  n 2 

   
n1  N 1  1  n 2  N 2  1 

If the populations are not normal or population variances are unknown and
estimated from sample data, then the sample sizes must be large enough.

If both random samples are independent and from the same population so that
μ1= μ2= μ and  12 =  22 = σ2, then E( X 1 + X 2 ) = 2μ and E( X 1 - X 2 ) = 0. The standard

deviation of the distribution of ( X 1 + X 2 ) is 

1 1 . If population variance is

n1 n2

unknown then it is estimated by a weighted average of sample variances. Thus,

 2
 
n1  1S12  n2  1S 22
, where information available in both samples are
n1  n2  2
used.

Exercise 6 The average lifetimes of Duracell Alkaline AA batteries and

Eveready Energizer Alkaline AA batteries are given as 4.1 hours
and 4.5 hours, respectively. The standard deviations of lifetimes
are 1.8 hours for Duracell batteries and 2.0 hours for Eveready
batteries. Random samples of 100 new batteries of each brand are
selected. What is the probability that the sample mean lifetime of
Duracell exceeds the sample mean lifetime of Eveready batteries?

Solution Let mean lifetime of samples of Duracell batteries be X hours and

mean lifetime of samples of Eveready batteries be Y hours.
 X   X  4.1 hours and Y  Y  4.5 hours   X  Y  0.4 hours

 X2 
1.82  0.0324 &  Y2 
22  0.04   X Y  0.0724  0.269
100 100
Since sample sizes are large, by CLT, the distribution of ( X - Y ) is
approximately normal with mean  X Y   X  Y  0.4 hours and

 X Y  0.269 hours. Now, P( X > Y )= P[( X - Y )>0] = 1- P[( X - Y )<0].

Therefore, required probability is: 1- P[( X - Y )<0]

 0   0.4
= 1- P Z 
  1  1.487  = 1- 0.9319=0.0681
 0.269 

Institute of Lifelong Learning, University of Delhi 21

 Statistics and Their Distributions

References:
1. Jay L. Devore, Probability and Statistics for Engineering and the Sciences,
8th edition, Cengage Learning
2. Irwin Miller and Marylees Miller, Mathematical Statistics, Seventh Edition,
Pearson.
3. A.L.Nagar and R.K.Das, Basic Statistics, Second Edition, Oxford University
Press

PRACTICE QUESTIONS

1. Given the population: 27 32 17 21 32

(a) Calculate μ and σ
(b) List the random samples of size n=2
(c) Derive the sampling distributions of mean and variance
(d) Compare the mean and variance of the sampling distributions with
the parameter values.

2. For the population given in question 1,

(i) Repeat all the parts (b) to (d) for n=3
(ii) How do the results vary from the ones obtained for n=2? Why?

3. State and explain the significance of the Central Limit Theorem

4. What properties of the SRS help in deriving the sampling distribution of means?
OR
Why are random samples commonly used to obtain estimates of unknown
parameters?

5. If X1, X2, X3 is a simple random sample of size three from a large

population with mean 5 and 4, evaluate the expected value and std error
of the statistic T = (2X1 + X2 – 3X3)

6. Distinguish between sampling with replacement and sampling without

replacement and explain the effect of method chosen on estimation of
population mean in each case

7. Let random samples of size 25 be drawn from a population that is

normally distributed with mean 2800 and standard deviation 5000. What is

Institute of Lifelong Learning, University of Delhi 22

 Statistics and Their Distributions

the mean and standard deviation of the distribution of sample means?

How would your answers be affected if sample size was 100?

8. An equipment manufacturer requires steel bars which must have a mean

length of at least 50 inches. The company can purchase bars from supplier
A whose bars have an average length of 47 inches with a standard
deviation of 12 inches, or from supplier B whose bars average 49 inches
with a standard deviation of 3.6 inches. If the manufacturer plans to buy
81 steel bars, which supplier, A or B, should be chosen?

9. The standard deviation of the amount of time it takes a college student to

complete a project is 40 minutes. A random of sample of 64 students is
taken. What is the probability the sample mean will be more than 8
minutes less than the population mean?

10. A random sample of size 81 is taken from an infinite population with the
mean  = 128 and the standard deviation σ = 6.3. With what probability

can we assert that the value we obtain for the sample mean X will not fall
between 126.6 and 129.4?

11. The mean production level at a firm is assumed to be 47.3 units per day
with a standard deviation of 12.7. The manager takes a sample of output
for 25 days. If the sample mean exceeds 49 units then the workers are
promised a Diwali bonus. How likely are the employees to get the bonus?
What assumption did you make, if any?

12. Independent random samples of size 400 are taken from each of two
populations having equal means and std deviations σ1 = 20 and σ2 = 30.
What can we assert with a probability of 0.99 about the value of the
difference in sample means?

13. What does standard error of a statistic measure? If a random sample of

size 25 is taken from a population distributed normally with mean 500 and
standard deviation 35, estimate the standard error of the sample mean.
Would you prefer a sample of size 49? Explain.

Institute of Lifelong Learning, University of Delhi 23

 Statistics and Their Distributions

14. Given that test scores in an entrance examination are normally distributed
with mean of 30 and a std. dev. of 6
(i) What is the probability that a single score drawn at random will be
greater than 34?
(ii) What is the probability that a sample of 9 scores will have a mean
greater than 34?
(iii) Explain the difference in the results obtained in parts (i) and (ii)

Institute of Lifelong Learning, University of Delhi 24