Scribd
Upload
61 views2 pages

Mean and Variance Estimation

The sample mean is an unbiased estimator of the population mean, with an expectation value equal to the population mean and a variance that decreases as the sample size increases. The sample variance is an unbiased estimator of the population variance, with an expectation value equal to the population variance. The denominator in the sample variance calculation is N-1 rather than N to account for the constraint imposed by estimating the sample mean from the data.

Uploaded by

ahmed22gouda22
Copyright
© Attribution Non-Commercial (BY-NC)
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
61 views2 pages

Mean and Variance Estimation

The sample mean is an unbiased estimator of the population mean, with an expectation value equal to the population mean and a variance that decreases as the sample size increases. The sample variance is an unbiased estimator of the population variance, with an expectation value equal to the population variance. The denominator in the sample variance calculation is N-1 rather than N to account for the constraint imposed by estimating the sample mean from the data.

Uploaded by

ahmed22gouda22
Copyright
© Attribution Non-Commercial (BY-NC)
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 2

ACM/ESE 118 Mean and variance estimation Consider a sample x1 , . . . , xN from a random variable X .

The sample may have been obtained through N independent but statistically identical experiments. From this sample, we want to estimate the mean and variance 2 of the random variable X (i.e., we want to estimate population quantities). The sample mean N 1 xi x = N
i=1

is an estimate of the mean . The expectation value of the sample mean is the population mean, E ( x) = , and the variance of the sample mean is var( x) = 2 /N . Since the expectation value of the sample mean is the population mean, the sample mean is said to be an unbiased estimator of the population mean. And since the variance of the sample mean approaches zero as the sample size increases (i.e., uctuations of the sample mean about the population mean decay to zero with increasing sample size), the sample mean is said to be a consistent estimator of the population mean. These properties of the sample mean are a consequence of the fact that if 2, . . . , 2 , x1 , . . . , xN are mutually uncorrelated random variables with variances 1 N the variance of their sum z = x1 + + xN is
2 2 2 z = 1 + + N .

(1)

If we view the members of the sample x1 , . . . , xN as realizations of identically distributed random variables with mean E (xi ) = and variance var(xi ) = 2 , it follows by the linearity of the expectation value operation that the expectation value of the sample mean is the population mean: E ( x) = N 1 i E (xi ) = . 2 = 2 /N . The variance of the sample mean follows from (1): var( x) = N 2 i i Moreover, the Central Limit Theorem states that, under fairly general conditions, the distribution of the sample mean x approaches a normal distribution N (, 2 /N ) with mean and variance 2 /N as the sample size N increases [see, e.g., Johnson and Wichern (2002, chapter 4.5) or Papoulis (1991, chapter 8)]. The sample variance s2 = 1 N 1
N

(xi x )2
i=1

is an unbiased estimator of the population variance 2 , that is, E (s2 ) = 2 . To see

this, we calculate 1 E [(xi )( x )] = N =


N

E [(xi )(xj )]
j =1

1 E (xi )2 N 2 = , N

where we have used the assumption that the xi are mutually uncorrelated. With var( x) = E ( x )2 = 2 /N , it then follows that E (xi x )2 = E [(xi ) ( x )]2 = 2 + 2 2 2 N N N 1 2 = . N
N

Thus, E (s2 ) = 1 N 1 E (xi x )2 = 2 .


i=1

The denominator N 1 in the sample variance is necessary to ensure unbiasedness of the variance estimator. The denominator N would only be correct if uctuations about the population mean and not about the sample mean x would appear in the expression for the sample variance. With the denominator N 1, one obtains an indenite sample variance for a sample of size N = 1, as expected. With the denominator N , the sample variance would vanish, yielding an obviously incorrect estimate of the population variance. The denominator N 1 appears because, after estimation of the sample mean, only N 1 degrees of freedom are available for the estimation of the variance, since the variables x 1 , . . . , xN and the sample mean satisfy the constraint
N

(xi x ) = 0.
i=1

References Johnson, R. A., and D. W. Wichern, 2002: Applied Multivariate Statistical Analysis. 5th ed., Prentice-Hall, 767 pp. Papoulis, A., 1991: Probability, Random Variables, and Stochastic Processes. 3rd ed., McGraw Hill, 666 pp. 2

You might also like