Test of Hypothesis

Random Sample
A simple random sample is a subset of a statistical population in which each member of the
subset has an equal probability of being chosen. An example of a simple random sample would be
the names of 25 employees being chosen out of a hat from a company of 250 employees. In this
case, the population is all 250 employees, and the sample is random because each employee has
an equal chance of being chosen.
A simple random sample is meant to be an unbiased representation of a group. It is
considered a fair way to select a sample from a larger population, since every member of the
population has an equal chance of getting selected.
A sampling error can occur with a simple random sample if the sample does not end up
accurately reflecting the population it is supposed to represent. For example, in our simple random
sample of 25 employees, it would be possible to draw 25 men even if the population consisted of
125 women and 125 men. For this reason, simple random sampling is more commonly used when
the researcher knows little about the population. If the researcher knew more, it would be better to
use a different sampling technique, such as stratified random sampling, which helps to account for
the differences within the population, such as age, race or gender.

Statistical Hypothesis
In order to understand the concept of a hypothesis, consider the following statements:
1. A coin is fair.
2. A die is balanced.
3. Small is beautiful but big is wonderful.
4. A rust resistant wheat variety has lower average yield than a non-resistant variety.
5. Educational material prepared by a researcher is better than the traditional material.
6. There are infinitely many prime numbers.
7. I am a good student.
8. 60% of my students are successful.
9. Managers in small scale factories experience more stress than the managers in large scale
industries.
Some of the above statements can be verified, confirmed or discarded. For example, consider the
statement (3) above. It cannot be verified by collecting data. It is the opinion of an individual.
Consider the statement (6). No data are required to prove the statement. It is a mathematical
hypothesis and can be proved by using mathematical induction. Statement numbers (1), (2), (4),
(5) (8) and (9) may be verified by designing appropriate experiments, collecting and analyzing
relevant data using appropriate statistical methods. Such unproved statements are called as
statistical hypothesis. Simply speaking, hypothesis is an unproved statement. We will restrict only
to statistical hypothesis.
A statistical hypothesis is an unproved statement about the distribution and or parameters
of the distribution of random variable(s) under consideration. In other words a statistical hypothesis
is an assertion about the unknown distribution and/or parameter of one or more random variables.
For example, suppose that a pharmaceutical company claims that its new drug brought in the
market is effective in curing the disease. To test the claim, an experiment is planned and the drug
is administered to 20 patients. Suppose X denotes the number of patients cured and p is the
probability that a patient is cured. We assume p to be the same for all patients. Here X is a
binomial random variable with parameters n = 20 and p. Suppose that company’s claim is that the
drug has 90% success rate. This claim needs to be verified by conducting appropriate clinical trial.
In statistical language the drug manufacturer’s claim is p = 0.90. It is a statistical hypothesis. If a
statistical hypothesis completely specifies the distribution and/or its parameters, then it is called a
simple hypothesis. If it does not completely specify the distribution and/or its parameters then it is
called composite hypothesis. In the above example the number of patients cured, X, follows
binomial distribution with n = 20 and p = 0.9. It means that the distribution of X is completely
specified, hence p = 0.90 is a simple hypothesis. Contrary to it, the claim was that the drug has
success rate above 70%, then p ≥ 0.70 is a composite hypothesis.

Sampling Distribution And Standard Error

A sampling distribution is a probability distribution of a statistic obtained through a large number of
samples drawn from a specific population. The sampling distribution of a given population is the
distribution of frequencies of a range of different outcomes that could possibly occur for a statistic
of a population.
A lot of data drawn and used by academicians, statisticians, researchers, marketers,
analysts, etc. are actually samples, not population. A sample is a subset of a population. For
example, a medical researcher that wanted to compare the average weight of all babies born in
North America from 1995 to 2005 to those born in South America within the same time period
cannot within a reasonable amount of time draw the data for the entire population of over a million
childbirths that occurred over the ten-year time frame. He will instead only use the weight of, say
100 babies, in each continent to make a conclusion. The weight of 200 babies used is the sample
and the average weight calculated is the sample mean.
Now suppose that instead of taking just one sample of 100 newborn weights from each
continent, the medical researcher takes repeated random samples from the general population,
and computes the sample mean for each sample group. So, for North America, he pulls up data
for 100 newborn weights recorded in the US, Canada, and Mexico as follows: four 100 samples
from select hospitals in the US, five 70 samples from Canada, and three 150 records from Mexico,
for a total of 1200 weights of newborn babies grouped in 12 sets. He also collects a sample data
of 100 birth weights from each of the 12 countries in South America. Each sample has its own
sample mean and the distribution of the sample means is known as the sample distribution.
The average weight computed for each of the 2400 sample set is the sampling distribution
of the mean. Not just the mean can be calculated from a sample. Other statistics, such as the
standard deviation, variance, proportion, and range can be calculated from a sample data. The
standard deviation and variance measure the variability of the sampling distribution. The number
of observations in a population, number of observations in a sample, and the procedure used to
draw the sample sets determine the variability of a sampling distribution. The standard deviation of
a sampling distribution is called the standard error. While the mean of a sampling distribution is
equal to the mean of the population, the standard error depends on the standard deviation of the
population, the size of the population, and the size of the sample. Knowing how spread apart the
mean of each of the sample sets are from each other and from the population mean, we get an
indication of how close the sample mean is to the population mean. The standard error of the
sampling distribution decreases as the sample size increases.

Null Hypothesis and Alternative Hypothesis

For example, suppose that a pharmaceutical company claims that its new drug brought in the
market is effective in curing the disease. To test the claim, an experiment is planned and the drug
is administered to 20 patients. Suppose X denotes the number of patients cured and p is the
probability that a patient is cured. We assume p to be the same for all patients. Here X is a
binomial random variable with parameters n = 20 and p = 0.90 because the company’s claim is
that the drug has 90% success rate.
A statistical hypothesis can be classified into two attributes:
(i) Simple or Composite, and
 (ii) Null or Alternative.
In the above example p = 0.90 is equivalent to the statement that p - 0.90 = 0. That is the
difference between parameter value and the specified value is zero (i.e., null). Null
Hypothesis is usually denoted by Ho. In statistical procedures we always test a null hypothesis. In
words of Sir Ronald Fisher “Null hypothesis is the hypothesis which is tested for the possible
rejection under the assumption that it is true”. To formally define “null hypothesis”: A null
hypothesis is a statistical hypothesis. It is a relation between a, function of the parameter(s) and its
possible value(s).
Null hypothesis is a hypothesis which the researcher would like to claim to be correct, unless the
data gives strong evidence against it. It is usually denoted by H o .
Examples:
 A coin is fair, i.e., Ho : p = 0.5
 A six faced die is balanced, i.e., Ho: pi = 1/6 for i = 1, 2…6.
 Average effect before treatment (µB) and after treatment (µA) is same. Here Ho : µB = µA
If a null hypothesis is rejected then the hypothesis that may be accepted as a consequence is
termed as the alternative hypothesis. It is denoted either by H1 or HA . For example in case of the
drug manufacturers claim a medical practitioner may set alternative hypotheses as H 1 : p < 0.90
whereas in case of random experiment of tossing of a coin H 1 : p ≠0.5 and in case of tossing of a
die it would be H1 : pi ≠1/6 for at least one i = 1, 2,…6.

Test of a Statistical Hypothesis

A test of a statistical hypothesis is a rule, which specifies for each possible observed data whether
to accept or reject the null hypothesis under consideration.

Test Statistic
In order to test Ho , we perform an experiment. The experiment generates sample data. The
sample data is then summarized into a single number. This number, which is a function of sample
values, is called a test statistic. The formula for computation of the test statistic depends on H o .
The conclusion ( i.e. acceptance/rejection) of the null hypothesis depends on the sample data
only, through the test statistic. We usually denote the test statistic by T. Note that this test statistic
is strictly a function of sample values and known constants, if any. Hence we write T = T(X). We
compare observed value of the test statistic with standard (or reference) value to arrive at a
conclusion.
Let us consider the drug manufacturer’s problem. Suppose we want to test H o: p ≥ 0.7 against H1 :
p < 0.7. The drug is tested upon 100 patients and X is the number of successfully treated patients,
observed. We can use T(X) = X itself as a test statistic. Common sense suggests that we should
accept Ho if X ≥ 70 and reject if X < 70 is observed.

Critical Region And Acceptance Region

When we make use of a statistical test procedure, it will involve a test statistic say T. Values of T
will vary from sample to sample. The possible values of T will always belong to sample space S of
possible values of T. Some values of T lead to acceptance of H o, whereas the others lead to
rejection of Ho . The set of values of T that are associated with rejection of H o is called the critical
region (C). All the other values of T will lead to acceptance of H o . It forms acceptance region (A).
Note that C and A are mutually exclusive and exhaustive subsets of S. By this we mean that, there
is nothing common in critical and acceptance region. In the earlier example of drug manufacturer,
we can set critical region as C = {0, 1, …59} and acceptance region as A = {60, 61 …100}.

Type I and Type II Error

When we apply appropriate statistical test to verify our claim, we have to either accept H o or
reject Ho . If we reject Ho , then as its consequence we will be favouring acceptance of H 1. Our
decision is based on whether the value of the test statistic belongs to acceptance region or critical
region. Notice that we come across following situations.

Thus two kinds of errors are possible: we may reject H o when it is true or we may accept H
o when it is false. These errors are respectively referred as Type I and Type II errors. We can
summarize this discussion in table form as follows:

Since our conclusion is based upon the data generated in the random experiment (or based
upon data available from random sample), these errors may occur with certain probabilities.

When a statistical test is constructed our aim is to reduce the possibilities of both type of
errors and take a correct decision. Remember that both the errors cannot be minimized
simultaneously. Therefore the proposed solution is to consider only that procedure for which
P(Type I error) ≤ α (a fixed number). This is so because wrong rejection of H o is considered
to be more serious error as opposed to wrong rejection of H 1 Suppose you are given a drug. Ho is
that the drug is poisonous; H1 is it is OK as a medicine. When you accept Ho it means you
conclude that drug is poisonous. So you decide not to use it. If you reject H o it means you believe
that it is OK for use. Suppose you commit type I error. It means you are allowing, eating of poison.
If you commit type II error, it means you throw good medicine. Which error is serious? Certainly
type I error. So we must minimize error that is more serious. However remember that, whenever
you apply a statistical test to a dataset and you arrive at a decision of rejecting H o, either the
decision is right or you have committed a type I error. Similarly if, you have accepted H o, either the
decision is right or you have committed a type II error.
 Level of Significance
The probability of committing type I error is called as the size of the test. The level of significance
is the maximum probability of type I error we are willing to tolerate. It is usually denoted by α. It is
many times expressed in percentage form. It is customary that in most of statistical work, α is
taken as 5% or 1%, but it is not a sacrosanct value. How to decide the level of significance and
who should decide it?. The answer to the first question is, it is a job of subject expert who plans
and executes the experiment. The statistical software can compute the probability of getting a
result as extreme as (or more extreme than) the observed result under H o. Such a probability is
known as p-value of the test. Now-a-days computer software provides p-value of the test.

One Sided and Two Sided Test (One Tailed and Two Tailed Test)
One Tailed Test
A one-tailed test allows you to determine if one mean is greater or less than another mean, but not
both. A direction must be chosen prior to testing.
In other words, a one-tailed test tells you the effect of a change in one direction and not the other.
Think of it this way, if you are trying to decide if you should buy a brand name product or a generic
product at your local drugstore, a one-tailed test of the effectiveness of the product would only tell
you if the generic product worked better than the brand name. You would have no insight into
whether the product was equivalent or worse.
Since the generic product is cheaper, you could see what looks like a minimal impact, but is in fact
a negative impact (meaning it doesn’t work very well at all!), but you go ahead and purchase the
generic product because it is cheaper.
If this is the case, you’re probably wondering when a one-tailed test should be used? One-tailed
tests should be used only when you are not worried about missing an effect in the untested
direction.
But how does this impact optimization? If you’re running a test and only using a one-tailed test,
you will only see significance if your new variant outperforms the default. There are 2 outcomes:
the new variant wins or we cannot distinguish it from the default.
Two Tailed Test
A two-tailed test allows you to determine if two means are different from one another. A direction
does not have to be specified prior to testing.
In other words, a two-tailed test will take into account the possibility of both a positive and a
negative effect.
Let’s head back to the drug store. If you were doing a two-tailed test of the generic against the
brand name product, you would have insight into whether the effectiveness of the product was
equivalent or worse than the brand name product. In this instance, you can make a more educated
decision because if the generic product is equivalent, you would purchase it because it is cheaper,
but if it is far less effective than the brand name product, you’d probably shell out the extra money
for the brand name product. You wouldn’t want to waste your money on an ineffective product,
would you?
So when a two-tailed test should be used? Two-tailed tests should be used when you are willing to
accept any of the following:
 One mean being greater, lower or similar to the other.
And how does this impact optimization? When running a test, if you are using a two-tailed test you
will see significance if your new variant’s mean is different from that of the default. There are 3
outcomes: the new variant wins, lost or is similar to the default.

Concept of Test of Hypothesis

Hypothesis testing is an act in statistics whereby an analyst tests an assumption regarding a
population parameter. The methodology employed by the analyst depends on the nature of the
data used and the reason for the analysis. Hypothesis testing is used to infer the result of a
hypothesis performed on sample data from a larger population.
In hypothesis testing, an analyst tests a statistical sample, with the goal of accepting or
rejecting a null hypothesis. The test tells the analyst whether or not his primary hypothesis is true.
If it isn't true, the analyst formulates a new hypothesis to be tested, repeating the process until
data reveals a true hypothesis.
 Statistical analysts test a hypothesis by measuring and examining a random sample of the
population being analyzed. All analysts use a random population sample to test two different
hypotheses: the null hypothesis and the alternative hypothesis. The null hypothesis is the
hypothesis the analyst believes to be true. Analysts believe the alternative hypothesis to be
untrue, making it effectively the opposite of a null hypothesis. This makes it so they are mutually
exclusive, and only one can be true. However, one of the two hypotheses will always be true.
If, for example, a person wants to test that a penny has exactly a 50% chance of landing heads,
the null hypothesis would be yes, and the alternative hypothesis would be no, it does not.
Mathematically, the null hypothesis would be represented as Ho: P = 0.5. The alternative
hypothesis would be denoted as "Ha" and be identical to the null hypothesis, except with the equal
sign struck-through, meaning that it does not equal 50%.
A random sample of 100 coin flips is taken from a random population of coin flippers, and the null
hypothesis is then tested. If it is found that the 100 coin flips were distributed as 40 heads and 60
tails, the analyst would assume that a penny does not have a 50% chance of landing heads, and
would reject the null hypothesis and accept the alternative hypothesis. Afterward, a new
hypothesis would be tested, this time that a penny has a 40% chance of landing heads.
All hypotheses are tested using a four-step process. The first step is for the analyst to state
the two hypotheses so that only one can be right. The next step is to formulate an analysis plan,
which outlines how the data will be evaluated. The third step is to carry out the plan and physically
analyze the sample data. The fourth and final step is to analyze the results and either accept or
reject the null.