Unit IX – EDUC 201

Testing Statistical Significance

 Statistical significance
 is used to provide evidence concerning the plausibility of the null
hypothesis, which hypothesizes that there is nothing more than
random chance at work in the data. Statistical hypothesis testing is
used to determine whether the result of a data set is statistically
significant.
  Tests for statistical significance are used to
address the question:

 What is the probability that what we think is a relationship

between two variables is really just a chance occurrence?
 Tests for statistical significance
tell us what the probability is that
the relationship we think we have
found is due only to random
chance.
 They tell us what the probability is
that we would be making an error
if we assume that we have found
that a relationship exists.
 We can never be completely 100% certain that a relationship
exists between two variables.
 There are too many sources of error to be controlled, for
example, sampling error, researcher bias, problems with
reliability and validity, simple mistakes, etc.
 But using probability theory and the normal curve, we
can estimate the probability of being wrong, if we
assume that our finding a relationship is true.
 If the probability of being wrong is small, then we say
that our observation of the relationship is a statistically
significant finding.
 Statistical significance means that there is a good chance
that we are right in finding that a relationship exists
between two variables. But statistical significance is not
the same as practical significance.
 We can have a statistically significant finding, but the
implications of that finding may have no practical
application.
 The researcher must always examine both the statistical
and the practical significance of any research finding.
 For example, we may find that there is a statistically
significant relationship between a citizen's age and
satisfaction with city recreation services. It may be that older
citizens are 5% less satisfied than younger citizens with city
recreation services. But is 5% a large enough difference to
be concerned about?
     Often times, when differences are small but statistically
significant, it is due to a very large sample size; in a sample
of a smaller size, the differences would not be enough to be
statistically significant.
Steps in Testing for Statistical
Significance
1) State the Research Hypothesis
2) State the Null Hypothesis
3) Select a probability of error level (alpha level)
4) Select and compute the test for statistical significance
5) Interpret the results
 1) State the Research Hypothesis

A research hypothesis states the expected relationship between two variables. It may be
stated in general terms, or it may include dimensions of direction and magnitude. For
example,
General: The length of the job training program is related to the rate of job placement of
trainees.
Direction: The longer the training program, the higher the rate of job placement of
trainees.
Magnitude: Longer training programs will place twice as many trainees into jobs as
shorter programs.
 2) State the Null Hypothesis

A null hypothesis usually states that there is no relationship

between the two variables. For example,
There is no relationship between the length of the job training
program and the rate of job placement of trainees.
Graduate assistant pay is not influenced by gender.
    A null hypothesis may also state that the relationship proposed
in the research hypothesis is not true.
 Researchers use a null hypothesis in research because it
is easier to disprove a null hypothesis than it is to prove a
research hypothesis.
 The null hypothesis is the researcher's "straw man." That
is, it is easier to show that something is false once than to
show that something is always true.
 It is easier to find disconfirming evidence against the
null hypothesis than to find confirming evidence for the
research hypothesis.
3) TYPE I AND TYPE II ERRORS
 Even in the best research project, there is always a possibility
(hopefully a small one) that the researcher will make a mistake
regarding the relationship between the two variables. There are
two possible mistakes or errors.   
 The first is called a Type I error. This occurs when the
researcher assumes that a relationship exists when in fact the
evidence is that it does not. In a Type I error, the researcher
should accept the null hypothesis and reject the research
hypothesis, but the opposite occurs. The probability of
committing a Type I error is called alpha.
 The second is called a Type II error. This occurs when the researcher assumes
that a relationship does not exist when in fact the evidence is that it does. In a
Type II error, the researcher should reject the null hypothesis and accept the
research hypothesis, but the opposite occurs. The probability of committing a
Type II error is called beta.
     Generally, reducing the possibility of committing a Type I error increases the
possibility of committing a Type II error and vice versa, reducing the possibility
of committing a Type II error increases the possibility of committing a Type I
error.
     Researchers generally try to minimize Type I errors, because when a
researcher assumes a relationship exists when one really does not, things may be
worse off than before. In Type II errors, the researcher misses an opportunity to
confirm that a relationship exists, but is no worse off than before.
4) The Chi Square Test
 For nominal and ordinal data, Chi Square is used as a
test for statistical significance. For example, we
hypothesize that there is a relationship between the
type of training program attended and the job
placement success of trainees. 
 5) Interpret the results

 If the computed value for Chi Square equals or

exceeds the value indicated in the table for the
given level of alpha and degrees of freedom,
then the researcher can assume that the observed
relationship between the two variables exists (at
the specified level of probability of error, or
alpha), and reject the null hypothesis. This gives
support to the research hypothesis.
Summary

1) state the research hypothesis:

There is a relationship between the type of training program attended and
the job placement success of trainees
2) state the null hypothesis:
There is no relationship between the type of training program attended and
the job placement success of trainees
3) calculate the test for statistical significance
4) calculate the degrees of freedom of the contingency table
5) select the level of alpha
6) look up the Chi Square
7) interpret the result
 Tests for statistical significance are used to estimate the
probability that a relationship observed in the data occurred only
by chance; the probability that the variables are really unrelated
in the population. They can be used to filter out unpromising
hypotheses.   
 Tests for statistical significance are used because they constitute
a common yardstick that can be understood by a great many
people, and they communicate essential information about a
research project that can be compared to the findings of other
projects.
   However, they do not assure that the research has been
carefully designed and executed. In fact, tests for statistical
significance may be misleading, because they are precise
numbers. But they have no relationship to the practical
significance of the findings of the research.
     Finally, one must always use measures of association along
with tests for statistical significance. The latter estimate the
probability that the relationship exists; while the former
estimate the strength (and sometimes the direction) of the
relationship. Each has its use, and they are best when used
together.
Ericka Paula Aquino Ison
EDUC 201 – BLOCK G