Scribd
Upload
14 views3 pages

Post Hoc

The document discusses post hoc comparisons, which are statistical tests conducted after an experiment to determine which sample means are significantly different. It explains that post hoc tests help reduce type 1 errors that can occur when making multiple comparisons, by using a higher threshold for significance. It also discusses type 1 and type 2 errors and how significance levels and p-values are used to determine statistical significance.

Uploaded by

prachee dash
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as DOCX, PDF, TXT or read online on Scribd
14 views3 pages

Post Hoc

The document discusses post hoc comparisons, which are statistical tests conducted after an experiment to determine which sample means are significantly different. It explains that post hoc tests help reduce type 1 errors that can occur when making multiple comparisons, by using a higher threshold for significance. It also discusses type 1 and type 2 errors and how significance levels and p-values are used to determine statistical significance.

Uploaded by

prachee dash
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as DOCX, PDF, TXT or read online on Scribd
You are on page 1/ 3

 Definition of Post Hoc Comparisons:

 Post hoc comparisons (or a posteriori comparisons) are statistical tests conducted after
an experiment to determine which specific groups' means are different.

 Purpose of Post Hoc Comparisons:

 These tests help prevent us from taking undue advantage of chance.


 When comparing many means, some differences may appear substantial due to
random sampling variation alone, even if the null hypothesis (H0) is true.

 Problem with Conventional t Tests:

 If we apply the t test to all possible differences between sample means or just the
largest ones, we increase the probability of a Type I error (rejecting H0 when it is
true).
 The t test is designed to compare the means of only two samples at a time.

 Post Hoc Tests and Type I Errors:

 Post hoc tests use sampling distributions that compare the means of many samples,
not just two.
 They require a larger difference between sample means before declaring it statistically
significant, reducing the likelihood of Type I errors.

 Conclusion:

 Post hoc tests protect us from making too many Type I errors by setting a higher
threshold for declaring differences statistically significant.

TYPE 1 AND 2- https://www.scribbr.com/statistics/type-i-and-type-ii-errors/

 What is a Type I Error?

 A Type I error happens when you conclude that your results are statistically
significant (meaning there's an effect or difference) when, in fact, there is no effect or
difference in reality. Essentially, it's a "false positive."

 Significance Level (Alpha, α)

 The significance level is a threshold you set at the start of your study to decide how
likely it is that your results are due to chance.
 Commonly, researchers set this level at 0.05 (or 5%).

 What Does the Significance Level Mean?

 If you set your significance level at 0.05, you are saying you're willing to accept a 5%
chance of making a Type I error.
 This means there's a 5% chance your results could happen just by chance if the null
hypothesis (no effect or difference) is actually true.

 P Value

 The p value is the probability that your results happened by chance.


 You compare the p value to your significance level to decide if your results are
statistically significant.

 Making a Decision

 If your p value is lower than the significance level (e.g., p < 0.05), you consider your
results statistically significant and reject the null hypothesis.
 If your p value is higher than the significance level, you do not reject the null
hypothesis.

 Example

 Suppose you're testing a new drug and you get a p value of 0.035.
 Since 0.035 is less than your significance level of 0.05, you conclude that your results
are statistically significant and reject the null hypothesis.
 However, there's still a 3.5% chance (0.035) that your results happened by chance,
meaning there's still a small risk of a Type I error.

 Reducing Type I Error Risk

 To reduce the risk of making a Type I error, you can set a lower significance level,
like 0.01 (1% chance of error).

SIGNIFICANCE LEVEL VS P VALUE

1. Definition and Purpose:


o Level of Significance (α): The threshold set by the researcher before
conducting the test. It defines the probability of rejecting the null hypothesis
when it is actually true (Type I error). Common values are 0.05, 0.01, and
0.10.
o p-Value: The probability of obtaining a test statistic at least as extreme as the
one observed, assuming the null hypothesis is true. It is calculated from the
data.
2. Role in Hypothesis Testing:
o Level of Significance (α): Determines the cut-off point for deciding whether
to reject the null hypothesis. It is a predetermined criterion.
o p-Value: Indicates how compatible the observed data is with the null
hypothesis. It helps in making the decision about the null hypothesis.
3. Decision Making:
o Level of Significance (α): If the p-value is less than or equal to α, the null
hypothesis is rejected. It sets the standard for significance.
o p-Value: If the p-value is smaller than the level of significance, it suggests
that the observed data is unlikely under the null hypothesis, leading to its
rejection.
4. Fixed vs. Variable:
o Level of Significance (α): A fixed value chosen before the experiment starts.
It does not change with the data.
o p-Value: A variable that depends on the sample data. It varies with different
datasets.
5. Interpretation:
o Level of Significance (α): Represents the maximum risk of making a Type I
error that the researcher is willing to accept.
o p-Value: Measures the strength of the evidence against the null hypothesis. A
smaller p-value indicates stronger evidence against the null hypothesis.

In summary, the level of significance is a predetermined threshold that dictates the criteria for
rejecting the null hypothesis, while the p-value is calculated from the data and indicates the
likelihood of the observed results under the null hypothesis.

You might also like