One-sample

proportion tests
HYPOTHESIS TESTING IN PYTHON

James Chapman
Curriculum Manager, DataCamp
Chapter 1 recap
Is a claim about an unknown population proportion feasible?

1. Standard error of sample statistic from bootstrap distribution

2. Compute a standardized test statistic

3. Calculate a p-value

4. Decide which hypothesis made most sense

Now, calculate the test statistic without using the bootstrap distribution

HYPOTHESIS TESTING IN PYTHON

Standardized test statistic for proportions
p: population proportion (unknown population parameter)

p^: sample proportion (sample statistic)

p0 : hypothesized population proportion

p^ − mean( p^) p^ − p
z= =
SE( p^) SE( p^)
Assuming H0 is true, p = p0 , so
p^ − p0
z=
SE( p^)

HYPOTHESIS TESTING IN PYTHON

Simplifying the standard error calculations
SE p^ = √
p0 ∗ (1 − p0 )
→ Under H0 , SE p^ depends on hypothesized p0 and sample size n
n
Assuming H0 is true,

p^ − p0
z=
√ p0 ∗ (1 − p0 )
n
^ and n) and the hypothesized parameter (p0 )
Only uses sample information ( p

HYPOTHESIS TESTING IN PYTHON

Why z instead of t?
(x̄child − x̄adult )
t=
√
s2child s2adult
+
nchild nadult

s is calculated from x̄
x̄ estimates the population mean
s estimates the population standard deviation
↑ uncertainty in our estimate of the parameter
t-distribution - fatter tails than a normal distribution

p^ only appears in the numerator, so z-scores are fine

HYPOTHESIS TESTING IN PYTHON

Stack Overflow age categories
H0 : Proportion of Stack Overflow users under thirty = 0.5

HA : Proportion of Stack Overflow users under thirty ≠ 0.5

alpha = 0.01

stack_overflow['age_cat'].value_counts(normalize=True)

Under 30 0.535604
At least 30 0.464396
Name: age_cat, dtype: float64

HYPOTHESIS TESTING IN PYTHON

Variables for z
p_hat = (stack_overflow['age_cat'] == 'Under 30').mean()

0.5356037151702786

p_0 = 0.50

n = len(stack_overflow)

2261

HYPOTHESIS TESTING IN PYTHON

Calculating the z-score
p^ − p0
z=
√
p0 ∗ (1 − p0 )
n

import numpy as np
numerator = p_hat - p_0
denominator = np.sqrt(p_0 * (1 - p_0) / n)
z_score = numerator / denominator

3.385911440783663

HYPOTHESIS TESTING IN PYTHON

Calculating the p-value
Two-tailed ("not equal"):

p_value = norm.cdf(-z_score) +
1 - norm.cdf(z_score)

p_value = 2 * (1 - norm.cdf(z_score))
Left-tailed ("less than"):
0.0007094227368100725
from scipy.stats import norm
p_value = norm.cdf(z_score)
p_value <= alpha

Right-tailed ("greater than"):

True

p_value = 1 - norm.cdf(z_score)

HYPOTHESIS TESTING IN PYTHON

 Let's practice!
HYPOTHESIS TESTING IN PYTHON
 Two-sample
proportion tests
HYPOTHESIS TESTING IN PYTHON

James Chapman
Curriculum Manager, DataCamp
Comparing two proportions
H0 : Proportion of hobbyist users is the same for those under thirty as those at least thirty

H0 : p≥30 − p<30 = 0

HA : Proportion of hobbyist users is different for those under thirty to those at least thirty

HA : p≥30 − p<30 ≠ 0

alpha = 0.05

HYPOTHESIS TESTING IN PYTHON

Calculating the z-score
z-score equation for a proportion test:
( p^≥30 − p^<30 ) − 0
z=
SE( p^≥30 − p^<30 )
Standard error equation:

SE( p^≥30 − p^<30 ) = √

p^ × (1 − p^) p^ × (1 − p^)
+
n≥30 n<30

p^ → weighted mean of p^≥30 and p^<30

n≥30 × p^≥30 + n<30 × p^<30
p^ =
n≥30 + n<30
Only require p^≥30 , p^<30 , n≥30 , n<30 from the sample to calculate the z-score

HYPOTHESIS TESTING IN PYTHON

Getting the numbers for the z-score
p_hats = stack_overflow.groupby("age_cat")['hobbyist'].value_counts(normalize=True)

age_cat hobbyist
At least 30 Yes 0.773333
No 0.226667
Under 30 Yes 0.843105
No 0.156895
Name: hobbyist, dtype: float64

n = stack_overflow.groupby("age_cat")['hobbyist'].count()

age_cat
At least 30 1050
Under 30 1211
Name: hobbyist, dtype: int64

HYPOTHESIS TESTING IN PYTHON

Getting the numbers for the z-score
p_hats = stack_overflow.groupby("age_cat")['hobbyist'].value_counts(normalize=True)

age_cat hobbyist
At least 30 Yes 0.773333
No 0.226667
Under 30 Yes 0.843105
No 0.156895
Name: hobbyist, dtype: float64

p_hat_at_least_30 = p_hats[("At least 30", "Yes")]

p_hat_under_30 = p_hats[("Under 30", "Yes")]
print(p_hat_at_least_30, p_hat_under_30)

0.773333 0.843105

HYPOTHESIS TESTING IN PYTHON

Getting the numbers for the z-score
n = stack_overflow.groupby("age_cat")['hobbyist'].count()

age_cat
At least 30 1050
Under 30 1211
Name: hobbyist, dtype: int64

n_at_least_30 = n["At least 30"]

n_under_30 = n["Under 30"]
print(n_at_least_30, n_under_30)

1050 1211

HYPOTHESIS TESTING IN PYTHON

Getting the numbers for the z-score
p_hat = (n_at_least_30 * p_hat_at_least_30 + n_under_30 * p_hat_under_30) /
(n_at_least_30 + n_under_30)

std_error = np.sqrt(p_hat * (1-p_hat) / n_at_least_30 +

p_hat * (1-p_hat) / n_under_30)

z_score = (p_hat_at_least_30 - p_hat_under_30) / std_error

print(z_score)

-4.223718652693034

HYPOTHESIS TESTING IN PYTHON

Proportion tests using proportions_ztest()
stack_overflow.groupby("age_cat")['hobbyist'].value_counts()

age_cat hobbyist
At least 30 Yes 812
No 238
Under 30 Yes 1021
No 190
Name: hobbyist, dtype: int64

n_hobbyists = np.array([812, 1021])

n_rows = np.array([812 + 238, 1021 + 190])
from statsmodels.stats.proportion import proportions_ztest
z_score, p_value = proportions_ztest(count=n_hobbyists, nobs=n_rows,
alternative="two-sided")

(-4.223691463320559, 2.403330142685068e-05)

HYPOTHESIS TESTING IN PYTHON

 Let's practice!
HYPOTHESIS TESTING IN PYTHON
 Chi-square test of
independence
HYPOTHESIS TESTING IN PYTHON

James Chapman
Curriculum Manager, DataCamp
Revisiting the proportion test
age_by_hobbyist = stack_overflow.groupby("age_cat")['hobbyist'].value_counts()

age_cat hobbyist
At least 30 Yes 812
No 238
Under 30 Yes 1021
No 190
Name: hobbyist, dtype: int64

from statsmodels.stats.proportion import proportions_ztest

n_hobbyists = np.array([812, 1021])
n_rows = np.array([812 + 238, 1021 + 190])
stat, p_value = proportions_ztest(count=n_hobbyists, nobs=n_rows,
alternative="two-sided")

(-4.223691463320559, 2.403330142685068e-05)

HYPOTHESIS TESTING IN PYTHON

Independence of variables
Previous hypothesis test result: evidence that hobbyist and age_cat are associated

Statistical independence - proportion of successes in the response variable is the same

across all categories of the explanatory variable

HYPOTHESIS TESTING IN PYTHON

Test for independence of variables
import pingouin
expected, observed, stats = pingouin.chi2_independence(data=stack_overflow, x='hobbyist',
y='age_cat', correction=False)
print(stats)

test lambda chi2 dof pval cramer power

0 pearson 1.000000 17.839570 1.0 0.000024 0.088826 0.988205
1 cressie-read 0.666667 17.818114 1.0 0.000024 0.088773 0.988126
2 log-likelihood 0.000000 17.802653 1.0 0.000025 0.088734 0.988069
3 freeman-tukey -0.500000 17.815060 1.0 0.000024 0.088765 0.988115
4 mod-log-likelihood -1.000000 17.848099 1.0 0.000024 0.088848 0.988236
5 neyman -2.000000 17.976656 1.0 0.000022 0.089167 0.988694

χ2 statistic = 17.839570 = (−4.223691463320559)2 = (z -score)2

HYPOTHESIS TESTING IN PYTHON

Job satisfaction and age category
stack_overflow['age_cat'].value_counts() stack_overflow['job_sat'].value_counts()

Under 30 1211 Very satisfied 879

At least 30 1050 Slightly satisfied 680
Name: age_cat, dtype: int64 Slightly dissatisfied 342
Neither 201
Very dissatisfied 159
Name: job_sat, dtype: int64

HYPOTHESIS TESTING IN PYTHON

Declaring the hypotheses
H0 : Age categories are independent of job satisfaction levels

HA : Age categories are not independent of job satisfaction levels

alpha = 0.1

Test statistic denoted χ2

Assuming independence, how far away are the observed results from the expected values?

HYPOTHESIS TESTING IN PYTHON

Exploratory visualization: proportional stacked bar plot
props = stack_overflow.groupby('job_sat')['age_cat'].value_counts(normalize=True)
wide_props = props.unstack()
wide_props.plot(kind="bar", stacked=True)

HYPOTHESIS TESTING IN PYTHON

Exploratory visualization: proportional stacked bar plot

HYPOTHESIS TESTING IN PYTHON

Chi-square independence test
import pingouin
expected, observed, stats = pingouin.chi2_independence(data=stack_overflow, x="job_sat", y="age_cat")
print(stats)

test lambda chi2 dof pval cramer power

0 pearson 1.000000 5.552373 4.0 0.235164 0.049555 0.437417
1 cressie-read 0.666667 5.554106 4.0 0.235014 0.049563 0.437545
2 log-likelihood 0.000000 5.558529 4.0 0.234632 0.049583 0.437871
3 freeman-tukey -0.500000 5.562688 4.0 0.234274 0.049601 0.438178
4 mod-log-likelihood -1.000000 5.567570 4.0 0.233854 0.049623 0.438538
5 neyman -2.000000 5.579519 4.0 0.232828 0.049676 0.439419

Degrees of freedom:

(No. of response categories − 1) × (No. of explanatory categories − 1)

(2 − 1) ∗ (5 − 1) = 4

HYPOTHESIS TESTING IN PYTHON

Swapping the variables?
props = stack_overflow.groupby('age_cat')['job_sat'].value_counts(normalize=True)
wide_props = props.unstack()
wide_props.plot(kind="bar", stacked=True)

HYPOTHESIS TESTING IN PYTHON

Swapping the variables?

HYPOTHESIS TESTING IN PYTHON

chi-square both ways
expected, observed, stats = pingouin.chi2_independence(data=stack_overflow, x="age_cat", y="job_sat")
print(stats[stats['test'] == 'pearson'])

test lambda chi2 dof pval cramer power

0 pearson 1.0 5.552373 4.0 0.235164 0.049555 0.437417

Ask: Are the variables X and Y independent?

Not: Is variable X independent from variable Y?

HYPOTHESIS TESTING IN PYTHON

 What about direction and tails?
Observed and expected counts squared must be non-negative

chi-square tests are almost always right-tailed 1

1Left-tailed chi-square tests are used in statistical forensics to detect if a fit is suspiciously good because the
data was fabricated. Chi-square tests of variance can be two-tailed. These are niche uses, though.

HYPOTHESIS TESTING IN PYTHON

 Let's practice!
HYPOTHESIS TESTING IN PYTHON
 Chi-square
goodness of fit tests
HYPOTHESIS TESTING IN PYTHON

James Chapman
Curriculum Manager, DataCamp
Purple links
How do you feel when you discover that you've already visited the top resource?

purple_link_counts = stack_overflow['purple_link'].value_counts()

purple_link_counts = purple_link_counts.rename_axis('purple_link')\
.reset_index(name='n')\
.sort_values('purple_link')

purple_link n
2 Amused 368
3 Annoyed 263
0 Hello, old friend 1225
1 Indifferent 405

HYPOTHESIS TESTING IN PYTHON

Declaring the hypotheses
hypothesized = pd.DataFrame({ purple_link prop
'purple_link': ['Amused', 'Annoyed', 'Hello, old friend', 'Indifferent'], 0 Amused 0.166667
'prop': [1/6, 1/6, 1/2, 1/6]}) 1 Annoyed 0.166667
2 Hello, old friend 0.500000
3 Indifferent 0.166667

H0 : The sample matches the hypothesized χ2 measures how far observed results are
distribution from expectations in each group

HA : The sample does not match the alpha = 0.01

hypothesized distribution

HYPOTHESIS TESTING IN PYTHON

Hypothesized counts by category
n_total = len(stack_overflow)
hypothesized["n"] = hypothesized["prop"] * n_total

purple_link prop n
0 Amused 0.166667 376.833333
1 Annoyed 0.166667 376.833333
2 Hello, old friend 0.500000 1130.500000
3 Indifferent 0.166667 376.833333

HYPOTHESIS TESTING IN PYTHON

Visualizing counts
import matplotlib.pyplot as plt

plt.bar(purple_link_counts['purple_link'], purple_link_counts['n'],
color='red', label='Observed')
plt.bar(hypothesized['purple_link'], hypothesized['n'], alpha=0.5,
color='blue', label='Hypothesized')

plt.legend()
plt.show()

HYPOTHESIS TESTING IN PYTHON

Visualizing counts

HYPOTHESIS TESTING IN PYTHON

chi-square goodness of fit test
print(hypothesized)

purple_link prop n
0 Amused 0.166667 376.833333
1 Annoyed 0.166667 376.833333
2 Hello, old friend 0.500000 1130.500000
3 Indifferent 0.166667 376.833333

from scipy.stats import chisquare

chisquare(f_obs=purple_link_counts['n'], f_exp=hypothesized['n'])

Power_divergenceResult(statistic=44.59840778416629, pvalue=1.1261810719413759e-09)

HYPOTHESIS TESTING IN PYTHON

 Let's practice!
HYPOTHESIS TESTING IN PYTHON