Topic Theory Manual Solving Simulation Coding

Introduction to T- - What is a t-test? - - Define the purpose - Simulate a sample - Write a Python function
Test Why is it used? - of a t-test.- Explain from a normal to check whether data
Assumptions of a t- why a t-test requires distribution and meets t-test assumptions.
test normally distributed check the validity of
data. t-test assumptions.
Types of T-Test - One-sample t-test - - Manually calculate - Simulate data for - Write Python functions
Two-sample t-test test statistics for all each type of t-test to implement all three
(independent) - three types of t-tests. and check if results types of t-tests.
Paired t-test align with
expectations.
Hypothesis Testing - Null and - Write down - Simulate data - Write a Python function
alternative hypotheses for a where you expect to that outputs a p-value and
hypotheses - P- practical example.- reject/fail to reject decision rule based on
value interpretation Compute test the null hypothesis. input data.
- Type I and Type II statistics and
errors interpret the p-value.
One-Sample T-Test - Formula and - Calculate test - Generate a single - Write a function to
interpretation - statistic and p-value sample and test if it perform a one-sample t-
When to use for a single dataset. differs from a test and display output.
known population
mean.
Two-Sample T-Test - Formula and - Calculate the test - Simulate two - Write a Python function
(Independent) interpretation - statistic for two samples and test if for a two-sample t-test
Equal vs unequal independent datasets they have equal with an option for
variance manually. means. equal/unequal variance.
Paired T-Test - Formula and - Manually calculate - Simulate - Write a Python function
interpretation - the test statistic for before/after to perform a paired t-test
When to use paired data. treatment data and and interpret the results.
test for a significant
effect.
Assumptions of T- - Normality - - Test whether - Simulate data to - Write a Python function
Test Independence - sample data meets t- test for violations of to check normality and
Homogeneity of test assumptions. assumptions. variance equality.
variance
Degrees of Freedom - Definition - How it - Calculate degrees of - Simulate how - Write a Python function
affects the t- freedom for a sample degrees of freedom to calculate degrees of
distribution manually. change with sample freedom for different
size. sample sizes.
P-Value and - What is a p-value? - Interpret a p-value - Simulate how the - Write a function to
Significance Level - How to choose a for different test p-value changes calculate and interpret a
significance level statistics. with sample size p-value.
and variability.
Confidence Intervals - How to calculate - - Manually compute - Simulate - Write a Python function
Relationship to t- confidence intervals confidence intervals to compute and visualize
test for a sample mean. and observe their confidence intervals.
variation.
T-Distribution vs - Differences and - Compute the shape - Simulate t- - Write a function to plot
Normal Distribution similarities - When of the t-distribution distribution vs and compare t-
to use each manually. normal distribution distribution vs normal
with different distribution.
degrees of freedom.
One-Tailed vs Two- - When to use each - Compute and - Simulate one- - Write a Python function
Tailed Tests type - Impact on p- compare one-tailed tailed vs two-tailed to perform one-tailed and
value vs two-tailed p- scenarios. two-tailed tests.
values.
Effect Size and - Cohen's d - - Manually calculate - Simulate datasets - Write a Python function
Power Statistical power Cohen's d and to estimate to calculate Cohen's d and
interpret results. statistical power. statistical power.
Alternative - When not to use a - Describe scenarios - Simulate data - Write Python code for a
Approaches t-test - Non- where a t-test is where assumptions Wilcoxon test or other
parametric inappropriate. are violated and non-parametric tests.
alternatives (e.g., explore
Wilcoxon test) alternatives.

Test Type When to Use

One-sample t-test Compare a sample mean to a known value (e.g., μ₀ = 7)
(scribbr.com)
Independent (two-sample) Compare means of two independent groups
t-test
Paired t-test Compare means of paired observations, like pre-post
measurements
Welch’s t-test Use when variances and/or sample sizes are unequal

One- vs Two-tailed Tests / Directionality

 Two-tailed: Testing for any difference.
 One-tailed: Testing a specific direction (e.g., Group A > Group B)

One-Sample t-test
🔍 Goal: Compare a sample mean to a known or target value.
📌 Use Case Example (Product Metrics – Amazon)
Scenario: Amazon’s product team claims that the average delivery time for a Prime order is within 2 days.
You collect a sample of 100 orders, and the average delivery time is 2.3 days.
💡 You want to test if this observed mean is significantly different from the target (2 days).

✅ Use:
# One-sample t-test checks if the sample average is different from 2.
# Null Hypothesis: μ = 2
💼 Why it's useful:
 Helps validate service level agreements (SLAs).
 Useful in benchmarking product KPIs against goals or promises.

✅ 2. Two-Sample (Independent) t-test

🔍 Goal: Compare means of two independent groups.
📌 Use Case Example (A/B Testing – Netflix UI)
Scenario: Netflix is testing two different UI layouts:
 Group A (old UI)
 Group B (new UI)
You want to test if the average watch time per user is higher with the new UI.
Why it's useful:
 Commonly used in A/B testing, marketing campaigns, pricing models.
 Helps data-driven decisions about feature rollouts or UI changes.

Paired t-test (Dependent Samples)

🔍 Goal: Compare means from the same group at two different times or under two conditions.
📌 Use Case Example (Before/After Feature Rollout – Spotify Personalization)
Scenario: Spotify introduces a new recommendation algorithm.
You compare user listening hours before and after the algorithm update for the same users.

Why it's useful:

 Helps measure impact of features or updates on the same user base.
 Controls for user-level variability.
Difference between 2-sample and paired t test
https://chatgpt.com/c/684eab1c-0ecc-8006-998a-5dfac04562c1

ASSUMPTION VIOLATION

Randomness: The data needs to be randomly sampled from the population.

 This assumption can be checked by examining the design of the experiment and the sampling strategy.
 If the randomness assumption is violated, we need to consider re-design the experiment and re-sample
the data.

Normality: The t-test assumes that both populations are normally distributed.
 The normality of the data can be checked using visualization such as histogram or QQ plot. It can also be
checked by statistical tests such as the Shapiro-Wilk test, the D’Agostino’s K-Squared test, or the
Kolmogorov-Smirnov test.
 When the normality assumption is violated, try data transformation such as logarithm or square root, and
check the normality of the transformed dataset.
 If the dataset is still not normally distributed after transformation, we can use the Wilcoxon test, a non-
parametric test that does not assume a normal data distribution.

Equal Variance: The population variances are the same.

 Equal variance can be checked using visualization such as boxplot or histogram. It can also be checked
by statistical tests such as Levene’s test or F test.
 The Welch test should be used when the equal variance assumption is violated and the normality
assumption is satisfied.
 When both the equal variance and the normality assumptions are violated, the non-parametric Wilcoxon
test should be used.

CODING
https://www.geeksforgeeks.org/t-test/

T-DISTRIBUTION
make me under the technical or conceptual understanding 'why though different t-test has different
formulas but all follow t-distribution'?
https://chatgpt.com/c/684eb2a6-e30c-8006-9d52-3e46216ab9c1

Z-TEST
CODING
https://www.geeksforgeeks.org/z-test/

Z-distribution
https://chatgpt.com/c/684eb63b-1a20-8006-a1ee-0fd33e27bc9c

chi-square
https://chatgpt.com/c/684eb7e9-4e90-8006-869c-76489a22b668
https://chatgpt.com/c/684eb7b6-8ce0-8006-a838-fd71c87ec132

ANOVA
https://www.datacamp.com/tutorial/anova-test
https://www.geeksforgeeks.org/maths/anova-formula/
https://chatgpt.com/c/684eb8c8-42ac-8006-a80b-2f0089558d74