assessment-test

November 19, 2024

1 Student Response data Analysis

1.1 Objective
1. How did the student’s ability to answer the questions change ?
2. Did the questions get difficult or easy?
3. Can you create a model that can predict if a student will answer a question correctly?
4. Note down any other observations you may have about the data.
[3]: # Importing Importan library
import pandas as pd
import pandas as pd
import plotly.graph_objects as go
import plotly.express as px
from scipy.stats import ttest_ind
from scipy.stats import shapiro
from scipy.stats import levene
from scipy.stats import chi2_contingency
from scipy.stats import pearsonr, spearmanr
from scipy.stats import wilcoxon
from scipy.stats import f_oneway
from scipy.stats import mannwhitneyu
from sklearn.model_selection import train_test_split
from sklearn.linear_model import LogisticRegression
from sklearn.ensemble import RandomForestClassifier, GradientBoostingClassifier
from sklearn.neighbors import KNeighborsClassifier
from sklearn.metrics import accuracy_score, precision_score, recall_score,␣
↪f1_score

[4]: # Load the CSV files for the years 2021 and 2022
data_2021 = pd.read_csv('C:/Users/nabee/Downloads/student_responses_2021.csv')
data_2022 = pd.read_csv('C:/Users/nabee/Downloads/student_responses_2022.csv')

[5]: # Display the first few rows and basic info of each dataset to understand their␣
↪structure and check for any anomalies

print(" ########## Student Data 2021 list Information ########## ")

print(data_2021.info())

1
print("\n")

print(" ########## Student Data 2021 list First Five Row ########## ")
print(data_2022.head())
print("\n")

print(" ########## Student Data 2021 list Information ########## ")

print(data_2022.info())
print("\n")

print(" ########## Student Data 2021 list First Five Row ########## ")
print(data_2022.head())

########## Student Data 2021 list Information ##########

<class 'pandas.core.frame.DataFrame'>
RangeIndex: 45000 entries, 0 to 44999
Data columns (total 5 columns):
# Column Non-Null Count Dtype
--- ------ -------------- -----
0 student_id 45000 non-null int64
1 question_id 45000 non-null int64
2 ability 44100 non-null float64
3 difficulty 44100 non-null float64
4 answered_correctly 45000 non-null bool
dtypes: bool(1), float64(2), int64(2)
memory usage: 1.4 MB
None

########## Student Data 2021 list First Five Row ##########

student_id question_id ability difficulty answered_correctly
0 1078 22 1.023237 -0.280792 True
1 1980 37 -2.243018 1.430540 False
2 1240 14 3.453396 -0.872714 True
3 1329 33 1.750230 0.816172 True
4 1995 17 2.288340 -0.558914 True

########## Student Data 2021 list Information ##########

<class 'pandas.core.frame.DataFrame'>
RangeIndex: 50000 entries, 0 to 49999
Data columns (total 5 columns):
# Column Non-Null Count Dtype
--- ------ -------------- -----
0 student_id 50000 non-null int64
1 question_id 50000 non-null int64
2 ability 49000 non-null float64

2
 3 difficulty 49000 non-null float64
4 answered_correctly 50000 non-null bool
dtypes: bool(1), float64(2), int64(2)
memory usage: 1.6 MB
None

########## Student Data 2021 list First Five Row ##########

student_id question_id ability difficulty answered_correctly
0 1078 22 1.023237 -0.280792 True
1 1980 37 -2.243018 1.430540 False
2 1240 14 3.453396 -0.872714 True
3 1329 33 1.750230 0.816172 True
4 1995 17 2.288340 -0.558914 True

1.1.1 Cecking the treands of ability and difficulty without handling missing value

[7]: # Combine the two datasets and add a 'year' column for easier year-based␣
↪analysis

data_2021['year'] = 2021
data_2022['year'] = 2022
combined_data = pd.concat([data_2021, data_2022], ignore_index=True)

# Analyze the average student ability and question difficulty by year

ability_trend = combined_data.groupby('year')['ability'].mean()
difficulty_trend = combined_data.groupby('year')['difficulty'].mean()

print(" ########## 2021 and 2022 Student's Ability Trend with missing value␣
↪########## ")

print(ability_trend)
print("\n")

print(" ########## 2021 and 2022 Student's Difficulty Trend with missing value␣
↪########## ")

print(difficulty_trend)
print("\n")

########## 2021 and 2022 Student's Ability Trend with missing value ##########
year
2021 -0.041824
2022 0.330433
Name: ability, dtype: float64

########## 2021 and 2022 Student's Difficulty Trend with missing value
##########
year

3
 2021 -0.057131
2022 0.116709
Name: difficulty, dtype: float64

[8]: # Plot average ability and difficulty by year using bar charts
fig1 = go.Figure()

fig1.add_trace(go.Bar(
x=ability_trend.index,
y=ability_trend.values,
name='Average Ability',
marker_color='cornflowerblue'
))

fig1.add_trace(go.Bar(
x=difficulty_trend.index,
y=difficulty_trend.values,
name='Average Difficulty',
marker_color='salmon'
))

fig1.update_layout(
title="Average Student Ability and Question Difficulty by Year",
xaxis_title="Year",
yaxis_title="Score",
barmode='group',
template='plotly_dark'
)

# Distribution of Ability Scores by Year (Cleaner Visualization)

fig2 = go.Figure()

fig2.add_trace(go.Histogram(
x=data_2021['ability'],
name='2021',
opacity=0.5,
marker_color='cornflowerblue',
nbinsx=20
))

fig2.add_trace(go.Histogram(
x=data_2022['ability'],
name='2022',
opacity=0.5,
marker_color='salmon',

4
 nbinsx=20
))

fig2.update_layout(
title="Distribution of Ability Scores in 2021 and 2022",
xaxis_title="Ability",
yaxis_title="Density",
barmode='overlay',
template='plotly_dark'
)

# Distribution of Difficulty Scores by Year (Cleaner Visualization)

fig3 = go.Figure()

fig3.add_trace(go.Histogram(
x=data_2021['difficulty'],
name='2021',
opacity=0.5,
marker_color='cornflowerblue',
nbinsx=20
))

fig3.add_trace(go.Histogram(
x=data_2022['difficulty'],
name='2022',
opacity=0.5,
marker_color='salmon',
nbinsx=20
))

fig3.update_layout(
title="Distribution of Difficulty Scores in 2021 and 2022",
xaxis_title="Difficulty",
yaxis_title="Density",
barmode='overlay',
template='plotly_dark'
)

# Show the plots

fig1.show()
fig2.show()
fig3.show()

5
 1.1.2 Cecking the treands of ability and difficulty after handling missing value

[10]: # Fill missing values with the mean using updated syntax to avoid warnings
data_2021_final = data_2021.assign(
ability=data_2021['ability'].fillna(data_2021['ability'].mean()),
difficulty=data_2021['difficulty'].fillna(data_2021['difficulty'].mean())
)

data_2022_final = data_2022.assign(
ability=data_2022['ability'].fillna(data_2022['ability'].mean()),
difficulty=data_2022['difficulty'].fillna(data_2022['difficulty'].mean())
)

# Add a year column for combined analysis

data_2021_final['year'] = 2021
data_2022_final['year'] = 2022
combined_data_final = pd.concat([data_2021_final, data_2022_final],␣
↪ignore_index=True)

# Average Ability and Difficulty by Year

average_ability_final = combined_data_final.groupby('year')['ability'].mean()
average_difficulty_final = combined_data_final.groupby('year')['difficulty'].
↪mean()

print(" ########## 2021 and 2022 Student's Ability Trend ########## ")
print(average_ability_final)
print("\n")

print(" ########## 2021 and 2022 Student's Difficulty Trend ########## ")
print(average_difficulty_final)
print("\n")

########## 2021 and 2022 Student's Ability Trend ##########

year
2021 -0.041824
2022 0.330433
Name: ability, dtype: float64

########## 2021 and 2022 Student's Difficulty Trend ##########

year
2021 -0.057131
2022 0.116709
Name: difficulty, dtype: float64

6
[11]: # Plot average ability and difficulty by year using bar charts
fig1 = go.Figure()

fig1.add_trace(go.Bar(
x=average_ability_final.index,
y=average_ability_final.values,
name='Average Ability',
marker_color='cornflowerblue'
))

fig1.add_trace(go.Bar(
x=average_difficulty_final.index,
y=average_difficulty_final.values,
name='Average Difficulty',
marker_color='salmon'
))

fig1.update_layout(
title="Average Student Ability and Question Difficulty by Year",
xaxis_title="Year",
yaxis_title="Score",
barmode='group',
template='plotly_dark'
)

# Distribution of Ability Scores by Year (Cleaner Visualization)

fig2 = go.Figure()

fig2.add_trace(go.Histogram(
x=data_2021_final['ability'],
name='2021',
opacity=0.5,
marker_color='cornflowerblue',
nbinsx=20
))

fig2.add_trace(go.Histogram(
x=data_2022_final['ability'],
name='2022',
opacity=0.5,
marker_color='salmon',
nbinsx=20
))

fig2.update_layout(
title="Distribution of Ability Scores in 2021 and 2022",
xaxis_title="Ability",

7
 yaxis_title="Density",
barmode='overlay',
template='plotly_dark'
)

# Distribution of Difficulty Scores by Year (Cleaner Visualization)

fig3 = go.Figure()

fig3.add_trace(go.Histogram(
x=data_2021_final['difficulty'],
name='2021',
opacity=0.5,
marker_color='cornflowerblue',
nbinsx=20
))

fig3.add_trace(go.Histogram(
x=data_2022_final['difficulty'],
name='2022',
opacity=0.5,
marker_color='salmon',
nbinsx=20
))

fig3.update_layout(
title="Distribution of Difficulty Scores in 2021 and 2022",
xaxis_title="Difficulty",
yaxis_title="Density",
barmode='overlay',
template='plotly_dark'
)

# Show the plots

fig1.show()
fig2.show()
fig3.show()

[12]: # Calculate the average answered_correctly for each student in 2021

student_performance_2021 = data_2021.
↪groupby('student_id')['answered_correctly'].mean().reset_index()

student_performance_2021 = student_performance_2021.
↪rename(columns={'answered_correctly': 'avg_correct_2021'})

# Find the top 10 and bottom 10 students

top_10_students = student_performance_2021.nlargest(10,␣
↪'avg_correct_2021')['student_id']

8
bottom_10_students = student_performance_2021.nsmallest(10,␣
↪'avg_correct_2021')['student_id']

# Get the ability of these students in 2021 and 2022

top_10_ability_2021 = data_2021[data_2021['student_id'].isin(top_10_students)].
↪groupby('student_id')['ability'].mean()

top_10_ability_2022 = data_2022[data_2022['student_id'].isin(top_10_students)].
↪groupby('student_id')['ability'].mean()

bottom_10_ability_2021 = data_2021[data_2021['student_id'].
↪isin(bottom_10_students)].groupby('student_id')['ability'].mean()

bottom_10_ability_2022 = data_2022[data_2022['student_id'].
↪isin(bottom_10_students)].groupby('student_id')['ability'].mean()

# Combine results into dataframes for comparison

top_10_comparison = pd.DataFrame({
'2021_ability': top_10_ability_2021,
'2022_ability': top_10_ability_2022
}).reset_index()

bottom_10_comparison = pd.DataFrame({
'2021_ability': bottom_10_ability_2021,
'2022_ability': bottom_10_ability_2022
}).reset_index()

# Display results
print("Top 10 Students Ability Comparison:")
print(top_10_comparison)

print("\nBottom 10 Students Ability Comparison:")

print(bottom_10_comparison)

Top 10 Students Ability Comparison:

student_id 2021_ability 2022_ability
0 4 1.784271 NaN
1 5 3.086568 NaN
2 6 1.960802 NaN
3 7 2.190667 NaN
4 22 2.937404 NaN
5 31 1.872750 NaN
6 33 3.889485 NaN
7 35 2.258828 NaN
8 38 4.105079 NaN
9 40 2.576292 NaN

Bottom 10 Students Ability Comparison:

student_id 2021_ability 2022_ability

9
 0 9 -3.104761 NaN
1 10 -3.958711 NaN
2 14 -2.726788 NaN
3 19 -2.163419 NaN
4 25 -2.602048 NaN
5 37 -5.302896 NaN
6 41 -2.389137 NaN
7 42 -2.415692 NaN
8 43 -4.696739 NaN
9 46 -2.334270 NaN

[13]: # Find common student IDs in both years

common_students = set(data_2021['student_id']).
↪intersection(set(data_2022['student_id']))

# Select the first 10 common student IDs

common_students_sample = list(common_students)[:10]

# Extract abilities for these students in 2021 and 2022

common_abilities_2021 = data_2021[data_2021['student_id'].
↪isin(common_students_sample)].groupby('student_id')['ability'].mean()

common_abilities_2022 = data_2022[data_2022['student_id'].
↪isin(common_students_sample)].groupby('student_id')['ability'].mean()

# Combine into a single dataframe for comparison

common_abilities_comparison = pd.DataFrame({
'2021_ability': common_abilities_2021,
'2022_ability': common_abilities_2022
}).reset_index()

# Display the comparison

print("Comparison of Abilities for Common Students in 2021 and 2022:")
print(common_abilities_comparison)

Comparison of Abilities for Common Students in 2021 and 2022:

Empty DataFrame
Columns: [student_id, 2021_ability, 2022_ability]
Index: []

[14]: # Visualize Top 10 Students

fig_top = go.Figure()

fig_top.add_trace(go.Bar(
x=top_10_comparison['student_id'],
y=top_10_comparison['2021_ability'],
name='2021 Ability',
marker_color='blue'

10
))

fig_top.add_trace(go.Bar(
x=top_10_comparison['student_id'],
y=top_10_comparison['2022_ability'],
name='2022 Ability',
marker_color='green'
))

fig_top.update_layout(
title="Top 10 Students: Ability Comparison (2021 vs 2022)",
xaxis_title="Student ID",
yaxis_title="Ability",
barmode='group',
template="plotly_white",
legend_title="Year"
)

fig_top.show()

# Visualize Bottom 10 Students

fig_bottom = go.Figure()

fig_bottom.add_trace(go.Bar(
x=bottom_10_comparison['student_id'],
y=bottom_10_comparison['2021_ability'],
name='2021 Ability',
marker_color='red'
))

fig_bottom.add_trace(go.Bar(
x=bottom_10_comparison['student_id'],
y=bottom_10_comparison['2022_ability'],
name='2022 Ability',
marker_color='orange'
))

fig_bottom.update_layout(
title="Bottom 10 Students: Ability Comparison (2021 vs 2022)",
xaxis_title="Student ID",
yaxis_title="Ability",
barmode='group',
template="plotly_white",
legend_title="Year"
)

fig_bottom.show()

11
[15]: # Perform t-tests for ability and difficulty
ability_ttest = ttest_ind(
data_2021_final['ability'],
data_2022_final['ability'],
equal_var=False
)
difficulty_ttest = ttest_ind(
data_2021_final['difficulty'],
data_2022_final['difficulty'],
equal_var=False
)

# Print t-test results

print("########## T-Test Results ##########")
print(f"Ability: t-statistic = {ability_ttest.statistic:.4f}, p-value =␣
↪{ability_ttest.pvalue:.4f}")

print(f"Difficulty: t-statistic = {difficulty_ttest.statistic:.4f}, p-value =␣

↪{difficulty_ttest.pvalue:.4f}")

########## T-Test Results ##########

Ability: t-statistic = -25.1121, p-value = 0.0000

12
 Difficulty: t-statistic = -24.3243, p-value = 0.0000

[16]: stat, p = shapiro(data_2021_final['ability'])

print(f"Shapiro-Wilk Test for 2021 Ability: Statistic={stat:.4f}, p-value={p:.
↪4f}")

Shapiro-Wilk Test for 2021 Ability: Statistic=0.9980, p-value=0.0000

C:\Users\nabee\anaconda3\Lib\site-packages\scipy\stats\_axis_nan_policy.py:531:
UserWarning:

scipy.stats.shapiro: For N > 5000, computed p-value may not be accurate. Current
N is 45000.

[17]: stat, p = levene(data_2021_final['ability'], data_2022_final['ability'])

print(f"Levene Test for Ability Variance: Statistic={stat:.4f}, p-value={p:.
↪4f}")

Levene Test for Ability Variance: Statistic=141.9266, p-value=0.0000

[18]: # Create a contingency table

contingency_table = pd.crosstab(data_2021_final['answered_correctly'],␣
↪data_2021_final['difficulty'])

# Perform the test

chi2, p, dof, expected = chi2_contingency(contingency_table)
print(f"Chi-Square Test: Statistic={chi2:.4f}, p-value={p:.4f}")

Chi-Square Test: Statistic=5668.9869, p-value=0.0000

[19]: pearson_corr, p_value = pearsonr(data_2021_final['ability'],␣

↪data_2021_final['difficulty'])

print(f"Pearson Correlation: {pearson_corr:.4f}, p-value={p_value:.4f}")

spearman_corr, p_value = spearmanr(data_2021_final['ability'],␣

↪data_2021_final['difficulty'])

print(f"Spearman Correlation: {spearman_corr:.4f}, p-value={p_value:.4f}")

Pearson Correlation: -0.0007, p-value=0.8798

Spearman Correlation: -0.0011, p-value=0.8158

[20]: stat, p = wilcoxon(common_abilities_comparison['2021_ability'],␣

↪common_abilities_comparison['2022_ability'])

print(f"Wilcoxon Signed-Rank Test: Statistic={stat:.4f}, p-value={p:.4f}")

Wilcoxon Signed-Rank Test: Statistic=nan, p-value=nan

13
[21]: stat, p = f_oneway(data_2021_final['ability'], data_2022_final['ability'])
print(f"One-Way ANOVA: Statistic={stat:.4f}, p-value={p:.4f}")

One-Way ANOVA: Statistic=622.6192, p-value=0.0000

[22]: stat, p = mannwhitneyu(data_2021_final['ability'], data_2022_final['ability'])

print(f"Mann-Whitney U Test: Statistic={stat:.4f}, p-value={p:.4f}")

Mann-Whitney U Test: Statistic=1040584864.0000, p-value=0.0000

[23]: # Shapiro-Wilk Test for 2021

shapiro_2021_ability = shapiro(data_2021_final['ability'])
shapiro_2021_difficulty = shapiro(data_2021_final['difficulty'])

# Shapiro-Wilk Test for 2022

shapiro_2022_ability = shapiro(data_2021_final['ability'])
shapiro_2022_difficulty = shapiro(data_2022_final['difficulty'])

print("Shapiro-Wilk Test Results:")

print(f"2021 Ability: Statistic={shapiro_2021_ability.statistic:.4f},␣
↪p-value={shapiro_2021_ability.pvalue:.4f}")

print(f"2021 Difficulty: Statistic={shapiro_2021_difficulty.statistic:.4f},␣

↪p-value={shapiro_2021_difficulty.pvalue:.4f}")

print(f"2022 Ability: Statistic={shapiro_2022_ability.statistic:.4f},␣

↪p-value={shapiro_2022_ability.pvalue:.4f}")

print(f"2022 Difficulty: Statistic={shapiro_2022_difficulty.statistic:.4f},␣

↪p-value={shapiro_2022_difficulty.pvalue:.4f}")

Shapiro-Wilk Test Results:

2021 Ability: Statistic=0.9980, p-value=0.0000
2021 Difficulty: Statistic=0.9684, p-value=0.0000
2022 Ability: Statistic=0.9980, p-value=0.0000
2022 Difficulty: Statistic=0.9565, p-value=0.0000
C:\Users\nabee\anaconda3\Lib\site-packages\scipy\stats\_axis_nan_policy.py:531:
UserWarning:

scipy.stats.shapiro: For N > 5000, computed p-value may not be accurate. Current
N is 45000.

C:\Users\nabee\anaconda3\Lib\site-packages\scipy\stats\_axis_nan_policy.py:531:
UserWarning:

scipy.stats.shapiro: For N > 5000, computed p-value may not be accurate. Current
N is 50000.

14
[24]: # Levene Test for Variance
levene_ability = levene(data_2021_final['ability'], data_2022_final['ability'])
levene_difficulty = levene(data_2021_final['difficulty'],␣
↪data_2022_final['difficulty'])

print("\nLevene Test Results:")

print(f"Ability: Statistic={levene_ability.statistic:.4f},␣
↪p-value={levene_ability.pvalue:.4f}")

print(f"Difficulty: Statistic={levene_difficulty.statistic:.4f},␣
↪p-value={levene_difficulty.pvalue:.4f}")

Levene Test Results:

Ability: Statistic=141.9266, p-value=0.0000
Difficulty: Statistic=1024.7584, p-value=0.0000

[25]: # Pearson Correlation (linear relationship)

pearson_corr_2021, p_pearson_2021 = pearsonr(data_2021_final['ability'],␣
↪data_2021_final['difficulty'])

pearson_corr_2022, p_pearson_2022 = pearsonr(data_2022_final['ability'],␣

↪data_2022_final['difficulty'])

# Spearman Correlation (non-linear or ordinal)

spearman_corr_2021, p_spearman_2021 = spearmanr(data_2021_final['ability'],␣
↪data_2021_final['difficulty'])

spearman_corr_2022, p_spearman_2022 = spearmanr(data_2022_final['ability'],␣

↪data_2022_final['difficulty'])

print("\nCorrelation Analysis:")
print(f"2021 Pearson Correlation: {pearson_corr_2021:.4f},␣
↪p-value={p_pearson_2021:.4f}")

print(f"2022 Pearson Correlation: {pearson_corr_2022:.4f},␣

↪p-value={p_pearson_2022:.4f}")

print(f"2021 Spearman Correlation: {spearman_corr_2021:.4f},␣

↪p-value={p_spearman_2021:.4f}")

print(f"2022 Spearman Correlation: {spearman_corr_2022:.4f},␣

↪p-value={p_spearman_2022:.4f}")

Correlation Analysis:
2021 Pearson Correlation: -0.0007, p-value=0.8798
2022 Pearson Correlation: -0.0004, p-value=0.9234
2021 Spearman Correlation: -0.0011, p-value=0.8158
2022 Spearman Correlation: -0.0000, p-value=0.9944

15
[26]: # Paired students in both years
paired_abilities = pd.merge(data_2021_final[['student_id', 'ability']],
data_2022_final[['student_id', 'ability']],
on='student_id', suffixes=('_2021', '_2022'))

# Wilcoxon Test
wilcoxon_stat, wilcoxon_p = wilcoxon(paired_abilities['ability_2021'],␣
↪paired_abilities['ability_2022'])

print("\nWilcoxon Signed-Rank Test:")

print(f"Statistic={wilcoxon_stat:.4f}, p-value={wilcoxon_p:.4f}")

Wilcoxon Signed-Rank Test:

Statistic=nan, p-value=nan

[27]: # Divide students into high and low performers

high_performers = data_2021_final[data_2021_final['answered_correctly'] ==␣
↪1]['ability']

low_performers = data_2021_final[data_2021_final['answered_correctly'] ==␣

↪0]['ability']

# Mann-Whitney U Test
mannwhitney_stat, mannwhitney_p = mannwhitneyu(high_performers, low_performers)

print("\nMann-Whitney U Test:")
print(f"Statistic={mannwhitney_stat:.4f}, p-value={mannwhitney_p:.4f}")

Mann-Whitney U Test:
Statistic=471964431.0000, p-value=0.0000

2 Conclusion from Tests of Distribution¶

2.0.1 1. Shapiro-Wilk Test (Normality)
• For 2021 and 2022, both ability and difficulty severely failed the normality test with the
respective p-value = 0.0000.

• This, therefore, means that the application of non-parametric tests best suits these variables.

2.0.2 2. Correlation Analysis

Pearson and Spearman Correlation Results:
• For 2021 and 2022, ability and difficulty bear no correlation at a statistically significant level
as indicated by the p-values > 0.05.

16
 • The correlation coefficients are near zero indicating that there is no significant linear or
monotonic relationship between the two variables.

2.0.3 3. Levene’s Test (Equality of Variances)

• The analysis produces a result which indicates that the variances associated with ability and
difficulty of 2021 and 2022 are significantly different with a p-value of 0.0000.

• It indicates that the spread in ability and difficulty has changed significantly over the two
years.

2.0.4 4. Independent Samples Mann-Whitney U Test

• The p-value is significant at 0.0000 for the ability scores that seem to be distributed otherwise
between high and low performers.

• This implies that the student performance levels, between high and low, have a sharp and
valid discrimination by their ability.

2.0.5 5. One-Way ANOVA

• The p-value 0.0000 indicates that there should be differences in the means between the groups
for the data set analyzed and provides a variance for both ability and difficulty in the different
groups.

2.0.6 6. T-Test Outputs

• There is a significant difference between the abilities and difficulties from 2021 and 2022; the
p-value = 0.0000.

• This indicates that student abilities and question difficulties have significantly changed during
these years.

2.0.7 7. Chi-Square Test

• The test has p-value = 0.0000, thus it implies that there is strong association between
categorical variables such as answered_correctly and others that are available in the dataset.

• This may indicate some interactions or relationships worth exploring more with the data.

2.0.8 8. Wilcoxon Signed-Rank Test

• The test failed since it returned NaN values because there was either missing or incompatible
data for certain samples.

• Pairwise samples will need caution when handled.

2.0.9 Analysis of Trends in Ability and Difficulty (2021–2022)

Trend in Student Ability:

17
 • Students were better prepared and better able in 2022 compared with 2021. This
has been observed from the increase in average ability from -0.0418 to 0.3304 in
2022..
• This might be due to better teaching, wide resource usage, or getting familiar
with the question format..

Trend in Question Difficulty:

• Questions were difficult in their level of difficulty, and there was an increase from
-0.0571 in 2021 to 0.1167 in 2022. This means that questions got increasingly
tough with time.
• This could be intended as a way of maintaining exam balance by keeping up the
rising levels of student competence, such that assessments fairly measure what is
understood and skills.

2.0.10 Effects of Missing Values

• Comparative trends for both the with-handling and without-handling of missing
values were found to be consistent. This is to say that the approach used in
handling missing data does not feature in the overall trends.
• Conclusion: Both student ability and question difficulty trends are reliable even in the
consideration of missing data.

2.0.11 Implications
1. Assessment Design:
• This interaction between increasing question difficulty and improving student
ability suggests a shift in assessment design. If question difficulty continues to
rise, it will be very important to continue monitoring student ability trends
so as to not allow assessments to become too difficult or unbalanced.
2. Instructional Strategy:
• Recognizing these patterns can help educators adjust curriculum difficulty,
focus on areas for improvement, and design assessments that challenge stu-
dents while remaining achievable as skills evolve..

3 Key Insights
3.0.1 Student Ability and Question Difficulty Over Time:
• Student ability and question difficulty both differ substantially between 2021 and 2022.

• These metrics are more inconsistent than in previous years, suggesting that tests and student
readiness may have shifted.

18
 3.0.2 No Correlation Between Ability and Difficulty:
• A lack of correlation implies that greater difficulty does not indicate lower ability and vice
versa.

3.0.3 Difference for High and Low Performers

• High and low performers are vastly different in their abilities, meaning that there are indeed
different preparation or skill levels among students.

3.0.4 Implications for Assessment Design:

• Mean differences between high and low performers suggest the importance of balanced test
design to prevent biased tests.

• Results suggest a need for specific intervention programs to improve low-achieving students.

3.0.5 Data Arangment for Machine learning

[32]: # Features and target

X = combined_data_final[['student_id','ability', 'difficulty','question_id']]
y = combined_data_final['answered_correctly']

# Train a Random Forest model

from sklearn.ensemble import RandomForestClassifier
rf_model = RandomForestClassifier(random_state=42)
rf_model.fit(X, y)

# Feature importance
importance = pd.DataFrame({
'Feature': ['student_id','ability', 'difficulty','question_id'],
'Importance': rf_model.feature_importances_
}).sort_values(by='Importance', ascending=False)

print("########## Feature Importance ##########")

print(importance)

########## Feature Importance ##########

Feature Importance
1 ability 0.742419
2 difficulty 0.130521
3 question_id 0.108167
0 student_id 0.018893

[33]: # Split the data: Train on 2021, Test on 2022

X_train = X[combined_data_final['year'] == 2021]
y_train = y[combined_data_final['year'] == 2021]
X_test = X[combined_data_final['year'] == 2022]
y_test = y[combined_data_final['year'] == 2022]

19
 3.0.6 Here we iniatialize four type of ML models
1. Logistic Regression
2. Random Forest
3. Gradient Boosting
4. K-Nearest Neighbors
[35]: # Initialize Logistic Regression
log_reg = LogisticRegression()
log_reg.fit(X_train, y_train)

# Predictions
y_pred_log_reg = log_reg.predict(X_test)

# Performance metrics
log_reg_performance = {
"Model": "Logistic Regression",
"Accuracy": accuracy_score(y_test, y_pred_log_reg),
"Precision": precision_score(y_test, y_pred_log_reg),
"Recall": recall_score(y_test, y_pred_log_reg),
"F1 Score": f1_score(y_test, y_pred_log_reg)
}

[36]: # Initialize Random Forest

rf = RandomForestClassifier(random_state=42)
rf.fit(X_train, y_train)

# Predictions
y_pred_rf = rf.predict(X_test)

# Performance metrics
rf_performance = {
"Model": "Random Forest",
"Accuracy": accuracy_score(y_test, y_pred_rf),
"Precision": precision_score(y_test, y_pred_rf),
"Recall": recall_score(y_test, y_pred_rf),
"F1 Score": f1_score(y_test, y_pred_rf)
}

[37]: # Initialize Gradient Boosting

gb = GradientBoostingClassifier(random_state=42)
gb.fit(X_train, y_train)

# Predictions
y_pred_gb = gb.predict(X_test)

# Performance metrics
gb_performance = {

20
 "Model": "Gradient Boosting",
"Accuracy": accuracy_score(y_test, y_pred_gb),
"Precision": precision_score(y_test, y_pred_gb),
"Recall": recall_score(y_test, y_pred_gb),
"F1 Score": f1_score(y_test, y_pred_gb)
}

[38]: # Initialize K-Nearest Neighbors

knn = KNeighborsClassifier()
knn.fit(X_train, y_train)

# Predictions
y_pred_knn = knn.predict(X_test)

# Performance metrics
knn_performance = {
"Model": "K-Nearest Neighbors",
"Accuracy": accuracy_score(y_test, y_pred_knn),
"Precision": precision_score(y_test, y_pred_knn),
"Recall": recall_score(y_test, y_pred_knn),
"F1 Score": f1_score(y_test, y_pred_knn)
}

3.0.7 Ploting graph to show different evalution of avove models

[40]: # Combine performance metrics
performance_df = pd.DataFrame([log_reg_performance, rf_performance,␣
↪gb_performance, knn_performance])

# Create a comparison graph in Plotly

fig = go.Figure()

# Add traces for each metric

metrics = ["Accuracy", "Precision", "Recall", "F1 Score"]
for metric in metrics:
fig.add_trace(go.Bar(
x=performance_df["Model"],
y=performance_df[metric],
name=metric
))

# Update layout for readability

fig.update_layout(
title="Model Performance Comparison",
xaxis_title="Model",
yaxis_title="Score",
barmode='group',

21
 template="plotly_dark",
legend_title_text="Metrics",
width=800,
height=500
)

fig.show()

3.0.8 Analysis of Model Performance

Based on the performance metrics:
• Logistic Regression has the highest Accuracy (0.98746) and F1 Score (0.987981), slightly
outperforming the other models in both categories.
• Random Forest and K-Nearest Neighbors (KNN) are close behind, with F1 Scores
of 0.984718 and 0.984441, respectively.
• Gradient Boosting has the lowest F1 Score among the models (0.982488), although it’s
still quite close to the others.

3.1 Conclusion
Logistic Regression performs the best overall, making it the preferred model based on
both accuracy and F1 Score. This model likely performs well because the relationship between
ability, difficulty, and the likelihood of answering correctly might be nearly linear, making
logistic regression an effective choice.
In a practical application, Logistic Regression would be favored here, but since the other models
perform very closely, Random Forest or KNN could also be considered, especially if the
underlying relationships prove to be more complex in future data.
[74]: !choco install pandoc

'choco' is not recognized as an internal or external command,

operable program or batch file.

[1]: pandoc --version

---------------------------------------------------------------------------
NameError Traceback (most recent call last)
Cell In[1], line 1
----> 1 pandoc --version

NameError: name 'pandoc' is not defined

[ ]:

22