logistic-regression
March 24, 2024
[2]: import pandas as pd
import numpy as np
from sklearn.linear_model import LogisticRegression #This is for logistic␣
↪regression
from sklearn.model_selection import train_test_split
from sklearn.metrics import confusion_matrix,␣
↪accuracy_score,classification_report,roc_curve,roc_auc_score #Metrics for␣
↪classification
import seaborn as sns
import matplotlib.pyplot as plt
from pandas_profiling import ProfileReport #pandas profiling for report␣
↪generation
[31]: #load dataset
data=pd.read_csv("C:/Users/ramaleer/Desktop/Practical 2/Datasets/Bank.CSV")
data.head()
[31]: age duration emp_var_rate cons_price_idx cons_conf_idx euribor3m \
0 44 210 1.4 93.444 -36.1 4.963
1 53 138 -0.1 93.200 -42.0 4.021
2 28 339 -1.7 94.055 -39.8 0.729
3 39 185 -1.8 93.075 -47.1 1.405
4 55 137 -2.9 92.201 -31.4 0.869
nr_employed y
0 5228.1 0
1 5195.8 0
2 4991.6 1
3 5099.1 0
4 5076.2 1
0.0.1 Data Description
• 1 - Age (numeric)
• 2 - Duration: last contact duration, in seconds (numeric)
• 3 - Emp.var.rate: employment variation rate - quarterly indicator (numeric)
• 4 - Cons.price.idx: consumer price index - monthly indicator (numeric)
1
• 5 - Cons.conf.idx: consumer confidence index - monthly indicator (numeric)
• 6 - Euribor3m: euribor 3 month rate - daily indicator (numeric)
• 7 - Nr.employed: number of employees - quarterly indicator (numeric)
The classification goal is to predict if the client will subscribe (yes/no) a term deposit (variable y).
Where variable y is encoded as yes = 1 and No = 0
[4]: #check if the data set is balanced or not.
data.y.value_counts()
#values show its imbalanced
[4]: 0 36548
1 4640
Name: y, dtype: int64
[5]: #get information about data
data.info()
<class 'pandas.core.frame.DataFrame'>
RangeIndex: 41188 entries, 0 to 41187
Data columns (total 8 columns):
# Column Non-Null Count Dtype
--- ------ -------------- -----
0 age 41188 non-null int64
1 duration 41188 non-null int64
2 emp_var_rate 41188 non-null float64
3 cons_price_idx 41188 non-null float64
4 cons_conf_idx 41188 non-null float64
5 euribor3m 41188 non-null float64
6 nr_employed 41188 non-null float64
7 y 41188 non-null int64
dtypes: float64(5), int64(3)
memory usage: 2.5 MB
[6]: #get the column and row count
print("Columns:",data.shape[1])
print("Rows:",data.shape[0])
Columns: 8
Rows: 41188
[7]: #define x, y
#Seperating independent data matrix & response vector
x = data.drop(columns = ['y'], axis=1) #independent variables selected
y = data.y #target vaiable is y
2
[8]: # Splitting data into training & testing sets (Validation set approach)
x_train,x_test,y_train,y_test=train_test_split(x,y,test_size=0.
↪2,random_state=0) # 20% of data is selected for test set-validation set
1 Creating the logistic regression model
[9]: model=LogisticRegression()
2 Training the model with training data
[10]: model.fit(x_train,y_train)
[10]: LogisticRegression()
3 Estimated coefficients for parameters and intercept of the model
[11]: model.coef_
[11]: array([[ 0.00102983, 0.00453544, -0.21667 , 0.4244128 , 0.05623385,
-0.27695426, -0.0078621 ]])
[12]: model.intercept_
[12]: array([0.00389662])
4 Predict the class of the unseen data
[13]: #here we use our test set for validation
y_pred=model.predict(x_test)
y_pred
[13]: array([0, 0, 0, …, 0, 0, 0], dtype=int64)
5 Predicted probabilities for each observation
[14]: #get the probabilities
y_pred_probs=model.predict_proba(x_test)
y_pred_probs #Left side column for 0 and right side column for 1
[14]: array([[0.93744048, 0.06255952],
[0.67198681, 0.32801319],
3
[0.9914955 , 0.0085045 ],
…,
[0.9921623 , 0.0078377 ],
[0.94365365, 0.05634635],
[0.99442537, 0.00557463]])
[15]: #probability is summation equals to 1
#Example:
0.93744108 + 0.06255892
#probability of outcome = 0 (no) and probability of outcome = 1 (yes)
[15]: 1.0
6 Confusion matrix
[16]: #obtain the confucion matrix
confusion_matrix(y_test,y_pred)
[16]: array([[7157, 168],
[ 606, 307]], dtype=int64)
[17]: sns.heatmap(confusion_matrix(y_test,y_pred),annot=True,fmt="g")
plt.xlabel("Predicted")
plt.ylabel("Actual")
plt.show()
4
7 Values inside the Confusion Matrix
[18]: tn,fp,fn,tp=confusion_matrix(y_test,y_pred).ravel()
[19]: tn #True Negatives : Negatives predicted correcly
[19]: 7157
[20]: fn #False Negatives : Predicted as No but Actualy its y= No.
[20]: 606
[21]: tp #True Positives : Positives predicted correcly
[21]: 307
[22]: fp #False Prositives : Predicted as positives but its actually false(because␣
↪its actually negative)
[22]: 168
5
8 Accuracy & Misclassification Error
[23]: #accuracy values from manual calculation
accuracy=(np.diag(confusion_matrix(y_test,y_pred)).sum())/len(y_test)
accuracy
[23]: 0.9060451565914057
[24]: #accuracy values: from function
accuracy_score(y_test,y_pred)
[24]: 0.9060451565914057
[25]: 1-accuracy_score(y_test,y_pred)
[25]: 0.09395484340859428
[26]: # MCE=miss classification error : incorrect classifications (FP and FN)
MCE=1-accuracy
MCE
[26]: 0.09395484340859428
9 Classification report with more metrics
[27]: print(classification_report(y_test,y_pred))
precision recall f1-score support
0 0.92 0.98 0.95 7325
1 0.65 0.34 0.44 913
accuracy 0.91 8238
macro avg 0.78 0.66 0.70 8238
weighted avg 0.89 0.91 0.89 8238
10 Receiver operating characteristic Curve (ROC Curve)
[28]: fpr, tpr, _ = roc_curve(y_test, y_pred_probs[:,1])
plt.plot(fpr,tpr)
plt.title("ROC Curve")
plt.show()
6
[29]: #Area under Curve
auc = roc_auc_score(y_test, y_pred_probs[:,1])
auc
[29]: 0.9121246014152794
[30]: # Homework : Check if there is a way for you to calculate the best threshold
# Homewrork : Experiment with other available classifier models on this Data to␣
↪classify the subscription of term deposit.
10.1 Generate Report using Pandas Profiling
[32]: profile = ProfileReport(data)
profile
Summarize dataset: 0%| | 0/5 [00:00<?, ?it/s]
Generate report structure: 0%| | 0/1 [00:00<?, ?it/s]
Render HTML: 0%| | 0/1 [00:00<?, ?it/s]
<IPython.core.display.HTML object>
7
[32]:
[ ]:
[ ]:
