Linear Regression study on the 3D printing dataset

# Importing the libraries

import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
import seaborn as sns

#To enable matplot visualization

%matplotlib inline

#Importing the Dataset using .read_csv() function of pandas

#We will call the dataset as 'printer'

printer = pd.read_csv('../input/3dprinter/data.csv')
#Let's check few rows of the dataset using the .head() function of pandas
printer.head()
 1. Layer Height in mm
2. Wall Thickness in mm
3. Infill Density in %
4. Infill Pattern in either Grid or Honeycomb
5. Nozzle Temperature in Degree C
6. Bed Temperature in degree C
7. Print speed in mm/s
8. Material in either abs or pla
9. Fan Speed in %

# Basic information of the dataset using .info() function of pandas

printer.info()

<class 'pandas.core.frame.DataFrame'>
RangeIndex: 50 entries, 0 to 49
Data columns (total 12 columns):
# Column Non-Null Count Dtype
--- ------ -------------- -----
0 layer_height 50 non-null float64
1 wall_thickness 50 non-null int64
2 infill_density 50 non-null int64
3 infill_pattern 50 non-null object
4 nozzle_temperature 50 non-null int64
5 bed_temperature 50 non-null int64
6 print_speed 50 non-null int64
7 material 50 non-null object
8 fan_speed 50 non-null int64
9 roughness 50 non-null int64
10 tension_strenght 50 non-null int64
11 elongation 50 non-null float64
dtypes: float64(2), int64(8), object(2)
memory usage: 4.8+ KB

printer['infill_pattern'].replace(['grid','honeycomb'], [0,1], inplace = True)

printer['material'].replace(['abs','pla'], [0,1], inplace = True)
In [7]:
linkcode
#let's view the first 10 observations
printer.head(10)
# Creating the heatmap
# First create a space for the heatmap and then draw the heatmap inside the space

fig, ax = plt.subplots(figsize = (20,7))

#Title for the heatmap

title = 'Correlation Heatmap of the 3D printing Dataset'
plt.title(title, fontsize = 20)
ttl = ax.title

# Correlation heatmap using .heatmap() function of sns library

sns.heatmap(printer.corr(), cbar = True, cmap = 'cool', annot = True,
linewidths = 1, ax = ax)

#enable visualization using .show() function of matplot

plt.show()
# Defining features and labels
X = printer.drop(['roughness','tension_strenght','elongation'], axis = 1)
y = printer['roughness']
In [10]:
linkcode
X.head()

#importing statsmodels library

import statsmodels.api as sm
In [12]:
linkcode
# let's define a function for the multiple regression

def linear_Regression(x,y):

x = sm.add_constant(x)

#defining the model, fitting the model and printing the results
multiple_model = sm.OLS(y,x).fit()
print(multiple_model.summary())
#calling the linear regression function
linear_Regression(X,y)
OLS Regression Results
=======================================================================
=======
Dep. Variable: roughness R-squared:
0.875
Model: OLS Adj. R-squared:
0.851
Method: Least Squares F-statistic:
35.95
Date: Thu, 07 Jul 2022 Prob (F-statistic):
3.83e-16
Time: 12:50:37 Log-Likelihood:
-248.19
No. Observations: 50 AIC:
514.4
Df Residuals: 41 BIC:
531.6
Df Model: 8
Covariance Type: nonrobust
=======================================================================
===============
coef std err t P>|t|
[0.025 0.975]
-----------------------------------------------------------------------
---------------
const -0.9534 0.159 -6.006 0.000 -
1.274 -0.633
layer_height 1269.4449 87.648 14.483 0.000
1092.437 1446.453
wall_thickness 2.3342 2.189 1.066 0.293 -
2.087 6.756
infill_density -0.0423 0.234 -0.181 0.857 -
0.515 0.430
infill_pattern -0.1255 11.280 -0.011 0.991 -
22.907 22.656
nozzle_temperature 15.0562 2.529 5.953 0.000
9.948 20.164
bed_temperature -55.6225 9.279 -5.995 0.000 -
74.362 -36.883
print_speed 0.6496 0.206 3.153 0.003
0.233 1.066
material 298.4514 58.364 5.114 0.000
180.582 416.321
fan_speed 7.8989 1.238 6.379 0.000
5.398 10.399
=======================================================================
=======
Omnibus: 1.107 Durbin-Watson:
1.467
Prob(Omnibus): 0.575 Jarque-Bera (JB):
0.749
Skew: 0.300 Prob(JB):
0.688
Kurtosis: 3.018 Cond. No.
3.59e+18
=======================================================================
=======

Notes:
[1] Standard Errors assume that the covariance matrix of the errors is
correctly specified.
[2] The smallest eigenvalue is 2.48e-31. This might indicate that there
are
strong multicollinearity problems or that the design matrix is
singular.

#check X
X.head()

X = X.drop(['wall_thickness','infill_density','infill_pattern'], axis = 1)
In [16]:
linkcode
X.head()
#calling the linear regression function
linear_Regression(X,y)
OLS Regression Results
=======================================================================
=======
Dep. Variable: roughness R-squared:
0.872
Model: OLS Adj. R-squared:
0.857
Method: Least Squares F-statistic:
59.78
Date: Thu, 07 Jul 2022 Prob (F-statistic):
1.67e-18
Time: 12:50:37 Log-Likelihood:
-248.88
No. Observations: 50 AIC:
509.8
Df Residuals: 44 BIC:
521.2
Df Model: 5
Covariance Type: nonrobust
=======================================================================
===============
coef std err t P>|t|
[0.025 0.975]
-----------------------------------------------------------------------
---------------
const -0.9307 0.151 -6.172 0.000 -
1.235 -0.627
layer_height 1246.5353 83.178 14.986 0.000
1078.901 1414.169
nozzle_temperature 14.7774 2.398 6.163 0.000
9.945 19.610
bed_temperature -54.3045 8.815 -6.160 0.000 -
72.070 -36.539
print_speed 0.5538 0.180 3.070 0.004
0.190 0.917
material 294.1610 56.159 5.238 0.000
180.981 407.341
fan_speed 7.6993 1.177 6.542 0.000
5.328 10.071
=======================================================================
=======
Omnibus: 0.850 Durbin-Watson:
1.367
Prob(Omnibus): 0.654 Jarque-Bera (JB):
0.720
Skew: 0.285 Prob(JB):
0.698
Kurtosis: 2.857 Cond. No.
3.59e+18
=======================================================================
=======

Notes:
[1] Standard Errors assume that the covariance matrix of the errors is
correctly specified.
[2] The smallest eigenvalue is 2.37e-31. This might indicate that there
are
strong multicollinearity problems or that the design matrix is
singular.

X.head()

#get the interaction terms by multiplying values

inter_mn = X['material']*X['nozzle_temperature']
inter_bn = X['bed_temperature']*X['nozzle_temperature']
inter_fn = X['fan_speed']*X['nozzle_temperature']
inter_fb = X['fan_speed']*X['bed_temperature']
In [21]:
linkcode
#adding these interaction terms to dataset using .concat() function of pandas
#we will call this dataset as interaction

interaction = pd.concat([X,inter_mn,inter_bn,inter_fn,inter_fb], axis = 1)

#chenge column names of this interaction terms

interaction = interaction.rename(columns = {0:'interct_mn', 1:'interact_bn',
2:'interact_fn',
3:'interact_fb'})

interaction.head(10)
# NOw let's fit this model to the linear regression function

linear_Regression(interaction,y)
OLS Regression Results
=======================================================================
=======
Dep. Variable: roughness R-squared:
0.925
Model: OLS Adj. R-squared:
0.910
Method: Least Squares F-statistic:
63.22
Date: Thu, 07 Jul 2022 Prob (F-statistic):
1.31e-20
Time: 12:50:38 Log-Likelihood:
-235.45
No. Observations: 50 AIC:
488.9
Df Residuals: 41 BIC:
506.1
Df Model: 8
Covariance Type: nonrobust
=======================================================================
===============
coef std err t P>|t|
[0.025 0.975]
-----------------------------------------------------------------------
---------------
const -1.0735 0.923 -1.163 0.252 -
2.938 0.791
layer_height 1246.5353 65.870 18.924 0.000
1113.508 1379.562
nozzle_temperature 0.0050 0.004 1.146 0.259 -
0.004 0.014
bed_temperature -57.8032 50.004 -1.156 0.254 -
158.788 43.182
print_speed 0.5538 0.143 3.877 0.000
0.265 0.842
material 5516.1791 2836.453 1.945 0.059 -
212.153 1.12e+04
fan_speed 33.0427 27.526 1.200 0.237 -
22.546 88.632
interct_mn -25.7036 12.956 -1.984 0.054 -
51.869 0.462
interact_bn 0.2588 0.227 1.138 0.262 -
0.201 0.718
interact_fn -0.2032 0.174 -1.169 0.249 -
0.554 0.148
interact_fb 0.1662 0.138 1.206 0.235 -
0.112 0.444
=======================================================================
=======
Omnibus: 0.934 Durbin-Watson:
1.345
Prob(Omnibus): 0.627 Jarque-Bera (JB):
0.333
Skew: 0.132 Prob(JB):
0.847
Kurtosis: 3.300 Cond. No.
4.80e+20
=======================================================================
=======

Notes:
[1] Standard Errors assume that the covariance matrix of the errors is
correctly specified.
[2] The smallest eigenvalue is 9.58e-32. This might indicate that there
are
strong multicollinearity problems or that the design matrix is
singular.

X = printer.drop(['roughness','tension_strenght','elongation'], axis = 1)
X.head()
y = printer['tension_strenght']
y
Out[24]:
0 18
1 16
2 8
3 10
4 5
5 24
6 12
7 14
8 27
9 25
10 37
11 12
12 16
13 9
14 10
15 27
16 23
17 26
18 33
19 29
20 16
21 12
22 10
23 19
24 8
25 11
26 12
27 18
28 34
29 14
30 27
31 19
32 18
33 9
34 13
35 33
36 24
37 26
38 22
39 4
40 35
41 34
42 28
43 28
44 21
45 28
46 14
47 30
48 29
49 27
Name: tension_strenght, dtype: int64
In [25]:
linkcode
linear_Regression(X,y)
OLS Regression Results
=======================================================================
=======
Dep. Variable: tension_strenght R-squared:
0.673
Model: OLS Adj. R-squared:
0.609
Method: Least Squares F-statistic:
10.55
Date: Thu, 07 Jul 2022 Prob (F-statistic):
6.91e-08
Time: 12:50:38 Log-Likelihood:
-151.94
No. Observations: 50 AIC:
321.9
Df Residuals: 41 BIC:
339.1
Df Model: 8
Covariance Type: nonrobust
=======================================================================
===============
coef std err t P>|t|
[0.025 0.975]
-----------------------------------------------------------------------
---------------
const 0.0663 0.023 2.863 0.007
0.020 0.113
layer_height 55.5972 12.788 4.348 0.000
29.772 81.423
wall_thickness 1.0687 0.319 3.346 0.002
0.424 1.714
infill_density 0.1629 0.034 4.769 0.000
0.094 0.232
infill_pattern -1.1427 1.646 -0.694 0.491 -
4.466 2.181
nozzle_temperature -1.0468 0.369 -2.837 0.007 -
1.792 -0.302
bed_temperature 3.8647 1.354 2.855 0.007
1.131 6.599
print_speed -0.0156 0.030 -0.519 0.607 -
0.076 0.045
material -17.3051 8.515 -2.032 0.049 -
34.502 -0.108
fan_speed -0.5719 0.181 -3.166 0.003 -
0.937 -0.207
=======================================================================
=======
Omnibus: 0.265 Durbin-Watson:
1.461
Prob(Omnibus): 0.876 Jarque-Bera (JB):
0.456
Skew: 0.060 Prob(JB):
0.796
Kurtosis: 2.548 Cond. No.
3.59e+18
=======================================================================
=======

Notes:
[1] Standard Errors assume that the covariance matrix of the errors is
correctly specified.
[2] The smallest eigenvalue is 2.48e-31. This might indicate that there
are
strong multicollinearity problems or that the design matrix is
singular.

X = X.drop(['infill_pattern','print_speed','material'], axis = 1)
X.head()

linear_Regression(X,y)
OLS Regression Results
=======================================================================
=======
Dep. Variable: tension_strenght R-squared:
0.634
Model: OLS Adj. R-squared:
0.593
Method: Least Squares F-statistic:
15.27
Date: Thu, 07 Jul 2022 Prob (F-statistic):
1.07e-08
Time: 12:50:38 Log-Likelihood:
-154.73
No. Observations: 50 AIC:
321.5
Df Residuals: 44 BIC:
332.9
Df Model: 5
Covariance Type: nonrobust
=======================================================================
===============
coef std err t P>|t|
[0.025 0.975]
-----------------------------------------------------------------------
---------------
const 0.0191 0.004 4.358 0.000
0.010 0.028
layer_height 56.6997 12.886 4.400 0.000
30.730 82.669
wall_thickness 1.1478 0.290 3.965 0.000
0.564 1.731
infill_density 0.1543 0.034 4.537 0.000
0.086 0.223
nozzle_temperature -0.3028 0.073 -4.141 0.000 -
0.450 -0.155
bed_temperature 1.1021 0.255 4.318 0.000
0.588 1.617
fan_speed -0.2052 0.043 -4.812 0.000 -
0.291 -0.119
=======================================================================
=======
Omnibus: 0.429 Durbin-Watson:
1.310
Prob(Omnibus): 0.807 Jarque-Bera (JB):
0.404
Skew: 0.202 Prob(JB):
0.817
Kurtosis: 2.824 Cond. No.
3.43e+18
=======================================================================
=======

Notes:
[1] Standard Errors assume that the covariance matrix of the errors is
correctly specified.
[2] The smallest eigenvalue is 2.54e-31. This might indicate that there
are
strong multicollinearity problems or that the design matrix is
singular.

interaction = pd.concat([X,inter_mn,inter_bn,inter_fn,inter_fb], axis = 1)

interaction = interaction.rename(columns = {0:'interct_mn', 1:'interact_bn',

2:'interact_fn',
3:'interact_fb'})
interaction.head()

linear_Regression(interaction,y)
OLS Regression Results
=======================================================================
=======
Dep. Variable: tension_strenght R-squared:
0.718
Model: OLS Adj. R-squared:
0.663
Method: Least Squares F-statistic:
13.03
Date: Thu, 07 Jul 2022 Prob (F-statistic):
4.06e-09
Time: 12:50:38 Log-Likelihood:
-148.27
No. Observations: 50 AIC:
314.5
Df Residuals: 41 BIC:
331.7
Df Model: 8
Covariance Type: nonrobust
=======================================================================
===============
coef std err t P>|t|
[0.025 0.975]
-----------------------------------------------------------------------
---------------
const -0.0870 0.068 -1.280 0.208 -
0.224 0.050
layer_height 56.6017 11.739 4.822 0.000
32.895 80.309
wall_thickness 1.1373 0.268 4.242 0.000
0.596 1.679
infill_density 0.1587 0.032 4.928 0.000
0.094 0.224
nozzle_temperature 0.0004 0.000 1.319 0.194 -
0.000 0.001
bed_temperature -4.7917 3.832 -1.251 0.218 -
12.530 2.946
fan_speed 2.1355 1.273 1.678 0.101 -
0.435 4.706
interct_mn 0.1191 0.096 1.235 0.224 -
0.076 0.314
interact_bn 0.0221 0.017 1.263 0.214 -
0.013 0.057
interact_fn -0.0138 0.007 -2.022 0.050 -
0.028 -1.34e-05
interact_fb 0.0077 0.004 1.830 0.075 -
0.001 0.016
=======================================================================
=======
Omnibus: 0.134 Durbin-Watson:
1.472
Prob(Omnibus): 0.935 Jarque-Bera (JB):
0.207
Skew: -0.114 Prob(JB):
0.902
Kurtosis: 2.782 Cond. No.
5.90e+20
=======================================================================
=======

Notes:
[1] Standard Errors assume that the covariance matrix of the errors is
correctly specified.
[2] The smallest eigenvalue is 6.34e-32. This might indicate that there
are
strong multicollinearity problems or that the design matrix is
singular.

printer.head()

linear_Regression(X,y)
OLS Regression Results
=======================================================================
=======
Dep. Variable: tension_strenght R-squared:
0.673
Model: OLS Adj. R-squared:
0.609
Method: Least Squares F-statistic:
10.55
Date: Thu, 07 Jul 2022 Prob (F-statistic):
6.91e-08
Time: 12:50:39 Log-Likelihood:
-151.94
No. Observations: 50 AIC:
321.9
Df Residuals: 41 BIC:
339.1
Df Model: 8
Covariance Type: nonrobust
=======================================================================
===============
coef std err t P>|t|
[0.025 0.975]
-----------------------------------------------------------------------
---------------
const 0.0663 0.023 2.863 0.007
0.020 0.113
layer_height 55.5972 12.788 4.348 0.000
29.772 81.423
wall_thickness 1.0687 0.319 3.346 0.002
0.424 1.714
infill_density 0.1629 0.034 4.769 0.000
0.094 0.232
infill_pattern -1.1427 1.646 -0.694 0.491 -
4.466 2.181
nozzle_temperature -1.0468 0.369 -2.837 0.007 -
1.792 -0.302
bed_temperature 3.8647 1.354 2.855 0.007
1.131 6.599
print_speed -0.0156 0.030 -0.519 0.607 -
0.076 0.045
material -17.3051 8.515 -2.032 0.049 -
34.502 -0.108
fan_speed -0.5719 0.181 -3.166 0.003 -
0.937 -0.207
=======================================================================
=======
Omnibus: 0.265 Durbin-Watson:
1.461
Prob(Omnibus): 0.876 Jarque-Bera (JB):
0.456
Skew: 0.060 Prob(JB):
0.796
Kurtosis: 2.548 Cond. No.
3.59e+18
=======================================================================
=======

Notes:
[1] Standard Errors assume that the covariance matrix of the errors is
correctly specified.
[2] The smallest eigenvalue is 2.48e-31. This might indicate that there
are
strong multicollinearity problems or that the design matrix is
singular.
In [33]:
linkcode
X = X.drop(['infill_pattern','print_speed'], axis = 1)
X.head()

linear_Regression(X,y)
OLS Regression Results
=======================================================================
=======
Dep. Variable: tension_strenght R-squared:
0.667
Model: OLS Adj. R-squared:
0.620
Method: Least Squares F-statistic:
14.33
Date: Thu, 07 Jul 2022 Prob (F-statistic):
6.78e-09
Time: 12:50:39 Log-Likelihood:
-152.42
No. Observations: 50 AIC:
318.8
Df Residuals: 43 BIC:
332.2
Df Model: 6
Covariance Type: nonrobust
=======================================================================
===============
coef std err t P>|t|
[0.025 0.975]
-----------------------------------------------------------------------
---------------
const 0.0646 0.023 2.840 0.007
0.019 0.111
layer_height 56.3670 12.448 4.528 0.000
31.262 81.472
wall_thickness 1.1117 0.280 3.967 0.000
0.547 1.677
infill_density 0.1664 0.033 4.985 0.000
0.099 0.234
nozzle_temperature -1.0289 0.363 -2.833 0.007 -
1.761 -0.296
bed_temperature 3.7661 1.330 2.831 0.007
1.084 6.449
material -17.1036 8.392 -2.038 0.048 -
34.028 -0.180
fan_speed -0.5565 0.177 -3.140 0.003 -
0.914 -0.199
=======================================================================
=======
Omnibus: 0.578 Durbin-Watson:
1.501
Prob(Omnibus): 0.749 Jarque-Bera (JB):
0.710
Skew: 0.195 Prob(JB):
0.701
Kurtosis: 2.566 Cond. No.
5.42e+18
=======================================================================
=======

Notes:
[1] Standard Errors assume that the covariance matrix of the errors is
correctly specified.
[2] The smallest eigenvalue is 1.02e-31. This might indicate that there
are
strong multicollinearity problems or that the design matrix is
singular.
In [35]:
linkcode
interaction = pd.concat([X,inter_mn,inter_bn,inter_fn,inter_fb], axis = 1)

interaction = interaction.rename(columns = {0:'interct_mn', 1:'interact_bn',

2:'interact_fn',
3:'interact_fb'})

interaction.head()
linear_Regression(interaction,y)
OLS Regression Results
=======================================================================
=======
Dep. Variable: tension_strenght R-squared:
0.718
Model: OLS Adj. R-squared:
0.654
Method: Least Squares F-statistic:
11.30
Date: Thu, 07 Jul 2022 Prob (F-statistic):
1.54e-08
Time: 12:50:39 Log-Likelihood:
-148.26
No. Observations: 50 AIC:
316.5
Df Residuals: 40 BIC:
335.6
Df Model: 9
Covariance Type: nonrobust
=======================================================================
===============
coef std err t P>|t|
[0.025 0.975]
-----------------------------------------------------------------------
---------------
const -0.0781 0.164 -0.477 0.636 -
0.409 0.253
layer_height 56.6034 11.884 4.763 0.000
32.585 80.622
wall_thickness 1.1375 0.271 4.191 0.000
0.589 1.686
infill_density 0.1587 0.033 4.868 0.000
0.093 0.225
nozzle_temperature 0.0004 0.001 0.482 0.633 -
0.001 0.002
bed_temperature -4.3166 8.863 -0.487 0.629 -
22.229 13.595
material -29.9509 502.401 -0.060 0.953 -
1045.341 985.440
fan_speed 1.8551 4.876 0.380 0.706 -
8.000 11.711
interct_mn 0.2557 2.295 0.111 0.912 -
4.382 4.893
interact_bn 0.0199 0.040 0.494 0.624 -
0.062 0.101
interact_fn -0.0120 0.031 -0.389 0.700 -
0.074 0.050
interact_fb 0.0062 0.024 0.256 0.800 -
0.043 0.056
=======================================================================
=======
Omnibus: 0.143 Durbin-Watson:
1.471
Prob(Omnibus): 0.931 Jarque-Bera (JB):
0.217
Skew: -0.118 Prob(JB):
0.897
Kurtosis: 2.779 Cond. No.
5.64e+20
=======================================================================
=======

Notes:
[1] Standard Errors assume that the covariance matrix of the errors is
correctly specified.
[2] The smallest eigenvalue is 6.94e-32. This might indicate that there
are