The Yenepoya Institute of Arts, Science, Commerce and

Management (YIASCM)

Course: IV Semester BCA ( All specializations) & BSc

Supervised Learning
Lecture 2: Unit-2
 Objective
• At the end of this session the leaner will able to understand
⮚ Simple Linear Regression in Machine Learning
⮚ Simple Linear Regression Model
⮚ Mean Squared Error
⮚ Implementation of Simple Linear Regression Algorithm
using Python
⮚ Multiple Linear Regression
⮚ Implementation of Multiple Linear Regression model using
Python
• Summary
• Reference
 Simple Linear Regression in Machine Learning

• Simple Linear Regression is a type of Regression

algorithms that models the relationship between a
dependent variable and a single independent variable.
• The relationship shown by a Simple Linear Regression
model is linear or a sloped straight line, hence it is
called Simple Linear Regression.
• The key point in Simple Linear Regression is that
the dependent variable must be a continuous/real
value.
• However, the independent variable can be measured
on continuous or categorical values.
• Simple Linear regression algorithm has mainly
two objectives:
1. Model the relationship between the two
variables. Such as the relationship between
Income and expenditure, experience and
Salary, etc.
2. Forecasting new observations. Such as
Weather forecasting according to temperature,
Revenue of a company according to the
investments in a year, etc.
 Simple Linear Regression Model:
The Simple Linear Regression model can be represented using
the below equation:

y= a0+a1x+ ε

Where,
• a0= It is the intercept of the Regression line (can be
obtained putting x=0)
• a1= It is the slope of the regression line, which tells
whether the line is increasing or decreasing.
• ε = The error term. (For a good model it will be negligible)
Mean Squared Error | What Is
Mean Squared Error?

https://youtu.be/beIgcdf0YDE?
si=YLwL9gw0X4GOfI1s
 Example
• In this example, we generate some random
data, split it into training and testing sets,
create a Linear Regression model, train it using
the training data, make predictions on the test
data, and then evaluate the model using Mean
Squared Error. Finally, we plot the training
data, test data, and the regression line.
 Code
# Import necessary libraries
import numpy as np
import matplotlib.pyplot as plt
from sklearn.model_selection import
train_test_split
from sklearn.linear_model import LinearRegression
from sklearn.metrics import mean_squared_error
# Generate some example data
np.random.seed(42)
X = 2 *np.random.rand(100, 1)
y = 4 + 3 * X + np.random.randn(100, 1)
# Split the data into training and testing sets
X_train, X_test, y_train, y_test = train_test_split(X,
y, test_size=0.2, random_state=42)

# Create a Linear Regression model

model = LinearRegression()
# Train the model
model.fit(X_train, y_train)

# Make predictions on the test set

y_pred = model.predict(X_test)

# Evaluate the model

mse = mean_squared_error(y_test, y_pred)
print(f'Mean Squared Error: {mse}')
# Plot the training data, test data, and the regression line

plt.scatter(X_train, y_train, label='Training Data')

plt.scatter(X_test, y_test, label='Test Data',
color='red')
plt.plot(X_test, y_pred, color='blue', linewidth=3,
label='Regression Line')
plt.xlabel('X')
plt.ylabel('y')
plt.legend()
plt.show()
 Output
Mean Squared Error: 0.6536995137170021
 Implementation of Simple Linear
Regression Algorithm using Python
• Problem Statement example for Simple Linear
Regression:
• Here we are taking a dataset that has two variables: salary
(dependent variable) and experience (Independent
variable).
• The goals of this problem is:
• We want to find out if there is any correlation between
these two variables
• We will find the best fit line for the dataset.
• How the dependent variable is changing by changing the
independent variable.
To implement the Simple Linear regression model in
machine learning usingPython, we need to follow the
below steps:
• Step-1: Data Pre-processing
• The first step for creating the Simple Linear
Regression model is data pre-processing. We have
already done it earlier in this tutorial. But there will
be some changes, which are given in the below
steps:
• First, we will import the three important libraries,
which will help us for loading the dataset, plotting
the graphs, and creating the Simple Linear
Regression model.
 .

import numpy as nm
import matplotlib.pyplot as mtp
import pandas as pd

Next, we will load the dataset into our code:

data_set= pd.read_csv('Salary_Data.csv')
 .

The above output shows the dataset, which has two variables: Salary and
Experience.
• After that, we need to extract the dependent and
independent variables from . the given dataset.
• The independent variable is years of experience, and
the dependent variable is salary.
• Below is code for it:
x= data_set.iloc[:, :-1].values
y= data_set.iloc[:, 1].values

In the above lines of code, for x variable, we have taken -1 value

since we want to remove the last column from the dataset. For y
variable, we have taken 1 value as a parameter, since we want to
extract the second column and indexing starts from the zero.
 By executing the above line of code, we will get the output for X and Y variable as:

In the above output image, we can see the X (independent) variable and Y (dependent)
variable has been extracted from the given dataset.
 .

• Next, we will split both variables into the test set and
training set.
• We have 30 observations, so we will take 20
observations for the training set and 10 observations
for the test set.
• We are splitting our dataset so that we can train our
model using a training dataset and then test the
model using a test dataset.
• The code for this is given below:
• from sklearn.model_selection import train_test_sp
lit
• x_train, x_test, y_train, y_test= train_test_split(x, y
, test_size= 1/3, random_state=0)
By executing the above code, we will get x-test, x-train and y-
test, y-train dataset. Consider the below images:
 .

For simple linear Regression, we will not use Feature Scaling. Because Python
libraries take care of it for some cases, so we don't need to perform it here. Now, our
dataset is well prepared to work on it and we are going to start building a Simple
Linear Regression model for the given problem.
Step-2: Fitting the Simple Linear Regression to
the Training Set:
• Now the second step is to fit our model to the training
dataset.
• To do so, we will import the LinearRegression class of
the linear_model library from the scikit learn.
• After importing the class, we are going to create an object
of the class named as a regressor.
• The code for this is given below:
#Fitting the Simple Linear Regression model to the training d
ataset
from sklearn.linear_model import LinearRegression
regressor= LinearRegression()
regressor.fit(x_train, y_train)
• In the above code, we have used a fit() method
.

to fit our Simple Linear Regression object to the

training set.
• In the fit() function, we have passed the x_train
and y_train, which is our training dataset for the
dependent and an independent variable.
• We have fitted our regressor object to the
training set so that the model can easily learn
the correlations between the predictor and
target variables. After executing the above lines
of code, we will get the below output.
 .
• Output:
• Out[7]: LinearRegression(copy_X=True,
fit_intercept=True, n_jobs=None,
normalize=False)
 Step: 3. Prediction of test set result:
• dependent (salary) and an independent variable
(Experience).
• So, now, our model is ready to predict the output for the
new observations. In this step, we will provide the test
dataset (new observations) to the model to check
whether it can predict the correct output or not.
• We will create a prediction vector y_pred, and x_pred,
which will contain predictions of test dataset, and
prediction of training set respectively.
• #Prediction of Test and Training set result
• y_pred= regressor.predict(x_test)
• x_pred= regressor.predict(x_train)
 .
• On executing the above lines of code, two
variables named y_pred and x_pred will
generate in the variable explorer options that
contain salary predictions for the training set
and test set.
• Output:
• compare the result by comparing values from
y_pred and y_test. By comparing these values,
we can check how good our model is
performing.
 Step: 4. visualizing the Training set results:
• Now in this step, we will visualize the training set result.
• To do so, we will use the scatter() function of the pyplot
library, which we have already imported in the pre-processing
step.
• The scatter () function will create a scatter plot of
observations.
• In the x-axis, we will plot the Years of Experience of employees
and on the y-axis, salary of employees.
• In the function, we will pass the real values of training set,
which means a year of experience x_train, training set of
Salaries y_train, and color of the observations.
• Here we are taking a green color for the observation, but it
can be any color as per the choice.
• Now, we need to plot the regression line, so for
. function of the
this, we will use the plot()
pyplot library.
• In this function, we will pass the years of
experience for training set, predicted salary for
training set x_pred, and color of the line.
• Next, we will give the title for the plot.
• So here, we will use the title() function of
the pyplot library and pass the name ("Salary
vs Experience (Training Dataset)".
• After that, we will assign labels for x-axis and y-
axis using xlabel() and ylabel() function.
Finally, we will represent all above things in a graph using
show(). The code is given below:

mtp.scatter(x_train, y_train, color="green")

mtp.plot(x_train, x_pred, color="red")
mtp.title("Salary vs Experience (Training Dataset)")

mtp.xlabel("Years of Experience")
mtp.ylabel("Salary(In Rupees)")
mtp.show()
 .

• Output:
• By executing the above lines of code, we will
get the below graph plot as an output.
• In the above plot, we can see the real values
observations in green. dots and predicted
values are covered by the red regression line.
The regression line shows a correlation
between the dependent and independent
variable.
• The good fit of the line can be observed by
calculating the difference between actual
values and predicted values. But as we can see
in the above plot, most of the observations
are close to the regression line, hence our
model is good for the training set.
 Step: 5. visualizing the Test set results:

• In the previous step, we have visualized the

performance of our model on the training set.
Now, we will do the same for the Test set. The
complete code will remain the same as the
above code, except in this, we will use x_test,
and y_test instead of x_train and y_train.
• Here we are also changing the color of
observations and regression line to differentiate
between the two plots, but it is optional.
#visualizing the Test set results
mtp.scatter(x_test, y_test, color="blue")
mtp.plot(x_train, x_pred, color="red")
mtp.title("Salary vs Experience (Test Dataset)")

mtp.xlabel("Years of Experience")
mtp.ylabel("Salary(In Rupees)")
mtp.show()
 • Output:
.

• By executing the above line of code, we will

get the output as:

In the above plot, there are observations given by the blue color,
and prediction is given by the red regression line. As we can see,
most of the observations are close to the regression line, hence
we can say our Simple Linear Regression is a good model and
 Multiple Linear Regression
• In the previous topic, we have learned about
Simple Linear Regression, where a single
Independent/Predictor(X) variable is used to model
the response variable (Y). But there may be various
cases in which the response variable is affected by
more than one predictor variable; for such cases,
the Multiple Linear Regression algorithm is used.
• “Multiple Linear Regression is one of the important
regression algorithms which models the linear
relationship between a single dependent
continuous variable and more than one
independent variable.”
Example: .
• Prediction of CO2 emission based on engine size and
number of cylinders in a car.

Some key points about MLR:

• For MLR, the dependent or target variable(Y) must be the
continuous/real, but the predictor or independent
variable may be of continuous or categorical form.
• Each feature variable must model the linear relationship
with the dependent variable.
• MLR tries to fit a regression line through a
multidimensional space of data-points.
 MLR equation
• In Multiple Linear Regression, the target
variable(Y) is a linear combination of multiple
predictor variables x1, x2, x3, ...,xn.

• Where,
• Y= Output/Response variable
• b0, b1, b2, b3 , bn....= Coefficients of the model.
• x1, x2, x3, x4,...= Various Independent/feature
variable
Implementation of Multiple Linear Regression model
using Python:
Problem Description:
• We have a dataset of 50 start-up companies. This dataset
contains five main information: R&D Spend, Administration
Spend, Marketing Spend, State, and Profit for a financial year.
Our goal is to create a model that can easily determine which
company has a maximum profit, and which is the most affecting
factor for the profit of a company.
• Since we need to find the Profit, so it is the dependent variable,
and the other four variables are independent variables. Below are
the main steps of deploying the MLR model:
• Data Pre-processing Steps
• Fitting the MLR model to the training set
• Predicting the result of the test set
 Step-1: Data Pre-processing
• Importing libraries: Firstly we will import the library
which will help in building the model. Below is the code
for it:
# importing libraries
import numpy as nm
import matplotlib.pyplot as mtp
import pandas as pd

• Importing dataset: Now we will import the

dataset(50_CompList), which contains all the variables.
Below is the code for it:
#importing datasets
data_set= pd.read_csv('50_CompList.csv')
Output: We will get the dataset as:

In above output, we can clearly see that there are five

variables, in which four variables are continuous and one is
categorical variable.
 .

• Extracting dependent and independent

Variables:
#Extracting Independent and dependent Variable
x= data_set.iloc[:, :-1].values
y= data_set.iloc[:, 4].values
• Output:

In the above output, the last column contains categorical variables which are not
suitable to apply directly for fitting the model. So we need to encode this variable.
 Encoding Dummy Variables:
• As we have one categorical variable (State), which
cannot be directly applied to the model, so we will
encode it.
• To encode the categorical variable into numbers, we
will use the LabelEncoder class.
• But it is not sufficient because it still has some
relational order, which may create a wrong model. So
in order to remove this problem, we will
use OneHotEncoder, which will create the dummy
variables.
 .

#Catgorical data
from sklearn.preprocessing import LabelEncoder, OneHotEncoder
lblencode= LabelEncoder()
x[:, 3]= lblencode.fit_transform(x[:,3])
onehotencoder= OneHotEncoder(categorical_features= [3])
x= onehotencoder.fit_transform(x).toarray()

Here we are only encoding one independent variable, which is state

as other variables are continuous.
 Output:

As we can see in the above output, the state column has been converted
into dummy variables (0 and 1). Here each dummy variable column is
corresponding to the one State. We can check by comparing it with the
original dataset. The first column corresponds to the California State, the
second column corresponds to the Florida State, and the third column
corresponds to the New York State.
• Now, we are writing a single line of
. code just to avoid the dummy variable
trap:
• #avoiding the dummy variable trap:
• x = x[:, 1:]
• If we do not remove the first dummy variable, then it may introduce
multicollinearity in the model.
• As we can see in the above. output image, the first
column has been removed.
• Now we will split the dataset into training and test set.
The code for this is given below:
# Splitting the dataset into training and test set.
from sklearn.model_selection import train_test_split
x_train, x_test, y_train, y_test= train_test_split(x, y, test_
size= 0.2, random_state=0)

• The above code will split our dataset into a training set
and test set.
Output: The above code will split the dataset into training set and test set.
.
 Step: 2- Fitting our MLR model to the Training set:
• Now, we have well prepared our dataset in order to
provide training, which means we will fit our
regression model to the training set. It will be similar
to as we did in Simple Linear Regression model. The
code for this will be:
#Fitting the MLR model to the training set:
from sklearn.linear_model import LinearRegression
regressor= LinearRegression()
regressor.fit(x_train, y_train)
Output:
Out[9]: LinearRegression(copy_X=True, fit_intercept=True, n_jobs=None, normalize=False)
Now, we have successfully trained our model using the training dataset. In the
next step, we will test the performance of the model using the test dataset.
 Step: 3- Prediction of Test set results:

• The last step for our model is checking the performance

of the model. We will do it by predicting the test set
result. For prediction, we will create a y_pred vector.
Below is the code for it:
#Predicting the Test set result;
y_pred= regressor.predict(x_test)
• By executing the above lines of code, a new vector will
be generated under the variable explorer option. We can
test our model by comparing the predicted values and
test set values.
.
• In the above output, we have predicted result set and
test set. We can check . model performance by
comparing these two value index by index. For
example, the first index has a predicted value
of 103015$ profit and test/real value
of 103282$ profit. The difference is only of 267$,
which is a good prediction, so, finally, our model is
completed here.
• We can also check the score for training dataset and
test dataset. Below is the code for it:

print('Train Score: ', regressor.score(x_train, y_train))

print('Test Score: ', regressor.score(x_test, y_test))
 .
• Output: The score is:
• Train Score: 0.9501847627493607
• Test Score: 0.9347068473282446
• The above score tells that our model is 95%
accurate with the training dataset and 93%
accurate with the test dataset.
 Summary
• In this session the leaner understand about
the
⮚ Simple Linear regression with implementation
code
⮚ Multiple Linear Regression with
implementation code
 References
• Alpaydin, E. (2020). Introduction to machine learning.
MIT press.
• Pedregosa, F., Varoquaux, G., Gramfort, A., Michel, V.,
Thirion, B., Grisel, O., ... &Vanderplas, J. (2011). Scikit-
learn: Machine learning in Python. the Journal of
machine Learning research, 12, 2825-2830.
• EthemAlpaydin, "Introduction to machine learning",
PHI learning, 2008. [T3] RajjanShinghal,
• Ripley, B. D. (2007). Pattern recognition and neural
networks. Cambridge university press.