machine

learning
Linear regression
Dr. Darkhan Zholtayev
Assistant professor at Department of Computational and Data
Science
[email protected]
Topics to cover
• What is the regression
• Linear regression
• Lest square error
 General graph

AI map

Joseph, B. (2020, June 17). Linear Regression Made Easy: How Does It Work and How to Use It in Python. Towards Da
Science. https://towardsdatascience.com/linear-regression-made-easy-how-does-it-work-and-how-to-use-it-in-pytho
Linear Regression
• Technique used for the modeling and analysis of
numerical data
• Exploits the relationship between two or more variables
so that we can gain information about one of them
through knowing values of the other
• Regression can be used for prediction, estimation,
hypothesis testing, and modeling causal relationships
Problem
 Data

Xie, Y. (2013). Lecture 11: Simple Linear Regression. H. Milton Stewart School of Industrial
and Systems Engineering, Georgia Institute of Technology. Retrieved from
Data

Xie, Y. (2013). Lecture 11: Simple Linear Regression. H. Milton Stewart School of Industrial
and Systems Engineering, Georgia Institute of Technology. Retrieved from
Data
Linear Regression
Linear regression
 Linear
regression

Xie, Y. (2013). Lecture 11: Simple Linear Regression. H. Milton Stewart School of Industrial
and Systems Engineering, Georgia Institute of Technology. Retrieved from
Linear regression: different forms
Linear regression
Linear regression
Linear regression
Estimate regression parameters
Method of least squares
 Least square estimates

Xie, Y. (2013). Lecture 11: Simple Linear Regression. H. Milton Stewart School of Industrial
and Systems Engineering, Georgia Institute of Technology. Retrieved from
Alternative notation
Example: oxygen and hydrocarcon level
Calculati
on 2
Calculati
on
Interpretat
ion of
regression
model
Estimation of variance
 Sammary

Xie, Y. (2013). Lecture 11: Simple Linear Regression. H. Milton Stewart School of Industrial
and Systems Engineering, Georgia Institute of Technology. Retrieved from
 Example
• import pandas as pd # for data manipulation
import numpy as np # for data manipulation
from sklearn.linear_model import LinearRegression # for
creating a model
import plotly.graph_objects as go # for visualizations
import plotly.express as px # for visualizations

• # Read data into a Pandas DataFrame

df = pd.read_csv('Real estate.csv', encoding='utf-8')

# Print DataFrame
df
Joseph, B. (2020, June 17). Linear Regression Made Easy: How Does It Work and How to Use It in Python. Towards Data Science.
https://towardsdatascience.com/linear-regression-made-easy-how-does-it-work-and-how-to-use-it-in-python-be0799d2f159
 Data

Joseph, B. (2020, June 17). Linear Regression Made Easy: How Does It Work and How to Use It in Python. Towards Data Science.
https://towardsdatascience.com/linear-regression-made-easy-how-does-it-work-and-how-to-use-it-in-python-be0799d2f159
 Code 1
• # Create a scatter plot
fig = px.scatter(df, x=df['X3 distance to the nearest MRT station'], y=df['Y house price of unit area'],
opacity=0.8, color_discrete_sequence=['black'])

# Change chart background color

fig.update_layout(dict(plot_bgcolor = 'white'))

# Update axes lines

fig.update_xaxes(showgrid=True, gridwidth=1, gridcolor='lightgrey',
zeroline=True, zerolinewidth=1, zerolinecolor='lightgrey',
showline=True, linewidth=1, linecolor='black')

fig.update_yaxes(showgrid=True, gridwidth=1, gridcolor='lightgrey',

zeroline=True, zerolinewidth=1, zerolinecolor='lightgrey',
showline=True, linewidth=1, linecolor='black')

# Set figure title

fig.update_layout(title_text="Scatter Plot")

# Update marker size

fig.update_traces(marker=dict(size=3))

Joseph,fig.show()
B. (2020, June 17). Linear Regression Made Easy: How Does It Work and How to Use It in Python. Towards Data Science.
https://towardsdatascience.com/linear-regression-made-easy-how-does-it-work-and-how-to-use-it-in-python-be0799d2f159
 Scatter plot

Joseph, B. (2020, June 17). Linear Regression Made Easy: How Does It Work and How to Use It in Python. Towards Data Science.
https://towardsdatascience.com/linear-regression-made-easy-how-does-it-work-and-how-to-use-it-in-python-be0799d2f159
 Training
• # Select variables that we want to use in a model
# Note, we need X to be a 2D array, hence reshape
X=df['X3 distance to the nearest MRT station'].values.reshape(-1,1)
y=df['Y house price of unit area'].values

# Fit linear regression model

model = LinearRegression()
reg = model.fit(X, y)

# Print the slope and intercept of the best-fit line

print(reg.coef_)
print(reg.intercept_)

Joseph, B. (2020, June 17). Linear Regression Made Easy: How Does It Work and How to Use It in Python. Towards Data Science.
https://towardsdatascience.com/linear-regression-made-easy-how-does-it-work-and-how-to-use-it-in-python-be0799d2f159
 Code 2
• # We will use below to draw a best-fit line on a chart
# Create 20 evenly spaced points from smallest X to largest X
x_range = np.linspace(X.min(), X.max(), 20)

# Predict y values for our set of X values

y_range = model.predict(x_range.reshape(-1, 1))

# Create a scatter plot

fig = px.scatter(df, x=df['X3 distance to the nearest MRT station'], y=df['Y house price of unit area'],
opacity=0.8, color_discrete_sequence=['black'])

# Add a best-fit line

fig.add_traces(go.Scatter(x=x_range, y=y_range, name='Regression Fit'))

# Change chart background color

fig.update_layout(dict(plot_bgcolor = 'white'))

# Update axes lines

fig.update_xaxes(showgrid=True, gridwidth=1, gridcolor='lightgrey',
zeroline=True, zerolinewidth=1, zerolinecolor='lightgrey',
showline=True, linewidth=1, linecolor='black')

fig.update_yaxes(showgrid=True, gridwidth=1, gridcolor='lightgrey',

zeroline=True, zerolinewidth=1, zerolinecolor='lightgrey',
showline=True, linewidth=1, linecolor='black')

# Set figure title

fig.update_layout(title_text="Scatter Plot with Linear Regression Line")

# Update marker size

fig.update_traces(marker=dict(size=3))

fig.show()

Joseph, B. (2020, June 17). Linear Regression Made Easy: How Does It Work and How to Use It in Python. Towards Data Science.
https://towardsdatascience.com/linear-regression-made-easy-how-does-it-work-and-how-to-use-it-in-python-be0799d2f159
Prediction line
 • # Select variables that we want to use in a
model
# Note, X in this case is already a 2D
array, hence no reshape
X=df[['X3 distance to the nearest MRT
Multiple station','X2 house age']]
y=df['Y house price of unit area'].values
linear # Fit linear regression model
regression model = LinearRegression()
reg = model.fit(X, y)

# Print slope(s) and intercept

print(reg.coef_)
print(reg.intercept_)
 Multiple
linear
regression
— Python
example

Joseph, B. (2020, June 17). Linear Regression Made Easy: How Does It Work and How to Use It in Python. Towards Data Science.
https://towardsdatascience.com/linear-regression-made-easy-how-does-it-work-and-how-to-use-it-in-python-be0799d2f159
 Fitted
multiple
linear
regression

Joseph, B. (2020, June 17). Linear Regression Made Easy: How Does It Work and How to Use It in Python. Towards Data Science.
https://towardsdatascience.com/linear-regression-made-easy-how-does-it-work-and-how-to-use-it-in-python-be0799d2f159
Basic statistics
• The sample mean is the sum of all the observations (∑Xi)
divided by the number of observations (n):
ΣXi = X1 + X2 + X3 + X4 + … + Xn

• Example. 1, 2, 2, 4, 5, 10. Calculate the mean. Note: n =

6 (six observations)

∑Xi = 1 + 2+ 2+ 4 + 5 + 10 = 24
= 24 / 6 = 4.0
The median

To get the median, we must first

rearrange the data into an
The median is the middle value of ordered array (in ascending or
the ordered data descending order). Generally, we
order the data from the lowest
value to the highest value.
The mode
• The mode is the value of the data that occurs with the
greatest frequency.

Example. 1, 1, 1, 2, 3, 4, 5
Answer. The mode is 1 since it occurs three times. The other values
each appear only once in the data set.

Example. 5, 5, 5, 6, 8, 10, 10, 10.

Answer. The mode is: 5, 10.
There are two modes. This is a bi-modal dataset.
Standart deviation

• The standard deviation, s, measures a kind of “average” deviation about the

mean. It is not really the “average” deviation, even though we may think of
it that way.

• Why can’t we simply compute the average deviation about the mean, if
that’s what we want?

• If you take a simple mean, and then add up the deviations about the mean,
as above, this sum will be equal to 0. Therefore, a measure of “average
deviation” will not work.
Standard Deviation
• Instead, we use:

• This is the “definitional formula” for standard deviation.

• The standard deviation has lots of nice properties, including:
• By squaring the deviation, we eliminate the problem of the deviations
summing to zero.
• In addition, this sum is a minimum. No other value subtracted from X and
squared will result in a smaller sum of the deviation squared. This is called
the “least squares property.”
• Note we divide by (n-1), not n. This will be referred to as a loss of
one degree of freedom.
Variance
The variance, s2, is the standard deviation (s) squared.
Conversely, .

Definitional formula:
Computational formula:
Thank you
for your
attention