LINEAR REGRESSION

Practice Problems:
Q1. Find predicted value of y for one epoch and RMSE using Linear Regression. [May – 19]
X 2 3 4 5 6 7 8 9 10
y-Actual 1 3 6 9 11 13 15 17 20
Solution:
x y-Actual 𝒙 − ̅𝒙 𝒚−𝒚 𝒙)𝟐
̅ (𝒙 − ̅ ̅)(𝒚 − 𝒚
(𝒙 − 𝒙 ̅)
2 1 -4 -9.56 16 38.22
3 3 -3 -7.56 9 22.67
4 6 -2 -4.56 4 9.11
5 9 -1 -1.56 1 1.56
6 11 0 0.44 0 0.00
7 13 1 2.44 1 2.44
8 15 2 4.44 4 8.89
9 17 3 6.44 9 19.33
10 20 4 9.44 16 37.78
Sum = 54 95 60 140
Average = 6 10.56
We have linear regression equation as y= b0+b1x
Using least square method,
∑(𝑥𝑖 − 𝑥
̅)(𝑦𝑖 − 𝑦
̅) 140
𝑏1 = 2
= = 2.33
∑(𝑥𝑖 − 𝑥
̅) 60

𝑏0 = 𝑦
̅ − 𝑏1 𝑥
̅ = 10.56 − 2.33 × 6 = −3.42

∴ The linear regression line becomes: −𝟑. 𝟒𝟐 + 𝟐. 𝟑𝟑 × 𝒙

To find the predicted value of y for one epoch of x:
Let x = 3
∴ 𝒚 = −𝟑. 𝟒𝟐 + 𝟐. 𝟑𝟑 × 𝟑 = 𝟑. 𝟓𝟕

1
We have 𝑀𝑆𝐸 = ∑𝑛𝑖=1(𝑦𝑖 − (𝑏1 𝑥𝑖 + 𝑏0 ))2
𝑁

For the above epoch when x = 3,

𝑀𝑆𝐸 = (3 − 3.57)2 = 0.3249
∴ 𝑹𝑴𝑺𝑬 = √𝑴𝑺𝑬 = √𝟎. 𝟑𝟐𝟒𝟗 = 𝟎. 𝟓𝟕
Q2. Given the following data for the sales of car of an automobile company for six consecutive
years. Predict the sales for next two consecutive years. [Dec – 2019]
Years 2013 2014 2015 2016 2017 2018
Sales 110 100 250 275 230 300
Solution:
We have linear regression equation as y= b0+b1x

Years (x) Sales (y) 𝒙 − 𝒙̅ 𝒚−𝒚 ̅)𝟐

̅ (𝒙 − 𝒙 ̅)(𝒚 − 𝒚
(𝒙 − 𝒙 ̅)
2013 110 -2.5 -100.83 6.25 252.08
2014 100 -1.5 -110.83 2.25 166.25
2015 250 -0.5 39.17 0.25 -19.58
2016 275 0.5 64.17 0.25 32.08
2017 230 1.5 19.17 2.25 28.75
2018 300 2.5 89.17 6.25 222.92
Sum = 12093 1265 17.5 682.5
Average = 2015.5 210.83
Using least square method,
∑(𝑥𝑖 − 𝑥
̅)(𝑦𝑖 − 𝑦
̅) 682.5
𝑏1 = 2
= = 39
∑(𝑥𝑖 − 𝑥
̅) 17.5

𝑏0 = 𝑦
̅ − 𝑏1 𝑥
̅ = 210.83 − 39 × 2015.5 = −78393.7

∴ The linear regression line becomes: −𝟕𝟖𝟑𝟗𝟑. 𝟕 + 𝟑𝟗 × 𝒙

The sales for next two consecutive years:
𝑾𝒉𝒆𝒏 𝒙 = 𝟐𝟎𝟏𝟗, 𝒚 = −𝟕𝟖𝟑𝟗𝟑. 𝟕 + 𝟑𝟗 × 𝟐𝟎𝟏𝟗 = 𝟑𝟒𝟕. 𝟑𝟑 ≈ 𝟑𝟒𝟕
𝑾𝒉𝒆𝒏 𝒙 = 𝟐𝟎𝟐𝟎, 𝒚 = −𝟕𝟖𝟑𝟗𝟑. 𝟕 + 𝟑𝟗 × 𝟐𝟎𝟐𝟎 = 𝟑𝟖𝟔. 𝟑𝟑 ≈ 𝟑𝟖𝟔
Q3. The following table shows the midterm and final exam grades obtained for students in
Database course. Use the method of least squares using regression to predict the final exam grade
of a student who received 86 on the midterm exam. [May – 2017]
Midterm Exam (x) 72 50 81 74 94 86 59 83 65 33 88 81
Final Exam (y) 84 63 77 78 90 75 49 79 77 52 74 90
Solution:
Midterm Exam Final Exam ̅)
(𝒙 − 𝒙 ̅)
(𝒚 − 𝒚 𝒙)𝟐
(𝒙 − ̅ ̅)(𝒚 − 𝒚
(𝒙 − 𝒙 ̅)
(x) (y)
72 84 -0.17 10 0.03 -1.67
50 63 -22.17 -11 491.36 243.83
81 77 8.83 3 78.03 26.50
74 78 1.83 4 3.36 7.33
94 90 21.83 16 476.69 349.33
86 75 13.83 1 191.36 13.83
59 49 -13.17 -25 173.36 329.17
83 79 10.83 5 117.36 54.17
65 77 -7.17 3 51.36 -21.50
33 52 -39.17 -22 1534.03 861.67
88 74 15.83 0 250.69 0.00
81 90 8.83 16 78.03 141.33
Sum 866 888 3445.67 2004
Average 72.17 74

We have linear regression equation as y= b0+b1x

Using least square method,
∑(𝑥𝑖 − 𝑥
̅)(𝑦𝑖 − 𝑦
̅) 2004
𝑏1 = 2
= = 0.58
∑(𝑥𝑖 − 𝑥
̅) 3445.67

𝑏0 = 𝑦
̅ − 𝑏1 𝑥
̅ = 74 − 0.58 × 72.17 = 32.14

∴ The linear regression line becomes: 𝒚 = 𝟑𝟐. 𝟏𝟒 + 𝟎. 𝟓𝟖 × 𝒙

The final exam grade of a student who received 86 on the midterm exam:
𝒚 = 𝟑𝟐. 𝟏𝟒 + 𝟎. 𝟓𝟖 × 𝟖𝟔 = 𝟖𝟐. 𝟎𝟐 ≈ 𝟖𝟐 𝒎𝒂𝒓𝒌𝒔
Exercise Questions
1. The relationship between the monthly e-commerce sales and the online advertising costs
for 7 online stores for the last year is given. Find the equation of the straight line that fits
the data best using least square method.
Online Monthly E-commerce Sales (1000 s) Online Advertising Dollars (1000 s)
Store
1 368 1.7
2 340 1.5
3 665 2.8
4 954 5
5 331 1.3
6 556 2.2
7 376 1.3
Solution: Y= 125.8 + 171.5*X
2. Examine the relationship between the age and price for used cars sold in the last year by a
car dealership company using linear regression least square method.
Car Age (in years) Price (in dollars)
4 6300
4 5800
5 5700
5 4500
7 4500
7 4200
8 4100
9 3100
10 2100
11 2500
12 2200
Solution: Y = 7836 – 502.4*X

3. Last year, five randomly selected students took a math aptitude test before they began their
statistics course. The Statistics Department has three questions.
Student Aptitude Test Statistic Grades
1 95 85
2 85 95
3 80 70
4 70 65
5 60 70

i. What linear regression equation best predicts statistics performance, based on math
aptitude scores? [Solution: y = 26.768 + 0.644x]
ii. If a student made an 80 on the aptitude test, what grade would we expect her to
make in statistics? [Solution: y = 78.288]