What Is Prediction?

 (Numerical) prediction is similar to classification

 construct a model

 use model to predict continuous or ordered value for a given

input
 Prediction is different from classification
 Classification refers to predict categorical class label

 Prediction models continuous-valued functions

 Major method for prediction: regression

 model the relationship between one or more independent or

predictor variables and a dependent or response variable

1
 Linear regression analysis

 Linear regression analysis is used to predict the

value of a variable based on the value of another
variable.
 The variable you want to predict is called the
dependent variable.
 The variable you are using to predict the other
variable's value is called the independent variable.
 Linear Regression
 Linear regression: involves a response variable y and a single predictor variable x
y = w0 + w1 x
where w0 (y-intercept) and w1 (slope) are regression coefficients
 Method of least squares: estimates the best-fitting straight line

| D|

 (x  x )( yi  y )
w 
i
i 1

1 | D|

 (x i  x )2
i 1

w  y w x
0 1

3
 Linear Regression using least square method example

Use the method of least square to

X Mid Term Y Final Exam
predicate the value of final exam 72 84
based on the given mid exam values. 50 63
81 77
74 78
y = w0 + w1 x 94
86
90
75
59 49
83 79
65 77
33 52
88 74
81 90

Step 1: Compute Average of both columns X and Y

Step 2: add a new column to compute Step 6 : compute W1
Step 7 : compute W0
Step 3: add a new column to compute

Step 4: add a new column to compute

Step 5: add a new column to compute

4
Numerical example
Linear Regression using least square method example | D|

 (x  x )( yi  y )
w 
i
Step2 Step3 Step 4 Step 5 i 1

X Mid Term Y Final Exam x-x' y-y' (x-x')(y-y') (x-x')^2 1 | D|

 (x i  x )2
72 84 0.17- 10 1.7- 0.0289 i 1
50 63 22.17- 11- 243.87 491.5089
81 77 8.83 3 26.49 77.9689
74 78 1.83 4 7.32 3.3489
94 90 21.83 16 349.28 476.5489
86 75 13.83 1 13.83 191.2689 W1= 2004 /3445.67
59 49 13.17- 25- 329.25 173.4489
83 79 10.83 5 54.15 117.2889
65 77 7.17- 3 21.51- 51.4089 =0.5816
33 52 39.17- 22- 861.74 1534.2889
88 74 15.83 0 0 250.5889
81 90 8.83 16 141.28 77.9689
sum sum
w  y w x
average x
72.17
Average y
74
2004 3445.67 0 1
W0= 74 – 0.5816 * 72.17
y = w0 + w1 x
=32.02
Y=32.02+0.5816 x

So, when the mid exam is 86, the predicated value of the final exam will be 82.04

5
 Regression accuracy metrics

MAE (Mean absolute error) represents the difference between the original and predicted values
extracted by averaged the absolute difference over the data set.

Y' Predicted Final Exam using

X Mid Term Y Final Exam Y=32.02+0.5816 x ERROR Mean absolute error

72 84 73.8952 10.1048
50 63 61.1 1.9
81 77 79.1296 2.1296
74 78 75.0584 2.9416
94 90 86.6904 3.3096
86 75 82.0376 7.0376
59 49 66.3344 17.3344
83 79 80.2928 1.2928
65 77 69.824 7.176
33 52 51.2128 0.7872
88 74 83.2008 9.2008
81 90 79.1296 10.8704

ERORR = 6.173733333