Scribd
Upload
9 views

Lecture 3

This document discusses machine learning techniques including regression, classification, linear regression, logistic regression and their differences. It provides details on regression techniques for predicting numeric targets, linear regression for modeling relationships between variables, and logistic regression for classification problems with nominal targets.

Uploaded by

somsonengda
Copyright
© © All Rights Reserved
Available Formats
Download as PDF, TXT or read online on Scribd
9 views

Lecture 3

This document discusses machine learning techniques including regression, classification, linear regression, logistic regression and their differences. It provides details on regression techniques for predicting numeric targets, linear regression for modeling relationships between variables, and logistic regression for classification problems with nominal targets.

Uploaded by

somsonengda
Copyright
© © All Rights Reserved
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 47

1

2
Wollo University ,Kombolicha Institute of Technology

Department of Software Engineering

Fundamental of Machine Learning

By Ashenafi Workie(MSc.)
KIOT@SE by Ashenafi Workie
Major chapters outlines

1 Chapter 1: Introduction to Machine Learning


2 Chapter 2: Classification based Supervised Learning
3 Chapter 3: Regression based Supervised Learning
4 Chapter 4: Unsupervised Learning
5 Chapter 5: Reinforcement Learning
6 Chapter 6: Advanced Machine Learning

4
Assessment Methods

• Assignments and Lab [15%]


• Quiz and Test [20%]
• Project and presentation [20%]
• Final Exam [45%]

5
Regression vs classification

6
Regression vs classification

7
What is Regression
▪ Regression is
▪ The difference between classification and regression is that in
regression our target variable is numeric with nominal as textual
and continuous.
▪ Companies may use this for doing things such as sales forecasts
or forecasting manufacturing defects.

▪ One creative example: predicting the probability of celebrity


divorce.

8
Regression

9
Regression

10
Regression

11
Regression

12
linear Regression

13
linear Regression

14
Linear Regression
▪ Linear regression: is probably the simplest approach for statistical
learning.
▪ Good starting point for more advanced approaches.
▪ Many fancy statistical learning techniques are extension of LR
.
▪ LR is very good to answer the following questions:
▪ Is there a relationship between two variables?
▪ How strong is the relationship?
▪ Which variable contribute the most?
▪ How accurately can we estimate the effect of each variable?
▪ How accurately can we predict the target?

15
linear Regression

16
linear Regression

17
Linear Regression
▪ Linear regression:
▪ Pros: Easy to interpret results, computationally inexpensive.
▪ Cons: Poorly models nonlinear data.
▪ Works with: Numeric values, nominal values.

▪ Our goal when using regression is to predict a numeric target


value.
▪ One way to do this is to write out an equation for the target value
with respect to the inputs.

18
linear Regression

19
linear Regression

20
linear Regression

21
linear Regression

22
linear Regression

23
linear Regression

24
Calculate the predicted values

25
Calculate the error and draw the line

26
linear Regression

27
More on Linear Regression

28
More on Linear Regression

29
Logistic Regression

30
Logistic Regression

31
Logistic Regression

32
Logistic Regression

33
Logistic Regression

34
Logistic Regression

35
Logistic Regression

36
Logistic Regression

37
Logistic Regression

38
Logistic Regression

39
Logistic Regression

40
Logistic Regression
❖ Stochastic gradient ascent can do as well as gradient ascent using far fewer
computing resources.

❖ Logistic regression:
❖ Pros: Computationally inexpensive, easy to implement, knowledge representation
easy to interpret.
❖ Cons: prone to underfitting, may have low accuracy.
❖ Works with: Numeric values, nominal values.

41
Logistic Regression(setting thourshold)
❖ We can treat a binary classification problem as a regression problem.
❖ The binary classes 1/0, and fit a regression algorithm.
❖ When a new unlabeled recorded comes in, try to guess the value with a
regression algorithm.
❖ If the value is higher than the midpoint (0.5), declare the prediction to be
the class 1, and vice versa.

42
Logistic Regression
❖ What if our data looks different as shown below; the line becomes
more slanted.

❖ This way, our algorithm is not skewed by the sample and it


manages to find the obvious visual threshold.

43
Logistic Regression

❖ Yb(x) is always going to be between 0 and 1.


❖ As opposed to linear regression approach; which can take on any
value.

44
Logistic Regression

45
Logistic Regression

46
End ….

47

You might also like