Introduction to

Machine Learning
Machine Learning Concepts
 Lecture Overview

• What is Data Mining/Machine Learning?

• What kind of data sources can we mine?
• Data Objects and Feature Types
• Data Preprocessing
• Pattern Types
• Machine Learning
• Underfitting and Overfitting
What is Data Mining?
 What is Machine Learning?

• Machine Learning: The ability of computer systems

to automatically learn from experience without being
explicitly programmed.
• Data Mining: Extraction of interesting patterns or
knowledge from a large amount of data.
 Data Mining vs. Machine Learning
Data Mining Machine Learning
• Finding interesting patterns in • Training data set to predict
data. future outcomes from already
• Requires human intervention mined data.
(pull together and sort through • No human intervention
data) necessary
• Used in retail, finance, banking, • Applications like image
sales, marketing, healthcare, etc. recognition, speech recognition,
chatbots, predictive analysis, etc.

Machine Learning uses Data Mining

 Models and Features and Instances
• Model: a mathematical function for describing
the relationship between inputs and outputs
and making predictions.
• Instance: an individual data point or
observational unit.
• Feature, or variable, is a characteristic that is
measured or observed on an instance

This symbol means let's

pause and look at data/code
Types of Data
Feature
Types
 Feature Types – Categorical

• Categorical: features that have non-numerical values,

or numerical values without mathematical
meaning. (Ex: The brand of shoes, the breed of dogs)
• Nominal Features: symbols of names of things (categorical).
• Eye color: brown, blue, green, gray
• Binary Features: two categories of 0 and 1 (Boolean).
• HIV test results: Positive (0) or Negative (1)
• Ordinal Features: values have meaningful order or ranking.
• Grades: A, A-, B+, B, B-
 Feature Types – Numerical

• Numerical: features that have numerical values with

mathematical meaning (Ex: Amount spent on a
transaction, square footage)
• Interval-scaled Features: a numeric feature measured on a
scale of equal units. Does not have an inherent-zero point.
• Ex: Ratings in a survey of 1 to 5.
• Ratio-Scaled Features: a numeric feature with an inherent
zero-point. Can speak of a value as being a multiple of
another value.
• Ex: count attributes such as the number of words in a document.
Types of
Machine
Learning
 Supervised Learning: Classification

• Machine Learning models that classify instances into

a class, similar to a category.
• May predict class directly or its probability of
membership.
• Input features will be categorical or numerical; output
features will be categorical.
 Supervised Learning: Regression

• Machine Learning models that predict the value of a

numerical output feature based on input features.
• Simple linear regression is an example.
• Input features will be categorical or numerical; output
features will be numerical.
 Unsupervised Learning: Clustering

• Machine Learning models that combine features or

instances while searching for patterns in the data.
Semi-supervised Learning Example
 Reinforcement Learning
• Type of machine learning where an agent learns to
make sequences of decisions by interacting with the
environment to maximize cumulative rewards
Kahoot
 Scikit-Learn
Workflows

1. Import your dataset

2. Split data into training and
test sets.
3. Choose and configure a
model
4. Fit the model on training data
using .fit().
5. Evaluate performance &
Predict on test data
using .score() or error metrics.

This symbol means let's

pause and look at data/code
 Bias and Variance
• Prediction error: difference between the observed value
and predicted value . Error is
• Bias: the mean of the prediction error. Unbiased means
prediction error = 0
• Variance: average squared difference between an
observation and the mean
• Error variance: is the variance of a model's
prediction errors. How wrong the model is on
average. Models with low error variance fit the
training data well.
 Error Variance vs. Prediction Variance

• Prediction Variance: the variance of the predicted values,

where with the bar is the mean predicted value.
• Relationship: models with low error variance have
high prediction variance.
• Low error variance models are highly complex, meaning
they are sensitive to small changes in training data, which
leads to high variability in predictions when applied to new
data.
Error Variance vs. Prediction Variance

Low Prediction Variance, High Error High Prediction Variance, Low Error
 Underfitting and Overfitting
Underfitting Overfitting
• Model is too simple to fit • Model is too complicated
the data well (capture and fits the data too
underlying patterns) closely.
• Underfit models do not • Do not generalize well to
explain changes in the output new data.
feature sufficiently. • Can be avoided by using
• Underfit models have high cross-fold validation or other
bias. model specific methods.
Overfitting and Underfitting Comparison

https://
www.geeksforge
eks.org/
underfitting-
and-overfitting-
in-machine-
learning/
 Bias-Variance Tradeoff
• Mean Square
Error (MSE):
average of squared
prediction errors.
• Composed of bias
and prediction
variance.
• Models with
accurate predictions
will have low MSE.
 Linear Regression
• Before we end today, we will learn of our first
machine learning model.
• Linear Regression uses numeric data to show a
relationship between an independent (x) and
dependent variable (y).
• Independent variable is the variable that we change, and
we see how the dependent variable is affected.
• Goal: to predict future values of one of the attributes
(features).
 Linear Regression Chart

If the independent If the independent

variable increases variable increases
and the dependent and the dependent
variable increases, variable decreases,
its called a positive its called a negative
correlation. correlation.

Your book calls independent variables predictor variable and

dependent variables a response variable.
 Linear Regression Chart 2

Taking a series Goal of the

of data points Least Squares
we can create a Method is to
regression line draw a line
using the least representing
squares method. the projection of the
two variables with
the least
amount of errors.
 Least Squares Method

m is the slope
of the line
b is the y-intercept

𝑦 =𝑚 𝑥+ 𝑏
 Correlation

Positive Correlation Negative Correlation

 Evaluating the Regression:
Coefficient of Determination

• Evaluation Metric:
• Linear Regression's metric needs to tell us the strength of
the correlation between the two variables.
• There is a measure for this called the Correlation
Coefficient, R, but oftentimes we instead use a different
metric that is more useful called the Coefficient of
Determination or R2
• So just what is the Coefficient of Determination?
 Evaluating the Regression:
Coefficient of Determination 2

• First let's define the Correlation Coefficient, R:

• measure the strength of association between two variables.
• Now, the Coefficient of Determination, R2
• the proportion of the variance in the dependent variable that is
predictable from the independent variable. This value shows how
well future outcomes can be predicted by the model.
Can the dependent variable be predicted from the independent variable??
0 1 Without
R
2
error

No, it cannot be predicted. Yes, it can be predicted.

Kahoot
 TPS

• Complete the TPS located in the module for this

week.