Artificial Intelligence

Lec.4
Assoc. Prof. Anwer Sayed
Lecture 4
Multivariate Linear Regression w. Gradient
Descent
• example, what if we want to predict the cost of a house and we have
access to the size, number of bedrooms, number of bathrooms, age
etc. For this kind of prediction we need to use Multivariate Linear
Regression.
Cost function
Gradient Descent
https://github.com/drbilo/multivariate-linear-
regression/blob/master/housepricelinearregression.py
Multivariate Linear Regression w. Gradient
Descent
• example, what if we want to predict the cost of a house and we have
access to the size, number of bedrooms, number of bathrooms, age
etc. For this kind of prediction we need to use Multivariate Linear
Regression.
Cost function
Gradient Descent
Classification
Classification
Algorithms
• The Classification algorithm classifies
the dataset into a number of classes.
Thus, is called classification. These
classes are such as, Yes or No, 0 or 1,
cat or dog, etc.

• The inputs can be continuous variables

or categories variables while the
output is class-1, class-2, or more.
Classification
Classification , Regression
Regression Vs.
Classification
Vs. Clustring
Classification Application

• Face Recognition
• Character recognition
• Spam detection
• Medical detection: from symptoms to illnesses
• Biometrics: Recognition / authentication using physical and/or
behavioral characteristics: Face, iris, signature ,…..etc
 Classification Algorithms
• The algorithm which implements the classification on a dataset is
known as a classifier.
There are two types of Classifications:
❖Binary Classifier
If the classification problem has only two possible outcomes, then
it is called as Binary Classifier.
Examples: YES or NO, MALE or FEMALE, CAT or DOG, etc.
❖Multi-class Classifier
If a classification problem has more than two outcomes, then it is
called as Multi-class Classifier.
Example: Classifications of types of crops , Classification of types
of careers .
Binary Classifier
Multi-class Classifier
 Classification Algorithms

• Classification Algorithms can be further divided into the Mainly two category:

❑Linear Models ❑Non-linear Models

▪ Logistic Regression ▪ Support Vector Machine (SVM)
▪ Support Vector Machine (SVM) ▪ K-Nearest Neighbors
▪ Naïve Bayes
▪ Decision Tree Classification
▪ Random Forest Classification
• we can apply binary transformation approaches such as one-versus-one and one-
versus-all to adapt native binary classification algorithms for multi-class classification
tasks.
• One-versus-one: this strategy trains as many classifiers as there are pairs of labels. If
we have a 3-class classification, we will have three pairs of labels, thus three
classifiers. In general, for N labels, we will have Nx(N-1)/2 classifiers. Each classifier is
trained on a single binary dataset, and the final class is predicted by a majority vote
between all the classifiers. One-vs-one approach works best for SVM
One-versus-rest: at this stage, we start by considering each label as an
independent label and consider the rest combined as only one label.
With 3-classes, we will have three classifiers.
• In general, for N labels, we will have N binary classifiers.
 Multi-Label Classification
• In multi-label classification tasks, we try to predict 0 or more classes for each input
example. In this case, there is no mutual exclusion because the input example can have
more than one label.
• such as auto-tagging in Natural Language Processing, where a given text can contain
multiple topics. Similarly to computer vision, an image can contain multiple objects, as
illustrated below: the model predicted that the image contains: a plane, a boat, a truck,
and a dog.
Examples
Multi-Label Classification
• It is not possible to use multi-class or binary classification models to
perform multi-label classification.

• Multi-label Decision Trees

• Multi-label Gradient Boosting
• Multi-label Random Forests
 Imbalanced Classification
• Let’s consider the following 3-class
classification scenario where the training
data contains: 60% of trucks, 25% of planes,
and 15% of boats.
• The imbalanced classification problem could
occur in the following scenario:
• Fraudulent transaction detections in financial
industries
• Rare disease diagnosis
• Customer churn analysis
• The most commonly used approaches include sampling
techniques or harnessing the power of cost-sensitive
algorithms.
Lazy Learners Vs. Eager Learners
• There are two types of learners in machine learning classification: lazy and
eager learners.
Eager learners are machine learning algorithms that first build a model from
the training dataset before making any prediction on future datasets. They
spend more time during the training process because of their eagerness to
have a better generalization during the training from learning the weights,
but they require less time to make predictions.
Most machine learning algorithms are eager learners, and below are some
examples:
• Logistic Regression.
• Support Vector Machine.
• Decision Trees.
• Artificial Neural Networks.
Lazy Learners Vs. Eager Learners
Lazy learners or instance-based learners, on the other hand, do not
create any model immediately from the training data, and this is where
the lazy aspect comes from. They just memorize the training data, and
each time there is a need to make a prediction, they search for the
nearest neighbor from the whole training data, which makes them very
slow during prediction.
Some examples of this kind are:
• K-Nearest Neighbor.
• Case-based reasoning.
 Logistic Regression
• Logistic regression is used for solving the classification problems. Logistic regression
predicts the output of a categorical dependent variable. It can be either Yes or No, 0 or 1, true
or False, etc. but instead of giving the exact value as 0 and 1, it gives the probabilistic values
which lie between 0 and 1.
• Type of Logistic Regression:
❖ Binomial:
It can be only two possible types of the dependent variables, such as 0 or 1, Pass or Fail, etc.
❖ Multinomial:
It can be 3 or more possible unordered types of the dependent variable, such as "cat", "dogs", or
"sheep"
❖ Ordinal:
It can be 3 or more possible ordered types of dependent variables, such as "low", "Medium", or
"High".
 Logistic Regression

• Using Binomial type if the values of dependent

variable (label/target) are closed to two values
such as 0 or 1 and the regression algorithms are
not suitable to learn these data.
 Logistic Regression
The logistic regression algorithm learns the
dataset that have:
• There are one input and one output.
• The output values are closed to two values
such as 0 or 1.
• These values are partitioned into two parts.
Most values at the right are closed to one
value such as 1 while the values at the left are
closed to the other value such as 0 or vice
versa.
Any
Questions ?