Classification in Artifical Intelligence

ICRL Research Lab

Report Supervisor
Kamran Khan

Submitted By
Muhammad Kabir Ahmad

CVIP Research Lab

University of Engineering and Technology, Lahore
Pakistan
‘

Contents
1 Introduction 1

2 Classification 1

3 Classification Algorithms 1
3.1 Logistic Regression . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2
3.1.1 Advantages and Disadvantages of Logistic Regression . . . . . . . . . . . 2
3.2 Decision Tree . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2
3.2.1 Advantages and Disadvantages of Decision Tree . . . . . . . . . . . . . . 2
3.3 Random Forest . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
3.3.1 Advantages and Disadvantages of Random Forest . . . . . . . . . . . . . 3
3.4 Support Vector Machine (SVM) . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
3.4.1 Advantages and Disadvantages of Support Vector Machine (SVM) . . . . 3
3.5 K-Nearest Neighbors (KNN) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
3.5.1 Advantages and Disadvantages of K-Nearest Neighbors (KNN) . . . . . . 3
3.6 Naive Bayes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
3.6.1 Advantages and Disadvantages of Naive Bayes . . . . . . . . . . . . . . . 4
3.7 Neural Network . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
3.7.1 Advantages and Disadvantages of Neural Network . . . . . . . . . . . . . 4

4 Conclusion 4

i
1 Introduction
Artificial Intelligence is basically the ability of machine to work like human, think like hu-
mans to performs various task and actions.To perform this classification plays an essential role.
Classification is the type of Supervised Learning 1 Unlike regression classification is the
identification of classes not the values. Classes can be called as categories or labels.
In this report we will discuss the classification and its algorithms and their advantages and
disadvantages and how the second algorithm covers the weakness of first one.

2 Classification
Classification2 is basically the identification of class of new observation / thing based on the
training dataset. In simple, suppose we have a new observation and we have two classes A
and B. Classification helps us to identify that this observation belongs to which class rather it
belongs to class A or class B.

Figure 1: Shows the Distribution of two classes A and B

3 Classification Algorithms
In this section , Algorithms of classification are discussed below:
• Logistic Regression
• Decision Tree
• Random Forest
• Support Vector Machine (SVM)
• K-Nearest Neighbors (KNN)
• Naive Bayes
• Neural Network

1
Supervised Learning is a technique to identify the class of new observed things on the basis of training data.
2
Leo Breiman, Jerome Friedman, Richard Olshen, and Charles Stone introduced the Classification in 1984.

1
3.1 Logistic Regression
Logistic Regression is one of the most popular Machine
Learning Binary Classification Algorithms. Logistic
regression predicts the probability of an event occurring,
such as voted or didn’t vote, based a given dataset of
independent variables. Since the outcome is the proba-
bility, the dependent variable is bounded between 0 and
1. Email fraud detection and disease detection, etc., can
be detected by Logistic Regression.
Figure 2: Logistic Regression

3.1.1 Advantages and Disadvantages of Logistic Regression

Some Advantages of Logistic Regression are: Suitable for linearly separable datasets. Easier to
Implement. Handles both categorical and numeric features etc.
Some Disadvantages or Weaknesses of Logistic Regression are :Limited to binary classification
tasks.Not suitable for problems with non-linear decision boundaries etc.

3.2 Decision Tree

Decision Tree is a supervised learning algorithm that can
used for both classification and Regression tasks, but it
majorly used in solving classification problem. It named
as Decision Tree because its structure similar to tree
structure as it contains the root node which expands
further to branches and construct the tree like structure.
A decision tree simply asks a question, and based on
the answer (Yes/No), it further split the tree into sub
trees. Decision Trees can capture nonlinear relationships Figure 3: Decision Tree
between target variables and through their recursive par-
titioning process(the divison of tree into subtree).

3.2.1 Advantages and Disadvantages of Decision Tree

Some Advantages of Decision Tree are: Easy to understand. Solve problems with non-linear
decision boundaries. Handle missing values easily by assuming them as new separate branch
etc.
Some Disadvantages or Weaknesses of Decision Tree are : Prone to overfitting (when tree
growns too long). Instability (Small data can cause large change of tree structure) etc.

2
3.3 Random Forest
Random Forest is famous Machine learning algorithm used for both regression and Classification
but majorly used for Classification. Random Forest Cop up the weakness of Decision Tree that
is ’Over-fitting’.It relates the concept of ensemble learning. Random Forest contains multiple
decision trees based on dataset provided and take the average to improve the prediction rather
then relying on one decision tree.

3.3.1 Advantages and Disadvantages of Random Forest

Some Advantages of Random Forest are: Reduces overfitting compared to decision trees. Pro-
vides feature importance.
Some Disadvantages or Weaknesses of Random Forest are : Less interpretable compared to
decision trees. Slower to train and predict compared to decision trees.

3.4 Support Vector Machine (SVM)

SVM algorithm seeks to identify the optimal line or de-
cision boundary that can divide n-dimensional space into
classes. A hyperplane is the name given to this optimal
decision boundary.
SVM chooses the extreme points/vectors that help in cre-
ating the hyperplane. These extreme cases are called as
support vectors, and hence algorithm is termed as Sup-
port Vector Machine
Figure 4: SVM

3.4.1 Advantages and Disadvantages of Support Vector Machine (SVM)

Some Advantages of SVM are: Effective in high-dimensional spaces.
Some Disadvantages or Weaknesses of SVM are : Memory-intensive for large datasets. Can be
sensitive to the choice of kernel parameters

3.5 K-Nearest Neighbors (KNN)

K-NN algorithm assumes the similarity between the new case/data and available cases and
put the new case into the category that is most similar to the available categories. K-NN is a
non-parametric algorithm 3 and lazy learner algorithm 4

3.5.1 Advantages and Disadvantages of K-Nearest Neighbors (KNN)

Some Advantages of KNN are: Simple and Observant. No training phase.
Some Disadvantages or Weaknesses of KNN are : Sensitive to irrelevant or duplicate features.

3
Non-parametric algorithm means it does not make any assumption on underlying data.
4
it does not learn from the training set immediately instead it stores the dataset and at the time of classifi-
cation, it performs an action on the dataset.

3
3.6 Naive Bayes
It based on Baye’s Theorem 5 It is probabilistic classifier.6 . Naive Bayes overcomes some
weaknesses of K-Nearest Neighbors (KNN) through its probabilistic approach and assumption
of feature independence.

3.6.1 Advantages and Disadvantages of Naive Bayes

Some Advantages of Naive Bayes are: Simple, Fast and easy to implement and detect the class
of dataset. Requires a small amount of training data.
Some Disadvantages or Weaknesses of Naive Bayes are : Makes the assumption that all features
are independent, which is frequently not the case in real data. Limited ability to express oneself.

3.7 Neural Network

A neural network is a method in artificial intelligence
that teaches computers to process data in a way that is
inspired by the human brain. It is a type of machine
learning process, called deep learning, that uses inter-
connected nodes or neurons in a layered structure that
resembles the human brain Figure 5: Neural Network

3.7.1 Advantages and Disadvantages of Neural Network

Some Advantages of Neural Network are: Can learn complex relationships in the data.Highly
flexible and scalable.
Some Disadvantages or Weaknesses of Neural Network are :Require large amounts of data and
computational resources for training.Overfitting in deep architecture.

4 Conclusion
The seven algorithms of Classification are briefly described in this report .Their Strengths and
weaknesses are highlighted above. Last type of classification that is Neural Network that is
reffered as Deep Learning plays essential role in the development of Artifical Intelligence now
a days.

5
Bayes’ theorem is also known as Bayes’ Rule or Bayes’ law, which is used to determine the probability of a
hypothesis with prior knowledge. It depends on the conditional probability.
6
Probabilistic classifier means it predicts on the basis of the probability of an object.