Artificial Intelligence (AI)

AI is a branch of computer science that aims to create intelligent machines that mimic
human behaviour such as knowledge, reasoning, problem-solving, perception, learning,
planning, ability to manipulate and move objects

AI is an area of computer science that emphasizes the creation of intelligent machines

that work and react like humans.

Artificial intelligence
In computer science, artificial intelligence ( AI), sometimes called machine intelligence, is intelligence demonstrated…
en.wikipedia.org

What is AI (artificial intelligence)? - Definition from WhatIs.com

Artificial intelligence (AI) is the simulation of human intelligence processes by machines, especially computer…
searchenterpriseai.techtarget.com

Machine Learning (ML)

Machine learning falls under the umbrella of AI, that provides systems with the ability to
automatically learn and improve from experience without being explicitly programmed.

The process of learning begins with observations or data, such as examples, direct
experience, or instruction, in order to look for patterns in data and make better decisions
in the future based on the examples we provide.

The primary aim is to allow the computers to learn automatically without human
intervention or assistance and adjust actions accordingly.

Supervised Learning
Supervised learning is a machine learning task of learning a function that maps an input
to an output based on example input-output pairs. A supervised learning algorithm
analyzes the training data and produces an inferred function, which can be used for
mapping new examples.

In supervised learning, we have labelled training data.

Supervised learning
Supervised learning is the machine learning task of learning a function that maps an input to an output based on…
en.wikipedia.org

Unsupervised Learning
Unsupervised learning is a machine learning task that draws inferences from datasets
consisting of input data without labelled responses. The goal of unsupervised learning is
to model the underlying structure or distribution in the data in order to learn more about
the data.

Clustering and association are some of the unsupervised learning subcategories.

Neural Network or Artificial Neural Network (ANN)

A neural network is a biologically-inspired programming paradigm which enables a
computer to learn from observational data. The design of an artificial neural network is
inspired by the biological neural network of the human brain, leading to a process of
learning that’s far more capable than that of standard machine learning models.

Neural networks, also known as artificial neural networks, consists of input and output
layers, as well as a hidden layer consisting of units that transform the input into
something that the output layer can use. They perform very well in tasks that require to
find patterns.

Back-propagation
It is a concept in neural networks, which allows networks to adjust their hidden layers of
neurons in situations where the outcome doesn’t match what the creator is hoping for.

Deep Neural Network (DNN) or Deep Learning

Deep learning is a subset of machine learning where multiple layers of neural networks
are stacked to create a huge network to map input into the output. It allows the network
to extract different features until it can recognize what it is looking for.

Linear regression
Linear regression is a machine learning algorithm based on supervised learning. It
performs a regression task. Regression models a target prediction value based on
independent variables. It is mostly used for finding out the relationship between variables
and forecasting. One example of a task where linear regression can be used is forecasting
housing price based on past values.

The cost function of linear regression is Root Mean Squared Error (RMSE) between
predicted y value (pred) and true y value (y).

Linear Regression (By Sewaqu — Own work, Public

Logistic regression
Logistic regression is a supervised machine learning algorithm which is used for the
classification problem. It is a classification algorithm used to assign observations to a
discrete set of classes. Some of the examples of classification problems are Email spam or
not spam, Online transaction fraud or not a fraud.

Logistic regression transforms its output using the logistic sigmoid function to return a
probability value.

There are two types of logistic regression:

1. Binary
 2. Multi-class

K-Nearest Neighbors (K-NN)

The k-nearest neighbors (KNN) algorithm is a simple, easy-to-implement supervised
machine learning algorithm that can be used to solve both classification and regression
problems.

The KNN algorithm assumes that similar things exist in close proximity. In other words,
similar things are near to each other.

Can be used on recommendation systems.

KNN works by finding the distances between a query and all the examples in the data,
selecting the specified number examples (K) closest to the query, then voting for the most
frequent label (in the case of classification) or averages the labels (in the case of
regression).
Random forest
Random forest is like a universal machine learning technique that can be used for both
regression and classification purpose. It consists of a large number of individual decision
trees that operate as an ensemble. Each individual decision tree in the random forest spits
out a class prediction and the class with the most votes become our model’s prediction.

In general, a random forest model does not overfit, and even if it does, it is easy to stop it
from overfitting.

There is no need for a separate validation set for a random forest model.

It makes only a few statistical assumptions. Does not assume that your data is normally
distributed, nor it assumes that the relationships are linear.

It requires very few pieces of feature engineering.

Ensemble learning
Ensemble learning helps improve machine learning results by combining several models.
This approach allows the production of better performance compared to a single model.

Ensemble methods are meta-algorithms that combine several machine learning

techniques into one predictive model in order to decrease variance (bagging), bias
(boosting), or improve prediction (stacking).

Examples are random forest, Gradient boosted decision trees, ADA boost.

Gradient boosted decision trees

Boosting is an ensemble technique in which the predictors are not made independently,
but sequentially.

It is a method of converting weak learners into strong learners. Gradient boosting is an

example of boosting. It is a machine learning technique for regression and classification
problems, which produces a prediction model in the form of an ensemble or weak
prediction models, typically decision trees.

Overfitting
Overfitting happens when a model that models the training data too well.

Overfitting happens when a model learns the detail and noise in the training data to the
extent that it negatively impacts the performance of the model on new data. It negatively
impacts the model's ability to generalize.

It can be prevented by:

1. Cross-validation

2. Regularization
Underfitting
Underfitting refers to a model that can neither model the training data nor generalize to
new data. It will have poor performance on the training data.

Regularization
Regularization is a technique to modify machine learning models to avoid the problem of
overfitting. You can apply regularization to any machine learning model. Regularization
simplifies overly complex models that are prone to be overfitted by adding penalty tern to
the objective function. If a model is overfitted, it will have problem generalizing and thus
will give inaccurate predictions when it is exposed to new data sets.

Regularization in Machine Learning

One of the major aspects of training your machine learning model is avoiding overfitting. The model will have a low…
towardsdatascience.com

All you need to know about Regularization

Alice : Hey Bob!!! I have been training my model for 10 hrs but my model is yielding very bad accuracy although it…
towardsdatascience.com

L1 vs L2 regularization
A regression model that uses the L1 regularization technique is called Lasso Regression. A
model which uses the L2 regularization technique is called Rigid Regression.
The key difference between the two is the penalty term which is added to the loss
function.

Rigid regression adds “squared magnitude” of coefficient as penalty term to the loss
function. Lasso regression (Least Absolute Shrinkage and Selection Operator) adds
“absolute value of magnitude” of coefficient as penalty term to the loss function.

L1 L2 Regularization
In this article we will understand why do we need regularization, what is regularization, what are different types of…
medium.com

Regularization for Simplicity: L₂ Regularization | Machine Learning Crash Course | Google…

Estimated Time: 7 minutes Consider the following generalization curve, which shows the loss for both the training set…
developers.google.com

Differences between L1 and L2 as Loss Function and Regularization

2014/11/30: Updated the L1-norm vs L2-norm loss function via a programmatic validated diagram. Thanks readers for
the…
www.chioka.in

Cross-validation
Cross-validation is a technique for evaluating machine learning models by training
several ML models on subsets of the available input data and evaluating them on a
complementary subset of the data. It is used to prevent overfitting of the model.

Different types of cross-validation techniques are:

1. Holdout method

2. K-fold (most popular)

3. Leave-P-out

Cross-Validation
Validation is probably in one of most important techniques that a data scientist use as there is always a need to…
towardsdatascience.com

Why and how to Cross Validate a Model?

Once we are done with training our model, we just can’t assume that it is going to work well on data that it has not…
towardsdatascience.com

Performance metrics for regression

Mean Absolute Error (MAE): measures the average of the absolute difference
between actual and predicted values.

Root Mean Squared Error (RMSE): measures the square root of the average of the
differences of the squares between the actual and the predicted values.

Performance metrics for classification problems

Confusion matrix: It is one of the most intuitive and easiest metrics used for finding
the correctness and accuracy of the model. It is used for classification problem where the
output can be of two or more types of classes.

Confusion Matrix (Source)

True Positives (TP): are the cases when the actual class of the data point was 1 (True)
and the predicted is also 1 (True).

True Negatives (TN): are the cases when the actual class of the data point was 0 (false)
and the predicted is also 0 (False).

False Positives (FP): are the cases when the actual class of the data point was 0 (False)
and the predicted is 1 (True). False is because the model has predicted incorrectly and
positive because the class predicted was a positive one.
False Negatives (FN): are the cases when the actual class of the data point was 1 (True)
and the predicted is 0 (False). False because the model has predicted incorrectly and
negative because the class predicted was a negative one (0).

Accuracy: Accuracy in classification problems is the number of correct predictions made

by the model over all the predictions made.

Accuracy in the confusion matrix (Source)

When to use accuracy: accuracy is a good measure when the target variable classes in
the data are nearly balanced.

When not to use accuracy: accuracy should never be used as a measure when the target
variable classes in the data are a majority of one class.

Precision (hits): Precision is a measure that tells us what proportion of predicted

values as True is actually True.
Recall or sensitivity (misses): Recall is a measure that tells of what proportion of
patients are actually true were predicted as being true by the model.
F1 score: Represents both precision and recall.

F1 Score (Source)

Receiver Operating Characteristic (ROC) curve: An ROC curve is a graph showing

the performance of a classification model at all classification thresholds.

The curve plots two parameters:

1. True Positive Rate (Recall)

2. False Positive Rate (Specificity)

ROC Curve (Source)

AUC (Area Under the ROC Curve): AUC measures the entire two-dimensional area
underneath the entire ROC curve.

It provides an aggregate measure of the performance across all possible classification

thresholds.
Area under ROC curve (Source)

Performance Metrics for Classification problems in Machine Learning

“Numbers have an important story to tell. They rely on you to give them a voice.” — Stephen Few
medium.com

Understanding Confusion Matrix

When we get the data, after data cleaning, pre-processing and wrangling, the first step we do is to feed it to an…
towardsdatascience.com

The topics discussed above were the basics of machine learning. We discussed the basic
terms such as AI, machine learning and deep learning, different types of machine
learning: supervised and unsupervised learning, some machine learning algorithms such
as linear regression, logistic regression, k-nn, and random forest, and performance
evaluation matrices for different algorithms.