Introduction To

Artificial Intelligence
 What is AI?
• Artificial Intelligence is composed of two words Artificial and
Intelligence, where Artificial defines "man-made," and intelligence
defines "thinking power", hence AI means "a man-made thinking
power.<
• It is a branch of computer science by which we can create intelligent
machines which can behave like a human, think like humans, and able
to make decisions.
 AI Definitions
• <Intelligence: The ability to learn and solve problems=
– Webster’s Dictionary.

• <Artificial intelligence (AI) is the intelligence exhibited by machines or

software’
– Wikipedia.

• <The science and engineering of making intelligent machines=

– McCarthy.

• <The study and design of intelligent agents, where an intelligent agent

is a system that perceives its environment and takes actions that
maximize its chances of success.=
– Russel and Norvig AI book.
 Why Artificial Intelligence?
• With the help of AI, you can create such software or devices which can
solve real-world problems very easily and with accuracy such as health
issues, marketing, traffic issues, etc.
• With the help of AI, you can create your personal virtual Assistant, such
as Cortana, Google Assistant, Siri, etc.
• With the help of AI, you can build such Robots which can work in an
environment where survival of humans can be at risk.
• AI opens a path for other new technologies, new devices, and new
Opportunities.
 Need for AI
• AI can be used to make an intelligent decision as compared to
human beings.
• AI can be used to analyze data more deeply.
• To maintain security and privacy of data more effectively and
efficiently.
• To create expert systems having the capability to explain and
give advice to users.
• AI can also be used to speed up the work performance.
 Applications of AI
• Business • Finance
• Healthcare • Data Security
• Education • Social Media
• Robotics • Travel & Transport
• Autonomous Vehicles • Automotive Industry
• Agriculture • Entertainment
• Astronomy • E-commerce
• Gaming
 Advantages of AI
• High Accuracy with less errors
• High-Speed and High Reliability
• Useful for risky areas
• Digital Assistant
• Useful as a public utility
 Disadvantages of AI
• High Cost
• Can replace humans
• In wrong hands, machines may lead to destruction.
• Makes human to dependent on it
• Can't think out of the box
• No feelings and emotions
• Increase dependency on machines
• No Original Creativity
Foundation of AI
 Components of AI
• Reasoning, problem-solving
• Knowledge Representation
• Planning
• Learning
• Natural Language Processing
• Perception
 History of AI

• Turing Machine and Turing Test

• The term AI was coined
• First Chatbot
• AI in the medical field.
• Voice of the Computer: NETtalk
• AI Today
 Types of AI
• Based on Capabilities
– Weak AI or Narrow AI
– General AI
– Super AI
• Based on functionality
– Reactive Machines
– Limited Memory
– Theory of Mind
– Self-Awareness
 Intelligent Agents
• Agent: An agent is anything that can be viewed as:
– perceiving its environment through sensors and
– acting upon that environment through actuators
 Agents in AI

• Human-Agent
• Robotic Agent
• Software Agent
• Sensor
• Actuators
• Effectors
 Rules of AI Agent
• Rule 1: An AI agent must have the ability to perceive the
environment.
• Rule 2: The observation must be used to make decisions.
• Rule 3: Decision should result in an action.
• Rule 4: The action taken by an AI agent must be a rational
action.
 Rational Agent

• For each possible percept sequence, a rational agent

should select an action that is expected to maximize
its performance measure, given the evidence
provided by the percept sequence and whatever
built-in knowledge the agent has.
 Rationality
• Rationality is relative to a performance measure.
• Judge rationality based on:
– Performance measure which defines the success criterion.
– Agent prior knowledge of its environment.
– Best possible actions that an agent can perform.
– The sequence of percepts.
 PEAS Representation
• When we define an AI agent or rational agent, then we can
group its properties under PEAS representation model.
• P: Performance measure
• E: Environment
• A: Actuators
• S: Sensors
 PEAS Representation
• PEAS for self-driving cars:
– Performance: Safety, time, legal drive, comfort.
– Environment: Roads, other cars, pedestrians, road signs.
– Actuators: Steering, accelerator, brake, signal, horn.
– Sensors: Camera, sonar, GPS, Speedometer, odometer,
accelerometer, engine sensors, keyboard.
 PEAS Representation
• PEAS for Vacuum Cleaner
– Performance: cleanness, efficiency: distance travelled to
clean, battery life, security.
– Environment: room, table, wood floor, carpet, different
obstacles.
– Actuators: wheels, different brushes, vacuum extractor.
– Sensors: camera, dirt detection sensor, cliff sensor, bump
sensors, infrared wall sensors.
 Agent Environment in AI
• Fully observable vs Partially Observable
• Static vs Dynamic
• Discrete vs Continuous
• Deterministic vs Stochastic
• Single-agent vs Multi-agent
• Episodic vs sequential
• Known vs Unknown
• Accessible vs Inaccessible
 Agent types
• Simple Reflex Agent
• Model-based reflex agent
• Goal-based agents
• Utility-based agent
• Learning agent
 Agent types
• Simple reflex agents select an action based on the current
state only ignoring the percept history.
 Agent types
• Model-based reflex agents - Handle partial observability by
keeping track of the part of the world it can’t see now.
 Agent types
• Goal-based agents - Knowing the current state of the
environment is not enough. The agent needs some goal
information
 Agent types
• Utility-based agents - Sometimes achieving the desired goal is
not enough. We may look for quicker, safer, cheaper trip to
reach a destination.
 Agent types
• Learning agents - A learning agent in AI is the type of agent
which can learn from its past experiences, or it has learning
capabilities.
 Agent’s organization
• Atomic Representation

• Factored Representation • Structured Representation

 Search Algorithms in AI
• Problem-solving agents (Rational agents)
• Goal-based agents
 Problem solving as search
• Define the problem through:
– Goal formulation
– Problem formulation
• Solving the problem as a 2-stage process:
– Search: <mental= or <offline= exploration of several
possibilities
– Execute the solution found
 Problem formulation
• Initial state
• States
• Actions
• Transition model
• Goal test
• Path cost
 Examples
• The 8-queen problem:

• 8 puzzles:
 Real-world examples
• Route finding problem
• Traveling salesperson problem
• VLSI layout
• Robot navigation
• Automatic assembly sequencing
• Protein design
 Search Algorithm Terminologies

• Search
– Search Space
– Start State
– Goal test
• Search tree
• Actions
• Transition model
• Path Cost
• Solution
• Optimal Solution
 Properties of Search Algorithm
• Completeness
• Optimality
• Time Complexity
• Space Complexity
 Types of search algorithms
• Uninformed/Blind Search:
– Breadth-first search
– Uniform cost search
– Depth-first search
– Iterative deepening depth-first search
– Bidirectional Search
• Informed Search
– Greedy Search
– A* Search
 Uninformed Search Algorithms
• Uninformed search is a class of general-purpose search
algorithms which operates in brute force-way. Uninformed
search algorithms do not have additional information about
state or search space other than how to traverse the tree, so it
is also called blind search.
 Types of uninformed search algorithms
• Breadth-first Search
 Types of uninformed search algorithms
• Depth-first Search
 Types of uninformed search algorithms
• Depth-Limited Search Algorithm
 Types of uninformed search algorithms
• Uniform-cost Search Algorithm
 Types of uninformed search algorithms
• Iterative deepening depth-first Search
 Types of uninformed search algorithms
• Bidirectional Search Algorithm
 Informed Search Algorithms
• Heuristics function: It is a function which is used in Informed
Search, and it finds the most promising path.
• h(n) <= h*(n) , Here h(n) is heuristic cost, and h*(n) is the
estimated cost. Hence heuristic cost should be less than or
equal to the estimated cost.
• Pure heuristic search is the simplest form of heuristic search
algorithms. It expands nodes based on their heuristic value
h(n).
 Best first search algorithm
• Best-first Search Algorithm (Greedy Search) - Greedy best-
first search algorithm always selects the path which appears
best at that moment.
• f(n)= h(n), where h(n)= estimated cost from node n to the
goal.
• The greedy best first algorithm is implemented by the
priority queue
 Best first search algorithm
• Step 1: Place the starting node into the OPEN list.
• Step 2: If the OPEN list is empty, Stop and return failure.
• Step 3: Remove the node n, from the OPEN list which has the lowest value
of h(n), and places it in the CLOSED list.
• Step 4: Expand the node n, and generate the successors of node n.
• Step 5: Check each successor of node n, and find whether any node is a
goal node or not. If any successor node is goal node, then return success
and terminate the search, else proceed to Step 6.
• Step 6: For each successor node, algorithm checks for evaluation function
f(n), and then check if the node has been in either OPEN or CLOSED list. If
the node has not been in both list, then add it to the OPEN list.
• Step 7: Return to Step 2.
Best first search algorithm
 A* Search Algorithm
• A* search is the most commonly known form of best-first
search. It uses heuristic function h(n), and cost to reach the
node n from the start state g(n).
• Combines:
– g(n): cost to reach node n
– h(n): cost to get from n to the goal
– f(n) = g(n) + h(n)
 A* Search Algorithm
• Step1: Place the starting node in the OPEN list.
• Step 2: Check if the OPEN list is empty or not, if the list is empty then
return failure and stops.
• Step 3: Select the node from the OPEN list which has the smallest value of
evaluation function (g+h), if node n is goal node then return success and
stop, otherwise
• Step 4: Expand node n and generate all of its successors, and put n into
the closed list. For each successor n', check whether n' is already in the
OPEN or CLOSED list, if not then compute evaluation function for n' and
place into Open list.
• Step 5: Else if node n' is already in OPEN and CLOSED, then it should be
attached to the back pointer which reflects the lowest g(n') value.
• Step 6: Return to Step 2.
A* Search Algorithm
A* Search Algorithm
 Hill Climbing Algorithm
• Hill climbing algorithm is a local search algorithm which
continuously moves in the direction of increasing
elevation/value to find the peak of the mountain or best
solution to the problem.
• A node of hill climbing algorithm has two components which
are state and value.
• In this algorithm, we don't need to maintain and handle the
search tree or graph as it only keeps a single current state.
 Features of Hill Climbing
• Generate and Test variant: Hill Climbing is the variant of
Generate and Test method. The Generate and Test method
produce feedback which helps to decide which direction to
move in the search space.
• Greedy approach: Hill-climbing algorithm search moves in the
direction which optimizes the cost.
• No backtracking: It does not backtrack the search space, as it
does not remember the previous states.
 State-space Diagram
• Local Maximum
• Global Maximum
• Current state
• Flat local maximum
• Shoulder
 Types of Hill Climbing Algorithm
• Simple Hill Climbing - It only evaluates the neighbor node state at a time and
selects the first one which optimizes current cost and set it as a current state.
• Step 1: Evaluate the initial state, if it is goal state then return success and Stop.
• Step 2: Loop Until a solution is found or there is no new operator left to apply.
• Step 3: Select and apply an operator to the current state.
• Step 4: Check new state:
– If it is goal state, then return success and quit.
– Else if it is better than the current state then assign new state as a current
state.
– Else if not better than the current state, then return to step2.
• Step 5: Exit.
 Types of Hill Climbing Algorithm
• Steepest-Ascent hill climbing – This algorithm examines all the
neighboring nodes of the current state and selects one neighbor node
which is closest to the goal state.
• Step 1: Evaluate the initial state, if it is goal state then return success and stop, else make
current state as initial state.
• Step 2: Loop until a solution is found or the current state does not change.
– Let SUCC be a state such that's any successor of the current state will be better than it.
– For each operator that applies to the current state:
• Apply the new operator and generate a new state.
• Evaluate the new state.
• If it is goal state, then return it and quit, else compare it to the SUCC.
• If it is better than SUCC, then set new state as SUCC.
• If the SUCC is better than the current state, then set current state to SUCC.
• Step 5: Exit.
 Types of Hill Climbing Algorithm
• Stochastic hill climbing
• Stochastic hill climbing does not examine for all its neighbor
before moving.
• Rather, this search algorithm selects one neighbor node at
random and decides whether to choose it as a current state or
examine another state.
 Difference between Informed
and Uninformed Search in AI
Parameters Informed Search Uninformed Search
Known as It is also known as Heuristic Search. It is also known as Blind Search.

It uses knowledge for the searching It doesn’t use knowledge for the
Using Knowledge
process. searching process.

It finds solution slow as compared to

Performance It finds a solution more quickly.
an informed search.

Completion It may or may not be complete. It is always complete.

Cost Factor Cost is low. Cost is high.

It consumes less time because of It consumes moderate time because

Time
quick searching. of slow searching.

There is a direction given about the No suggestion is given regarding the

Direction
solution. solution in it.
 Difference between Informed
and Uninformed Search in AI
Parameters Informed Search Uninformed Search
Implementation It is less lengthy while implemented. It is more lengthy while implemented.

It is more efficient as efficiency takes into

It is comparatively less efficient as
account cost and performance. The
Efficiency incurred cost is more and the speed of
incurred cost is less and speed of finding
finding the Breadth-Firstsolution is slow.
solutions is quick.

Computational requirements are Comparatively higher computational

Computational requirements
lessened. requirements.

Having a wide scope in terms of handling Solving a massive search task is

Size of search problems
large search problems. challenging.

•Greedy Search
•Depth First Search (DFS)
•A* Search
Examples of Algorithms •Breadth First Search (BFS)
•AO* Search
•Branch and Bound
•Hill Climbing Algorithm
 Adversarial Search
• Adversarial search is a search, where we examine the problem
which arises when we try to plan ahead of the world and
other agents are planning against us.
• The environment with more than one agent is termed as
multi-agent environment, in which each agent is an opponent
of other agent and playing against each other.
• Searches in which two or more players with conflicting goals
are trying to explore the same search space for the solution,
are called adversarial searches, often known as Games.
 Types of Games in AI
• Perfect information
– Chess, Checkers, Go, etc.
• Imperfect information
– Tic-tac-toe, Battleship, blind, Bridge, etc.
• Deterministic games
– Checkers, Go, tic-tac-toe, etc.
• Non-deterministic games
– Backgammon, Monopoly, Poker, etc.
 Zero-Sum Game
• Zero-sum games are adversarial search which involves pure
competition.
• Zero-sum game: Embedded thinking
• What to do.
• How to decide the move
• Needs to think about his opponent as well
• The opponent also thinks what to do
 Zero-Sum Game
• Formalization of the problem:
– Initial state
– Player(s)
– Action(s)
– Result(s, a)
– Terminal-Test(s)
– Utility(s, p)
 Example:
Tic-Tac-Toe game tree
• There are two players
MAX and MIN.
• Players have an alternate
turn and start with MAX.
• MAX maximizes the
result of the game tree
• MIN minimizes the result.
 Mini-Max Algorithm in AI
• Mini-max algorithm is a recursive or backtracking algorithm
which is used in decision-making and game theory. It provides
an optimal move for the player assuming that opponent is also
playing optimally.
• Min-Max algorithm is mostly used for game playing in AI. Such
as Chess, Checkers, tic-tac-toe, go, and various two-players
game. This Algorithm computes the minimax decision for the
current state.
• The minimax algorithm performs a depth-first search
algorithm for the exploration of the complete game tree.
 Working of Min-Max Algorithm
• Step 1: In the first step, the algorithm generates the entire game-
tree and apply the utility function to get the utility values for the
terminal states.
• Step 2: Now, first we find the utilities value for the Maximizer, its
initial value is -∞, so we will compare each value in terminal state
with initial value of Maximizer and determines the higher nodes
values.
• Step 3: In the next step, it's a turn for minimizer, so it will compare
all nodes value with +∞, and will find the 3rd layer node values.
• Step 4: Now it's a turn for Maximizer, and it will again choose the
maximum of all nodes value and find the maximum value for the
root node.
 Properties of Mini-Max algorithm
• Complete
• Optimal
• Time complexity
• Space Complexity

• Limitation - The main drawback of the minimax algorithm is

that it gets really slow for complex games such as Chess, go,
etc. This type of games has a huge branching factor, and the
player has lots of choices to decide.
 Alpha-Beta Pruning
• Alpha-beta pruning is a modified version of the minimax
algorithm. It is an optimization technique for the minimax
algorithm.
• There is a technique by which without checking each node of
the game tree we can compute the correct minimax decision,
and this technique is called pruning.
• It is also called as Alpha-Beta Algorithm.
 Alpha-Beta Pruning
• The two-parameter can be defined as:
– Alpha: The best (highest-value) choice we have found so far
at any point along the path of Maximizer. The initial value
of alpha is -∞.
– Beta: The best (lowest-value) choice we have found so far
at any point along the path of Minimizer. The initial value
of beta is +∞.
• Condition for Alpha-beta pruning: α>=β
 Working of Alpha-Beta Pruning
• Step 1: At the first step the,
Max player will start first
move from node A where α=
-∞ and β= +∞, these value
of alpha and beta passed
down to node B where again
α= -∞ and β= +∞, and Node
B passes the same value to
its child D.
 Working of Alpha-Beta Pruning
• Step 2: At Node D, the value of α will be
calculated as its turn for Max. The value
of α is compared with firstly 2 and then
3, and the max (2, 3) = 3 will be the
value of α at node D and node value will
also 3.
• Step 3: Now algorithm backtrack to node
B, where the value of β will change as
this is a turn of Min, Now β= +∞, will
compare with the available subsequent
nodes value, i.e. min (∞, 3) = 3, hence at
node B now α= -∞, and β= 3.
 Working of Alpha-Beta Pruning
• Step 4: At node E, Max will take
its turn, and the value of alpha
will change. The current value
of alpha will be compared with
5, so max (-∞, 5) = 5, hence at
node E α= 5 and β= 3, where
α>=β, so the right successor of E
will be pruned, and algorithm
will not traverse it, and the
value at node E will be 5.
 Working of Alpha-Beta Pruning
• Step 5: At next step, algorithm again
backtrack the tree, from node B to
node A. At node A, the value of alpha
will be changed the maximum
available value is 3 as max (-∞, 3)= 3,
and β= +∞, these two values now
passes to right successor of A which
is Node C.
• Step 6: At node F, again the value of α
will be compared with left child
which is 0, and max(3,0)= 3, and then
compared with right child which is 1,
and max(3,1)= 3 still α remains 3, but
the node value of F will become 1.
 Working of Alpha-Beta Pruning
• Step 7: Node F returns the
node value 1 to node C, at C α=
3 and β= +∞, here the value of
beta will be changed, it will
compare with 1 so min (∞, 1) =
1. Now at C, α=3 and β= 1, and
again it satisfies the condition
α>=β, so the next child of C
which is G will be pruned, and
the algorithm will not compute
the entire sub-tree G.
 Working of Alpha-Beta Pruning
• Step 8: C now returns the
value of 1 to A here the best
value for A is max (3, 1) = 3.
Following is the final game tree
which is the showing the
nodes which are computed
and nodes which has never
computed. Hence the optimal
value for the maximizer is 3 for
this example.
 Move Ordering andRules
to find good ordering
• Move Ordering can be of two types:
– Worst ordering:The time complexity for such an order is O(bm).
– Ideal ordering: Complexity in ideal ordering is O(bm/2).
• Rules to find good ordering:
– Occur the best move from the shallowest node.
– Order the nodes in the tree such that the best nodes are checked first.
– Use domain knowledge while finding the best move. Ex: for Chess, try
order: captures first, then threats, then forward moves, backward moves.
– We can bookkeep the states, as there is a possibility that states may
repeat.
 Machine Learning
• Machine learning is a growing technology which enables
computers to learn automatically from past data.
• Machine learning uses various algorithms for building
mathematical models and making predictions using historical
data or information
• Machine Learning is said as a subset of artificial intelligence
that is mainly concerned with the development of algorithms
which allow a computer to learn from the data and past
experiences on their own.
 Machine Learning
• Machine learning enables a machine to automatically learn
from data, improve performance from experiences, and
predict things without being explicitly programmed.
• A machine has the ability to learn if it can improve its
performance by gaining more data.
• A Machine Learning system learns from historical data, builds
the prediction models, and whenever it receives new data,
predicts the output for it.
 Features of Machine Learning
• Machine learning uses data to detect various patterns in a
given dataset.
• It can learn from past data and improve automatically.
• It is a data-driven technology.
• Machine learning is much similar to data mining as it also
deals with the huge amount of the data.
 Need for Machine Learning
• The reason behind the need for machine learning is that it is
capable of doing tasks that are too complex for a person to
implement directly.
• We can train machine learning algorithms by providing them
the huge amount of data and let them explore the data,
construct the models, and predict the required output
automatically.
• Currently, machine learning is used in self-driving cars, cyber
fraud detection, face recognition, etc.
 Need for Machine Learning
• Rapid increment in the production of data
• Solving complex problems, which are difficult for a human
• Decision making in various sector including finance
• Finding hidden patterns and extracting useful information
from data.
 Classification of Machine Learning
• Supervised Learning
– Supervised learning is a type of machine learning method
in which we provide sample labeled data to the machine
learning system in order to train it, and on that basis, it
predicts the output.
• Classification
• Regression
 Classification of Machine Learning
• Unsupervised Learning
– Unsupervised learning is a learning method in which a
machine learns without any supervision.
• Clustering
• Association
 Classification of Machine Learning
• Reinforcement Learning
– Reinforcement learning is a feedback-based learning
method, in which a learning agent gets a reward for each
right action and gets a penalty for each wrong action.
– The agent learns automatically with these feedbacks and
improves its performance.
 Applications of ML
• Image Recognition
• Speech Recognition
• Traffic prediction
• Product recommendations
• Self-driving cars
• Email Spam and Malware Filtering
• Virtual Personal Assistant
• Online Fraud Detection
• Stock Market trading
• Medical Diagnosis
 Difference between AI and ML
• Artificial intelligence and machine learning are the part of
computer science that are correlated with each other.
• AI is a bigger concept to create intelligent machines that can
simulate human thinking capability and behavior, whereas,
machine learning is an application or subset of AI that allows
machines to learn from data without being programmed
explicitly.
 Difference between AI and ML
• Artificial Intelligence - Artificial intelligence is a technology
using which we can create intelligent systems that can
simulate human intelligence.
– Weak AI
– General AI
– Strong AI
 Difference between AI and ML
• Machine learning - Machine learning is a subfield of artificial
intelligence, which enables machines to learn from past data
or experiences without being explicitly programmed.
– Supervised learning
– Reinforcement learning
– Unsupervised learning
 Machine Learning Life cycle

• Gathering Data
– Identify various data sources
– Collect data
– Integrate the data obtained from different sources
• Data preparation
– Data exploration
– Data pre-processing
 Machine Learning Life cycle
• Data Wrangling
– Missing Values
– Duplicate data
– Invalid data
– Noise
• Data Analysis
– Selection of analytical techniques
– Building models
– Review the result
 Machine Learning Life cycle

• Train Model
– We train our model to improve its performance for better
outcome of the problem.
• Test Model
– We check for the accuracy of our model by providing a test
dataset to it.
• Deployment
– Where we deploy the model in the real-world system.
 Supervised Machine Learning
• Supervised learning is the types of machine learning in which
machines are trained using well "labelled" training data, and
on basis of that data, machines predict the output.
• The aim of a supervised learning algorithm is to find a
mapping function to map the input variable(x) with the output
variable(y).
• In the real-world, supervised learning can be used for Risk
Assessment, Image classification, Fraud Detection, spam
filtering, etc.
 Steps Involved in Supervised Learning
• First Determine the type of training dataset
• Collect/Gather the labelled training data.
• Split the training dataset into training dataset, test dataset, and validation
dataset.
• Determine the input features of the training dataset, which should have
enough knowledge so that the model can accurately predict the output.
• Determine the suitable algorithm for the model, such as support vector
machine, decision tree, etc.
• Execute the algorithm on the training dataset. Sometimes we need
validation sets as the control parameters, which are the subset of training
datasets.
• Evaluate the accuracy of the model by providing the test set. If the model
predicts the correct output, which means our model is accurate.
 Types of supervised Machine learning Algorithms
• Regression algorithms are used if there is a relationship
between the input variable and the output variable. It is used
for the prediction of continuous variables, such as Weather
forecasting, Market Trends, etc.
• Classification algorithms are used when the output variable is
categorical, which means there are two classes such as Yes-No,
Male-Female, True-false, etc.
 Advantages and
Disadvantages
• The model can predict the output on the basis of prior
experiences.
• We can have an exact idea about the classes of objects.
• Helps us to solve various real-world problems such as fraud
detection, spam filtering, etc.

• Not suitable for handling the complex tasks.

• Cannot predict the correct output if the test data is different from
the training dataset.
• Training required lots of computation times.
• We need enough knowledge about the classes of object.
 Regression Analysis in ML
• Regression analysis is a statistical method to model the
relationship between a dependent (target) and independent
(predictor) variables with one or more independent variables.
• It predicts continuous/real values such as temperature, age,
salary, price, etc.
• Examples:
• Prediction of rain using temperature and other factors
• Determining Market trends
• Prediction of road accidents due to rash driving.
 Terminologies
• Dependent Variable
• Independent Variable
• Outliers
• Multicollinearity
• Underfitting and Overfitting
 Why Regression Analysis?
• Regression estimates the relationship between the target and
the independent variable.
• It is used to find the trends in data.
• It helps to predict real/continuous values.
• By performing the regression, we can confidently determine
the most important factor, the least important factor, and how
each factor is affecting the other factors.
 Types of Regression
• Linear Regression
• Logistic Regression
• Polynomial Regression
• Support Vector Regression
• Decision Tree Regression
• Random Forest Regression
• Ridge Regression
 Linear Regression
• Linear regression is a statistical regression method which is
used for predictive analysis.
• It is used for solving the regression problem in machine
learning.
• If there is only one input variable (x), then such linear
regression is called simple linear regression. And if there is
more than one input variable, then such linear regression is
called multiple linear regression.
Linear Regression
 Logistic Regression
• Logistic regression is another supervised learning algorithm
which is used to solve the classification problems. In
classification problems, we have dependent variables in a
binary or discrete format such as 0 or 1.
• Logistic regression algorithm works with the categorical
variable such as 0 or 1, Yes or No, True or False, Spam or not
spam, etc.
• Logistic regression is a type of regression, but it is different
from the linear regression algorithm in the term how they are
used.
Logistic Regression
 Polynomial Regression
• Polynomial Regression is a type of regression which models
the non-linear dataset using a linear model.
• It is similar to multiple linear regression, but it fits a non-linear
curve between the value of x and corresponding conditional
values of y.
• In Polynomial regression, the original features are transformed
into polynomial features of given degree and then modeled
using a linear model. Which means the datapoints are best
fitted using a polynomial line.
Polynomial Regression
 Support Vector Regression
• Kernel: It is a function used to map a lower-dimensional data
into higher dimensional data.
• Hyperplane: In general SVM, it is a separation line between
two classes, but in SVR, it is a line which helps to predict the
continuous variables and cover most of the datapoints.
• Boundary line: Boundary lines are the two lines apart from
hyperplane, which creates a margin for datapoints.
• Support vectors: Support vectors are the datapoints which are
nearest to the hyperplane and opposite class.
Support Vector Regression
 Decision Tree Regression
• Decision Tree is a supervised learning algorithm which can be
used for solving both classification and regression problems.
• It can solve problems for both categorical and numerical data
• Decision Tree regression builds a tree-like structure in which
each internal node represents the "test" for an attribute, each
branch represent the result of the test, and each leaf node
represents the final decision or result.
Decision Tree Regression
 Random Forest Regression
• Random forest is one of the most powerful supervised
learning algorithms which is capable of performing regression
as well as classification tasks.
• The Random Forest regression is an ensemble learning
method which combines multiple decision trees and predicts
the final output based on the average of each tree output.
• The combined decision trees are called as base models, and it
can be represented more formally as:
• g(x)= f0(x)+ f1(x)+ f2(x)+....
Random Forest Regression
 Ridge Regression
• Ridge regression is one of the most robust versions of linear
regression in which a small amount of bias is introduced so
that we can get better long term predictions.
• The amount of bias added to the model is known as Ridge
Regression penalty. We can compute this penalty term by
multiplying with the lambda to the squared weight of each
individual features.
• Ridge regression is a regularization technique, which is used to
reduce the complexity of the model. It is also called as L2
regularization.
 Classification Algorithm in ML

• The Classification algorithm is a Supervised Learning

technique that is used to identify the category of new
observations on the basis of training data.
• Yes or No, 0 or 1, Spam or Not Spam, cat or dog, etc. Classes
can be called as targets/labels or categories.
• y=f(x), where y = categorical output
– Binary Classifier
– Multi-class Classifier
Classification Algorithm in ML
 Learners in Classification Problems
• Lazy Learners:
– Lazy Learner firstly stores the training dataset and wait until it
receives the test dataset.
– Lazy learner case, classification is done on the basis of the most
related data stored in the training dataset.
– Example: K-NN algorithm, Case-based reasoning
• Eager Learners:
– Eager Learners develop a classification model based on a
training dataset before receiving a test dataset.
– Example: Decision Trees, Naïve Bayes, ANN.
 Evaluating a Classification model
• Log Loss or Cross-Entropy Loss:
– It is used for evaluating the performance of a classifier,
whose output is a probability value between the 0 and 1.
– For a good binary Classification model, the value of log loss
should be near to 0.
– The value of log loss increases if the predicted value
deviates from the actual value.
– The lower log loss represents the higher accuracy of the
model.
 Evaluating a Classification model
• Confusion Matrix:
– The confusion matrix provides us a matrix/table as output and
describes the performance of the model.
– It is also known as the error matrix.
– The matrix consists of predictions result in a summarized form,
which has a total number of correct predictions and incorrect
predictions. The matrix looks like as below table:
Actual Positive Actual Negative
Predicted Positive True Positive False Positive
Predicted Negative False Negative True Negative
 Evaluating a Classification model
• AUC-ROC curve:
– ROC curve stands for Receiver Operating Characteristics Curve
and AUC stands for Area Under the Curve.
– It is a graph that shows the performance of the classification
model at different thresholds.
– To visualize the performance of the multi-class classification
model, we use the AUC-ROC Curve.
– The ROC curve is plotted with TPR and FPR, where TPR (True
Positive Rate) on Y-axis and FPR(False Positive Rate) on X-axis.
 Types of Classification Algorithm
• Linear Models
– Logistic Regression
– Support Vector Machines
• Non-linear Models
– K-Nearest Neighbours
– Kernel SVM
– Naïve Bayes
– Decision Tree Classification
– Random Forest Classification
 Logistic Regression
• Logistic regression is one of the most popular Machine
Learning algorithms, which comes under the Supervised
Learning technique. It is used for predicting the categorical
dependent variable using a given set of independent variables.
• Assumptions for Logistic Regression:
– The dependent variable must be categorical in nature.
– The independent variable should not have multi-
collinearity.
 Support Vector Machine Algorithm

• The goal of the SVM algorithm is to create the best line or

decision boundary that can segregate n-dimensional space
into classes so that we can easily put the new data point in the
correct category in the future. This best decision boundary is
called a hyperplane.
• SVM chooses the extreme points/vectors that help in creating
the hyperplane.
Support Vector Machine Algorithm
 K-Nearest Neighbour(KNN)
• K-NN algorithm assumes the similarity between the new
case/data and available cases and put the new case into the
category that is most similar to the available categories.
• K-NN is a non-parametric algorithm, which means it does not
make any assumption on underlying data.
• KNN algorithm at the training phase just stores the dataset
and when it gets new data, then it classifies that data into a
category that is much similar to the new data.
K-Nearest Neighbour(KNN)
 K-Nearest Neighbour(KNN)
• Step-1: Select the number K of the neighbors
• Step-2: Calculate the Euclidean distance of K number of
neighbors
• Step-3: Take the K nearest neighbors as per the calculated
Euclidean distance.
• Step-4: Among these k neighbors, count the number of the
data points in each category.
• Step-5: Assign the new data points to that category for which
the number of the neighbor is maximum.
• Step-6: Our model is ready.
 Advantages & Disadvantages of KNN

• It is simple to implement.
• It is robust to the noisy training data
• It can be more effective if the training data is large.

• Always needs to determine the value of K which may be

complex some time.
• The computation cost is high because of calculating the
distance between the data points for all the training samples.
 Naïve Bayes Classifier Algorithm
• Naïve Bayes algorithm is a supervised learning algorithm,
which is based on Bayes theorem and used for solving
classification problems.
• It is mainly used in text classification that includes a high-
dimensional training dataset.
• It is a probabilistic classifier, which means it predicts on the
basis of the probability of an object.
 Naïve Bayes Classifier Algorithm
• Naïve: It is called Naïve because it assumes that the
occurrence of a certain feature is independent of the
occurrence of other features. Such as if the fruit is identified
on the bases of color, shape, and taste, then red, spherical,
and sweet fruit is recognized as an apple. Hence each feature
individually contributes to identify that it is an apple without
depending on each other.
• Bayes: It is called Bayes because it depends on the principle of
Bayes' Theorem.
 Working of Naïve Bayes' Classifier
• Step 1: Convert the given dataset into frequency tables.
• Step 2: Generate Likelihood table by finding the probabilities
of given features.
• Step 3: Now, use Bayes theorem to calculate the posterior
probability.
 Types of Naïve Bayes Model
• Gaussian: The Gaussian model assumes that features follow a
normal distribution.
• Multinomial: The Multinomial Naïve Bayes classifier is used
when the data is multinomial distributed.
• Bernoulli: The Bernoulli classifier works similar to the
Multinomial classifier, but the predictor variables are the
independent Booleans variables.
 Advantages & Disadvantages

• Naïve Bayes is one of the fast and easy ML algorithms to predict a

class of datasets.
• It can be used for Binary as well as Multi-class Classifications.
• It performs well in Multi-class predictions as compared to the
other Algorithms.
• It is the most popular choice for text classification problems.

• Naive Bayes assumes that all features are independent or

unrelated, so it cannot learn the relationship between features.
 Decision Tree Classification Algorithm
• It is a tree-structured classifier, where internal nodes
represent the features of a dataset, branches represent the
decision rules and each leaf node represents the outcome.
• It is a graphical representation for getting all the possible
solutions to a problem/decision based on given conditions.
• In order to build a tree, we use the CART algorithm, which
stands for Classification and Regression Tree algorithm.
Decision Tree Classification Algorithm
 Decision Tree Terminologies

• Root Node
• Leaf Node
• Splitting
• Branch/Sub Tree
• Pruning
• Parent/Child node
 Working of Decision Tree
• Step-1: Begin the tree with the root node, says S, which contains
the complete dataset.
• Step-2: Find the best attribute in the dataset using Attribute
Selection Measure (ASM).
• Step-3: Divide the S into subsets that contains possible values for
the best attributes.
• Step-4: Generate the decision tree node, which contains the best
attribute.
• Step-5: Recursively make new decision trees using the subsets of
the dataset created in step -3. Continue this process until a stage is
reached where you cannot further classify the nodes and called
the final node as a leaf node.
 Advantages & Disadvantages
• It is simple to understand as it follows the same process which a
human follow while making any decision in real-life.
• It can be very useful for solving decision-related problems.
• It helps to think about all the possible outcomes for a problem.
• There is less requirement of data cleaning compared to other
algorithms.

• The decision tree contains lots of layers, which makes it complex.

• For more class labels, the computational complexity of the
decision tree may increase.
 Random Forest Algorithm
• Random Forest is a classifier that contains a number of
decision trees on various subsets of the given dataset and
takes the average to improve the predictive accuracy of that
dataset.
• Assumptions for Random Forest
– There should be some actual values in the feature variable
of the dataset so that the classifier can predict accurate
results rather than a guessed result.
– The predictions from each tree must have very low
correlations.
 Working of Random Forest
• Step-1: Select random K data points from the training set.
• Step-2: Build the decision trees associated with the selected
data points (Subsets).
• Step-3: Choose the number N for decision trees that you want
to build.
• Step-4: Repeat Step 1 & 2.
• Step-5: For new data points, find the predictions of each
decision tree, and assign the new data points to the category
that wins the majority votes.
 Unsupervised ML
• Unsupervised learning is a type of machine learning in which
models are trained using unlabeled dataset and are allowed to
act on that data without any supervision.
• The goal of unsupervised learning is to find the underlying
structure of dataset, group that data according to similarities,
and represent that dataset in a compressed format.
 Why Unsupervised Learning
• Unsupervised learning is helpful for finding useful insights
from the data.
• Unsupervised learning is much similar as a human learns to
think by their own experiences, which makes it closer to the
real AI.
• Unsupervised learning works on unlabeled and uncategorized
data which make unsupervised learning more important.
• In real-world, we do not always have input data with the
corresponding output so to solve such cases, we need
unsupervised learning.
 Types of
Unsupervised Learning Algorithm

• Clustering: Clustering is a method of grouping the objects into

clusters such that objects with most similarities remains into a
group and has less or no similarities with the objects of
another group.
• Association: An association rule is an unsupervised learning
method which is used for finding the relationships between
variables in the large database. It determines the set of items
that occurs together in the dataset.
 Advantages & Disadvantages
• Unsupervised learning is used for more complex tasks as compared
to supervised learning because, in unsupervised learning, we don't
have labeled input data.
• Unsupervised learning is preferable as it is easy to get unlabeled
data in comparison to labeled data.

• Unsupervised learning is intrinsically more difficult than supervised

learning as it does not have corresponding output.
• The result of the unsupervised learning algorithm might be less
accurate as input data is not labeled, and algorithms do not know
the exact output in advance.
 Clustering in ML
• A way of grouping the data points into different clusters,
consisting of similar data points. The objects with the possible
similarities remain in a group that has less or no similarities
with another group.
• After applying this clustering technique, each cluster or group
is provided with a cluster-ID. ML system can use this id to
simplify the processing of large and complex datasets.
• The clustering technique is commonly used for statistical data
analysis.
Clustering in ML
 Types of Clustering Methods
• Partitioning Clustering
• Density-Based Clustering
• Distribution Model-Based Clustering
• Hierarchical Clustering
• Fuzzy Clustering
 Partitioning Clustering
• It is a type of clustering that divides the data into non-
hierarchical groups. It is also known as the centroid-based
method. The most common example of partitioning clustering
is the K-Means Clustering algorithm.
• In this type, the dataset is divided into a set of k groups, where
K is used to define the number of pre-defined groups. The
cluster center is created in such a way that the distance
between the data points of one cluster is minimum as
compared to another cluster centroid.
Partitioning Clustering
 Density-Based Clustering
• The density-based clustering method connects the highly-
dense areas into clusters, and the arbitrarily shaped
distributions are formed as long as the dense region can be
connected. This algorithm does it by identifying different
clusters in the dataset and connects the areas of high densities
into clusters. The dense areas in data space are divided from
each other by sparser areas.
Density-Based Clustering
 Distribution Model Based Clustering
• In the distribution model-based clustering method, the data is
divided based on the probability of how a dataset belongs to a
particular distribution. The grouping is done by assuming
some distributions commonly Gaussian Distribution.
• The example of this type is the Expectation-Maximization
Clustering algorithm that uses Gaussian Mixture Models
(GMM).
Distribution Model Based Clustering
 Hierarchical Clustering
• Hierarchical clustering can be used as an alternative for the
partitioned clustering as there is no requirement of pre-
specifying the number of clusters to be created.
• In this technique, the dataset is divided into clusters to create
a tree-like structure, which is also called a dendrogram.
• The observations or any number of clusters can be selected by
cutting the tree at the correct level.
Hierarchical Clustering
 Fuzzy Clustering
• Fuzzy clustering is a type of soft method in which a data object
may belong to more than one group or cluster.
• Each dataset has a set of membership coefficients, which
depend on the degree of membership to be in a cluster.
• Fuzzy C-means algorithm is the example of this type of
clustering; it is sometimes also known as the Fuzzy k-means
algorithm.
 Clustering Algorithms
• K-Means algorithm
• Mean-shift algorithm
• DBSCAN Algorithm
• Expectation-Maximization Clustering using GMM
• Agglomerative Hierarchical algorithm
• Affinity Propagation
 Association Rule Learning
• It tries to find some interesting relations or associations
among the variables of dataset. It is based on different rules to
discover the interesting relations between variables in the
database.
• It is employed in Market Basket analysis, Web usage mining,
continuous production, etc.
 Metrics of Association Rule Learning
• Support - Support is the frequency of A or how frequently an
item appears in the dataset. It is defined as the fraction of the
transaction T that contains the itemset X.
• Confidence - Confidence indicates how often the rule has been
found to be true. Or how often the items X and Y occur together
in the dataset when the occurrence of X is already given.

• Lift - It is the strength of any rule, which can be defined as

below formula:
 Types of Association Rule Learning
• Apriori Algorithm - This algorithm uses frequent datasets to
generate association rules. It is designed to work on the
databases that contain transactions.
• Eclat Algorithm - Eclat algorithm stands for Equivalence Class
Transformation. This algorithm uses a depth-first search
technique to find frequent itemsets in a transaction database.
• F-P Growth Algorithm - The F-P growth algorithm stands for
Frequent Pattern, and it is the improved version of the Apriori
Algorithm. It represents the database in the form of a tree
structure that is known as a frequent pattern or tree.
 Applications of Association Rule Learning
• Market Basket Analysis: It is one of the popular examples and
applications of association rule mining. This technique is
commonly used by big retailers to determine the association
between items.
• Medical Diagnosis: With the help of association rules, patients can
be cured easily, as it helps in identifying the probability of illness
for a particular disease.
• Protein Sequence: The association rules help in determining the
synthesis of artificial Proteins.
• It is also used for the Catalog Design and Loss-leader Analysis and
many more other applications.
 Difference between AI and ML

Artificial Intelligence Machine learning

Artificial intelligence is a technology which Machine learning is a subset of AI which allows
enables a machine to simulate human a machine to automatically learn from past
behavior. data without programming explicitly.

The goal of AI is to make a smart computer The goal of ML is to allow machines to learn
system like humans to solve complex from data so that they can give accurate
problems. output.
In AI, we make intelligent systems to perform In ML, we teach machines with data to
any task like a human. perform a particular task and give an accurate
result.
Machine learning and deep learning are the Deep learning is a main subset of machine
two main subsets of AI. learning.
AI has a very wide range of scope. Machine learning has a limited scope.
 Difference between AI and ML

Artificial Intelligence Machine learning

AI is working to create an intelligent system Machine learning is working to create machines
which can perform various complex tasks. that can perform only those specific tasks for
which they are trained.
AI system is concerned about maximizing the Machine learning is mainly concerned about
chances of success. accuracy and patterns.
On the basis of capabilities, AI can be divided Machine learning can also be divided into mainly
into three types, which are, Weak AI, General three types that are Supervised
AI, and Strong AI. learning, Unsupervised learning,
and Reinforcement learning.
It includes learning, reasoning, and self- It includes learning and self-correction when
correction. introduced with new data.
AI completely deals with Structured, semi- Machine learning deals with Structured and
structured, and unstructured data. semi-structured data.
 Deep Learning
• Deep learning is a collection of statistical techniques of
machine learning for learning feature hierarchies that are
actually based on artificial neural networks.
• Deep learning is implemented with the help of Neural
Networks, and the idea behind the motivation of Neural
Network is the biological neurons, which is nothing but a brain
cell.
 Deep Learning Architectures

• Deep Neural Networks - It is a neural network that

incorporates the complexity of a certain level, which means
several numbers of hidden layers are encompassed in
between the input and output layers.
• Deep Belief Networks - A deep belief network is a class of
Deep Neural Network that comprises of multi-layer belief
networks.
• Recurrent Neural Networks - It permits parallel as well as
sequential computation.
 Types of Deep Learning Networks
• Feed Forward Neural Network
• Recurrent Neural Network
• Convolutional Neural Network
• Restricted Boltzmann Machine
• Autoencoders
 Advantages
& Disadvantages of DL
• It lessens the need for feature engineering.
• It eradicates all those costs that are needless.
• It easily identifies difficult defects.
• It results in the best-in-class performance on problems.

• It requires an ample amount of data.

• It is quite expensive to train.
• It does not have strong theoretical groundwork.
 Deep Learning Algorithms
• Deep learning algorithms are dynamically made to run
through several layers of neural networks, which are nothing
but a set of decision-making networks that are pre-trained to
serve a task.
• It becomes quite unfeasible for a traditional machine learning
algorithm to handle great depths and this is where deep
learning.
 Importance of Deep Learning
• Deep learning algorithms play a crucial role in determining the
features and can handle the large number of processes for the
data that might be structured or unstructured.
• Deep learning algorithms are highly progressive algorithms
that learn about the image that we discussed previously by
passing it through each neural network layer.
• There are various cases like multiclass classification where
deep learning can be effective because it involves smaller but
more structured datasets but is not preferred usually.
 Deep Learning Algorithms
• Convolutional Neural Networks (CNNs)
 Deep Learning Algorithms
• Long Short Term Memory Networks (LSTMs)
 Deep Learning Algorithms
• Recurrent Neural Networks (RNNs)
 Deep Learning Algorithms
• Generative Adversarial Networks
 Deep Learning Algorithms
• Radial Basis Function Networks
 Deep Learning Algorithms
• Multilayer Perceptrons (MLPs)
 Deep Learning Algorithms
• Self Organizing Maps (SOMs)
 Deep Learning Algorithms
• Deep Belief Networks (DBNs)
 Deep Learning Algorithms
• Restricted Boltzmann Machines (RBMs)
 Deep Learning Algorithms
• Autoencoders