Unit 5

Planning – planning problem – Simple planning agent

– Blocks world problem – Mean Ends analysis
Learning - Machine learning - Learning concepts,
methods and models Introduction to expert system –
architecture of expert systems.
 Planning
• The task of coming up with a sequence of actions that
will achieve a goal is called planning.
• Planning Environments
1. Classical Planning Environments
• Fully observable, deterministic, finite, static and
discrete.
2. Non classical Planning Environments
• Partially observable, stochastic with different
algorithms and agent designs.
 Planning Problem
• The planning problem is actually the question how to go to next state or the
goal state from the current state. It involves two things 'how' and 'when'.
• The planning problem is defined with:
1. Domain model
• The domain model defines the actions along with the objects.
• It is necessary to specify the operators too that actually describe the action.
• Along with this, information about actions and state constraints while acting
should also be given
2. Initial state
• The initial state is the state where any action is yet to take place (the stage
when the exam schedule is put up!).
3. Goal state (next state)
• The final state or the goal state is the state which the plan is intended to
achieve.
Basic Planning Representation

• Planning representation
involves
1) State
Representation
2) Goal Representation
3) Action
Representation
 Planning Representation
Consider a
delivery robot
with mail and
coffee to
deliver. Assume
a simplified
problem
domain with
four locations
 Planning Representation
State Representation Goal Representation
• A state is represented with • Goal is a partially
a conjunction of positive specified state
literals using • A proposition satisfies a
• Logical Propositions: Poor Λ goal if it contains all the
Unknown
atoms of the goal and
• FOL literals: At(Plane1,OMA)
Λ At(Plane2,JFK) possibly others.
Example: Rich ^ Famous ^
• Closed World Assumption
• What is not stated are
Miserable satisfies the goal
assumed false Rich ^ Famous
Action Representation
 Simple planning agent
• A simple planning agent is very similar to problem-solving agents.
• It constructs plans that achieve its goals, and then executes them.
• The limitations of the problem- solving approach motivates the
design of planning systems.
• To solve a planning problem using a state-space search approach
we would let the:
• initial state = initial situation
• goal-test predicate = goal state description
• successor function computed from the set of operators
• Once a goal is found, solution plan is the sequence of operators in
the path from the start node to the goal node
 Algorithm of a simple planning
agent
1.Generate a goal to achieve
2. Construct a plan to achieve goal from current state
3. Execute plan until finished
4. Begin again with new goal
• The agent first generates a goal to achieve, and then
constructs a plan to achieve it from the current state.
• Once it has a plan, it keeps executing it until the plan is
finished, then begins again with a new goal.
 Languages for Planning Problems
• STRIPS
– Stanford Research Institute Problem Solver
– Historically important
• ADL
– Action Description Languages
• PDDL
– Planning Domain Definition Language
 Stanford Research Institute Problem Solver (STRIPS)

• Makes use of the first

order predicates.
• STRIPS allows function-
free literals.
• Example of a robot: The
example involves a robot,
a cup tea, guest and two
rooms. We want the robot
to get the tea and give it
to the guest.
 Action Description Language (ADL)
• STRIPS language lacks the expressive power.
• It can be extended very well to overcome some of the
limitations and ADL does that.
• The properties of ADL are as follows:
1. It allows negative literals.
2. It makes use of quantified variables along with the
disjunctions and the conjunctions.
3. Conditional post-conditions are allowed.
4. Variables with different types at the same time are
allowed and also equality property is available.
 Planning Domain Description
Language (PDDL)
• It is a standardization of the planning languages.
• It is defined by the researchers as a standard language.
• We can say it is a superset of STRIPS and ADL that
allows features like
1. Objects can have type specifications.
2. It can have negative pre-conditions.
3. The add and delete lists can be conditional.
4. In some cases, it also allows numeric values.
 Block World
• There are ‘N’ number of Blocks resting on table with
specified sequence.
• Goal is to arrange in desired sequence.
• Available moves
– Put block on table
– Put a block on another block top
• State is represented using sequence of blocks in
current pos.
Block World
 Block World

The Robot Arm can perform 4 operations:

1.STACK(X,Y) : Stacking Block X on Block Y
2.UNSTACK(X,Y) : Picking up Block X which is on top of Block Y
3.PICKUP(X) : Picking up Block X which is on top of the table
4.PUTDOWN(X) : Put Block X on the table
 Means - Ends Analysis
• Means-Ends Analysis is problem-solving techniques used in
Artificial intelligence for limiting search in AI programs.
• It is a mixture of Backward and forward search technique.
• The means -ends analysis process centers around finding the
difference between current state and goal state.
• Search strategies are either reason forward of backward.
Steps
1. First, evaluate the difference between Initial State and final
State.
2. Select the various operators which can be applied for each
difference.
3. Apply the operator at each difference, which reduces the
difference between the current state and goal state.
 MEA Algorithm
Step 1: Compare CURRENT to GOAL, if there are no differences between both
then return Success and Exit.
• Step 2: Else, select the most significant difference and reduce it by doing the
following steps until the success or failure occurs.
• A) Select a new operator O which is applicable for the current difference, and
if there is no such operator, then signal failure.
• B) Attempt to apply operator O to CURRENT. Make a description of two
states.
i) O-Start, a state in which O?s preconditions are satisfied.
ii) O-Result, the state that would result if O were applied In O-start.
• C) If
(First-Part <------ MEA (CURRENT, O-START)
And
(LAST-Part <----- MEA (O-Result, GOAL), are successful, then signal
Success and return the result of combining FIRST-PART, O, and LASTPART.
Example of Mean-Ends Analysis:
Example of Mean-Ends Analysis:
 Non-linear planning
• A plan that consists of sub-problems,
which are solved simultaneously is said
non-linear plan.
• In case of the goal stack planning there
are some problems. To achieve any goal,
it could have an impact on the one that
has been achieved.
There is a concept of constraint posting
that comes with non-linear planning. The
constraint posting states that the plan
can be built by
1. Addition of operators or suggesting
operators
2. Ordering them
3. Binding the variables to the operators
 Conditional Planning
• Conditional planning has to work regardless of the outcome of an
action.
• The outcome of actions cannot be determined so the environment is
said to be nondeterministic.
• It’s a way to deal with uncertainty by checking what is actually
happening in the environment at predetermined points in the plan.
(Conditional Steps)
Example:
• Check whether SFO airport (San Francisco International Airport) is oper
Three kind of Environments
• Fully Observable - The agent always knows the current state
• Partially Observable - The agent knows only a certain amount about
the actual state. (much more common in real world)
• Unknown - The agent knows nothing about the current state.
Conditional Planning in Fully Observable
Environments
• Agent used conditional steps to check the state of the
environment to decide what to do next.
• Plan information stores in a library
Ex:

• Syntax: If then plan_A else plan_B

 Reactive Planning
• Reactive planning is planning under uncertainty.
• Makes use of the if-then rules.
• The reactive planners are based on the concept that they should be able to
handle an unknown situation too. So, the reaction rules are used that help
them in doing so.
• A rule selection is based on the priority and a holding condition that
maximises the priority.
• The rule which is at present in execution is said to be active whereas the
ones with holding priority (we can call them possible competitors) are pre-
active others are inactive.
• A B-tree structure is used in reactive planning, where the things are
algorithm selects the rule. Sometimes, no rule can be selected. In such a
case, dependent on the algorithm implementation for rule selection.
 Machine Learning
• Machine learning is building and exploring of methods for
programming computer to make them Learn.
 Machine Learning
• Machine Learning • Scope of ML
Approach
 Machine Learning
Goals of machine learning
• To produce learning algorithms with practical value.
• Development and enhancement of computer algorithms and
models to meet the decision making requirements in practical
scenarios.
• To facilitate in building Intelligent System (IS) that can be used
in solving real time problems.
Challenges of Machine Learning
• Availability of limited learning data and unknown perspectives.
• Acquiring Accurate , compact and precise knowledge building.
• Require large working memory to store data.
• Focusing Too Much on Algorithms and Theories
• Monitoring and maintenance
 Machine Learning
Types of Machine Learning
• Rote learning
Computational structure
used in Machine learning: • Learning from observations
• Learning from agents
1. Functions. • Inductive learning
2. Logic programs and rule • Decision tree learning
sets • Supervised learning
3. Finite state machines • Unsupervised learning
4. Grammars • Semi-supervised learning
5. Problem solving system • Ensemble learning
• Discovery based learning
• Learning by problem solving
 ML - Rote Learning
• Rote learning is a rudimentary form of learning, which
focusses on memorization.
• Here, the values are stored so that these are not re-
computed.
• When computations are more expensive than re-
computing from scratch, this strategy can save a
significant amount of time.
• Selective paging and caching are some of the examples
of rote learning.
 ML - Learning from agents
•An agent is defined the computational entity which is
capable of receiving the environment and can act based
on the situation.
•The agent is composed of learning element performance
element and a curiosity element.
•Based on the coordination between these elements the
outcome of the agent’s behavior is measured.
 ML - Inductive Learning
• Inductive learning involves learning generalized rules from
specific examples (can think of this as the “inverse” of deduction)
• Main task: given a set of examples, each classified as positive or
negative produce a concept description that matches exactly the
positive examples.
• The examples are coded in some representation language, e.g.
they are coded by a finite set of real-valued features.
• The concept description is in a certain language that is
presumably a superset of the language of possible example
encodings.
• A “correct” concept description is one that classifies correctly ALL
possible examples, not just those given in the training set.
 ML - Decision Tree Learning
• The learned function is represented by a decision tree.
• In terms of programming it is also represented as if then
rules.
• Decision tree depicts the simple learning from the
observation, method.
• Based on the observation, at every node, decision is
taken.
 ML - Supervised learning
• Uses labelled dataset.
• Supervised learning is the learning algorithm that is
provided with the set of training data and the algorithm
further induces the classifier to classify the unseen or
new data.
• A line (hyperplane) which is generated after learning
separating two classes class A and class the in two parts
the classifier and the decision-making engine minimize
the false positives and false negative.
 Hyperplane
• As a simple example, for a classification
task with only two features, hyperplane is
a line that linearly separates and classifies
a set of data.
• Intuitively, the further from the
hyperplane our data points lie, the more
confident we are that they have been
correctly classified.
• We therefore want our data points to be
as far away from the hyperplane as
possible, while still being on the correct
side of it.
• So when new testing data are added,
whatever side of the hyperplane it lands
will decide the class that we assign to it.
 ML - Unsupervised Learning
• Use unlabeled dataset.
• Learning is more based on similarities and differences
which are visible.
• These differences and similarities are mathematically
represented in unsupervised learning
• Grouping and categorization of the objects is based on the
understanding of similarities and visualization of their
relations
• Unsupervised learning performs hierarchical clustering.
 ML - Semi supervised Learning
• Semi supervised learning is developed to
cope up with the issues of learning in
supervised or unsupervised mode in
isolation.
• Semi-supervised learning tries to learn
from the labelled as well as unlabeled data.
• Let U be a set of unlabeled data and L be
a set of labelled data.
• As the learning process the learning
approach identifies the unlabeled data U
with reference to a labelled data L and
keeps on labelling the unlabeled data.
• This method is also called as self training
in semi supervised learning.
ML - Ensemble Learning
• Ensemble learning method is the one where
multiple learners or learning algorithms are
trained.
• In ensemble learning method the whole collection
or
ensemble of hypothesis is selected from the
hypothesis space and their predictions are
combined.
• In this approach, the learners or referred to as
base learners.
• The most commonly used ensemble learning
methods are
1.Boosting:
• Boosting can probably be defined as the method
for generating accurate predictions by combining
the rules that
are comparatively inaccurate.
• Boosting works on the weighted training sets. The
weights of
the training example reflects the importance of
 ML
Discovery-based Learning
• In case of discovery learning the learning, takes place without any help of
teacher or supervisor.
• It is called unsupervised.
• It can be said to be inquiry-based learning, where the learner on the basis of
past experience and knowledge tries to discover the outcomes.
Learning by Problem Solving
• In learning from problem solving, various parameters related to solution and
problem are considered.
• These parameters are used and effectively desirability of a particular
outcome or decision is determined.
• It is typically based on the way a problem is solved and the outcomes at
every step.
• Even in pattern-based decision- making, the correct category is determined
based on the previous results.
 ML - Learning Methods

• Artificial neural network-based learning - Back

propagation
• Support vector machines
• Reinforcement learning
• Adaptive learning
• Multi agent based learning
• Distributed learning
• Speedup learning
 Artificial neural network based
learning
• An Artificial Neural Network (ANN) is a computational
model inspired by the human brain’s neural structure.
• It consists of interconnected nodes (neurons) organized
into layers.
• Information flows through these nodes, and the
network adjusts the connection strengths (weights)
during training to learn from data, enabling it to
recognize patterns, make predictions, and solve various
tasks in machine learning and artificial intelligence.
 ANN - Back propagation Algorithm
• Back-propagation is the
essence of neural net
training.
• It is the method of fine-
tuning the weights of a
neural net based on the
error rate obtained in the
previous epoch (i.e.,
iteration).
• Proper tuning of the
weights allows you to
reduce error rates and to
make the model reliable by
increasing its
generalization.
 ANN - Back propagation Algorithm
1.Inputs X, arrive through the preconnected path
2. Input is modeled using real weights W. The weights are
usually randomly selected.
3. Calculate the output for every neuron from the input
layer, to the hidden layers, to the output layer.
4. Calculate the error in the outputs.
ErrorB= Actual Output – Desired Output
5. Travel back from the output layer to the hidden layer
to adjust the weights such that the error is decreased.
6. Keep repeating the process until the desired output is
achieved.
 Support Vector Machine (SVM)
• Support vector machines (SVM)
are supervised learning models
with associated learning
algorithms that analyze data used
for classification and regression
analysis.
• It is a machine learning approach.
• They analyze the large amount of
data to identify patterns from them.
• SVMs are based on the idea of
finding a hyperplane that best
divides a dataset into two classes,
as shown in the image below.
Identify the right hyperplane (Scenario-1):
• Here, we have three hyperplanes (A, B and C). Now,
identify the right hyperplane to classify star and circle.

Hyperplane “B” has excellently performed this job.

 Identify the right hyperplane (Scenario-2):
• Here, we have three hyperplanes (A, B and C) and all
are segregating the classes well. Now, how can we
identify the right hyperplane?
• Here, maximizing the distances between
nearest
data point (either class) and hyperplane
will help us to decide the right hyperplane.
• This distance is called as Margin. Let’s
look at the second fig
• We can see that the margin for
hyperplane C is high as compared to
both A and B. Hence, we name the right
hyperplane as C.
• Another reason for selecting the
hyperplane with higher margin is
robustness. If we select a hyperplane
 Reinforcement Learning
• Learning from interaction with an environment to
achieve some long-term goal that is related to the state
of the environment
• The goal is defined by reward signal, which must be
maximised.
• Agent must be able to partially/fully sense the
environment state and take actions to influence the
environment state
• The state is typically described with a feature vector
 Reinforcement Learning Systems
• Reinforcement learning systems have 4 main elements:
1.Policy-A policy is a mapping from the perceived states of the
environment to actions to be taken when in those states
• A reinforcement learning agent uses a policy to select actions given
the current environment state
• An on-policy agent learns only about the policy that it is executing
• An off-policy agent learns about a policy or policies different from
the one that it is executing.
2. Reward signal
• The reward signal defines the goal
• On each time step, the environment sends a single number called
the reward to the reinforcement learning agent,
• The agent’s objective is to maximize the total reward that it
receives over the long run
• The reward signal is used to alter the policy
 Reinforcement Learning Systems

3.Value function
• The value of a state is the total amount of reward an agent
can expect to accumulate over the future, starting in that state.
• Values must continually be re-estimated from the sequence of
observations that an agent makes over its lifetime.
4. Optional model of the environment
• A model of the environment allows inferences to be made
about how the environment will behave.
• Example: Given a state and an action to be taken while in that
state, the model could predict the next state and the next
reward.
• Models are used for planning, which means deciding on a
course of action by considering possible future situations before
they are experienced
 Adaptive learning

• Adaptive machine learning algorithms are the machine

learning models, where the changes in the environment
help in selecting the algorithm or learning method.
• The adaptive learning solves some of the complex
problems for which a single learning method is not
enough.
• This method is even more appropriate when the
environment is continuously changing and real time
response is expected.
 Multi-Agent System
• A system with multiple autonomous
entities, with distributed
information, computational ability,
and possibly divergent interests.
• A multiagent system is one that
consists of a number of agents,
which interact with one-another
• In the most general case, agents
will be acting on behalf of users
with different goals and motivations
• To successfully interact, they will
require the ability to cooperate,
coordinate, and negotiate with each
other, much as people do
 Distributed learning
• In distributed learning the task of learning is distributed.
• Need for distributed learning - Arises due to large data
sets and time constraints.
• More than one agent in different parts of the data set.
• There will be distributed learning algorithms taken part
in each partition to get the desired outcome, which
would then be combined.
• Multi agent systems can be thought of as a subset of
distributed learning.
 Speedup learning
• Speedup learning typically deals with speeding up problem
solving by effective use of problem solving experience.
• Hence, prior problem solving experience is an input for speedup
learning.
• In this learning,
1. There is no interaction with the environment.
2. New problems cannot be solved.
• So, speedup learning accelerates the process based on the
previous experiences and prior observations.
 Expert System
• An expert system is a computer program that is
designed to solve complex problems and to provide
decision-making ability like a human expert.
• It performs this by extracting knowledge from its
knowledge base using the reasoning and inference rules
according to the user queries.
 Characteristics of Expert System

• High Performance: The expert system provides high

performance for solving any type of complex problem of a
specific domain with high efficiency and accuracy.
• Understandable: It responds in a way that can be easily
understandable by the user. It can take input in human
language and provides the output in the same way.
• Reliable: It is much reliable for generating an efficient
and accurate output.
• Highly responsive: ES provides the result for any
complex query within a very short period of time.
Components of Expert System
 Components of Expert System

1. User Interface
• With the help of a user interface, the expert system interacts with the user, takes
queries as an input in a readable format, and passes it to the inference engine.
• After getting the response from the inference engine, it displays the output to the user.
• In other words, it is an interface that helps a non-expert user to communicate with the
expert system to find a solution.
2. Inference Engine(Rules of Engine)
• The inference engine is known as the brain of the expert system as it is the main
processing unit of the system.
• It applies inference rules to the knowledge base to derive a conclusion or deduce new
information.
• It helps in deriving an error-free solution of queries asked by the user.
3. Knowledge Base
• The knowledgebase is a type of storage that stores knowledge acquired from the
different experts of the particular domain.
• It is considered as big storage of knowledge.
• The more the knowledge base, the more precise will be the Expert System.
 Expert Systems
• DENDRAL: It was an artificial intelligence project that was made as a
chemical analysis expert system. It was used in organic chemistry to
detect unknown organic molecules with the help of their mass spectra
and knowledge base of chemistry.
• MYCIN: It was one of the earliest backward chaining expert systems
that was designed to find the bacteria causing infections like
bacteraemia and meningitis. It was also used for the recommendation
of antibiotics and the diagnosis of blood clotting diseases.
• PXDES: It is an expert system that is used to determine the type and
level of lung cancer. To determine the disease, it takes a picture from
the upper body, which looks like the shadow. This shadow identifies
the type and degree of harm.
• CaDeT: The CaDet expert system is a diagnostic support system that
can detect cancer at early stages.
Architecture of Expert System
 Architecture of Expert System
Explanation Facility
Knowledge acquisition facility
• Allows a user to understand how the
expert system arrived at certain • Provide convenient and efficient
conclusions or results. means of capturing and storing all
• For example: it allows a doctor to find out the components of the knowledge
the logic or rationale of the diagnosis base.
made by a medical expert system.
• Acts as an interface between
Inference Engine
experts and the knowledge base.
• Seeks information and relationships from
the knowledge base and provides User Interface
answers, predictions, and suggestions the • Specialized user interface software
way a human expert would. employed for designing, creating,
Knowledge Base updating, and using expert systems.
• Stores all relevant information, data, • The main purpose of the user
rules, cases, and relationships used by the interface is to make the
expert system.
development and use of an expert
• It uses rules, If-then statements, fuzzy
logic system easier for users and decision
Advantages and Limitations of
Expert Systems