Introduction

to
Artificial Intelligence

Dr. Pulak Sahoo

Associate Professor
Silicon Institute of Technology
 Syllabus - 18CS2T29 : Artificial Intelligence
B.Tech – CSE-A – 6th sem

• Module I
• Artificial Intelligence: Introduction
• Intelligent Agents: Agents, Behavior, Nature of Env., Structure of Agents
• Problem Solving by Searching - Problem-Solving Agents, Example Problems,
Searching for Solutions, Uninformed search strategies, Searching with Partial Info

• Module II
• Informed Search & Exploration: Informed (Heuristic) search strategies,
Heuristic functions, Local Search Algorithms & Optimization Problems
• Constraint Satisfaction Problems: Introduction, Backtracking search for
CSPs, Local Search for CSPs
• Adversarial Search: Games, Optimal Decisions in Games, Alpha-Beta
Pruning;
• Knowledge & Reasoning: Knowledge-Based Agents, TheWumpusWorld.
2
 Syllabus - 18CS2T29 : Artificial Intelligence
B.Tech – CSE – 6th sem

• Module III

• Knowledge and Reasoning: Logic, Propositional Logic, Reasoning Patterns

in Propositional Logic

• First-Order Logic: Syntax and Semantics of First-Order Logic, Using First-

Order Logic, Knowledge Engineering in First-Order Logic

• Inference in First-Order Logic: Propositional vs. First-Order Logic,

Unification and Lifting, Forward Chaining, Backward Chaining, Resolution

• Knowledge Representation: Ontological Engineering , Categories &

Objects, Semantic Nets, Frames

3
 Syllabus - 18CS2T29 : Artificial Intelligence
B.Tech – CSE – 6th sem

• Module IV
• Planning: The Planning Problem, Planning with State-Space Search, Partial-
Order Planning, Planning Graphs
• Uncertain Knowledge & Reasoning: Acting under Uncertainty, Bayes Rule & its
use
• Probabilistic Reasoning: Representing Knowledge in an Uncertain Domain,
Semantics of Bayesian Networks

• Module V
• Learning: Learning from Observations, Forms of Learning, Inductive Learning,
• Learning Decision Trees: Statistical Learning, Instance Based Learning, Neural
Networks
• Reinforcement Learning: Passive & Active Reinforcement Learning
• Expert Systems: Introduction, Architecture, Representations.

4
 Books

1. Stuart Russell & Peter Norvig, Artificial Intelligence: A Modern

Approach, 2nd Edition, Pearson Education.

2. Elaine Rich, Kevin Knight, Shivshankar B Nair, Artificial Intelligence,

McGraw Hill, 3rd Edition.

3. Nills J. Nilson, “Artificial Intelligence: A New Synthesis”, 2nd Edition,

2000, Elsevier India Publications, New Delhi.

4. Michael Negnevitsky, “Artificial Intelligence: A Guide to Intelligent

Systems”, Second Edition, 2005, Pearson Education, Inc. New Delhi.

5. Dan W. Patterson, “Introduction to Artificial Intelligence and Expert

Systems”, 1st Edition, 1996, PHI Learning Pvt. Ltd., New Delhi.

6. E. Charniak and D. McDermott, Introduction to AI, 1st Edition, Addison-

Wesley, 1985
5
 Introduction to AI - Topics

• What is AI

• Four Approaches to AI
▪ Acting Humanly
▪ Thinking Humanly
▪ Acting Rationally
▪ Thinking Rationally

• Foundations of AI

• History of AI

6
What is AI?

7
 What is AI?

• Artificial intelligence is an area of computer science

▪ that emphasizes the creation of intelligent machines
▪ that think, work & react like humans

• Computers with artificial intelligence can do activities like:

▪ Speech recognition
▪ Car responding to master’s voice
▪ Learning
▪ Amazon Alexa
▪ Planning
▪ Decision making by a Robot
▪ Problem solving
▪ Tic-tac-toe, TSP or n-Queen problem 8
 Four Approaches to AI
• Computers with AI can :

9
Four Approaches to AI

10
 Four Approaches to AI

• (1) Think humanly – Cognitive modeling approach

▪ AI application thinks like a human

▪ For this it needs to know “working of human mind”

▪ 3 ways to do this:

(1) By Introspection – catching your “own thoughts”

(2) By Psychological experiments – observing a “person’s action”
(3) By Brain imaging – observing “brains in action”

▪ Cognitive Science - scientific study of the mind & its processes

▪ The study of how we do, what we do (Ex: Car driving)

▪ How to make the computer do it in the same way
11
• (2) Act humanly - Turing Test approach

▪ Turing Test (1950) of a computer with AI (by Alan Turing)

• The computer & a human are interrogated by another human

behind a barrier via written questions

• Computer passes the test if the human cannot tell if the written
response is from a computer or human

12
A Computer with AI needs to have below capabilities :

▪ Natural Language Processing –to communicate in English

▪ Knowledge Representation –to store what it knows or hears

▪ Automated Reasoning – to use the stored info to answer questions

▪ Machine Learning – to adapt to new/changing circumstances

To pass the Total Turing Test, a machine needs :

▪ Computer vision – to perceive objects (sensor)

▪ Robotics – to manipulate objects (actuator)

13
 Introduction
to
Artificial Intelligence

Dr. Pulak Sahoo

Associate Professor
Silicon Institute of Technology
 Introduction to AI - Topics

• What is AI

• Four Approaches to AI
▪ Thinking Humanly
▪ Acting Humanly
▪ Thinking Rationally
▪ Acting Rationally

• Foundations of AI

• History of AI

15
• (3) Think Rationally – The “Laws of thought” approach

▪ Rational thinking – “Right Thinking” with unquestionable reasoning or Logic

▪ A program can solve any solvable problem described in “logical

notation” (Ex: TSP, 8-Puzzle, Chess…)
▪ Ex - Socrates is a man; All men are moral; Socrates is moral

▪ Two obstacles to this approach –

(1) Not easy to represent all problems in logical notations

(2) Big difference between solving a problem “in principle” than “in practice”
16
• (4) Act Rationally – The “Rational Agent” approach

▪ Agent – “One that Acts” – It perceives the env. through sensors & acts on
the env. through actuators (Ex: Robot, Auto-pilot, Vacuum cleaner…)

▪ Rational Agent – The Agent that acts to achieve the “best outcome”
▪ In case of uncertainty, the “best expected outcome”

▪ All the skills needed for Turing test also applies here

17
 Foundations of AI
• Following disciplines have contributed to ideas, viewpoints &
techniques to AI

▪ Philosophy
▪ Made AI conceivable with the idea that “Mind is in someway like a machine”
▪ It operates based on “knowledge”
▪ “Thought” can be used to choose “action”

▪ Mathematics
▪ Provided “tools” to represent & manipulate logically certain or uncertain
statements
▪ Provided algorithms & computations

▪ Economics
▪ Helped in making decision that Maximize the expected outcome
18
 Foundations of AI

▪ Neuroscience
▪ Provided the knowledge about “how brain works”
▪ How brain is similar & different from computers

▪ Psychology
▪ Provided the idea that humans & animals can be considered as “info.
processing machines” 19
 Foundations of AI

▪ Computer Engg.
▪ Provided highly efficient & powerful machines to “implement AI
applications”

▪ Control theory and cybernetics

▪ Helped in designing devices that act optimally based on “feedback from
the env.”

▪ Linguistics
▪ Provided the idea that language for communication can fit into AI
models
20
 History of AI

Gestation of AI (1943-55)

• McCulloch & Pitt propose AI based on knowledge of physiology & neurons in brain
• Russel proposed formal analysis
•Turing proposed theory of computation
• Hebb (1949) proposed Hebbian learning on connection between neurons

Birth of AI (1956)

• John McCarthy (Dartmouth College) – official birth place of AI – 2 month workshop

• Newell & Simon – Created a reasoning program “Logic Theorist”

21
 History of AI

Early enthusiasm, Great Expectations (1952-69)

• Early success but in limited way by Newell & Simons

• General Problem Solver (GPS) – program designed to imitate human problem-solving
• Physical symbol system hypothesis by Newell & Simons
• Lisp AI programming language by McCarthy
• Neural Network – adalines & perceptrons – by Widrow & Rosenblatt

A dose of reality (1966-73)

• Simons prediction of extraordinary success could not be achieved

• Computer chess champion & theorem proving by computer took 40 years than
prediction of 10 years
• A number of difficulties were encountered while implementing complicated AI
applications

22
 History of AI

Knowledge-based systems (1969-79)

• General purpose applications (called weak methods) were unable to handle complex
problems
• Powerful applications using domain-specific knowledge like DENDRAL were needed

AI becomes an Industry (1980-present)

• R.I – 1st commercial Expert System started in 1982 – saved $40m per year for the org
• DEC’s AI group installed 40 E.S.s
• 5th Generation project (10 yrs) of Intelligent AI system – started in Japan using prolog

The return of neural networks (1986-present)

• Back-Propagation – reinvented in 1980’s by 4 different books

• Parallel Distributed processing was introduced
23
 History of AI

AI adopts the scientific method (1987-present)

• Revolution in both content & methodology of AI applications
• Hidden Markov model (HMMs) – Dominates AI applications
• Data-Mining technology – spread into AI area
• Bayesian network – dominates uncertain reasoning AI & ES

Emergence of Intelligent Agents (1995-present)

• Intelligent Agents architecture by Newell, Laird & Rosenbloom
• Internet – Most important Intelligent agent env.
• AI systems have become common in Web-based applications
• Human-level AI (HLAI) – Machines that think

Availability of very large data sets (2001-present)

•In many AI applications “Data” is more important than “Algorithm”
•In recent times, availability of large data sources have improved AI applications
24
• Ex: Images from web, Genometic sequences, words of English
 Course Outline - Module I

• Introduction to AI
• AI Problems & AI techniques
• Solving problems by searching
• Problem Formulation

• Intelligent Agents:
• Structure of Intelligent agents
• Types of Agents
• Agent Environments

• Uninformed Search Techniques:

• DFS, BFS, Uniform cost search, Depth. Limited Search,
• Iterative Deepening, Bidirectional search

• Comparing Different Techniques 25

26
 Chapter 2

Intelligent Agents

Dr. Pulak Sahoo

Associate Professor
Silicon Institute of Technology
 Intelligent Agent
An agent is something that perceives it’s environment
through sensors & acts upon it through actuators
Ex:
 A robot with cameras (sensors) & motors (actuators)
 An AC detecting and regulating room temperature
 An automatic Vacuum Cleaner detecting dirt & cleaning
Intelligent Agent - Diagram
 Some simple terms

Percept
 Agent’s perceptual inputs from env. at any given
instant

Percept sequence
 Complete history of everything that the agent has ever
perceived (Ex: temp or location reading at short time intervals…)
 Agent function & program

Agent’s behavior is mathematically described by

 Agent function

 A function mapping any given percept sequence

to an action
Architecture
Percept Seq. Action Agent (Data)

Program
Agent’s behavior is Practically described by (Algorithm)

 Agent program •Agent Function

•Agent Program
 The real implementation
 Vacuum-World

Perception:
 Clean or Dirty?
 Where it is in? (location A & B)

Actions: Move left, Move right, Suck, Do nothing

 Vacuum-World
 Agent function mapping percept sequences to actions partial
tabulation

---
[A;Clean], [A;Clean], [A;Clean] Right
[A;Clean], [A;Clean], [A;Dirty] Suck
---
Agent Program - implementation
Function Reflex-Vacuum-Agent ([location,state]) return an
action
If state = Dirty then return Suck
else if location = A then return Right // state = Clean
else if location = B then return Left
 Concept of Rationality
Rational agent
 One that does the right thing (see example below)
 Every entry in the agent function table is correct (rational)

What is correct?
 The actions that cause the agent to be most successful

 Need ways to measure success

 Performance measure

Performance measure
 An Objective function determines:
 Criteria of Success for an agent
 Ex: Passed the exam if scored 45%
• Success if 90% clean

An agent, based on its percepts performs action sequence

if result is desirable, it is said to be performing well

No “universal performance measure” exists for all agents

A general rule:
 Design “performance measures” according to
 “What is required to be achieved” in the env.
 Rather than “How the agent should behave”

Ex. in vacuum-cleaner
 We want the “floor clean”
 We don’t restrict “how the agent behaves”
 Rationality

What is rational behavior at any given time

depends on Four factors:

1. The “performance measure” defining the “criterion of

success”
2. The agent’s “prior knowledge” of the env
3. The “actions” that the agent can perform
4. The agent’s “percept sequence” up to now
For each possible “percept sequence”:
 A rational agent should select
 An action expected to “maximize performance” with
whatever “built-in knowledge” the agent has

Ex. An exam
 Maximize marks, based on the given questions & own
knowledge
Example rational agent – Vacuum Cleaner
Performance measure
 Awards “one point” for each “clean square” at each time step,
measured over 10000 time steps

Prior knowledge about the env

 The “geography” of the env
 Only two squares

 The “effect” of the actions

Actions it can perform
 Left, Right, Suck & NoOp
Percept sequences
 “Where” is the agent? (A or B)
 Whether the “location contains dirt”? (Dirty or Clean)
Under this circumstance, the agent is rational
 Omniscience Agent: different from rational
Knows the actual outcome of it’s actions in advance
This is impossible in real world
Ex – While crossing an empty street, man dies of the fallen cargo door
from an aircraft  was the man irrational?
Outcome depends only on current percept not past percept sequence
Rationality maximizes - Expected performance
Perfection maximizes - Actual performance
Rational agents are not omniscient

Learning Agent : different from rational

• A rational agent depends on “current percept” as well as “past
percept sequence”
• This is called learning
• After “experiencing an episode”, the agent should adjust it’s
behaviors to “perform better” next time
 Autonomous Agent
If an agent just relies on the “prior knowledge of it’s designer”
rather than
 its “own percepts” then the agent lacks autonomy
 A rational agent is autonomous - it should learn to compensate
for partial or incorrect prior knowledge

Software Agents

In some cases the env is not real world but “artificial”

Ex: flight simulator, video games, Internet
Those agents working in these env.s are called “Software agents”
(softbots) - All parts of the agent are software
 Task environment (PEAS Description)
• Task env. describes the problem While the rational agent provides the
solution
• In designing an agent, the first step is to describe the task env. fully
 Specifying the task env (PEAS description)-Ex-Automated taxi driver
 Performance (the measure success)
How to judge the performance of a automated driver?
Factors - Reaching correct destination, Minimizing fuel consumption, Trip
time, Non-violations of traffic laws, Maximizing the safety etc.
 Environment (the external world in which the agent operates)
A taxi must deal with Road condition, Traffic lights, other Vehicles,
Pedestrians, Animals, Road works etc. & also interact with the customer
 Actuators (Implements agents action on the env) – the output
Provides control over the steering, gear, brake & communicates with the
customer
 Sensors (Reads or takes inputs from the env)
Detect other vehicles, road situation (camera), GPS to know where the taxi is
etc…
Task environments - Automated taxi driver

Properties of Task environments

Fully observable vs. Partially observable
 If an agent’s sensors give it access to the complete state of the env
at each point in time then the env is fully observable
 Sensors detect all aspects relevant to the choice of action
 Partially observable - An env is partially observable because of
noisy & inaccurate sensors or missing sensor data
• Ex: A local dirt sensor of the cleaner cannot tell squares are clean or not

of images
Task Env Examples- PEAS description

skill
 Properties Task environments

Deterministic vs. Stochastic

 Next state of the env Completely determined by the Current state &
the actions, then the env is Deterministic, otherwise, it is Stochastic

 Auto-Pilot Car - Stochastic because of some unobservable aspects

 noise or unknown

Episodic vs. Sequential

 An episode - “A single pair of perception & action”
 Eposodic - The “quality of an agent’s action” does not depend on
“other episodes” (Every episode is independent of other) – Ex:
Smoke detection
 Episodic env is simpler - The agent does not need to think ahead
 Sequential – “Current action” may affect “all future decisions”
 Ex. Taxi driving & Chess
 Properties Task environments
Static vs. Dynamic
 Dynamic env – is changing over time (Ex: The no. of people in street)
 Static env – don’t change with time (Ex: The destination)
 Semidynamic env - env is not changed over time, but the agent’s
performance score does (Soccer player)

Discrete vs. Continuous

 Discrete env - There are a limited no. of distinct states, clearly defined
percepts & actions (Ex: Vacuum World)
 Continuous env: Taxi driving

Single agent VS. Multiagent

 Single agent - Playing a crossword puzzle
 Two agent (Competitive multi-agent env) - Chess playing
 Cooperative multi-agent env – Automated tutors (english & comm skill
 Properties Task environments
Known vs. Unknown

 This distinction refers not to the env itself but to the agent’s (or
designer’s) state of knowledge about the env

 Known env - The outcomes for all actions are known (Ex: Solitaire)

 Unknown env - the agent will have to learn how make good
decisions (Ex: new video game)
 Structure of agents

Architecture
Agent (Components)

Program
(Algorithm)
•Agent Function
•Agent Program
 Structure of agents
Agent = architecture + program
 Architecture = the components of agent (sensors, actuators..)

 (Agent) Program = the functions (job of AI) that implement the

agent actions based on the current percept

Agent programs
Input for “Agent Program” - only the current percept
Input for “Agent Function” - the entire percept sequence
The agent must remember all of them
Can be implemented as a look up table

P1 P2 P3 Pn Action
 Vacuum-World
 Agent function mapping percept sequences to actions
partial tabulation

---
[A;Clean], [A;Clean], [A;Clean] Right
[A;Clean], [A;Clean], [A;Dirty] Suck
---
 Agent programs
P = the set of possible percepts (Ex: 2 (Clean or Dirty))
T= max length of the percept sequence (Ex- 150)
 The total number of percepts it receives


T t 1 2 150
Size of the look up table
t 1
P (Ex- 2 + 2 + … + 2 )

Ex: Chess
 P =10, T=150

 Look up table contain >10150 entries

Despite of huge size, look up table does “what we want it to do”

The key challenge of AI
 How to write programs that produces rational behaviour from a
small amount of code
 It is ok to have large amount of table entries
 Chapter 2

Intelligent Agents

Dr. Pulak Sahoo

Associate Professor
Silicon Institute of Technology
 Types of Agent programs

Four types

 Simple reflex agents

 Model-based reflex agents

 Goal-based agents

 Utility-based agents
1. Simple reflex agents
1. Simple reflex agents (for fully observable env) Lab

It uses just condition-action rules

 The rules are in form of “if … then …”

 Efficient but have narrow range of applicability

 Because knowledge sometimes “cannot be stated explicitly”

 Works only if the environment is “fully observable”

// From percept get state

// From State get rule
// From rule find action
 A Simple Reflex Agent in Nature

percepts
(size, motion)

RULES:
(1) If small moving object,
then activate SNAP
(2) If large moving object,
then activate AVOID and inhibit SNAP
ELSE (not moving) then NOOP
needed for
completeness Action: SNAP or AVOID or NOOP
 Example Reflex Agent With Internal State:
Wall-Following

start

Actions: left, right, straight, open-door

Rules:
1. If open(left) & open(right) and open(straight) then
choose randomly between right & left
2. If wall(left) & open(right) & open(straight) then straight
3. If wall(right) & open(left) & open(straight) then straight
4. If wall(right) & open(left) & wall(straight) then left
5. If wall(left) & open(right) & wall(straight) then right
6. If wall(left) and door(right) and wall(straight) then open-door
7. If wall(right) and wall(left) and open(straight) then straight
8. (Default) Move randomly
1. Simple reflex agents
2. Model-based Reflex Agents
(for partially observable env)
2. Model-based Reflex Agents (for partially observable env)
For the world that is “partially observable”
 the agent has to keep track of the state of the env.

 Ex- Driving a car & changing lane (traffic situation is imp)

We also require two more types of knowledge

 How the world evolves independent of the agent

 How the agent’s actions affect the world

Example – Self-driven car

IF THEN
Example Mapping Saw an object ahead, turned right, Go straight
Table and it’s now road clear ahead
Saw an object ahead, turned right, Slow down
State – Road Clear, and another object ahead
Object ahead, Dead end See no objects ahead Go straight
See dead end Take U Turn
Model-based Reflex Agents

// From current state & percept get nex

// From State get rule.
// From rule find the action.
// From new state & action update next
3. Goal-based agents
 3. Goal-based agents

The objective is to achieve the goal

Current state of the env.& action are not enough to reach the goal
Need to find out the action sequences required to achieve the goal

Choose Actions that will lead to the goal, based on

 the current state

 the current percept

Conclusion
 Goal-based agents are less efficient but more flexible
 Searching & Planning (Two other sub-fields in AI )
 4. Utility-based agents A2

degree of success
 4. Utility-based agents A2

Reaching the Goals is not enough. Achieving high-quality behavior

(reaching goal in a better approach) is important

A no. of action sequences required to achieve the goal

 If goal => success, then utility => degree of success

State A has higher utility if it is preferred more than other states

Utility is a function that maps a state onto a real number indicating
the degree of success
Utility has below advantages:
 In case of conflicting goals, only some of the goals can be achieved
 Utility describes the appropriate trade-off

When there are several goals & none of them can be achieved with
certainty
 Utility provides a way for the decision-making
 Learning Agents A2

Suggest actions
 Learning Agents A2

After an agent is programmed, it can not work immediately

 It needs to taught
 Teach it by giving it a set of examples
 Test it by using another set of examples
 We then say the agent - A learning agent

Four conceptual components

 Learning element – For Making continuous improvements
 Performance element - Selecting proper actions
 Critic
 Tell how well the agent is doing with respect to a fixed performance std
 Feedback from users
 Problem generator
 Suggest actions that will lead to new & informative experiences
 Dr. Pulak Sahoo
Associate Professor
Silicon Institute of Technology
 Contents
 Introduction

 Problem Solving Agents

 Goal Formulation
 Problem Formulation (Defining a problem)

 Searching
(1) Uninformed (Blind) Search
(2) Informed Search

 Example problems
 Traveler's route finding problem (Travelling Romania)
 Vacuum World
 Toy problems
 8-Puzzle & 8-Queen
 Contents
 Example problems
 Missionaries & Cannibals
 Real-World Problems
 Route Finding Problem

 Uninformed (Blind) Search

 BREADTH-FIRST Search
 DEPTH-FIRST Search
 UNIFORM-COST Search
 DEPTH-LIMITED Search
 Iterative Deepening Depth-First Search
 Bidirectional Search

 Comparison of Uninformed search techs

 Introduction
 Problem Solving Agent
 Problem solving by searching:
 How an agent find a “sequence of actions” to achieve it’s goal

 Goal-based Agents: 2 Types

 Problem-solving agents (use atomic representation)

 Planning agents (use advanced structured representation)

 Introduction
 We will discuss :
 A no. of example problem definitions & their solutions

 A no. of general purpose search algorithms (2 types)

1. Uninformed search algorithm – Have no other info than problem definition

2. Informed search algorithm – Guidance provided for finding solution

 In “Simple task envs” –Solution is a “Fixed sequence of actions”

Water Jog problem
Water Jog problem
 Water Jog problem - Solution

6
Traveler's path finding problem
(Travelling Romania)

•nodes are states

•links are actions
 Problem Solving Agents
 Traveler’s path finding problem
 A traveler is visiting Romania with performance measures like
distance traveled, sight seeing, shopping & nightlife

 Goal Formulation:
 An agent’s task is simplified if it can formulate a goal based on current
situation & performance measure

 Goals limit objectives of the agent (simplifies the problem)

 Goal is a “set of world states” (Cities to be visited by a traveler)

 An agent must act to reach a goal state (Reach Bucharest)

 Problem Formulation:
 Is the process of deciding what type of Actions (driving) & States (Cities)
to consider for a given goal
(Driving from one city to another & reach destination city)
 Problem Solving Agents
 Let us assume that the env is:
 Observable (agent knows current state - current city)
 Discrete (finite set of actions to choose from - connected cities)
 Known (which states are reached by which action - having map of cities)
 Deterministic (Each action has only one outcome)

 Then the solution to any problem is a <fixed sequence of actions>

 The process of looking for a sequence of actions that reaches the

goal is called SEARCH

 When a solution is found, the recommended action sequence is

carried out (called EXECUTION)

 Agent Design - <Formulate -> Search -> Execute>

Traveler's path finding problem
(Travelling Romania)

•nodes are states

•links are actions
 Problem Solving Agents
Ex – Problem – Getting to Bucharest from Arad (in Romania)
Step-1: Defining the problem
 A problem can be defined using 5 components
1. Initial State – starting state (Ex: In(Arad))
2. Actions (available to the agent) – ACTIONS (s) – set of actions
available in state ‘s’ (Ex: Go(Sibiu), Go(Timisoara), Go(Zerind))
3. Transition Model – RESULT (s, a) – Returns the “state reached”
after action ‘a’ in state ‘s’ – (Ex: RESULT(In(Arad), Go(Zerind)) =
In(Zerind))
 State Space - <Initial state, Actions, Transition model> - A directed graph
with nodes are states and links are actions
4. Goal Test – Determines if a given state is a goal state (Ex: Goal is
{In(Bucharest)}
 Problem Solving Agents
5. Path Cost – Assigns a “numeric cost” to each path (Ex: Length in km)
 Path – A sequence of states connected by a sequence of actions

 Step Cost – sum of costs - c(s1, a, s2) - cost of taking action a in state s1 to
reach state s2

 Optimal Solution – Has the lowest path cost among all solutions

 Step-2: Formulating a problem

 Problem – Getting to Bucharest from Arad (in Romania)
 Formulation – In terms of <Initial state, Actions, Transition Model, Goal Test,
Path Cost> - Is an abstraction not the real thing – Irrelevant details are removed

 Ex: In (Arad) is an abstract description excluding many things like Road condition,
Weather, Travel companions, Driving Rules etc..
 Example Problems – 8 - Puzzle
 State – A state desc. specifies the location of 8 tiles & the blank space
 Initial State – Any state can be the initial state
 Actions – Movement of the blank space Left, Right, Up & Down
 Transition Model – For a given state & action, this returns the
resulting state
 Goal Test – Checks if the resulting state matches goal state given below

 Path Cost– Each step costs 1 (Path cost is the ‘no. of steps in the path’)
 8-Queens problem
Placing 8 queens on a chess board, so that none attacks the other

Problem Formulation
 State - Any arrangement of 0 to 8 queens on board

 Initial State – No queen on the board

 Actions - Add a queen to any empty square

 Transition Model –
 Returns the board with
a queen added to the specific square
 Goal Test– 8 queens on board, none attacked
 Missionaries & Cannibals
 3 missionaries & 3 cannibals are on one side of a river
 The boat can hold 1 or 2 people
 Find a way to get everyone to the other side, without ever leaving a
group of missionaries outnumbered by cannibals

 State: (#m, #c, 1/0)

 #m - denotes the no. of missionaries in the first bank
 #c - denotes the no. of cannibals in the first bank
 The last bit - indicates whether the boat is in the first bank

 Initial state: (3, 3, 1)

 Goal state: (0, 0, 0) m – 0 or >= c

 Actions: Boat carries (1, 0) or (0, 1) or (1, 1) or (2, 0) or (0, 2)

Initial state: (3, 3, 1)

Goal state: (0, 0, 0)

 Example Problems
Vacuum World - Problem formulation
 States
 If Agent has n (=2) locations, each location has 2 possibilities (Dirt or No
2
Dirt), Possible world states = n . 2n ( = 2.2 = 8 )

 Initial State
 Any state can be the starting state

 Actions – Each state has 3 actions (Left, Right, Suck)

 Transition Model
 All actions have expected effects except below which have no effect : (1)
Moving Left in leftmost square (2) Moving Right in rightmost square (3)
Sucking in a clean square
 Goal Test – Checks if all squares are clean

 Path Cost – Each step costs 1 (Path cost is the ‘no. of steps in the path’)
 Real-World Problems
Route Finding Problem - Problem formulation
 Used in applications like Driving Directions, Touring, Travelling Salesman
(TSP), ROBOT Navigation etc.. Described by <S, s, O, G>

 Set of States (S)

 Each state is a combination of < location & current time >

 Initial State (s)

 Starting location as specified by user

 Actions (O) – Taking flight from current location (seat, class …)

 Transition Model
 State resulting from taking a flight -> Destination (current location),
Arrival time (current time)
 Goal Test – Final destinations (G – Goal Set) as specified by the user

 Path Cost – Monetary cost, Travel time, Airplane details etc..

 Problem space & search

 To build a system for problem solving we need to do

the following:

1. Define the problem precisely

2. Analyze the problem

3. Represent the knowledge necessary for the problem

4. Choose the best problem solving technique & apply

it to solve the problem
 Definitions – State & State Space

 State - is an instance of the problem

 Ex: current city, state of 8-puzzle board

 State Space:

 Is a graph :

 Whose nodes are set of states

 Whose links are actions that transform one state to

another state
 State space search
 Basic Search Problem:
 Given: [S, s, O, G] where
 S - is the (implicitly specified) set of states
 s - is the Initial state
 O - is the set of state transition operators
 G - is the set of goal states

 To find a sequence of state transitions leading from Initial state s

to a goal state G

 Variants (to be considered later)

 Finding the shortest sequence
 When operators have an associated cost
 Outline of a search algorithm
1. Initialize: Set OPEN = {s} //Set of unexplored state(s)

2. Fail: If OPEN = { }, then terminate with failure // no more states

3. Select: Select a state, n, from OPEN

4. Terminate: If n  G, terminate with success //If reached a Goal state

5. Expand: Generate the successors of n using O and insert them in OPEN

6. Loop: Go To Step 2

 Compare: OPEN is a queue (FIFO) versus a stack (LIFO)

 Uninformed (Blind) Search
Searching with no additional info available about states
other than the problem definition
 Approach – Generate successors from current state & distinguish a
goal state from non-goal state

BREADTH-FIRST Search

 Root node of the search tree is expanded first

 All the successors of Root node are expended next & then their
successors…so on
 All nodes at a given level (depth) are expanded before next level

 BFS is an instance of general Graph-Search Algorithm

 Lowest unexpanded node is chosen for expansion by using FIFO

queue Shallowest node from unexplored node set (frontier) is explored next
Traveler's path finding problem
(Travelling Romania)

•nodes are states

•links are actions
Search Tree – Arad to Bucharest

Pitesti

Bucharest Bucharest
Search Tree – Arad to Bucharest
 BREADTH-FIRST Search
 New nodes go to the back of the queue & Older nodes are
Expended first FIFO queue O1 O2 N1 N2 N3

 Goal test is applied to each node when generated

 Complete – Shallowest goal node is at a finite depth ‘d’, provided

branching factor ‘b’ is finite Check
diagram
in next
 Shallowest (least depth) goal node may not be optimal slide

 BFS search is optimal if path cost is increasing with depth of node

 Searching an “Uniform tree” where each node has same ‘b’ successors
2 3 d d
 Total no. of nodes generated : b (1st level) + b (2nd level) + b + … + b = O (b )

 Time Complexity: O (bd)

d-1 d
 Every generated node remains in memory – O (b ) in explored set & O (b ) in
frontier (unexplored)
d
 Space Complexity: O (b )
Search Tree – Arad to Bucharest

Pitesti

Bucharest Bucharest
BFS on a Graph Start
node

Initialization
// List of nodes to explored next

// List of nodes already explored

// If no nodes to explore

// node is added to explored list

// Generate successors Iter-1

Iter-2
// Reached
goal state
// Add node to frontier
 BFS on a Binary tree

Time & Memory Requirements for BFS

 BREADTH-FIRST Search

 Memory requirements are a bigger problem than Execution time

 For problem with depth 12, Petabytes of memory is required

 Execution time is still a major factor

 350 years for solution for problem with depth 16

 Exponential Complexity search problems cannot be solved by

uninformed methods
 DEPTH-FIRST Search
 DFS expands the deepest node in the frontier of the search tree 1st
 The expanded nodes are then dropped from the frontier
 Search “Backs up” to the next deepest node with unexplored successors
 DFS uses LIFO queue (most recently generated node is expanded)
 Implementations: (1) Graph-Search (2) Recursive function call
Lower level nodes
 DFS is an non-optimal search technique (successors) are
expanded first
 Let ‘b’ be the branching factor & ‘m’ be max depth of any node
m
 Time Complexity: O (b ) ( m can be >> d of BFS)

 DFS have no clear adv. over BFS except for space complexity (for Tree search)
 Once a node is expanded it can be removed from memory
 Space Complexity: O (bm)
 For d=16, DFS needs 156 kb, but BFS - 10 exabytes (7 Trillion times more)
 Depth-First Search

Ex: Recursive
Once Expanded, the
node is removed from function call
memory & Frontier
(Open set)

Implemented
using LIFO queue
DEPTH-FIRST Search
 DFS on a Graph Start
node

DEPTH

Initialization
// List of nodes to explored next
LIFO
// List of nodes already explored

// If no nodes to explore

// node is added to explored list

// Generate successors Iter-1

Iter-2
// Reached
goal state
// Add node to frontier
 DEPTH-LIMITED Search
 Backtracking Search
 Only one successor is generated at a time To save memory
 Uses less memory – O(m) m – max depth
Depth-Limited Search
 Searches with a predetermined depth limit of ‘l’ // Ex: l = 19 for Romania
 Assumed that no successors exist after level-l
 This solves the indefinite path problem faced by DFS & BFS
 DFS is a special case of DLS with l = infinitive
 But introduces ‘incompleteness’ if l < d (shallowest goal is beyond the depth limit)
 DLS is non-optimal if l > d
l
 Time Complexity: O (b )
 Space Complexity: O (bl)
 Ex: Romania has 20 cities with solution length <= 19. So, l = 19 can be chosen
 Iterative Deepening Depth-First Search
 Finds the “Best depth limit”
 By gradually increasing the depth limit 0, 1, 2.. until goal is found
 Goal – Shallowest goal node (d)
Algorithm

 Combines the benefits of DFS & BFS

 Like DFS - Space Complexity: O (bd)

d
 Like BFS - Time Complexity: O (b )
 Preferred uninformed search method when the search space is large &
depth of the solution is not known
Iterative Deepening Search (ID-DFS):
Example
Iterative Deepening Search (ID-DFS):
Iterative Deepening Search (ID-DFS):
Properties
 Bidirectional Search
 Idea behind BS
 Search “from both ends”
 Search “Forward from initial state” & “Backwards from
goal” – till intersect

 Analysis
 Solution depth (in levels from root, i.e., edge depth): d

bi nodes generated at level i

At least this many nodes to test
Total: I bi = 1 + b + b2 + … + bd/2 = (bd/2)

 Space Complexity: (bd/2) & Time Complexity: (bd/2)

 Bidirectional Search

Properties

 Space complexity is a weakness of BS

 But, Time complexity makes BS attractive

Backward search

 8-Puzzle & Romania problems have only one goal state

(backward search is similar to forward search)

 In case of many goal states a new dummy goal state is

created (predecessor to all goal states)
 UNIFORM-COST Search A1

 When all step(path) costs are equal => BFS search is the optimal search
Bcoz, shallowest goal node is found first
 When steps costs are unequal =>
 Uniform-cost search expands the node ‘n’ with the lowest path cost g(n)
instead of the shallowest node

Initialization

Iteration-1

Iter-2
If frontier node has
higher cost, replace it
 UNIFORM-COST Search A1

 Difference with BFS

 Frontier (Open set) –Is a priority queue ordered by ‘g’ (Path cost)
 Goal Test is applied on node when it is selected for expansion (rather
than on generation)
 A Cost Test (path cost calc) is added to check if a better path is found to
a node currently in frontier

310

Path cost = g(n)

278
Final Path: A, C, D, E, G
 Comparison of Uninformed search techs
 See table below:

 Complete => Solution is guaranteed

 b – branching factor
 d – depth of the shallowest solution
 m – max depth of search tree
 l – depth limit
 Tradeoff between space and time

 Iterative deepening
 Perform DFS repeatedly using increasing depth bounds
 Works in O(bd) time and O(bd) space

 Bi-directional search
 Possible only if the operators are reversible
 Works in O(bd/2) time and O(bd/2) space
53
Beyond Classical Search – Hill Climbing &
Simulated Annealing

Dr. Pulak Sahoo

Associate Professor
Silicon Institute of Technology
 Topics

 Beyond Classical Search

 Local Search Algorithms

(1) Hill-climbing search (Local Greedy Search)

 HCS: n-queens problem
 HCS: Algorithm
 HCS: TSP, 8-Puzzle Problems

(2) Simulated Annealing

 HCS: n-queens problem
 HCS: Algorithm

2
 Local search algorithms

 Previous search algorithms use to explore search space

systematically to find the optimal path to the goal

 Ex: TSP, Route finding problems etc.

 But, in many large & complex search problems, the path to

the goal is irrelevant; the goal state itself is the solution

 Ex: 8-puzzle, n-queens, Chess etc.

 Local search algorithms

 In such cases: we can use local search algorithms

 LCA’s keep track of the "current" state

 Then try to improve it by comparing it with it’s
neighbors or successors (if current state is better, it is a peak)

 Comparison is done based on “Objective Function”

(elevation – similar to evaluation function)
 Ex: No. of conflicts for n-Queen, Manhattan distance for 8-Puzzle…

 Advantages –
1. Requires small memory – a constant amount (to store states & values)
2. Can find a reasonable solution in large search space where
systematic algorithms are unsuitable
 Local search algorithms
 Are useful for solving large search space problems fast
 By finding the best state according to “Objective Function”

 Ex: Darwinian Evolution, Fitness Function – Reproductive fitness

State Space
Landscape
 Local search algorithms

 LSAs explore the “State-space landscape” containing 2 things

 Which has both location (state) & elevation (cost defined by
objective function)
 Aim – to find lowest valley (global minimum) or highest peak
(global maximum)
 Complete LSA – always finds a goal
 Optimal LSA - always finds a global maximum/minimum
 Hill-climbing search (steepest ascent)

 HCS algorithm is a loop that continuously moves in the

direction of increasing value (uphill)
 Terminates when reaches a “Peak” where no neighbor has a
higher value
 Like climbing Everest in thick fog

 Data structure: Stores State & Value of objective function (not

search tree)
 Also called “Greedy Local Search”

 Makes rapid progress towards a solution by improving on

previous state
 Ex: n-Queen problem
 HCS: Algorithm

 At each step current node is replaced by it’s best neighbor

(with highest/lowest VALUE)

\\ Current node set to start node

\\ Loop through each neighbor
\\ Select the nbr with best obj .function

\\ If current node’s obj function is higher, return current node

else set nbr node as the current node
 Example: TSP

 Start with any complete tour, perform pair-

wise exchange
C D
A B

C A

D E
B E

 Variants of this approach get within 1% of

optimal very quickly with thousands of cities
 Example: n-queens

 Put n queens on an n × n board with no two queens

attacking each other (i.e not on the same row, column or
diagonal)

 Objective function: Number of conflicts (solution is

global minimum – preferably 0 conflicts(perfect solution))
or Manhattan distance (how many positions away)

 Conflict => the number of pairs of queens that are

attacking each other
 Example: n-queens

 Reached a solution in 2 steps

 HCS: 8-queens problem

 Complete state formation

 Each state has 8 queens, one per column
 Successor states – All possible states by moving a queen to
another square in the same column (8X7=56 successors)

 Heuristic cost function (h) – No. of queen pairs that attack each other

 Global minimum = 0 (perfect solutions)

 Fig-1 in next page shows initial state with h=17

 Value of best successors h=12

 It takes only 5 steps to reach the state in Fig-2 with h=1 (very
nearly a solution)
 – HCS makes rapid progress towards solution
 HCS: 8-queens problem

local minimum with h = 1

Min h

 h = no. of queen pairs attacking each other, either directly or indirectly

 h = 17 for the above state
 Each square contains h values of the successors (h=12 is next best state)
 HCS: 8-queens problem

Fig-1

 h = no. of queen pairs attacking each other, either directly or indirectly

 h = 17 for the above state
 Each square contains h values of the successors (h=12 is next best state)
 HCS: 8-queens problem

Fig-2

• A local minimum with h = 1

• Achieved in 5 steps
 HCS-Problems

 Local maxima: is a peak that is higher than it’s

neighboring states but lower than global maximum
 HCS can get stuck in local maxima

 Ridges– Sequence of local maxima that are difficult to navigate

 Plateaux – Flat local maximum from which no uphill exit exists
An 8-puzzle problem solved by a HCS

Solution
8-puzzle: HCS stuck at local maximum

1 + 1 + 3 + 1 + 2 + 1 + 1 + 1 + 1= 12
 HCS: Algorithm

 At each step current node is replaced by it’s best neighbor

(with highest/lowest VALUE)

\\ Current node set to start node

\\ Loop through each neighbor
\\ Select the nbr with best obj .function

\\ If current node’s obj function is higher, return current node

else set nbr node as the current node
 Simulated Annealing

 HCS algorithm - can “get stuck in local maxima” as it never

makes “downhill” moves (hence remains incomplete)

 Solution
 Random walk – “Randomly selects a successor” from the
neighborhood
 Is complete but inefficient

 Simulated Annealing algorithm combines both HCS &

Random Walk
 Simulated Annealing

 Gradient descent:

 Roll a ping pong ball on bumpy surface it will rest at a local

minimum. Then shake the ball hard enough to dislodge it
from local minimum to global minimum

 Simulated Annealing –

 Shake the ball hard (by increasing the temp in the beginning) &
then gradually reduce the intensity of shaking (reduce the temp)
 Simulated Annealing - Algorithm

 Innermost loop is similar to HCS algorithm

 In stead of picking the nearest best move it picks a random

move
 If it improves the situation, it is accepted

 Otherwise accept the move with a probability < 1

 Probability decreases exponentially with badness of the move

 Probability also decreases as temp goes down

 Used extensively to solve VLSI layout problems

Thanks!!!
Simulated Annealing - Algorithm
An 8-puzzle problem solved by a HCS

Solution
 Dr. Pulak Sahoo
Associate Professor
Silicon Institute of Technology
 Contents
 Introduction
 Heuristic Function

 Informed Search Strategies

 Best First Search

 Greedy Search

 A* Search

 A* Search – Algorithm

 A* Search – Conditions for Optimality

 A* Algorithm for 8-Puzzle problem
 Memory bounded A*: MA*
 Introduction

 Informed Search Strategy uses

 Problem-Specific Knowledge or Domain Knowledge

 along with the Problem definition

 to find Solutions More Efficiently
 than Uninformed search

 Best-First Search is an example of Informed search

 Heuristic Function
 A node is selected for expansion based on the Evaluation Function
F(n) <cost estimate>
 The node with “lowest evaluation” is expanded first

 Choice of “ F ” determines the search strategy

 Heuristic function “H(n)” is a component of F(n)

 H(n) – Estimated cost of cheapest path to goal from node ‘n’

 If (n = goal node) then H(n) = 0

• F(n) = H(n) + G(n)

• H(n) – How far the goal is
• G(n) – No. of nodes travelled from start node to current node
 Heuristic Function
• F(n) = H(n) + G(n)
• H(n) – How far the goal G is
• G(n) – No. of nodes travelled from start node S to current node n

G
S I2 In
H(n)

n
I1
I3

G(n)
 Domain Knowledge & Heuristic Function

 Domain Knowledge is used for :

 For guiding the search & Generating next states

 Heuristics uses “domain specific knowledge” to estimate the

“quality of potential solutions”

 Examples of Heuristic Functions:

 Manhattan distance heuristic for “8 puzzle”

 Minimum Spanning Tree heuristic for “TSP”
 Heuristics are fundamental to “Chess programs”

 A strategy is defined by picking the order of node expansion

 8-Puzzle (Initial & Goal States)
Manhattan distance heuristic

Operations

Step cost - 1

• F(n) = H(n) + G(n)

• H(n) – How far the goal is – No. of misplaced tiles (4)
• G(n) – Number of nodes travelled from start node to current node
 The informed search problem

• Problem definition:

▪ Given: [S, s, O, G, h] where

▪ S is the set of states
▪ s is the start state
▪ O is the set of state transition operators having some cost
▪ G is the set of goal states
▪ h( ) is a heuristic function estimating the distance to a goal

▪ Goal is to find:
▪ A minimum cost “sequence of transitions” to a goal state
 Evaluation Function (f(n))
for different search algorithms
• f(n) – Evaluation function (Estimated cost from start node to
goal node thru current node ‘n’)
• h(n) – Heuristics – Estimated cost of how far the goal node is from
current node ‘n’
• g(n) – Cost so far from start node to node ‘n’

• Greedy Search
• f(n) = h(n) // heuristic (estimated cost) to goal node
• Uniform Cost Search
• f(n) = g(n) // path cost so far
• A* Search
• f(n) = h(n) + g(n) // path cost so far + heuristic to goal node
 Greedy – Best first search (pg-93)

 Greedy search is one of the “Best First Search” Algorithm

 Greedy best-first search algorithm always selects the path

which appears best at that moment
 May not always find the optimal solution

 A greedy search algorithm uses a heuristic for making

locally optimal choices at each stage with the hope of
finding a global optimum
 We will use of Greedy search to solve the route-finding
problem from Arad to Bucharest (in Romania)
Route Finding problem (Travelling Romania)
 Greedy – Best first search (pg-93)

 Use of Greedy search to solve the route-finding problem from

Arad to Bucharest (in Romania)

 Heuristic used = hsld = straight line distance (based on experience)

 The initial state=Arad

 hsld(In(Arad)) = 366

 The first expansion step produces:

 Sibiu, Timisoara & Zerind

 Greedy best-first will select Sibiu

 As the heuristic (hsld ) is minimum
 Greedy – Best first search
 Greedy best-first will select Sibiu

 If Sibiu is expanded we get:

 Arad, Fagaras, Oradea & Rimnicu
Vilcea

 Greedy best-first search will select: Fagaras

 If Fagaras is expanded we get:

 Sibiu & Bucharest
Goal
State
(211)
Route Finding problem (Travelling Romania)
 Greedy – Best first search
 Goal reached !!!
 Search cost is minimal (only expanded the nodes on solution path)

 But not optimal (also incomplete) (also leads to false paths)

 Ex: Path Arad, Sibiu, Rimnicu Vilcea, Pitesti is 32 km shorter

m
 Time & Space complexity for Tree is : O(b )

 m – max depth of the search tree

 With quality heuristics, space & time complexity can be reduced

substantially
 A* Search - Algorithm
 A* Search is Best-known form of best-first search

 Idea: Avoid expanding already expensive paths

 Evaluation function f(n) = g(n) + h(n)

 f(n) - Total cost of path from start node to “goal” through “n”
 g(n) - The cost of reaching the node “n” from “start”
 h(n) - Estimated cost of the cheapest solution through node
“n” to the goal

 In order to get the cheapest solution, try the node with

lowest value of g(n) + h(n)
 A* search is – Complete & Optimal
 A* Search - Algorithm

STEP-1 (Initialize): <Initialize the node lists - OPEN (currently known but not
evaluated) & CLOSED (already evaluated)>

// Initially, only the start node is known & no node is evaluated

 Set OPEN = {s} & CLOSED = { }
 g(s) = 0 // The cost of reaching from start node to start node is zero
 f(s) = h(s) // For first node, the evaluation function is completely based on heuristic

STEP-2 (Fail): <If nothing in OPEN node list, then terminate with failure>
 If OPEN = { }, then terminate with failure

STEP-3 (Select): <If OPEN list has nodes, then select the cheapest cost node>
 Select the minimum cost node n from OPEN //like done in route finding problem
 Save n in CLOSED // Move n into already evaluated list ‘CLOSED’

STEP-4 (Goal test): <if node n is the goal, stop search, return Success & f(n)>
 If n  G, terminate with success, return f(n)
STEP-5 (Expand): <Generate the successors of node n>
 For each successor m of n // For each of the successors do below steps

// New Node - If m is neither in OPEN or CLOSED => neither known nor evaluated before
 If m  [OPEN or CLOSED] // g(m) is not yet calculated
// Calculate evaluation function f(m)
 Set g(m) = g(n) + Cost (n,m) // Find the distance from start to node m
 Set f(m) = g(m) + h(m) // Evaluation function for node m is updated
 Insert m in OPEN // Insert m in OPEN (known nodes list)

// Old Node - If m is either in OPEN or in CLOSED => either known or evaluated before
 If m  [OPEN or CLOSED] // g(m) is already calculated
 Set g(m) = min {g(m), g(n) + Cost(n,m)} // set g(m) to lowest value till now
 Set f(m) = g(m) + h(m) // Calculate evaluation function f(m)

// If f(m) value of this node is less than earlier nodes at same level
 If f(m) has decreased & m  CLOSED
 move it to OPEN

STEP-6 Loop: Go To Step 2.

A* Search – Example-1
A* Search – Example-1
A* Search – Example-2
A* Search – Example-2
 A* Algorithm
for
8-Puzzle problem
 8-Puzzle (Initial & Goal States)

Operations

Step cost - 1

• F(n) = H(n) + G(n)

• H(n) – How far the goal is – No. of misplaced tiles
• G(n) – Number of nodes travelled from start node to current node
8-Puzzle
(Solving
steps)
 Memory bounded A*: MA*
 Working A* within a given memory bound, M

 Whenever |OPEN  CLOSED| approaches M

 Some of the least promising states are removed

 To guarantee that the algorithm terminates

 we need to back up the cost of the most promising leaf of
the subtree being deleted at the root of that subtree

 Many variants of this algorithm have been studied

 Recursive Best-First Search (RBFS) is a linear space version
of this algorithm
 Dr. Pulak Sahoo
Associate Professor
Silicon Institute of Technology
 Contents
 Introduction
 Heuristic Function & Evaluation Function

 Informed Search Strategies

 Best First Search

 Greedy Search

 A* Search

 A* Search – Algorithm
 A* Search – Conditions for Optimality

 A* Algorithm for 8-Puzzle problem

 Memory bounded A*: MA*

 Searching with partial information

 A* Search – Conditions for Optimality
 1st Condition for Optimality – Admissibility

 h (n) should be an admissible heuristic

 Optimistic - it never over-estimates cost to reach the goal
 => f(n) = g(n) + h(n) – never overestimates true cost of solution

 Ex: Straight Line distance to reach Bucharest hSLD

 2nd Condition for Optimality – Consistency (monotonicity)

 Required only for A* Graph Search applications
 h() is monotonic if: h(n) – h(m)  c(n,m), where m is a successor of n
 If monotonicity is satisfied, then A* has already found an optimal path to
the current node
 The f-values of the nodes expanded by A* is non-decreasing
 Searching with partial information
This occurs If the environment is not fully observable or deterministic.
Given below are two reasons for this:
1. Sensorless problems
If the agent has no sensors:
Then the agent cannot know it’s current state
It would have to take repeated action paths to ensure that
the goal state is reached regardless of it’s initial state.

2. Contingency problems
Occurs when the env. is partially observable or when actions are uncertain
After each action the agent needs to verify what effects that action has
caused
Rather than planning for every possible contingency after an action, it is
usually better to start acting and see which contingencies do arise.
37
Constraint Satisfaction
Problems

Dr. Pulak Sahoo

Associate Professor
Silicon Institute of Technology
1
 Contents

 Constraint Satisfaction Problems (CSP)

 Backtracking search for CSPs

 Local search for CSPs

2
 Constraint satisfaction problems (CSPs)

 In Standard search problem:

 A State (Ex: city (Arad) or water jug (3,2)) contains all of the
info necessary :
 to predict the effects of an action

 Transition operator – Go (Zerind)

 to determine if it is a goal state
 In (Bucharest) ?

 Requires a Data structure that supports state info,

heuristic function & goal test
3
 Constraint satisfaction problems (CSPs)

 A Constraint Satisfaction Problem (CSP) consists of

 States - a set of variables – Xi

 Domain for each variable – Di (Xi can take values from Di )

 Goal test - a set of constraints

 The aim is to choose a value for each variable so that the

resulting possible world satisfies the constraints
 Example: Map-Coloring

4
 Example: Map-Coloring

Australia

 Variables WA, NT, Q, NSW, V, SA, T

 Domains Di = {red, green, blue}
 Constraints: Adjacent regions must have different colors
 Ex: WA ≠ NT, or (WA,NT) in {(red,green),(red,blue),(green,red),
(green,blue),(blue,red),(blue,green) 5
 Ex: Map-Coloring - solution

 Solutions are complete & consistent assignments

 Ex: WA = red, NT = green,Q = red, NSW = green, V = red,
SA = blue,T = green
6
 Constraint graph

 Binary CSP: each constraint relates two variables (Ex: map

coloring problem)
 Constraint graph: nodes are variables, arcs are constraints

7
 Varieties of CSPs
 Discrete variables

 Finite domains:
 n variables, domain size d (Ex: map coloring problem)
 O(d n) complete assignments
 Ex: Boolean CSPs (NP-complete)
 Formula "a AND NOT b" is satisfiable because one can find the
values a = TRUE and b = FALSE, which make (a AND NOT b) = TRUE

 Infinite domains:
 Integers, strings variables etc.

 Ex: job scheduling, variables are start/end days for each job

 need a constraint language, e.g., StartJob1 + 5 ≤ StartJob3

8
 Varieties of CSPs
 Continuous variables
 Ex1: start/end times for Hubble Space Telescope observations (time
is variable in a continuous domain)

 Ex2: linear constraints solvable in polynomial time by linear

programming

 Ex3: Finding the intersections of two circles

 Find the Cartesian coordinates of all the intersection points of the two
circles

 Hubble is solar-powered telescope that takes sharp pictures of

objects in the sky such as planets, stars & galaxies. Hubble has
made more than 1 million observations including pictures of the birth
& death of stars, galaxies billions of light years away.
9
 Varieties of constraints

 Unary constraints involve a single variable,

 e.g., SA ≠ green

 Binary constraints involve pairs of variables,

 e.g., SA ≠ WA

 Higher-order constraints involve 3 or more

variables,
 e.g., cryptarithmetic column constraints
10
 Example: Cryptarithmetic

Constraints: Alldiff(F,T,U,W,R,O)
Variables: F T U W R O X1 X2 X3  O + O = R + 10 · X1
 X1 + W + W = U + 10 · X2
Domains: {0,1,2,3,4,5,6,7,8,9}  X2 + T + T = O + 10 · X3
 X3 = F, T ≠ 0, F ≠ 0

11
 Real-world CSPs

 Assignment problems
 Ex: who teaches in what class

 Timetabling problems
 Ex: which class is offered when and where?

 Transportation scheduling

 Factory scheduling

Notice that many real-world problems involve real-valued variables

12
 Standard search formulation (incremental)

Let's start with the straightforward approach, then fix it

States are defined by the values assigned so far

 Initial state: Empty assignment { }

 Successor function: Assign a value to an unassigned

variable that does not conflict with current assignments 
fail if no legal assignments

 Goal test: The current assignment is complete

 Ex: Which teacher teaches in which class

1. Every solution appears at depth n with n variables  use DFS
2. b = (n - l )d at depth l, hence n! · dn leaves
13
 Backtracking search

 Variable assignments are commutative

 Ex: [WA = red then NT = green] same as [NT = green then WA = red]

 Only need to consider assignments to a single variable at each node

 Depth-first search for CSPs with single-variable assignments is

called Backtracking search

 Backtracking search is the basic uninformed algorithm for CSPs

 Can solve n-queens for n ≈ 25

14
Backtracking search - Algorithm

15
Backtracking example

16
Backtracking example

17
Backtracking example

18
Backtracking example

19
 Improving backtracking efficiency

 General-purpose methods can give huge

gains in speed:

 Which variable (province) should be assigned next?

 In what order (color) should its values be tried?

 Can we detect inevitable failure early?

20
 Most constrained variable
 Most constrained variable:
choose the variable with the fewest legal values

3 2
3 2
3 1
3
3 2 1
3 3 3 2
2
3 3 3

 Ex: minimum remaining values (MRV) heuristic

21
 Most constraining variable

 Tie-breaker among most constrained variables

 Most constraining variable:
 Choose the variable with the most constraints on

remaining variables

3 2
3 3 2 1
3
3 2 1
3 3 2
3
2
3 3 3

22
Least constraining value

 Given a variable, choose the Complete

Solution
least constraining value:
the one that rules out the fewest values in the remaining
variables

2 constraints - selected
LCV

3 constraints

MCV

 Combining these heuristics makes 1000 queens feasible

23
 Forward checking
 Idea:
 Keep track of remaining legal values for unassigned variables

 Terminate search when any variable has no legal values

Possible colors in each province

24
 Forward checking
 Idea:
 Keep track of remaining legal values for unassigned variables

 Terminate search when any variable has no legal values



25
 Forward checking
 Idea:
 Keep track of remaining legal values for unassigned variables

 Terminate search when any variable has no legal values



26
 Forward checking
 Idea:
 Keep track of remaining legal values for unassigned variables

 Terminate search when any variable has no legal values



Search terminated with “no legal values” for “SA”

27
 Constraint propagation
 Forward checking propagates information from assigned
to unassigned variables, but doesn't provide early
detection for all failures:


 NT and SA cannot both be blue!

 Constraint propagation repeatedly enforces constraints
locally 28
 Arc consistency

 Simplest form of propagation makes

each arc consistent with another
X Y is consistent
Iff for every value x of X there is some allowed y of Y

B
R

29
 Arc consistency

 Simplest form of propagation makes

each arc consistent
X Y is consistent
Iff for every value x of X there is some allowed y of Y

B
R

30
 Arc consistency

 Simplest form of propagation makes

each arc consistent
X Y is consistent
Iff for every value x of X there is some allowed y of Y

 If X loses a value, neighbors of X need to be rechecked 31

 Arc consistency
 Simplest form of propagation makes
each arc consistent
X Y is consistent
Iff for every value x of X there is some allowed y of Y

 If X loses a value, neighbors of X need to be rechecked

 Arc consistency detects failure earlier than forward checking
 Can be run as a preprocessor or after each assignment 32
 Local search for CSPs
 Hill-climbing, simulated annealing typically work with
"complete" states, i.e., all variables assigned

 To apply to CSPs:

 Allow states with unsatisfied constraints



 Operators reassign variable values

 Variable selection: randomly select any conflicted

variable

 Value selection by min-conflicts heuristic:

 Choose value that violates the fewest constraints
 i.e., Hill-climb with h(n) = total number of violated constraints
33
 Example: 4-Queens
 States: 4 queens in 4 columns (44 = 256 states)
 Actions: move queen in column
 Goal test: no attacks
 Evaluation: h(n) = number of attacks

 Given random initial state, can solve n-queens in almost

constant time for arbitrary n with high probability (e.g., n =
10,000,000)
34
 Summary
 CSPs are a special kind of problem:
 states defined by values of a fixed set of variables

 goal test defined by constraints on variable values

 Backtracking = depth-first search with one variable assigned per

node
 Variable ordering and value selection heuristics help significantly
 Forward checking prevents assignments that guarantee later
failure

 Constraint propagation (e.g., arc consistency) does additional

work to constrain values and detect inconsistencies

 Iterative min-conflicts is usually effective in practice

35
The End

36
 Dr. Pulak Sahoo
Associate Professor
Silicon Institute of Technology
 Adversarial Search

 Introduction

 Game Playing

 Min-Max Search

 Alpha Beta Pruning

 Adversarial Search problems (Game Playing)
 Multi-Agent env – Need to consider action of other agents

 Competitive env – The agent’s goals are in conflict (adversarial)

 Ex: Chess or Tic-Tac-Toe

 Deterministic, turn-taking, two-player, zero-sum games

 Zero sum game - Total payoff to all players is same (0+1, 1+0, ½ + ½)

 One agents action has equal & opposite effect on other

(One wins, other loses)
Games vs. Search problems
 More than one agent
100
 Harder to solve (search tree of chess has 35 nodes)

 Deal with "Unpredictable" opponent  Determine a

counter-move for every possible opponent move

 Time limits  If can not find the exact goal within

time, must approximate

 Ex: Chess & Tic-Tac-Toe

 Formally defining a game

 S0 : Initial State – How the game is setup at the start

 PLAYER (s)– Defines which player has the move in a state

 ACTION (s)– Returns the set of legal moves in a state

 RESULT (s,a)– The transition model that define result of a move

 TERMINAL_TEST (s) – True if game is over & false otherwise

 UTILITY (s,p) – (objective / payoff function) – Final numeric

value of a game that ends in terminal state ‘s’ for player ‘p’
 Ex: In Chess, the outcomes are win(1), loss(0) or draw(1/2)
 GAME TREE
 Tree with nodes as game states & edges are moves (could
be very large)

 Combination of Initial state, ACTION & RESULT functions

 See diagram on next page

MAX places ‘X’ & MIN places ‘O’ alternatively
Terminal state – One player has 3 ‘X’ or 3 ‘O’ in a row
Tic-Tac-Toe has < 9! (362880) terminal nodes

MAX has 9
possible moves

MIN has 8
possible moves
 Optimal Decisions in games
 MAX & MIN are playing, MAX responds to all possible moves from MIN

 Optimal Strategy – Respond to “best possible move” of opponent

 Idea: choose the move with highest minimax value (given by utility/
payoff function) = best achievable move against best play by opponent

 Ex: 2-Ply Game Tree (See diagram on next page)

MINIMAX (s) = UTILITY (s) <<Game finished>> if TERMINAL-TEST (s)

Ex: In Chess, the Utility is win(1), loss(0) or draw(1/2)
maxa Actions(s) MINIMAX (RESULT (s,a)) if PLAYER (s) = MAX
mina Actions(s) MINIMAX (RESULT (s,a)) if PLAYER (s) = MIN
Three possible moves by MAX (at root) – A1, A2, A3

Possible replies by MIN to A1 – A11, A12, A13 and so on..

This one-move game ends after ½ moves (called Ply) each by MAX & MIN

Optimal move is determined by Utility (Minimax value) of each node (A1

– 3, A2 – 2, A3 – 2) – MAX chooses maximum value i.e. 3, MIN chooses
minimum value i.e. 3
3 Players are there – A, B, C with utilities (u1, u2, u3) – Not 0 sum game
(u1 – utility of A, u2 – utility of B & u3 – utility of C)
We move upwards by selecting “Max” utilities at each level
 Computes the minimax decision from the current state

 Uses Recursive Computation of minimax values of each successor

state

 Recursion proceeds down the leaves of the tree & minimax values
are backed up (in the example it returns value 3)

 Performs a complete DFS of the game tree

m
 Time Complexity – O (b )
 Space Complexly – O (bm) or O(m) (all actions generated at once
or one at a time)

 Note: m – max depth of the tree, b – no. of legal moves at each point
 For chess, b ≈ 35, m ≈100 for "reasonable" games  exact solution
completely infeasible
 // Action – a, State – s, v - utility

// v = max utility value of the state

// Returns action corresponding
to the best utility value

// if terminal node return it’s utility

// v initialized to -infinity

// Visit successor level (apply action on states)

& find the max utility value (v) & return it

// if terminal node return it’s utility

// v initialized to -infinity

// Visit successor level (apply action on states)

& find the min utility value (v) & return it
 Complete? Yes (if tree is finite)

 Optimal? Yes (against an optimal opponent)

 Problem with minimax – no. of game states to be explored is
exponential in the depth of the game tree

 This exponent can be cut down (in half) by pruning to eliminate

large parts of the tree (called Alpha-Beta Pruning)

 Consider the same 2-ply game

 We can identify the minimax decision without evaluating 2 leaf nodes

Z = min (2,x,y)
<= 2

No need to
evaluate. Can
 Simplified minimax formula is given below be pruned

 Consider the 2 unevaluated successors of node ‘A2’ have values ‘x’ & ‘y’

 The value of the root node is given below (root value does not depend on
pruned nodes x & y)

MINIMAX (root) = max ( min (3, 12, 8), min (2, x, y), min (14, 5, 2) )
= max ( 3, min (2, x, y), 2)
= max ( 3, z, 2) where z = min (2, x, y) <= 2
= 3
 Alpha-Beta Pruning can be applied to tree of any depth

 Often it is possible to prune entire sub tree than just leaves

 Alpha – the value of the best (highest value) choice we have

found so far at any choice point along the path for MAX

 Beta – the value of the best (lowest value) choice we have found
so far at any choice point along the path for MIN

 Alpha-Beta search updates values of alpha & beta as it goes along

 It prunes the remaining branches at a node

 When value of current node is worse than alpha (for MAX) & beta (for MIN)
 No need to
evaluate. Can
be pruned
 Need to
evaluate. Can
not be pruned
 Pruning does not affect final result

 Good move ordering improves effectiveness of pruning

 With "perfect ordering" time complexity = O(bm/2)

 Checkers: Chinook ended 40-year-reign of human world
champion Marion Tinsley in 1994. Used a pre-computed
endgame database defining perfect play for all positions
involving 8 or fewer pieces on the board, a total of 444
billion positions

 Chess: Deep Blue defeated human world champion Garry

Kasparov in a six-game match in 1997. Deep Blue searches
200 million positions per second, uses very sophisticated
evaluation, and undisclosed methods for extending some
lines of search up to 40 ply

 Othello: human champions refuse to compete against

computers, who are too good
27
Knowledge-based Agents
Logic

Dr. Pulak Sahoo

Associate Professor
Silicon Institute of Technology
 Knowledge-Based Agents& Logic
- Topics

• Introduction

• Knowledge-Based Agents

• WUMPUS WORLD Environment

• Propositional Logic

• First Order Predicate Logic

• Forward and Backward Chaining

2
 Introduction
• Human beings know things

• This helps them to do things

intelligently based on reasoning

• Process of reasoning operates

based on internal representation (storage) of knowledge

• Same approach is followed by Knowledge-based Agents

• Logic is a class of representation that supports Knowledge-based

Agents

• They can adopt to changes in env. by updating knowledge 3

 Knowledge-based Agents (Design)
• Central component – knowledge base (KB)
• Knowledge Base – Set of sentences expressed in Knowledge
Representation Language

• Operations
 TELL – Add new sentence to KB
 ASK – Query what is known

• An KB Agent program takes a percept as input & returns an action

• The KB initially contains some “background knowledge”

• The Agent program does 3 things

 TELLs the KB what it perceives
 ASKs the KB what action should be performed
 TELLs the KB what action was chosen & executes the action 4
 KB Agents Program
Agent KB-Agent (Percept) Returns an action

Persistent: KB – a knowledge base // Maintain a KB

t (time) = 0 //time is initialized to 0

// Input percept sequence & time to KB

TELL ( KB, Make-Percept-Sentence ( percept, t ) )
// Find suitable action to be taken from KB
action = ASK (KB, Make-Action-Query ( t ) )
// Update KB with action corresponding to the percept seq at time t
TELL (KB, Make-Action-Sentence ( percept, t)

t = t + 1 // Increment time

return action // Return action

5
 KB Agents Program

• Two System building approaches employed by a designer to

an empty KB

1. Declarative approach
 TELL sentences one-by-one until the agent knows how to
operate

2. Procedural approach
 Encodes desired behavior directly into program code

• A successful agent must combine both approaches

6
The Wumpus World Environment

 Wumpus World
• A cave containing rooms connected by passageways

• The Wumpus (beast) is hidden in the cave – Eats anyone

entering the room

• The Agent has only one arrow to shoot

• Some rooms has bottom-less pits to trap anyone

entering

• Only Reward - Possibility of finding a gold heap

7
8
 Task Environment Description - PEAS

 Performance measure
• +1000 – Coming out of cave with gold
• -1000 – Falling into Pit or Eaten by Wumpus
• -1 – For each action
• -10 – For using the arrow
• End of game – Agent dies or climbs out of cave
 Env
• A 4X4 grid of rooms
• Agent starts in [1,1]
• Location of Gold & Wumpus chosen randomly (except starting
one )
• Each square (except starting one) can be a pit with probability9 0.2
 Task Environment Description - PEAS
 Actuators
• Agent Moves – Forward, TurnLeft, TurnRight
• Death – Falling into Pit or Eaten by Wumpus (Safe to enter room with dead
wumpus)
• Forward move against wall– Not allowed
• Actions– Grab (pickup gold), Shoot (one Arrow), Climb (out of cave from
[1,1])
• End of game – Agent dies or climbs out of cave

 Sensors
• Stench: Perceived in squares containing & adjacent to wumpus
• Breeze: Perceived in squares adjacent to a pit
• Glitter: Perceived in squares containing Gold
• Bump: Perceived when walking into a Wall
10
• Kill Wumpus: Perceived Scream anywhere in the cave
 Wumpus World - Steps

• Challenges for Agent - Initial ignorance of env configuration

(require logical reasoning)

• Good possibility of agent getting out with gold

• Sometimes, agent will have to choose between empty-hand
return or death
• 21% times gold is in a pit or surrounded by pits

• Knowledge Representation Language (KRL) used – writing

symbols in the grids
• Initial KB – contains rules of the game
11
• Start grid [1,1] & it is safe – denoted by A (agent) & OK
• 1st percept is [None, None, None, None] => neibouring grids [1,2] & [2,1] are safe
(OK)
• If Agent moves to [2,1] => Perceives breeze (B)=>Pit(s) present in [2,2] or [3,1] or
both (P?)
• Only safe square is [1,2]. Hence agent should move back to [1,1] & then to [1,2]

12
• In [1,2] perceived Stench & No breeze – denoted by S - [Stench, None, None,
None, None]
• After 5th move perceived [Stench, Breeze, Glitter, None, None] => Found Gold

13
 Logic & Deduction

 A formal system for describing states of affairs, consisting of:

 The syntax of the language describing how to make sentences

 The semantics of the language describing the relation between
the sentences & the states of affairs

 A proof theory – a set of rules for logically deducing

entailments of a set of sentences

 Improper definition of logic or incorrect proof theory can result in

absurd reasoning

14
 Types of Logics

Language What exists Belief of agent

Propositional Logic Facts True/False/Unknown

First-Order Logic Facts, Objects, Relations True/False/Unknown

Temporal Logic Facts, Objects, Relations, True/False/Unknown

Times
Probability Theory Facts Degree of belief 0..1
Fuzzy Logic Degree of truth Degree of belief 0..1

15
 1. Propositional Logic
 Simple but Powerful
 Contains a set of atomic propositions AP
 It contains Syntax, Semantics & Entailment -
 Syntax - Defines allowable sentences
 Sentences – 2 types
 Atomic sentence – Single symbol that can be True | False | AP
Ex- P, Q, R, W1,3 (means Wumpus in [1,3]), North…
 Complex sentence - ( Sentence ) | [Sentence]
 | : Logical Connective like
 ( (negation),  (and) , (or),  (if & only if), (implies))
Ex: W1,3   W2,2
 Semantics – Defines the rules for determining the truth of the
statement in a given model
 Entailment – Relation between 2 successive sentences
16
 Inference Rules

• Inference rule - transformation rule - is a logical form that takes

premises, analyzes their syntax, and returns a conclusion

1. Modus Ponens or Implication Elimination:

• Premise-1 : "If then "

• Premise-2 : , Given
• Conclusion : Conclusion

=> if the premises are true, then so is the conclusion.

17
 Inference Rules

2. Unit Resolution:

• If is True & is True, Then is True

3. Resolution:

or

• The 2 premises are said to be resolved and the variable is said

to be resolved away.

…. and several other rules

18
• A sentence/premise may have:

 Validity (always true)

 Satisfiability (sometimes true)

 No Satisfiability (always false)

19
 Propositional Logic
 Semantics (Defines the rules for determining the truth of the statement)
 Atomic sentence – 2 rules (True & False)
Ex: B1,1  (P1,2  P2,1)
 Complex sentence – 5 rules
A square is breezy if the
  P is true iff P is false neighboring squares have pit
and vice versa
 P  Q is true iff both P & Q are true
 P  Q is true iff either P or Q is true
 P  Q is true unless P is true & Q is false
 P  Q is true iff P & Q are both true or both false
 Truth Tables - Specify truth value of complex sentence for each possible
value

20
 Propositional Logic – Example – Wumpus world

 A Simple Knowledge Base

 Example KB for Wumpus world
 Px,y – True, if pit is there in [x,y]
 Wx,y - True, if wumpus is there in [x,y]
 Bx,y - True, if breeze is there in [x,y]
 Sx,y - True, if stench is there in [x,y]
 Sentences (Enumerate)

 R1 : P1,1 // There is no pit in [1,1]

 R2 : B1,1  (P1,2  P1,2) // A square is breezy if the neighboring
squares have pit & vice versa
21
 Propositional Logic

 A Simple Inference Procedure

 Models are assignments of True or False to every symbol
 Check the sentences are true in every model
 Example Inference Procedure (Wumpus world)
 Seven symbols – B1,1,B2,2, P1,1, P1,2, P2,1, P2,2, P3,1
7
 2 = 128 possible models
 In three cases, KB is true
n
 Time Complexity = O(2 ) n = no. of symbols in KB
 Space Complexity = O(n)
22
 Examples - Automated Reasoning
Example-1: Deducing the position of the wumpus based on
information like Stench, Breeze etc..

Example-2:
• If the unicorn is mythical, then it is immortal, (premises)
• But, if it is not mythical, then it is a mortal mammal.
• If the unicorn is either immortal (P1 ) or a mammal (P2 ) , then it is
horned (Q) .
• The unicorn is magical if it is horned

Q: Can we prove that the unicorn is mythical? Magical? Horned?

• In general, the inference problem is NP-complete (Cook’s Theorem)
• If we restrict ourselves to Horn sentences, then repeated use of Modus
Ponens gives us a polytime procedure. Horn sentences are of the form:

P1  P2  …  Pn  Q 23
 Conjunctive Normal Form (CNF)

 Conjunctive normal form (CNF) is an approach to Boolean

logic that expresses formulas as :

 Conjunctions of clauses with an AND or OR

 Each clause connected by a conjunction, (AND) must be either

a literal or contain a disjunction (OR) operator.

 CNF is useful for automated theorem proving

24
Conjunctive Normal Form (CNF)

// Replace all

// De Morgan’s theorem

// in normal form

25
Propositional Logic – Example-1
3rd inference rule (resolution)

// B is resolved away

// Q is resolved away

26
 Propositional Logic – Example-2
Problem: Solution: H: It is “Humid” (sentences)
• If it is “Hot”, Then it is “Humid” R: it will “Rain”
• If it is “Humid”, Then it will “Rain” O: It is “Hot”

• If it is “Hot”, Then it is “Humid”: O => H

Q: If it is “Hot”, Show that it will “Rain”
• If it is “Humid”, Then it will “Rain : H => R
• It is “Hot” : O
• Add “Negation of Goal”:

CNF: Step-1
(eliminate =>)

Apply Resolution
Inference rule on
H,O&R

Resolution rule

27
Propositional Logic – Example-3

28
 2. First-order Predicate Logic (FOPL)

 FOPL is a symbolized reasoning system in which each

sentence is broken down into (1) a subject (a variable) & (2) a
predicate (a function)

 The predicate modifies or defines the properties of the subject

 A predicate can only refer to a single subject

• A sentence in FOPL is written in the form

 Px or P(x), where P is the predicate & x is the subject (a variable)

• Complete sentences are logically combined & manipulated as

done in Boolean algebra
29
 First-order Predicate Logic (FOPL)
• Sentence  Atomic Sentence (P(x) or x = y)
| Sentence Connective Sentence ( |  |  |  )
| Quantifier Variable ( | ), … Sentence
|  Sentence

• Atomic Sentence  Predicate(Term, …)| Term = Term

• Term  Function(Term, …) | Constant | Variable
• Connective   |  |  | 
• Quantifier   |
• Constant  A | 5 | Kolkata | …
• Variable  a | x | s | …
• Predicate  Before | HasColor | …
Function  Is-Prof () | Is_Person () | Is_Dean ()| …
30
•
 First-order Predicate Logic (FOPL)

 Consider a subject as a variable represented by x

 Let A be a predicate "is an apple"
 F be a predicate "is a fruit"
 S be a predicate "is sour“
 M be a predicate "is mushy"

 Then we can say -

31
 Example-1

1st – If x is a apple => All x are fruits,

2nd – If x is a fruit => some x are apple
3rd – Some apples are sour
4th – Some apples are mushy

32
Examples-2 Formal definition – using FOPL

33
 FOPL – Example-3
Problem:
• Show the validity of the following sentence
• All men are mortal. John is a man. Therefore John is Mortal.

34
 FOPL – Example-3
Problem:
• Show the validity of the following sentence
• All men are mortal. John is a man. Therefore John is Mortal.

Replace =>

Negation of Goal

35
 FOPL – Example-4
Problem:
• Given the following predicate show how resolution process can be applied

Negation of Goal

36
 Forward Chaining
 Forward chaining is a data driven method of
 Deriving a particular goal
 from a given knowledge base & a set of inference rules
 The application of inference rules results in new knowledge
 which is then added to the knowledge base
 Used to answer the question “What can happen next”
 The inference engine applies chain of conditions, facts & rules
 to arrive at a solution (decision or goal)

37
 Forward Chaining
• The system starts from a set of facts & a set of rules
 Tries to find ways of using them to deduce a conclusion (goal)

• Called data-driven reasoning because the reasoning starts from a

set of data and ends up at the goal

• 1st Step – Take the facts from the fact database & see if any
combination of these matches any of the components of rules in the
rule database

• 2nd Step – In case of a match, the rule is triggered (fired)

• 3rd Step – Then it’s conclusion is added to the facts database

• 4th Step - If the conclusion is an action, then the system causes

38
that action to take place
Forward Chaining – Example - Elevator

// Fact-1

// Fact-2

// Conclusion added to KB
39
Forward Chaining - Example - Elevator

// Fact-4 added to KB

40
 Forward Chaining - Example - Elevator

 The system examines the rules & finds that Facts 1 & 2 match the
components of Rule 3

 Rule 3 fired & its conclusion “Go to 3rd floor” is added to the facts
database

 This results in the elevator heading to the 3rd floor

 Note that Fact 3 (today is Tuesday) was ignored because it did not
match the components of any rules

 Assuming the elevator is going to the 3rd floor & has reached the
2nd floor, when the button is pressed on the 1st floor

The fact “Button pressed on first floor” Is now added to the

database, which results in Rule 4 firing (remember to go to first floor)
41
Forward Chaining - Example – Knowledge Base

• Question

 The law says that it is a crime for an American to sell

weapons to hostile nations.

 The country Nono, an enemy America, has some

missiles, and all of its missiles were sold to it by Col.
West, who is an American.

 Prove that Col. West is a criminal.

42
 Sentences in FOPL
• It is a crime for an American to sell weapons to hostile
nations:
American(x) Weapon(y) Sells(x,y,z) Hostile(z)  Criminal(x)
< x – person, y – weapon, z – country >
• Nono…has some missiles
x Owns(Nono, x)  Missiles(x) // some of the weapons owned by Nono are missiles
Owns(Nono, M1) and Missle(M1)
< x – weapon, y – missile >
• All of its missiles were sold to it by Col. West
x Missle(x)  Owns(Nono, x)  Sells( West, x, Nono)
< x – missile >
• Missiles are weapons
Missle(x)  Weapon(x) 43
• An enemy of America counts as “hostile”
Enemy( x, America )  Hostile(x)

• Col. West who is an American

American( Col. West )

• The country Nono, an enemy of America

Enemy(Nono, America)

44
45
And

46
Conclusion

47
 Backward Chaining
 Backward chaining is a goal driven method of
 Deriving a particular goal from a given knowledge base & a set
of inference rules
 Inference rules are applied by matching the goal to the results of
the relations stored in the knowledge base
 Used to answer the question “Why this happened”
 Based on what has already happened, the inference engine tries to
find out which conditions (causes or reasons) could have happened
for this result

48
 Backward Chaining
• The system starts from a conclusion (hypothesis to prove or goal)
 Tries to show how the conclusion has been reached from the
rules & facts in the database
• Reasoning in this way is called as goal-driven reasoning
• Steps – Start with the goal state & see what actions could lead to
it of the components of rules in the rule database

• Ex:
 If the goal state is “blocks arranged on a table”
 Then one possible action is to “place a block on the table”
 This action may not be possible from the start state
 Further actions need to be added before this action
 In this way, a plan is formulated starting from the goal & working back
toward the start state 49
 Backward Chaining

• Backward chaining ensures that each action that is taken is one

that will definitely lead to the goal

• In many cases, Backward Chaining will make the planning

process far more efficient compared to Forward Chaining

50
 Example - WEATHER FORECAST SYSTEM

// Premise-1 – R1 (conclusion of rule-2)

3 // Rule-1

1 // Premise-1 – R2 (This is ‘Known’)

// Premise-2 – R2 (This is ‘Known’)
// Fire R-2

// Premise-1 – R3 (This is ‘Known’)

2 // Conclusion of R-3

 Suppose we have been given the following facts,

a) Sun is behind the clouds
b) Air is very heavy & cool
 Problem: Use Backward chaining to conclude there are chances of 51
rain
Example - WEATHER FORECAST SYSTEM

52
Example - WEATHER FORECAST SYSTEM

53
Example - WEATHER FORECAST SYSTEM

54
Backward Chaining Example

55
56
57
58
59
60
61
Backward Chaining Algorithm

62
63
Planning

Dr. Pulak Sahoo

Associate Professor
Silicon Institute of Technology
 Planning - Topics

• Planning: Introduction to Planning

• Planning vs Problem Solving
 Example – The Blocks World
 Example: Transportation (Air Cargo) problem
 Example: Spare Tire problem

• Representation of Action

• Partial Order Planning

 Example: Shoes & Socks Problem

• Hierarchical Planning
• Example: House Construction
2
 Introduction
• Planning: the task of finding a sequence of actions that will
achieve a given goal when executed from a given world state

• Given are:

• The computed plan is:

• Classical planning env.s are fully observable, deterministic,

finite, static & discrete 3
 Introduction

• Planning agents :
 Construct plans to achieve goals, then executes them

• Planning Agent specifies:

 States & Goal – as logical sentences
 Actions – as preconditions/outcomes
 Plan – as sequence of actions

4
 Introduction

• Key ideas:

 Use descriptions in a formal language – FOPL

 Divide-and-conquer algorithms

 States/goals represented by sets of sentences

 Actions represented by logical descriptions of preconditions & effects

 Planner can add actions to plan whenever needed

 Connection between order of planning & order of execution not

essential 5
Example – The Blocks World

Robot Robot
hand
A hand
C B
A B C
START GOAL
 Example – The Blocks World

• A planning system has rules with precondition deletion &

addition list

Robot Robot
hand
A hand
C B
A B C
START GOAL

Sequence of actions : 5. Pickup B

1. Grab C 6. Stack B on C
2. Pickup C 7. Grab A
3. Place on table C 8. Pickup A
4. Grab B 9. Stack A on B
 Example – The Blocks World

• Fundamental Problem :
 The “frame problem” is concerned with the question

“What piece of knowledge is relevant to the situation”

• Fundamental Assumption : “Closed world assumption”

 If something is not asserted (specified) in the knowledge base,
it is assumed to be false

 Called “Negation by failure”

 Example – The Blocks World

•A planning system – Has rules with precondition,

deletion list & addition list

Robot Robot
hand
A hand
C B
A B C
START GOAL

on(B, table)
on(A, table) on(C, table)
on(C, A) on(B, C)
hand empty on(A, B)
clear(C) hand empty
clear(B) clear(A)
 Rules
R1 : pickup(x)
 Precondition & Deletion List: hand empty, on(x,table), clear(x)
 Add List: holding(x)

R2 : putdown(x)
 Precondition & Deletion List: holding(x)
 Add List: hand empty, on(x,table), clear(x)
R3 : stack(x,y) // place x on y
 Precondition & Deletion List: holding(x), clear(y)
 Add List : on(x,y), clear(x)
R4 : unstack(x,y) // Remove x placed on y
 Precondition & Deletion List: on(x,y), clear(x)
 Add List: holding(x), clear(y)
 Plan for the block world problem

• For the given problem, Start  Goal can be achieved by the

following sequence:

1. Unstack(C,A) // remove C placed on A

2. Putdown(C) // put C on table
3. Pickup(B)
4. Stack(B,C) // put B on C
5. Pickup(A)
6. Stack(A,B) // put A on B

• Execution of a plan: achieved through a data structure called

Triangular Table
 Planning vs Problem Solving

• Planning agent is similar to problem solving agent as both

 Construct plans to achieve goals, then executes them
• Note: Issues with the problem solving techniques - Huge state space &
large branching factor

• Planning agent is different from problem solving agent in:

 Representation of goals, states, actions
 Use of logical representations & Way it searches for solutions

• Planning systems follow “Divide-and-conquer” approach

12
Example: Transportation (Air Cargo) problem

\\ swap cargos
\\ load cargo c into plane p at airport a

\\ unload cargo c from plane p at airport a

\\ plane p flies from airport ‘from’ to ‘to’

13
Example: Spare Tire problem

\\ If requirements are not satisfied, leave it for next day

14
Example: Spare Tire problem
 Representation of Action
Example – Transportation (Air Cargo) problem

• Representation for Actions

• Action description – serves as a name for possible action

• Precondition – a conjunction of positive literals // state required before
action can be applied
• Effect – a conjunction of literals (positive or negative)

Op ( ACTION: Go(there),
PRECOND: At(here)  Path(here, there),
EFFECT: At(there)  At(here) )
 Partial Order Planning

Idea: (Plan to reach goal is divided into sub-plans with sub-goals)

• Work on several sub-goals independently (Divide & conquer)
• Solve them with sub-plans & Combines the sub-plans
• Flexibility in ordering the sub-plans
• Least commitment strategy:
 Delaying the ordering choice during the search
 Ex: Leave actions unordered unless they must be sequential

Strengths of POP:
• Early pruning of the search space parts in case of irresolvable conflicts
• The solution is partial-order plan (produce flexible plans)
 Partial Order Planning
Example – Shoes & Socks Problem
Actions
Op( ACTION: RightShoe, PRECOND: RightSockOn, EFFECT: RightShoeOn)
Op( ACTION: RightSock, EFFECT: RightSockOn)
Op( ACTION: LeftShoe, PRECOND: LeftSockOn, EFFECT: LeftShoeOn)
Op( ACTION: LeftSock, EFFECT: LeftSockOn)

Initial plan
Plan (
STEPS: { S1: Op (ACTION: start),
S2: Op (ACTION: finish, PRECOND: RightShoeOn  LeftShoeOn )},
ORDERINGS: {S1  S2}, // S1 should take place before S2
BINDINGS: {},
LINKS: {}
)
Example – Shoes & Socks Problem
Example – Shoes & Socks Problem
 Hierarchical Planning
• Hierarchical task network (HTN) planning is an approach to
automated planning in which the dependency among actions can
be given in the form of hierarchically structured networks

• Hierarchical Task Network

 At each “level”, only a small number of planning actions
 Then descend to lower levels to “solve these” for real

 At higher “levels”, the planner ignores “internal effects”

of decompositions
 But these have to be resolved at some (later) level
 HTN Example – House Construction

• Construction Domain:
 Actions: // arrows show dependency
 Buy Land: Money  Land

 Get Loan: Good Credit  Money

 Get Permit: Land  Permit

 Hire Builder:  Contract

 Construction: Permit  Contract  House Built

 Pay Builder: Money  House Built  House …

 HTN Example – House Construction

• Macro Action in Library:

 Build House:
HTN Example – House Construction
25
Uncertain Knowledge and
Reasoning

Dr. Pulak Sahoo

Associate Professor
Silicon Institute of Technology
 Topics
 Uncertainty-Introduction
 Uncertainty-Example
 Reasons for using probability
 Uncertain knowledge and reasoning
 Probability theory
 Axioms of probability
 Conditional probabilities
 Bayes’ Rule, Theorem and its use
2
 Topics
 Inferences

 Bayesian networks

 Belief Network for the example

 Bayesian network semantics

 Incremental Network Construction

 Representing the joint probability distribution

3
 Uncertainty-Introduction
 Agent may need to handle uncertainty due to :
 Partially Observable Env (don’t know the current state)
 Non-deterministic Env (don’t know result of a sequence of actions)

 Ex: In up & down economy, whether to invest in stock market

 Uncertainty is expressed as A  B (if A is true, then B is true. But
not sure whether A is true)

 Uncertainty is dealt by:

 Keeping track of belief state (all possible current states)
 Generating contingency plan (for all possible eventualities)

 Drawbacks
 Very large & complex belief state representations & Contingency plan
4
 Uncertainty-Example
 Uncertainty is expressed as A  B

 Example of uncertainty: Diagnosis of a dental patient’s toothache

 Consider below rule –

 p symptom(p,Toothache)  disease(p,cavity)
 Rule is wrong – because not all patients have cavity

 Fixing of the above rule –

 p symptom(p,Toothache)  disease(p,cavity) 
disease(p,gum_disease)  disease(p,abscess) …
 Exhaustive list of possible problèms
5
 Uncertain knowledge and reasoning
can be dealt using:

 Probability theory

 Bayesian networks

 Certainty factors

6
 Reasons for using probability

 Three main reasons

 Laziness
 Too much work to find & list all antecedents (influencing factors).

 Theoretical ignorance
 Medical science has no complete theory for this domain.

 Practical ignorance
 Uncertain about a patient condition as so many tests are impossible
to run.
 In such judgemental domains like law, business, design.., the
agent’s knowledge is only a degree of belief
 Which can be dealt with probability theory
7
 1. Probability theory

 Probability provides a way of summarizing the uncertainity

 Ex- The chance of cavity for toothache of a patient with history of
gum desease is 40% or 0.4 probablity

 Probability theory  degree of belief or plausibility of a

statement – a numerical measure in the range [0,1]

8
 Probability theory - representation

 Unconditional or prior probability of A – the degree of belief in

A in the absence of any other information – P(A)
 A – random variable
 Probability distribution – P(A), P(A,B)
Example-
P(Weather = Sunny) = 0.1 P(Weather = Rain) = 0.7
P(Weather = Snow) = 0.2 Weather – random variable
 P(Weather) = (0.1, 0.7, 0.2) – probability distribution

 Conditional probability – posterior – The agent has obtained some

evidence B (is true) for A - P(A|B) – Probability of A given B
 Example- P(Cavity | Toothache) = 0.4
9
 Axioms of probability
 The measure of the occurrence of an event (random variable) A – a
function P:S  R satisfying the axioms:
 0  P(A)  1
 P(S) = 1 ( or P(true) = 1 and P(false) = 0)
 P(A  B) = P(A) + P(B) - P(A  B)
 P(A  ~A) = P(A)+P(~A) –P(false) = P(true)
 P(~A) = 1 – P(A)

 A & B mutually exclusive  P(A  B) = P(A) + P(B) // P(A  B) is null

 P(e1  e2  e3  … en) = P(e1) + P(e2) + P(e3) + … + P(en)
The probability of a proposition a = the sum of the probabilities of the
atomic events (e(a) ) in which a holds P(a) =  P(ei)
eie(a)

10
Uncertain Knowledge and
Reasoning

Dr. Pulak Sahoo

Associate Professor
Silicon Institute of Technology
 Topics
 Uncertainty-Introduction
 Uncertainty-Example
 Reasons for using probability
 Uncertain knowledge and reasoning
 Probability theory
 Axioms of probability
 Conditional probabilities
 Bayes’ Rule, Theorem and its use
12
 Topics
 Inferences

 Bayesian networks

 Belief Network for the example

 Bayesian network semantics

 Incremental Network Construction

 Representing the joint probability distribution

13
 Conditional probabilities

 Conditional probabilities (for A) can be defined in terms of

unconditional probabilities (for B)

 The condition probability of the occurrence of A if event B

actually occurs ( or given B (is true))
 P(A|B) = P(A  B) / P(B)
This can be written also as:
 P(A  B) = P(A|B) * P(B)

Probability distributions (for different outcomes a1, b1, b2…)

 P(A=a1  B=b1) = P(A=a1|B=b1) * P(B=b1)

 P(A=a1  B=b2) = P(A=a1|B=b2) * P(B=b2) ….

 P(X,Y) = P(X|Y)*P(Y)

14
 Bayes’ Rule, Theorem and its use

 P(A  B) = P(A|B) *P(B) & P(A  B) = P(B|A) *P(A)

 Bayes Theorem

 P(B|A) = P(A | B) * P(B) / P(A) // P(B|A)*P(A) = P(A|B)*P(B)

 Inferences

Probability distribution P(Cavity, Toothache)

Toothache  Toothache
Cavity 0.04 0.06 // cavity with/wo toothache
 Cavity 0.01 0.89 // no cavity

 P(Cavity) = 0.04 + 0.06 = 0.1

 P(Cavity  Toothache) = 0.04 + 0.01 + 0.06 = 0.11
 P(Cavity | Toothache) = P(Cavity  Toothache) / P(Toothache) =
0.04 / 0.05
16
 Inferences
Toothache ~ Toothache
Catch ~ Catch Catch ~ Catch
Cavity 0.108 0.012 0.072 0.008
~ Cavity 0.016 0.064 0.144 0.576

Probability distributions P(Cavity, Toothache, Catch)

 P(Cavity) = 0.108 + 0.012 + 0.72 + 0.008 = 0.2

 P(Cavity  Toothache) = 0.108 + 0.012 + 0.072 + 0.008 + 0.016

 + 0.064 = 0.28

 P(Cavity | Toothache) = P(Cavity  Toothache) / P(Toothache)

= [P(Cavity  Toothache  Catch) + P(Cavity  Toothache  ~ Catch)]

* / P(Toothache)
= (0.108 + 0.012 ) / (0.108 + 0.012 + 0.016 + 0.064) = 0.12 / 0.2 = 0.6
17
 Bayesian networks

 Represents dependencies among random variables

 Gives a short specification of conditional probability
distribution
 Many random variables are conditionally independent
 Simplifies computations
 Graphical representation
 DAG – Causal relationships among random variables
 Allows inferences based on the network structure
18
 Bayesian networks

A BN is a DAG in which each node is annotated with

quantitative probability information:
 Nodes represent random variables (discrete or continuous)
 Directed links XY: X has a direct influence on Y, X is
said to be a parent of Y
 Each node X has an associated conditional probability
table, P(Xi | Parents(Xi)) that quantify the effects of the
parents on the node

 Example: Weather, Cavity, Toothache, Catch

 Weather, Cavity  Toothache, Cavity  Catch

19
Belief Network for the Burglar alarm
example
P(B) P(E)
0.001 0.002
Burglary Earthquake

B E P(A)
T T 0.95
Alarm T F 0.95
F T 0.29
F F 0.001

A P(J) A P(M)
JohnCalls MaryCalls
T 0.90 T 0.70
F 0.05 F 0.01

20
 Example: Burglar alarm at your home

 Burglar alarm at your home

 Fairly reliable at detecting a burglary
 Also responds on occasion to minor earthquakes

 Two neighbors who, on hearing the alarm calls you at

office
 John always calls when he hears the alarm, but sometimes
confuses the telephone ringing with the alarm & calls
 Mary likes loud music and sometimes misses the alarm
altogether
21
 Belief Network for the Burglar alarm example

P(B) P(E)
Parent 0.001 0.002
Burglary Earthquake

B E P(A)
T T 0.95
Alarm T F 0.95
F T 0.29
F F 0.001
Child A P(J)
JohnCalls MaryCalls A P(M)
T 0.90 T 0.70
F 0.05 F 0.01

22
 Representing the joint probability distribution

 A generic entry in the joint probability distribution P(x1, …, xn)

is given by:
n
P( x1 ,..., xn )   P( xi | Parents( X i ))
i 1

 Probability of the event that “the alarm has sounded but neither
a burglary nor an earthquake has occurred, and both Mary and
John call”:

P(J  M  A  B  E)
= P(J | A) P(M | A) P(A | B  E) P(B) P(E)
= 0.9 X 0.7 X 0.001 X 0.999 X 0.998 = 0.00062

23
 Bayesian network semantics

A) Represent a probability distribution (table)

B) Specify conditional independence – build the network
A) Each value of the probability distribution can be
computed as:
P(X1=x1  … Xn=xn) = P(x1,…, xn) =
i=1,n P(xi | Parents(xi))
 where Parents(xi) represent the specific values of
Parents(Xi)
24
 Incremental Network Construction

1. Choose the set of relevant variables Xi that describe the

domain

2. Choose an ordering for the variables (very important step)

3. While there are variables left:

a) Pick a variable X & add a node to the network for it
b) Set Parents(X) to some minimal set of nodes already in
the net such that the conditional independence property
is satisfied
c) Define the conditional probability table for X
25
THE END

26
 Learning

Dr. Pulak Sahoo

Associate Professor
Silicon Institute of Technology
 Contents
• Introduction
• General models of Learning agents
 Four elements of learning agent

• Paradigms of learning
• Supervised & Unsupervised Learning
• Inductive Learning
 Decision Tree Induction

 Artificial Neural Networks

 Perceptron

• Reinforcement Learning
 Introduction

• The idea behind learning:

• Percepts should be used not only for acting, but also for
improving the agent's ability to act in the future

• Learning takes place as a result of :

• Interaction between the agent & the world
• From observation by the agent of its own decision-making
processes
 GENERAL MODEL OF LEARNING AGENTS

Improves the
Performance
element

Suggests
new actions
 Four elements of learning agent

1. Performance element
– It takes the percepts as input & decides on external actions

2. Critic
– Tells the learning element how well the agent is doing

3. Learning element
– Takes feedback on how the agent is doing from Critic
– Determines how the performance element can do better in future

4. Problem generator
– Suggests actions that will lead to new & informative experiences
 Paradigms of learning

1. Supervised Learning
– Uses a labeled training set to teach the model to yield the desired output.
– This training dataset includes inputs & correct outputs (labeled by a
friendly teacher) allowing the model to learn over time
2. Unsupervised Learning
– The training dataset are not labeled (output not provided)
– The agent has to discover the patterns within the dataset on its own
3. Inductive Learning
– Is used for generating a set of a classification rules which are of the form
“IF-THEN”
– These rules are produced at each iteration on the dataset & appended to
the existing set of rules
4. Reinforcement Learning
– Helps in taking suitable action to maximize reward in a particular situation
– The agent receives some feedbacks (positive or negative) on its action, but
is not told what is the correct action
 1. Supervised learning
• Supervised learning method involves the training of the system or
model where the output labels are provided
• The model is trained with guidance of already known outputs/labels
• to facilitate the prediction of outputs/labels for future unknown
instances (Test Data)

• SL is used in Classification & Regression problems

 2. Unsupervised learning
• This model does not involve the target output/label (unlabeled data)
• No training is provided to the system/model
• The system has to learn by its own through analysis of the structural
characteristics in the input patterns.
• USL is used in Clustering problems
 Differences - Supervised & Unsupervised Learning

1. SL technique deals with the labeled data where the output data
patterns are known to the system. But, the USL technique works with
unlabeled data

2. The SL method is less complex while USL method is more

complicated

3. The outcome of SL technique is more accurate & reliable. USL

technique generates moderately reliable results

4. Classification & Regression are the types of problems solved

under the SL method. The USL method includes clustering
problems
 3. Inductive Learning
• Is an iterative learning algorithm:
 Used for generating a set of classification rule of the form “IF-THEN”
 For each set of examples, it produces rules at each iteration & appends
to the existing set of rules

New
examples

h: X  Y
Inductive Learning Example – Restaurant selection

Food Can Fast Price Bar Should

(3) Chat Service (3) avl go?
(2) (2) (2)
great yes yes normal no yes
great no yes normal no yes
mediocre yes no high no no
great yes yes normal yes yes

Instance Space: Each row is an instance & each column is a feature (attribute)
Label: Last column is the label (or output)
Training Data : Set of instances labeled with target values
Hypothesis Space: Set of all classification rules hi that we allow
 Learning

Dr. Pulak Sahoo

Associate Professor
Silicon Institute of Technology
 Contents
• Introduction
• General models of Learning agents
 Four elements of learning agent

• Paradigms of learning
• Supervised & Unsupervised Learning
• Inductive Learning
 Decision Tree Induction

 Artificial Neural Networks

 Perceptron

• Reinforcement Learning
 Classification by Decision Tree Induction

• DT induction is the learning using decision trees created from class-

labeled training tuples (training instance with output value/label
provided)

• A DT is a flowchart-like tree structure, where each internal node

denotes a test on an attribute (eg: age > 18, income < 25000…)

• The topmost node in a tree is the root node

• Each branch represents an outcome of the test

• Each leaf node holds a class label

14
Example: Computer Buying

Labeled
Training
Tuple

15
Example: Finding Customers likely to buy a
computer

16
How are decision trees used for classification?

• Given a tuple or instance, X, the attribute values of the tuple

are tested using the decision tree

• A path is traced from the root to a leaf node, which holds the
class or label prediction for that tuple

17
 Why DT classifiers so popular?
• The construction of DT classifiers does not require any domain
knowledge or parameter setting

• Can handle high dimensional data

• The learning & classification steps of DT induction are simple & fast

• Have good accuracy

• Simple to understand & interpret

• Able to handle both numerical & categorical data (salary figures or

low/medium/high scales)
18
Example: Computer Buying

19
The attribute age has the highest information gain (has more
influence on buying decision) & therefore becomes the splitting
attribute at the root node of the DT. Branches are grown for each
outcome of age.

20
 Artificial Neural Networks (ANN)
• An artificial neural network (ANN) is an AI based computing
system designed to simulate the way the human brain analyzes
and processes information
• It is used to solve complex problems the same way human’s do
• ANN consists of a pool of processing units (neurons) which
communicate by sending signals to each other over a large
number of weighted connections

• In Testing stage, each unit receives input from neighbors or

external sources & uses this to compute an output signal
which is propagated to other units

• In Learning stage, the adjustment of the weights takes place

to yield the best result

• The system is inherently parallel as many units can carry out

their computations at the same time
 Perceptron
The perceptron is the simplest form of ANN that:
• Is a single layer feed-forward network of neurons
• Takes the input signals x1, x2, …, xn connected with a weighting factor wi
• Computes a weighted sum z of those inputs
• Then passes it through a threshold/activation function & outputs the
result a
Perceptron (used for classification problems)

x1
x x

x x
x
x
x
x x

x2

5/8/2021
 4. Reinforcement Learning
• RL is a feedback-based learning technique in which an agent learns to
behave in an env. by performing the actions & seeing results of actions
• The agent gets positive feedback for each good action & gets negative
feedback or penalty for each bad action
• The agent learns automatically using feedbacks & experience without
any labeled data
• It solves problem with long-term goals, such as game-playing, robotics,
etc.
Reinforcement Learning - Example
The End!!!