Intelligent Agents

AIML lecture-2
By
Dr. Rupa Mehta
COED, SVNIT
 Outline
• Agents and environments
• Rationality
• PEAS (Performance measure, Environment,
Actuators, Sensors)
• Environment types
• Agent types
 Agents
• An agent is anything that can be viewed as perceiving its
environment through sensors and acting upon that
environment through actuators
• Human agent:
– eyes, ears, and other organs for sensors; hands,legs, mouth,
and other body parts for actuators
• Robotic agent:
– cameras and infrared range finders for sensors, various motors
for actuators
 Agents and environments

• The agent function maps from percept histories to

actions
[f: P*  A]
• The agent program runs on the physical architecture to
produce f
• agent = architecture + program
 Agents and environments
• Percept : the agent's perceptual inputs at any given
instant.
• An agent's percept sequence : complete history of
everything the agent has ever perceived.
• An agent's choice of action at any given instant can
depend on the entire percept sequence observed to
date, but not on anything it hasn't perceived
• An agent's behavior : the agent function that maps any
given percept sequence to an action.
 Agents and environments
• Normally agent functions are described in tabular form
where
• This table is large(infinite)
– Need to put bottom on the length of percept sequences
– construct this table by trying out all possible percept sequences
and recording which actions the agent does in response.
• The agent program is set of instruction to store and
retrieve these agent function depending on the
environmental information
 Vacuum-cleaner world

• Percepts: location and contents, e.g., [A,Dirty]

• Actions: Left, Right, Suck, NoOp

Partial Percept Sequence

 Rational agents
• An agent should strive to "do the right thing", based on
what it can perceive and the actions it can perform.
– The right action is the one that will cause the agent to be most
successful

• Performance measure: An objective criterion for success

of an agent's behavior
– E.g., performance measure of a vacuum-cleaner agent could be
• amount of dirt cleaned up
• amount of time taken
• amount of electricity consumed
• amount of noise generated, etc.

• Designer will devise a performance measure for all tasks

and agents, one appropriate to the circumstances.
 Rational agents
• Rational Agent:
– For each possible percept sequence, a rational agent
should
• select an action
– that is expected to maximize its performance measure,
• given
– the evidence provided by the percept sequence
– and whatever built-in knowledge the agent has.
 Rational agents
• Rationality is distinct from omniscience (all-knowing with infinite
knowledge)
• Rational at any given time depends on four things:
– The performance measure that defines the criterion of success.
– The agent's prior knowledge of the environment.
– The actions that the agent can perform.
– The agent's percept sequence to date.
• Rational Agent:
– For each possible percept sequence, a rational agent should select an
action that is expected to maximize its performance measure, given the
evidence provided by the percept sequence and whatever built-in
knowledge the agent has.
– Agents can perform actions in order to modify future percepts so as to
obtain useful information (information gathering, exploration)
• Autonomous Agent : if its behavior is determined by its own
experience (with ability to learn and adapt)
 Rational agents
• Rational choice depends only on the percept sequence
to date.
• the agent is not allowed to engage in decidedly
unintelligent activities.
– The rational Agent will look both ways before crossing a busy road. So
the percept sequence for crossing road without the looking action on
both side is uninformative
– Second, a rational agent shou1d choose the "looking" action before
stepping into the street, because looking helps maximize the expected
perfomance.
• Doing actions in order to modify future percepts
sometimes called information gathering
 PEAS
• PEAS:
– Performance measure
– Environment
– Actuators
– Sensors
• First specify the setting for intelligent agent
design
 PEAS
• e.g. task of designing an

– Agent : Automated taxi driver

– Performance measure: Safe, fast, legal, comfortable
trip, maximize profits
– Environment: Roads, other traffic, pedestrians,
customers
– Actuators: Steering wheel, accelerator, brake, signal,
horn
– Sensors: Cameras, sonar, speedometer, GPS,
odometer, engine sensors, keyboard
 PEAS
• Agent: Medical diagnosis system
• Performance measure: Healthy patient, minimize costs,
lawsuits
• Environment: Patient, hospital, staff
• Actuators: Screen display (questions, tests, diagnoses,
treatments, referrals)
• Sensors: Keyboard (entry of symptoms, findings,
patient's answers)
 PEAS
• Agent: Part-picking robot
• Performance measure: Percentage of parts in
correct bins
• Environment: Conveyor belt with parts, bins
• Actuators: Jointed arm and hand
• Sensors: Camera, joint angle sensors
 PEAS
• Agent: Interactive English tutor
• Performance measure: Maximize student's
score on test
• Environment: Set of students
• Actuators: Screen display (exercises,
suggestions, corrections)
• Sensors: Keyboard
 Environment
• A rational agent not only gather information but also
learn as much as possible from what it perceives.
• The agent's initial configuration could reflect some prior
knowledge of the environment, but as the agent gains
experience this may be modified and augmented.
• When environment is completely known a priori, the
agent need not perceive it simply acts correctly. (Fragile
agent)
– E.g. the lowly dung beetle : After digging its nest and laying its eggs, it
fetches a ball of dung from a nearby heap to plug the entrance. If the
ball of dung is removed from its grasp en route, the beetle continues its
task and pantomimes plugging the nest with the nonexistent dung ball,
never noticing that it is missing.
• Evolution has built an assumption into the beetle's
behavior, and when it is violated, unsuccessful behavior
results.
 Environment
• A rational agent not only gather information but also
learn as much as possible from what it perceives.

– the sphex wasp. The female sphex will dig a burrow, go out and sting a
caterpillar and drag it to the burrow, enter the burrow again to check all
is well, drag the caterpillar inside, and lay its eggs.
– The caterpillar serves as a food source when the eggs hatch.
– if an entomologist moves the caterpillar a few inches away while the
sphex is doing the check, it will revert to the "drag " step of its plan and
will continue the plan without modification, even after dozens of
caterpillar-moving interventions.

• The sphex is unable to lean that its innate plan is failing,

and thus will not change it.
 Environment types
• Fully observable (vs. Partially observable):

– An agent's sensors give it access to the complete state of the

environment at each point in time.
– A task environment is effectively fully observable if the sensors
detect all aspects that are relevant to the choice of action and
relevance, in turn, depends on the performance measure. Fully
observable environments are convenient because the agent
need not maintain any internal state to keep track of the world.
– An environment might be partially observable because of noisy
and inaccurate sensors or because parts of the state are simply
missing from the sensor data. E.g. a vacuum agent with only a
local dirt sensor cannot tell whether there is dirt in other squares,
and an automated taxi cannot see what other drivers are
thinking.
– If the agent has no sensors at all then the environment is
unobservable..
 Environment types
• Episodic (vs. Sequential):
– The agent's experience is divided into atomic "episodes" (each
episode consists of the agent perceiving and then performing a
single action), and the choice of action in each episode depends
only on the episode itself.
– In sequential environments the current decision could affect all
future decisions.
– Chess and taxi driving are sequential: in both cases, short term
actions can have long term consequences.
– Episodic environments are much simpler than sequential
environments because the agent does not need to think ahead
 Environment types
• Static (vs. Dynamic):
– The environment is unchanged while an agent is deliberating
• Static environments are easy to deal with because the agent need
not keep looking at the world while it is deciding on an action, nor
need worry about the passage of time.
– Crossword puzzles are static.
• Dynamic environments are continuously asking the agent what it
wants to do. It keeps track of no operation state of agent.
– Taxi driving is clearly dynamic: the other cars and the taxi itself keep moving
while the driving algorithm dithers about what to do next
• If the environment itself does not change with the passage of time
but the agent's performance score does, then it is Semi dynamic.
– Chess, when played without a clock, is semi dynamic.
 Environment types
• Discrete (vs. Continuous):
– state of the environment, to the way time is handled, and to the percepts
and actions of the agent.
• E.g. chess environment has a finite number of distinct states (excluding the clock).
Chess also has a discrete set of percepts and actions.
– A limited number of distinct, clearly defined percepts and actions.

– Taxi driving is a continuous-state and continuous-time problem

• the speed and location of the taxi and of the other vehicles sweep through a range of
continuous values and do so smoothly over time.
• Taxi-driving actions are also continuous (steering angles, etc.).
• Input from digital cameras is discrete, strictly speaking, but is typically treated as
representing continuously varying intensities and locations.
 Environment types
• Deterministic (vs. Stochastic):
– The next state of the environment is completely determined by the current state
and the action executed by the agent. (If the environment is deterministic except
for the actions of other agents, then the environment is strategic)
• In a fully observable, deterministic environment an agent need not
worry about uncertainty
– Uncertainty may be generated due to the actions of other agents in a multiagent
environment; but a game can be deterministic if each agent be able to predict
the actions of the others.
– The vacuum world as we described it is deterministic, but variations can include
stochastic elements such as randomly appearing dirt and an unreliable suction
mechanism
• The partially observable environment is stochastic.
– In the complex real application it is impossible to keep track of all the unobserved
aspects.
– Taxi driving is clearly stochastic in this sense, because one can never predict the
behavior of traffic exactly; moreover, one's tires blow out and one's engine seizes
up without warning.
 Environment types
• Nondeterministic environment is one in which actions are
characterized by their possible outcomes, but no probabilities are
attached to them.
– Nondeterministic environment descriptions are usually associated with
performance measures that require the agent to succeed for all possible
outcomes of its actions.
 Environment types
• Single agent (vs. Multiagent):
– An agent operating by itself in an environment.
• Here deciding the entity as an agent is added issue
– Does an agent A (the taxi driver for example) have to treat an object B (another
vehicle) as an agent, or can it be treated merely as an object behaving according
to the laws of physics, analogous to waves at the beach or leaves blowing in the
wind?
• Another issue is whether B's behavior is best described as
maximizing a performance measure whose value depends on agent
A's behavior?
– E.g in chess,
• the opponent entity B is trying to maximize its performance measure, which,
by the rules of chess, minimizes agent A's performance measure. chess is a
competitive Multiagent environment.
– In the taxi-driving environment,
• avoiding collisions maximizes the performance measure of all agents, so it
is a partially cooperative multiagent environment.
• It is also partially competitive because, for example, only one car can occupy
a parking space.
 Environment types
• Single agent (vs. Multiagent):
–
• The agent-design problems in multiagent environments are often
quite different from those in single-agent environments;
– E.g. communication often emerges as a rational behavior in multiagent
environments; in some competitive environments, randomized behavior is
rational because it avoids the pitfalls of predictability.
 Environment types
• Known vs. unknown
– Strictly speaking, this distinction refers not to the environment itself but
to the agent's (or designer's) state of knowledge about the "laws of
physics" of the environment.
• In a known environment, the outcomes (or outcome probabilities if the
environment is stochastic) for all actions are given. Obviously,
• if the environment is unknown, the agent will have to learn how it works in
order to make good decisions.
• distinction between known and unknown environments is not the same as
the one between fully and partially observable environments.
– It is quite possible for a known environment to be partially observable-
• for example, in solitaire card games, I know the rules but am still unable to see the
cards that have not yet been turned over. Conversely, an unknown

• environment can be fully observable-in a new video game, the screen may
show the entire game state but I still don't know what the buttons do until I
try them.
 Environment types
Chess with Chess without Taxi driving
a clock a clock
Fully observable Yes Yes No
Deterministic Strategic Strategic No
Episodic No No No
Static Semi Yes No
Discrete Yes Yes No
Single agent No No No

• The environment type largely determines the agent design

•
• The real world is (of course) partially observable, stochastic,
sequential, dynamic, continuous, multi-agent
•
 Agent functions and programs

• An agent is completely specified by the

– agent function which is mapping percept sequences to actions
– agent function (or a small equivalence class) is rational

• Aim :
– find a way to implement the rational agent function concisel
 Table-lookup agent
• Drawbacks:
– Huge table
– Take a long time to build the table
– No autonomy
– Even with learning, need a long time to learn
the table entries
 Agent types
• Four basic types in order of increasing
generality:
•
• Simple reflex agents
• Model-based reflex agents
• Goal-based agents
• Utility-based agents
Simple reflex agents
 Simple reflex agents
• \input{algorithms/d-agent-algorithm}
•
Model-based reflex agents
 Model-based reflex agents
• \input{algorithms/d+-agent-algorithm}
•
Goal-based agents
Utility-based agents
Learning agents