Chapter 2

Agents
KAMALA L
Contents
o Agents and Environments

o Good Behaviour

o The Concept of Rationality

o The Nature of Environments

o Structure of Agents
Agents And Environment
 Agents and Environments
• An Agent is anything that can be viewed as perceiving its
environment through sensors and acting upon that environment
through actuators.

• A human agent has eyes, ears, and other organs for sensors and
hands, legs, mouth, and other body parts for actuators.

• A robotic agent might have cameras and infrared range finders for
sensors and various motors for actuators.

• A software agent receives keystrokes, file contents, and network

packets as sensory inputs and acts on the environment by displaying
on the screen, writing files, and sending network packets.
 Agents And Environment
Percept:
 We use the term percept to refer to the agent's perceptual inputs at
any given instant

Percept Sequence:
 An agent's percept sequence is the complete history of everything the
agent has ever perceived.
Agent function:
 Mathematically speaking, we say that an agent's behaviour is described by the agent
function that maps any given percept sequence to an action.

Agent program
 Internally, the agent function for an artificial agent will be implemented by an agent
program.
 The agent function is an abstract mathematical description; the agent program is a
concrete implementation, running on the agent architecture.
 To illustrate these ideas, we will use a very simple example-the vacuum-cleaner world
shown in Fig 2.1.5.
 This particular world has just two locations: squares A and B. The vacuum agent
perceives which square it is in and whether there is dirt in the square.
 It can choose to move left, move right, suck up the dirt, or do nothing.

 One very simple agent function is the following: if the current square is dirty,
then suck, otherwise move to the other square.

A[dirty]Suck,
A[dirty]A[clean]Right,
A[dirty]A[clean]B[Dirty]Suck,
A[dirty]A[clean]B[Dirty] B[Clean]Left
Agent Function
 Concept of Rationality:
A Rational agent is one that does the right thing-conceptually
speaking;

 Every entry in the table for the agent function is filled out correctly.

 Theright action is the one that will cause the agent to be most
successful.
 Performance measures:
A performance measure embodies the criterion for success of an agent's behaviour.
When an agent is plunked down in an environment, it generates a sequence of actions
according to the percept's it receives.

 Thissequence of actions causes the environment to go through a sequence of states. If

the sequence is desirable, then the agent has performed well.

 Consider
the vacuum-cleaner agent from the preceding section. We might propose to
measure performance by the amount of dirt cleaned up in a single eight-hour
shift.

A rational agent can maximize this performance measure by cleaning up the dirt, then
dumping it all on the floor, then cleaning it up again, and so on.
 For example, one point could be awarded for each clean square at each
time.
 The selection of a performance measure is not always easy. It is based on
average cleanliness over time.
 Yet the same average cleanliness can be achieved by two different agents,
one of which does a job all the time:
 while the other cleans energetically but takes long breaks.
 Rationality
• What is rational at any given time depends on four things:
• The performance measure that defines the criterion of success.
• The agent's prior knowledge of the environment.
• The actions that the agent can perform.
• The agent's percept sequence to date.(Sensors)
• PEAS

• For each possible percept sequence, a rational agent should

select an action that is expected to maximize its performance
measure, given the evidence provided by the percept sequence and
whatever knowledge the agent has.
 Rationality
• Consider the simple vacuum-cleaner agent that cleans a square if it is dirty and moves to the other
square if not;

• This is the agent function tabulated in Fig

• Is this a rational agent?
• That depends! First, we need to say what the performance measure is, what is known about the
environment, and what sensors and actuators the agent has.
 The only available actions are Left; Right, Suck, and NoOp(donothing).
 The agent correctly perceives its location and whether that location contains dirt
 Omniscience

• An omniscient agent knows the actual outcome of its actions and can act accordingly;
but omniscience is impossible in reality.

• Example: You will stop crossing road all of a sudden, if something falls from sky in
front of you.

• The Rationality is not the same as perfection. Rationality maximizes expected

performance.

• Definition of rationality does not require omniscience, because the rational choice
depends only on the percept sequence to date.
 LEARNING:
 Doing actions in order to modify future prospects.

 A rational agent not only gather information but also to learn as much as possible from what is
perceives.

 The agents initial configuration could reflect some knowledge of the environment but as the agent gain
experience this may be modified and augmented.

 There are extreme cases in which the environment is completely known a priori.

 In such cases, the agent need not perceive or learn, it's simply acts correctly.
 Autonomy
A system is autonomous to extent that its own behaviour is determined
by its own experience.
 Therefore a system is not autonomous if it is guided by its designer
accordingly to a per your decisions.
 To survive agents must have

-enough built in knowledge to survive

-ability to learn.
 Task environment which are essentially the problems to which rational
agents are the solutions.
 Task environments:
 We must think about task environments, which are essentially the
"problems" to which rational agents are the "solutions.“

Specifying the task environment

 The rationality of the simple vacuum-cleaner agent, needs
specification of
a. The performance measure
b. The environment
c. The agent's actuators and sensors.
PEAS:
• All these are grouped together under the heading of the task
environment. We call this the PEAS (Performance
Measure, Environment, Actuators, and Sensors)
description.

• In designing an agent, the first step must always be to specify

the task environment as fully as possible.
 DRIVER LESS CAR
• Safe, fast, legal, comfortable trip, maximize profits :Performance
Measure

• Roads, other traffic, pedestrians, customers: Environment

• Steering, accelerator, brake, Signal, horn, display: Actuators

• Cameras, sonar, Speedometer, GPS, Odometer, engine sensors,

keyboards:Sensors
 Medical Diagnosis
• P:Healthy patient ,minimize costs, lawsuits
• E:Patient, hospital, staff
• A:Display questions, tests, diagnosis, treatments, referrals
• S:Keyboard entry of symptoms, findings,patients answers
 INTERACTIVE ENGLISH TUTOR

• Maximize student’s score on tests: Performance Measure

• Set of students, testing agency: Environment

• Display, Suggestion, Corrections, exercises: Actuators

• Keyboard Entry: Sensors

 Properties of task environments:

• Fully observable vs. partially observable

• Deterministic vs. stochastic
• Episodic vs. sequential
• Static vs. dynamic
• Discrete vs. continuous
• Single agent vs. Multiagent
• Known vs unknown
 Fully observable vs. partially
observable
 If an agent's sensors give its access to the complete state of the
environment at each point in time, then we say that the task
environment is fully observable.

A task environment is effectively fully observable if the sensors detect all

aspects that are relevant to the choice of action

 For example, Chess is an example of Fully observable, and an

automated taxi cannot be fully observable.
 Deterministic vs. stochastic
• If the next state of the environment is completely determined by the
current state and the action executed by the agent, then we say the
environment is deterministic; otherwise, it is stochastic.
For example,
• Taxi driving is clearly stochastic, because one can never predict the
behaviour of traffic exactly; moreover, one's tires blow out and one's
engine seizes up without warning.
• The washing machine as we described it is deterministic.
 Episodic vs. sequential
 In an episodic task environment, the agent's experience is divided into
atomic episodes.

 Each episode consists of the agent perceiving and then performing a

single action.

 Crucially, the next episode does not depend on the actions taken
in previous episodes.

 For example, an agent that has to spot defective parts , each decision
on the current part, regardless of previous decisions; where as in
sequential environments, the current decision could affect all future
decisions. Chess and taxi driving are sequential.
 Static vs. dynamic
• If the environment can change while an agent is deliberating, then we
say the environment is Dynamic for that agent; otherwise, it is static.
• Static environments are easy to deal with because the agent need not
keep looking at the world while it is deciding on an action, nor need it
worry about the passage of time.
• Dynamic environments, are continuously monitors the agent what it
wants to do.
• If the environment itself does not change with the passage of time but
the agent’s performance score does, then we say the environment is
semi dynamic.
• Taxi driving is clearly dynamic. Chess, when played with a clock, is
semi dynamic. Crossword puzzles are static.
 Discrete vs.
continuous
• The discrete/continuous distinction can be applied to the state of the
environment, to the way
Time is handled, and to the percepts and actions of the agent.

• For example, a discrete-state environment such as a chess game has a

finite number of distinct states. Chess also has a discrete set of
percept's and actions.

• Taxi driving is a continuous state and continuous-time problem: the speed

and location of the taxi and of the other vehicles sweep through a range of
continuous values. Taxi-driving actions are also continuous (steering
angles,etc.).
 Single agent vs. Multiagent
 An agent solving a crossword puzzle by itself is clearly in a single-agent
environment, whereas an agent playing chess is in a two-agent environment.

 As one might expect, the hardest case is partially observable, stochastic,

sequential, dynamic, continuous, and multiagent.

 Chess is a competitive multiagent environment. In the taxi-driving environment,

on the other hand, avoiding collisions maximizes the performance measure of
all agents, so it is a partially cooperative multiagent environment.

 It is also partially competitive because, for example, only one car can occupy a
parking space.
 Known vs unknown
• In a known environment, the outcomes for all actions are
given.

• If the environment is unknown, the agent will have to learn

how it works in order to make good decisions.
 Structure of Agents:
• The job of AI is to design the agent program that implements the agent
function mapping percept's to actions.
• We assume this program will run on some sort of computing device with
physical sensors and actuators-we call this the architecture:
• Obviously, the program we choose has to be one that is appropriate for
the architecture.
• If the program is going to recommend actions like Walk, the architecture
had better have legs.
• Agent = architecture + program
 Agent programs
• The agent programs take the current percept as input from the sensors
and return an action to the actuators.
• Notice the difference between the agent program, which takes the current
percept as input, and the agent function, which takes the entire percept
history.
• The agent program takes just the current percept as input because
nothing more is available from the environment
• if the agent's actions depend on the entire percept sequence, the agent
will have to remember the percepts
 Drawbacks:
 Tablelookup of percept-action pairs defining all possible condition-action
rules necessary to interact in an environment
Problems
 Too big to generate and to store
 Not adaptive to changes in the environment; requires entire table to be
updated if changes occur i.e., No autonomy
 Not suitable for all kinds of agents.
 Take a long time to build the table
 Agent types
• Four basic kinds of agent programs that embody the principles underlying almost
all intelligent systems:
• Simple reflex agents
• Model-based reflex Agents
• Goal-based Agents
• Utility-based Agent
• Learning Based Agent
 Simple Reflex Agent
• The simplest kind of agent is the simple reflex agent.

• These agents select actions on the basis of the current percept,

ignoring the rest of the percept history.

• For example, the vacuum agent whose Its decision is based only on the
current location and on whether that contains dirt
 Simple Reflex Agent
 Select action on the basis of only the current percept.
 E.g. the vacuum-agent

 Large reduction in possible percept/action situations.

 Implemented through condition-action rules

Function REFLEX-VACUUM-AGENT ([location, status]) return an
action
if status == Dirty then return Suck
else if location == A then return Right
else if location == B then return Left
 Characteristics:
 Only works if the environment is fully observable.

 Lacking history.
 Model-based reflex agents
 The agent should maintain some sort of internal state that depends on the percept
history and thereby reflects at least some of the unobserved aspects of the current
state.
 Updating this internal state information as time goes by requires two kinds of
knowledge to be encoded in the agent program.
 First, we need some information about how the world evolves independently of
the agent-
 Second, we need some information about how the agent's own actions affect the
world
 This knowledge about how the world working is called a model of the world. An
agent that uses such a model is called a model-based agent.
Function Model_based_agent (percept)
persistent: rules, a set of condition-action rules
state, a description of the current world state
action, the most recent action
Model, a description of how next state depends on current state
and action
State UPDATESTATE(state, action, percept, model)
Rule RULE-MATCH(state, rules)
Action RULE.ACTION
return action
 Model-based reflex agents
• The model-based reflex agent with internal state, showing how the current
percept is combined with the old internal state to generate the updated
description of the current state, based on the agent’s model of how the world
works.
• The interesting part is the function UPDATE-STATE, which is responsible for
creating the new internal state description.
• One use for Model-Based Reflex Agents is when initially your agent does not know
the map, but you want it to explore and remember the places. So as it moves
around and sees more of the level, it creates an internal map.
Goal Based agent
 Goal based agent
 It keeps track of the world state as well as a set of goals it is trying to
achieve, and chooses an action that will (eventually) lead to the
achievement of its goals.
 Knowing about the current state of the environment is not always enough
to decide what
to do.
For example, at a road junction, the taxi can turn left, turn right, or
go straight on.
 The correct decision depends on where the taxi is trying to get to.
 In other words, as well as a current state description, the agent needs
some sort of goal information that describes situations that are desirable-
for example, being at the passenger's destination
• The agent program can combine this with information about the results of possible
actions in order to choose actions that achieve the goal.
• With reflex agents, the goal is implicit in the condition-action rules. With goal-based
agents, we make the goal explicit.
• This allows us to change the goal easily, and Use search and planning in deciding the
best action at any given time.
• The heart of the goal based agent is the search function.
• It returns a path from the current state to the goal—a plan.
• Ideally we want an Agent that considers all possibilities and choose the most useful
one.
• This means that in case of multiple competing goals, the agent should choose the one
with the highest Utility.
• Agents that use utility are also known as Utility-Based Agents.
 Goal based agent
• Like the Model-Based Agents, Goal-Based agents also have an internal
model of the game state.
• Whereas Model-Based Agents only need to know how to update their
internal model of the state using new observations,
• Goal-based agents have the additional requirement of knowing how their
actions will affect to reach goal state.
• While Goals can be defined in any way that suits your need and
implementation of the task.
• It also helps to provide a priority with goals, so the agents know when it
has to choose between two goals, whose utility should have a higher
weightage.
 Utility-based agents
• Goals alone are not really enough to generate high-quality behaviour
in most environments.
• For example, there are many action sequences that will get the taxi to its
destination (thereby achieving the goal) but some are quicker, safer,
more reliable, or cheaper than others.
• Goals just provide a crude binary distinction between "happy" and
"unhappy" states, whereas a more general performance measure should
allow a comparison of different world states according to exactly how
happy they would make the agent if they could be achieved.
 Utility-based agents
• Because "happy" does not sound very scientific, the customary
terminology is to say that if one world state is preferred to another,
then it has higher utility for the agent.

• Sometimes agents will have multiple conflicting goals. In this case, a

utility function is more appropriate.

• A utility function assigns a number proportional to the "happiness" of

the agent to each state.
 Utility-based agents
• An ideal agent will attempt to maximize the expected value of the
utility function, so it must know the probability of each of the possible
outcomes of an action in a given state.

• The SEARCH function differs from that of a goal-based agent in that it

attempts to find the optimal, reachable state in terms of the expected
value of the utility function.
 Learning agents:
 A learning agent has a performance element which performs
actions, and a learning
element that can change the performance element to be more efficient as
the agents learns.

 The learning element uses feedback from the critic on how the agent is
doing and determines how the performance element should be modified
to do better in the future.

 Problem generator is responsible for suggesting actions that will lead

to new and informative experiences.
 Learning agents:
A critic component is used to evaluate how well the agent is doing and provide
feedback to the learning component.

 The critic and the problem generator can actually be considered as part of the
learning component since they both help to learn.

 Learning in intelligent agents can be summarized as a process of modification of

each
component of the agent to bring the components into closer agreement with the
available
feedback information, thereby improving the overall performance of the agent.
 How the components of agents
programs work
1)Atomic representation, each state of the world is indivisible – it has no internal
structure. The algorithms underlying search and game-playing all work with
atomic representations

2) Factored representation, splits up each state into a fixed set of variables or

attributes, each of which has value. Many important areas of AI are based on
factored representations, including propositional logic and the machine
learning algorithms.

3) A structured representation, in which objects such as cows and trucks and

their various and varying relationships can be described explicitly. Structured
representations underlie relational databases and first-order logic