Chapter Two

Intelligent Agent
 Objective:
 Define what an agent is in general
 Gives ideas about agent, agent function, agent program and
architecture, environment, percept, sensor, actuator (effectors),
 Gives idea on how agent should act
 How to measure agent success
 Rational agent, autonomous agent
 Types of environment
 Types of agent

1
 What is an agent?
 Is any thing that can be viewed as perceiving its environment through
sensors and acting upon the environment through the actuators or
effectors .
 An operation involving an effector is called an action
 The agent can change the environment through actuators or effectors
 Human as an agent has eyes, ears, and other organs for sensors; and
hands, legs, mouth, and other organs as effectors
 Robots as an agent has camera, sound recorder, infrared range finder for
sensors; and various motors for effectors.
 A software agent receives keystrokes, file contents, and network packets
as sensory inputs and acts on the environment by displaying on the
screen, writing files and sending network packets.
 Autonomous spacecrafts
 Expert systems like the Cardiologist is an agent

2
 Cont’d…
 Percept refer to the agent’s perceptual inputs at any given
instant
 An agent’s percept sequence is the complete history of
everything the agent has ever perceived
 In general, an agent’s choice of action at any given instant
can depend on the entire percept sequence observed to
date, but not on anything it hasn’t perceived
 Basic faculties of agent
 Acting
 Sensing
 Understanding, reasoning, learning

3
 Agents and environments

4
 Cont’d…

 An agent’s behavior is described by the agent function

that maps any given percept sequence to an action
 The agent function maps from percept histories to
actions: [f: P*  A]
 The agent program runs on the physical architecture
to produce f
 agent = architecture + program
5
 Cont’d…
 An Intelligent Agent must sense, must act, must be
autonomous (to some extent),. It also must be rational.
 AI is about building rational agents. An agent is
something that perceives and acts.
 A rational agent always does the right thing.

6
Ideal Example of Agent
Vacuum-cleaner world
 Percepts: location and contents
 e.g., [A, Dirty]

 Actions: Left, Right, Suck

7
 How should Agents act?

Agents may be rational or human like

We have seen how human act or think is difficult to understand
since due to the complex structure of human intelligence.
Our agent should be designed from rationality view that act
rationally
A rational agent is an agent that does the right thing for the
perceived data from the environment
What is right is an ambiguous concept but we can consider
the right thing as the one that makes the agent more
successful.
 Success is also measured by using performance measure
 Question: how and when do you measure success in
performance?
8
 Performance measure (how?)

 Subjective measure using the agent

 How happy is the agent at the end of the action
 Agent should answer based on his opinion
 Some agents are unable to answer, some are delude them
selves, some over estimate and some under estimate their
success
 Therefore, subjective measure is not a better way.
 Objective Measure imposed by some authority is an
alternative

9
 Objective Measure
 Needs standard to measure success
 Provides quantitative value of success measure of agent
 Involves factors that affect performance and weight to each
factors
 E.g., performance measure of a vacuum-cleaner agent could be
 amount of dirt cleaned up,
 amount of time taken,
 amount of electricity consumed,
 amount of noise generated, etc.
 The time to measure performance is also important for success.
 It may include knowing starting time, finishing time, duration of
job, etc

10
 Omniscience vs Rational Agent
 Omniscience agent is distinct from Rational agent
 Omniscience agent is an agent that knows the actual outcome of its action.
However, rational agent is an agent that tries to achieve more success from
its decision
 Rational agent could make a mistake because of unpredictable factors at the
time of making decision.
 Rationality maximizes expected performance
 The rational choice depends only on the percept sequence to date
 Omniscient agent that act and think rationally never make a mistake
 Omniscient agent is an ideal agent in real world
 Agents can perform actions in order to modify future percepts so as to
obtain useful information (information gathering, exploration)
 11Rational agent not only to gather information but also to learn
 Factors to measure rationality of agents
1. Percept sequence perceived so far (do we have the entire
history of how the world evolve or not)
2. The set of actions that the agent can perform (agents
designed to do the same job with different action set will
have different performance)
3. Performance measures ( is it subjective or objective? What
are the factors and their weights)
4. The agent knowledge about the environment (what kind of
sensor does the agent have? Does the agent knows every
thing about the environment or not)

12
 Ideal rational Agent

 For each possible percept sequence, an ideal rational

agent should do what every action is expected to
maximize its performance measure, on the basis of the
evidence provided by the percept sequence and what ever
built-in knowledge the agent has.
 Ideal rational agent implementation require perfection
 In real situation such agent is difficult to achieve
 Why car accident happened? Because drivers are not
perfect agent
It leads to the definition of a rational agent:
For each possible percept sequence, a rational agent should
select an action that is expected to maximize its performance
measure, given the evidence provided by the percept
sequence
13
and whatever built-in knowledge the agent has.
 Autonomy
 An agent is autonomous if its behavior is determined by its
own experience (with ability to learn and adapt)
 A rational agent should be autonomous—it should learn what
it can to compensate for partial or incorrect prior knowledge.
 Agent that lacks autonomous, if its actions are based
completely on built-in knowledge
 Example: student grade decider agent:
 Knowledge base given: rules for converting numeric grade to letter
grade
 Case 1: agent always follows the rule (lacks autonomous)
 Case 2: agent that modify the rules by learning exceptions from the
knowledge base as well as grade distribution.

14
 The Nature of Environments

 In designing an agent, the first step must always be to

specify the task environment as fully as possible.
 Task environment, which are essentially the “problems” to
which rational agents are the “solutions.”
 PEAS (Performance, Environment, Actuators, Sensors)
grouped under task environment

15
 Structure of Intelligent Agent
 Structure of AI Agent refers to the design of intelligent agent
program (function that implement agent mapping from percept to
actions) that will run on some sort of computing device called
architecture
 This course focus on intelligent agent program function, theory,
design and implementation
 Design of intelligent agent needs prior knowledge of
 Performance measure or Goal the agent supposed to achieve,
 On what kind of Environment it operates
 What kind of Actuators it has (what are the possible Actions),
 What kind of Sensors the agent has (what are the possible
Percepts)
 Performance measure  Environment  Actuators  Sensors are
abbreviated as PEAS
 Percepts Actions Goal  Environment are abbreviated as PAGE
16
 Examples of agents structure and sample PEAS
 Agent: automated taxi driver:
 Environment: Roads, traffic, pedestrians, customers
 Sensors: Cameras, sonar, speedometer, GPS, odometer, engine
sensors, keyboard
 Actuators: Steering wheel, accelerator, brake, signal, horn
 Performance measure: Safe, fast, legal, comfortable trip,
maximize profits
 Agent: Medical diagnosis system
 Environment: Patient, hospital, physician, nurses, …
 Sensors: Keyboard (percept can be symptoms, findings, patient's
answers)
 Actuators: Screen display (action can be questions, tests,
diagnoses, treatments, referrals)
 Performance measure: Healthy patient, minimize costs, lawsuits
17
 Examples of agents structure and sample PEAS
 Agent: Interactive English tutor
 Environment: Set of students
 Sensors: Keyboard (typed words)
 Actuators: Screen display (exercises, suggestions, corrections)
 Performance measure: Maximize student's score on test
 Agent: Satellite image analysis system
 Environment: Images from orbiting
 Sensors: Pixels of varying intensity, color
 Actuators: print categorization of scene
 Performance measure: Correct categorization
 Agent: Part picking robot
 Environment: Conveyor belt with parts
 Sensors: pixels of varying intensity
 Actuators: pickup parts and sort into bins
 Performance measure: place parts in correct bins
18
 Agent programs
 An agent is completely specified by the agent function that maps
percept sequences into actions
 Aim: find a way to implement the rational agent function
concisely
 Skeleton of the Agent
FUNCTION SKELETON-AGENT (percept) returns action
static memory, the agent’s memory of the world
memory UPDATE-MEMORY (memory, percept)
action  CHOOSE-BEST-ACTION (memory)
memory UPDATE-MEMORY (memory, action)
RETURN action
Note:
1. the function gets only a single percept at a time
Q: how to get the percept sequence?
2. The goal or performance measure is not part of the skeleton
19
 Table-lookup agent

 Table look up agent store all the percept sequences –action pair
into the table
 For each percept, this type of agent will search for the percept
entry and return the corresponding actions.
 Table look up couldn’t be the right option to implement
successful agent
 Why?
 Drawbacks:
 Huge table
 Take a long time to build the table
 No autonomy
 Even with learning, need a long time to learn the table entries

20
 Agent types
 Based on memory of the agent, and the way the agent takes
action we can divide agents into five basic types:
 These are (according to their increasing order of
generality) :
1. Simple reflex agents
2. Model-based reflex agents
3. Goal-based agents
4. Utility-based agents
5. Learning agent
Each will be discussed soon with models.
Notation of model:
 Rectangles: used to represent the current internal state of the
agent decision process
 Ovals: used to represent the background information used in
21
the process
Simple reflex agents
 These agents select actions on the basis of the current
percept, ignoring the rest of the percept history

22
 Simple reflex agent function prototype
 Function SIMPLE_REFLEX_AGENT(percept) return action
static : rules, a set of condition –action-rule

stateINTERPRET-INPUT(percept)
ruleRULE-MATCH(state, rules)
ActionRULE-ACTION[rule]
Return action
The above logical representation only work if the environment
is fully observable
e.g., Vacuum cleaner is a simple reflex agent, because its
decision is based only on the current location and on whether
that location contains dirt.
23
Simple reflex agents example
 Consider an artificial robot that stand at the center of
Meskel square (environment)
Agent has camera and microphone (sensor)
if the agent perceive a sound of very high frequency
(say above 20Khz) then it start to fly up to the sky as
much as possible
if the agent perceive an image which looks like a car it
run away in the forward direction
Otherwise it just turn in any direction randomly

24
Simple reflex agents example
 Imagine yourself as the driver of the automated taxi. If the
car in front brakes and its brake lights come on, then
you should notice this and initiate braking. Some
processing is done on the visual input to establish the
condition we call “The car in front is braking.” Then, this
triggers some established connection in the agent program
to the action “initiate braking.” We call
such a connection a condition–action rule, written as
if car-in-front-is-braking then initiate-braking.

25
Model-based reflex agents (also called a reflex
agent with internal state)
 Model-based reflex agents are made to deal with partial
accessibility; they do this by keeping track of the part of
the world it can’t see now.
 It does this by keeping an internal state that depends on
what it has seen before so it holds information on the
unobserved aspects of the current state.
 Updating the internal state information as time goes by
requires two kinds of knowledge:
 Some information about how the world evolves independently
of the agent
 Some information about how the agent’s own actions affect the
world
26
Model-based reflex agents (also called a reflex
agent with internal state)

27
 Model-based reflex agents
 Function MODEL_BASED_AGENT(PERCEPT) return action
static state, a description of the current world state
rules, a set of condition action rules

stateUPDATE_STATE(state, percept)
ruleRULE_MATCH(state, rules)
actionRULE_ACTION[rule]
stateUPDATE_STATE(state,action)
return action

28
Goal-based agents
 Knowing something about the current state of the environment
is not always enough to decide what to do
 For example, at a road junction, the taxi can turn left, turn
right, or go straight on . The correct decision depends on
where the taxi is trying to get to- agent needs some sort of
goal information that describes situations that are
desirable(being at the passenger’s destination)
 The agent program can combine this with the model (the same
information as was used in the model based reflex agent) to
choose actions that achieve the goal.
 Decision making of goal based is different from the condition-
action rules. In that it involves consideration of the future
“what will happen if I do such and such?” and “what makes me
29happy?”
Goal-based agents

30
 Goal-based agents structure
 Function GOAL_BASED_AGENT(PERCEPT) return action
static state, a description of the current world state
goal, a description of the goal to achieve may be in
terms of state
stateUPDATE_STATE(state, percept)
actionSetPOSSIBLE_ACTIONS(state)
actionACTION_THAT_LEADS_TO_GOAL(actionSet)
stateUPDATE_STATE(state,action)
return action

31
Utility-based agents
 It uses a model of the world, along with a utility function that measures its
preferences among states of the world.
 Then it chooses the action that leads to the best expected utility, where
expected utility is computed by averaging over all possible outcome states,
weighted by the probability of the outcome.
 Goals just provide a crude binary distinction between “happy” and
“unhappy” states. A more general performance measure should allow a
comparison of different world states according to exactly how happy
they would make the agent-Utility
 An agent’s utility function is essentially an internalization of the
performance measure
 If the internal utility function and the external performance measure are in
agreement, then an agent that chooses actions to maximize its utility will be
rational according to the external performance measure

32
Utility-based agents

33
Utility-based agents structure
 Function UTILITY_BASED_AGENT(PERCEPT) return action
static state, a description of the current world state
goal, a description of the goal to achieve may be in
terms of state
stateUPDATE_STATE(state, percept)
actionSetPOSSIBLE_ACTIONS(state)
actionBEST_ACTION(actionSet)
stateUPDATE_STATE(state,action)
return action

34
Learning agents (1)
 Learning agents divided into four conceptual components:
 Learning element: Responsible for making improvements.
 The learning element uses feedback from the critic on how the agent
is doing and determines how the performance element should be
modified to do better in the future.
 The design of the learning element depends very much on the design
of the performance element
Performance element: Responsible for selecting external actions
 Critic is necessary because the percepts themselves provide no
indication of the agent’s success.

35
Learning agents (2)
 Problem generator :It is responsible for suggesting
actions that will lead to new and informative experiences.
 The point is that if the performance element had its way,
it would keep doing the actions that are best, given what it
knows.
 But if the agent is willing to explore a little and do some
perhaps suboptimal actions in the short run, it might
discover much better actions for the long run.
 The problem generator’s job is to suggest these
exploratory actions .

36
Learning agents (3)
Automated taxi example
 The performance element consists of whatever collection of knowledge and
procedures the taxi has for selecting its driving actions.
 The taxi goes out on the road and drives, using this performance element
 The critic observes the world and passes information along to the learning
element e.g after the taxi makes a quick left turn across three lanes of traffic, the
critic observes the shocking language used by other driver
 The learning element is able to formulate a rule saying this was a bad action,
and the performance element is modified
 The problem generator might identify certain areas of behavior in need of
improvement and suggest experiments, such as trying out the brakes on different
road surfaces under different conditions
 Learning in intelligent agents can be summarized as a process of modification of
each component of the agent to bring the components into closer agreement
with the available feedback information, thereby improving the overall
37performance of the agent.
Learning agents

38
 Types of Environment(Reading Assignment)
 Based on the portion of the environment observable
 Fully observable: An agent's sensors give it access to the complete
state of the environment at each point in time. (chess vs. driving)
 A task environment is effectively fully observable if the sensors
detect all aspects that are relevant to the choice of action; relevance, in turn,
depends on the performance measure
 Fully observable environments are convenient because the agent need not
maintain any internal state to keep track of the world
 An environment might be partially observable because of noisy and
inaccurate sensors or because parts of the state are simply
missing from the sensor data.
 If the agent has no sensors at all then the environment is Fully
unobservable

39
 Types of Environment Cont’d…
 Based on the effect of the agent action
 Deterministic : The next state of the environment is completely
determined by the current state and the action executed by the
agent. E.g., The vacuum world
 Strategic: If the environment is deterministic except for the
actions of other agents, then the environment is strategic
 Stochastic or probabilistic:
 Most real situations are so complex that it is impossible to keep track
of all the unobserved aspects; for practical purposes
 Taxi driving is clearly stochastic in this sense, because one can never
predict the behavior of traffic exactly; moreover, one’s tires blow out
and one’s engine seizes up without warning
 Environment is uncertain if it is not fully observable or not
deterministic
40
 Types of Environment Cont’d
 The word “stochastic” generally implies that uncertainty
about outcomes is quantified in terms of probabilities; a
nondeterministic environment is one in which actions are
characterized by their possible outcomes, but no
probabilities are attached to them
 Nondeterministic environment descriptions are usually
associated with performance measures that require the
agent to succeed for all possible outcomes of its actions

41
 Types of Environment

 Based on the number agent involved

 Single agent A single agent operating by itself in an
environment. E.g., an agent solving a crossword puzzle by itself
 Multi-agent: multiple agents are involved in the environment
 E.g., an agent playing chess
 Based on the state, the way time handle, action and
percept space pattern
 Discrete: A limited number of distinct, clearly defined state,
percepts and actions.
 E.g., the chess environment has a finite number of distinct
states (excluding the clock).Chess also has a discrete set of
percepts and actions.
42
 Types of Environment Cont’d
 Continuous: state, percept and action are consciously changing
variables
 Note: one or more of them can be discrete or continuous
 Taxi driving is a continuous-state and continuous-time problem:
the speed and location of the taxi and of the other vehicles
sweep through a range of continuous values and do so smoothly
over time. Taxi-driving actions are also continuous (steering
angles, etc.)

43
 Types of Environment Cont’d
 Based on the effect of time
 Static: The environment is unchanged while an agent is deliberating.
 Static environments are easy to deal with because the agent need not keep
looking at the world while it is deciding on an action, nor need it worry about
the passage of time e.g., Crossword puzzles
 Dynamic: The environment changes while an agent is deliberating.
 It continuously asking the agent what it wants to do; if it hasn’t decided yet,
that counts as deciding to do nothing. E.g., Taxi driving is clearly dynamic: the
other cars and the taxi itself keep moving while the driving algorithm dithers
about what to do next
 semi-dynamic: The environment is semi-dynamic if the environment
itself does not change with the passage of time but the agent's
performance score does. E.g., Chess, when played with a clock

44
 Types of Environment Cont’d
 Based on loosely dependent sub-objectives
 Episodic: The agent's experience is divided into atomic "episodes" (each
episode consists of the agent perceiving and then performing a single
action), and the choice of action in each episode depends only on the
episode itself.
 The next episode does not depend on the actions taken in previous episodes
 Many classification tasks are episodic
 E.g.,an agent that has to spot defective parts on an assembly line bases each
decision on the current part, regardless of previous decisions
 Sequential: The agent's experience is a single atomic
"episodes“
 The current decision could affect all future decisions.
 E.g., Chess and taxi driving are sequential: in both cases, short-term
actions can have long-term consequences.
 Episodic environments are much simpler than sequential
45 environments because the agent does not need to think ahead.
 Environment types example

Chess with Chess without Taxi driving

a clock a clock
Fully observable Yes Yes No
Deterministic Strategic Strategic No
Episodic No No No
Static Semi Yes No
Discrete Yes Yes No
Single agent No No No

 The environment type largely determines the agent design

 The real world is (of course) partially observable, stochastic,
sequential, dynamic, continuous, multi-agent

46
Types of Environment More Example

47
 The end

48