Chapter2:

Intelligent Agents

Ch2:Intelligent Agents Artificial Intelligence 1

 Outline

❖ Agents and environments

❖ Rationality
❖ PEAS (Performance measure, Environment, Actuators,
Sensors)
❖ Environment types
❖ Agent types

Ch2:Intelligent Agents Artificial Intelligence 2

 2.1 Agents and environments
❖ An agent is anything that can be viewed as perceiving its
environment through sensors and acting upon that
environment through actuators

❖ It is no accident that this diagram is of exactly the same form

as the diagram for the interaction of the human nervous
system with its environment.
Ch2:Intelligent Agents Artificial Intelligence 3
 2.1 Agents and environments (Cont’d)
❖ Examples
Agent Sensors Actuators
Human eyes, ears, and other hands, legs, mouth, and other
organs body parts
Robot cameras and infrared range Various motors
finders
Software Keystrokes, file contents Displaying on screen, writing
and network packets file, and sending network
packets

Ch2:Intelligent Agents Artificial Intelligence 4

 2.1 Agents and environments: Terminology
❖ Percept ➔ refers to the agent’s perceptual input at any given time
❖ Percept Sequence ➔ the complete history of every thing the agent
has ever perceived.
 An agent’s choice of action at any given instant can depend on the entire
percept sequence observed to date but not on any thing it hasn’t perceived
❖ The agent function ➔ describe the agent behavior.
➔ map any given percept sequence to an action:
[f: P* → A]
➔ it is an abstract mathematical description.
❖ The agent program ➔ the concrete implementation, running within
some physical system to produce f:
Agent = Architecture + Program

Ch2:Intelligent Agents Artificial Intelligence 5

 2.1 Agents and environments: Vacuum-
cleaner world
❖ Percepts : location and state of the environment,
❖ Actions : Left, Right, Suck, NoOp

❖ There are only two locations: square A and square B, the agent
perceives which square it is in and whether there is dirt in the
square. It can choose to move left, right or suck.

Ch2:Intelligent Agents Artificial Intelligence 6

 2.2 Good Behavior: The Concept of rationality
❖ Rational Agent: one that does the right thing.
❖ What does it mean to do the right thing?
 We answer by considering the consequences of the agent’s behavior:
▪ an agent generate a sequence of actions according to the percepts it
receives.
▪ This sequence of actions causes the environment to go through a sequence
of states.
▪ If the sequence is desirable, then the agent has performed well.
▪ Desirability is captured by a performance measure ➔ that evaluates any
given sequence of environment states.
▪ Example : performance measure of a vacuum-cleaner agent could be
amount of dirt cleaned up, amount of time taken, amount of electricity
consumed, amount of noise generated, etc.
▪ When do we evaluate the performance is also important. If we measured
how much dirt the agent cleaned up in the first hour of the day, we would be
rewarding those agents that start fast and punishing those work late.

Ch2:Intelligent Agents Artificial Intelligence 7

 2.2.1 Rationality
❖ Rationality of Agents Depends on:
 The performance measure that defines the criterion of success.
 The agent ‘s prior knowledge of the environment.
 The actions that the agents can perform.
 The agent’s percept sequence to date

❖ A definition of a rational agent:

 For each possible percept sequence, a rational agent should
select an action that is expected to maximize its performance
measure, given the evidence provided by the percept
sequence and whatever built-in knowledge the agent has.

Ch2:Intelligent Agents Artificial Intelligence 8

 2.3.1 Specifying the Task Environment
❖ To design a rational agent, First we must specify the
task environment:
 Which are the “problem” specification for which the agent
is a solution

❖ PEAS: to specify a task environment

 Performance measure
 Environment
 Actuators
 Sensors

Ch2:Intelligent Agents Artificial Intelligence 9

 2.3.1 Specifying the Task Environment : Example 1

❖ PEAS : Taxi Driver Agent

 Performance measure : Safe, fast, legal, comfortable trip,

maximize profits
 Environment : Roads, other traffic, pedestrians, customers
 Actuators : Steering wheel, accelerator, brake, signal, horn
 Sensors : Cameras, sonar, speedometer, GPS, odometer,
engine sensors, keyboard

Ch2:Intelligent Agents Artificial Intelligence 10

 2.3.1 Specifying the Task Environment : Example 2

❖ PEAS : Medical Diagnosis Agent

 Performance measure : Healthy patient, minimize costs,

 Environment : Patient, hospital, staff
 Actuators : Screen display (questions, tests, diagnoses,
treatments, referrals)
 Sensors : Keyboard (entry of symptoms, patient's answers)

Ch2:Intelligent Agents Artificial Intelligence 11

 2.3.1 Specifying the Task Environment : Example 3

❖ PEAS : Interactive English Tutor

 Performance measure : Maximize student's score on test

 Environment : Set of students
 Actuators : Screen display (exercises, suggestions,
corrections)
 Sensors : Keyboard

Ch2:Intelligent Agents Artificial Intelligence 12

 2.3.1 Specifying the Task Environment : Example 4

❖ PEAS : Part-Picking Robot

 Performance measure : Percentage of parts in correct bins

 Environment : Conveyor belt with parts, bins
 Actuators : Jointed arm and hand
 Sensors : Camera, joint angle sensors

Ch2:Intelligent Agents Artificial Intelligence 13

 2.3.2 properties of task environment
❖ Fully observable (vs. partially observable):
 )‫اليمكن‬/‫غير مرصود) (يمكن الوصول اليه‬/‫(مرصود‬

 If an agent's sensors give it access to the complete state of the

environment at each point in time, then the task environment is
fully observable.

 An environment might be partially observable because of

noisy inaccurate sensors or because parts of the state are
simply missing from the sensor data
▪ For example: a vacuum agent with only a local dirt sensor
cannot tell whether there is dirt in other squares.

Ch2:Intelligent Agents Artificial Intelligence 14

 2.3.2 properties of task environment (Cont’d)

❖ Single agent (vs. multiagent):

 Single agent is an agent operating by itself in an environment.

 For multiagent environment may be competitive or

cooperative

Ch2:Intelligent Agents Artificial Intelligence 15

 2.3.2 properties of task environment (Cont’d)
❖ Deterministic (vs. stochastic):
 )‫ عشوائى‬/‫(محدد‬

 If the next state of the environment is completely determined by

the current state and the action executed by the agent, then the
environment is deterministic.
 otherwise, it is stochastic, implies that uncertainty about
outcomes is quantified in terms of probabilities
 An environment is uncertain if it is not fully observable or not
deterministic.
 A nondeterministic environment is one in which actions are
characterized by their possible outcomes, but no probabilities
are attached to them

Ch2:Intelligent Agents Artificial Intelligence 16

 2.3.2 properties of task environment (Cont’d)
❖ Episodic (vs. sequential):
 )‫ غير مرحلى‬/ ‫(مرحلى‬

 In an episodic environment an agent’s action is divided into

atomic episodes. Decisions do not depend on previous
decisions/actions.
▪ For example: a defective parts detection agent.

 In sequential environment the current decisions affect all future

decisions

Ch2:Intelligent Agents Artificial Intelligence 17

 2.3.2 properties of task environment (Cont’d)
❖ Static (vs. dynamic):
 If the environment can change while an agent is deliberating,
then it is dynamic for that agent.
 Otherwise it is static.
▪ Static environments are easy to deal with because the agent
– Need not keep looking at the world while it is deciding on an action
– Need not to worry about the passage of time

 The environment is semidynamic if the environment itself does

not change with the passage of time but the agent's
performance score does

Ch2:Intelligent Agents Artificial Intelligence 18

 2.3.2 properties of task environment (Cont’d)
❖ Discrete (vs. continuous):
 )‫ مستمر‬/ ‫(متقطع‬
 Applies to
▪ The state of the environment.
– The chess environment has a finite number of states.
– Taxi-driving is a continuous state environment
▪ The way time is handled.
– Taxi-driving is a continuous time problem
▪ The percept and actions of the agents.
– Chess has discrete set of percepts and actions
– Taxi-driving actions are continuous

Ch2:Intelligent Agents Artificial Intelligence 19

 2.3.2 properties of task environment (Cont’d)
❖ Known (vs. unknown):
 This is not a property of the environment, it refers to the agent’s
state of knowledge about the lows of the environment.
 In a known environment, the outcomes for all actions are given.
 If the environment is unknown the agent will have to learn how it
works in order to make good decisions
 It is possible for a known environment to be partially observable
▪ In solitaire card game, I know the rules but still unable to see the
cards that have not yet been turned over.
 Conversely, an unknown environment can be fully observable
▪ In a new video game, the screen may show the entire game state
but I still don’t know what the buttons do until I try them.

Ch2:Intelligent Agents Artificial Intelligence 20

 2.3.2 properties of task environment (Cont’d)

task observable deterministic/ episodic/ static/ discrete/ agents

environment stochastic sequential dynamic continuous
crossword fully deterministic sequential static discrete single
puzzle
chess with fully deterministic sequential semi discrete multi
clock
taxi partial stochastic sequential dynamic continuous multi
driving
medical partial stochastic sequential dynamic continuous single
diagnosis
Part-picking partial stochastic episodic dynamic continuous single
robot

Ch2:Intelligent Agents Artificial Intelligence 21

 2.3.2 properties of task environment (Cont’d)
❖ The environment type largely determines the agent
design
❖ The real world is (of course)
 partially observable,
 stochastic,
 sequential,
 dynamic,
 continuous,
 multi-agent

Ch2:Intelligent Agents Artificial Intelligence 22

 2.4 The structure of the agent
❖ Goal of AI
 Given a PEAS task environment,
 Construct agent function f,
 Design an agent program that implements f on a particular
architecture (Agent = Architecture + Program)
❖ In general, the architecture
 Makes the percept from the sensors available to the program
 Run the program, and
 Feeds the program’s action choices to the actuators.
❖ The agent program take the current percept as inputs from the
sensors and return an action to the actuators.
 If the agent’s actions depend on the entire percept sequence, the agent
will have to remember the percepts

Ch2:Intelligent Agents Artificial Intelligence 23

 2.4.1 Agent programs
❖ Agent Types
 Table-Driven Agents )‫– (العميل المعتمد على جدول التقابل‬Use a percept
❖Do what we want sequence/ action table in memory to find the next action. They
(not intelligent) are implemented by a (large) lookup table.
 Simple Reflex agents )‫– (العميل المبنى على رد الفعل‬Are based on
condition-action rules and implemented with an appropriate
production (rule-based) system. They are stateless devices,
❖Four basic kind
of agent programs which do not have memory of past world states.
that embody the  Model-based reflex )‫– (العميل المبنى على رد الفعل مع ذاكرة‬Have internal
principles state, which is used to keep track of past states of the world.
underlying all
intelligent systems  Agents with Explicit Goals )‫– (العميل المبنى على هدف واضح‬Are
agents, which in addition to state information have a kind of goal
❖All these can be information that describes desirable situations. Agents of this
turned into kind take future events into consideration.
learning agents
 Utility-Based Agents )‫–(العميل المبنى على تقدير فائدة قراراته‬Base their
decision on classic axiomatic utility-theory in order to act
rationally.
 Learning Agents )‫(العميل المبنى على قابلية التعلم‬
Ch2:Intelligent Agents Artificial Intelligence 24
 2.4.1 Agent programs : Table-Driven Agents
❖ Problems
 Huge number of possible
percepts (consider an automated
taxi with a camera as the sensor)
=> lookup table would be huge.
 Takes long time to build the table.
 Not adaptive to changes in the
Lookup-table environment; requires entire table
to be updated if changes occur.

Ch2:Intelligent Agents Artificial Intelligence 25

 2.4.2 Simple reflex agents
❖ These agents select actions based on the
basis of the current percept, ignoring the rest
of the percept history.
 No memory, Fails if environment is partially
observable
❖ There behavior is based on a condition-
action rule
 For example:
If car-in-front-is-braking then initiate-braking
 A simple reflex agent works by finding a rule
whose condition matches the current
situation (as defined by the percept) and then
doing the action associated with that rule.

Ch2:Intelligent Agents Artificial Intelligence 26

 2.4.2 Simple reflex agents (Cont’d)

❖ Simple reflex agents will work only if the current decision can
be made on the basis of only the current percept
 That is, if the environment is fully observable

❖ A little bit of unobservability can cause serious trouble.

Ch2:Intelligent Agents Artificial Intelligence 27

 2.4.3 Model-based reflex agents
❖The current percept is combined with ❖ To handle partial observability the
the old internal state to generate the agent has to keep track of the part
updated description of the current state, of the world it can’t see now.
based on the agent’s model of the world.
❖ The agent has to maintain some
sort of internal state (a model of
the world):
 How the world evolves
independently of the agent?
 How the agent’s own action
affects the world?

internal state ❖ A model-based reflex agent keeps

track of the current state of the
world, using an internal model . it
then chooses an action in the same
way as the reflex agent

Ch2:Intelligent Agents Artificial Intelligence 28

 2.4.3 Model-based reflex agents (Cont’d)

❖ The function UPDATE-STATE is responsible for creating the new internal

state description.
❖ The detailed of how models and states are represented depending on the
type of the environment and the technology used in the agent design.
 examples of models and updating algorithms appear in chapters 4, 12,11,15,17,
and 25.

Ch2:Intelligent Agents Artificial Intelligence 29

 2.4.3 Model-based reflex agents (Cont’d)

❖ It is seldom possible for the agent to determine the current

state of a partially observable environment exactly
❖ Instead, the box labeled “what the world is like now” represent
the agent’s best guess

Ch2:Intelligent Agents Artificial Intelligence 30

 2.4.4 Goal-based agents
❖ A model-based, goal based agent
keeps track of the world state as
well as a set of goals it is trying to
achieve, and chooses an action that
will lead to the achievement of its
goals.
❖ Sometimes goal satisfaction result
immediately from a single action.
❖ sometimes the agent has to
consider long sequences of actions
in order to achieve the goal.
 Search (ch3-5) and planning (ch10-
11) are used to find action
sequences that achieve the agent’s
❖We need to predict the future: we need goal
to plan & search

Ch2:Intelligent Agents Artificial Intelligence 31

 2.4.5 Utility-based agents

❖ Some solutions to goal states are

better than others.
❖ Which one is best given by a
utility function.
❖ A model-based, utility based
agent uses a model of the world,
along with a utility function that
measures its preferences among
states of the world.
❖ Then it chooses the action that
leads to the best expected utility.

Ch2:Intelligent Agents Artificial Intelligence 32

 2.4.6 Learning agents
❖ A learning agent can be divide into  The critic element
four components: ➢ tell the learning element how well the agent
is doing w.r.t a fixed performance standard.
 The performance element
➢ responsible for selecting external
 The problem generator
actions. ➢ responsible for suggesting actions that will
lead to new and informative experience.
➢ is what we have previously
considered to be the entire agent.
 The learning element
➢ responsible for making
improvements.
➢ uses feedback from the critic and
determine how the performance
element should be modified to do
better in the future
➢ its design depend on the design of
the performance element, i.e given
an agent design, learning
mechanisms can be constructed to
improve every part of the agent.

Ch2:Intelligent Agents Artificial Intelligence 33