Artificial Intelligence

Ch 02 Intelligent Agents
part1

Instructor: Kamal Darwish

1
 Outline

⚫ Agents and environments

⚫ Rationality
⚫ PEAS (Performance measure, Environment,
Actuators, Sensors)
⚫ Environment types
⚫ Agent types

2
 An agent is any entity that can be viewed as:
• perceiving its environment through sensors, and
• acting upon that environment through actuators.

3
 Agents and Environments [cont.]
Agents include humans, robots, softbots, thermostats, etc.
• human:
perceives: with eyes, ears, nose, hands, ...,
acts: with voice, hands, arms, legs, ...
• robot:
perceives: with video-cameras, infra-red sensors, radar, ...
acts: with wheels, motors,
• softbot:
perceives: receiving keystrokes, files, network packets, ...
acts: displaying on the screen, writing files, sending network packets
• thermostat:
perceives: with heat sensor, ...
acts: electric impulses to valves, devices, ...
4
Agents and Environments [cont.]
▪
▪
▪
▪

5
Key concepts
Percept and Percept sequences
 percept: the collection of agent’s perceptual inputs at any given
instant
 percept sequence: the complete history of everything the agent has
ever perceived

An agent’s choice of action at any given instant

 can depend on the entire percept sequence observed to date
 does not depend on anything it hasn’t perceived
Remark
An agent can perceive its own actions, but not always it effects.
6
Key concepts [cont.]
Agent function
An agent’s behavior is described by the agent function [f: P* → A]
which maps any given percept sequence into an action.
• ideally, can be seen as a table [percept sequence, action]

Agent program
Internally, the agent function for an artificial agent is implemented by an agent program.

Note: Agent function vs. agent program

 The agent function is an abstract mathematical description
• possibly-infinite description
 The agent program is a concrete implementation of the agent function
• finite description
• runs on the physical architecture to produce the agent function f
7
• agent = architecture + program
Example : Vacuum cleaner
A very-simple vacuum cleaner
 Environment: squares A and B
 Percepts: location ({A, B}) and content ({Dirty, Clean})
• e.g. [A, Dirty]
 Actions: {left, right, suck, no_op}

8
Example : Vacuum cleaner [cont.]
A simple agent function
If the current square is dirty, then suck; otherwise, move to the other square.
⚫ Agent’s function -> look-up table
⚪ For many agents this is a very large table
Percept sequence Action
[A,Clean] Right
[A,Dirty] Suck
[B,Clean] Left
[B,Dirty] Suck
[A,Clean],[B,Clean] Left
[A,Clean],[B,Dirty] Suck
etc etc

Note: this agent function depends only on the last percept, not on the whole percept
sequence. 9
Example : Vacuum cleaner [cont.]
Corresponding agent program
function Reflex-Vacuum-Agent([location,status]) returns an action
if status = Dirty then return Suck
else if location = A then return Right
else if location = B then return Left

10
 Rational Agents
 Intuition: a rational agent is one that “does the right thing”
 i.e., every entry in the agent function-table is filled out correctly
 What is the right thing?
 Approximation: the most “succesfull” thing:
 In a given environment, according to the percept sequence it
receives, an agent generates a sequence of actions, ...
 ... causing the environment to go through a sequence of states.
 If such sequence is desirable, then the agent has performed well.
⇒ need a performance measure to evaluate any sequence of
environment states
 Performance measure should be objective.
11
Rational Agents [cont.]

 Performance measure according to what is wanted in the

environment, not to how the agents should behave

 e.g. “how clean the floor is” is a better measure than “the
amount of dirt cleaned within a certain time”

12
Rational Agents [cont.]
What is rational at any given time depends on four things:
 The performance measure that defines the criterion of success
 The agent’s prior knowledge of the environment
 The actions that the agent can perform
 The agent’s percept sequence to date (from sensors)
Definition of a rational agent
For each possible percept sequence, a rational agent should select an action
that is expected to maximize its performance measure, given the evidence
provided by the percept sequence and whatever built-in knowledge the agent has.

13
 Example of Rational Agents: vacuum-cleaner agent
Under the following assumptions:
 Performance measure: one point for each clean square at each time step, over 1000
time steps
 Environment knowledge:
• “geography” known a priori,
• dirt distribution and agent initial location unknown
• [clean squares cannot become dirty again ]
 Perception: self location, presence of dirt
 Actions: Left, Right, Suck
 Is the agent rational?
⇒ Yes! (provided the given performence measure)

Beware: if a penalty for each move is given, the agent behaves poorly
⇒ better agent: do nothing once it is sure all the squares are clean
14
 Rationality vs. Omniscience vs. Perfection
 Remark
 Rationality ≠ Omniscience!
• An omniscient agent knows for sure the outcome of its actions
⇒ omniscience impossible in reality
• A rational agent may only know “up to a reasonable confidence”
(e.g., when crossing a road, what if something falling from a plane
flattens you? if so, would you be considered irrational?)

 Rational behaviour is not perfect behaviour!

• perfection maximizes actual performance
• (given uncertainty) rationality maximizes expected performance
15
Other important features

 Rationality requires other important features

Information gathering/exploration:
the rational choice depends only on the percept sequence to date
⇒ actions needed in order to modify future percepts
Ex: look both ways before crossing a busy road
Learning:
agent’s prior knowledge of the environment incomplete
⇒ learning from percept sequences improves & augments it
Ex: a baby learns from trial&errors the right movements to walk
16
Autonomy in Agents
The autonomy of an agent is the extent to which its behaviour is determined by its own
experience, rather than knowledge of designer.

⚫ Therefore, a system is not autonomous if it is guided by its designer according

to a priori decisions.
⚫ An autonomous agent can always say “no”.
⚫ Extremes
 No autonomy – ignores environment/data
 Complete autonomy – must act randomly/no program
⚫ Ideal: design agents to have some autonomy
 Possibly become more autonomous with experience
⚫ Example: a child learns how to climb a tree

17
Task Environments - PEAS

 To design a rational agent we must specify its task environment

i.e. the “problems” to which rational agents are the “solutions

 Task environment described in terms of four elements (“PEAS”):

Performance measure
Environment
Actuators
Sensors

18
PEAS: Vacuum Cleaner

Simple Vacuum Cleaner

Performance measure: 1 point per clean square per time step
Environment: squares A and B, possibly dirty
Actuators: move left/right, suck
Sensors: self location, presence of dirt

19
PEAS: Automated taxi driver
 Performance measure
• safety, destination, profits, comfort, ...
 Environment
• streets/freeways, other traffic, pedestrians, ...
 Actuators
• steering, accelerator, brake, horn, speaker/display, ...
 Sensors
• video, sonar, speedometer, engine sensors, GPS, ...
Remark
 Some goals to be measured may conflict!
e.g. profits vs. safety, profits vs. comfort, ...
=⇒ tradeoffs are required 20
▪
▪
▪
▪
▪
▪
▪
▪

21
PEAS: Part-picking robot

 Performance measure
• Percentage of parts in correct bins
 Environment
• Conveyor belt with parts, bins
 Actuators
• Jointed arm and hand
 Sensors
• Camera, joint angle sensors

22
PEAS: Interactive English tutor

 Performance measure
• Maximize student's score on test
 Environment
• Set of students
 Actuators
• Screen display (exercises, suggestions, corrections)
 Sensors
• Keyboard, microphone

23
Properties of Task Environments
Task environments can be categorized along six dimensions:
 Fully observable vs. partially observable
 Single-agent vs. multi-agent
 Deterministic vs. stochastic
 Episodic vs. sequential
 Static vs. dynamic
 Discrete vs. continuous

24
 Properties of Task Environments [cont.]
Fully observable vs. partially observable

 A task environment is (effectively) fully observable iff the sensor

detect the complete state of the environment needed to choose an
action.
 ”relevant” depends on the performance measure
 no need to maintain internal state to keep track of the environment
 A task environment may be partially observable
 Ex: Taxi driving
• noisy and inaccurate sensors
• parts of the state are not accessible for sensors
 A t.e. might be even unobservable (no sensors)
 e.g. fully-deterministic actions
25
Properties of Task Environments [cont.]
Deterministic vs. stochastic
 A task environment is deterministic iff its next state is completely
determined by its current state and by the action of the agent.
 (Ex: a crossword puzzle).
 A task environment is stochastic if uncertainty about outcomes is
quantified in terms of probabilities (Ex: dice, poker game, component
failure,...)
 A t.e. is nondeterministic iff actions are characterized by their possible
outcomes, but no probabilities are attached to them
A partially observable environment could appear to be stochastic.
• ⇒ for practical purposes, when it is impossible to keep track of all
the unobserved aspects, they must be treated as stochastic.
26
• (Ex: Taxi driving)
Properties of Task Environments [cont.]
Episodic vs. sequential
 In an episodic task environment
 the agent’s experience is divided into atomic episodes
 in each episode the agent receives a percept and then performs a
single action
⇒ episodes do not depend on the actions taken in previous
episodes, and they do not influence future episodes
 Ex: an agent that has to spot defective parts on an assembly line,
 In sequential environments the current decision could affect future
decisions ⇒ actions can have long-term consequences
 Ex: chess, taxi driving, ...
 Episodic environments are much simpler than sequential ones
 No need to think ahead!
27
Properties of Task Environments [cont.]
Static vs. dynamic
 The task environment is dynamic iff it can change while the
agent is choosing an action, static otherwise
⇒ agent needs keep looking at the world while deciding an
action
 Ex: crossword puzzles are static, taxi driving is dynamic
 The t.e. is semidynamic if the environment itself does not change
with time, but the agent’s performance score does
 Ex: chess with a clock
 Static environments are easier to deal wrt. [semi]dynamic ones

28
Properties of Task Environments [cont.]
Discrete vs. continuous
 The state of the environment, the way time is handled,
and agents percepts & actions can be discrete or
continuous
 Ex: Crossword puzzles: discrete state, time, percepts & actions
 Ex : Chess: : discrete state, time, percepts & actions
 Ex: Taxi driving: continuous state, time, percepts & actions
...

29
Properties of Task Environments [cont.]
Single-agent vs. multi-agent
 A task environment is multi-agent iff contains other agents who are
also maximizing some performance measure that depends on the
current agent’s actions
 latest condition essential
 distinction between single- and multi-agent sometimes subtle
 Two important cases
 competitive multi-agent environment: other agents’ goals conflict with
or oppose to the agent’s goals
Ex: chess, war scenarios, taxi driving (compete for parking lot), ...
 cooperative multi-agent environment: other agents’ goals coincide in
full or in part with the agent’s goals
• Ex: ants’ nest, factory, taxi driving (avoid collisions), ...
30
Properties of Task Environments [cont.]
Single-agent vs. multi-agent
 different design problems for multi-agent wrt. single-agent
 competitive: randomized behaviour often rational
(unpredictable)
 collaborative: communication with other agents often
rational

 In a multi-agent environment we ignore uncertainty that

arises from the actions of other agents
 Ex: chess is deterministic even though each agent is
unable to predict the actions of the others.
31
Properties of Task Environments [cont.]

Note
 The simplest environment is fully observable, single-agent,
deterministic, episodic, static and discrete.
 Ex: simple vacuum cleaner
 Most real-world situations are partially observable, multi-agent,
stochastic, sequential, dynamic, and continuous.
 Ex: taxi driving

32
Properties of Task Environments [cont.]
Example properties of task Environments

Several of the answers in the table depend on how the task

environment is defined.