CS 5/7320

Artificial
Intelligence

Intelligent Agents
AIMA Chapter 2

Slides by Michael Hahsler

based on slides by Svetlana Lazepnik
with figures from the AIMA textbook.

This work is licensed under a Creative Commons Image: "Robot at the British Library Science Fiction Exhibition"
Attribution-ShareAlike 4.0 International License.
by BadgerGravling
Outline

PEAS
(Performance
What is an
measure, Environment
intelligent Rationality Agent types
Environment, types
agent?
Actuators,
Sensors)
Outline

PEAS
(Performance
What is an
measure, Environment
intelligent Rationality Agent types
Environment, types
agent?
Actuators,
Sensors)
What is an Agents?
• An agent is anything that can be viewed as perceiving its environment
through sensors and acting upon that environment through actuators.

• Control theory: A closed-loop control system (= feedback control system)

is a set of mechanical or electronic devices that automatically regulate a
process variable to a desired state or set point without human interaction.
The agent is called a controller.
• Softbot: Agent is a software program that runs on a host device.
Agent Function and Agent Program
The agent function maps from the set of all possible percept sequences 𝑃𝑃∗ to the
set of actions 𝐴𝐴 formulated as an abstract mathematical function.

𝒑𝒑

𝒂𝒂 = 𝒇𝒇(𝒑𝒑)
𝑓𝑓 ∶ 𝑃𝑃∗ → 𝐴𝐴
𝒂𝒂

The agent program is a concrete implementation of this function for a given

physical system.

Agent = architecture (hardware) + agent program (implementation of 𝑓𝑓)

• Sensors
• Memory
• Computational power
 Example:
Vacuum-cleaner World
• Percepts:
Location and status,
e.g., [A, Dirty]
• Actions:
Most recent
Left, Right, Suck, NoOp
Percept 𝑝𝑝

Agent function: 𝑓𝑓 ∶ 𝑃𝑃∗ → 𝐴𝐴 Implemented agent program:

Percept Sequence Action function Vacuum-Agent([location, status])

[A, Clean] Right returns an action 𝑎𝑎
[A, Dirty] Suck
… if status = Dirty then return Suck
[A, Clean], [B, Clean] Left else if location = A then return Right
else if location = B then return Left
…
[A, Clean], [B, Clean], [A, Dirty] Suck
…

Problem: This table can become infinitively large!

Outline

PEAS
(Performance
What is an
measure, Environment
intelligent Rationality Agent types
Environment, types
agent?
Actuators,
Sensors)
Rational Agents: What is Good Behavior?
Foundation
• Consequentialism: Evaluate behavior by its consequences.
• Utilitarianism: Maximize happiness and well-being.

Definition of a rational agent:

“For each possible percept sequence, a rational agent should select an
action that maximizes its expected performance measure, given the
evidence provided by the percept sequence and the agent’s built-in
knowledge.”

• Performance measure: An objective criterion for success of an agent's

behavior (often called utility function or reward function).
• Expectation: Outcome averaged over all possible situations that may
arise.

Rule: Pick the action that maximize the expected utility

𝑎𝑎 = argmax𝑎𝑎∈A 𝐸𝐸 𝑈𝑈 𝑎𝑎)
Rational Agents
Rule: Pick the action that maximize the expected utility

𝑎𝑎 = argmax𝑎𝑎∈A 𝐸𝐸 𝑈𝑈 𝑎𝑎)

This means:

• Rationality is an ideal – it implies that no one can build a better agent

• Rationality ≠ Omniscience – rational agents can make mistakes if percepts and

knowledge do not suffice to make a good decision

• Rationality ≠ Perfection – rational agents maximize expected outcomes not actual

outcomes

• It is rational to explore and learn – I.e., use percepts to supplement prior knowledge
and become autonomous

• Rationality is often bounded by available memory, computational power, available

sensors, etc.
 Example:
Vacuum-cleaner World
• Percepts:
Location and status,
e.g., [A, Dirty]
• Actions:
Left, Right, Suck, NoOp
Agent function: Implemented agent program:

Percept Sequence Action function Vacuum-Agent([location, status])

[A, Clean] Right returns an action
[A, Dirty] Suck
… if status = Dirty then return Suck
[A, Clean], [B, Clean] Left else if location = A then return Right
else if location = B then return Left
…

What could be a performance measure?

Is this agent program rational?
Outline

PEAS
(Performance
What is an
measure, Environment
intelligent Rationality Agent types
Environment, types
agent?
Actuators,
Sensors)
Problem Specification: PEAS Performance
measure

Performance
Environment Actuators Sensors
measure

Components and Defines

Defines utility Defines
rules of how actions percepts
and what is available
affect the
rational actions
environment.
Example: Automated Taxi Driver

Performance
Environment Actuators Sensors
measure
• Safe • Roads • Steering • Cameras
• fast • other traffic wheel • sonar
• legal • pedestrians • accelerator • speedometer
• comfortable • customers • brake • GPS
trip • signal • Odometer
• maximize • horn • engine
profits sensors
• keyboard
Example: Spam Filter

Performance
Environment Actuators Sensors
measure
• Accuracy: • A user’s email • Mark as spam • Incoming
Minimizing account • delete messages
false • email server • etc. • other
positives, information
false about user’s
negatives account
Outline

PEAS
(Performance
What is an
measure, Environment
intelligent Rationality Agent types
Environment, types
agent?
Actuators,
Sensors)
Environment Types
Fully observable: The agent's sensors Partially observable: The agent cannot see all
give it access to the complete state of vs. aspects of the environment. E.g., it can’t see
the environment. The agent can “see” through walls
the whole environment.

Stochastic: Changes cannot be determined from

the current state and the action (there is some
Deterministic: Changes in the environment
randomness).
is completely determined by the current vs.
Strategic: The environment is stochastic and
state of the environment and the agent’s
adversarial. It chooses actions strategically to
action.
harm the agent. E.g., a game where the other
player is modeled as part of the environment.

Known: The agent knows the rules of the Unknown: The agent cannot predict the outcome
environment and can predict the
vs. of actions.
outcome of actions.
Environment Types
Static: The environment is not changing
Dynamic: The environment is changing while
while agent is deliberating. vs. the agent is deliberating.
Semidynamic: the environment is static,
but the agent's performance score
depends on how fast it acts.

Discrete: The environment provides a fixed Continuous: Percepts, actions, state variables or
number of distinct percepts, actions, and vs. time are continuous leading to an infinite state,
environment states. Time can also evolve in a percept or action space.
discrete or continuous fashion.

Episodic: Episode = a self-contained

Sequential: Actions now affect the outcomes
sequence of actions. The agent's choice of vs. later. E.g., learning makes problems
action in one episode does not affect the
sequential.
next episodes. The agent does the same task
repeatedly.

Single agent: An agent operating by itself in vs. Multi-agent: Agent cooperate or compete in the
an environment. same environment.
Examples of Different Environments

Word jumble Chess with Scrabble Taxi driving

solver a clock

Observable Fully Fully Partially Partially

Determ. game
Deterministic Stochastic
Deterministic Mechanics Stochastic
+Strategic
+ Strategic*
Episodic? Episodic Episodic Episodic Sequential
Static Static Semidynamic Static Dynamic

Discrete Discrete Discrete Discrete Continuous

Single agent Single Multi* Multi* Multi*

* Can be models as a single agent problem with the other agent(s) in the environment.
Outline

PEAS
(Performance
What is an
measure, Environment
intelligent Rationality Agent types
Environment, types
agent?
Actuators,
Sensors)
 Designing a Rational Agent
Remember the definition of a
rational agent:
𝑓𝑓 “For each possible percept sequence, a
action rational agent should select an action
that maximizes its expected
performance measure, given the
evidence provided by the percept
sequence and the agent’s built-in
knowledge.”

Percept to the
Agent Function agent function
• Assess Note: Everything
𝑓𝑓 outside the agent
performance function can be
measure seen as the
• Remember environment.
percept sequence Action from the
• Built-in knowledge agent function
Hierarchy of Agent Types

Utility-based agents

Goal-based agents

Model-based reflex agents

Simple reflex agents

 Simple Reflex Agent
• Uses only built-in knowledge in the form of rules that select action only based
on the current percept. This is typically very fast!
• The agent does not know about the performance measure! But well-designed
rules can lead to good performance.
• The agent needs no memory and ignores all past percepts.

𝑎𝑎 = 𝑓𝑓(𝑝𝑝)

The interaction is a sequence: 𝑝𝑝0 , 𝑎𝑎0 , 𝑝𝑝1 , 𝑎𝑎1 , 𝑝𝑝2 , 𝑎𝑎2 , … 𝑝𝑝𝑡𝑡 , 𝑎𝑎𝑡𝑡 , …

Example: A simple vacuum cleaner that uses rules based on its current sensor input.
Model-based Reflex Agent
• Maintains a state variable to keeps track of aspects of the environment that
cannot be currently observed. I.e., it has memory and knows how the
environment reacts to actions (called transition function).
• The state is updated using the percept.
• There is now more information for the rules to make better decisions.

𝑠𝑠

𝑠𝑠 ′ = 𝑇𝑇(𝑠𝑠, 𝑎𝑎)

𝑎𝑎 = 𝑓𝑓(𝑝𝑝, 𝑠𝑠)

The interaction is a sequence: 𝑠𝑠0 , 𝑎𝑎0 , 𝑝𝑝1 , 𝑠𝑠1 , 𝑎𝑎1 , 𝑝𝑝2 , 𝑠𝑠2 , 𝑎𝑎2 , 𝑝𝑝3 , … , 𝑝𝑝𝑡𝑡 , 𝑠𝑠𝑡𝑡 , 𝑎𝑎𝑡𝑡 , …

Example: A vacuum cleaner that remembers were it has already cleaned.

Transition Function
• The environment is modeled as a discrete dynamical system.
• Changed in the environment are a sequence of states 𝑠𝑠0 , 𝑠𝑠1 , … 𝑠𝑠𝑇𝑇 , where the
index is the time step.
• Example of a state diagram: switch on

Light is Light is
off on

switch off
• States change because of:
a. System dynamics of the environment.
b. The actions of the agent.

• Both types of changes are represented by the transition function written as

𝑇𝑇: 𝑆𝑆 × 𝐴𝐴 → 𝑆𝑆 or 𝑠𝑠𝑠 = 𝑇𝑇(𝑠𝑠, 𝑎𝑎) 𝑆𝑆 … set of states

𝐴𝐴 … set of available actions
𝑎𝑎 ∈ 𝐴𝐴 … an action
𝑠𝑠 ∈ 𝑆𝑆 … current state
𝑠𝑠 ′ ∈ 𝑆𝑆 … next state
State Representation
States help to keep track of the environment and the agent in the environment. This is
often also called the system state. The representation can be
• Atomic: Just a label for a black box. E.g., A, B
• Factored: A set of attribute values called fluents.
E.g., [location = left, status = clean, temperature = 75 deg. F]

Action causes Variables describing the

transition system state are called
“fluents”

We often construct atomic labels from factored information. E.g.: If the agent’s state is
the coordinate x = 7 and y = 3, then the atomic state label could be the string “(7, 3)”.
With the atomic representation, we can only compare if two labels are the same. With
the factored state representation, we can reason more and calculate the distance
between states!

The set of all possible states is called the state space 𝑺𝑺. This set is typically very large!
Old-school vs. Smart Thermostat

Old-school thermostat Smart thermostat

Percepts States Percepts States
Old-school vs. Smart Thermostat
Change
Set temperatur
temperature e when you
range are too
cold/warm.

Old-school thermostat Smart thermostat

Percepts Percepts States
States
• Temp: deg. F Factored states
• Outside temp. • Estimated
• Weather report time to cool
temperature: • Energy the house
Low, ok, high No states need curtailment • Someone
• Someone walking home?
by • How long till
• Someone changes someone is
temp. coming
• Day & time home?
• … • A/C: on, off
Goal-based Agent
• The agent has the task to reach a defined goal state and is then finished.
• The agent needs to move towards the goal. It can use search algorithms to
plan actions that lead to the goal.
• Performance measure: the cost to reach the goal.

𝑇𝑇

𝑎𝑎 = argmin𝑎𝑎0∈A � 𝑐𝑐𝑡𝑡 � 𝑠𝑠𝑇𝑇 ∈ 𝑆𝑆 𝑔𝑔𝑔𝑔𝑔𝑔𝑔𝑔

𝑡𝑡=0

plan Sum of the cost

of a planed sequence of
actions that leads to a
goal state

The interaction is a sequence: 𝑠𝑠0 , 𝑎𝑎0 , 𝑝𝑝1 , 𝑠𝑠1 , 𝑎𝑎1 , 𝑝𝑝2 , 𝑠𝑠2 , 𝑎𝑎2 , … , 𝑠𝑠 𝑔𝑔𝑔𝑔𝑔𝑔𝑔𝑔
cost
Example: Solving a puzzle. What action gets me closer to the solution?
 Utility-based Agent
• The agent uses a utility function to evaluate the desirability of each possible
states. This is typically expressed as the reward of being in a state 𝑅𝑅(𝑠𝑠).
• Choose actions to stay in desirable states.
• Performance measure: The discounted sum of expected utility over time.
∞

𝑎𝑎 = arg𝑚𝑚𝑚𝑚𝑚𝑚𝑎𝑎0∈A 𝔼𝔼 � 𝛾𝛾 𝑡𝑡 𝑟𝑟𝑡𝑡
𝑡𝑡=0

Utility is the
expected future
discounted reward

Techniques: Markov decision

processes, reinforcement learning

The interaction is a sequence: 𝑠𝑠0 , 𝑎𝑎0 , 𝑝𝑝1 , 𝑠𝑠1 , 𝑎𝑎1 . 𝑝𝑝2 , 𝑠𝑠2 , 𝑎𝑎2 , …
reward
Example: An autonomous Mars rover prefers states where its battery is not critically low.
Agents that Learn

The learning element modifies the agent program (reflex-based, goal-

based, or utility-based) to improve its performance.

How is the agent

currently performing?

Update the agent

Agent
program program

Exploration
Example: Smart Thermostat
Change
temperature
when you are
too
cold/warm.

Smart thermostat
Percepts States
• Temp: deg. F Factored states
• Outside temp. • Estimated
• Weather report time to cool
• Energy the house
curtailment • Someone
• Someone walking home?
by • How long till
• Someone changes someone is
temp. coming
• Day & time home?
• … • A/C: on, off
Example: Modern Vacuum Robot
Features are:
• Control via App
• Cleaning Modes
• Navigation
• Mapping
• Boundary blockers

Source: https://www.techhive.com/article/3269782/best-robot-
vacuum-cleaners.html
PEAS Description of a
Modern Robot Vacuum

Performance
Environment Actuators Sensors
measure
What Type of Intelligent Agent is a
Modern Robot Vacuum?

Does it collect utility over

Utility-based agents time? How would the utility for
each state be defined?
Is it learning?

Goal-based agents Does it have a goal state?

Does it store state information.

Model-based reflex agents How would they be defined
(atomic/factored)?

Does it use simple rules based

Simple reflex agents on the current percepts?

Check what applies

What Type of Intelligent
Agent is this?
PEAS Description of ChatGPT

Performance
Environment Actuators Sensors
measure
How does ChatGPT work?
What Type of Intelligent Agent is
ChatGPT?
Does it collect utility over
Utility-based agents time? How would the utility for
each state be defined?
Is it learning?

Goal-based agents Does it have a goal state?

Does it store state information.

Model-based reflex agents How would they be defined
(atomic/factored)?

Does it use simple rules based

Simple reflex agents
on the current percepts?

Answer the following questions:

Check what applies • Does ChatGPT pass the Touring test?
• Is ChatGPT a rational agent? Why?

We will talk about knowledge-based agents later.

Intelligent Systems as
Sets of Agents:
Self-driving Car

Make sure the passenger has a pleasant drive High-level

Utility-based agents (not too much sudden breaking = utility) planning
It should learn!

Goal-based agents Plan the route to the destination.

Remember where every other car is and

Model-based reflex agents calculate where they will be in the next few
seconds.
React to unforeseen issues like a child Low-level
Simple reflex agents running in front of the car quickly.
planning
Some Environment Types Revisited
Fully observable: The agent’s sensors Partially observable: The agent only
vs. perceives part of the state and needs to
always show the whole state.
remember or infer the test.

Stochastic:
Deterministic: Percepts are 100% reliable • Percepts are unreliable (noise distribution,
and changes in the environment is vs. sensor failure probability, etc.). This is called a
completely determined by the current state stochastic sensor model.
of the environment and the agent’s action. • The transition function is stochastic leading
to transition probabilities and a Markov
process.

Known: The agent knows the transition Unknown: The needs to learn the transition
vs. function by trying actions.
function.

We will spend the whole semester on discussing algorithms that can deal with
environments that have different combinations of these three properties.
Conclusion

Intelligent agents inspire the research areas of modern AI

Stay within given

Search for a goal Optimize functions constraints
(e.g., navigation). (e.g., utility). (constraint satisfaction problem;
e.g., reach the goal without
running out of power)

Learn a good agent

Deal with uncertainty program from data Sensing
(e.g., current traffic on the (e.g, natural language
road). and improve over time processing, vision)
(machine learning).