Intelligent Agents

CHAPTER 2
Oliver Schulte
Summer2011
 Outline
2

Agents and environments

Rationality
PEAS (Performance measure, Environment,
Actuators, Sensors)
Environment types
Agent types

Artificial Intelligence a modern approach

 Agents
3

An agent is anything that can be viewed as

perceiving its environment through sensors and
acting upon that environment through actuators

Human agent:
eyes, ears, and other organs for sensors;
hands, legs, mouth, and other body parts for actuators

Robotic agent:
cameras and infrared range finders for sensors
various motors for actuators

Artificial Intelligence a modern approach

 Agents and environments
4

The agent function maps from percept histories to

actions:
[f: P* A]
The agent program runs on the physical architecture to
produce f
agent = architecture + program

Artificial Intelligence a modern approach

 Vacuum-cleaner world
5

Demo:
http://www.ai.sri.com/~oreilly/aima3ejava/aima3ejavademos.h
tml

Percepts: location and contents, e.g., [A,Dirty]

Actions: Left, Right, Suck, NoOp
Agents function look-up table
For many agents this is a very large table

Artificial Intelligence a modern approach

 Rational agents
6

Rationality
Performance measuring success
Agents prior knowledge of environment
Actions that agent can perform
Agents percept sequence to date

Rational Agent: For each possible percept sequence, a

rational agent should select an action that is expected to
maximize its performance measure, given the evidence
provided by the percept sequence and whatever built-in
knowledge the agent has.

Artificial Intelligence a modern approach

 Examples of Rational Choice
7

See File: intro-choice.doc

Artificial Intelligence a modern approach

 Rationality
8

Rational is different from omniscience

Percepts may not supply all relevant information
E.g., in card game, dont know cards of others.

Rational is different from being perfect

Rationality maximizes expected outcome while perfection
maximizes actual outcome.

Artificial Intelligence a modern approach

 Autonomy in Agents

The autonomy of an agent is the extent to which its

behaviour is determined by its own experience,
rather than knowledge of designer.

Extremes
No autonomy ignores environment/data
Complete autonomy must act randomly/no program
Example: baby learning to crawl
Ideal: design agents to have some autonomy
Possibly become more autonomous with experience
 PEAS
10

PEAS: Performance measure, Environment,

Actuators, Sensors

Must first specify the setting for intelligent agent

design

Consider, e.g., the task of designing an automated taxi

driver:
Performance measure: Safe, fast, legal, comfortable trip,
maximize profits
Environment: Roads, other traffic, pedestrians, customers
Actuators: Steering wheel, accelerator, brake, signal, horn
Sensors: Cameras, sonar, speedometer, GPS, odometer,
engine sensors, keyboard
Artificial Intelligence a modern approach
 PEAS
11

Agent: Part-picking robot

Performance measure: Percentage of parts in correct
bins
Environment: Conveyor belt with parts, bins
Actuators: Jointed arm and hand
Sensors: Camera, joint angle sensors

Artificial Intelligence a modern approach

 PEAS
12

Agent: Interactive English tutor

Performance measure: Maximize student's score on
test
Environment: Set of students
Actuators: Screen display (exercises, suggestions,
corrections)
Sensors: Keyboard

Artificial Intelligence a modern approach

 Environment types
13

Fully observable (vs. partially observable)

Deterministic (vs. stochastic)
Episodic (vs. sequential)
Static (vs. dynamic)
Discrete (vs. continuous)
Single agent (vs. multiagent):

Artificial Intelligence a modern approach

 Fully observable (vs. partially observable)
14

Is everything an agent requires to choose its actions

available to it via its sensors? Perfect or Full
information.
If so, the environment is fully accessible
If not, parts of the environment are inaccessible
Agent must make informed guesses about world.
In decision theory: perfect information vs. imperfect
information.
Cross WordPoker Backgammon Taxi driver Part picking robot
Image analys
Fully Partially Partially Partially Fully Fully

Artificial Intelligence a modern approach

 Deterministic (vs. stochastic)
15

Does the change in world state

Depend only on current state and agents action?
Non-deterministic environments
Have aspects beyond the control of the agent
Utility functions have to guess at changes in world

Cross WordPoker Backgammon

Taxi driver Part picking robot
Image analys
Deterministic
Stochastic Stochastic Stochastic Stochastic Deterministic

Artificial Intelligence a modern approach

 Episodic (vs. sequential):
16

Is the choice of current action

Dependent on previous actions?
If not, then the environment is episodic
In non-episodic environments:
Agent has to plan ahead:
Current choice will affect future actions

Cross WordPoker Backgammon Taxi driver Part picking robot

Image analys
Sequential
Sequential Sequential Sequential Episodic Episodic

Artificial Intelligence a modern approach

 Static (vs. dynamic):
17
Static environments dont change
While the agent is deliberating over what to do
Dynamic environments do change
So agent should/could consult the world when choosing actions
Alternatively: anticipate the change during deliberation OR make
decision very fast
Semidynamic: If the environment itself does not change
with the passage of time but the agent's performance
score does.
Cross WordPoker Backgammon Image analys
Taxi driver Part picking robot
Static Static Static Dynamic Dynamic Semi

Another example: off-line route planning vs. on-board navigation system

Artificial Intelligence a modern approach
 Discrete (vs. continuous)
18

A limited number of distinct, clearly defined percepts and

actions vs. a range of values (continuous)

Cross WordPoker Backgammon Image analys

Taxi driver Part picking robot
Discrete Discrete Discrete Conti Conti Conti

Artificial Intelligence a modern approach

 Single agent (vs. multiagent):
19

An agent operating by itself in an environment or there are

many agents working together

Cross WordPoker Backgammon Image analys

Taxi driver Part picking robot
Single Multi Multi Multi Single Single

Artificial Intelligence a modern approach

 Summary.
ObservableDeterministic
Episodic Static Discrete Agents

Cross Word Fully Deterministic

Sequential Static Discrete Single

Poker Fully Stochastic Sequential Static Discrete Multi

Backgammon Partially Stochastic Sequential Static Discrete Multi

Taxi driver Partially Multi

Stochastic SequentialDynamicConti

Partially
Part picking robot Stochastic Episodic DynamicConti Single

Image analysisFully DeterministicEpisodic Semi Conti Single

Artificial Intelligence a modern approach

 Choice under (Un)certainty
21

Fully
Observable
yes

no
Deterministic no

yes

Certainty: Uncertainty
Search

Artificial Intelligence a modern approach

 Agent types
22

Four basic types in order of increasing generality:

Simple reflex agents
Reflex agents with state/model
Goal-based agents
Utility-based agents
All these can be turned into learning agents
http://www.ai.sri.com/~oreilly/aima3ejava/aima3ejava
demos.html

Artificial Intelligence a modern approach

 Simple reflex agents
23

Artificial Intelligence a modern approach

 Simple reflex agents
24

Simple but very limited intelligence.

Action does not depend on percept history, only on current
percept.
Therefore no memory requirements.
Infinite loops
Suppose vacuum cleaner does not observe location. What do you do

given location = clean? Left of A or right on B -> infinite loop.

Fly buzzing around window or light.

Possible Solution: Randomize action.

Thermostat.

Chess openings, endings

Lookup table (not a good idea in general)

35100 entries required for the entire game

Artificial Intelligence a modern approach

 States: Beyond Reflexes
25

Recall the agent function that maps from percept histories

to actions:
[f: P* A]
An agent program can implement an agent function by
maintaining an internal state.
The internal state can contain information about the state
of the external environment.
The state depends on the history of percepts and on the
history of actions taken:
[f: P*, A* S A] where S is the set of states.
If each internal state includes all information relevant to
information making, the state space is Markovian.

Artificial Intelligence a modern approach

 States and Memory: Game Theory
26

If each state includes the information about the

percepts and actions that led to it, the state space has
perfect recall.
Perfect Information = Perfect Recall + Full
Observability + Deterministic Actions.

Artificial Intelligence a modern approach

 Model-based reflex agents
27
Know how world evolves
Overtaking car gets closer from
behind
How agents actions affect the
world
Wheel turned clockwise takes you
right

Model base agents update their

state

Artificial Intelligence a modern approach

 Goal-based agents
28

knowing state and environment? Enough?

Taxi can go left, right, straight

Have a goal
A destination to get to

Uses knowledge about a goal to guide its actions

E.g., Search, planning

Artificial Intelligence a modern approach

 Goal-based agents
29

Reflex agent breaks when it sees brake lights. Goal based agent
reasons
Brake light -> car in front is stopping -> I should stop -> I should use brake

Artificial Intelligence a modern approach

 Utility-based agents
30

Goals are not always enough

Many action sequences get taxi to destination
Consider other things. How fast, how safe..
A utility function maps a state onto a real number
which describes the associated degree of
happiness, goodness, success.
Where does the utility measure come from?
Economics: money.
Biology: number of offspring.
Your life?

Artificial Intelligence a modern approach

 Utility-based agents
31

Artificial Intelligence a modern approach

 Learning agents
32

Performance element is
what was previously the
whole agent
Input sensor
Output action
Learning element
Modifies performance
element.

Artificial Intelligence a modern approach

 Learning agents
33

Critic: how the agent is

doing
Input: checkmate?
Fixed

Problem generator
Tries to solve the problem
differently instead of
optimizing.
Suggests exploring new
actions -> new problems.
Artificial Intelligence a modern approach
 Learning agents(Taxi driver)
34

Performance element
How it currently drives
Taxi driver Makes quick left turn across 3 lanes
Critics observe shocking language by passenger and other drivers
and informs bad action
Learning element tries to modify performance elements for future
Problem generator suggests experiment out something called
Brakes on different Road conditions
Exploration vs. Exploitation
Learning experience can be costly in the short run
shocking language from other drivers
Less tip
Fewer passengers
Artificial Intelligence a modern approach
 The Big Picture: AI for Model-Based Agents
35

Planning
Action Reinforcement
Decision Theory
Learning
Game Theory

Knowledge Learning
Logic Machine Learning
Probability Statistics
Heuristics
Inference

Artificial Intelligence a modern approach

 The Picture for Reflex-Based Agents
36

Action
Reinforcement
Learning
Learning

Studied in AI, Cybernetics, Control Theory, Biology,

Psychology.

Artificial Intelligence a modern approach

 Discussion Question
37

Model-based behaviour has a large overhead.

Our large brains are very expensive from an
evolutionary point of view.
Why would it be worthwhile to base behaviour on a
model rather than hard-code it?
For what types of organisms in what type of
environments?

Artificial Intelligence a modern approach

 Summary
38

Agents can be described by their PEAS.

Environments can be described by several key properties:
64 Environment Types.
A rational agent maximizes the performance measure for
their PEAS.
The performance measure depends on the agent function.
The agent program implements the agent function.
3 main architectures for agent programs.
In this course we will look at some of the common and
useful combinations of environment/agent architecture.

Artificial Intelligence a modern approach