Intelligent Agents

and
Environment

1
Outline
 Agents and environments
 Good Behavior: Rationality
 PEAS (Performance measure, Environment,
Actuators, Sensors)
 Environment types
 Agent types
 Agents
 An agent is anything that can perceive its
environment through sensors and act upon that
environment through actuators

 Human agent: eyes, ears, and other organs for

sensors; hands, legs, mouth, and other body parts
for actuators

 Robotic agent: camera and microphone for sensors;

various motors for actuators

3
Agents
 Percept: Agent’s perceptual inputs at any
given instance.

 Percept sequence: Complete history of

everything the agent has ever perceived.

 Agent’s choice of action at any given instant

can depend on the entire percept sequence
observed to date.
Agents and environments

 The agent function maps from percept histories to actions:

[f: P*  A]

 The agent program runs on the physical architecture to produce

f

 Agent function is an abstract mathematical description; the

agent program is a concrete implementation, running on the
5
agent architecture.
The vacuum-cleaner world

 Environment: square A and B

 Percepts: [location and content] e.g. [A, Dirty]
 Actions: left, right, suck, and no-op

6
The vacuum-cleaner world

Percept sequence Action

[A,Clean] Right
[A, Dirty] Suck
[B, Clean] Left
[B, Dirty] Suck
[A, Clean],[A, Clean] Right
[A, Clean],[A, Dirty] Suck
… …

Partial tabulation of a simple agent function for the vacuum cleaner world
7
The vacuum-cleaner world

function REFLEX-VACUUM-AGENT ([location, status]) return an action

if status == Dirty then return Suck
else if location == A then return Right
else if location == B then return Left

8
Rational agents
 For each possible percept sequence, a rational
agent should select an action that is expected to
maximize its performance measure,

 given the evidence provided by the percept

sequence, and whatever built-in knowledge the
agent has.
Rationality
 What is rational at any given time depends
on four things:
 The performance measure that defines the
criterion of success.
 The agent’s prior knowledge of the
environment
 The actions that the agent can perform
 The agent’s percept sequence to date
Rational agents
 Agents can perform actions in order to
modify future percepts so as to obtain useful
information (information gathering,
exploration)

 An agent is autonomous if its behavior is

determined by its own experience (with
ability to learn and adapt)

 Right action is the one that cause the

agent to be most successful. Therefore,
we will need some way to measure
success.
Performance measures
 Performance measure: An objective criterion
for success of an agent's behavior

 E.g., performance measure of a vacuum-cleaner

agent could be amount of dirt cleaned up,
amount of time taken, amount of electricity
consumed, amount of noise generated, etc.
Environments
 To design an agent we must
specify its task environment.
 PEAS description of the task
environment:
 Performance
 Environment
 Actuators
 Sensors

13
 PEAS
 Consider, e.g., the task of designing an
automated taxi driver:
 Performance measure: Safe, fast, comfortable, maximize
profits

 Environment: Roads, pedestrians, customers

 Actuators: Steering wheel, accelerator, brake, signal, horn

 Sensors: Cameras, sonar, speedometer, GPS, engine sensors

14
PEAS
 Agent: Medical diagnosis system
 Performance measure: Healthy patient,
minimize costs, lawsuits
 Environment: Patient, hospital, staff
 Actuators: Screen display (questions, tests,
diagnoses, treatments, referrals)
 Sensors: Keyboard (entry of symptoms,
findings, patient's answers)
PEAS
 Agent: Part-picking robot
 Performance measure: Percentage of parts
in correct bins
 Environment: Conveyor belt with parts, bins
 Actuators: Jointed arm and hand
 Sensors: Camera, joint angle sensors
PEAS
 Agent: Interactive English tutor
 Performance measure: Maximize student's
score on test
 Environment: Set of students
 Actuators: Screen display (exercises,
suggestions, corrections)
 Sensors: Keyboard
Environment Types
 The range of task environments is vast in AI.

 Task environments are categorized into

multiple categories/dimesions

 Dimensions determine the appropriate agent

design and its implementation
Environment types
Fully observable vs. partially observable
 An agent's sensors give it access to the
complete state of the environment at each point
in time is called fully observable environment.
 A task environment is effectively fully
observable if the sensors detect all aspects that
are relevant to the choice of action
 Partially observable environment is due to
noise and inaccurate sensors
 Examples include Solitaire is fully observable,
automated taxi driving is partially.
Environment types
Deterministic vs. stochastic
 The next state of the environment is completely

determined by the current state and the action

executed by the agent then the environment is
deterministic.
 If the environment is partially observable then it
is stochastic. E.g environment is complex, hard
to keep track of all the unobserved aspects
 If the environment is deterministic except the
actions of other agents, then the environment is
strategic.
 For example, Vacuum world is deterministic, Taxi

driving is stocastic i.e Tyres blow out or engine seizes

Environment types
Episodic vs. sequential
 The agent's experience is divided into atomic

"episodes" (each episode consists of the agent

perceiving and then performing a single action),
and the choice of action in each episode
depends only on the episode itself.
 E.g agent has to pick the defected parts from

conveyer belt, decision depends on the current part.

 In sequential, current decision could affect all future

decisions.
 Chess and taxi driving are sequential. Short term

action has long term consequences.

Environment types
Static vs. dynamic
 The environment is unchanged while an agent is

deliberating then it is called static, otherwise

dynamic.
 The environment is semidynamic if the

environment itself does not change with the

passage of time but the agent's performance
score does
 Taxi driving is dynamic, the other cars and taxi

itself is moving and the driving algorithm

dithers about what to do next.
 Crossworld puzzle is static.
Environment types
Discrete vs. continuous
 A limited number of distinct, clearly defined
percepts and actions. E.g chess environment has
finite number of states, percepts and actions whereas
Continuous environment is taxi driving i.e its speed,
location etc

Single agent vs. multiagent

 An agent operating by itself in an environment.
E.g agent solving a crossworld puzzle by itself
 Agent playing chess in a two-agent environment is an

example of multiagent.
Environment types

 The simplest environment is

 Fully observable, deterministic, episodic,
static, discrete and single-agent.

 Most real situations are:

 Partially observable, stochastic,
sequential, dynamic, continuous and
multi-agent.

24
Environment types
Chess with Chess without Taxi driving
a clock a clock
Fully observable Yes Yes No
Deterministic Strategic Strategic No
Episodic No No No
Static Semi Yes No
Discrete Yes Yes No
Single agent No No No

 The environment type largely determines the agent design.

Agent functions and programs
 An agent is completely specified by the
agent function mapping percept sequences
to actions
 Aim: find a way to implement the rational
agent function concisely

 agent=architecture+program
Agent Program
Function TABLE-DRIVEN_AGENT(percept) returns an action

static: percepts, a sequence initially empty

table, a table of actions, indexed by percept
sequence

append percept to the end of percepts

action  LOOKUP(percepts, table)
return action
Agent Program
 Drawbacks:
 Huge table
 Take a long time to build the table
 Even with learning, need a long time to learn
the table entries
Agent types
 Four basic kind of agent programs
will be discussed:
 Simple reflex agents
 Model-based reflex agents
 Goal-based agents
 Utility-based agents

 All these can be turned into learning

agents.
1. Simple reflex agents
 Select action on the basis of only the current
percept.
 E.g. the vacuum-agent
 Implemented through condition-action rules
 If dirty then suck
 If car in-front is braking then initiate braking.
Simple reflex agents
The vacuum-cleaner world

function REFLEX-VACUUM-AGENT ([location, status]) return an action

if status == Dirty then return Suck
else if location == A then return Right
else if location == B then return Left

32
Agent types; simple reflex
function SIMPLE-REFLEX-AGENT(percept) returns an action

static: rules, a set of condition-action rules

Generates abstracted
state  INTERPRET-INPUT(percept) description of the current
rule  RULE-MATCH(state, rule) state from the percept
action  RULE-ACTION[rule] Returns the first rule in
return action the set of rules that
matches the given state
description
Will only work if the environment is fully
observable.
For example if car would either brake
continuously and unnecessarily or worse, never
brake at all.
2-Model-based reflex agents
 To tackle partially observable environments.
 Maintain internal state that depends on percept history
 Reflects at least some of the unobserved aspects of the
current state
 Over time update state using world knowledge
 Model of the World

 Agent uses such model is called model based agent.

 Current percept is combined with old internal state to

generate the updated description of the current state.
Model-based reflex agents
Agent types; reflex and state
function REFLEX-AGENT-WITH-STATE(percept) returns an
action

static: rules, a set of condition-action rules

state, a description of the current world state
action, the most recent action.

state  UPDATE-STATE(state, action, percept) Responsible for

rule  RULE-MATCH(state, rule) creating the new
action  RULE-ACTION[rule] internal state
description
return action
Goal-based agents
 The agent needs a goal to know which situations are
desirable.

 Typically investigated in search and planning research.

 Major difference: future is taken into account

Goal-based agents
Utility-based agents
 Certain goals can be reached in different ways.
 Some are better, have a higher utility.
 For example, many action sequences that will get the taxi
to its destination but some are quicker, safer, more
reliable and cheaper than others.

 Utility function maps a (sequence of) state(s) onto a real

number, which describes the associated degree of
happiness.
Utility-based agents
Learning agents
 All previous agent-programs describe methods
for selecting actions.
 Yet it does not explain the origin of these programs.
 Learning mechanisms can be used to perform this task.
 Teach them instead of instructing them.
 Advantage is the robustness of the program toward initially
unknown environments.
Learning agents
Learning agents
 Performance element: selecting actions based on
percepts.
 Corresponds to the previous agent programs

 Learning element: introduce improvements in

performance element.
 Critic provides feedback on agents performance based on fixed performance
standard.

 Problem generator: suggests actions that will lead to new

and informative experiences.
 Suggest experiments