Artificial Intelligence

INTELLIGENT AGENTS

Nguyễn Ngọc Thảo – Nguyễn Hải Minh

{nnthao, nhminh}@fit.hcmus.edu.vn
Outline
• Agents and environments
• Good behavior: The concept of rationality
• The nature of environments
• The structure of agents

2
Agents and Environments
3
What is Agent?
• AI studies how to make computers do things that people are
better at if they could
• Extend what they do to huge data sets
• Do it fast, in near real-time
• Not make mistakes
• Such systems are called Agents.

4
 What is Agent?
• An agent perceives its environment through sensors and
acts upon that environment through actuators.

What AI should fill

Agents interact with environments through sensors and actuators.

5
Examples of agents

Human agent Robotic agent Software agent

Sensors: eyes, ears, Sensors: cameras, Sensors: keystrokes,

and other organs. infrared range finders, file contents, network
Actuators: hands, etc. packets, etc.
legs, vocal tract, etc. Actuators: levels, Actuators: monitor,
motors, etc. physical disk, routers,
etc.

6
The agent’s behavior
• Percept: the agent’s perceptual inputs at any given instant
• Percept sequence: the complete history of everything the
agent has ever perceived
• An agent’s behavior is described by the agent function that
maps any given percept sequence to an action.
𝒇: 𝓟 → 𝓐
• Agent program: the implementation of the agent function

agent = architecture + program

mathematical practical
7
The Vacuum-cleaner world

A vacuum-cleaner world with just two locations

• Percepts: location and contents, e.g., [A,Dirty]

• Actions: Left, Right, Suck, Do Nothing

8
The Vacuum-cleaner world
Percept sequence Action
[A, Clean] Right
[A, Dirty] Suck
[B, Clean] Left
[B, Dirty] Suck
[A, Clean], [A, Clean] Right
[A, Clean], [A, Dirty] Suck
...
...
Partial tabulation of a [A, Clean], [A, Clean], [A, Clean] Right
simple agent function for
the vacuum-cleaner world [A, Clean], [A, Clean], [A, Dirty] Suck
9
The Vacuum-cleaner world

function REFLEX-VACUUM-AGENT([location,status]) returns an action

if status = Dirty then return Suck
else if location = A then return Right
else if location = B then return Left

The agent program for a simple reflex agent in the two-state vacuum environment.

10
Why do we need agents?
• A tool for analyze systems
• All areas of engineering can be seen as designing artifacts
that interact with the world.
• AI designs artifacts that have significant computational
resources and the task environment requires nontrivial
decision making

11
The concept of rationality
• Rationality
• Omniscience, learning, and autonomy

12
Rational agents
• A rational agent is one that does the right thing.
• Every entry in the table for the agent function is filled out correctly.
• What is “right” thing?
• The actions that cause the agent to be most successful
• We need ways to measure success.

Performance measure

13
Performance measure
• An agent, based on its percepts → generates actions
sequence → environment goes to sequence of states
• If this sequence of states is desirable, then the agent performed well.
• Performance measure evaluates any given sequence of
environment states (remember, not agent states!!!).
• An objective function that decides how the agent does successfully.
E.g., 90%? 30%?

14
Design performance measures
• General rule: Design performance measures according to
What one actually wants in the environment
Not how one thinks the agent should behave

• For example, in vacuum-cleaner world

• The amount of dirt cleaned up in a single eight-hour shift, or
• The floor clean, no matter how the agent behaves
• Which one is better?

15
Rationality
• What is rational at any given time depends on

Performance measure Prior knowledge

Define the criterion of success What the agent knows about the
environment

Percept sequence Actions

The agent’s percept to date What the agent can perform

16
Definition of a rational agent
For each possible percept sequence, a rational agent should select
an action that is expected to maximize its performance measure,
given the evidence provided by the percept sequence and whatever
built-in knowledge the agent has.

• For example, in an exam,

• Maximize marks based on the questions
on the paper and your knowledge

17
The Vacuum-cleaner agent
• Performance measure
• Award one point for each clean square at each time step, over 10000
time steps
• Prior knowledge about the environment
• The geography of the environment (2 squares)
• The effect of the actions
• Actions that can perform
• Left, Right, Suck and Do Nothing
• Percept sequences
• Where is the agent?
• Whether the location contains dirt?
• Under this circumstance, the agent is rational.
18
Omniscience, learning, and autonomy

19
Omniscience vs. Rationality
Omniscience Rationality
• Know the actual outcome of Maximize performance measure
actions in advance given the percepts sequence to
• No other possible outcomes date and prior knowledge
• However, impossible in real
world Rationality is not perfection
• Example?

20
Information gathering
• The agent must not engage in unintelligent activities due to
inadvertency.
• Information gathering – Doing actions in order to modify
future percepts (e.g., exploration)
• This is an important part of rationality.

21
Learning
• A rational agent must learn as much as possible from what it
perceives.
• Its initial configuration may be modified and augmented as it gains
experience.
• There are extreme cases in which the environment is
completely known a priori.

22
Autonomy
• A rational agent should be autonomous – Learn what it can
to compensate for partial or incorrect prior knowledge.
• If an agent just relies on the prior knowledge of its designer rather
than its own percepts, then the agent lacks autonomy.
• E.g., a clock
• No input (percepts)
• Run its own algorithm (prior knowledge)
• No learning, no experience, etc.

23
The Nature of
Environments
• Specifying the task environment
• Properties of task environments

24
The task environment
• Task environments are essentially the “problems” to which
rational agents are the “solutions.”

Environments Rational agents

Problems – Solutions

• They come in a variety of flavors, which directly affects the

appropriate design for the agent program.

25
The task environment
• The task environment includes

• Performance measure
• Environment
PEAS • Agent’s Actuators
• Agent’s Sensors

• It must always be the first step in designing an agent and

should be specified as fully as possible.

26
An example: Automated taxi driver
• Performance measure
• How can we judge the automated driver?
• Which factors are considered?
• getting to the correct destination
• minimizing fuel consumption
• minimizing the trip time and/or cost
• minimizing the violations of traffic laws
• maximizing the safety and comfort
• etc.

27
An example: Automated taxi driver
• Environment
• A variety of roads (rural lane, urban alley, etc.)
• Traffic lights, other vehicles, pedestrians, stray animals, road works,
police cars, puddles, potholes, etc.
• Interaction with the passengers
• Actuators (for outputs)
• Control over the accelerator, steering, gear, shifting and braking
• A display to communicate with the customers
• Sensors (for inputs)
• Controllable cameras for detecting other vehicles, road situations
• GPS (Global Positioning System) to know where the taxi is
• Many more devices are necessary: speedometer, accelerometer, etc.
28
An example: Automated taxi driver

PEAS description of the task environment for an automated taxi.

29
Software agents
• Sometimes, the environment may not be the real world.
• E.g., flight simulator, video games, Internet
• They are all artificial but very complex environments

• Those agents working in these environments are called

software agent (softbots).
• All parts of the agent are software.
30
Agents and their PEAS descriptions

31

Examples of agent types and their PEAS description

Quiz 01: PEAS description
• For each of the following activities, give a PEAS description
of the task environment

Playing a tennis match in Practicing tennis against a wall

a tournament

32
Properties of Task environment

Fully observable Partially observable

Single agent Multiagent
Deterministic Stochastic
Episodic Sequential
Static Dynamic
Discrete Continuous
Known Unknown

• These dimensions determine the appropriate agent design

and the applicability of techniques for agent implementation.

33
Fully Observable vs. Partially observable

• Fully observable: The agent’s sensory gives it access to the

complete state of the environment.
• The agent need not maintain internal state to keep track of the world.
• Partially observable
• Noisy and inaccurate sensors
• Parts of the state are simply missing from the sensor data, e.g., a
vacuum agent with only a local dirt sensor cannot tell whether there
is dirt in other squares
• Unobservable: The agent has no sensors at all

34
Single agent vs. Multiagent
• Single agent: An agent operates by itself in an environment.
• E.g., solving crossword → single-agent, playing chess → two-agent
• Which entities must be viewed as agents?
• Whether B’s behavior is described as maximizing a performance
measure whose value depends on A’s behavior.
• Competitive vs. Cooperative multiagent environment
• E.g., playing chess → competitive, driving on road → cooperative

35
Deterministic vs. Stochastic
• Deterministic: The next state of the environment is
completely determined by the current state and the action
executed by the agent.
• E.g., the vacuum world → deterministic, driving on road → stochastic
• Most real situations are so complex that they must be
treated as stochastic.

36
Episodic vs. Sequential
• Episodic: The agent’s experience is divided into atomic
episodes, in each of which the agent receives a percept and
then performs a single action.
• Quality of action depends just on the episode itself
• Do not need to think ahead
• Sequential: A current decision could affect future decisions.

• E.g., spotting defective parts on an assembly line vs. playing

chess

37
Static vs. Dynamic
• Static: The environment is unchanged while an agent is
deliberating.
• E.g., crossword puzzles → static, taxi driving → dynamic
• Dynamic: The agent is continuous asked what it wants to do
• If it has not decided yet, that counts as deciding to do nothing.

• Semi dynamic: The environment itself does not change with

the passage of time but the agent’s performance score does
• E.g., chess playing with a clock

38
Properties of Task environment
• Discrete vs. continuous
• The distinction applies to the state of the environment, to the way
time is handled, and to the agent’s percepts and actions
• E.g., the chess has a finite number of distinct states, percepts and
actions; while the vehicles’ speeds and locations sweep through a
range of continuous values smoothly over time.
• Known vs. unknown
• Known environment: the outcomes (or outcome probabilities if the
environment is stochastic) for all actions are given.
• Unknown environment: the agent needs to learn how it works to
make good decisions.

39
Environments and their characteristics

Examples of task environments and their characteristics.

40
Properties of Task environment
• The simplest environment: Fully observable, deterministic,
episodic, static, discrete and single-agent.
• Most real situations: Partially observable, stochastic,
sequential, dynamic, continuous and multi-agent.

41
Quiz 02: Task environment
• For each of the following activities, characterize its task
environment in term of properties listed.
• Playing a tennis match in a tournament
• Practicing tennis against a wall

42
The structure of agents
• Agent programs
• Simple reflex agents
• Model-based reflex agents
• Goal-based agents
• Utility-based agents
• Learning agents

43
The agent architecture

agent = architecture + program

• Architecture: some sort of computing device with physical

sensors and actuators that this program will run on.
• Ordinary PC, robotic car with several onboard computers, cameras,
and other sensors, etc.
• The program must be appropriate for the architecture.
• Program: Walk action → Architecture: legs

44
The agent programs
• They take the current percept as input from the sensors and
return an action to the actuators.
• Agent program vs. Agent function
• The agent program takes only the current percept, because nothing
more is available from the environment.
• The agent function gets the entire percept sequence that the agent
must remember.

45
A trivial agent program
• Keep track of the percept sequence and index into a table of
actions to decide what to do.

function TABLE-DRIVEN-AGENT(percept) returns an action

persistent: percepts, a sequence, initially empty
table, a table of actions, indexed by percept sequences,
initially fully specified
append percept to the end of percepts
action ← LOOKUP(percepts, table)
return action

The TABLE-DRIVEN-AGENT program is invoked for each new percept and

returns an action each time. It retains the complete percept sequence in memory.

46
A trivial agent program
• 𝑃 = the set of possible percepts
• 𝑇 = lifetime of the agent
• I.e., the total number of percepts it receives
• The size of the look up table is σ𝑇𝑡= 𝑃 𝑡

• For example, consider playing chess

• 𝑃 = 10, 𝑇 = 150 → A table of at least 10150 entries

• Despite of huge size, look up table does what we want

47
The key challenge of AI
• Write programs that produce rational behavior from a small
amount of code rather than a large amount of table entries
• E.g., calculate square roots – a five-line program of Newton’s
Method vs. a huge lookup tables

https://en.wikipedia.org/wiki/Newton%27s_method

48
Types of agent programs

Utility-
based
Goal-based
agents
agents
Model-
based
Simple
reflex agent
reflex agent

49
Simple reflex agents
• The simplest kind of agent, limited intelligence
• Select actions based on the current percept, ignoring the
rest of the percept history
• The connection from percept to action is represented by
condition-action rules.
IF current percept THEN action
• E.g., IF car-in-front-is-braking THEN initiate-braking.

• Limitations
• Knowledge sometimes cannot be stated explicitly → low applicability
• Work only if the environment is fully observable
50
function SIMPLE-REFLEX-AGENT(percept) returns an action
persistent: rules, a set of condition–action rules
state ← INTERPRET-INPUT(percept)
rule ← RULE-MATCH(state, rules)
action ← rule.ACTION
return action
51
A Simple reflex agent in nature
percepts
(size, motion)

Action: SNAP or AVOID or NOOP

RULES:
(1) If small moving object, then activate SNAP
(2) If large moving object, then activate AVOID and inhibit SNAP
ELSE (not moving) then NOOP

Needed for completeness

52
Model-based reflex agents
• The agent must keep track of an internal state in partially
observable environments.
• It depends on the percept history and reflects some of the
unobserved aspects, e.g., driving a car and changing lane.
• The agent program updates the internal state information as
time goes by by encoding two kinds of knowledge
• How the world evolves independently of the agent model of
• How the agent’s actions affect the world the world

53
IF THEN
Saw an object ahead and turned right, and it’s now clear ahead Go straight
Saw an object ahead and turned right, and object ahead again Halt
See no object ahead Go straight
See an object ahead Turn randomly
Example table agent with internal state
54
Goal-based agents
• Current state of the environment is always not enough
• The agent further needs some sort of goal information that
describes desired situations.
• E.g., at a road junction, the taxi can turn left, turn right, or go straight
on, depending on where the taxi is trying to get to.
• Less efficient but more flexible
• Knowledge supporting the decisions is represented explicitly and can
be modified.

55
Goal-based agents

56
Utility-based agent
• Goals are inadequate to generate high-quality behavior in
most environments.
• Many action sequences can get the goals, some are better, and
some are worse, e.g., go home by taxi or Grab car?
• An agent’s utility function is essentially an internalization of
the performance measure.
• Goal → success, utility → degree of success (how successful it is)
• If state A is more preferred than others, then A has higher utility.

57
Utility-based agent

58
Utility-based agent: Advantages
• When there are conflicting goals
• Only some of which can be achieved, e.g., speed and safety
• The utility function specifies the appropriate tradeoff.
• When there are several goals that the agent can aim for
• None of which can be achieved with certainty
• The utility weights the likelihood of success against the importance of
the goals.
• The rational utility-based agent chooses the action that
maximizes the expected utility of the action outcomes

59
Learning agents
• After an agent is programmed, can it work immediately?
• No, it still need teaching
• Once an agent is done, what can we do next?
• Teach it by giving it a set of examples
• Test it by using another set of examples
• We then say the agent learns → learning agents

60
Learning agents

61
Learning agents
• A learning agent is divided into four conceptual components
1. Learning element → Make improvement
2. Performance element → Select external actions
3. Critic → Tell the Learning element how well the agent is doing with
respect to fixed performance standard. (Feedback from user or
examples, good or not?)
4. Problem generator → Suggest actions leading to new and
informative experiences

Learning in intelligent agents is a process of modification of each

component of the agent to bring the components into closer agreement
with the available feedback information, thereby improving the overall
performance of the agent.
62
Learning agents: An example
• Performance element
• Whatever collection of knowledge and procedures the taxi has for selecting
its driving actions (may be further modified)
• Critic
• Observe the world and pass information to the learning element
• E.g., quick left turn across three lanes of traffic → shocking language used
by other drivers observed → bad action
• Learning element
• Formulate new rules from the experience told by the critic
• E.g., a new rule for the above bad action
• Problem generator
• Identify certain behaviors in need of improvement and suggest experiments
• E.g., try out the brakes on different road surfaces under different conditions
63
Quiz 03: Learning agents
• Give an example of learning rational agent following four
conceptual elements.

64
THE END

65