AI- Intelligent Agents

Dr. Azhar Mahmood

Associate Professor
Email: [email protected]
 Outline
• Agents and Environments
• Agent Rationality
• PEAS (Performance measure, Environment,
Actuators, Sensors)
• Environment types
• Agent types
 Agents: Definition
“An agent perceives its environment via
sensors and acts upon that environment through
its actuators”
 Agents
– Agent: entity in a program or environment capable of
generating action.
– An agent uses perception of the environment to make decisions
about actions to take.
– The perception capability is usually called a sensor.
– The actions can depend on the most recent perception or on
the entire history (percept sequence).
• Examples:
– Human agent:
– eyes, ears, and other organs for sensors;
– hands, legs, mouth, and other body parts for actuators
– Robotic agent:
– cameras and infrared range finders for sensors;
– various motors for actuators
 Agent Function
• The agent function is a mathematical function that
maps a sequence of perceptions into action.
[f: P*  A]

• The function is implemented as the agent program. The

program runs on the physical architecture to produce f
agent = architecture + program
• The part of the agent taking an action is called an
actuator.
environment  sensors  agent actuators  environment
Environment
Environment

Sensors

Percept (Observations)

Agent Function
Agent
Actuator

Action
Environment Environment
 E.g., vacuum-cleaner world
Agent / Robot

iRobot Corporation

Founder Rodney Brooks (MIT)

• Percepts: location and
contents, e.g., [A, Dirty]
• Actions: Left, Right, Suck,
NoOp
 Rational Agents
• An agent should strive to "do the right thing", based on
what:
– it can perceive and
– the actions it can perform.
• The right action is the one that will cause the agent to
be most successful

Performance measure: An objective criterion for success of an agent's behavior.

Performance measures self-driving car: time to reach destination (minimize), safety,

predictability of behavior for other agents, reliability, etc.

Performance measure of game-playing agent: win/loss percentage (maximize),

robustness, unpredictability (to “confuse” opponent), etc.

Performance measures of a vacuum-cleaner agent: amount of dirt cleaned up,

amount of time taken, amount of electricity consumed, level of noise generated,
 Definition of Rational Agents

• For each possible percept sequence, a rational agent should

select
– an action that maximizes its performance measure (in expectation)
– given the evidence provided by the percept sequence and whatever
built in knowledge the agent has.

Why “in expectation”?

Captures actions with stochastic / uncertain effects or actions performed in

stochastic environments.

We can then look at the expected value of an action ??

In high-risk settings, we may also want to limit the worst-case behavior.

 Rational Agents
Rationality is distinct from omniscience (“all
knowing”).
We can behave rationally even when faced with incomplete
information.

Agents can perform actions in order to modify future

percepts so as to obtain useful information:
information gathering, exploration.
An agent is autonomous if its behavior is determined
by its own experience (with ability to learn and adapt).

•
 Characterizing a Task Environment

• Must first specify the setting for intelligent

agent design.
• PEAS: Performance measure, Environment, Actuators,
Sensors

• Example: the task of designing a self-driving car

– Performance measure Safe, fast, legal, comfortable trip
– Environment Roads, other traffic, pedestrians
– Actuators Steering wheel, accelerator, brake, signal, horn
– Sensors Cameras, LIDAR (light/radar), speedometer, GPS, odometer
engine sensors, keyboard

–
•
Agent Types and their PEAS

Examples of Agent Types and their PEAS description

 Types- Task Environments

Tic-Tac-Toe Game ??
Poker game ??
 Types- Task Environments
2) Deterministic vs Stochastic
– An environment is deterministic if the next state of the
environment is completely determined by the current state of
the environment and the action of the agent;
– In a stochastic environment, there are multiple, unpredictable
outcomes. (If the environment is deterministic except for the
actions of other agents, then the environment is strategic).
In a fully observable, deterministic environment, the agent need
not to deal with uncertainty.

Note: Uncertainty can also arise because of computational

limitations. E.g., we may be playing an omniscient (“all
knowing”) opponent but we may not be able to compute
his/her moves. 16
 Deterministic vs Stochastic
• Deterministic AI environments are those on which
the outcome can be determined base on a specific
state. In other words, deterministic environments
ignore uncertainty.

• Most real world AI environments are not

deterministic. Instead, they can be classified as
stochastic. Self-driving vehicles are a classic example
of stochastic AI processes.
 Types- Task Environments

3) Episodic vs Sequential
– In an episodic environment, the agent’s experience is divided into
atomic
episodes. Each episode consists of the agent perceiving and then
performing a single action.
– Subsequent episodes do not depend on what actions occurred in
previous
episodes. Choice of action in each episode depends only on the
episode itself.
For example, an agent that has to spot defective parts on an assembly
line bases each decision on the current part, regardless of previous
decisions;
(classifying images)
– In a sequential environment, the agent engages in a series of connected
episodes. Current decision can affect future decisions. (E.g., chess and driving)
 Types- Task Environments
4) Static vs Dynamic (Time Based)
–A static environment does not change while the agent is
thinking. The passage of time as an agent deliberates is irrelevant.

–If the environment can change while an agent is deliberating, then

we say the environment is dynamic for that agent;

–The environment is semi-dynamic if the environment itself does

not change with the passage of time but the agent's performance
score does. Chess, when played with a clock

Taxi driving is clearly dynamic: the other cars and the taxi itself
keep moving while the driving algorithm think about what to do next.
Chess, when played with a clock, is semi-dynamic. Crossword
puzzles are static.
 Types- Task Environments
5) Discrete / Continuous
– If the number of distinct percepts and actions is limited, the
environment is discrete, otherwise it is continuous.

6) Single agent / Multi-agent

– If the environment contains other intelligent agents, the agent
needs to be concerned about strategic, game-theoretic aspects
of the environment (for either cooperative or competitive
agents).

Most engineering environments don’t have multi-agent properties,

whereas most social and economic systems get their complexity from the
interactions of (more or less) rational agents.
Example Tasks and Environment Types

How to make the right decisions? Decision theory

21
Examples of task environments and
their characteristics
Types of Agents
 Agents Structure

• The agent function maps from percept histories to actions

f: P*  Action (Abstract mathematical function)
• The agent program runs (internally, Implement the agent function) on the physical architecture to
produce f
• The architecture makes the percepts from the sensors available to the program, runs the program,
and feeds the program’s action choices to the actuators as they are generated.

agent = architecture + program

The architecture might be just an ordinary PC, or it might

be a robotic car with several onboard computers,
cameras, and other sensors.
our focus

Job of AI is to Design an agent program assuming an architecture that will make the percepts from
the sensors available to the program.
 Other Agents types
• Table-lookup driven Agents

• Reflex based Agents

• Model based Agents

• Goal based Agents

• Utility based Agents

• Learning Agents
 Types of Agent

I) --- Table-lookup driven agents

• Uses a percept sequence / action table in memory to find the next action.
• Implemented as a (large) lookup table.

Drawbacks:
– Huge table (often simply too large)
– Takes a long time to build/learn the table
 Toy example: Vacuum world.
Percepts: robot senses it’s location and “cleanliness.”
So, location and contents, e.g., [A, Dirty], [B, Clean].
With 2 locations, we get 4 different possible sensor inputs.
Actions: Left, Right, Suck, NoOp
 Table lookup
• Action sequence of length K, gives 4^K different possible sequences.
• At least many entries are needed in the table.
• So, even in this very toy world, with K = 20, you need a table with over 4^20
> 10^12 entries.

• In more real-world scenarios, one would have many more different percepts
(eg many more locations), e.g., >=100.
• There will therefore be 100^K different possible sequences of length K. For
K = 20, this would require a table with over 100^20 = 10^40 entries.
Infeasible to even store.

• So, table lookup formulation is mainly of theoretical interest. For practical

agent systems, we need to find much more compact representations.
• For example, logic-based representations, Bayesian net representations,
or neural net style representations, or use a different agent architecture,
• e.g., “ignore the past” --- Reflex agents.
 II) --- Simple Reflex Agents
Agents do not have memory of past world states or percepts.
So, actions depend solely on current percept.
Action becomes a “reflex.”

Uses condition-action rules.

Agent selects actions on the basis of current percept only.

If tail-light of car in front is red, then brake.

 A Simple Reflex Agent in Nature
percepts
(size, motion)

RULES:
(1) If small moving object,
then activate SNAP
(2) If large moving object,
then activate AVOID and inhibit SNAP
ELSE (not moving) then NOOP
needed for
completeness Action: SNAP or AVOID or NOOP
 III) --- Model-based Reflex Agents
• Key difference (wrt simple reflex agents):
– Agents have internal state, which is used to keep
track of past states of the world.
• That depends on the percept history
• Reflecting some of the unobserved aspects
• E.g., driving a car and changing lane
– Agents have the ability to represent change in the
World.
• How the world evolves independently of the agent
• How the agent’s actions affect the world
 Model-based Reflex Agents
Logical Agents
Knowledge Representation and Reasoning:
later in the course

“Infers potentially
dangerous driver
in front.”

If “dangerous driver in front,”

then “keep distance.”
Example Table Agent with Internal State
IF THEN
Saw an object ahead, and Go straight
turned right, and it’s now
clear ahead
Saw an object Ahead, turned Halt
right, and object ahead again

See no objects ahead Go straight

See an object ahead Turn randomly
Example Reflex Agent With Internal State:
Wall-Following

start
Actions: left, right, straight, open-door
Rules:
1. If open(left) & open(right) and open(straight) then
choose randomly between right and left
2. If wall(left) and open(right) and open(straight) then straight
3. If wall(right) and open(left) and open(straight) then straight
4. If wall(right) and open(left) and wall(straight) then left
5. If wall(left) and open(right) and wall(straight) then right
6. If wall(left) and door(right) and wall(straight) then open-door
7. If wall(right) and wall(left) and open(straight) then straight.
8. (Default) Move randomly
 IV) --- Goal-based agents
• Key difference wrt Model-Based Agents:
– In addition to state information, have goal information that
– describes desirable situations to be achieved.

• Agents of this kind take future events into consideration.

• What sequence of actions can I take to achieve certain goals?

• Choose actions so as to (eventually) achieve a (given or computed)

goal.
 problem solving and search!
 Goal-based agents
Problem Solving

Considers
“Clean “future”
kitchen”

Agent keeps track of the world state as well as set of goals it’s trying to achieve: chooses
actions that will (eventually) lead to the goal(s).
More flexible than reflex agents  may involve search and planning
 V) --- Utility-based agents
• When there are multiple possible alternatives, how to decide which one is
best?
• Goals are qualitative: A goal specifies a crude distinction between a happy and
unhappy state, but often need a more general performance measure that describes “degree
of happiness.”

• Utility function U: State  R indicating a measure of success or happiness

when at a given state.
• Important for making tradeoffs: Allows decisions comparing choice between
conflicting goals, and choice between likelihood of success and importance of
goal (if achievement is uncertain).

Use decision theoretic models: e.g., faster vs. safer.

 Utility-based Agents
• Goals alone are not enough to generate high-quality behavior
⁻ E.g. meals in Canteen, good or not ?
• Many action sequences  the goals, some are better and some worse
⁻ If goal means success, then utility means the degree of success (how successful it
is)
• It is said state A has higher utility, If state A is more preferred than others
• Utility is therefore a function that maps a state onto a real number, the degree of
success
• Utility has several advantages:
• When there are conflicting goals,
• Only some of the goals but not all can be achieved
• utility describes the appropriate trade-off
• When there are several goals
• None of them are achieved certainly
• utility provides a way for the decision-making
 Ma
king Utility-based agents
isi on
c
De

Decision theoretic actions:

e.g. faster vs. safer
 VI) --- Learning agents: Adapt and improve
over time
• After an agent is programmed, can it work immediately?
⁻ No, it still need teaching

• In AI,
⁻ Once an agent is done
⁻ We teach it by giving it a set of examples
⁻ Test it by using another set of examples
• We then say the agent learns
⁻ A learning agent
 Learning Agents
• Four conceptual components
⁻ Learning element
• Making improvement
⁻ Performance element
• Selecting external actions
⁻ Critic
• Tells the Learning element how well the agent is doing
with respect to fixed performance standard.
• (Feedback from user or examples, good or not?)
⁻ Problem generator
• Suggest actions that will lead to new and informative
experiences.
 Learning Agents
More complicated when agent needs to learn utility
information: Reinforcement learning (based on action
payoff)

“Quick turn is not safe”

More detail: No quick turn

Deep Learning

Road conditions, etc

Takes percepts
and selects actions

Try out the brakes on

different road surfaces
 Summary: Agent types
(1) Table-driven agents
use a percept sequence/action table in memory to find the next action. They are
implemented by a (large) lookup table.
(2) Simple reflex agents
are based on condition-action rules, implemented with an appropriate production
system. They are stateless devices which do not have memory of past world
states.
(3) Agents with memory - Model-based reflex agents
have internal state, which is used to keep track of past states of the world.
(4) Agents with goals – Goal-based agents
are agents that, in addition to state information, have goal information that
describes desirable situations. Agents of this kind take future events into
consideration.
(5) Utility-based agents
base their decisions on classic axiomatic utility theory in order to act rationally.
(6) Learning agents
they have the ability to improve performance through learning.
 Summary
• An agent perceives and acts in an environment, has an architecture, and is implemented by
an agent program.
• A rational agent always chooses the action which maximizes its expected performance,
given its percept sequence so far.
• An autonomous agent uses its own experience rather than built-in knowledge of the
environment by the designer.
• An agent program maps from percept to action and updates its internal state.
– Reflex agents (simple / model-based) respond immediately to percepts.
– Goal-based agents act in order to achieve their goal(s), possible sequence of steps.
– Utility-based agents maximize their own utility function.
– Learning agents improve their performance through learning.
• Representing knowledge is important for successful agent design.
•
• The most challenging environments are partially observable, stochastic, sequential, dynamic,
and continuous, and contain multiple intelligent agents.

Reading: Chapter 2 -Artificial Intelligence A Modern Approach Third Edition