Types of AI Agents

Agents can be grouped into five classes based on their degree of perceived intelligence and capability. All these
agents can improve their performance and generate better action over the time.
These are given below:
1.Simple Reflex Agent 2.Model-based reflex agent 3. Goal-based agents 4. Utility-based agent 5. Learning agent

The Simple reflex agents are the simplest agents. These agents
take decisions on the basis of the current percepts and ignore the
rest of the percept history.

o These agents only succeed in the fully observable environment.

o The Simple reflex agent does not consider any part of percepts
history during their decision and action process.
o The Simple reflex agent works on Condition-action rule, which
means it maps the current state to action. Such as a Room Cleaner
agent, it works only if there is dirt in the room.
o Problems for the simple reflex agent design approach:
o They have very limited intelligence
o They do not have knowledge of non-perceptual parts of the
current state
o Mostly too big to generate and to store.
o Not adaptive to changes in the environment.
2. Model-based reflex agent
o The Model-based agent can work in a partially observable environment, and track the situation.
o A model-based agent has two important factors:
o Model: It is knowledge about "how things happen in the world," so it is called a Model-based agent.
o Internal State: It is a representation of the current state based on percept history.
o These agents have the model, "which is knowledge of the world" and based on the model they perform actions.
o Updating the agent state requires information about:
a. How the world evolves
b. How the agent's action affects the world.
3. Goal-based agents
o The knowledge of the current state environment is not always sufficient to decide for an agent to what to do.
o The agent needs to know its goal which describes desirable situations.
o Goal-based agents expand the capabilities of the model-based agent by having the "goal" information.
o They choose an action, so that they can achieve the goal.
o These agents may have to consider a long sequence of possible actions before deciding whether the goal is achieved or not. Such
considerations of different scenario are called searching and planning, which makes an agent proactive.
4. Utility-based agents
o These agents are similar to the goal-based agent but provide an extra component of utility
measurement which makes them different by providing a measure of success at a given state.
o Utility-based agent act based not only goals but also the best way to achieve the goal.
o The Utility-based agent is useful when there are multiple possible alternatives, and an agent has
to choose in order to perform the best action.
o The utility function maps each state to a real number to check how efficiently each action achieves
the goals.
5. Learning Agents
o A learning agent in AI is the type of agent which can learn from its past experiences, or it has learning capabilities.
o It starts to act with basic knowledge and then able to act and adapt automatically through learning.
o A learning agent has mainly four conceptual components, which are:
a. Learning element: It is responsible for making improvements by learning from environment
b. Critic: Learning element takes feedback from critic which describes that how well the agent is doing with respect
to a fixed performance standard.
c. Performance element: It is responsible for selecting external action
d. Problem generator: This component is responsible for suggesting actions that will lead to new and informative
experiences.
o Hence, learning agents are able to learn, analyze performance, and look for new ways to improve the
performance.
 Agents in Artificial Intelligence
An AI system can be defined as the study of the rational agent and its environment. The agents sense the
environment through sensors and act on their environment through actuators. An AI agent can have mental
properties such as knowledge, belief, intention, etc.

What is an Agent?
An agent can be anything that perceive its environment through sensors and act upon that environment through
actuators. An Agent runs in the cycle of perceiving, thinking, and acting. An agent can be:
o Human-Agent: A human agent has eyes, ears, and other organs which work for sensors and hand, legs,
vocal tract work for actuators.
o Robotic Agent: A robotic agent can have cameras, infrared range finder, NLP for sensors and various
motors for actuators.
o Software Agent: Software agent can have keystrokes, file contents as sensory input and act on those
inputs and display output on the screen.
Hence the world around us is full of agents such as thermostat, cellphone, camera, and even we are also agents.
Before moving forward, we should first know about sensors, effectors, and actuators.
Sensor: Sensor is a device which detects the change in the environment and sends the information to other
electronic devices. An agent observes its environment through sensors.
Actuators: Actuators are the component of machines that converts energy into motion. The actuators are only
responsible for moving and controlling a system. An actuator can be an electric motor, gears, rails, etc.
Effectors: Effectors are the devices which affect the environment. Effectors can be legs, wheels, arms, fingers,
wings, fins, and display screen.
 Intelligent Agents:

An intelligent agent is an autonomous entity which act upon an environment using sensors and
actuators for achieving goals. An intelligent agent may learn from the environment to achieve
their goals. A thermostat is an example of an intelligent agent.
Following are the main four rules for an AI agent:
o Rule 1: An AI agent must have the ability to perceive the environment.
o Rule 2: The observation must be used to make decisions.
o Rule 3: Decision should result in an action.
o Rule 4: The action taken by an AI agent must be a rational action.

Rational Agent:
A rational agent is an agent which has clear preference, models uncertainty, and acts in a way to
maximize its performance measure with all possible actions.
A rational agent is said to perform the right things. AI is about creating rational agents to use for
game theory and decision theory for various real-world scenarios.
For an AI agent, the rational action is most important because in AI reinforcement learning
algorithm, for each best possible action, agent gets the positive reward and for each wrong
action, an agent gets a negative reward.
Note: Rational agents in AI are very similar to intelligent agents.
Rationality:
The rationality of an agent is measured by its performance measure. Rationality can be
judged on the basis of following points:
1 Performance measure which defines the success criterion. 2 Agent prior knowledge of its
environment.
2 Best possible actions that an agent can perform. 4 The sequence of percepts.

Note: Rationality differs from Omniscience because an Omniscient agent knows the actual
outcome of its action and act accordingly, which is not possible in reality.
Structure of an AI Agent
The task of AI is to design an agent program which implements the agent
function. The structure of an intelligent agent is a combination of architecture
and agent program. It can be viewed as:
1. Agent = Architecture + Agent program
Following are the main three terms involved in the structure of an AI agent:
Architecture: Architecture is machinery that an AI agent executes on.
Agent Function: Agent function is used to map a percept to an action.
1. f:P* → A
Agent program: Agent program is an implementation of agent function. An agent program executes
on the physical architecture to produce function f.
PEAS Representation

PEAS is a type of model on which an AI agent works upon. When we define an AI agent or
rational agent, then we can group its properties under PEAS representation model. It is
made up of four words:
o P: Performance measure E: Environment A: Actuators S: Sensors

Here performance measure is the objective for the success of an agent's behavior.

PEAS for self-driving cars:

Let's suppose a self-driving car then PEAS representation will be:

Performance: Safety, time, legal drive, comfort
Environment: Roads, other vehicles, road signs, pedestrian
Actuators: Steering, accelerator, brake, signal, horn
Sensors: Camera, GPS, speedometer, odometer, accelerometer,
sonar.
 Adversarial Search
Adversarial search is a search, where we examine the problem
which arises when we try to plan ahead of the world and other
agents are planning against us.
o In previous topics, we have studied the search strategies which are only
associated with a single agent that aims to find the solution which often
expressed in the form of a sequence of actions.
o But, there might be some situations where more than one agent is searching
for the solution in the same search space, and this situation usually occurs in
game playing.
o The environment with more than one agent is termed as multi-agent
environment, in which each agent is an opponent of other agent and playing
against each other. Each agent needs to consider the action of other agent
and effect of that action on their performance.
o So, Searches in which two or more players with conflicting goals are
trying to explore the same search space for the solution, are called
adversarial searches, often known as Games.
Types of Games in AI:
Deterministic Chance Moves

Perfect information Chess, Checkers, go, Othello Backgammon, monopoly

Imperfect information Battleships, blind, tic-tac-toe Bridge, poker, scrabble, nuclear

war

o Perfect information: A game with the perfect information is that in which agents can look into the
complete board. Agents have all the information about the game, and they can see each other moves also.
Examples are Chess, Checkers, Go, etc.
o Imperfect information: If in a game agents do not have all information about the game and not aware
with what's going on, such type of games are called the game with imperfect information, such as tic-tac-
toe, Battleship, blind, Bridge, etc.
o Deterministic games: Deterministic games are those games which follow a strict pattern and set of
rules for the games, and there is no randomness associated with them. Examples are chess, Checkers,
Go, tic-tac-toe, etc.
o Non-deterministic games: Non-deterministic are those games which have various unpredictable
events and has a factor of chance or luck. This factor of chance or luck is introduced by either dice or cards.
These are random, and each action response is not fixed. Such games are also called as stochastic games.
Example: Backgammon, Monopoly, Poker, etc.
Adversarial Search Formalization of the problem:
A game can be defined as a type of search in AI which can be formalized of the
following elements:
o Initial state: It specifies how the game is set up at the start.

o Player(s): It specifies which player has moved in the state space.

o Action(s): It returns the set of legal moves in state space.

o Result(s, a): It is the transition model, which specifies the result of moves in the state

space.
o Terminal-Test(s): Terminal test is true if the game is over, else it is false at any case.

The state where the game ends is called terminal states.

o Utility(s, p): A utility function gives the final numeric value for a game that ends in

terminal states s for player p. It is also called payoff function. For Chess, the
outcomes are a win, loss, or draw and its payoff values are +1, 0, ½. And for tic-tac-
toe, utility values are +1, -1, and 0.
Game tree:
A game tree is a tree where nodes of the tree are the game states and Edges of the tree are
the moves by players.
Game tree involves initial state, actions function, and result Function.
Mini-Max Algorithm in Artificial Intelligence
o Mini-max algorithm is a recursive or backtracking algorithm which is used in decision-

making and game theory. It provides an optimal move for the player assuming that
opponent is also playing optimally.
o Mini-Max algorithm uses recursion to search through the game-tree.

o Min-Max algorithm is mostly used for game playing in AI. Such as Chess, Checkers,

tic-tac-toe, go, and various tow-players game. This Algorithm computes the minimax
decision for the current state.
o In this algorithm two players play the game, one is called MAX and other is called MIN.

o Both the players fight it as the opponent player gets the minimum benefit while they

get the maximum benefit.

o Both Players of the game are opponent of each other, where MAX will select the

maximized value and MIN will select the minimized value.

o The minimax algorithm performs a depth-first search algorithm for the exploration of

the complete game tree.

o The minimax algorithm proceeds all the way down to the terminal node of the tree,

then backtrack the tree as the recursion.

Working of Min-Max Algorithm:
o The working of the minimax algorithm can be easily described using an

example. Below we have taken an example of game-tree which is

representing the two-player game.
o In this example, there are two players one is called Maximizer and other is

called Minimizer.
o Maximizer will try to get the Maximum possible score, and Minimizer will try to

get the minimum possible score.

o This algorithm applies DFS, so in this game-tree, we have to go all the way

through the leaves to reach the terminal nodes.

o At the terminal node, the terminal values are given so we will compare those

value and backtrack the tree until the initial state occurs. Following are the
main steps involved in solving the two-player game tree:
Working of Min-Max Algorithm:
o The working of the minimax algorithm can be easily described using an

example. Below we have taken an example of game-tree which is

representing the two-player game.
o In this example, there are two players one is called Maximizer and other is

called Minimizer.
o Maximizer will try to get the Maximum possible score, and Minimizer will try to

get the minimum possible score.

o This algorithm applies DFS, so in this game-tree, we have to go all the way

through the leaves to reach the terminal nodes.

o At the terminal node, the terminal values are given so we will compare those

value and backtrack the tree until the initial state occurs. Following are the
main steps involved in solving the two-player game tree:
Properties of Mini-Max algorithm:
o Complete- Min-Max algorithm is Complete. It will definitely find a solution (if exist), in

the finite search tree.

o Optimal- Min-Max algorithm is optimal if both opponents are playing optimally.

o Time complexity- As it performs DFS for the game-tree, so the time complexity of

Min-Max algorithm is O(bm), where b is branching factor of the game-tree, and m is the
maximum depth of the tree.
o Space Complexity- Space complexity of Mini-max algorithm is also similar to DFS

which is O(bm).

Limitation of the minimax Algorithm:

o The main drawback of the minimax algorithm is that it gets really slow for complex games such as Chess, go,
etc. This type of games has a huge branching factor, and the player has lots of choices to decide. This
limitation of the minimax algorithm can be improved from alpha-beta pruning which we have discussed in
the next topic.
 Alpha-Beta Pruning
o Alpha-beta pruning is a modified version of the minimax algorithm. It is an optimization technique for the
minimax algorithm.
o As we have seen in the minimax search algorithm that the number of game states it has to examine are
exponential in depth of the tree. Since we cannot eliminate the exponent, but we can cut it to half. Hence
there is a technique by which without checking each node of the game tree we can compute the correct
minimax decision, and this technique is called pruning.
o This involves two threshold parameter Alpha and beta for future expansion, so it is called alpha-beta pruning.
It is also called as Alpha-Beta Algorithm.
o Alpha-beta pruning can be applied at any depth of a tree, and sometimes it not only prune the tree leaves
but also entire sub-tree.
o The two-parameter can be defined as:
a. Alpha: The best (highest-value) choice we have found so far at any point along the path of Maximizer. The initial
value of alpha is -∞.
b. Beta: The best (lowest-value) choice we have found so far at any point along the path of Minimizer. The initial
value of beta is +∞.
o The Alpha-beta pruning to a standard minimax algorithm returns the same move as the standard algorithm
does, but it removes all the nodes which are not really affecting the final decision but making algorithm slow.
Hence by pruning these nodes, it makes the algorithm fast.

Condition for Alpha-beta pruning:

The main condition which required for alpha-beta pruning is:

1. α>=β
Key points about alpha-beta pruning:

o The Max player will only update the value of alpha.

o The Min player will only update the value of beta.
o While backtracking the tree, the node values will be passed to upper nodes instead of values of alpha and
beta.
o We will only pass the alpha, beta values to the child nodes.
 The Wumpus world
The Wumpus world is a cave which has 4/4 rooms connected with passageways. So there are total 16 rooms
which are connected with each other. We have a knowledge-based agent who will go forward in this
world.

The cave has a room with a beast which is called Wumpus, who eats anyone who enters the room.

The Wumpus can be shot by the agent, but the agent has a single arrow.

In the Wumpus world, there are some Pits rooms which are bottomless, and if agent falls in Pits, then he
will be stuck there forever.

The exciting thing with this cave is that in one room there is a possibility of finding a heap of gold.

So the agent goal is to find the gold and climb out the cave without fallen into Pits or eaten by Wumpus.

The agent will get a reward if he comes out with gold, and he will get a penalty if eaten by Wumpus or falls
in the pit.

Note: Here Wumpus is static and cannot move.

Following is a sample diagram for representing the Wumpus world. It is showing some rooms with Pits, one room
with Wumpus and one agent at (1, 1) square location of the world.

There are also some components which can help the agent to navigate the cave.

These components are given as follows:

a. The rooms adjacent to the Wumpus room are smelly, so that it would have some stench.
b. The room adjacent to PITs has a breeze, so if the agent reaches near to PIT, then he will perceive the breeze.
c. There will be glitter in the room if and only if the room has gold.
d. The Wumpus can be killed by the agent if the agent is facing to it, and Wumpus will emit a horrible scream which
can be heard anywhere in the cave.
PEAS description of Wumpus world:

To explain the Wumpus world we have given PEAS description as below:

Performance measure:

o +1000 reward points if the agent comes out of the cave with the gold.
o -1000 points penalty for being eaten by the Wumpus or falling into the pit.
o -1 for each action, and -10 for using an arrow.
o The game ends if either agent dies or came out of the cave.

Environment:

o A 4*4 grid of rooms.

o The agent initially in room square [1, 1], facing toward the right.
o Location of Wumpus and gold are chosen randomly except the first square [1,1].
o Each square of the cave can be a pit with probability 0.2 except the first square.

Actuators:

o Left turn,
o Right turn
o Move forward
o Grab
o Release
 o Shoot.

Sensors:

o The agent will perceive the stench if he is in the room adjacent to the Wumpus. (Not diagonally).
o The agent will perceive breeze if he is in the room directly adjacent to the Pit.
o The agent will perceive the glitter in the room where the gold is present.
o The agent will perceive the bump if he walks into a wall.
o When the Wumpus is shot, it emits a horrible scream which can be perceived anywhere in the cave.
o These percepts can be represented as five element list, in which we will have different indicators for each
sensor.
o Example if agent perceives stench, breeze, but no glitter, no bump, and no scream then it can be represented
as:
[Stench, Breeze, None, None, None].

The Wumpus world Properties:

o Partially observable: The Wumpus world is partially observable because the agent can only perceive the close
environment such as an adjacent room.
o Deterministic: It is deterministic, as the result and outcome of the world are already known.
o Sequential: The order is important, so it is sequential.
o Static: It is static as Wumpus and Pits are not moving.
o Discrete: The environment is discrete.
o One agent: The environment is a single agent as we have one agent only and Wumpus is not considered as
an agent.