MODULE 01

1.1 WHAT IS AI?

Ar ficial Intelligence (AI) is the study and development of systems that can perform tasks
requiring intelligence. These tasks include reasoning, learning, problem-solving, and
decision-making. AI can be understood in different ways, depending on whether it focuses
on thinking or ac ng and whether the goal is to replicate human behaviour or achieve
logical ra onality.

1. Thinking Humanly: Systems designed to think like humans.

2. Thinking Ra onally: Systems designed to follow logical principles.

3. Ac ng Humanly: Systems that act like humans.

4. Ac ng Ra onally: Systems that act to achieve the best outcomes.

1.1.1 Ac ng Humanly: The Turing Test

The Turing Test, proposed by Alan Turing in 1950, is a prac cal way to define intelligence. A
computer is intelligent if a human interrogator, a er asking ques ons, cannot dis nguish
between responses from the computer and a human.

To pass this test, a computer needs the following capabili es:

1. Natural Language Processing: To understand and reply in human language.

2. Knowledge Representa on: To store and organize knowledge.

3. Automated Reasoning: To answer ques ons and make decisions using stored
knowledge.

4. Machine Learning: To learn from experience and adapt to new situa ons.

The Total Turing Test expands this to include:

 Computer Vision: To perceive and interpret visual data.

  Robo cs: To interact physically with the world.

Although the Turing Test inspired much of AI's early development, researchers today focus
more on the principles of intelligence than on passing the test.

1.1.2 Thinking Humanly: Cogni ve Modeling

To design machines that think like humans, we need to understand how humans think. This
involves:

1. Introspec on: Observing your own thought processes.

2. Psychological Experiments: Studying human behaviour in controlled condi ons.

3. Brain Imaging: Analysing how the brain works during specific tasks.

Once we understand human cogni on, we can build computer models to replicate it. For
example, the General Problem Solver (GPS) program developed by Allen Newell and
Herbert Simon simulated human problem-solving by tracing reasoning steps similar to
humans.

This approach is part of cogni ve science, which combines AI with psychology to understand
the mind.

1.1.3 Thinking Ra onally: The "Laws of Thought"

The laws of thought approach, origina ng with Aristotle, focuses on crea ng systems that
follow logical reasoning. Aristotle's syllogisms (e.g., "All men are mortal; Socrates is a man;
therefore, Socrates is mortal") represent early a empts at formal logic.

In the 19th century, logic was expanded to represent complex rela onships, and by the mid-
20th century, AI programs could solve problems expressed in logical terms. This method,
called the logicist tradi on, uses formal logic to build intelligent systems.

Challenges:

1. Transla ng informal, uncertain knowledge into strict logical rules is difficult.

 2. Solving problems logically can be computa onally expensive and imprac cal for large
datasets.

Despite these challenges, logical reasoning remains a key tool for developing intelligent
systems.

1.1.4 Ac ng Ra onally: Ra onal Agents

A ra onal agent is a system that acts to achieve the best outcome or maximize its goals
based on its knowledge. Unlike logic-based systems, ra onal agents can act intelligently even
when perfect solu ons are impossible.

Key Features of Ra onal Agents:

 Operate autonomously.

 Perceive and interpret their environment.

 Adapt to changes over me.

 Make decisions to achieve goals.

For example, recoiling from a hot stove is a ra onal ac on based on reflex, not logical
reasoning. Ra onal agents use knowledge, reasoning, and learning to act effec vely in
diverse situa ons.

Advantages:

1. Ra onal agents are more flexible than logic-based systems.

2. Ra onality is mathema cally defined and general, allowing for systema c

development.

1.4 THE STATE OF THE ART

AI systems demonstrate remarkable capabili es across various fields.

 1. Robo c Vehicles: AI powers autonomous vehicles like STANLEY, a robo c car that
navigated rough terrain to win the 2005 DARPA Grand Challenge. It u lized cameras,
radar, and onboard so ware for driving decisions.

2. Speech Recogni on: Automated systems enable voice-driven interac ons, such as
booking flights via phone, using advanced speech recogni on and dialog
management.

3. Autonomous Planning and Scheduling: NASA's Remote Agent and successor

programs like MAPGEN manage spacecra opera ons and mission planning,
demonstra ng AI's role in space explora on.

4. Game Playing: IBM's DEEP BLUE defeated chess champion Garry Kasparov in 1997,
showcasing AI's strategic reasoning capabili es.

5. Spam Detec on: Learning algorithms classify billions of emails daily as spam,
adap ng to new spamming tac cs effec vely.

6. Logis cs Planning: Tools like DART op mized logis cs for the U.S. military during the
Persian Gulf crisis, reducing planning me from weeks to hours.

7. Robo cs: AI-driven robots like Roomba vacuum cleaners and PackBot handle
household tasks and hazardous opera ons in conflict zones.

8. Machine Transla on: AI translates languages (e.g., Arabic to English) using sta s cal
models built from large datasets, enabling cross-language communica on without
requiring linguists.

2.1 AGENTS AND ENVIRONMENTS

An agent in Ar ficial Intelligence is any system that can observe its surroundings (using
sensors) and take ac ons to change or interact with the surroundings (using actuators). The
concept applies to humans, robots, and so ware programs.

Examples of Agents:

1. Human Agent:

o Sensors: Eyes (to see), ears (to hear), and other senses like touch and smell.
 o Actuators: Hands, legs, and vocal cords, which allow us to perform ac ons
like wri ng, walking, or speaking.

2. Robo c Agent:

o Sensors: Cameras to "see," infrared sensors to detect objects, and

microphones to "hear."

o Actuators: Motors to move wheels or arms, allowing the robot to perform

tasks like picking up objects or naviga ng.

3. So ware Agent:

o Sensors: Inputs like keystrokes, file data, or network packets (messages sent
over the internet).

o Actuators: Outputs like showing informa on on a screen, saving a file, or

sending an email.

Key Components of an Agent

1. Percept: What the agent senses at a par cular moment.

o Example: A vacuum cleaner agent senses that the square it is on is dirty.

2. Percept Sequence: The complete history of what the agent has sensed so far.

o Example: A sequence like [Square A dirty → cleaned → moved to Square B] is

the percept sequence of the vacuum agent.
 3. Ac on: What the agent does in response to what it has sensed.

o Example: The vacuum cleaner sucks up dirt or moves to a new square.

How Agents Work

 An agent decides its ac ons based on the informa on it has gathered (precepts).

 Its behaviour is governed by an agent func on, which specifies what ac on to take
for every possible percept sequence.

For example, in a vacuum cleaner world:

 If the agent senses dirt, it cleans the square.

 If there is no dirt, it moves to the next square.

Agent Func on vs. Agent Program

1. Agent Func on: A theore cal or mathema cal descrip on of how the agent
responds to every possible situa on (percept sequence).

o Example: A table showing "If the square is dirty, clean it; otherwise, move."

2. Agent Program: The actual implementa on or code inside the agent that makes it act
according to the agent func on.

o Example: The program running inside a robot vacuum cleaner.

Example: Vacuum Cleaner World

 Environment: Two squares, A and B. Each square can be clean or dirty.

 Sensors: The vacuum cleaner senses:

1. Its loca on (A or B).

2. Whether the square is dirty.

 Ac ons:

o Suck: Clean the current square.

o Move Le or Move Right: Navigate to the other square.

 o Do Nothing: Stop if everything is clean.

A Simple Vacuum Agent's Rule

 If the square is dirty, suck.

 If the square is clean, move to the next square.

For example:

1. Start in Square A. If A is dirty, clean it.

2. Move to Square B. If B is dirty, clean it.

3. Stop when both squares are clean.

What Makes a Good Agent?

A good agent is one that performs well in achieving its goals. It should:

 Perceive well: Gather accurate informa on from the environment.

  Decide efficiently: Take the best possible ac on based on the precepts.

 Act effec vely: Perform the ac ons in a way that achieves the goal.

In the vacuum cleaner world, a good agent would clean all the dirt in the shortest amount of
me and with minimal movement.

Why Use the Concept of Agents?

The concept of agents is a tool for understanding and designing intelligent systems. For
example:

 Robots: They need to decide how to navigate and interact with their environment.

 AI programs: They must take ac ons based on data, like classifying spam emails or
recommending movies.

 So ware systems: They handle tasks like managing calendars, answering queries, or
automa ng processes.

2.2 GOOD BEHAVIOR: THE CONCEPT OF RATIONALITY

A ra onal agent is an agent that always does the right thing. "Right thing" means that the
agent’s ac ons lead to desirable outcomes in its environment.

 How do we define the "right thing"?

By looking at the consequences of the agent's ac ons. An agent interacts with its
environment, and the sequence of its ac ons changes the state of the environment.
If the resul ng states are good or desirable, the agent is considered to have behaved
ra onally.

 How do we measure success?

Success is determined by a performance measure. This is a way to evaluate how well
the environment's states match the desired outcomes a er the agent's ac ons.

Key Ideas about Performance Measures

1. The performance measure should be based on the environment's states, not on the
agent’s opinion of its performance.
 o Example: If we let the agent decide its own success, it might deceive itself
into thinking it performed perfectly even when it did not.

2. Performance measures need to be designed carefully.

o Example: For a vacuum-cleaner agent, if the measure only rewards the

amount of dirt cleaned, the agent could cheat by repeatedly cleaning and
dumping the dirt.

o A be er performance measure would be one that rewards having a clean

floor at every me step while penalizing unnecessary energy use or noise.

3. Even well-designed measures can raise deeper ques ons.

o Example: Should we reward consistent but average performance or allow

highs and lows in performance?

o This applies to broader topics like whether a steady life is be er than a risky
one.

2.2.1 Ra onality

Defini on of Ra onality

For an agent to be ra onal, its ac ons should maximize the performance measure based on:

1. The performance measure (what defines success).

2. The agent’s prior knowledge about the environment.

3. The ac ons available to the agent.

4. The agent’s percept sequence (what it has observed so far).

Ra onal Agent Example: The vacuum-cleaner agent

 Assump ons:

o It scores one point for each clean square at every me step during a life me
of 1000 me steps.

o The environment’s layout is known, but the dirt’s ini al loca on is unknown.
 o The agent can move le , right, or suck up dirt.

o The agent can perceive its current loca on and whether there’s dirt.

Under these condi ons, the agent behaves ra onally because:

 It cleans dirty squares.

 It moves to the next square if the current one is clean.

 It maximizes cleanliness within the constraints.

Irra onal Behaviour

If the environment or performance measure changes, the same agent might no longer
behave ra onally.

 Example 1: If all dirt is cleaned and the agent keeps moving unnecessarily, it wastes
energy and loses points.

Solu on: A be er agent would stop moving once it confirms all squares are clean.

 Example 2: If clean squares can become dirty again, the agent should periodically
check and re-clean them.

 Example 3: If the environment’s layout is unknown, the agent needs to explore

instead of s cking to specific squares.

2.2.2 Omniscience, learning, and autonomy

Ra onality vs. Omniscience

 What is omniscience?
Omniscience means knowing the actual outcome of every ac on before it happens.
However, this is impossible in real-world scenarios because no one can predict the
future with certainty.

 Example:
Imagine walking across the Champs Élysées to meet an old friend. There’s no traffic,
so crossing the street seems ra onal. But if a cargo door from an airplane
 unexpectedly falls and hits you, it doesn’t mean you were irra onal—it just means
you couldn’t have foreseen the event.

 Key takeaway:
Ra onality maximizes expected performance based on the informa on the agent
has, not the actual performance a er the fact. Expec ng agents to act perfectly (as if
they had a crystal ball) is unrealis c.

Ra onality and Informa on Gathering

Ra onal agents need to take ac ons to improve their knowledge of the environment. This is
called informa on gathering.

 Example:
When crossing a busy road, an agent that doesn’t look both ways is missing crucial
informa on (like a fast-approaching truck). A ra onal agent would first look, then
decide whether it’s safe to cross.

 Why gather informa on?

To maximize performance, agents must take ac ons to improve their understanding
of the environment before ac ng.

o A vacuum-cleaning agent in an unknown environment must explore to learn

about the layout and dirt loca ons.

Learning

Ra onal agents should not only gather informa on but also learn from their experiences to
improve their behaviour over me.

 Agents with prior knowledge:

An agent might start with some knowledge about the environment (designed by its
creator). However, if the environment changes or this knowledge is incomplete, the
agent must learn to adapt.

 Examples of poor learning:

 1. Dung beetle:
It ins nc vely plugs its nest with dung. If the dung is removed, it s ll pretends
to plug the nest, unaware of its failure. This is an example of behaviour failing
due to reliance on fixed, innate ac ons.

2. Sphex wasp:
It drags a caterpillar to its burrow and checks the burrow before pu ng the
caterpillar inside. If the caterpillar is moved slightly during the check, the
wasp reverts to dragging it again and repeats this endlessly. It cannot learn or
modify its behaviour.

 Why learn?
Fixed behaviours work only in predictable environments. Learning allows agents to
adapt when condi ons change or when their ini al assump ons fail.

Autonomy

 What is autonomy?
An agent is autonomous if it relies on its own observa ons and learning rather than
just the knowledge programmed by its designer.

 Example:
A vacuum-cleaning agent that learns pa erns of dirt accumula on (e.g., dirt appears
more near doorways) performs be er than one that follows a rigid cleaning
schedule.

 Why not full autonomy at the start?

A completely autonomous agent might act randomly in the beginning due to lack of
experience. It’s prac cal to provide some ini al knowledge (like reflexes in animals)
to help the agent survive and learn in its environment.

 Goal of autonomy:
Over me, the agent learns enough from its environment to operate effec vely, even
if the designer's prior knowledge was incomplete or incorrect.
2.3 THE NATURE OF ENVIRONMENTS

Task environment—the "problem" that ra onal agents aim to solve. Understanding the task
environment is essen al for designing ra onal agents.

2.3.1 Specifying the Task Environment (PEAS)

The task environment is the se ng in which an agent operates and must be defined clearly.
The first step in designing any agent is to specify its task environment using the PEAS
framework:

 P: Performance measure

 E: Environment

 A: Actuators

 S: Sensors

This descrip on helps define the problem and the agent’s components.

Example: Automated Taxi Driver

An automated taxi driver is a complex task environment, and its PEAS descrip on looks like
this:

Performance Measure

The agent's goal is to perform a task as well as possible. In the case of the taxi, the
performance measure includes:

 Safe, fast, legal, and comfortable trips

  Maximizing profits

 Minimizing trip me, fuel consump on, and wear and tear

 Minimizing traffic law viola ons

 Providing good customer service (comfort and safety)

Environment

The environment the taxi will operate in consists of:

 Roads (ranging from rural lanes to urban streets to highways)

 Other traffic (cars, pedestrians, stray animals)

 Environmental condi ons (e.g., weather, roadworks, potholes)

 Passengers

This environment is dynamic, so the taxi must adapt to new situa ons.

Actuators

These are the physical components the agent uses to interact with the environment. The
taxi's actuators include:

 Steering wheel (for turning)

 Accelerator and brake (to control speed)

 Signal and horn (for communica on)

 Display screen or voice synthesizer (for communica on with passengers)

Sensors

Sensors allow the agent to perceive the environment. The taxi's sensors include:

 Video cameras (for visual informa on)

 Infrared or sonar sensors (for distance to other cars and obstacles)

 Speedometer (for controlling speed)

 GPS (to know its loca on and avoid ge ng lost)

  Accelerometer (to measure speed changes and handle curves)

 Engine and fuel sensors (for vehicle health)

 Keyboard or microphone (for passenger input)

Other Examples of Agents

In addi on to the taxi, there are other agent types with their own PEAS descrip ons. These
include:

1. So ware Agents (So bots):

These agents operate in ar ficial environments. For example, a so bot might be used
to operate a website, scanning news sources and displaying relevant content to
users. It may need to:

o Process natural language

o Learn what each user is interested in

o Change plans dynamically (e.g., if a news source goes down)

The environment here is the internet, which is as complex as the physical
world. The so bot needs to handle various unexpected situa ons (such as a
server failure) and interact with both human and other ar ficial agents.

2. So ware robots on conveyor belts:

This agent operates in a simpler environment, where it inspects parts on a conveyor
belt. The agent assumes condi ons (e.g., fixed ligh ng, known parts) and has fewer
ac ons (accept or reject parts). The task environment is less complex, making it
easier to design.
Real vs. Ar ficial Environments

 Real Environments: Environments in the physical world are complex and

unpredictable. For example, driving on roads with varying condi ons, traffic, and
weather.

 Ar ficial Environments: So ware agents may operate in environments like websites

or virtual systems, where complexity arises from mul ple agents (human or
ar ficial), dynamic changes, and large amounts of informa on.

2.3.2 Proper es of task environments

1. Task Environment Dimensions

Task environments can be categorized along these dimensions:

a. Fully Observable vs. Par ally Observable

  Fully Observable: The agent's sensors provide complete informa on about the
environment at every point in me.

o Example: Chess, where the en re board is visible.

 Par ally Observable: The agent’s sensors are limited or noisy, so it cannot get full
informa on about the environment.

o Example: Taxi driving, where the agent cannot know the inten ons of other
drivers.

b. Single Agent vs. Mul agent

 Single Agent: The environment has only one agent ac ng, like solving a crossword
puzzle.

 Mul agent: Mul ple agents interact, and their ac ons can affect one another.

o Compe ve Mul agent Environment: Agents compete, such as in chess.

o Coopera ve Mul agent Environment: Agents collaborate, such as avoiding

collisions in traffic.

c. Determinis c vs. Stochas c

 Determinis c: The next state of the environment is fully predictable based on the
current state and agent's ac on.

o Example: Chess (no randomness).

 Stochas c: Outcomes are uncertain due to randomness or unpredictability.

o Example: Taxi driving (other vehicles' behavior is uncertain).

d. Episodic vs. Sequen al

 Episodic: Each decision is independent of previous decisions.

o Example: Spo ng defec ve parts on an assembly line.

 Sequen al: Each decision affects future decisions.

o Example: Chess or taxi driving.

e. Sta c vs. Dynamic

 Sta c: The environment doesn’t change while the agent is making decisions.

o Example: Crossword puzzles.

 Dynamic: The environment changes during the agent's decision-making process.

o Example: Taxi driving (traffic condi ons keep changing).

 Semi dynamic: The environment is sta c, but the performance score may change
over me, as seen in med chess games.

f. Discrete vs. Con nuous

 Discrete: The environment has a limited set of dis nct states, ac ons, or me steps.

o Example: Chess (finite board posi ons).

 Con nuous: States, me, or ac ons vary con nuously.

o Example: Taxi driving (speed and posi on vary con nuously).

g. Known vs. Unknown

 Known: The agent (or designer) knows the rules or "laws" governing the
environment.

o Example: Chess, where rules are fully known.

 Unknown: The agent must learn how the environment works.

o Example: Playing a new video game.

2. Task Environment Examples

The text provides several examples of task environments with their characteris cs:

Task
Observable Agents Determinis c Episodic Sta c Discrete/Con nuous
Environment

Crossword
Fully Single Determinis c Sequen al Sta c Discrete
Puzzle

Chess with a
Fully Mul Determinis c Sequen al Semi Discrete
Clock

Poker Par ally Mul Stochas c Sequen al Sta c Discrete

Taxi Driving Par ally Mul Stochas c Sequen al Dynamic Con nuous

Medical
Par ally Single Stochas c Sequen al Dynamic Con nuous
Diagnosis

Image Analysis Fully Single Determinis c Episodic Semi Con nuous

Part-Picking
Par ally Single Stochas c Episodic Dynamic Con nuous
Robot

Insights:

 The task environment type impacts how the agent operates.

 For instance, an agent in a dynamic, stochas c, mul agent environment (like taxi
driving) must deal with uncertainty, plan for unpredictable changes, and coordinate
ac ons with others.

3. Complexity of Task Environments

 The hardest environments are par ally observable, mul agent, stochas c,
sequen al, dynamic, con nuous, and unknown.
 o Example: Driving a rented car in a new country with unfamiliar geography and
traffic laws.

4. Evalua ng Agents in Task Environments

 Environment Class: Instead of tes ng an agent in a single scenario, it is evaluated

across a variety of environments (e.g., different traffic or weather condi ons for a
taxi-driving agent).

 Environment Generators: Simulate different scenarios within the environment class.

o Example: A vacuum-cleaner agent might be tested with random dirt pa erns

and star ng loca ons.

 The goal of the agent is to maximize its average performance over all environments
in the class.

2.4 THE STRUCTURE OF AGENTS

2.4.1 Agent programs

 The agent program is the logic or so ware that decides the agent's ac ons based on
its observa ons (precepts).

 It implements the agent func on, which maps percept sequences to ac ons.

 Agent = Architecture + Program:

o Architecture: The physical system, such as a robot, a PC, or a car with sensors
and actuators, that runs the agent program.

o Program: The logic designed to perform ac ons based on the agent's inputs.

For example:

 A robo c vacuum cleaner with wheels and suc on requires a program to decide
where and when to clean.

 If the program says "Walk," the architecture must have legs to execute it.
How Does an Agent Program Work?

 An agent program takes the current percept from sensors and decides an ac on to
pass to actuators.

 Key dis nc on:

o The agent program uses only the current percept.

o The agent func on considers the en re percept history to make decisions.

 If the agent program needs to remember past percepts, it must include a memory
component.

Types of Agent Programs

There are four main types of agent programs, each suited to different environments:

a. Simple Reflex Agents

b. Model-Based Reflex Agents

c. Goal-Based Agents

d. U lity-Based Agents

TABLE-DRIVEN AGENT

This is the simplest agent type:

 How it works: Uses a lookup table with all possible percept-ac on pairs.

 Why it fails:

1. The table size grows exponen ally with the number of percepts.

2. It’s impossible to store or compute for complex environments like driving.

3. Designers can’t manually create the table for all situa ons.
For example:

 A taxi agent would need a table to handle every possible road, traffic, and weather
combina on, which is imprac cal.

Challenges with Table-Driven Agents

 Memory: No physical system can store the enormous table for real-world
environments.

 Time: Designers can’t build such a massive table manually.

 Learning: Agents can’t learn or adapt from experiences.

 Guidance: Even for simple environments, there’s no easy way to define table entries.

Key Idea in AI

The goal of AI is to create compact programs that approximate ra onal behaviour without
relying on massive tables. For example:

2.4.2 Simple reflex agents

 Defini on: These agents choose ac ons based only on the current percept, ignoring
historical data. Their behaviour is dictated by a set of predefined condi on-ac on
rules, such as "if dirty, then clean."

 Example:
 o A vacuum cleaner agent that decides whether to clean or move based on
whether the current square is dirty.

o A driver braking when they see the brake lights of the car in front.

 Implementa on:

o The SIMPLE-REFLEX-AGENT func on uses:

 Sensors to interpret the current state (INTERPRET-INPUT).

  A rule-matching mechanism to determine the ac on based on
condi on-ac on rules.

o The process involves directly mapping condi ons to ac ons without

considering past percepts.

 Limita ons:

o Fully Observable Environments: Simple reflex agents work well only in

environments where the agent can observe everything it needs to make a
decision.

o Par al Observability: If the environment isn't fully observable, these agents

may fail (e.g., a vacuum agent with no loca on sensor may loop indefinitely).

o Randomiza on as a Solu on: In par ally observable se ngs, randomiza on

can help avoid infinite loops, though this is o en a subop mal fix.

2.4.3 Model-based reflex agents

 Defini on: These agents maintain an internal state to account for unobservable
aspects of the environment. This state is updated using a model of the world that
predicts how the environment evolves and how the agent’s ac ons affect it.

 Structure:

o In addi on to the current percept, the agent uses:

 State informa on updated from past percepts.

 A model of the environment’s dynamics (e.g., knowing that overtaking

cars will move closer over me).

 Condi on-ac on rules to decide the next step.

 Implementa on:

o The MODEL-BASED-REFLEX-AGENT func on involves:

  Upda ng State: Combining the current percept with the previous
state using a model of the world.

 Matching Rules: Finding the appropriate ac on based on the updated

state.

o This allows the agent to handle situa ons where not all necessary informa on
is directly observable.

 Example:

o A taxi driver uses an internal model to track other vehicles' posi ons, even
when they're temporarily out of sight.

o It can also consider goals (e.g., deciding when to refuel) as part of its internal
state.
  Advantages:

o Can handle par al observability more effec vely.

o Makes educated guesses about the current state and plans ac ons
accordingly.

2.4.4 Goal-Based Agents

 These agents consider goals to decide their ac ons. A goal represents a desirable
state the agent strives to achieve (e.g., a taxi reaching its des na on).

 They use a model of the environment to plan ac ons that lead to the goal.

 Example: A taxi at a road junc on needs to decide whether to turn or go straight. The
decision depends on the goal des na on.

Advantages:

 Flexibility: A change in the goal (des na on) doesn’t require reprogramming the
en re agent. Only the goal needs to be updated.

 Adaptability: It can adjust its decisions if environmental condi ons (e.g., rain
affec ng brakes) change.

Comparison with Reflex Agents:

  Reflex agents work based on predefined condi on-ac on rules and do not explicitly
reason about future outcomes.

 Goal-based agents predict future states and choose ac ons accordingly, making
them more adaptable.

2.4.5 U lity-Based Agents

 Goals alone are insufficient for high-quality behavior because they don’t evaluate
the desirability of different ac ons. U lity-based agents address this.

 U lity Func on: Measures the desirability of states, allowing the agent to make
trade-offs between conflic ng goals (e.g., speed vs. safety).

 Expected U lity: In uncertain environments, the agent chooses ac ons that

maximize the average u lity based on probabili es of outcomes.

Benefits:

 Can handle conflic ng goals by specifying trade-offs using a u lity func on.

 Works well in uncertain environments, where outcomes aren’t guaranteed.

Challenges:

 Building u lity-maximizing agents involves significant research in percep on,

reasoning, and learning.
  Computa onal complexity o en makes achieving perfect ra onality imprac cal.

2.4.6 Learning Agents

 These agents improve over me by learning from their experiences, making them
suitable for ini ally unknown environments.

 The structure of a learning agent consists of four components:

1. Performance Element: Selects ac ons based on knowledge.

2. Learning Element: Improves the agent by analysing feedback.

3. Cri c: Evaluates the agent’s ac ons and provides feedback.

4. Problem Generator: Suggests exploratory ac ons for gaining new

experiences.

Process of Learning:

 Agents learn how the world works (“How the world evolves”) and the effects of their
ac ons (“What my ac ons do”).

 Feedback can also involve u lity-based learning, where performance standards

define rewards or penal es (e.g., a taxi agent learns that rough driving reduces ps).

Example:
  A taxi learning to avoid sharp turns by observing other drivers’ reac ons (e.g.,
honking or shou ng). The cri c flags such ac ons as undesirable, and the agent
updates its rules.

Core Idea of Learning:

 Learning modifies the agent’s components to align be er with feedback, improving

performance.

2.4.7 How the components of agent programs work

How the components of agent programs work, focusing on the representa on of the
environment. The discussion categorizes these representa ons into atomic, factored, and
structured, emphasizing their increasing complexity and expressiveness.

1. Atomic Representa on:

o Descrip on: The simplest form of representa on, where each state is treated
as a single indivisible unit, like a "black box."

o Example: In a route-finding problem, each state could be the name of the city
you are currently in.

o Advantages: Simple and efficient for problems like search and game-playing.

o Limita ons: Cannot capture internal details or rela onships within a state.

o Applica ons: Used in algorithms like Hidden Markov Models and Markov
Decision Processes, where states are considered indivisible.
2. Factored Representa on:

o Descrip on: Breaks a state into a fixed set of a ributes or variables, each
with its own value. This enables shared a ributes across states.

o Example: In the driving problem, a ributes might include GPS loca on, fuel
level, and toll money.

o Advantages: More detailed and flexible than atomic representa ons, allowing
for par al informa on and uncertainty.

o Limita ons: S ll lacks the ability to capture rela onships between en es.

o Applica ons: Found in areas like constraint sa sfac on, proposi onal logic,
Bayesian networks, and planning.

3. Structured Representa on:

o Descrip on: Describes states in terms of objects and their rela onships,
making it the most expressive representa on.

o Example: Instead of a simple variable, it could describe a truck reversing into

a driveway and being blocked by a loose cow.

o Advantages: Captures complex interac ons and rela onships, enabling a

more natural representa on of real-world scenarios.

o Limita ons: More computa onally intensive for reasoning and learning.
o Applica ons: Underpins rela onal databases, first-order logic, natural
language understanding, and advanced learning algorithms.