DIGITAL NOTES

ON
ARTIFICIAL INTELLIGENCE
[BAD402]

B.E IIIYEAR-IV SEM

(2024-2025)

DEPARTMENT OF ARTIFICIAL INTELLIGENCE & MACHINE LEARNING

RNS INSTITUTE OF TECHNOLOGY

Affiliated to VTU, Recognized by GOK, Approved by AICTE, New Delhi
(NAAC ‘A+ Grade’ Accredited, NBA Accredited (UG - CSE, ECE, ISE, EIE and EEE)
Channasandra, Dr. Vishnuvardhan Road, Bengaluru - 560 098
Ph:(080)28611880,28611881 URL: www.rnsit.ac.in
 Sub Name: ARTIFICIAL INTELLIGENCE Sub Code: BAD402

Table of Contents
Module-1
1. Introduction
I. What is AI? ……………………………………………3-6
II. Foundations of AI ……………………………………..6-11
III. History of AI …………………......................................11- 15
2. Intelligent Agents:
IV. Agents and environment ……………………………….15-17
V. Concept of Rationality …………………………………17-18
VI. The nature of environment …………………………......18-20
VII. The structure of agents …………………………………20-29

Prof. Rashmi B C, Assistant Professor., AI&ML, RNSIT Page- 2

Sub Name: ARTIFICIAL INTELLIGENCE Sub Code: BAD402

Introduction:
Foundation of AI was laid with Boolean theory by a mathematician, Boole & other researchers.
Since the invention of computer in 1943, AI has been of interest to the researchers. They always
aimed to make machines more intelligent than humans.

Intelligence is a property of mind that encompasses many related mental abilities, such as the
capabilities to
 reason
 plan
 solve problems
 think abstractly
 comprehend ideas and language and
 learn

I. What is AI?
John McCarthy in mid-1950’s coined the term Artificial Intelligence‖ which he would define
as the science and engineering of making intelligent machines‖ AI is about teaching the
machines to learn, to act, and think as humans would do.

We can organize AI definition into 4 categories:

 The definitions on top are concerned with thought processes and reasoning, whereas
the ones on the bottom address behaviour.
 The definitions on the left measure success in terms of conformity to human
performance whereas the ones on the right measure against an ideal performance
measure called rationality.
 A system is rational if it does the "right thing," given what it knows.
 Historically, all four approaches to AI have been followed, each by different people
with different methods.
 A human-centred approach must be in part an empirical science, involving observations
and hypotheses about human behaviour.
 A rationalist’s approach involves a combination of mathematics and engineering. The
various groups have both disparaged and helped each other.

Prof. Rashmi B C, Assistant Professor., AI&ML, RNSIT Page- 3

Sub Name: ARTIFICIAL INTELLIGENCE Sub Code: BAD402

Let us look at the four approaches in more detail.

Thinking humanly:
The cognitive modelling approach If we are going to say that a given program thinks like a
human, we must have some way of determining how humans think. We need to get inside the
actual workings of human minds.
There are three ways to do this:
1. through introspection-trying to catch our own thoughts as they go by
2. through psychological experiments observing a person in action; and
3. through brain imaging—observing the brain in action.
Acting humanly: The Turing Test approach
 The Turing Test, proposed by Alan Turing (1950), was designed to provide a satisfactory
operational definition of intelligence.
 A computer passes the test if a human interrogator, after posing some written questions,
cannot tell whether the written responses come from a person or from a computer.
 This test is used to evaluate a computer acting like humanly.
 For current scenarios the computer would need to possess the following capabilities:
 natural language processing to enable it to communicate successfully in English
 knowledge representation to store what it knows or hears; o automated reasoning to use
the stored information to answer questions and to draw new conclusions
 machine learning to adapt to new circumstances and to detect and the patterns.

Prof. Rashmi B C, Assistant Professor., AI&ML, RNSIT Page- 4

Sub Name: ARTIFICIAL INTELLIGENCE Sub Code: BAD402

 Total Turing Test includes a video signal so that the interrogator can test the subject’s
perceptual abilities, as well as the opportunity for the interrogator to pass physical objects
through the hatch.
 To pass the total Turing Test, the computer will need computer vision to perceive objects,
and robotics to manipulate objects and move about.
These six disciplines compose most of AI.
 The Turing test, proposed by Alan Turing (1950)

 Human interrogator is connected to either another human or a machine in another room.

 Interrogator may ask text questions via a computer terminal, gets answers from the
other room.
 If the human interrogator cannot distinguish the human from the machine, the machine
is said to be intelligent.
Thinking rationally:
The “laws of thought” approach Aristotle was one of the first to attempt to codify right thinking
that is, irrefutable reasoning processes. His syllogisms provided patterns for argument
structures that always yielded correct conclusions when given correct premises.
Example:
Socrates is a man;
all men are mortal;
therefore, Socrates is mortal. -- logic
There are two main obstacles to this approach.
1. It is not easy to take informal knowledge and state it in the formal terms required by logical
notation, particularly when the knowledge is less than 100% certain.
2. Second, there is a big difference between solving a problem in principle‖ and solving it in
practice.

Prof. Rashmi B C, Assistant Professor., AI&ML, RNSIT Page- 5

Sub Name: ARTIFICIAL INTELLIGENCE Sub Code: BAD402

Acting rationally:
The rational agent approaches:
 An agent is just something that acts.
 All computer programs do something, but computer agents are expected to do more: operate
autonomously, perceive their environment, persist over a prolonged time period, and adapt to
change, and create and pursue goals.
 A rational agent is one that acts so as to achieve the best outcome or, when there is
uncertainty, the best expected outcome.
 In the laws of thought approach to AI, the emphasis was on correct inferences.
 On the other hand, correct inference is not all of rationality; in some situations, there is no
provably correct thing to do, but something must still be done.
 For example, recoiling from a hot stove is a reflex action that is usually more successful than
a slower action taken after careful deliberation.
II. The Foundations of AI
1. Philosophy (the study of the fundamental nature of knowledge):
 Can formal rules be used to draw valid conclusions?
 How does the mind arise from a physical brain?
 Where does knowledge come from?
 How does knowledge lead to action?
 Aristotle (384–322 B.C.), was the first to formulate a precise set of laws governing the
rational part of the mind. He developed an informal system of syllogisms for proper
reasoning, which in principle allowed one to generate conclusions mechanically, given
initial premises. E.g. all dogs are animals; all animals have four legs; therefore, all dogs
have four legs.
 Thomas Hobbes (1588–1679) proposed that reasoning was like numerical
computation that we add and subtract in our silent thoughts.
 Rene Descartes (1596–1650) gave the first clear discussion of the distinction between
mind and matter and of the problems that arise.
 The empiricism movement, starting with Francis Bacon's (1561— 1626).
 The confirmation theory of Carnap and Carl Hempel (1905-1997) attempted to
analyse the acquisition of knowledge from experience.

Prof. Rashmi B C, Assistant Professor., AI&ML, RNSIT Page- 6

Sub Name: ARTIFICIAL INTELLIGENCE Sub Code: BAD402

 Carnap's book The Logical Structure of the World (1928) defined an explicit
computational procedure for extracting knowledge from elementary experiences. It was
probably the first theory of mind as a computational process.
 The final element in the philosophical picture of the mind is the connection between
knowledge and action. This question is vital to Al because intelligence requires action
as well as reasoning.
2. Mathematics:
 What are the formal rules to draw valid conclusions?
 What can be computed?
 Formal science required a level of mathematical formalization in three fundamental
areas: logic, computation, and probability.
 Logic: George Boole (1815–1864), who worked out the details of propositional, or
Boolean, logic. In 1879, Gottlob Frege (1848–1925) extended Boole’s logic to include
objects and relations, creating the first order logic that is used today.
 First order logic – Contains predicates, quantifiers and variables
 Example:
Philosopher(a) ⇒ Scholar(a)
∀x, effect_carona(x) ⇒ quarantine(x)
∀x, King(x) ^ Greedy (x) ⇒ Evil (x)
 Alfred Tarski (1902–1983) introduced a theory of reference that shows how to relate
the objects in a logic to objects in the real world.
 Logic and Computation: The first nontrivial algorithm is thought to be Euclid’s
algorithm for computing greatest common divisors(GCD).
 Beside logic and computation, the third great contribution of mathematics to AI is the
probability. The Italian Gerolamo Cardanao (1501-1576) first framed the idea of
probability, describing it in terms of the possible outcomes of gambling events.
 Thomas Bayes (1702-1761) proposed a rule for updating probabilities in the light of
new evidence. Baye’s rule underlies most modern approaches to uncertain reasoning in AI
systems.
3. Economics:
 How should we make decisions so as to maximize payoff?
 How should we do this when the payoff may be far in the future?

Prof. Rashmi B C, Assistant Professor., AI&ML, RNSIT Page- 7

Sub Name: ARTIFICIAL INTELLIGENCE Sub Code: BAD402

 The science of economics got its start in 1776, when Scottish philosopher Adam Smith
treat it as a science, using the idea that economies can be thought of as consisting of individual
agents maximizing their own economic well-being.
 Decision theory, which combines probability theory with utility theory, provides a
formal and complete framework for decisions (economic or otherwise) made under
uncertainty that is, in cases where probabilistic descriptions appropriately capture the
decision maker’s environment.
 Von Neumann and Morgenstern’s development of game theory included the
surprising result that, for some games, a rational agent should adopt policies that are
randomized. Unlike decision theory, game theory does not offer an unambiguous
prescription for selecting actions.
 Operations Research, For the most part, economists did not address the third question
listed above, namely, how to make rational decisions when payoffs from actions are not
immediate but instead result from several actions taken in sequence. This topic was
pursued in the field of operations research, which emerged in World War II from efforts
in Britain to optimize radar installations, and later found civilian applications in
complex management decisions. The work of Richard Bellman (1957) formalized a
class of sequential decision problems called Markov decision processes.
 SATISFICING: Work in economics and operations research has contributed much to
our notion of rational agents, yet for many years AI research developed along entirely
separate paths. One reason was the apparent complexity of making rational decisions.
The pioneering AI researcher Herbert Simon (1916–2001) won the Nobel Prize in
economics in 1978 for his early work showing that models based on satisficing making
decisions that are “good enough,” rather than laboriously calculating an optimal
decision—gave a better description of actual human behavior (Simon, 1947). Since the
1990s, there has been a resurgence of interest in decision-theoretic techniques for agent
systems (Wellman, 1995).

4. Neuroscience:
 How do brain process information?
 Neuroscience is the study of the nervous system, particularly the brain.
 335 B.C. Aristotle wrote, "Of all the animals, man has the largest brain in proportion to
his size."

Prof. Rashmi B C, Assistant Professor., AI&ML, RNSIT Page- 8

Sub Name: ARTIFICIAL INTELLIGENCE Sub Code: BAD402

 Nicolas Rashevsky (1936, 1938) was the first to apply mathematical models to the study
of the nervous system.

 The measurement of intact brain activity began in 1929 with the invention by Hans
Berger of the electroencephalograph (EEG).
 The recent development of functional magnetic resonance imaging (fMRI) (Ogawa et
al., 1990; Cabeza and Nyberg, 2001) is giving neuroscientists unprecedentedly detailed
images of brain activity, enabling measurements that correspond in interesting ways to
ongoing cognitive processes.
5. Psychology:
 How do humans and animals think and act?
 Behaviourism movement, led by John Watson (1878-1958). Behaviourist’s insisted on
studying only objective measures of the percept’s (stimulus) given to an animal and its
resulting actions (or response). Behaviourism discovered a lot about rats and pigeons
but had less success at understanding human.
 Cognitive psychology, views the brain as an information processing device. Common
view among psychologist that a cognitive theory should be like a computer

Prof. Rashmi B C, Assistant Professor., AI&ML, RNSIT Page- 9

Sub Name: ARTIFICIAL INTELLIGENCE Sub Code: BAD402

program.(Anderson 1980) i.e. It should describe a detailed information processing

mechanism whereby some cognitive function might be implemented.
 Craik specified the three key steps of a knowledge-based agent:
o the stimulus must be translated into an internal representation,
o the representation is manipulated by cognitive processes to derive new internal
representations, and
o these are in turn retranslated back into action. He clearly explained why this
was a good design for an agent

6. Computer engineering:
 How can we build an efficient computer?
 For artificial intelligence to succeed, we need two things: intelligence and an artefact.
The computer has been the artefact (object) of choice.
 The first operational computer was the electromechanical Heath Robinson, built in
1940 by Alan Turing's team for a single purpose: deciphering German messages.
 The first operational programmable computer was the Z-3, the invention of KonradZuse
in Germany in 1941.
 The first electronic computer, the ABC, was assembled by John Atanasoff and his
student Clifford Berry between 1940 and 1942 at Iowa State University.
 The first programmable machine was a loom, devised in 1805 by Joseph Marie
Jacquard (1752-1834) that used punched cards to store instructions for the pattern to be
woven.
7. Control theory and cybernetics:
 How can artifacts operate under their own control?
 Ktesibios of Alexandria (c. 250 B.C.) built the first self-controlling machine: a water
clock with a regulator that maintained a constant flow rate. This invention changed the
definition of what an artifact could do.
 Modern control theory, especially the branch known as stochastic optimal control, has
as its goal the design of systems that maximize an objective function over time. This
roughly OBJECTIVE FUNCTION matches our view of Al: designing systems that
behave optimally.
 Calculus and matrix algebra- the tools of control theory The tools of logical inference
and computation allowed AI researchers to consider problems such as language, vision,
and planning that fell completely outside the control theorist’s purview.

Prof. Rashmi B C, Assistant Professor., AI&ML, RNSIT Page- 10

Sub Name: ARTIFICIAL INTELLIGENCE Sub Code: BAD402

8. Linguistics:
 How does language relate to thought?
 In 1957, B. F. Skinner published Verbal Behaviour. This was a comprehensive, detailed
account of the behaviourist approach to language learning, written by the foremost
expert in the field.
 Noam Chomsky, who had just published a book on his own theory, Syntactic
Structures. Chomsky pointed out that the behaviourist theory did not address the notion
of creativity in language.
 Modern linguistics and AI were born at about the same time, and grew up together,
intersecting in a hybrid field called computational linguistics or natural language
processing.
 The problem of understanding language soon turned out to be considerably more
complex than it seemed in 1957. Understanding language requires an understanding of
the subject matter and context, not just an understanding of the structure of sentences.
 Knowledge representation (the study of how to put knowledge into a form that a
computer can reason with)- tied to language and informed by research in linguistics.

III. The History of AI

Important research that laid the groundwork for AI:

 The gestation of artificial intelligence (1943–1955)

1943: Warren McCulloch and Walter Pitts: a model of artificial Boolean circuit
model of neurons to perform computations.
 First steps towards connectionist computation and learning (Hebbian learning)
 Marvin Minsky and Dann Edmonds (1951) constructed the first neural network
computer.
1949: Donald Hebb: Hebbaian learning rule.
1950: Alan Turing’s “Computing Machinery and Intelligence”
 First complete vision of AI.
1950: Marvin Minsky SNARC (first neural computer)

 The birth of artificial intelligence (1956)

Dartmouth Workshop bringing together top minds on automata theory, neural nets
and the study of intelligence.

Prof. Rashmi B C, Assistant Professor., AI&ML, RNSIT Page- 11

Sub Name: ARTIFICIAL INTELLIGENCE Sub Code: BAD402

 Allen Newell and Herbert Simon: The logic theorist (first nonnumeric thinking
program used for theorem proving)
 For the next 20 years the field was dominated by these participants.

 Early enthusiasm, great expectations (1952–1969)

1952: Newell and Simon-Logic Theorist (LT)
 Introduced the General Problem Solver (GPS).
 Imitation of human problem solving.
1952: Arthur Samuel investigated game playing (checkers) with great success.
1958: John McCarthy
 Inventor of Lisp (second-older-high-level language)
 Logic oriented, Advice Taker (separation between knowledge and
reasoning)
1958: Marvin Minsky
 Introduction of microworlds that appear to require intelligence to solve: e.g.
blocks world.
 Anti-logic orientation. society of the mind.
1959: David Gelernter: Geometry Theorem Prover
1965: J A Robinson: Resolution, a complete algorithm for logical reasoning.
1970: Patrick Winston: Blocks World Learning Theory.
1972: Terry Winograd: SHRDLU
1975: David Waltz, Vision & Constraint propagation.
The most famous microworld was the blocks world, which consists of a set of solid blocks
placed on a tabletop (or more often, a simulation of a tabletop), as shown in Figure 1.4.

Prof. Rashmi B C, Assistant Professor., AI&ML, RNSIT Page- 12

Sub Name: ARTIFICIAL INTELLIGENCE Sub Code: BAD402

 A dose of reality (1966–1973)

1966-73: Minsky and Papert Perceptrons
 AI systems do not scale up well
 Neural network research almost disappears.
 Automatic translation fails.
 Progress as slower than expected (Unrealistic predictions)
 Some systems lacked scalability (Combinatorial explosion in search)
 Fundamental limitations on techniques and representations.
 Knowledge-based systems: The key to power? (1969–1979)
1969: B. Buchanan:General-purpose vs. Domian specific
 Dendral (expert system to infer the molecular structure from MS data)-First
successful knowledge intensive system.
 Ed Feigenbaum et al. Heuristic Programming Project (Stanford U.)
1975: Shortlifee, Feigenbaum, Buchanan: Expert systems
 MYCIN (expert system to diagnose blood infections)- Introduction of
uncertainty in reasoning.
1976: Newell & Simon: Physical Symbol Systems Hypothesis.
Increase in knowledge representation research
 Logic, frames, semantic nets, …
 AI becomes an industry (1980–present)

Prof. Rashmi B C, Assistant Professor., AI&ML, RNSIT Page- 13

Sub Name: ARTIFICIAL INTELLIGENCE Sub Code: BAD402

1981: Fifth generation project in Japan.

1982: McDermott
 Digital Equipment, R1, expert system to configure VAX computers
 Teknowledge (expert system company)
 Intellicorp (expert system company)
 The return of neural networks (1986–present)
1982: John Hopfield, Hopfield networks.
1985: Rumelhart, Hinton, Williams: Backpropagation.
1986: Rumelhart, McClelland: PDP books.
Puts an end to the AI winter.
Connectionist revival(1986-present): (Return of Neural Network):
 Parallel distributed processing (RumelHart and McClelland, 1986):
Backpropogation.

 AI adopts the scientific method (1987–present)

Scientific method: hypothesis-rigorous empirical experiment-results.
Neats vs. Scruffies
Probabilistic framework:
 In speech recognition: Hidden Markov Models (HMMs)
 In uncertain reasoning and expert systems: Bayesian Network Formalism
In neural networks
 The emergence of intelligent agents (1995–present)
SOAR, complete agent architecture (Newell, Laird, Rosenbloom)
The whole agent problem:
 “How does an agent act/behave embedded in real environments with continuous
sensory inputs”.
 The availability of very large data sets (2001–present)
2002: For the first time, AI entered the home in the form of Roomba, a vacuum
cleaner.
2006: AI came in the Business world till the year 2006. Companies like Facebook,
Twitter, and Netflix also started using AI.
2011: In the year 2011, IBM's Watson won jeopardy, a quiz show, where it had to
solve the complex questions as well as riddles. Watson had proved that it could
understand natural language and can solve tricky questions quickly.

Prof. Rashmi B C, Assistant Professor., AI&ML, RNSIT Page- 14

Sub Name: ARTIFICIAL INTELLIGENCE Sub Code: BAD402

2012: Google has launched an Android app feature "Google now", which was able
to provide information to the user as a prediction.
2014: In the year 2014, Chatbot "Eugene Goostman" won a competition in the
infamous "Turing test."
2018: The "Project Debater" from IBM debated on complex topics with two master
debaters and also performed extremely well.
2019: Google has demonstrated an AI program "Duplex" which was a virtual
assistant and which had taken hairdresser appointment on call, and lady on other
side didn't notice that she was talking with the machine.
2020-Present: The AI boom started with the initial development of key
architectures and algorithms such as the transformer architecture in 2017, leading
to the scaling and development of large language models exhibiting human-like
traits of reasoning, cognition, attention and creativity. The AI era has been said to
have begun around 2022-2023, with the public release of scaled large language
models such as ChatGPT.
Now AI has developed to a remarkable level. The concept of Deep learning, big data, and data
science are now trending like a boom. Nowadays companies like Google, Facebook, IBM, and
Amazon are working with AI and creating amazing devices. The future of Artificial
Intelligence is inspiring and will come with high intelligence.

Intelligent Agents:

An Intelligent Agent perceives it environment via sensors and acts rationally upon that
environment with its effectors (actuators). Hence, an agent gets percept’s one at a time, and
maps this percept sequence to actions.
IV. Agents and environment

Agent: An Agent is anything that can be viewed as perceiving its environment through sensors
and acting upon that environment through actuators.

 A human agent has eyes, ears, and other organs for sensors and hands, legs, mouth, and
other body parts for actuators.
 A robotic agent might have cameras and infrared range finders for sensors and various
motors for actuators.

Prof. Rashmi B C, Assistant Professor., AI&ML, RNSIT Page- 15

Sub Name: ARTIFICIAL INTELLIGENCE Sub Code: BAD402

 A software agent receives keystrokes, file contents, and network packets as sensory
inputs and acts on the environment by displaying on the screen, writing files, and
sending network packets.

This simple idea is illustrated in Figure 2.1.

Percept: We use the term percept to refer to the agent's perceptual inputs at any given instant.
Percept Sequence: An agent's percept sequence is the complete history of everything the agent
has ever perceived.
Agent function: Mathematically speaking, we say that an agent's behaviour is described by
the agent function that maps any given percept sequence to an action.
Agent program: Internally, the agent function for an artificial agent will be implemented by
an agent program. It is important to keep these two ideas distinct. The agent function is an
abstract mathematical description; the agent program is a concrete implementation, running on
the agent architecture.
To illustrate these ideas, we will use a very simple example-the vacuum-cleaner world shown
in Fig 2.2. This particular world has just two locations: squares A and B. The vacuum agent
perceives which square it is in and whether there is dirt in the square. It can choose to move
left, move right, suck up the dirt, or do nothing. One very simple agent function is the
following: if the current square is dirty, then suck, otherwise move to the other square. A partial
tabulation of this agent function is shown in Fig 2.3.

Prof. Rashmi B C, Assistant Professor., AI&ML, RNSIT Page- 16

Sub Name: ARTIFICIAL INTELLIGENCE Sub Code: BAD402

V. Concept of Rationality
A Rational agent is one that does the right thing. we say that the right action is the one that will
cause the agent to be most successful. That leaves us with the problem of deciding how and
when to evaluate the agent's success.
We use the term performance measure for the how—the criteria that determine how successful
an agent is.
✓ Ex-Agent cleaning the dirty floor

✓ Performance Measure-Amount of dirt collected

✓ When to measure-Weekly for better results
What is rational at any given time depends on four things:
• The performance measure defining the criterion of success
• The agent’s prior knowledge of the environment
• The actions that the agent can perform
• The agent’s percept sequence up to now.

Prof. Rashmi B C, Assistant Professor., AI&ML, RNSIT Page- 17

Sub Name: ARTIFICIAL INTELLIGENCE Sub Code: BAD402

Omniscience, Learning and Autonomy:

➢ We need to distinguish between rationality and omniscience. An Omniscient agent knows
the actual outcome of its actions and can act accordingly but omniscience is impossible in
reality.
➢ Rational agent not only gathers information but also learns as much as possible from what
it perceives.
➢ If an agent just relies on the prior knowledge of its designer rather than its own percepts
then the agent lacks autonomy.
➢ A system is autonomous to the extent that its behaviour is determined its own experience.

➢ A rational agent should be autonomous.

E.g., a clock (lacks autonomy)
➢ No input (percept’s)
➢ Run only but its own algorithm (prior knowledge)

➢ No learning, no experience, etc.

VI. The nature of environment

Specifying the task environment:
Environments: The Performance measure, the environment and the agent’s actuators and
sensors come under the heading task environment. We also call this as PEAS (Performance,
Environment, Actuators, Sensors)
Figure 2.4 summarizes the PEAS description for the taxi’s task environment

Prof. Rashmi B C, Assistant Professor., AI&ML, RNSIT Page- 18

Sub Name: ARTIFICIAL INTELLIGENCE Sub Code: BAD402

Properties of task environments:

The range of task environments that might arise in AI is obviously vast. We can, however,
identify a fairly small number of dimensions along which task environments can be
categorized. These dimensions determine, to a large extent, the appropriate agent design and
the applicability of each of the principal families of techniques for agent implementation. First,
we list the dimensions, then we analyse several task environments to illustrate the ideas.

Environment-Types:
1. Accessible vs. inaccessible or Fully observable vs Partially Observable: If an agent
sensor can sense or access the complete state of an environment at each point of time then it is
a fully observable environment, else it is partially observable.
2. Deterministic vs. Stochastic: If the next state of the environment is completely determined
by the current state and the actions selected by the agents, then we say the environment is
deterministic
3. Episodic vs. nonepisodic:

Prof. Rashmi B C, Assistant Professor., AI&ML, RNSIT Page- 19

Sub Name: ARTIFICIAL INTELLIGENCE Sub Code: BAD402

➢ The agent's experience is divided into "episodes." Each episode consists of the agent
perceiving and then acting. The quality of its action depends just on the episode itself, because
subsequent episodes do not depend on what actions occur in previous episodes.
➢ Episodic environments are much simpler because the agent does not need to think ahead.
4. Static vs. dynamic: If the environment can change while an agent is deliberating, then we
say the environment is dynamic for that agent; otherwise it is static.
5. Discrete vs. continuous: If there are a limited number of distinct, clearly defined percept’s
and actions we say that the environment is discrete. Otherwise, it is continuous.
Figure 2.6 lists the properties of a number of familiar environments.

VII. The structure of agents:

➢ The job of AI is to design the agent program: a function that implements the agent
mapping from percept’s to actions. We assume this program will run on some sort of computing
device, which we will call the architecture.
➢ The architecture might be a plain computer, or it might include special-purpose hardware
for certain tasks, such as processing camera images or filtering audio input. It might also
include software that provides a degree of insulation between the raw computer and the agent
program, so that we can program at a higher level. In general, the architecture makes the
percept’s from the sensors available to the program, runs the program, and feeds the program's
action choices to the effectors as they are generated.
➢ The relationship among agents, architectures, and programs can be summed up as
follows: agent = architecture + program

Prof. Rashmi B C, Assistant Professor., AI&ML, RNSIT Page- 20

Sub Name: ARTIFICIAL INTELLIGENCE Sub Code: BAD402

Agent programs:
➢ Intelligent agents accept percepts from an environment and generates actions. The
early versions of agent programs will have a very simple form (Figure 2.7)
➢ Each will use some internal data structures that will be updated as new percepts
arrive.
➢ These data structures are operated on by the agent's decision-making procedures to
generate an action choice, which is then passed to the architecture to be executed

Types of agents: Agents can be grouped into four classes based on their degree of perceived
intelligence and capability:
➢ Simple Reflex Agents

➢ Model-Based Reflex Agents

➢ Goal-Based Agents

➢ Utility-Based Agents
Simple reflex agents:
➢ Simple reflex agents ignore the rest of the percept history and act only on the basis
of the current percept.
➢ The agent function is based on the condition-action rule.

➢ If the condition is true, then the action is taken, else not. This agent function only
succeeds when the environment is fully observable.
The program in Figure 2.8 is specific to one particular vacuum environment. A more general
and flexible approach is first to build a general-purpose interpreter for condition– action rules
and then to create rule sets for specific task environments. Figure 2.9 gives the structure of this
general program in schematic form, showing how the condition–action rules allow the agent to
make the connection from percept to action. The agent program, which is also very simple, is

Prof. Rashmi B C, Assistant Professor., AI&ML, RNSIT Page- 21

Sub Name: ARTIFICIAL INTELLIGENCE Sub Code: BAD402

shown in Figure 2.10. The agent in Figure 2.10 will work only if the correct decision can be
made on the basis of only the current percept that is, only if the environment is fully observable.

Model-based reflex agents:

➢ The Model-based agent can work in a partially observable environment, and track
the situation.
➢ A model-based agent has two important factors:

Prof. Rashmi B C, Assistant Professor., AI&ML, RNSIT Page- 22

Sub Name: ARTIFICIAL INTELLIGENCE Sub Code: BAD402

 Model: It is knowledge about "how things happen in the world," so it is called

a Model-based agent.
 Internal State: It is a representation of the current state based on percept history.
Figure 2.11 gives the structure of the model-based reflex agent with internal state, showing
how the current percept is combined with the old internal state to generate the updated
description of the current state, based on the agent’s model of how the world works. The agent
program is shown in Figure 2.12. The interesting part is the function UPDATE-STATE, which
is responsible for creating the new internal state description.

Goal-based agents:
➢ A goal-based agent has an agenda.
➢ It operates based on a goal in front of it and makes decisions based on how best to
reach that goal.

Prof. Rashmi B C, Assistant Professor., AI&ML, RNSIT Page- 23

Sub Name: ARTIFICIAL INTELLIGENCE Sub Code: BAD402

➢ A goal-based agent operates as a search and planning function, meaning it targets

the goal ahead and finds the right action in order to reach it.
➢ Expansion of model-based agent.
Figure 2.13 shows the goal-based agent’s structure.

Utility-based agents:
➢ A utility-based agent is an agent that acts based not only on what the goal is, but the
best way to reach that goal.
➢ The Utility-based agent is useful when there are multiple possible alternatives, and
an agent has to choose in order to perform the best action.
➢ The term utility can be used to describe how "happy" the agent is.
The utility-based agent structure appears in Figure 2.14.

Prof. Rashmi B C, Assistant Professor., AI&ML, RNSIT Page- 24

Sub Name: ARTIFICIAL INTELLIGENCE Sub Code: BAD402

Learning agents:
We have described agent programs with various methods for selecting actions. We have not,
so far, explained how the agent programs come into being. In his famous early paper, Turing
(1950) considers the idea of actually programming his intelligent machines by hand. He
estimates how much work this might take and concludes “Some more expeditious method
seems desirable.” The method he proposes is to build learning machines and then to teach them.
In many areas of AI, this is now the preferred method for creating state-of-the-art systems.
Learning has another advantage, as we noted earlier:
It allows the agent to operate in initially unknown environments and to become more competent
than its initial knowledge alone might allow.
A learning agent can be divided into four conceptual components, as shown in Figure 2.15.

Prof. Rashmi B C, Assistant Professor., AI&ML, RNSIT Page- 25

Sub Name: ARTIFICIAL INTELLIGENCE Sub Code: BAD402

Learning Element:
The most important distinction is between the learning element, which is responsible for
making improvements, and the performance element, which is responsible for selecting
external actions.
Performance Element:
The performance element is what we have previously considered to be the entire agent It takes
in percepts and decides on actions.
Critic:
The learning element uses feedback from the critic on how the agent is doing and determines
how the performance element should be modified to do better in the future. The design of the
learning element depends very much on the design of the performance element. When trying
to design an agent that learns a certain capability, the first question is not “How am I going to
get it to learn this?” but “What kind of performance element will my agent need to do this once
it has learned how?” Given an agent design, learning mechanisms can be constructed to
improve every part of the agent. The critic tells the learning element how well the agent is
doing with respect to a fixed performance standard.
The critic is necessary because the percepts themselves provide no indication of the agent’s
success. For example, a chess program could receive a percept indicating that it has checkmated
its opponent, but it needs a performance standard to know that this is a good thing; the percept
itself does not say so. It is important that the performance. standard be fixed. Conceptually, one
should think of it as being outside the agent altogether because the agent must not modify it to
fit its own behavior.
The last component of the learning agent is the problem generator.
Problem Generator:
It is responsible for suggesting actions that will lead to new and informative experiences. The
point is that if the performance element had its way, it would keep doing the actions that are
best, given what it knows. But if the agent is willing to explore a little and do some perhaps
suboptimal actions in the short run, it might discover much better actions for the long run. The
problem generator’s job is to suggest these exploratory actions. This is what scientists do when
they carry out experiments. Galileo did not think that dropping rocks from the top of a tower
in Pisa was valuable in itself. He was not trying to break the rocks or to modify the brains of
unfortunate passers-by. His aim was to modify his own brain by identifying a better theory of
the motion of objects.

Prof. Rashmi B C, Assistant Professor., AI&ML, RNSIT Page- 26

Sub Name: ARTIFICIAL INTELLIGENCE Sub Code: BAD402

To make the overall design more concrete, let us return to the automated taxi example. The
performance element consists of whatever collection of knowledge and procedures the taxi has
for selecting its driving actions. The taxi goes out on the road and drives, using this performance
element. The critic observes the world and passes information along to the learning element.
For example, after the taxi makes a quick left turn across three lanes of traffic, the critic
observes the shocking language used by other drivers. From this experience, the learning
element is able to formulate a rule saying this was a bad action, and the performance element
is modified by installation of the new rule. The problem generator might identify certain areas
of behavior in need of improvement and suggest experiments, such as trying out the brakes on
different road surfaces under different conditions.
The simplest cases involve learning directly from the percept sequence. Observation of pairs
of successive states of the environment can allow the agent to learn “How the world evolves,”
and observation of the results of its actions can allow the agent to learn “What my actions do.”
For example, if the taxi exerts a certain braking pressure when driving on a wet road, then it
will soon find out how much deceleration is actually achieved. Clearly, these two learning tasks
are more difficult if the environment is only partially observable.
The forms of learning in the preceding paragraph do not need to access the external
performance standard—in a sense, the standard is the universal one of making predictions that
agree with experiment. The situation is slightly more complex for a utility-based agent that
wishes to learn utility information. For example, suppose the taxi-driving agent receives no tips
from passengers who have been thoroughly shaken up during the trip. The external
performance standard must inform the agent that the loss of tips is a negative contribution to
its overall performance; then the agent might be able to learn that violent maneuvers do not
contribute to its own utility. In a sense, the performance standard distinguishes part of the
incoming percept as a reward (or penalty) that provides direct feedback on the quality of the
agent’s behavior. Hard-wired performance standards such as pain and hunger in animals can
be understood in this way.
How the components of agent programs work:
We have described agent programs (in very high-level terms) as consisting of various
components, whose function it is to answer questions such as: “What is the world like now?”
“What action should I do now?” “What do my actions do?” The next question for a student of
AI is, “How on earth do these components work?” It takes about a thousand pages to begin to
answer that question properly, but here we want to draw the reader’s attention to some basic

Prof. Rashmi B C, Assistant Professor., AI&ML, RNSIT Page- 27

Sub Name: ARTIFICIAL INTELLIGENCE Sub Code: BAD402

distinctions among the various ways that the components can represent the environment that
the agent inhabits.
Roughly speaking, we can place the representations along an axis of increasing complexity and
expressive power—atomic, factored, and structured. To illustrate these ideas, it helps to
consider a particular agent component, such as the one that deals with “What my actions do.”
This component describes the changes that might occur in the environment as the result of
taking an action, and Figure 2.16 provides schematic depictions of how those transitions might
be represented.

Atomic Representation:
In an atomic representation each state of the world is indivisible it has no internal structure.
Consider the problem of finding a driving route from one end of a country to the other via some
sequence of cities. For the purposes of solving this problem, it may suffice to reduce the state
of world to just the name of the city we are in a single atom of knowledge; a “black box” whose
only discernible property is that of being identical to or different from another black box.
Factored Representation Variable:
A factored representation splits up each state into a fixed set of variables or attributes, each of
which can have a value. While two different atomic states have nothing in common—they are
just different black boxes—two different factored states can share some attributes (such as
being at some particular GPS location) and not others (such as having lots of gas or having no
gas); this makes it much easier to work out how to turn one state into another.

Prof. Rashmi B C, Assistant Professor., AI&ML, RNSIT Page- 28

Sub Name: ARTIFICIAL INTELLIGENCE Sub Code: BAD402

With factored representations, we can also represent uncertainty for example, ignorance
about the amount of gas in the tank can be represented by leaving that attribute blank.
For many purposes, we need to understand the world as having things in it that are related to
each other, not just variables with values. For example, we might notice that a large truck ahead
of us is reversing into the driveway of a dairy farm but a cow has got loose and is blocking the
truck’s path. A factored representation is unlikely to be pre-equipped with the attribute
TruckAheadBackingIntoDairyFarmDrivewayBlockedByLooseCow with value true or false.
Instead, we would need a structured representation, in which objects such as cows and trucks
and their various and varying relationships can be described explicitly. (See Figure 2.16(c).)
In fact, almost everything that humans express in natural language concerns objects and their
relationships. As we mentioned earlier, the axis along which atomic, factored, and structured
representations lie is the axis of increasing expressiveness. Often, the more expressive
language is much more concise; for example, the rules of chess can be written in a page or two
of a structured-representation language such as first-order logic but require thousands of pages
when written in a factored-representation language such as propositional logic. On the other
hand, reasoning and learning become more complex as the expressive power of the
representation increases. To gain the benefits of expressive representations while avoiding their
drawbacks, intelligent systems for the real world may need to operate at all points along the
axis simultaneously.

-----------------------------------------END OF MODULE-1---------------------------------------------

Prof. Rashmi B C, Assistant Professor., AI&ML, RNSIT Page- 29