Module-1

AI, or artificial intelligence, aims not only to understand human intelligence but also to create
intelligent systems. Emerging after World War II and formally named in 1956, AI remains a dynamic
field with vast opportunities. It spans diverse subfields, from learning and perception to specialized
tasks like chess, poetry, and medical diagnosis. AI is considered a universal discipline, relevant to a
wide range of intellectual activities.

WHAT IS AI?
AI can be defined along two dimensions::
In Figure 1.1 we see eight definitions of AI, laid out along two dimensions. The definitions on top are
concerned with thought processes and reasoning, whereas the ones on the bottom address behavior.
The definitions on the left measure success in terms of fidelity to human performance, whereas the
ones on the right measure against an ideal performance measure, called rationality. A system is rational
if it does the ―right thing,‖ given what it knows.

1.1.1 Acting humanly: The Turing Test approach

The Turing Test, proposed by Alan Turing (1950), was designed to provide a satisfactory operational
definition of intelligence. A computer passes the test if a human interrogator, after posing some written
questions, cannot tell whether the written responses come from a person or from a computer. For now,
we note that programming a computer to pass a rigorously applied test provides plenty to work on. The
computer would need to possess the following capabilities:
 natural language processing to enable it to communicate successfully in English;
 knowledge representation to store what it knows or hears;
 automated reasoning to use the stored information to answer questions and to draw new
conclusions;
 machine learning to adapt to new circumstances and to detect and extrapolate patterns.
 computer vision to perceive objects, and
 robotics to manipulate objects and move about.

Thinking humanly: The cognitive modeling approach

Understanding human thinking is crucial for assessing whether AI can think like humans. There are
three primary ways to study human thought: introspection, psychological experiments, and brain
imaging. Once a precise theory of the mind is developed, it can be expressed as a computer program. If
the program’s input-output behavior aligns with human behavior, it suggests similarities in underlying
mechanisms. Allen Newell and Herbert Simon developed the General Problem Solver (GPS) to model
human reasoning by comparing AI reasoning steps to human problem-solving processes. Cognitive
science integrates AI models and psychological experiments to construct precise, testable theories
about the human mind. This interdisciplinary field has contributed significantly to our understanding of
intelligence and has been explored extensively in academic literature.
AI research occasionally references human cognition but primarily focuses on computational methods.
In its early days, AI research often confused efficient task performance with human-like intelligence,
leading to misconceptions. Modern researchers distinguish between AI’s problem-solving abilities and
actual human cognition, which has helped both AI and cognitive science advance more effectively.
Despite this distinction, the two fields continue to influence each other, particularly in areas like
computer vision, where neurophysiological insights contribute to AI development. Unlike AI, cognitive
science relies on experimental studies involving humans or animals, while AI research assumes only a
computer as its testing ground. Cognitive science aims to replicate and understand natural intelligence,
whereas AI focuses on building efficient computational intelligence.
Both fields benefit from cross-disciplinary insights, with AI taking inspiration from human cognition
without necessarily mimicking it. The clear distinction between AI and cognitive science prevents
misinterpretation and allows for specialized advancements in each domain. AI has led to breakthroughs
in automated reasoning and perception, while cognitive science continues to refine our understanding
of human intelligence. Future AI developments may further integrate findings from cognitive science,
fostering collaboration between the two disciplines. This ongoing exchange of knowledge enhances
progress in both fields, contributing to the development of more advanced intelligent systems.
Thinking rationally: The “laws of thought” approach
Aristotle was one of the first to define structured reasoning through syllogisms, which led to the study
of logic. In the 19th century, logicians developed precise notations for statements about objects and
their relationships. By 1965, AI programs could theoretically solve problems expressed in logical
notation, forming the foundation of the logicist tradition in AI. This approach aims to create intelligent
systems based on logical reasoning. However, two major challenges arise: translating informal
knowledge into precise logical terms and managing computational complexity. Many problems require
significant computational resources, making practical implementation difficult. Even small problems
can overwhelm computers without proper guidance on reasoning steps. These challenges, first
encountered in the logicist tradition, remain relevant to computational reasoning today.
Acting rationally: The rational agent approach
Computer agents differ from regular programs by operating autonomously, perceiving their
environment, persisting over time, adapting to change, and pursuing goals. A rational agent aims to
achieve the best possible outcome or, under uncertainty, the best expected outcome.
The ―laws of thought‖ approach in AI emphasizes correct inferences, which can help rational agents
make logical decisions. However, rationality goes beyond inference, as some actions, like reflexes, are
more effective without deliberate reasoning. For example, recoiling from a hot stove is a rational action
without requiring logical deduction.
Skills needed for the Turing Test also contribute to rational behavior. Knowledge representation and
reasoning help agents make sound decisions. Natural language processing enables communication in
complex societies, while learning enhances effective behavior.
The rational-agent approach has two key advantages. First, it is broader than the ―laws of thought‖
approach because rationality can be achieved through multiple mechanisms, not just inference. Second,
it is more suitable for scientific development than approaches based on human behavior or thought.
Rationality is mathematically well-defined and can be used to create agents that provably achieve their
goals. In contrast, human behavior is complex, environment-dependent, and difficult to formally define.

THE FOUNDATIONS OF ARTIFICIAL INTELLIGENCE

In this section, we provide a brief history of the disciplines that contributed ideas, viewpoints,
and techniques to AI. Like any history, this one is forced to concentrate on a small number
of people, events, and ideas and to ignore others that also were important. We organize the
history around a series of questions. We certainly would not wish to give the impression that
these questions are the only ones the disciplines address or that the disciplines have all been
working toward AI as their ultimate fruition.
Philosophy
• Can formal rules be used to draw valid conclusions?
• How does the mind arise from a physical brain?
• Where does knowledge come from?
• How does knowledge lead to action?
The foundation of logical reasoning dates back to Aristotle, who introduced syllogisms as a structured
way to derive conclusions from premises. His work initiated formal logic, influencing centuries of
thought. Later, Ramon Lull envisioned mechanical artifacts capable of reasoning. Thomas Hobbes
compared reasoning to numerical computation, suggesting that thought processes could be quantified.
By the 1500s, Leonardo da Vinci designed a mechanical calculator, though it remained unbuilt. In the
early 1600s, Wilhelm Schickard constructed the first known calculating machine, while Blaise Pascal's
Pascaline became more widely recognized. Pascal believed these machines were closer to thought than
any action performed by animals. Leibniz further advanced mechanical computation by developing a
device capable of performing arithmetic operations such as addition, subtraction, multiplication, and
root extraction. He speculated about machines performing conceptual reasoning, leading to early
discussions about artificial intelligence.
Hobbes expanded on this idea by proposing an "artificial animal," likening human biological functions
to mechanical systems. He argued that the mind operated similarly to mechanical processes, sparking
early debates about the nature of thought. René Descartes, however, introduced dualism, arguing that
the mind and body were distinct entities. According to Descartes, while animals were purely
mechanical beings, humans possessed a unique, non-physical mind exempt from physical laws. This
perspective contrasted with materialism, which held that the mind's operations were entirely physical,
governed by the same natural laws as any other physical system.
As the debate over the nature of thought continued, different schools of philosophy emerged.
Rationalists like Descartes and Leibniz emphasized reasoning as the primary source of knowledge,
believing that logical principles could explain the world. In contrast, empiricists such as Francis Bacon,
John Locke, and David Hume argued that knowledge arises from sensory experiences. Hume
introduced the principle of induction, asserting that general rules develop through repeated
observations. This idea became a cornerstone of scientific reasoning.
Building on these philosophical traditions, the Vienna Circle, led by Rudolf Carnap, developed logical
positivism. This doctrine combined rationalism and empiricism, arguing that all knowledge could be
characterized through logical theories connected to sensory input. Logical positivists sought to create a
structured, scientific method for understanding thought. Carnap and Carl Hempel advanced
confirmation theory, which analyzed how knowledge is derived from experience. Carnap’s book, The
Logical Structure of the World, outlined a computational procedure for knowledge acquisition,
marking one of the earliest attempts to describe cognition algorithmically.
Beyond reasoning, philosophers also explored the connection between knowledge and action. Aristotle
addressed this in De Motu Animalium, suggesting that actions result from logical connections between
goals and knowledge. He explained that reasoning about unchanging truths is akin to reasoning about
action, with conclusions leading to decisions. For instance, a person needing warmth might reason that
a cloak provides warmth, leading to the action of making or acquiring a cloak.
Aristotle expanded this idea in Nicomachean Ethics, where he described an algorithmic approach to
decision-making. He argued that people deliberate about means rather than ends, analyzing the best
methods to achieve a goal. If a desired outcome was impossible, the reasoning process halted; if it
seemed feasible, action was taken. This method of goal-based analysis laid the foundation for modern
decision-making processes in AI. Over two millennia later, Newell and Simon implemented this
approach in their General Problem Solver (GPS), which we now recognize as a regression planning
system.
While goal-based reasoning is useful, it does not address situations where multiple actions could
achieve a goal or where no action is entirely sufficient. In such cases, philosophers like Antoine
Arnauld proposed quantitative formulas for decision-making. John Stuart Mill expanded on these ideas
in Utilitarianism, advocating rational decision-making in all areas of life. This line of thought
eventually led to the formalization of decision theory, which provides structured methods for selecting
optimal actions based on given information.
The historical development of reasoning, knowledge acquisition, and decision-making has directly
influenced artificial intelligence. From Aristotle’s syllogisms to Hobbes’ mechanical metaphors,
Descartes’ dualism, and Hume’s empiricism, each philosophical tradition has contributed to our
understanding of thought processes. Logical positivism and confirmation theory provided a structured
approach to knowledge representation, while Aristotle’s goal-based reasoning laid the groundwork for
AI planning and decision-making systems. The transition from purely theoretical discussions to
practical implementations, such as the GPS system, illustrates how ancient ideas have evolved into
modern AI methodologies.
The field of AI continues to build on these foundational concepts, striving to develop intelligent
systems that can reason, learn, and act autonomously. Philosophical debates about the nature of
intelligence, free will, and rationality persist, influencing how AI is designed and applied. The interplay
between historical thought and technological advancements underscores the enduring relevance of
these ideas, demonstrating how centuries-old theories shape the future of artificial intelligence.

Mathematics
• What are the formal rules to draw valid conclusions?
• What can be computed?
• How do we reason with uncertain information?
Here’s the summary in paragraph form:
Philosophers laid the foundation for AI, but its development into a formal science required advances in
three key areas of mathematics: logic, computation, and probability. The origins of formal logic date
back to ancient Greece, but its mathematical development began with George Boole (1815–1864), who
introduced Boolean logic. Gottlob Frege (1848–1925) extended it to first-order logic, which remains
fundamental today. Alfred Tarski (1902–1983) contributed a theory connecting logical objects to real-
world entities. Computation traces back to Euclid’s algorithm for computing the greatest common
divisor, and the word ―algorithm‖ originates from al-Khwarizmi, a 9th-century Persian mathematician.
Boole and others explored algorithms for logical deduction, and by the late 19th century, efforts
emerged to formalize mathematical reasoning as logical deduction. Kurt Gödel (1906–1978) proved
that while first-order logic could express many truths, it could not capture mathematical induction. His
1931 incompleteness theorem revealed that any sufficiently strong formal system contains true but
unprovable statements, implying that some functions cannot be computed.
This limitation led Alan Turing (1912–1954) to investigate computability, culminating in the Church–
Turing thesis, which states that Turing machines can compute any computable function. However,
Turing also showed that some functions, such as determining whether a program halts, are non-
computable. While decidability and computability define the limits of AI, tractability is even more
crucial. A problem is intractable if its solving time grows exponentially with instance size. The
distinction between polynomial and exponential complexity, emphasized in the 1960s by Cobham and
Edmonds, underlines why AI must focus on solving tractable subproblems. The theory of NP-
completeness, pioneered by Steven Cook (1971) and Richard Karp (1972), classifies certain problems
as NP-complete, meaning they are computationally hard. Though it remains unproven whether all NP-
complete problems are intractable, most researchers believe they are. Despite early optimism about
computers as ―Electronic Super-Brains,‖ AI research emphasizes efficient problem-solving due to
computational limitations.
Besides logic and computation, probability plays a crucial role in AI. Gerolamo Cardano (1501–1576)
first conceptualized probability, framing it in terms of gambling outcomes. Blaise Pascal and Pierre
Fermat advanced probability theory in 1654 by analyzing unfinished gambling games. Probability
became essential in science, allowing researchers to deal with uncertainty. Contributions from James
Bernoulli, Pierre Laplace, and others led to new statistical methods. Thomas Bayes (1702–1761)
proposed Bayes’ rule, which underlies modern AI approaches to uncertain reasoning. AI integrates
logic for structured reasoning, computation for problem-solving, and probability for handling
uncertainty. These mathematical foundations shape AI’s capabilities and limitations, influencing its
development and applications. Gödel’s incompleteness theorem revealed logical constraints, Turing’s
work defined computational limits, and NP-completeness theory classified problem difficulty. AI
researchers navigate these constraints while leveraging probability for adaptability. Future
advancements in AI depend on deeper mathematical insights and the interplay between logic,
computation, and probability.
Economics
 How should we make decisions so as to maximize payoff?
 How should we do this when others may not go along?
 How should we do this when the payoff may be far in the future?
The science of economics began in 1776 with Adam Smith’s The Wealth of Nations, marking the first
systematic approach to economic study. While ancient thinkers contributed to economic thought, Smith
treated it as a science, viewing economies as collections of individuals maximizing their well-being.
Economics is not merely about money but about how people make choices that lead to preferred
outcomes. The mathematical formalization of utility theory, which quantifies these preferences, began
with Léon Walras (1834–1910) and was later improved by Frank Ramsey, John von Neumann, and
Oskar Morgenstern in their 1944 work The Theory of Games and Economic Behavior. Decision theory,
which combines probability and utility theory, provides a structured framework for making choices
under uncertainty, particularly in large economies where individual agents act independently. However,
in smaller economies, the actions of one agent significantly impact others, making decision-making
more game-like. Game theory, pioneered by von Neumann and Morgenstern, revealed that in certain
situations, rational agents should adopt randomized policies. Unlike decision theory, game theory does
not always prescribe a single best action. Traditional economics largely ignored sequential decision-
making, where payoffs result from a series of actions rather than immediate consequences. This
problem was later addressed by operations research, a field that emerged during World War II to
optimize radar installations and later expanded into complex management decisions. Richard Bellman
formalized sequential decision-making in 1957 through Markov decision processes. Although
economics and operations research contributed to the understanding of rational decision-making, AI
research initially followed a separate trajectory due to the complexity of making optimal decisions.
Herbert Simon (1916–2001), a pioneering AI researcher, won the Nobel Prize in economics in 1978 for
demonstrating that human decision-making often follows a satisficing approach—seeking solutions
that are "good enough" rather than strictly optimal. This contrasted with classical economic models that
assumed perfectly rational agents. Since the 1990s, AI research has increasingly incorporated decision-
theoretic techniques, with scholars like Michael Wellman contributing to this shift. As a result,
economic principles such as utility theory, decision theory, and game theory have become integral to
AI, helping model rational behavior, optimize decision-making under uncertainty, and manage
interactions between intelligent agents.

Neuroscience
How do brains process information?
Neuroscience is the study of the nervous system, particularly the brain, which has long been recognized
as the center of thought and consciousness. While the exact mechanisms by which the brain enables
cognition remain a mystery, its role in mental function has been acknowledged for thousands of years,
as evidenced by the effects of head injuries on cognitive abilities. Aristotle, in 335 B.C., noted that
humans have the largest brain in proportion to body size, yet it was only in the mid-18th century that
the brain was widely accepted as the seat of consciousness, replacing earlier beliefs that attributed
thought to the heart or spleen. A major breakthrough came in 1861 when Paul Broca identified a
specific brain area responsible for speech production, now called Broca’s area, demonstrating that
cognitive functions are localized to distinct regions. Around the same time, scientists had discovered
that the brain consists of nerve cells, or neurons, but it was not until 1873 that Camillo Golgi developed
a staining technique allowing the visualization of individual neurons. This technique enabled Santiago
Ramón y Cajal to make pioneering discoveries about neuronal structures, and later, Nicolas Rashevsky
applied mathematical models to study the nervous system. Over time, researchers have mapped
connections between brain areas and their corresponding motor and sensory functions, but these
mappings can change significantly in just a few weeks, and some animals exhibit multiple maps.
Despite these findings, many mysteries remain, such as how the brain compensates for damaged areas
and how individual memories are stored. The first technique for measuring brain activity in living
humans emerged in 1929 with Hans Berger’s invention of the electroencephalograph (EEG). More
recently, functional magnetic resonance imaging (fMRI) has provided neuroscientists with detailed
images of brain activity, allowing correlations between brain function and cognitive processes.
Advances in single-cell recording have also enabled researchers to stimulate neurons electrically,
chemically, or optically, helping to map input-output relationships in neural circuits. Despite these
technological advances, our understanding of how cognition actually works remains incomplete. The
astonishing reality is that thought, action, and consciousness arise from a collection of simple cells,
leading John Searle to conclude in 1992 that ―brains cause minds.‖ The only alternative to this view is
mysticism, which suggests that the mind exists beyond the realm of physical science. While brains and
digital computers share some similarities, they also have fundamental differences. Computers operate
at speeds millions of times faster than the brain but lack the brain’s vast storage capacity and dense
interconnectivity. Although the most powerful supercomputers now have capacities comparable to the
human brain, the brain does not appear to use all its neurons simultaneously. Futurists speculate about
an impending technological singularity, a point where computers surpass human intelligence, but such
comparisons are of limited value. Even if we had an infinitely powerful computer, we still would not
know how to replicate the intelligence of the human brain.
Figure 1.2
The parts of a nerve cell or neuron. Each neuron consists of a cell body, or soma, that contains a cell
nucleus. Branching out from the cell body are a number of fibers called dendrites and a single long
fiber called the axon. The axon stretches out for a long distance, much longer than the scale in this
diagram indicates. Typically, an axon is 1 cm long (100 times the diameter of the cell body), but can
reach up to 1 meter. A neuron makes connections with 10 to 100,000 other neurons at junctions called
synapses. Signals are propagated from neuron to neuron by a complicated electrochemical reaction.
The signals control brain activity in the short term and also enable long-term changes in the
connectivity of neurons. These mechanisms are thought to form the basis for learning in the brain.
Most information processing goes on in the cerebral cortex, the outer layer of the brain. The basic
organizational unit appears to be a column of tissue about 0.5 mm in diameter, containing about
20,000 neurons and extending the full depth of the cortex about 4 mm in humans).

Psychology

How do humans and animals think and act?

The origins of scientific psychology trace back to Hermann von Helmholtz and Wilhelm Wundt.
Helmholtz applied scientific methods to vision, producing the influential Handbook of Physiological
Optics. Wundt, who established the first psychology lab in 1879 at Leipzig, emphasized controlled
experiments where subjects introspected on their thought processes. However, the subjective nature of
introspection limited its reliability. Meanwhile, biologists studying animal behavior lacked
introspective data and developed objective methodologies, as seen in H. S. Jennings' Behavior of the
Lower Organisms (1906). This led to behaviorism, championed by John Watson, which rejected mental
processes as unmeasurable and focused only on observable stimuli and responses. While behaviorism
advanced understanding of simple behaviors in animals, it struggled with explaining human cognition.
Cognitive psychology, which views the brain as an information processor, has roots in William James
and Helmholtz’s idea that perception involves unconscious inference. Though behaviorism dominated
in the U.S., Cambridge’s Applied Psychology Unit, under Frederic Bartlett, allowed cognitive
modeling to persist. Bartlett’s student, Kenneth Craik, in The Nature of Explanation (1943), argued that
mental constructs like beliefs and goals were scientifically valid, likening them to concepts like
pressure and temperature in physics. Craik outlined three key steps of cognitive processing: translating
stimuli into internal representations, manipulating these representations, and using them to guide action.
He suggested that organisms benefit from maintaining internal models of reality, allowing them to
anticipate future events and act effectively.
After Craik’s untimely death in 1945, Donald Broadbent continued his work, pioneering information-
processing models in Perception and Communication (1958). Meanwhile, in the U.S., cognitive science
emerged alongside developments in computer modeling. The field’s official start is often attributed to a
1956 MIT workshop where George Miller presented The Magic Number Seven, Noam Chomsky
introduced Three Models of Language, and Allen Newell and Herbert Simon demonstrated The Logic
Theory Machine. These papers showed how computational models could explain memory, language,
and logical thinking. Today, many psychologists believe cognitive theories should resemble computer
programs, specifying mechanisms for cognitive functions in an information-processing framework.

Computer engineering

How can we build an efficient computer?

For artificial intelligence to succeed, it requires both intelligence and an artifact, with the modern
digital computer serving as the preferred artifact. The first operational computer, the electromechanical
Heath Robinson, was built in 1940 by Alan Turing’s team for codebreaking. In 1943, this team
developed Colossus, a general-purpose vacuum tube-based machine. Around the same time, Konrad
Zuse in Germany created the Z-3, the first programmable computer, also pioneering floating-point
numbers and the first high-level programming language, Plankalkül. Meanwhile, in the U.S., John
Atanasoff and Clifford Berry built the ABC (Atanasoff-Berry Computer) between 1940 and 1942,
though it remained largely unrecognized. Instead, the ENIAC, developed at the University of
Pennsylvania by John Mauchly and John Eckert, became the most influential precursor of modern
computers.
Since then, each generation of computer hardware has improved in speed, capacity, and affordability.
Performance doubled roughly every 18 months until 2005, when power dissipation issues shifted focus
to multi-core processors instead of increasing clock speed. Future advancements are expected to rely on
massive parallelism, resembling the brain’s structure. Before electronic computers, various calculating
devices existed. Early automated machines date back to the 17th century, and the first programmable
machine, Joseph Marie Jacquard’s loom (1805), used punched cards to control weaving patterns. In the
19th century, Charles Babbage designed two groundbreaking but unfinished machines. The Difference
Engine was meant to compute mathematical tables and was successfully built in 1991 at the Science
Museum in London. His more ambitious Analytical Engine, featuring addressable memory, stored
programs, and conditional jumps, was the first universal computing concept. Ada Lovelace, Babbage’s
collaborator and daughter of Lord Byron, is recognized as the first programmer, having written
programs for the Analytical Engine and speculating on its potential to play chess or compose music.
AI has also benefited from software developments, including operating systems, programming
languages, and development tools. However, AI has, in turn, contributed significantly to mainstream
computer science. Innovations from AI research, such as time-sharing, interactive interpreters, personal
computing with graphical interfaces, rapid development environments, linked lists, automatic storage
management, and key programming paradigms like symbolic, functional, declarative, and object-
oriented programming, have shaped modern computing.

Control theory and cybernetics

How can artifacts operate under their own control?

Ktesibios of Alexandria (c. 250 B.C.) built the first self-regulating machine, a water clock with a
constant flow rate, changing the perception of what artifacts could do. Before this, only living beings
could adjust their behavior based on environmental changes. Other early feedback control systems
include James Watt’s steam engine governor and Cornelis Drebbel’s thermostat. The mathematical
foundations of stable feedback systems emerged in the 19th century.
Norbert Wiener (1894–1964), a key figure in control theory, worked with Bertrand Russell before
exploring the connections between biological and mechanical control systems. With colleagues like
Arturo Rosenblueth and Julian Bigelow, he challenged behaviorist views, seeing behavior as a
mechanism minimizing the difference between current and goal states. In the late 1940s, Wiener, along
with Warren McCulloch, Walter Pitts, and John von Neumann, organized influential conferences on
cognition and computation. His 1948 book Cybernetics introduced the public to the idea of artificial
intelligence.
In Britain, W. Ross Ashby and others, including Alan Turing and Grey Walter, formed the Ratio Club
to discuss Wiener’s ideas. Ashby’s Design for a Brain (1948, 1952) explored how intelligence could
emerge from homeostatic devices using feedback loops for stable adaptive behavior.
Modern control theory, particularly stochastic optimal control, aims to design systems that maximize
an objective function over time. This aligns with AI’s goal of creating optimal behavior. Despite their
shared origins, AI and control theory remain distinct due to differences in mathematical approaches.
Control theory, using calculus and matrix algebra, focuses on continuous systems, while AI developed
alternative tools like logical inference and computation to tackle problems such as language, vision,
and planning.

Linguistics
• How does language relate to thought?
In 1957, B. F. Skinner published Verbal Behavior, a detailed behaviorist account of language learning.
However, Noam Chomsky’s review of the book became equally famous and undermined behaviorism’s
dominance. Chomsky argued that behaviorism failed to explain linguistic creativity—how children
understand and generate sentences they have never heard before. His own theory, inspired by ancient
linguist Panini, provided a formal model of syntax that could, in principle, be programmed.
This marked a shift in linguistics, which developed alongside artificial intelligence, leading to the
emergence of computational linguistics or natural language processing (NLP). Initially, researchers
assumed that understanding language was mainly about mastering sentence structures. However, by the
1960s, it became clear that true language comprehension required an understanding of context and
subject matter, not just syntax.
Early research in AI and knowledge representation was closely tied to linguistics, as scientists sought
ways to encode knowledge in forms that computers could reason with. This, in turn, was influenced by
decades of philosophical inquiry into the nature of language and meaning. The intersection of AI and
linguistics continues to shape modern NLP, which remains a complex challenge due to the nuances of
human language.

THE HISTORY OF ARTIFICIAL INTELLIGENCE

1. The gestation of artificial intelligence (1943–1955)

The foundations of artificial intelligence were laid by Warren McCulloch and Walter Pitts in 1943.
Drawing from neuroscience, propositional logic, and Turing’s theory of computation, they proposed a
model of artificial neurons. In their model, neurons were either ―on‖ or ―off,‖ activating based on
sufficient input from neighboring neurons. They demonstrated that such networks could compute any
function and implement logical operations like AND, OR, and NOT. They also speculated that neural
networks could learn, a concept later developed by Donald Hebb in 1949. Hebb introduced an
influential rule—Hebbian learning—which suggested that connections between neurons strengthen
with repeated activation, forming the basis of modern neural network learning.
In 1950, Harvard undergraduates Marvin Minsky and Dean Edmonds built the first neural network
computer, SNARC, using 3,000 vacuum tubes and surplus airplane components to simulate 40 neurons.
Minsky later studied computation in neural networks at Princeton. His work faced skepticism, but John
von Neumann reportedly supported it, predicting its future significance. Minsky later proved key
limitations of early neural networks, shaping AI research directions.
Amid these developments, Alan Turing played a central role in defining AI’s vision. As early as 1947,
he gave lectures on intelligent machines, and in 1950, he published Computing Machinery and
Intelligence, introducing the famous Turing Test. He also proposed machine learning, genetic
algorithms, and reinforcement learning, advocating the idea of training AI like a child rather than
attempting to replicate an adult mind from the start. His ideas provided a roadmap for AI, influencing
generations of researchers and establishing key concepts that still shape the field today.
2.The birth of artificial intelligence (1956)
John McCarthy, a key AI pioneer, played a crucial role in defining AI as a distinct discipline. After
earning his Ph.D. at Princeton in 1951, he worked at Stanford before moving to Dartmouth, where AI’s
official birth took place. In 1956, he, along with Marvin Minsky, Claude Shannon, and Nathaniel
Rochester, organized the historic Dartmouth Summer Research Project on Artificial Intelligence. This
workshop gathered leading scientists in automata theory, neural networks, and intelligence research to
explore the possibility of simulating human intelligence in machines. Among the attendees were
Trenchard More, Arthur Samuel, Ray Solomonoff, and Oliver Selfridge. The most significant
contribution came from Allen Newell and Herbert Simon, who introduced the Logic Theorist, a
program capable of reasoning symbolically. Simon claimed that their program had solved the mind-
body problem by demonstrating that machines could think non-numerically. Despite its innovations, AI
faced skepticism, as reflected in the rejection of a paper coauthored by Newell, Simon, and the Logic
Theorist. Although the Dartmouth workshop did not yield immediate breakthroughs, it laid the
foundation for AI as an independent research field. Over the next two decades, McCarthy, Minsky,
Newell, and Simon advanced AI at institutions like MIT, Stanford, Carnegie Mellon, and IBM,
distinguishing it from control theory and operations research. AI’s unique focus on replicating human
cognition, creativity, and adaptability necessitated a distinct identity. McCarthy’s development of LISP
at Stanford further cemented AI’s connection to computer science, enabling advances in natural
language processing, expert systems, and early machine learning models. Minsky, collaborating with
Seymour Papert, explored neural networks, but their book Perceptrons revealed limitations in early
models, temporarily slowing research in the field. Meanwhile, Newell and Simon developed the
General Problem Solver (GPS), introducing heuristic search techniques that shaped automated
reasoning. During the 1960s, AI research gained momentum as governments and private industries
increased funding, leading to the establishment of research labs at top universities and corporations.
Early AI programs demonstrated intelligent behavior in areas like chess, algebra, and language
translation, proving the feasibility of AI. However, challenges emerged, including the divide between
symbolic AI and connectionist approaches. Symbolic AI, championed by McCarthy and Newell, relied
on rule-based knowledge encoding, which led to the development of expert systems in the 1970s and
1980s. In contrast, connectionist approaches, inspired by neural networks, sought to mimic human
learning processes. Though neural networks fell out of favor due to computational limitations, they
resurged in the 21st century with the advent of deep learning. Despite early optimism, AI researchers
underestimated the complexity of human cognition, struggling with common-sense reasoning,
perception, and experiential learning. This led to periods of reduced funding and stalled progress,
known as "AI winters." Nonetheless, the foundational ideas from the Dartmouth workshop remained
influential, ensuring AI’s eventual resurgence. The workshop’s long-term impact is undeniable, setting
the stage for innovations in expert systems, symbolic reasoning, deep learning, and robotics. Today, AI
is a transformative force in healthcare, finance, autonomous systems, and entertainment. The ideas
proposed by McCarthy, Minsky, Newell, and their colleagues in 1956 have largely materialized, albeit
more slowly than expected. Machines can now learn, adapt, and outperform humans in tasks like
strategic games, medical diagnostics, and language translation. The journey that began at Dartmouth
continues to shape AI’s future as researchers strive to develop more advanced, autonomous systems.
The impact of the historic Dartmouth workshop remains a cornerstone of AI’s evolution, guiding the
ongoing quest to create truly intelligent machines.
3.Early enthusiasm, great expectations (1952–1969)

The early years of AI saw remarkable progress despite limited computing power. Initially, computers
were believed to be capable of only basic arithmetic, so any demonstration of intelligence by a machine
was groundbreaking. AI pioneers such as John McCarthy, Marvin Minsky, Allen Newell, and Herbert
Simon faced skepticism but systematically proved that machines could perform intelligent tasks.
McCarthy referred to this era as the ―Look, Ma, no hands!‖ period, emphasizing the excitement
surrounding AI’s capabilities. A major achievement of the time was Newell and Simon’s General
Problem Solver (GPS), which mimicked human problem-solving and supported the Physical Symbol
System Hypothesis, suggesting that intelligence requires symbol manipulation. Symbolic AI dominated
early AI research, but alternative approaches like connectionism and neural networks later challenged it.
IBM’s AI research also made significant strides, with Herbert Gelernter’s Geometry Theorem Prover
solving mathematical problems and Arthur Samuel’s checkers-playing program demonstrating machine
learning by improving its performance autonomously. McCarthy’s contributions included developing
Lisp, a dominant AI programming language, and introducing time-sharing to optimize computing
resources. His Advice Taker concept anticipated future AI reasoning systems. Minsky, on the other
hand, focused on practical AI system implementations. McCarthy established Stanford’s AI lab, which
emphasized logic-based AI, while MIT’s research explored controlled environments called
"microworlds," such as the blocks world. These projects led to advances in robotics, AI planning, and
natural language understanding, with contributions from Terry Winograd and Scott Fahlman. Parallel
to symbolic AI, neural networks progressed with work by Warren McCulloch, Walter Pitts, Bernie
Widrow, and Frank Rosenblatt, whose perceptron model could learn simple patterns. However,
limitations in perceptrons, highlighted by Minsky and Seymour Papert in 1969, led to a temporary
decline in neural network research. Early AI faced challenges in replicating human cognition,
struggling with tasks requiring common sense and real-world adaptability. By the 1970s, AI labs were
established worldwide, supported by government and private funding. AI applications expanded to
game playing, medical diagnosis, and automated reasoning, though early methods reached limitations,
leading to new approaches. Symbolic AI was eventually challenged by machine learning and neural
networks, which made a resurgence. Despite setbacks, AI pioneers laid the groundwork for modern
advancements in deep learning, natural language processing, and robotics. AI evolved beyond
microworlds into real-world applications, yet its fundamental goal remains the same: to create
intelligent, adaptable machines capable of reasoning, learning, and interacting with the world in
meaningful ways.

A dose of reality (1966–1973)

From the beginning, AI researchers were highly optimistic about their progress, making bold
predictions. In 1957, Herbert Simon claimed that machines could think, learn, and create, predicting
that within ten years, a computer would be a chess champion and prove a mathematical theorem.
However, these achievements took around 40 years instead of 10. Early AI systems performed well on
simple problems but failed on more complex tasks, leading to significant setbacks. One major difficulty
was the lack of subject knowledge in early AI programs. These systems relied on syntactic
manipulations without understanding the actual meaning of the content. A notable failure was early
machine translation, heavily funded after the 1957 Sputnik launch. Researchers believed that simple
grammatical transformations and word-for-word replacements would be sufficient, but translation
required deeper contextual understanding. This became evident when an AI system translated ―the
spirit is willing but the flesh is weak‖ into Russian and back into English as ―the vodka is good but the
meat is rotten.‖ In 1966, an advisory report found no substantial progress in machine translation,
leading to the cancellation of U.S. government funding. However, machine translation later improved
and became widely used in various domains.
Another significant challenge AI researchers faced was the intractability of many problems. Early AI
programs relied on brute-force methods, testing different sequences of steps until finding a solution.
While effective in simple environments with few objects and actions, these methods failed to scale in
complex real-world scenarios. Initially, researchers believed that increasing computational power
would solve this issue, but computational complexity theory later showed that many AI problems were
inherently difficult to solve efficiently. The optimism surrounding resolution theorem proving also
faded when AI struggled with complex theorems, demonstrating that finding a solution in principle did
not guarantee practical success.
Early attempts at machine evolution, now known as genetic algorithms, also faced difficulties.
Researchers in the late 1950s tried evolving programs through random mutations and selection,
expecting improved solutions. However, despite thousands of hours of computing, little progress was
made due to weak representation methods. Modern genetic algorithms have since improved, but early
failures revealed the limitations of naive approaches. The 1973 Lighthill report further criticized AI’s
inability to address real-world complexity, leading the British government to cut most AI research
funding.
A third fundamental limitation was the weakness of early AI structures. In 1969, Marvin Minsky and
Seymour Papert’s Perceptrons showed that simple neural networks had severe limitations, such as
being unable to distinguish whether two inputs were different. Although their findings did not apply to
more complex multilayer networks, research on neural networks nearly stopped. Ironically, the
backpropagation algorithm, which later revived neural networks, had already been discovered in 1969
but was ignored until the 1980s.
Despite early failures, AI research persisted, and foundational ideas remained influential. Though AI
faced multiple ―winters‖ due to unrealistic expectations, advancements in computation, data, and
algorithms eventually revived the field. Challenges once thought insurmountable, such as natural
language processing and intelligent decision-making, are now being addressed with modern AI
techniques. While early AI pioneers were overly optimistic, their vision paved the way for today’s AI
breakthroughs.

Knowledge-based systems: The key to power? (1969–1979)

During the first decade of AI research, problem-solving was largely seen as a general-purpose search
process, where simple reasoning steps were strung together to form solutions. However, these weak
methods did not scale well for complex problems. An alternative approach emerged, emphasizing
domain-specific knowledge to allow for larger reasoning steps. This was exemplified by the
DENDRAL program, developed at Stanford by Ed Feigenbaum, Bruce Buchanan, and Joshua
Lederberg. DENDRAL was designed to infer molecular structures from mass spectrometry data.
Initially, it attempted to generate all possible molecular structures and compare predicted spectra with
observed spectra, but this approach proved computationally intractable. The researchers then
incorporated expert knowledge from chemists, using heuristic rules to recognize common molecular
substructures. By mapping general theoretical knowledge into efficient problem-solving rules,
DENDRAL became the first successful knowledge-intensive AI system.
Building on this success, Feigenbaum and colleagues expanded their research into expert systems,
leading to the Heuristic Programming Project (HPP). The most notable product of this effort was
MYCIN, a system for diagnosing blood infections. Unlike DENDRAL, MYCIN’s rules could not be
derived from a theoretical model but were instead acquired from expert interviews, textbooks, and case
studies. Additionally, MYCIN introduced a certainty factor system to handle the uncertainty inherent in
medical diagnosis. It performed comparably to medical experts and outperformed junior doctors,
demonstrating the effectiveness of expert systems in complex domains.
The importance of domain knowledge was also evident in natural language understanding. Winograd’s
SHRDLU system excited researchers with its ability to process natural language but was limited to the
blocks world, making it unsuitable for general applications. Eugene Charniak and others argued that
robust language understanding required extensive world knowledge. At Yale, Roger Schank
emphasized this point, controversially claiming that ―there is no such thing as syntax‖ to highlight the
need for knowledge-driven approaches. Schank and his students developed programs that focused on
knowledge representation and reasoning rather than syntax. These efforts explored stereotypical
situations, human memory organization, and understanding plans and goals.
The growing demand for real-world AI applications led to the development of better knowledge
representation schemes. Various approaches emerged, including logic-based systems like Prolog in
Europe and the PLANNER family in the U.S. Others followed Marvin Minsky’s concept of frames,
structuring facts about objects and events into hierarchical taxonomies similar to biological
classifications. This shift from general problem-solving to specialized knowledge-driven systems
significantly improved AI’s capabilities, paving the way for modern expert systems and natural
language processing advancements.

AI becomes an industry (1980–present)

The first successful commercial expert system, R1, was launched by Digital Equipment Corporation in
1982, saving the company $40 million annually by 1986. By 1988, major corporations like DEC and
DuPont had deployed numerous expert systems, with AI gaining widespread corporate interest. In 1981,
Japan’s ―Fifth Generation‖ project aimed to develop intelligent computers using Prolog, prompting the
U.S. to form the MCC consortium and Britain to reinstate AI funding through the Alvey report. Despite
high ambitions, these projects failed to meet expectations. The AI industry grew rapidly in the 1980s,
reaching billions in value by 1988. However, unrealistic promises led to the ―AI Winter,‖ causing
many AI companies to collapse.
AI adopts the scientific method (1987–present)
Artificial intelligence (AI) has undergone a significant transformation in recent years, shifting from
intuition-driven approaches to methodologies rooted in rigorous mathematical theories and empirical
validation. Early AI research often sought to develop entirely new computational paradigms, isolating
itself from fields such as control theory, statistics, and optimization. However, this isolationism is now
being abandoned in favor of an integrative approach that leverages established mathematical
foundations. David McAllester (1998) noted that AI researchers now recognize the necessity of
integrating machine learning with information theory, uncertain reasoning with stochastic modeling,
search with classical optimization, and automated reasoning with formal methods. The field has fully
embraced the scientific method, where hypotheses must be tested through rigorous empirical
experiments, and results are subjected to statistical analysis to ensure their significance. Shared
repositories of test data and code have further facilitated reproducibility and validation of experimental
results.
One striking example of this shift is in speech recognition. In the 1970s, AI researchers experimented
with various architectures and approaches, many of which were fragile and ad hoc, demonstrated only
on a few selected examples. Over time, hidden Markov models (HMMs) emerged as the dominant
technique because they provided a mathematically rigorous framework that leveraged decades of
research from other fields. These models are trained on vast amounts of real speech data, ensuring
robustness and continuous improvement through rigorous testing. Similar progress has occurred in
machine translation, where initial enthusiasm for statistical models in the 1950s waned in the 1960s but
resurged in the late 1990s, now forming the backbone of modern translation systems. Neural networks
have also followed this trajectory. While the 1980s saw exploratory research aimed at understanding
their differences from traditional techniques, the field has matured to the point where neural networks
are now directly compared with statistical and pattern recognition methods. This has enabled the most
effective techniques to be applied to practical applications, fostering the rapid expansion of data mining
technologies.
The resurgence of probabilistic reasoning in AI is another key development, largely influenced by
Judea Pearl’s (1988) work on Bayesian networks. In the past, probabilistic reasoning systems struggled
with efficiency and accuracy, but Bayesian networks provided a structured way to represent uncertainty
and reason effectively under it. This approach led to normative expert systems that act rationally
according to decision theory rather than attempting to imitate human cognitive processes. Microsoft
Windows, for instance, incorporates normative diagnostic expert systems for problem correction.
Similar advancements have taken place in robotics, computer vision, and knowledge representation.
The growing mathematical sophistication of these fields, coupled with a better understanding of
complexity properties, has resulted in more structured research agendas and more reliable methods.
While increased formalization in the 1990s led some AI subfields, such as robotics and vision, to
become somewhat isolated from mainstream AI, recent reintegration—especially through machine
learning—has bridged these gaps. The benefits of this renewed connectivity are evident in the
improved performance and applicability of AI technologies across multiple domains. AI's evolution
towards a more rigorous, interdisciplinary, and scientifically grounded approach continues to drive
innovation, leading to increasingly robust and impactful real-world applications.

The emergence of intelligent agents (1995–present)

Encouraged by progress in solving AI subproblems, researchers have begun revisiting the challenge of
building complete intelligent agents. A prominent example is the SOAR architecture developed by
Allen Newell, John Laird, and Paul Rosenbloom, which aims to integrate different cognitive functions
into a unified system. The Internet has emerged as a crucial environment for AI agents, with AI
technologies driving search engines, recommender systems, and web aggregators. The prevalence of
AI-powered systems has even introduced the "-bot" suffix into everyday language. One key realization
from efforts to build complete agents is that AI subfields must be more interconnected. Sensory
systems, including vision, sonar, and speech recognition, provide imperfect information, necessitating
reasoning and planning methods capable of handling uncertainty. This perspective has led AI to engage
more closely with control theory and economics, as these fields also study decision-making agents.
Advances in robotic vehicle control illustrate this interdisciplinary approach, combining sensor
improvements, localization, mapping, and high-level planning.
Despite these advances, some AI pioneers, including John McCarthy, Marvin Minsky, Nils Nilsson,
and Patrick Winston, have voiced dissatisfaction with the field’s focus on narrow, application-specific
AI, such as chess-playing programs and self-driving cars. They argue that AI should return to its
original goal of developing machines that think, learn, and create, an effort they term Human-Level AI
(HLAI). The first HLAI symposium took place in 2004, with researchers acknowledging the need for
extensive knowledge bases to support human-like reasoning. A closely related idea is Artificial General
Intelligence (AGI), which seeks to develop a universal algorithm for learning and acting in any
environment. AGI traces its origins to Ray Solomonoff's work in the 1960s and gained formal
recognition with its first conference in 2008, followed by the establishment of the Journal of Artificial
General Intelligence. A significant concern within AGI research is ensuring that AI systems align with
human values and act in ways beneficial to society, leading to discussions on "Friendly AI."
Researchers like Eliezer Yudkowsky and Stephen Omohundro emphasize the importance of designing
AI with built-in safeguards to prevent unintended consequences. As AI continues evolving, balancing
specialized applications with the broader vision of general intelligence remains an ongoing challenge.
The pursuit of AGI and HLAI requires advances in learning algorithms, reasoning mechanisms, and
knowledge representation, with researchers striving to create AI systems capable of human-like thought,
creativity, and adaptability.
The availability of very large data sets (2001–present)
For much of computer science’s 60-year history, the focus has been on algorithms as the primary
subject of study. However, recent developments in AI suggest that, for many problems, the availability
of vast amounts of data can be more critical than the choice of algorithm. The growing accessibility of
massive datasets—such as trillions of words of English text, billions of images from the Web, and
genomic sequences—has shifted attention toward data-driven methods. Instead of relying solely on
sophisticated algorithms, researchers have demonstrated that even simple techniques can achieve
outstanding results when given enough data.
A pivotal example of this shift is Yarowsky’s (1995) work on word-sense disambiguation. Rather than
relying on human-labeled examples, he showed that high accuracy (above 96%) could be achieved
using only dictionary definitions and a large corpus of unannotated text. The model could bootstrap
patterns from the data to correctly label new examples. Banko and Brill (2001) further demonstrated
that increasing the amount of training data has a greater impact on performance than selecting the best
algorithm. Their research showed that a mediocre algorithm trained on 100 million words of data
outperformed the best-known algorithm trained on just 1 million words, highlighting the power of
large-scale data.
Another compelling example is Hays and Efros’s (2007) work on filling in missing parts of
photographs. Their algorithm, which searches a database for matching image patches, performed
poorly with only ten thousand reference images but improved significantly when the dataset expanded
to two million. This underscores how AI performance often depends more on dataset size than on the
intricacy of the algorithm itself.
Such research suggests that AI’s long-standing ―knowledge bottleneck‖—the challenge of encoding
vast amounts of knowledge into a system—may be overcome through data-driven learning rather than
manual knowledge engineering. With sufficient data, learning algorithms can extract meaningful
patterns and generalize effectively across applications. This trend has fueled optimism about AI’s
progress, with reports suggesting that the field is moving beyond its historical "AI Winter" into a new
era of rapid advancements. Kurzweil (2005) highlights how AI is now deeply embedded in industries
worldwide, powering countless applications across domains. These developments indicate that AI's
future will be increasingly driven by data availability, making large-scale learning a cornerstone of
future advancements.
AGENTS AND ENVIRONMENTS
An agent is anything that can be viewed as perceiving its environment through
sensors and acting upon that environment through actuators. This simple idea is
illustrated in Figure 2.1.
A human agent has eyes, ears, and other organs for sensors and hands, legs, vocal
tract, and so on for actuators. A robotic agent might have cameras and infrared range
finders for sensors and various motors for actuators. A software agent receives
keystrokes, file contents, and network packets as sensory inputs and acts on the
environment by displaying on the screen, writing files, and sending network packets.
We use the term percept to refer to the agent’s perceptual inputs at any given instant.
An agent’s percept sequence is the complete history of everything the agent has ever
perceived. In general, an agent’s choice of action at any given instant can depend on
the entire percept sequence observed to date, but not on anything it hasn’t perceived.
By specifying the agent’s choice of action for every possible percept sequence, we
have said more or less everything there is to say about the agent. Mathematically
speaking, we say that an agent’s behavior is described by the agent function that maps
any given percept sequence to an action.
The agent function describes an agent’s behavior by mapping percept sequences to
actions, forming a potentially infinite table unless bounded. This function can be
externally characterized by testing all possible percepts and recording responses.
However, the agent function is distinct from the agent program, which is the actual
implementation running on a physical system. The agent function is an abstract
mathematical concept, whereas the agent program is its concrete realization.
The vacuum-cleaner world is a simple, two-location environment (squares A and B)
where a vacuum agent perceives its location and whether there is dirt. The agent can
move left, move right, suck up dirt, or do nothing. A basic agent function follows a
simple rule: if the current square is dirty, it sucks; otherwise, it moves to the other
square. This behavior can be represented in a table (Figure 2.3) and implemented
through an agent program (Figure 2.8). This example illustrates the distinction
between an abstract agent function and its concrete implementation.
Looking at Figure 2.3, we see that various vacuum-world agents can be defined
simply by filling in the right-hand column in various ways. The obvious question,
then, is this: What is the right way to fill out the table? In other words, what makes an
agent good or bad, intelligent or stupid?
GOOD BEHAVIOR: THE CONCEPT OF RATIONALITY
A rational agent is one that does the right thing—conceptually speaking, every entry in the
table for the agent function is filled out correctly. Obviously, doing the right thing is better
than doing the wrong thing, but what does it mean to do the right thing?
An agent’s performance is judged by the consequences of its actions in the environment, not by its own
perception of success. The performance measure evaluates how desirable the sequence of environment
states is, rather than relying on the agent’s opinion. Designing an effective performance measure is
crucial but challenging, as poorly designed metrics can lead to unintended behaviors. For example, a
vacuum-cleaner agent judged by the amount of dirt cleaned might exploit the system by repeatedly
making the floor dirty and cleaning it again. A better measure would reward maintaining cleanliness
over time while considering energy use and noise. Performance measures should align with the actual
goals of the environment rather than assumptions about how the agent should behave. Even with well-
designed measures, complex issues arise, such as how to define and prioritize cleanliness over time.
Similar dilemmas appear in broader contexts, questioning whether a consistent, mediocre performance
is preferable to one with highs and lows. These considerations extend to philosophical debates about
quality of life and economic distribution. Is a stable but unremarkable existence better than one with
extreme fluctuations? Should an economy aim for universal moderate wealth or allow disparities where
some thrive while others struggle? Such questions highlight the difficulty of defining optimal
performance, whether for artificial agents or human societies.

Rationality
What is rational at any given time depends on four things:
• The performance measure that defines the criterion of success.
• The agent’s prior knowledge of the environment.
• The actions that the agent can perform.
• The agent’s percept sequence to date.
This leads to a definition of a rational agent:

For each possible percept sequence, a rational agent should select an action that is expected to
maximize its performance measure, given the evidence provided by the percept sequence and whatever
built-in knowledge the agent has.
Consider the simple vacuum-cleaner agent that cleans a square if it is dirty and moves to the other
square if not; this is the agent function tabulated in Figure 2.3. Is this a rational agent? That depends!
First, we need to say what the performance measure is, what is known about the environment, and
what sensors and actuators the agent has. Let us assume the following:
• The performance measure awards one point for each clean square at each time step, over a ―lifetime‖
of 1000 time steps.
• The ―geography‖ of the environment is known a priori (Figure 2.2) but the dirt distribution and the
initial location of the agent are not. Clean squares stay clean and sucking cleans the current square. The
 Left and Right actions move the agent left and right except when this would take the agent outside the
environment, in which case the agent remains where it is.
• The only available actions are Left, Right, and Suck.
• The agent correctly perceives its location and whether that location contains dirt.

We claim that under these circumstances the agent is indeed rational; its expected performance is at
least as high as any other agent’s.
One can see easily that the same agent would be irrational under different circumstances. For example,
once all the dirt is cleaned up, the agent will oscillate needlessly back and forth; if the performance
measure includes a penalty of one point for each movement left or right, the agent will fare poorly. A
better agent for this case would do nothing once it is sure that all the squares are clean. If clean squares
can become dirty again, the agent should occasionally check and re-clean them if needed. If the
geography of the environment is unknown, the agent will need to explore it rather than stick to squares
A and B.

Omniscience, learning, and autonomy

Rationality differs from omniscience, as a rational agent makes decisions based on available percepts, whereas an
omniscient agent would know actual outcomes in advance. Since omniscience is impossible, rationality focuses on
maximizing expected performance rather than perfection. For example, if someone crosses an empty street and is
unexpectedly hit by falling debris, their decision was still rational given the information they had. Rational agents
must also avoid underinformed actions, such as crossing a road without looking. Information gathering, such as
looking before crossing, is essential to rationality because it improves decision-making by modifying future percepts.
Similarly, exploration helps agents navigate unknown environments, as seen in a vacuum-cleaning agent that must
first explore before cleaning efficiently. Rational agents must also learn from their experiences. While some agents
rely solely on built-in knowledge, such as dung beetles or sphex wasps, they fail when their predetermined behaviors
are disrupted. The dung beetle, for example, continues its task even when its dung ball is removed, demonstrating an
inability to adapt. The sphex wasp follows a rigid sequence of actions, failing to recognize changes in its
environment. Such rigid behavior indicates a lack of autonomy, as the agent depends entirely on prior knowledge
rather than real-time percepts. A truly rational agent should be autonomous, learning from experience to improve
performance. For instance, a vacuum-cleaning agent that learns patterns of dirt accumulation will outperform one
that blindly follows a preset routine. While complete autonomy is unnecessary from the start, agents should be
designed with some built-in knowledge, similar to how animals have innate reflexes to survive until they learn more.
Over time, an intelligent agent’s behavior should become independent of its initial programming, allowing it to
succeed in diverse environments. This adaptability makes learning a crucial component of rationality. With a clear
understanding of rationality, the next step is to consider task environments, which define the problems that rational
agents must solve. By specifying task environments and their variations, designers can develop suitable agent
programs tailored to different scenarios. The characteristics of a task environment directly influence how an agent
should be designed to achieve optimal performance. Understanding these variations helps in constructing rational
agents that can function effectively across multiple settings.

Specifying the task environment

The task environment of an agent consists of its Performance measure, Environment, Actuators, and Sensors, known
as the PEAS framework. Defining this framework is the first step in designing an intelligent agent. While the
vacuum-cleaner world was a simple example, a more complex case is an automated taxi driver. Though fully
autonomous taxis are not yet a reality, driving is an open-ended task with endless unpredictable scenarios. The PEAS
framework for a self-driving taxi includes performance measures like safety and efficiency, an environment of roads
and traffic, actuators such as steering and braking, and sensors like cameras and GPS. Clearly defining these
elements helps develop agents capable of handling real-world challenges.Figure 2.4 summarizes the PEAS
description for the taxi’s task environment. We discuss each element in more detail in the following paragraphs.
The performance measure for an automated driver includes reaching the correct destination efficiently while
minimizing fuel use, wear, trip time, and costs. It should also follow traffic laws, ensure safety and passenger
comfort, and maximize profits. Since some goals conflict, tradeoffs are necessary to balance efficiency, safety, and
cost-effectiveness.
The driving environment for an automated taxi includes diverse road types, from rural lanes to multi-lane highways,
and various obstacles like traffic, pedestrians, animals, and road hazards. It must also interact with passengers and
adapt to regional conditions, such as weather variations or differing traffic rules. The complexity increases if the taxi
must handle multiple environments, such as snowy roads in Alaska or left-side driving in Britain. A more restricted
environment simplifies the design challenge, while broader adaptability requires advanced capabilities.
An automated taxi requires actuators similar to those used by human drivers, including controls for acceleration,
steering, and braking. Additionally, it needs a way to communicate with passengers through a display screen or voice
synthesizer. To enhance coordination on the road, it may also require a system to interact with other vehicles,
whether for cooperative driving or signaling intentions.
An automated taxi requires various sensors to navigate and operate efficiently. It needs controllable video cameras to
see the road, along with infrared or sonar sensors to detect nearby obstacles. A speedometer helps prevent speeding,
while an accelerometer ensures proper handling, especially on curves. Standard vehicle sensors monitor the engine,
fuel, and electrical systems. GPS is essential for navigation, preventing the taxi from getting lost. Additionally, a
keyboard or microphone allows passengers to input their destination.
In Figure 2.5, we have sketched the basic PEAS elements for a number of additional agent types. The distinction
between "real" and "artificial" environments is less important than the complexity of the agent's interactions with its
environment, percept sequence, and performance measure. Some artificial environments, like programs operating
through keyboard input and screen output, can be as complex as real-world scenarios. Conversely, some real
environments, like a conveyor belt inspection system, can be simple due to controlled conditions, predefined object
types, and limited actions such as accepting or rejecting parts.
Some software agents, or softbots, operate in highly complex and dynamic environments like the Internet. A web-
based softbot, for example, might scan news sources, present relevant content to users, and manage advertising to
generate revenue. To succeed, it must process natural language, learn user and advertiser preferences, and adapt
dynamically to changes, such as lost connections or new sources. The Internet, with its vast interactions among
artificial and human agents, presents challenges comparable to those in the physical world.

Properties of task environments

AI task environments vary widely but can be categorized using a few key dimensions that influence agent design and
implementation techniques. These dimensions help determine the most suitable approach for creating intelligent
agents. By analyzing different task environments, we can better understand how these factors shape agent behavior.
While the definitions are introduced informally here, later chapters provide more precise explanations and examples.

1.Fully observable vs. partially observable

Fully Observable Environment

An environment is fully observable when an agent's sensors provide complete and relevant information about the
entire environment at every moment. This ensures that the agent does not need to maintain an internal memory to
track past states.
Examples:
A chess game is fully observable because the entire board state is visible at all times.
A tic-tac-toe game is also fully observable, as all pieces and moves are in plain sight.
Partially Observable Environment
An environment is partially observable when an agent has limited or incomplete information due to sensor
constraints, environmental complexity, or missing data.
Examples:
A self-driving car operates in a partially observable environment because it cannot see around corners or predict
other drivers' intentions.
A poker game is partially observable since players cannot see their opponents' cards.
2. Single agent vs. multiagent
Single-Agent Environment
A single-agent environment involves only one decision-making entity (agent) operating within the environment. The
agent's actions do not need to consider responses from other agents, making decision-making relatively
straightforward.
Examples:
Maze-solving robot: The robot navigates a maze without interference from other agents.
A multi-agent environment involves multiple decision-making agents that interact with each other, either
cooperatively or competitively. Each agent's actions can affect the environment and influence other agents' decisions.
Example: Online multiplayer games where teams compete but members collaborate within their teams.
3.Deterministic vs. stochastic

Deterministic Environment
An environment is deterministic if the outcome of an action is completely predictable and does not involve any
randomness.
Given a specific state and action, the next state is always the same.
Example: Chess—Every move leads to a specific, predictable result without randomness.
Stochastic Environment
An environment is stochastic if there is some degree of randomness or uncertainty in the outcomes.
The same action in the same state might lead to different results due to probabilistic factors.
Example: Poker or Self-Driving Cars—Uncertainty in opponent behavior, road conditions, or external disturbances
makes the environment unpredictable.
3. Episodic vs. sequential:
Episodic Environment
Each decision is independent of past and future decisions.
The environment does not carry over state information between episodes.
Example: Image classification—Classifying an image does not depend on past images or affect future ones.
Sequential Environment
Actions influence future states and decisions.
The agent needs to consider how current actions affect future rewards.
Example: Chess or Self-Driving Cars—A move affects the future game state, and changing lanes impacts future
driving conditions.
4.Static vs. dynamic

Static Environment
The environment remains unchanged while the agent is making a decision.
The agent does not need to worry about time constraints or external changes.
Example: Chess—The board remains the same until the player makes a move.
Dynamic Environment
The environment changes over time, even if the agent does nothing.
The agent must account for real-time changes and possibly act under time constraints.
Example: Self-driving cars—Traffic, pedestrians, and road conditions continuously change, requiring real-time
decisions.
5.Discrete vs. continuous:

Discrete Environment
The environment has a finite number of states and actions.
The agent moves between distinct states, like steps in a grid.
Example: Chess—There are a limited number of board positions and legal moves.
Continuous Environment
The environment has an infinite or highly granular number of states and actions.
Actions and states vary smoothly over a range, requiring more complex calculations.
Example: Self-driving cars—The car can move in any direction at varying speeds, and the environment includes
infinitely variable positions, angles, and velocities.
6. Known vs. unknown:

Known Environment
The agent has complete knowledge of the environment's rules and dynamics.
The agent can plan optimal actions based on this knowledge.
Example: Chess—The agent knows all possible moves and outcomes in advance.
Unknown Environment
The agent does not have full knowledge of how the environment works.
The agent must learn through exploration (e.g., Reinforcement Learning).
Example: Self-driving cars—The agent must learn how traffic behaves in different scenarios.

As one might expect, the hardest case is partially observable, multiagent, stochastic, sequential, dynamic, continuous,
and unknown. Taxi driving is hard in all these senses, except that for the most part the driver’s environment is known.
Driving a rented car in a new country with unfamiliar geography and traffic laws is a lot more exciting. Figure 2.6
lists the properties of a number of familiar environments. Note that the answers are not always cut and dried. For
example, we describe the part-picking robot as episodic, because it normally considers each part in isolation. But if
one day there is a large batch of defective parts, the robot should learn from several observations that the distribution
of defects has changed, and should modify its behavior for subsequent parts

THE STRUCTURE OF AGENTS

So far, we have discussed agents based on their behavior—how they act given a sequence of percepts. Now, we need
to examine their internal workings. AI focuses on designing an agent program that implements the agent function,
which maps percepts to actions. This program runs on a computing architecture equipped with sensors and actuators
to interact with the environment.

AGENT = ARCHITECTURE + PROGRAM .

The agent program must be suitable for its architecture. If the program suggests actions like Walk, the system must
have legs. The architecture can range from a simple PC to a robotic car with sensors and onboard computers. Its role
is to process percepts from sensors, run the program, and send action commands to actuators.

Agent programs

All agent programs follow the same structure: they take the current percept as input and return an action to the
actuators. The key difference is that the agent function considers the entire percept history, while the agent program
only processes the current percept. If past percepts are needed for decision-making, the agent must store them in
memory.
We describe the agent programs in the simple pseudocode language
For example, Figure 2.7 shows a rather trivial agent program that keeps track of the percept sequence and then uses it
to index into a table of actions to decide what to do.
The table—an example of which is given for the vacuum world in Figure 2.3—represents explicitly the agent
function that the agent program embodies. To build a rational agent in this way, we as designers must construct a
table that contains the appropriate action for every possible percept sequence.
The table-driven approach to agent construction is impractical due to its enormous size. Given a percept set P and a
lifetime T, the lookup table would require exponential entries.
For example, an automated taxi processing visual input would need over 10 250,000,000,000entries for just an hour of
driving. Even chess would require at least 10150 entries, far exceeding the 1080atoms in the universe.
This approach fails because:
 Storage: No physical agent can store such a massive table.
 Design Time: Creating the table manually is impossible.
 Learning: No agent can experience and memorize all necessary entries.
 Guidance: Even with a feasible table, there’s no clear way to define its contents.
Thus, a more efficient and adaptive approach is needed for AI agents.

Although TABLE-DRIVEN-AGENT works, its impractical size makes it unusable. The challenge in AI is to design
compact programs that produce rational behavior without relying on vast lookup tables. History shows this is
possible—square root tables were replaced by a simple Newton’s method program. The key question is whether AI
can achieve the same for general intelligence, and the belief is that it can.

we outline four basic kinds of agent programs that embody the principles underlying almost all intelligent systems:
• Simple reflex agents;
• Model-based reflex agents;
• Goal-based agents; and
• Utility-based agents.

Simple reflex agents

A simple reflex agent selects actions based only on the current percept, ignoring percept history. For example, a
vacuum agent decides based only on its current location and whether there is dirt. This significantly reduces the
number of possible states, making the agent program much smaller than a table-driven alternative. The biggest
reduction comes from ignoring history, cutting possibilities from 4T to just 4.
Simple reflex behavior is also useful in complex environments. For instance, in an automated taxi, if the car ahead
brakes and its brake lights turn on, the agent detects this condition and initiates braking immediately. This is an
example of a condition-action rule, where certain conditions in the environment trigger specific actions. Such rules
allow AI agents to respond quickly and efficiently without requiring complete knowledge of past percepts.

Figure 2.3 is a simple reflex agent, because its decision is based only on the current location and on whether that
location contains dirt. An agent program for this agent is shown in Figure 2.8.
The program in Figure 2.8 is specific to a particular vacuum environment, but a more
general approach involves creating a general-purpose interpreter for condition-action
rules, which can be adapted for various tasks. Figure 2.9 illustrates this approach,
showing how the agent connects percepts to actions using condition-action rules. The
agent’s decision process is depicted with rectangles representing its internal state and
ovals representing background information. The program, shown in Figure 2.10,
includes the INTERPRET-INPUT function, which abstracts the current state from the
percept, and the RULE-MATCH function, which finds the rule that matches the state
description. These rules and matching processes are conceptually implemented
through logic gates in actual implementations.

Simple reflex agents, though straightforward, have limited intelligence. The agent in
Figure 2.10 works only when decisions can be made based on the current percept,
meaning the environment must be fully observable. Partial observability can cause
significant issues, as shown by the braking rule example, where the agent might
misinterpret a car's brake lights. Similarly, a vacuum agent with only a dirt sensor and
no location sensor would encounter problems, such as failing to make progress or
entering infinite loops in partially observable environments.

One solution to this is randomization, which helps escape infinite loops. For example,
if the vacuum agent perceives [Clean], it might randomize its next action, like
choosing randomly between Left or Right. This allows the agent to eventually reach
the other square and complete the task if that square is dirty. While randomization can
help in certain situations, it is generally not rational in single-agent environments. A
more sophisticated, deterministic agent would typically perform better in most cases.
2. Model-based reflex agents

To address partial observability, agents should maintain an internal state that reflects aspects of the
world they cannot currently observe. This internal state is updated based on percept history and helps
the agent track unobserved elements of the environment. For example, in the braking problem, the
agent only needs to remember the previous camera frame to detect when red lights are activated. In
more complex tasks, like changing lanes, the agent must track the locations of other cars that are not
visible all at once. Similarly, for any driving task, an agent needs to remember where its keys are.

To update this internal state, two types of knowledge are required in the agent's program. First, the
agent needs knowledge of how the world evolves independently of its actions—such as the fact that an
overtaking car typically becomes closer to the agent. Second, the agent needs knowledge of how its
actions affect the environment—such as how turning the steering wheel changes the car’s direction or
how driving for a set amount of time results in the car being a certain distance away.

This knowledge of how the world works is called a "model of the world." Agents that use such models
are known as model-based agents. In a model-based reflex agent (as shown in Figure 2.11), the agent
combines its current percept with its internal state to update its description of the world. The key
function, UPDATE-STATE, is responsible for updating the agent's internal state based on its model.

The representation of models and states can vary widely depending on the environment and technology
used in the agent's design. However, it is generally impossible for the agent to precisely determine the
current state of a partially observable environment. Instead, the agent forms a "best guess" about the
world’s current state. For instance, an automated taxi may not see around a large truck blocking its
view but can still make decisions based on the uncertainty of the situation.

The agent’s internal state doesn't have to represent the literal state of the world. It can include aspects
of the agent’s intentions. For example, a taxi may have an internal state representing its intention to fill
up with gas unless it already has enough fuel, even though its current location doesn’t indicate this
need. This distinction highlights that an agent’s internal state often includes not just the environment’s
state but also the agent’s goals or plans, which may be influenced by its perceptions and actions.
3.Goal-based agents

Knowing the current state of the environment isn't always enough for decision-
making. For example, at a road junction, a taxi must decide whether to turn left, right,
or go straight. The correct decision depends on the taxi's goal, such as reaching a
passenger's destination. Therefore, in addition to the current state description, the
agent needs goal information that defines desirable outcomes, like arriving at the
destination. The agent program combines the goal with the world model to choose
actions that achieve this goal.

The structure of a goal-based agent is shown in Figure 2.13. In some cases, achieving
the goal may be straightforward, where a single action leads directly to goal
satisfaction. In more complex situations, such as navigating through twists and turns,
the agent may need to consider a sequence of actions to reach its goal. The AI
subfields of search (Chapters 3-5) and planning (Chapters 10-11) focus on finding
these action sequences.

This decision-making differs from simple condition-action rules, as it involves future

planning—assessing the outcomes of potential actions and whether they will help
achieve the goal. Reflex agents, by contrast, map directly from percepts to actions,
like braking when brake lights are seen. A goal-based agent might reason that if the
car in front has brake lights on, it will slow down, and the agent should brake to avoid
a collision.

While goal-based agents are less efficient than reflex agents, they offer greater
flexibility. Their behavior is driven by explicit knowledge that can be modified. For
example, if it starts raining, the agent can update its knowledge about brake
performance, automatically adjusting its behavior. Reflex agents, however, would
require rewriting multiple condition-action rules. A goal-based agent can easily
change destinations by simply updating its goal, while reflex agents need their rules
rewritten to accommodate a new destination.

Utility-based agents

Goals alone are insufficient for generating high-quality behavior in most

environments. While achieving a goal like getting a taxi to its destination is important,
some action sequences are better than others in terms of speed, safety, reliability, or
cost. Goals merely distinguish between "happy" and "unhappy" states, but a more
refined performance measure is needed to compare different world states by how
desirable they are to the agent. This is where the concept of utility comes in.

Utility is a performance measure used to evaluate and compare the desirability of

various states. An agent’s utility function is an internal representation of this
measure. If the utility function aligns with the external performance measure, the
agent will act rationally by maximizing its utility. This approach offers more
flexibility than simple goal-based agents and supports learning.

Utility-based agents are particularly useful in two cases where goals alone are
insufficient. First, when goals conflict, such as speed versus safety, the utility function
helps specify the appropriate tradeoff. Second, when multiple uncertain goals exist,
the utility function allows the agent to weigh the likelihood of success against the
importance of each goal.
In real-world scenarios, partial observability and stochasticity mean agents often must
make decisions under uncertainty. A rational utility-based agent chooses the action
that maximizes the expected utility of possible outcomes. This means the agent
calculates the expected utility by considering the probabilities of each outcome and
their respective utilities.

A rational agent, according to its utility function, will maximize expected utility by
following a general-purpose algorithm that doesn’t depend on the specifics of the
utility function. The utility-based agent structure is illustrated in Figure 2.14, and
agents designed to handle uncertainty in stochastic or partially observable
environments..

However, designing utility-based agents is complex. Although the goal of maximizing

expected utility sounds simple, the agent must model and track its environment, a task
requiring advanced techniques in perception, representation, reasoning, and learning.
Therefore, creating such agents involves substantial research and effort.

Learning agents

Agent programs, which we’ve discussed for selecting actions, don’t explain how these
programs come into existence. Turing (1950) first proposed a more efficient method
for building intelligent machines: learning. He believed manually programming
machines was inefficient and suggested building learning machines and teaching them
instead. Today, learning is a preferred method for creating state-of-the-art AI systems.
Learning also allows agents to adapt to unknown environments and become more
competent than they were initially.

Learning agents are built from four key components. The performance element
selects external actions based on percepts, and the learning element is responsible for
improving the performance element by modifying its actions based on feedback. The
performance element, which we have previously considered the whole agent, operates
by taking in percepts and deciding actions. The learning element, in contrast, receives
feedback from the critic, which tells the agent how well it's performing according to a
fixed performance standard.

The critic is crucial because percepts alone don’t indicate whether the agent’s actions
are correct. For instance, a chess program might recognize that it has checkmated an
opponent, but the percept alone doesn’t tell the agent that this is a successful outcome.
The performance standard must be fixed and separate from the agent itself, as the
agent should not modify it to fit its behavior.

The problem generator is the final component, responsible for suggesting actions
that lead to new, informative experiences. The performance element tends to stick
with actions it already knows to be good, but the problem generator encourages the
agent to explore new actions, potentially discovering better strategies in the long term.
This idea mirrors scientific experimentation, where scientists, like Galileo, test
suboptimal actions to gain valuable insights, not for immediate gains but to improve
future understanding.

How the components of agent programs work

Agent programs are designed to answer fundamental questions like "What is the
world like now?" and "What action should I take?" The components of these
programs must represent the environment in a way that enables intelligent behavior.
These representations can vary in complexity and expressive power, falling into three
general categories: atomic, factored, and structured representations.

Atomic representations are the simplest form of environment modeling. In this type,
the state of the world is treated as a single, indivisible unit. For instance, when
considering a driving problem, the world might be reduced to the city the agent is
in—an atom of knowledge with no internal structure. Many AI algorithms, such as
those used in search problems, game-playing, and Hidden Markov Models (HMM),
rely on atomic representations, which simplify the problem by treating world states as
isolated entities without considering their internal characteristics.

Moving to a factored representation, the world state is broken down into multiple
variables or attributes, each with a value. In the driving example, this might include
the amount of gas in the tank, the GPS coordinates, whether the oil warning light is
on, and other relevant details. Factored representations allow for more detailed
modeling of the environment, where attributes can be known or unknown.
Uncertainty is manageable, and different attributes may be updated independently.
This kind of representation is used in various AI techniques like constraint
satisfaction, propositional logic, planning, Bayesian networks, and machine learning,
where the relationships between attributes are essential for making informed
decisions.

The structured representation is even more sophisticated, explicitly modeling

objects and their relationships. In this approach, agents don't just track individual
attributes; they recognize objects and the interactions between them. For example, in
the driving scenario, a structured representation would capture the situation of a truck
reversing into a dairy farm driveway, with a cow blocking its path. This complexity
goes beyond simply tracking state variables—objects and their interactions (like
trucks and cows) are considered integral to understanding the world. Structured
representations are fundamental in areas like relational databases, first-order logic,
knowledge-based learning, and natural language understanding.

The progression from atomic to factored to structured representations illustrates

increasing expressiveness. A more expressive representation can handle all that a less
expressive one can, but also capture more complex relationships and details. For
instance, structured representations can succinctly describe complex situations like
chess rules using first-order logic, while propositional logic would require thousands
of pages to convey the same information. However, greater expressiveness comes
with trade-offs: reasoning and learning become more computationally demanding and
complex.

In practice, intelligent systems may need to operate across these levels of

representation simultaneously. They may use atomic representations for quick
decisions where simplicity is sufficient, factored representations when more detail is
needed, and structured representations to handle complex relationships. The challenge
lies in balancing the expressive power of the representation with the need for efficient
reasoning and decision-making, especially in dynamic, real-world environments.