UNIT – I

INTRODUCTION TO AI
The term ‗Artificial Intelligence‘ was coined by John McCarthy in 1956. This term is used to
describe the ‗Intelligence‘ demonstrated by a system. It plays a key role in problem solving.
Production systems help in searching for a solution to a problem. Certain heuristic algorithms
have been developed to solve a problem within the scheduled time and space. Artificial
Intelligence (AI) involves the task of creating intelligent computers that can perform
activities similar to those performed by a human being, but more efficiently. The main
objective of AI is to create an information processing theory, which can help develop
intelligent computers. Currently, AI is used in various areas, such as games, natural language
processing, vision processing, speech processing, robotics and expert system. Banks use
software systems that are created using AI to organize operations, invest in stocks and
manage property.
Artificial Intelligence (AI) is the branch of computer science that deals with the creation of
computers with human skills. It recognizes its surrounding and initiates actions that maximize
its change of success. The term AI is used to describe the ‗intelligence‘ that the system
demonstrates. Tools and insights from fields, including linguistics, psychology, computer
science, cognitive science, neuroscience, probability, optimization and logic are used. Since
the human brain uses many techniques to plan and cross-check facts system integrations is
also essential for AI.
Definitions
Artificial intelligence is the study of how to make computers do things, which at the moment
people do better. This is ephemeral as it refers to the current state of computer science and
excludes major problem areas that cannot be solved well, either by computers or by people at
the moment.
Artificial intelligence is also known as the branch of computer science that is concerned with
the automation of intellectual performance. AI is based upon the principles of computer
science, namely data structures used in knowledge representation, the algorithms needed to
apply that knowledge and the languages and programming techniques used in their
implementation.
It is a field of study that encompasses computational techniques for performing tasks that
seem to require intelligence when performed by humans. It seeks to explain and follow
intellectual performance in terms of computational processes.
Artificial intelligence is about generating representations and procedures that automatically or
autonomously solve problems heretofore solved by humans.
There are various definitions of AI according to different authors. Most of these definitions
take a very technical direction and avoid philosophical problems connected with the idea that
AI‘s purpose is to construct an artificial human. These definitions are categorized into the
following four categories:
1. Systems that think like humans (focus on reasoning and human framework).
 2. Systems that think rationally (focus on reasoning and a general concept of
intelligence).
3. Systems that act like humans (focus on behaviour and human framework).
4. Systems that act rationally (focus on behaviour and a general concept of intelligence).

Goals of AI
The goals of Artificial Intelligence (AI) revolve around enhancing human capabilities,
improving efficiency, and solving complex problems. Below are the key objectives of AI:
1. Automation of Tasks
 Automate repetitive and time-consuming tasks to improve efficiency and reduce
human error.
 Examples: Manufacturing robots, automated customer support systems.
2. Problem Solving
 Provide innovative solutions for complex real-world challenges in fields like
healthcare, finance, transportation, and education.
 Examples: Disease diagnosis, fraud detection, optimization problems.
3. Improving Decision-Making
 Assist humans in making more informed and accurate decisions by analyzing large
datasets.
 Examples: Financial forecasting, predictive maintenance.
4. Enhancing Human Capabilities
 Augment human intelligence by providing tools for creative, cognitive, and analytical
tasks.
 Examples: AI-powered content creation, virtual assistants.
5. Personalization
 Provide customized experiences based on user preferences and behaviors.
 Examples: Personalized recommendations in e-commerce and streaming platforms.
6. Advancing Scientific Research
 Accelerate discoveries by analyzing complex data and simulating experiments.
 Examples: Drug discovery, climate modeling.
7. Human Interaction
 Enhance human-machine interaction through natural language processing, speech
recognition, and vision systems.
 Examples: Chatbots, AI-driven virtual tutors.
8. Global Challenges
  Address global challenges like climate change, food security, and sustainable energy.
 Examples: Smart agriculture, renewable energy management systems.
9. Safety and Security
 Improve safety and security systems to protect people and assets.
 Examples: Surveillance, cybersecurity systems.
10. Ethical and Responsible AI
 Develop AI systems that are ethical, unbiased, and transparent, ensuring their use for
the betterment of society.

AI approaches
AI approaches are methodologies and strategies used to build intelligent systems that mimic
human cognitive abilities. These approaches can be broadly categorized into the following:
1. Symbolic AI (Logic-Based Approaches)
 Description: Based on symbolic representations and rule-based systems.
 Key Techniques:
o Knowledge representation (ontologies, semantic networks).
o Logical reasoning (predicate logic, propositional logic).
 Examples: Expert systems like MYCIN for medical diagnosis.
 Limitations: Struggles with uncertainty, ambiguity, and scalability.
2. Machine Learning (Data-Driven Approaches)
 Description: Systems learn patterns and relationships from data instead of being
explicitly programmed.
 Types:
o Supervised Learning: Learning from labeled data.
 Example: Classification (Spam detection), regression (house price
prediction).
o Unsupervised Learning: Learning patterns in unlabeled data.
 Example: Clustering (customer segmentation).
o Reinforcement Learning: Learning through trial and error by receiving
rewards or penalties.
 Example: Game-playing AI (AlphaGo).
 Limitations: Requires large datasets and computational power, prone to overfitting.
3. Neural Networks and Deep Learning
  Description: Inspired by the human brain, deep learning uses multi-layered neural
networks to learn complex patterns.
 Key Concepts:
o Convolutional Neural Networks (CNNs) for image processing.
o Recurrent Neural Networks (RNNs) for sequential data like text and speech.
o Transformers for natural language processing (NLP), e.g., GPT.
 Examples: Image recognition, language translation, autonomous vehicles.
 Limitations: High computational cost, lack of interpretability.
4. Evolutionary Algorithms (Optimization-Based Approaches)
 Description: Inspired by natural evolution, these algorithms optimize solutions
iteratively.
 Key Techniques:
o Genetic algorithms.
o Particle swarm optimization.
 Examples: Solving optimization problems in engineering and design.
 Limitations: Computationally expensive for large problems.
5. Hybrid Approaches
 Description: Combines different approaches for improved performance.
o Example: Neural-symbolic systems that integrate symbolic reasoning with
neural networks.
 Benefits: Leverages strengths of multiple approaches.
6. Bayesian Networks (Probabilistic Reasoning)
 Description: Uses probability theory to model uncertainty and causal relationships.
 Examples: Predictive modeling, medical diagnosis, fraud detection.
 Limitations: Computational complexity increases with network size.
7. Natural Language Processing (NLP)
 Description: Focuses on understanding, interpreting, and generating human language.
 Key Techniques:
o Syntax and semantic analysis.
o Sentiment analysis, machine translation, chatbots.
 Examples: Google Translate, virtual assistants like Siri.
8. Cognitive Computing
 Description: Mimics human thought processes to solve complex problems.
  Examples: IBM Watson for question answering and decision support.
9. Expert Systems
 Description: Rule-based systems that emulate the decision-making ability of a human
expert.
 Examples: Medical diagnosis systems, troubleshooting software.
 Limitations: Requires extensive domain knowledge for rule creation.
10. Behavioral and Agent-Based AI
 Description: Focuses on developing autonomous agents that interact with their
environment and other agents.
 Examples: Robotics, multi-agent systems for logistics.
Branches of AI
AI is a broad field with various branches, each focusing on a specific aspect of creating
intelligent systems. Below are the main branches of AI:
1. Machine Learning (ML)
 Description: Enables systems to learn from data and improve over time without
explicit programming.
 Key Areas:
o Supervised Learning.
o Unsupervised Learning.
o Reinforcement Learning.
 Applications: Fraud detection, recommendation systems, predictive analytics.
2. Natural Language Processing (NLP)
 Description: Focuses on enabling machines to understand, interpret, and generate
human language.
 Key Areas:
o Sentiment analysis.
o Language translation.
o Chatbots and virtual assistants.
 Applications: Google Translate, Siri, automated customer support.
3. Computer Vision
 Description: Enables machines to interpret and analyze visual data from the world.
 Key Areas:
o Image recognition.
o Object detection.
 o Facial recognition.
 Applications: Autonomous vehicles, medical imaging, surveillance systems.
4. Robotics
 Description: Involves designing, building, and programming robots that can perform
tasks autonomously.
 Key Areas:
o Industrial robots.
o Humanoid robots.
o Swarm robotics.
 Applications: Manufacturing, healthcare (surgical robots), space exploration.
5. Expert Systems
 Description: Mimics the decision-making ability of a human expert by using a set of
rules.
 Key Areas:
o Knowledge representation.
o Rule-based reasoning.
 Applications: Medical diagnosis systems, troubleshooting software.
6. Deep Learning
 Description: A subset of machine learning that uses artificial neural networks to
model and solve complex problems.
 Key Areas:
o Convolutional Neural Networks (CNNs) for images.
o Recurrent Neural Networks (RNNs) for sequential data.
o Transformers for language processing.
 Applications: Image classification, voice assistants, autonomous vehicles.
7. Fuzzy Logic
 Description: Deals with reasoning that is approximate rather than fixed or exact,
mimicking human decision-making.
 Applications: Control systems (washing machines, air conditioners), stock market
prediction.
8. Evolutionary Computation
 Description: Uses concepts from natural evolution, such as selection and mutation, to
solve optimization problems.
 Key Techniques:
 o Genetic Algorithms.
o Genetic Programming.
 Applications: Engineering design, game strategy optimization.
9. Reinforcement Learning
 Description: Focuses on training agents to take actions in an environment to
maximize cumulative rewards.
 Applications: Game-playing AI (AlphaGo), robotics, autonomous navigation.
10. Cognitive Computing
 Description: Simulates human thought processes in complex situations.
 Applications: IBM Watson, decision support systems.
11. Neural Networks
 Description: Modeled after the human brain, used to recognize patterns and process
complex data.
 Applications: Speech recognition, image processing, NLP tasks.
12. Artificial General Intelligence (AGI)
 Description: The pursuit of creating machines with general intelligence comparable
to human cognitive abilities.
 Current Status: Still in research; not yet achieved.
13. Knowledge Representation and Reasoning
 Description: Focuses on how machines can represent, store, and use knowledge
effectively.
 Applications: Semantic web, expert systems.
14. Speech Recognition
 Description: Enables systems to convert spoken language into text and understand it.
 Applications: Voice assistants (Alexa, Google Assistant), transcription software.
15. Autonomous Systems
 Description: Develops systems that can operate without human intervention.
 Applications: Drones, self-driving cars.
Applications of AI
AI has numerous applications across various industries, enhancing efficiency, accuracy, and
innovation. Here are some key applications:
1. Healthcare
 Applications:
o Disease diagnosis and prediction (e.g., cancer detection using imaging).
o Personalized medicine and drug discovery.
o Robotic surgeries.
o Virtual health assistants and chatbots for patient interaction.
o Analyzing medical records and optimizing hospital workflows.
 Examples: IBM Watson Health, AI-powered radiology tools.
2. Finance
 Applications:
o Fraud detection and prevention.
o Algorithmic trading.
o Credit risk assessment and underwriting.
o Personalized financial planning and robo-advisors.
o Customer service via AI-powered chatbots.
 Examples: PayPal's fraud detection system, Betterment robo-advisor.
3. Retail and E-commerce
 Applications:
o Personalized product recommendations.
o Inventory management and demand forecasting.
o Visual search and augmented reality shopping.
o Chatbots for customer service.
o Automated checkout systems.
 Examples: Amazon's recommendation engine, AI-powered virtual try-ons.
4. Education
 Applications:
o Personalized learning paths and adaptive learning platforms.
o Automated grading and assessment systems.
o AI tutors and virtual teaching assistants.
o Predicting student performance and dropout rates.
  Examples: Duolingo for language learning, Grammarly for writing assistance.
5. Transportation and Logistics
 Applications:
o Autonomous vehicles and drones.
o Traffic management and optimization.
o Route optimization for delivery and logistics.
o Predictive maintenance for vehicles and infrastructure.
 Examples: Tesla's Autopilot, UPS's ORION route optimization.
6. Entertainment and Media
 Applications:
o Content recommendation (e.g., movies, music, articles).
o Automated video and image editing.
o AI-generated content and animations.
o Sentiment analysis for social media trends.
 Examples: Netflix's recommendation engine, Deepfake technology.
7. Manufacturing
 Applications:
o Predictive maintenance for machinery.
o Quality control through AI-powered visual inspections.
o Supply chain optimization.
o Collaborative robots (cobots) for assembly lines.
 Examples: Siemens' AI-driven manufacturing systems.
8. Agriculture
 Applications:
o Precision farming using AI-driven sensors and drones.
o Predicting crop yields and weather patterns.
o Automated harvesting with robotics.
o Pest and disease detection.
 Examples: Blue River Technology, AI-powered irrigation systems.
9. Energy
 Applications:
o Smart grids for efficient energy distribution.
 o Predicting energy demand and optimizing supply.
o Renewable energy forecasting (e.g., wind and solar).
o Maintenance of power plants and equipment.
 Examples: Google‘s AI for energy efficiency in data centers.
10. Customer Service
 Applications:
o AI chatbots for 24/7 customer support.
o Sentiment analysis for understanding customer feedback.
o Automated ticket routing and issue resolution.
 Examples: Zendesk AI, Intercom‘s AI chatbot.
11. Security and Surveillance
 Applications:
o Facial recognition for access control.
o Threat detection in cyber security.
o Monitoring and analyzing surveillance footage.
o Fraud and anomaly detection.
 Examples: Clearview AI, AI-driven cyber security platforms.
12. Environment and Sustainability
 Applications:
o Monitoring and predicting climate change.
o Wildlife conservation through AI-powered monitoring.
o Optimizing waste management and recycling.
o Disaster prediction and response.
 Examples: AI for deforestation monitoring, AI-driven clean energy solutions.
13. Gaming
 Applications:
o NPC behavior and decision-making.
o AI opponents in strategy and simulation games.
o Procedural content generation.
 Examples: AI opponents in Chess, Dota 2 (OpenAI Five).
14. Legal and Compliance
 Applications:
 o Contract analysis and review.
o Legal research through AI-powered tools.
o Compliance monitoring for regulatory adherence.
 Examples: Kira Systems, ROSS Intelligence.
15. Human Resources (HR)
 Applications:
o Resume screening and candidate matching.
o Employee sentiment analysis.
o Workforce planning and performance management.
 Examples: LinkedIn‘s AI recruitment tools.

Agents in Artificial Intelligence

An AI system can be defined as the study of the rational agent and its environment. The
agents sense the environment through sensors and act on their environment through actuators.
An AI agent can have mental properties such as knowledge, belief, intention, etc.
What is an Agent?
An agent can be anything that perceive its environment through sensors and act upon that
environment through actuators. An Agent runs in the cycle of perceiving, thinking,
and acting.
We use the term percept to refer to the agent‘s perceptual inputs at any given instant. An
agent‘s percept sequence is the complete history of everything the agent has ever perceived.
In general, an agent‘s choice of action at any given instant can depend on the entire percept
sequence observed to date, but not on anything it hasn‘t perceived. By specifying the agent‘s
choice of action for every possible percept sequence, we have said more or less everything
there is to say about the agent. Mathematically speaking, we say that an agent‘s behavior is
described by the agent function that maps any given percept sequence to an action.
Internally, the agent function for an artificial agent will be implemented by an agent program.
It is important to keep these two ideas distinct. The agent function is an abstract mathematical
description; the agent program is a concrete implementation, running within some physical
system.
Structure of an AI Agent
The task of AI is to design an agent program which implements the agent function. The
structure of an intelligent agent is a combination of architecture and agent program. It can be
viewed as:
Agent = Architecture + Agent program
Following are the main three terms involved in the structure of an AI agent:
Architecture: Architecture is machinery that an AI agent executes on.
Agent Function: Agent function is used to map a percept to an action.
f:P* → A
Agent program: Agent program is an implementation of agent function. An agent program
executes on the physical architecture to produce function f.
An agent can be:
o Human-Agent: A human agent has eyes, ears, and other organs which work for
sensors and hand, legs, vocal tract work for actuators.
o Robotic Agent: A robotic agent can have cameras, infrared range finder, NLP for
sensors and various motors for actuators.
o Software Agent: Software agent can have keystrokes, file contents as sensory input
and act on those inputs and display output on the screen.

Hence the world around us is full of agents such as thermostat, cellphone, camera, and even
we are also agents.
Before moving forward, we should first know about sensors, effectors, and actuators.
Sensor: Sensor is a device which detects the change in the environment and sends the
information to other electronic devices. An agent observes its environment through sensors.
Actuators: Actuators are the component of machines that converts energy into motion. The
actuators are only responsible for moving and controlling a system. An actuator can be an
electric motor, gears, rails, etc.
Effectors: Effectors are the devices which affect the environment. Effectors can be legs,
wheels, arms, fingers, wings, fins, and display screen.

Intelligent Agents:
An intelligent agent is an autonomous entity which acts upon an environment using sensors
and actuators for achieving goals. An intelligent agent may learn from the environment to
achieve their goals. A thermostat is an example of an intelligent agent.
Following are the main four rules for an AI agent:
o Rule 1: An AI agent must have the ability to perceive the environment.
o Rule 2: The observation must be used to make decisions.
o Rule 3: Decision should result in an action.
o Rule 4: The action taken by an AI agent must be a rational action.
Rational Agent:
A rational agent is an agent which has clear preference, models uncertainty, and acts in a way
to maximize its performance measure with all possible actions.
A rational agent is said to perform the right things. AI is about creating rational agents to use
for game theory and decision theory for various real-world scenarios.
For an AI agent, the rational action is most important because in AI reinforcement learning
algorithm, for each best possible action, agent gets the positive reward and for each wrong
action, an agent gets a negative reward.
Rationality:
The rationality of an agent is measured by its performance measure. Rationality can be
judged on the basis of following points:
o Performance measure which defines the success criterion.
o Agent prior knowledge of its environment.
o Best possible actions that an agent can perform.
o The sequence of precepts.

PEAS Representation
PEAS is a type of model on which an AI agent works upon. When we define an AI agent or
rational agent, then we can group its properties under PEAS representation model. It is made
up of four words:
 P: Performance measure
 E: Environment
 A: Actuators
 S: Sensors

Here performance measure is the objective for the success of an agent's behavior.
PEAS for self-driving cars:
Let's suppose a self-driving car then PEAS representation will be:
Performance: Safety, time, legal drive, comfort
Environment: Roads, other vehicles, road signs, pedestrian
Actuators: Steering, accelerator, brake, signal, horn
Sensors: Camera, GPS, speedometer, odometer, accelerometer, sonar.
Types of AI Agents
Agents can be grouped into five classes based on their degree of perceived intelligence and
capability. All these agents can improve their performance and generate better action over the
time. These are given below:
 Simple Reflex Agent
 Model-based reflex agent
 Goal-based agents
 Utility-based agent
 Learning agent

1. Simple Reflex agent:

 The Simple reflex agents are the simplest agents. These agents take decisions on the
basis of the current precepts and ignore the rest of the percept history.
 These agents only succeed in the fully observable environment.
 The Simple reflex agent does not consider any part of precepts history during their
decision and action process.
 The Simple reflex agent works on Condition-action rule, which means it maps the
current state to action. Such as a Room Cleaner agent, it works only if there is dirt in
the room.
 Problems for the simple reflex agent design approach:
o They have very limited intelligence
o They do not have knowledge of non-perceptual parts of the current state
o Mostly too big to generate and to store.
o Not adaptive to changes in the environment.

function SIMPLE-REFLEX-AGENT(percept ) returns an action

persistent: rules, a set of condition–action rules
state←INTERPRET-INPUT(percept )
rule←RULE-MATCH(state, rules)
action ←rule.ACTION
return action
2. Model-based reflex agent
 The Model-based agent can work in a partially observable environment, and track the
situation.
 A model-based agent has two important factors:

o Model: It is knowledge about "how things happen in the world," so it is called

a Model-based agent.
o Internal State: It is a representation of the current state based on percept
history.

 These agents have the model, "which is knowledge of the world" and based on the
model they perform actions.
 Updating the agent state requires information about:

o How the world evolves

o How the agent's action affects the world.

The most effective way to handle partial observability is for the agent to keep track of the
part of the world it can’t see now. That is, the agent should maintain some sort of internal
INTERNAL STATE state that depends on the percept history and thereby reflects at least
some of the unobserved aspects of the current state. For the braking problem, the internal
state is not too extensive just the previous frame from the camera, allowing the agent to
detect when two red lights at the edge of the vehicle go on or off simultaneously. For other
driving tasks such as changing lanes, the agent needs to keep track of where the other cars are
if it can‘t see them all at once. And for any driving to be possible at all, the agent needs to
keep track of where its keys are.
Updating this internal state information as time goes by requires two kinds of knowledge to
be encoded in the agent program. First, we need some information about how the world
evolves independently of the agent—for example, that an overtaking car generally will be
closer behind than it was a moment ago. Second, we need some information about how the
agent‘s own actions affect the world—for example, that when the agent turns the steering
wheel clockwise, the car turns to the right, or that after driving for five minutes northbound
on the freeway, one is usually about five miles north of where one was five minutes ago. This
knowledge about ―how the world works‖—whether implemented in simple Boolean circuits
or in complete scientific theories—is called a model of the world. An agent that uses such a
MODEL-BASED model is called a model-based agent.
Figure 2.11 gives the structure of the model-based reflex agent with internal state, showing
how the current percept is combined with the old internal state to generate the updated
description of the current state, based on the agent‘s model of how the world works. The
agent program is shown in Figure 2.12. The interesting part is the function UPDATE-
STATE, which is responsible for creating the new internal state description. The details of
how models and states are represented vary widely depending on the type of environment and
the particular technology used in the agent design. Regardless of the kind of representation
used, it is seldom possible for the agent to determine the current state of a partially
observable environment exactly. Instead, the box labeled ―what the world is like now‖
(Figure 2.11) represents the agent‘s ―best guess‖ (or sometimes best guesses). For example,
an automated taxi may not be able to see around the large truck that has stopped in front of it
and can only guess about what may be causing the hold-up. Thus, uncertainty about the
current state may be unavoidable, but the agent still has to make a decision.
A perhaps less obvious point about the internal ―state‖ maintained by a model-based agent is
that it does not have to describe ―what the world is like now‖ in a literal sense. For example,
the taxi may be driving back home, and it may have a rule telling it to fill up with gas on the
way home unless it has at least half a tank. Although ―driving back home‖ may seem to an
aspect of the world state, the fact of the taxi‘s destination is actually an aspect of the agent‘s
internal state. If you find this puzzling, consider that the taxi could be in exactly the same
place at the same time, but intending to reach a different destination.
function MODEL-BASED-REFLEX-AGENT(percept ) returns an action
persistent: state, the agent‘s current conception of the world state
model , a description of how the next state depends on current state and action
rules, a set of condition–action rules
action, the most recent action, initially none
state←UPDATE-STATE(state, action, percept ,model )
rule←RULE-MATCH(state, rules)
action ←rule.ACTION
return action
3. Goal-based agents
 The knowledge of the current state environment is not always sufficient to decide
for an agent to what to do.
 The agent needs to know its goal which describes desirable situations.
 Goal-based agents expand the capabilities of the model-based agent by having the
"goal" information.
 They choose an action, so that they can achieve the goal.
 These agents may have to consider a long sequence of possible actions before
deciding whether the goal is achieved or not. Such considerations of different
scenario are called searching and planning, which makes an agent proactive.
4. Utility-based agents
 These agents are similar to the goal-based agent but provide an extra component
of utility measurement which makes them different by providing a measure of
success at a given state.
 Utility-based agent act based not only goals but also the best way to achieve the
goal.
 The Utility-based agent is useful when there are multiple possible alternatives, and
an agent has to choose in order to perform the best action.
 The utility function maps each state to a real number to check how efficiently each
action achieves the goals.

Goals alone are not enough to generate high-quality behavior in most environments. For
example, many action sequences will get the taxi to its destination (thereby achieving the
goal) but some are quicker, safer, more reliable, or cheaper than others. Goals just provide a
crude binary distinction between ―happy‖ and ―unhappy‖ states. A more general performance
measure should allow a comparison of different world states according to exactly how happy
they would make the agent. Because ―happy‖ does not sound very scientific, economists and
computer scientists use the term utility instead.
We have already seen that a performance measure assigns a score to any given sequence of
environment states, so it can easily distinguish between more and less desirable ways of
getting to the taxi‘s destination. An agent‘s utility function is essentially an internalization of
the performance measure. If the internal utility function and the external performance
measure are in agreement, then an agent that chooses actions to maximize its utility will be
rational according to the external performance measure.
Let us emphasize again that this is not the only way to be rational—we have already seen a
rational agent program for the vacuum world (Figure 2.8) that has no idea what its utility
function is—but, like goal-based agents, a utility-based agent has many advantages in terms
of flexibility and learning. Furthermore, in two kinds of cases, goals are inadequate but a
utility-based agent can still make rational decisions. First, when there are conflicting goals,
only some of which can be achieved (for example, speed and safety), the utility function
specifies the appropriate trade off. Second, when there are several goals that the agent can
aim for, none of which can be achieved with certainty, utility provides a way in which the
likelihood of success can be weighed against the importance of the goals.
Partial observability and stochasticity are ubiquitous in the real world, and so, therefore, is
decision making under uncertainty. Technically speaking, a rational utility-based agent
chooses the action that maximizes the expected utility of the action outcomes—that is, the
utility the agent expects to derive, on average, given the probabilities and utilities of each
outcome.
5. Learning Agents
o A learning agent in AI is the type of agent which can learn from its past experiences,
or it has learning capabilities.
o It starts to act with basic knowledge and then able to act and adapt automatically
through learning.
o A learning agent has mainly four conceptual components, which are:

o Learning element: It is responsible for making improvements by learning

from environment
o Critic: Learning element takes feedback from critic which describes that how
well the agent is doing with respect to a fixed performance standard.
o Performance element: It is responsible for selecting external action
o Problem generator: This component is responsible for suggesting actions
that will lead to new and informative experiences.

o Hence, learning agents are able to learn, analyze performance, and look for new ways
to improve the performance.
Agent Environment in AI
An environment is everything in the world which surrounds the agent, but it is not a part of
an agent itself. An environment can be described as a situation in which an agent is present.
The environment is where agent lives, operate and provide the agent with something to sense
and act upon it. An environment is mostly said to be non-feministic.
As per Russell and Norvig, an environment can have various features from the point of view
of an agent:
1. Fully observable vs Partially Observable
2. Static vs Dynamic
3. Discrete vs Continuous
4. Deterministic vs Stochastic
5. Single-agent vs Multi-agent
6. Episodic vs sequential
7. Known vs Unknown
8. Accessible vs Inaccessible

1. Fully observable vs Partially Observable:

o If an agent sensor can sense or access the complete state of an environment at each
point in time then it is a fully observable environment, it is partially observable. For
reference, Imagine a chess-playing agent. In this case, the agent can fully observe the
state of the chessboard at all times. Its sensors (in this case, vision or the ability to
access the board's state) provide complete information about the current position of all
pieces. This is a fully observable environment because the agent has perfect
information about the state of the world.
 o A fully observable environment is easy as there is no need to maintain the internal
state to keep track of the history of the world. For reference, Consider a self-driving
car navigating a busy city. While the car has sensors like cameras, lidar, and radar, it
can't see everything at all times. Buildings, other vehicles, and pedestrians can
obstruct its sensors. In this scenario, the car's environment is partially observable
because it doesn't have complete and constant access to all relevant information. It
needs to maintain an internal state and history to make informed decisions even when
some information is temporarily unavailable.
o An agent with no sensors in all environments then such an environment is called
unobservable. For reference, think about an agent designed to predict earthquakes but
placed in a sealed, windowless room with no sensors or access to external data. In this
situation, the environment is unobservable because the agent has no way to gather
information about the outside world. It can't sense any aspect of its environment,
making it completely unobservable.

2. Deterministic vs Stochastic:
o If an agent's current state and selected action can completely determine the next state
of the environment, then such an environment is called a deterministic environment.
For reference, Chess is a classic example of a deterministic environment. In chess, the
rules are well-defined, and each move made by a player has a clear and predictable
outcome based on those rules. If you move a pawn from one square to another, the
resulting state of the chessboard is entirely determined by that action, as is your
opponent's response. There's no randomness or uncertainty in the outcomes of chess
moves because they follow strict rules. In a deterministic environment like chess,
knowing the current state and the actions taken allows you to completely determine
the next state.
o A stochastic environment is random and cannot be determined completely by an
agent. For reference, The stock market is an example of a stochastic environment. It's
highly influenced by a multitude of unpredictable factors, including economic events,
investor sentiment, and news. While there are patterns and trends, the exact behavior
of stock prices is inherently random and cannot be completely determined by any
individual or agent. Even with access to extensive data and analysis tools, stock
market movements can exhibit a high degree of unpredictability. Random events and
market sentiment play significant roles, introducing uncertainty.
o In a deterministic, fully observable environment, an agent does not need to worry
about uncertainty.

3. Episodic vs Sequential:
o In an episodic environment, there is a series of one-shot actions, and only the current
percept is required for the action. For example, Tic-Tac-Toe is a classic example of an
episodic environment. In this game, two players take turns placing their symbols (X
or O) on a 3x3 grid. Each move by a player is independent of previous moves, and the
goal is to form a line of three symbols horizontally, vertically, or diagonally. The
game consists of a series of one-shot actions where the current state of the board is the
only thing that matters for the next move. There's no need for the players to remember
past moves because they don't affect the current move. The game is self-contained
and episodic.
o However, in a Sequential environment, an agent requires memory of past actions to
determine the next best actions. For example, Chess is an example of a sequential
 environment. Unlike Tic-Tac-Toe, chess is a complex game where the outcome of
each move depends on a sequence of previous moves. In chess, players must consider
the history of the game, as the current position of pieces, previous moves, and
potential future moves all influence the best course of action. To play chess
effectively, players need to maintain a memory of past actions, anticipate future
moves, and plan their strategies accordingly. It's a sequential environment because the
sequence of actions and the history of the game significantly impact decision-making.

4. Single-agent vs Multi-agent
o If only one agent is involved in an environment, and operating by itself then such an
environment is called a single-agent environment. For example, Solitaire is a classic
example of a single-agent environment. When you play Solitaire, you're the only
agent involved. You make all the decisions and actions to achieve a goal, which is to
arrange a deck of cards in a specific way. There are no other agents or players
interacting with you. It's a solitary game where the outcome depends solely on your
decisions and moves. In this single-agent environment, the agent doesn't need to
consider the actions or decisions of other entities.
o However, if multiple agents are operating in an environment, then such an
environment is called a multi-agent environment. For reference, A soccer match is an
example of a multi-agent environment. In a soccer game, there are two teams, each
consisting of multiple players (agents). These players work together to achieve
common goals (scoring goals and preventing the opposing team from scoring). Each
player has their own set of actions and decisions, and they interact with both their
teammates and the opposing team. The outcome of the game depends on the
coordinated actions and strategies of all the agents on the field. It's a multi-agent
environment because there are multiple autonomous entities (players) interacting in a
shared environment.
o The agent design problems in the multi-agent environment are different from single-
agent environments.

5. Static vs Dynamic:
o If the environment can change itself while an agent is deliberating then such an
environment is called a dynamic environment it is called a static environment.
o Static environments are easy to deal with because an agent does not need to continue
looking at the world while deciding on an action. For reference, A crossword puzzle is
an example of a static environment. When you work on a crossword puzzle, the
puzzle itself doesn't change while you're thinking about your next move. The
arrangement of clues and empty squares remains constant throughout your problem-
solving process. You can take your time to deliberate and find the best word to fill in
each blank, and the puzzle's state remains unaltered during this process. It's a static
environment because there are no changes in the puzzle based on your deliberations.
o However, for a dynamic environment, agents need to keep looking at the world at
each action. For reference, Taxi driving is an example of a dynamic environment.
When you're driving a taxi, the environment is constantly changing. The road
conditions, traffic, pedestrians, and other vehicles all contribute to the dynamic nature
of this environment. As a taxi driver, you need to keep a constant watch on the road
and adapt your actions in real time based on the changing circumstances. The
environment can change rapidly, requiring your continuous attention and decision-
 making. It's a dynamic environment because it evolves while you're deliberating and
taking action.

6. Discrete vs Continuous:
o If in an environment, there are a finite number of percepts and actions that can be
performed within it, then such an environment is called a discrete environment it is
called a continuous environment.
o Chess is an example of a discrete environment. In chess, there are a finite number of
distinct chess pieces (e.g., pawns, rooks, knights) and a finite number of squares on
the chessboard. The rules of chess define clear, discrete moves that a player can make.
Each piece can be in a specific location on the board, and players take turns making
individual, well-defined moves. The state of the chessboard is discrete and can be
described by the positions of the pieces on the board.
o Controlling a robotic arm to perform precise movements in a factory setting is an
example of a continuous environment. In this context, the robot arm's position and
orientation can exist along a continuous spectrum. There are virtually infinite possible
positions and orientations for the robotic arm within its workspace. The control inputs
to move the arm, such as adjusting joint angles or applying forces, can also vary
continuously. Agents in this environment must operate within a continuous state and
action space, and they need to make precise, continuous adjustments to achieve their
goals.

7. Known vs Unknown
o Known and unknown are not actually a feature of an environment, but it is an agent's
state of knowledge to perform an action.
o In a known environment, the results of all actions are known to the agent. While in an
unknown environment, an agent needs to learn how it works in order to perform an
action.
o It is quite possible for a known environment to be partially observable and an
Unknown environment to be fully observable.
o The opening theory in chess can be considered as a known environment for
experienced chess players. Chess has a vast body of knowledge regarding opening
moves, strategies, and responses. Experienced players are familiar with established
openings, and they have studied various sequences of moves and their outcomes.
When they make their initial moves in a game, they have a good understanding of the
potential consequences based on their knowledge of known openings.
o Imagine a scenario where a rover or drone is sent to explore an alien planet with no
prior knowledge or maps of the terrain. In this unknown environment, the agent (rover
or drone) has to explore and learn about the terrain as it goes along. It doesn't have
prior knowledge of the landscape, potential hazards, or valuable resources. The agent
needs to use sensors and data it collects during exploration to build a map and
understand how the terrain works. It operates in an unknown environment because the
results and consequences of its actions are not initially known, and it must learn from
its experiences.

8. Accessible vs Inaccessible
If an agent can obtain complete and accurate information about the state's environment, then
such an environment is called an Accessible environment else it is called inaccessible.
o For example, Imagine an empty room equipped with highly accurate temperature
sensors. These sensors can provide real-time temperature measurements at any point
within the room. An agent placed in this room can obtain complete and accurate
information about the temperature at different locations. It can access this information
at any time, allowing it to make decisions based on the precise temperature data. This
environment is accessible because the agent can acquire complete and accurate
information about the state of the room, specifically its temperature.
o For example, Consider a scenario where a satellite in space is tasked with monitoring
a specific event taking place on Earth, such as a natural disaster or a remote area's
condition. While the satellite can capture images and data from space, it cannot access
fine-grained information about the event's details. For example, it may see a forest fire
occurring but cannot determine the exact temperature at specific locations within the
fire or identify individual objects on the ground. The satellite's observations provide
valuable data, but the environment it is monitoring (Earth) is vast and complex,
making it impossible to access complete and detailed information about all aspects of
the event. In this case, the Earth's surface is an inaccessible environment for obtaining
fine-grained information about specific events.