UNIT-3

Bayes' theorem in Artificial intelligence

The Bayes Theorem in AI is perhaps the most fundamental basis for probability and statistics, more popularly known
as Bayes’ rule or Bayes’ law. It allows us to revise our assumptions or the probability that an event will occur, given
new information or evidence. In this article, we will see how the Bayes theorem is used in AI.

Bayes’ Theorem in AI
In probability theory, Bayes’ theorem talks about the relation of the conditional probability of two random events
and their marginal probability. In short, it provides a way to calculate the value of P(B|A) by using the knowledge of
P(A|B).

Bayes’ theorem is the name given to the formula used to calculate conditional probability. The formula is as follows:

P(A∣B)=P(A∩B)/P(B)=(P(A)∗P(B∣A))/P(B)

where,

P(A) is the probability that event A occurs.

P(B) defines the probability that event B occurs.
P(A|B) is the probability of the occurrence of event A given that event B has already occurred.
P(B∣A) can now be read as: Probability of event B occurring given that event A occurred.
p(A∩B) is the probability events A and B will happen together.
Key terms in Bayes’ Theorem
The Bayes’ Theorem is a basic concept in probability and statistics. It gives a model of updating beliefs or
probabilities when the new evidence is presented. This theorem was named after Reverend Thomas Bayes and has
been applied in many fields, ranging from artificial intelligence and machine learning to data analysis.

The Bayes’ Theorem encompasses four major elements:

Prior Probability (P(A)): The probability or belief in an event A prior to considering any additional evidence, it
represents what we know or believe about A based on previous knowledge.
Likelihood P(B|A): the probability of evidence B given the occurrence of event A. It determines how strongly the
evidence points toward the event.
Evidence (P(B)): Evidence is the probability of observing evidence B regardless of whether A is true. It serves to
normalize the distribution so that the posterior probability is a valid probability distribution.
Posterior Probability P(A|B): The posterior probability is a revised belief regarding event A, informed by some new
evidence B. It answers the question, “What is the probability that A is true given evidence B observed?”
Using these components, Bayes’ Theorem computes the posterior probability P(A|B), which represents our updated
belief in A after considering the new evidence.

In artificial intelligence, probability and the Bayes Theorem are especially useful when making decisions or
inferences based on uncertain or incomplete data. It enables us to rationally update our beliefs as new evidence
becomes available, making it an indispensable tool in AI, machine learning, and decision-making processes.
How bayes theorem is relevant in AI?
Bayes’ theorem is highly relevant in AI due to its ability to handle uncertainty and make decisions based on
probabilities. Here’s why it’s crucial:

Probabilistic Reasoning: In many real-world scenarios, AI systems must reason under uncertainty. Bayes’ theorem
allows AI systems to update their beliefs based on new evidence. This is essential for applications like autonomous
vehicles, where the environment is constantly changing and sensors provide noisy information.

Machine Learning: Bayes’ theorem serves as the foundation for Bayesian machine learning approaches. These
methods allow AI models to incorporate prior knowledge and update their beliefs as they see more data. This is
particularly useful in scenarios with limited data or when dealing with complex relationships between variables.

Classification and Prediction: In classification tasks, such as spam email detection or medical diagnosis, Bayes’
theorem can be used to calculate the probability that a given input belongs to a particular class. This allows AI
systems to make more informed decisions based on the available evidence.

Anomaly Detection: Bayes’ theorem is used in anomaly detection, where AI systems identify unusual patterns in
data. By modeling the normal behavior of a system, Bayes’ theorem can help detect deviations from this norm,
signaling potential anomalies or security threats.
Importance of Bayes’ Theorem in AI

Bayes’ Theorem is extremely important in artificial intelligence (AI) and related fields.

Probabilistic Reasoning: In AI, many problems involve uncertainty, so probabilistic reasoning is an important technique.
Bayes’ Theorem enables artificial intelligence systems to model and reason about uncertainty by updating beliefs in
response to new evidence. This is important for decision-making, pattern recognition, and predictive modeling.

Machine Learning: Bayes’ Theorem is a fundamental concept in machine learning, specifically Bayesian machine learning.
Bayesian methods are used to model complex relationships, estimate model parameters, and predict outcomes. Bayesian
models enable the principled handling of uncertainty in tasks such as classification, regression, and clustering.

Data Science: Bayes’ Theorem is used extensively in Bayesian statistics. It is used to estimate and update probabilities in a
variety of settings, including hypothesis testing, Bayesian inference, and Bayesian optimization. It offers a consistent
framework for modeling and comprehending data.
Uses of Bayes Rule in Artificial Intelligence
Bayesian Inference: In Bayesian statistics, the Bayes’ rule is used to update the probability distribution over a set
of parameters or hypotheses using observed data. This is especially important for machine learning tasks like
parameter estimation in Bayesian networks, hidden Markov models, and probabilistic graphical models.

Naive Bayes Classification: In the field of natural language processing and text classification, the Naive Bayes
classifier is widely used. It uses Bayes’ theorem to calculate the likelihood that a document belongs to a specific
category based on the words it contains. Despite its “naive” assumption of feature independence, it works
surprisingly well in practice.

Bayesian Networks: Bayesian networks are graphical models that use Bayes’ theorem to represent and predict
probabilistic relationships between variables. They are used in a variety of AI applications, such as medical
diagnosis, fault detection, and decision support systems.

Spam Email Filtering: In email filtering systems, Bayes’ theorem is used to determine whether an incoming email is
spam or not. The model calculates the likelihood of seeing specific words or features in spam or non-spam emails
and adjusts the probabilities accordingly.

Reinforcement Learning: Bayes’ rule can be used to model the environment in a probabilistic manner. Bayesian
reinforcement learning methods can help agents estimate and update their beliefs about state transitions and
rewards, allowing them to make more informed decisions.
Bayesian Optimization: In optimization tasks, Bayes’ theorem can be used to represent the objective
function as a probabilistic surrogate. Bayesian optimization techniques make use of this model to iteratively
explore and exploit the search space in order to efficiently find the optimal solution. This is commonly used
for hyperparameter tuning and algorithm parameter optimization.

Anomaly Detection: The Bayes theorem can be used to identify anomalies or outliers in datasets. Deviations
from the normal distribution can be quantified by modeling it, which aids in anomaly detection for a variety
of applications, including fraud detection and network security.

Personalization: In recommendation systems, Bayes’ theorem can be used to update user preferences and
provide personalized recommendations. By constantly updating a user’s preferences based on their
interactions, the system can recommend more relevant content.

Robotics and Sensor Fusion: In robotics, the Bayes’ rule is used to combine sensors. It uses data from
multiple sensors to estimate the state of a robot or its environment. This is necessary for tasks like
localization and mapping.

Medical Diagnosis: In healthcare, Bayes’ theorem is used in medical decision support systems to update the
likelihood of various diagnoses based on patient symptoms, test results, and medical history.
Problem:
Suppose you are building a spam filter for emails. You want to determine whether an
email is spam or not based on the presence of the word "offer" in the email. You have
the following data:
P(Spam): Probability of an email being spam = 0.4
P(Not Spam): Probability of an email not being spam = 0.6
P(Offer | Spam): Probability that the word "offer" appears in a spam email = 0.7
P(Offer | Not Spam): Probability that the word "offer" appears in a non-spam email =
0.2
You want to calculate the probability that an email is spam given that it contains the
word "offer”.
Solution:
To find P(Spam | Offer), we use Bayes'
theorem:𝑃(Spam | Offer)=𝑃(Offer | Spam)×𝑃(Spam)/ P(Offer)​

First, we need to calculate P(Offer) using the law of total

probability:𝑃(Offer)=𝑃(Offer | Spam)×𝑃(Spam)+𝑃(Offer | Not Spam)×𝑃(Not Spam)

Plugging in the values: 𝑃(Offer)=(0.7×0.4)+(0.2×0.6)=0.28+0.12=0.4

Now, calculate
P(Spam | Offer)=0.7×0.40./0.40=0.7
So, the probability that an email is spam given that it contains the word "offer" is 0.7 or
70%.
Semantic Networks in Artificial Intelligence
Semantic networks are a powerful tool in the field of artificial intelligence
(AI), used to represent knowledge and understand relationships between
different concepts. They are graphical representations that connect nodes
(representing concepts) with edges (representing relationships). Semantic
networks are widely used in natural language processing (NLP), knowledge
representation, and reasoning systems.

These relationships can indicate various connections, such as "is a," "part
of," "has a," or any other meaningful association.
Example:
Consider the following example to represent knowledge about animals:

•Nodes: "Dog," "Cat," "Animal," "Mammal“

•Edges:
• "Dog" → "is a" → "Mammal"
• "Cat" → "is a" → "Mammal"
• "Mammal" → "is a" → "Animal“

This network indicates that both dogs and cats are mammals, and mammals are a type
of animal.
Components of Semantic Networks
Semantic networks are made up of several key components:

1. Lexical Components
Nodes: The fundamental units of a semantic network, representing concepts, entities, or
objects within the domain of knowledge. Examples include "Dog," "Animal," or "Tree.“

Labels: Descriptive names or identifiers associated with the nodes, providing a way to
refer to the concepts they represent.

2. Structural Components
Edges/Links: The connections between nodes, representing relationships such as "is a,"
"part of," "causes," or "associated with.“

Types of Relationships: These can include hierarchical relationships (e.g., "is a"),
associative relationships (e.g., "related to"), and functional relationships (e.g., "causes"
or "results in").
3. Semantic Components
Meanings of Nodes: The specific meanings or interpretations of the nodes within the
context of the network.
Interpretation of Relationships: The understanding of what the edges or links between
nodes signify in real-world terms, ensuring the relationships are meaningful and
accurately reflect the domain.

4. Procedural Part
Inference Rules: Rules that allow the network to derive new knowledge from existing
relationships. For example, if "Dog is a Mammal" and "Mammal is an Animal," the
network can infer that "Dog is an Animal.“
Query Mechanisms: Procedures for retrieving information from the network based on
specific queries or criteria.
Update Mechanisms: Rules and processes for adding, modifying, or removing nodes and
links as new information is introduced.
Examples of Semantic Networks in AI
Semantic networks are a powerful tool for representing relationships and classifications
across various domains. Here are some examples illustrating how semantic networks can
be applied in different fields to organize and understand complex information.

1. Technology Stack Classification

Nodes: Frontend, Backend, HTML, CSS, JavaScript, Python, Django, API
Links: “is a” relation, “uses” relation
Labels: Web Development, Framework, Language

In this semantic network, different components of a technology stack are represented.

"HTML," "CSS," and "JavaScript" are linked to "Frontend" with an "is a" relation, while
"Python" and "Django" are linked to "Backend." The "uses" relation connects "API" to
both "Frontend" and "Backend," indicating its role in web development.
Applications of Semantic Networks in AI
Semantic networks are used in various AI applications, such as:

Natural Language Processing (NLP): In NLP, semantic networks help in understanding the meaning of
words and sentences by representing the relationships between different words and concepts.

Expert Systems: In expert systems, semantic networks are used to represent the knowledge of human
experts, enabling the system to make decisions or provide recommendations based on that knowledge.

Ontology Development: Ontologies, which define the structure of knowledge in a particular domain, often
use semantic networks to represent the relationships between concepts within that domain.

Information Retrieval: Semantic networks enhance information retrieval by allowing systems to

understand the context and relationships between different pieces of information, leading to more
accurate search results.

Machine Learning: In some machine learning applications, semantic networks are used to improve the
interpretability of models by providing a structured representation of the knowledge the model has
learned.
Advantages of Semantic Networks

Intuitive Representation: Semantic networks provide a clear and intuitive way to

represent knowledge, making it easier for both humans and machines to understand
complex relationships.

Flexibility: They can represent various types of relationships and are flexible enough
to be applied across different domains and applications.

Support for Reasoning: Semantic networks facilitate reasoning by enabling AI

systems to infer new knowledge based on existing relationships.
Challenges and Limitations
While semantic networks are powerful, they come with certain challenges:

Scalability: As the number of concepts and relationships increases, semantic

networks can become complex and difficult to manage.

Ambiguity: Representing ambiguous or unclear relationships can be challenging,

leading to potential misinterpretations by the AI system.

Computational Complexity: Complex networks with numerous relationships can

require significant computational resources to process and reason about.
Script is a structured framework that represents a stereotypical sequence of events
within a specific context. Introduced by cognitive scientist Roger Schank in the 1970s,
scripts are designed to help AI systems understand and predict human behavior by
modeling common scenarios.

Key Components of a Script:

1.Entry Conditions: Preconditions that must be met for the script to initiate.
2.Scenes: Sequential events or actions that occur within the script.
3.Roles: Participants involved in the script, each with specific functions.
4.Props: Objects or items utilized during the events.
5.Results: Outcomes or states resulting from the script's execution.
Example: Visiting a Restaurant
•Entry Conditions: The individual is hungry and has money.
•Scenes:
• Entering the restaurant.
• Being seated by a host.
• Ordering food from a waiter.
• Eating the meal.
• Paying the bill.
• Leaving the restaurant.
•Roles: Customer, host, waiter, cashier.
•Props: Menu, table, food, bill, payment method.
•Results: The individual is no longer hungry and has less money.
Schema refers to an organized framework or structure that represents knowledge about a
particular concept, entity, or scenario. Schemas are instrumental in enabling AI systems to
understand, interpret, and generate human-like responses by providing context and
facilitating the organization of information.
Key Aspects of Schemas in AI:
1.Knowledge Representation: Schemas serve as structured representations of knowledge,
allowing AI systems to model complex information about the world.
2.Learning and Adaptation: AI systems utilize schemas to learn from experiences and adapt
to new information. By organizing knowledge into schemas, AI can generalize from specific
instances to broader concepts, enhancing its learning capabilities.
3.Natural Language Processing (NLP): In NLP, schemas help AI understand and generate
human language by providing context and structure. They enable the system to comprehend
the relationships between words and phrases, facilitating more accurate language processing.
4.Cognitive Architectures: Schemas are integral to cognitive architectures in AI, where they
function as building blocks for perception, cognition, and action. These architectures model
human cognitive processes, allowing AI systems to perform tasks that require human-like
understanding and reasoning.
Applications of Schemas in AI:
•Question Answering Systems: Schemas enable AI to interpret and respond to user
queries by providing a structured understanding of the information domain. For instance,
the Schema2QA toolkit generates question-answering agents by leveraging database
schemas, facilitating efficient information retrieval.
•Autonomous Robotics: In robotics, schemas guide the behavior of autonomous agents
by providing frameworks for perception and action. This approach allows robots to
navigate and interact with their environment effectively.
•Knowledge Base Question Answering: Schemas assist AI in understanding and querying
knowledge bases by providing context and structure, enabling accurate information
retrieval. The In-Context Schema Understanding method leverages schemas to enhance
AI's ability to answer questions based on knowledge bases.
In summary, schemas play a pivotal role in AI by structuring knowledge, facilitating
learning, and enabling systems to perform complex tasks that require human-like
understanding and reasoning.
Frame is a data structure used to represent stereotyped situations, encompassing various
attributes and their corresponding values. Introduced by Marvin Minsky in 1974, frames
serve as a means to organize knowledge into structured formats.
Key Components of a Frame:
•Slots: These are attributes or properties of the frame, each representing a specific aspect
of the situation or object.
•Fillers: These are the values assigned to the slots, detailing specific information pertinent
to the attribute.
Example:
Consider a frame representing a "Restaurant Visit":
•Frame Name: Restaurant Visit
• Slots and Fillers:
• Location: Downtown
• Cuisine Type: Italian
• Seating Capacity: 50
• Average Cost per Person: $25
Conceptual Dependency in AI
what is conceptual dependency in artificial intelligence?

Conceptual Dependency (CD) emerges as a pivotal hypothesis and representation

technique within the field of artificial intelligence. Originating in the late 1960s and mid-
1970s by Roger Schank, CD aims to elucidate the meaning of natural language sentences in
a manner devoid of linguistic constraints. This approach enables more effective
comprehension, manipulation, and generation of sentences by computer programs. Here,
we introduce the core concepts and components of Applied Reliance.
1. Primitive Acts (Primitives)
Primitive Acts demonstrations are the fundamental structural blocks of all activities in CD.
They address central activities that can happen in different circumstances and are
language-autonomous. Some common primitive acts include:

ATRANS (Dynamic Exchange): The exchange of a theoretical relationship, like giving data or
proprietorship.
PTRANS (Actual Exchange): The actual development of an item starting with one spot and
then onto the next.
PROPEL: The application of physical force to an object, causing it to move.
MOVE: A self-motivated change in position by an animate object.
INGEST: Taking something into the body, such as eating or drinking.
EXPEL: Forcing something out of the body, such as exhaling or vomiting.
SPEAK: Producing verbal output.
ATTEND: Directing sensory organs towards a stimulus (like looking or listening).
2. Conceptual Cases (Cases)
Conceptual Cases (Cases) portray the jobs played by various substances in an activity. They
assist with determining who is doing what to whom, with what, and under what
conditions. Normal theoretical cases include:

Agent(AG): The substance playing out the activity.

Object (OB): The substance that is impacted by the activity.
Recipient (RE): The substance that gets the aftereffect of the activity.
Instrument (IN): The means or device used to play out the activity.
Source (SRC): The beginning stage of an exchange activity.
Objective (DEST): The endpoint of an exchange activity.
Experiencer (EX): The element that encounters a sensation or feeling.
3. Modifiers
Modifiers give extra insights concerning activities, objects, or different components in the
CD structure. They can determine qualities like time, location, manner, and purpose.
Instances of modifiers include:

Time: When the activity happens (e.g., yesterday, presently).

Area: Where the activity happens (e.g., in the recreation area, at home).
Way: How the activity is performed (e.g., rapidly, cautiously).
Reason: Why the activity is performed (e.g., to get cash, for entertainment only).
4. Conceptual Tenses
Conceptual tenses show the transient parts of activities, for example, when
they happen and their term. This aids in figuring out the timing and
arrangement of occasions. Instances of applied tenses include:

Past: Activities that have previously happened.

Present: Activities that are as of now happening.
Future: Activities that will happen.
Constant: Activities that are progressing.
Finished: Activities that have been done.
5. Dependencies
These are connections between activities that show how activities are connected.

Causal Dependency: One activity causes another.

Temporal Dependency: The arrangement of activities in time.

Conditional Dependency: One activity is subject to the event of another activity.

6. State Descriptions
These depict the condition of elements when activities. They can incorporate actual
states (e.g., area, ownership) or mental states (e.g., convictions, wants).

By utilizing these parts, Conceptual Dependency intends to make a language-free

portrayal of importance, which can be utilized for different purposes, for example,
natural language figuring out, machine interpretation, and artificial intelligence
consciousness.
Six primitive conceptual categories provide building blocks which are the
set of allowable dependencies in the concepts in a sentence:

PP-- Real world objects.

ACT-- Real world actions.
PA-- Attributes of objects.
AA-- Attributes of actions.
T-- Times.
LOC-- Locations.
Arrows indicate the direction of dependency. Letters above indicate certain
relationships:
o-- object.
R-- recipient-donor.
I-- instrument e.g. eat with a spoon.
D-- destination e.g. going home.
Double arrows indicate two-way links between the actor (PP) and action (ACT).
The actions are built from the set of primitive acts (see above).
These can be modified by tense etc.
The use of tense and mood in describing events is extremely important and introduced
the following modifiers:
p-- past
f-- future
t-- transition
ts-- start transition
tf-- finished transition
k-- continuing
?-- interrogative
/-- negative
delta-- timeless
c-- conditional
Fuzzy refers to things that are not clear or are vague. In the real world many times we
encounter a situation when we can’t determine whether the state is true or false, their
fuzzy logic provides very valuable flexibility for reasoning. In this way, we can consider the
inaccuracies and uncertainties of any situation.

Fuzzy Logic is a form of many-valued logic in which the truth values of variables may be
any real number between 0 and 1, instead of just the traditional values of true or false. It
is used to deal with imprecise or uncertain information and is a mathematical method for
representing vagueness and uncertainty in decision-making.

Fuzzy Logic is based on the idea that in many cases, the concept of true or false is too
restrictive, and that there are many shades of gray in between. It allows for partial truths,
where a statement can be partially true or false, rather than fully true or false.

Fuzzy Logic is used in a wide range of applications, such as control systems, image
processing, natural language processing, medical diagnosis, and artificial intelligence.
In the boolean system truth value, 1.0 represents the absolute truth value and 0.0
represents the absolute false value. But in the fuzzy system, there is no logic for the
absolute truth and absolute false value. But in fuzzy logic, there is an intermediate value
too present which is partially true and partially false.
What is Forward Reasoning?
Forward reasoning is a process in artificial intelligence that finds all the possible solutions
of a problem based on the initial data and facts. Thus, the forward reasoning is a data-
driven task as it begins with new data. The main objective of the forward reasoning in AI is
to find a conclusion that would follow. It uses an opportunistic type of approach.

Forward reasoning flows from incipient to the consequence. The inference engine
searches the knowledge base with the given information depending on the constraints.
The precedence of these constraints have to match the current state.

In forward reasoning, the first step is that the system is given one or more constraints. The
rules are then searched for in the knowledge base for every constraint. The rule that fulfils
the condition is selected. Also, every rule can generate a new condition from the
conclusion which is obtained from the invoked one. This new conditions can be added and
are processed again.

The step ends if no new conditions exist. Hence, we can conclude that forward reasoning
follows the top-down approach.
What is Backward Reasoning?
Backward reasoning is the reverse process of the forward reasoning in which a goal or
hypothesis is selected and it is analyzed to find the initial data, facts, and rules. Therefore,
the backward reasoning is a goal driven task as it begins with conclusions or goals that are
uncertain. The main objective of the backward reasoning is to find the facts that support
the conclusions.

Backward reasoning uses a conservative type of approach and flows from consequence to
the incipient. The system helps to choose a goal state and reasons in a backward direction.
The first step in the backward reasoning is that the goal state and rules are selected. Then,
sub-goals are made from the selected rule, which need to be satisfied for the goal state to
be true.

The initial conditions are set such that they satisfy all the sub-goals. Also, the established
states are matched to the initial state provided. If the condition is fulfilled, the goal is the
solution, otherwise the goal is rejected. Therefore, backward reasoning follows bottom-up
technique.
Forward Reasoning & Backward Reasoning
1.

It is a data-driven task.

It is a goal driven task.

2.

It begins with new data.

It begins with conclusions that are uncertain.

3.

The objective is to find a conclusion that would follow.

The objective is to find the facts that support the conclusions.

4.

It uses an opportunistic type of approach.

It uses a conservative type of approach.

5.

It flows from incipient to the consequence.

It flows from consequence to the incipient.

6.

Forward reasoning begins with the initial facts.

Backward reasoning begins with some goal (hypothesis).

7.

Forward reasoning tests all the rules.

Backward reasons tests some rules.

8.

Forward reasoning is a bottom-up approach.

Backward reasoning is a top-down approach

9.

Forward reasoning can produce an infinite number of conclusion.

Backward reasoning produces a finite number of conclusions.

10.

In the forward reasoning, all the data is available.

In the backward reasoning, the data is acquired on demand.

11.

Forward reasoning has a small number of initial states but a large number of conclusions.

Backward reasoning has a smaller number of goals and a larger number of rules.
12.

In forward reasoning, the goal formation is difficult.

In backward reasoning, it is easy to form a goal.

13.

Forward reasoning works in forward direction to find all the possible conclusions from facts.

Backward reasoning work in backward direction to find the facts that justify the goal.

14.

Forward reason is suitable to answer the problems such as planning, control, monitoring, etc.

Backward reasoning is suitable for diagnosis like problems.