Shraddha Kavale

A09 / TU4F2122008
Experiment 1
Aim: Implement Inferencing with Bayesian Network in Python
REQUIREMENTS: Anaconda Distribution – Python Platform/ Jupyter Notebook/ PyCharm
IDE/ python IDE/ Google COLAB – Cloud based Jupyter Notebook Environment
PRE-REQUISITES: Python Standard Libraries
THEORY:
What is Bayesian Network?
A Bayesian network (also spelt Bayes network , Bayes net, belief network , or judgment
network) is a probabilistic graphical model that depicts a set of variables and their conditional
dependencies using a directed acyclic graph (DAG). Bayesian networks are perfect for taking
an observed event and forecasting the likelihood that any of numerous known causes played a
role. A Bayesian network , for example , could reflect the probability correlations between
diseases and symptoms . Given a set of symptoms , the network may be used to calculate the
likelihood of the presence of certain diseases.
What is Directed Acyclic Graph (DAG)?
In graph theory and computer science , a directed acyclic graph (DAG ) is a directed graph
with no directed cycles. In other words, it’s made up of vertices and edges (also called arcs),
with each edge pointing from one vertex to the next in such a way that following those
directions would never lead to a closed-loop as depicted in below picture.

A directed graph is one in which all edge directions are consistent and the vertices can be
topologically arranged in a linear order. DAGs have various scientific and computing
applications, including biology evolution, family trees, and epidemiology, and sociology.
The Maths Behind the Bayesian Network
An acyclic directed graph is used to create a Bayesian network, which is a probability model.
It’s factored by utilizing a single conditional probability distribution for each variable in the
model, whose distribution is based on the parents in the graph. The simple principle of
probability underpins Bayesian models. So, first, let’s define conditional probability and joint
probability distribution.
Conditional Probability
Conditional probability is a measure of the likelihood of an event occurring provided that
another event has already occurred (through assumption, supposition, statement, or evidence).
If A is the event of interest and B is known or considered to have occurred, the conditional
probability of A given B is generally stated as P(A|B) or, less frequently, PB(A) if A is the
event of interest and B is known or thought to have occurred. This can also be expressed as a
percentage of the likelihood of B crossing with A.
Joint Probability
The chance of two (or more) events together is known as the joint probability. The sum of
the probabilities of two or more random variables is the joint probability distribution. For
example, the joint probability of events A and B is expressed formally as: The letter P is the
first letter of the alphabet (A and B).
The upside-down capital “U” operator or, in some situations, a comma “,” represents the
“and” or conjunction. P(A ^ B) P(A, B) By multiplying the chance of event A by the
likelihood of event B, the combined probability for occurrences A and B is calculated.
Posterior Probability
In Bayesian statistics, the conditional probability of a random occurrence or an ambiguous
assertion is the conditional probability given the relevant data or background. “After taking
into account the relevant evidence pertinent to the specific subject under consideration,”
“posterior” means in this case. The probability distribution of an unknown quantity
interpreted as a random variable based on data from an experiment or survey is known as the
posterior probability distribution.
Inference over a Bayesian network can come in two forms.
The first is simply evaluating the joint probability of a particular assignment of values for
each variable (or a subset) in the network. For this, we already have a factorized form of the
joint distribution, so we simply evaluate that product using the provided conditional
probabilities. If we only care about a subset of variables, we will need to marginalize out the
ones we are not interested in. In many cases, this may result in underflow, so it is common to
take the logarithm of that product, which is equivalent to adding up the individual logarithms
of each term in the product. The second, more interesting inference task, is to find P(x|e), or,
to find the probability of some assignment of a subset of the variables (x) given assignments
of other variables (our evidence, e). In the above example, an example of this could be to find
P(Sprinkler, WetGrass
| Cloudy), where {Sprinkler, WetGrass} is our x, and {Cloudy} is our e. In order to calculate
this, we use the fact that P(x|e) = P(x, e) / P(e) = αP(x, e), where α is a normalization constant
that we will calculate at the end such that P(x|e) + P(¬x | e) = 1. In order to calculate P(x, e),
we must marginalize the joint probability distribution over the variables that do not appear in
x or e, which we will denote as Y.For the given example, we can calculate P(Sprinkler,
WetGrass | Cloudy) as follows: We would calculate P(¬x | e) in the same fashion, just setting
the value of the variables in x to false instead of true. Once both P(x | e) and P(¬x | e) are
calculated, we can solve for α, which equals 1 / (P(x | e) + P(¬x | e)). Note that in larger
networks, Y will most likely be quite large, since most inference tasks will only directly use a
small subset of the variables. In cases like these, exact inference as shown above is very
computationally intensive, so methods must be used to reduce the amount of computation.
One more efficient method of exact inference is through variable elimination, which takes
advantage of the fact that each factor only involves a small number of variables. This means
that the summations BEIT Sem VII Data Science Lab Manual4can be rearranged such that
only factors involving a given variable are used in the marginalization of that variable.
Alternatively, many networks are too large even for this method, so approximate inference
methods such as MCMC are instead used; these provide probability estimations that require
significantly less computation than exact inference methods.
IMPLEMENATATION
1. Installing pgmpy, a python library for working with graphical models. It al- lows the
user to create their own graphical models and answer inference or map queries over
them. pgmpy has implementation of many inference algorithms like
VariableElimination, Belief Propagation etc.

2. Creating class to represent Naive Bayes. Naive Bayes is a special case of Bayesian
Model where the only edges in the model are from the feature variables to the
dependent variable. Creating a discrete random variable; conditional on zero or more
parent Variables; a class for Probability Distribution mapping; a class for normalizing
to sum to 1, a class for mapping of {row: ProbDist, ...} where each row is a tuple of
values of the parent variables; a class for `bool`, except values display as 'T' and 'F'
instead of 'True' and 'False
3. Creating some associated functions for the probability distribution for P(variable |
evidence), when all parent variables are known (in evidence), Normalizing a {key:
value} distribution so values sum to 1.0. Mutates dist and returns it and for Random
 sample an outcome from a probability distribution. Creating create the joint distribution
over all variables, the probability that var = xi, given the values in this row

4. Defining network structure and making Burglary, Earthquake, etc. be global variables

5. Probability distribution of a Burglary, Probability distribution of a Earthquake,

7. Probability that "the alarm has not sounded, but neither a burglary nor an earthquake
has occurred, and both John and Mary call"

Conclusion:
Bayesian inference is based on using probability to represent all forms of uncertainty.
Bayesian inference is a method of statistical inference in which Bayes' theorem is used to
update the probability for a hypothesis as more evidence or information becomes available.
The idea of Bayesian learning is to compute the posterior probability distribution of the
target features of a new example conditioned on its input features and all of the training
examples.