EXPERIMENT NO.

: 1

Aim: Learning rules and activation functions in Neural Network

Software: SCILAB

Theory:
Activation functions are an integral building block of neural networks that enable
them to learn complex patterns in data. They transform the input signal of a node in a neural
network into an output signal that is then passed on to the next layer. Without activation
functions, neural networks would be restricted to modeling only linear relationships between
inputs and outputs.

Why Are Activation Functions Essential?

Without activation functions, neural networks would just consist of linear operations like
matrix multiplication. All layers would perform linear transformations of the input, and no
non-linearities would be introduced.
Most real-world data is non-linear. For example, relationships between house prices and size,
income, and purchases, etc., are non-linear. If neural networks had no activation functions,
they would fail to learn the complex non-linear patterns that exist in real-world data.
Activation functions enable neural networks to learn these non-linear relationships by
introducing non-linear behaviors through activation functions. This greatly increases the
flexibility and power of neural networks to model complex and nuanced data.

Types of Activation Functions

Neural networks leverage various types of activation functions to introduce non-linearities and
enable learning complex patterns. Each activation function has its own unique properties and
is suitable for certain use cases.
For example, the sigmoid function is ideal for binary classification problems, softmax is useful
for multi-class prediction, and ReLU helps overcome the vanishing gradient problem.
Using the right activation function for the task leads to faster training and better performance.
Let’s look at some of the common activation functions:
Linear activation

The linear activation function is the simplest activation function, defined as:
f(x) = x
It simply returns the input x as the output. Graphically, it looks like a straight line with a slope
of 1.
The main use case of the linear activation function is in the output layer of a neural network
used for regression. For regression problems where we want to predict a numerical value,
using a linear activation function in the output layer ensures the neural network outputs a
numerical value. The linear activation function does not squash or transform the output, so the
actual predicted value is returned.
However, the linear activation function is rarely used in hidden layers of neural networks. This
is because it does not provide any non-linearity. The whole point of hidden layers is to learn
non-linear combinations of the input features. Using a linear activation throughout would
restrict the model to just learning linear transformations of the input.

Sigmoid activation
The sigmoid activation function, often represented as σ(x), is a smooth, continuously
differentiable function that is historically important in the development of neural networks.
The sigmoid activation function has the mathematical form:
f(x) = 1 / (1 + e^-x)
It takes a real-valued input and squashes it to a value between 0 and 1. The sigmoid function
has an "S"-shaped curve that asymptotes to 0 for large negative numbers and 1 for large
positive numbers. The outputs can be easily interpreted as probabilities, which makes it natural
for binary classification problems.

Tanh (hyperbolic tangent) activation

The tanh (hyperbolic tangent) activation function is defined as: f(x)

= (e^x - e^-x) / (e^x + e^-x)
The tanh function outputs values in the range of -1 to +1. This means that it can deal with
negative values more effectively than the sigmoid function, which has a range of 0 to 1.
Unlike the sigmoid function, tanh is zero-centered, which means that its output is symmetric
around the origin of the coordinate system. This is often considered an advantage because it
can help the learning algorithm converge faster.
Because the output of tanh ranges between -1 and +1, it has stronger gradients than the sigmoid
function. Stronger gradients often result in faster learning and convergence during training
because they tend to be more resilient against the problem of vanishing gradients when
compared to the gradients of the sigmoid function.

ReLU (rectified linear unit) activation

The Rectified Linear Unit (ReLU) activation function has the form:
f(x) = max(0, x)
It thresholds the input at zero, returning 0 for negative values and the input itself for positive
values.

For inputs greater than 0, ReLU acts as a linear function with a gradient of 1. This means that
it does not alter the scale of positive inputs and allows the gradient to pass through unchanged
during backpropagation. This property is critical in mitigating the vanishing gradient problem.
Even though ReLU is linear for half of its input space, it is technically a non-linear function
because it has a non-differentiable point at x=0, where it abruptly changes from x. This
nonlinearity allows neural networks to learn complex patterns
Since ReLU outputs zero for all negative inputs, it naturally leads to sparse activations; at any
time, only a subset of neurons are activated, leading to more efficient computation.

Softmax activation
The softmax activation function, also known as the normalized exponential function, is
particularly useful within the context of multi-class classification problems. This function
operates on a vector, often referred to as the logits, which represents the raw predictions or
scores for each class computed by the previous layers of a neural network. For input vector x
with elements x1, x2, ..., xC, the SoftMax function is defined as: f(xi) = e^xi / Σj e^xj The
output of the SoftMax function is a probability distribution that sums up to one. Each element
of the output represents the probability that the input belongs to a particular class.
The use of the exponential function ensures that all output values are non-negative. This is
crucial because probabilities cannot be negative.

Choosing the Right Activation Function

The choice of activation function depends on the type of problem you are trying to solve. Here
are some guidelines: For binary classification
Use the sigmoid activation function in the output layer. It will squash outputs between 0 and
1, representing probabilities for the two classes.
For multi-class classification
Use the SoftMax activation function in the output layer. It will output probability distributions
over all classes.
If unsure
Use the ReLU activation function in the hidden layers. ReLU is the most common default
activation function and usually a good choice.

Conclusion
 Experiment No.: 02

Aim: Development of logic using MP and Hebb neuron model.

Theory:

1.The MP (McCulloch-Pitts) model :

The MP (McCulloch-Pitts) model, also known as the threshold logic unit, is a
simplified model of a neuron in a neural network. It was introduced by Warren McCulloch
and Walter Pitts in 1943 and served as a foundational concept for subsequent developments
in artificial neural networks.

The MP model represents a binary neuron that takes multiple binary inputs and produces a
binary output based on a fixed threshold. It captures the basic functionality of a real neuron
by modelling the integration of input signals and the activation of the neuron based on a
threshold.

Here are the key characteristics of the MP model:

1.Binary Inputs: The model takes binary inputs, where each input can be either 0 or 1. These
inputs represent the input signals to the neuron.
2.Weights (w): Each input is associated with a weight, which determines the strength or
importance of that input. The weights can be positive, negative, or zero.
3.Threshold (𝜃): The model has a fixed threshold value, which is a predetermined threshold
that the sum of the weighted inputs must exceed for the neuron to produce an output.
4.Weighted Sum: The inputs are multiplied by their respective weights, and the weighted sum
of these inputs is computed.

2. Activation Function:
The weighted sum is compared to the threshold using an activation function. If the
weighted sum exceeds the threshold, the neuron fires and produces an output of 1; otherwise,
it remains inactive and produces an output of 0.

The MP model provides a binary decision mechanism, making it suitable for tasks that require
simple classification or decision-making. It forms the basis for more complex and powerful
neural network models, such as the perceptron and artificial neural networks with multiple
layers (multilayer perceptron), which allow for more flexible and sophisticated computations.
McCulloch-Pitts Neuron Activation Function

Logic Gate Implementation

AND Gate:

• Inputs: x1, x2
• Weights: w1=1, w2=1
• Threshold: 𝜃 =2

Logic for AND Gate

• If both inputs are 1 (i.e., 𝑥1+x2= 2), the output is 1.

• For other combinations of inputs, the output is 0.

X1 X2 Output(AND)
0 0 0
0 1 0
1 0 0
 1 1 1
The Hebbian learning rule :

The Hebbian learning rule, proposed by Donald Hebb in 1949, is a simple and
fundamental principle in neural network learning. It states that "cells that fire together wire
together," meaning that the strength of the connection between two neurons increases when
they are activated simultaneously.

The Hebbian learning rule is based on the idea of synaptic plasticity, which refers to the ability
of the connections (synapses) between neurons to change in strength. It provides a mechanism
for learning and memory formation in neural networks.

The basic principle of the Hebbian learning rule can be summarized as follows:

1.If two connected neurons are both activated (fire) at the same time, the strength of the
connection between them is strengthened.

2. If two connected neurons are not activated together, or only one of them is activated, the
connection between them remains unchanged or weakens.

This rule is often expressed mathematically as:

ΔW = η * X * Y

• ΔW represents the change in synaptic weight (connection strength).

• η (eta) is the learning rate or the scaling factor that determines the magnitude of weight
change.
• X and Y are the activities (outputs) of the pre-synaptic and post-synaptic neurons,
respectively.

The Hebbian learning rule can be applied in a single neuron or in a network of neurons. It
allows the network to learn associations between input patterns and modify its connection
strengths accordingly.

It's important to note that while the Hebbian learning rule is a fundamental concept in neural
network learning, it has certain limitations and can lead to instability or over fitting in certain
scenarios. Therefore, modern neural network architectures often employ more advanced
learning algorithms, such as back propagation, that address these limitations and offer more
efficient and robust learning mechanisms
AND Gate using Hebbian Learning

• Inputs: x1, x2
• Initially: w1=0, w2=0
• Learning rate: η =1

Training Process

1.For input (0, 0), desired output is 0: Weights remain unchanged.

2. For input (0, 1), desired output is 0: Weights remain unchanged.

3. For input (1, 0), desired output is 0: Weights remain unchanged.

4. For input (1, 1), desired output is 1: Update the weights using Hebb's rule

Final weights: w_{1} = 1 w_{2} = 1

Conclusion:
 EXPERIMENT NO.: 3

Aim: To study Development of supervised learning using NN Toolbox

Software: SCILAB Theory:
Supervised learning is the most common type of machine learning algorithms. It uses a
known datasets (called the training datasets) to train an algorithm with a known set of input
data (called features) and known responses to make predictions. The training datasets includes
labelled input data that pair with desired outputs or response values. From it, the supervised
learning algorithm seeks to create a model by discovering relationships between the features
and output data and then makes predictions of the response values for a new datasets.

In supervised learning with a neural network, the network learns a mapping from inputs to
outputs by adjusting its internal weights and biases. The process involves forward propagation,
where inputs are passed through the network to produce outputs, and backpropagation, where
the errors are used to update the weights. Key elements include the choice of activation
function, loss function, optimization method, and regularization techniques to prevent
overfitting. The Neural Network Toolbox simplifies these processes, allowing users to focus
on high-level model design and evaluation.

Supervised Learning Algorithms

 Supervised learning algorithms can be broadly categorized into two main types based on
the nature of the output variable:

1. Regression Algorithms
These algorithms are used when the output variable is continuous, meaning the target is
a real value. The goal of regression is to predict a continuous number based on the input data.
Used for numerical continuous-response values. Regression models can be easily trained with
the Regression Learner app.
Common Algorithms:
Linear Regression: A method that models the relationship between a dependent variable and
one or more independent variables using a linear equation.
Ridge and Lasso Regression: Extensions of linear regression that include regularization to
reduce overfitting.
Polynomial Regression: Extends linear regression to account for non-linear relationships by
introducing polynomial terms.
Support Vector Regression (SVR): Uses support vectors for regression tasks, similar to
SVMs for classification.
Decision Trees for Regression: Builds a tree-like model to predict a continuous value by
splitting data based on feature values.
Supervised learning algorithms can be broadly categorized into two main types based on the
nature of the output variable: Uses deep learning models to map inputs to continuous
outputs.

2. Classification Algorithms
These algorithms are used when the output variable is categorical, meaning the target belongs
to a set of predefined categories or classes. The goal of classification is to assign the input to
one of these categories. Used for categorical response values, where the data can be separated
into specific classes. A binary classification model has two classes and a multiclass
classification model has more. You can train classification models with the Classification
Learner app with MATLAB.

Common Algorithms:
Logistic Regression: A statistical method for binary classification that models the probability
of a binary outcome.
Support Vector Machines (SVM): A classification algorithm that finds the optimal
hyperplane to separate data points of different classes.
K-Nearest Neighbours (KNN): A simple algorithm that classifies new data points based on
the majority class of their nearest neighbours.
Decision Trees for Classification: A tree-like model where internal nodes represent features,
branches represent decision rules, and leaves represent class labels.
Random Forest: An ensemble method using multiple decision trees to improve classification
accuracy and reduce overfitting.
Naive Bayes: A probabilistic classifier based on Bayes' theorem, often used for text
classification.
Neural Networks for Classification: Deep learning models like feedforward and
convolutional neural networks (CNNs) are commonly used for image, text, and other complex
classification tasks.

Supervised Learning using NN Toolbox

Supervised learning is a machine learning approach where a model is trained on a labelled
dataset, meaning the input data is associated with the correct output. The goal is to learn a
function that maps input data to output labels as accurately as possible. When using a Neural
Network (NN) Toolbox for supervised learning, this approach can be divided into several
theoretical concepts.
1. Supervised Learning Overview
In supervised learning, the datasets consist of input-output pairs:
Input (X): A set of features representing the data.
Output (Y): The corresponding label or target for each input.
The neural network learns to map inputs to outputs by adjusting the weights and biases of its
internal neurons during the training process. After training, the network can predict the outputs
for new, unseen inputs based on what it has learned.

2. Neural Network Structure

A neural network is composed of multiple layers of interconnected nodes or neurons:
Input Layer: Takes the input features (e.g., image pixels, sensor readings).
Hidden Layers: These layers perform the actual computation, learning complex patterns
through weighted connections.
Output Layer: Produces the predicted output (for classification, regression, etc.).
Each neuron performs a weighted sum of its inputs, passes this sum through an activation
function, and propagates the result forward to the next layer.
Mathematically, the process can be represented as:
y = f (W x + b) is the matrix of weights, is
the input vector,
is the bias vector, is the activation function (e.g., sigmoid, Re
LU),
is the output of the neuron.

3. Forward Propagation
In supervised learning with neural networks, the model is trained to predict outputs given
inputs. During forward propagation, the input data is passed through each layer of the network.
Each neuron in the hidden and output layers applies its activation function to the weighted
sum of its inputs and passes the result forward.
The final output is compared with the true target using a loss function (e.g., mean squared
error for regression or cross-entropy for classification), and the difference (error) is computed.

4. Loss Function
The loss function (also called the cost function) measures how well the neural network’s
predictions match the true targets. For different tasks, different loss functions are used:
Regression: Mean Squared Error (MSE) is often used.
MSE = \frac{1}{n}/sum {I =1} ^{n} (y _i - \hat{y}_i) ^2
Classification: Cross-Entropy Loss is used for tasks where the output is categorical.
(p, q) = - \sum p(x) \log(q(x))
The goal is to minimize this loss function, which means the neural network is producing
predictions close to the actual labels.

5. Backpropagation and Weight Update

Once the loss is calculated, the neural network needs to adjust its weights to reduce this loss.
This is done through backpropagation, an algorithm that computes the gradient of the loss
function with respect to each weight in the network.
6. Activation Functions
The activation function introduces non-linearity to the network, allowing it to learn complex
patterns. Without these non-linearities, a neural network would essentially act like a linear
regression model. Common activation functions include:
Sigmoid: Squashes the output between 0 and 1, commonly used for binary classification tasks.
\sigma(x) = \frac{1}{1 + e^{-x}}
Re LU (Rectified Linear Unit): Introduces non-linearity by setting all negative values to 0 and
leaving positive values unchanged. f(x) = \max(0, x)
Tanh: Similar to sigmoid but ranges between -1 and 1.
\tanh(x) = \frac{2}{1 + e^{-2x}} - 1
The choice of activation function affects the performance and behaviour of the network.

7.Training and Generalization

The goal of training the neural network is to make it generalize well to unseen data, not just
to memorize the training set. Overfitting happens when the network performs well on the
training data but poorly on validation/test data.

8.Learning Rate and Optimization

The learning rate controls how much the weights are adjusted during each step of
backpropagation. If the learning rate is too high, the network may overshoot the optimal
weights and fail to converge. If it’s too low, training can become very slow or get stuck in
local minima.
Common optimization algorithms used in neural networks include:
Gradient Descent: Iteratively adjusts weights in the direction of the negative gradient.
Adam: An adaptive optimization method that adjusts the learning rate based on first and
second moments of the gradients.
9. Supervised Learning in NN Toolbox
In MATLAB’s Neural Network Toolbox, these steps are abstracted through built-in functions.
The toolbox provides simple ways to:
Define network architectures (input, hidden, and output layers).
Choose appropriate activation functions and loss functions.
Train the network using various algorithms.
Split data into training, validation, and testing subsets.
Evaluate the model's performance on new data.
Code:-
% Load Iris datasets
[X, T] = iris_datasets;

% Create a feedforward neural network with one hidden layer of 10 neurons net =
patternnet(10);
% Set the training, validation, and testing ratios net.divideParam.trainRatio =
0.7; % 70% for training net.divideParam.valRatio = 0.15; % 15% for
validation net.divideParam.testRatio = 0.15; % 15% for testing

% Train the network

[net, tr] = train(net, X, T);

% Test the trained network

testX = X(:, tr.testInd); testT
= T(:, tr.testInd); testY =
net(testX);
% Evaluate the performance on the test set testPerformance = perform(net, testT, testY);
disp(['Test Performance (Cross-Entropy Loss): ',
num2str(testPerformance)]);
% Plot confusion matrix to visualize classification results figure,
plotconfusion(testT, testY); % View network architecture
view(net);
Output
Conclusion:
 EXPERIMENT NO.: 04

Aim: Development and testing of perceptron NN algorithm.

Software Requirement: Jupyter Notebook

Theory :-
Perceptron is one of the first and most straightforward models of artificial neural
networks. Despite being a straightforward model, the perceptron has been proven to be
successful in solving specific categorization issues. Perceptron is one of the simplest
Artificial neural network architectures. It was introduced by Frank Rosenblatt in 1957s. It is
the simplest type of feedforward neural network, consisting of a single layer of input nodes
that are fully connected to a layer of output nodes. It can learn the linearly separable patterns.
it uses slightly different types of artificial neurons known as threshold logic units (TLU).

Types of Perceptron

• Single-Layer Perceptron: This type of perceptron is limited to learning linearly

separable patterns. effective for tasks where the data can be divided into distinct
categories through a straight line.
• Multilayer Perceptron: Multilayer perceptrons possess enhanced processing
capabilities as they consist of two or more layers, adept at handling more complex
patterns and relationships within the data.

Basic Components of Perceptron

A perceptron, the basic unit of a neural network, comprises essential components that
collaborate in information processing.

• Input Features: The perceptron takes multiple input features, each input feature
represents a characteristic or attribute of the input data.
• Weights: Each input feature is associated with a weight, determining the significance
of each input feature in influencing the perceptron’s output. During training, these
weights are adjusted to learn the optimal values.
• Summation Function: The perceptron calculates the weighted sum of its inputs
using the summation function. The summation function combines the inputs with
their respective weights to produce a weighted sum.
• Activation Function: The weighted sum is then passed through an activation
function. Perceptron uses Heaviside step function functions. which take the summed
values as input and compare with the threshold and provide the output as 0 or 1.

• Output: The final output of the perceptron, is determined by the activation function’s
result. For example, in binary classification problems, the output might represent a
predicted class (0 or 1).
• Bias: A bias term is often included in the perceptron model. The bias allows the model
to make adjustments that are independent of the input. It is an additional parameter
that is learned during training.
• Learning Algorithm (Weight Update Rule): During training, the perceptron learns
by adjusting its weights and bias based on a learning algorithm. A common approach
is the perceptron learning algorithm, which updates weights based on the difference
between the predicted output and the true output.

How does Perceptron work?

A weight is assigned to each input node of a perceptron, indicating the significance
of that input to the output. The perceptron’s output is a weighted sum of the inputs
that have been run through an activation function to decide whether or not the
perceptron will fire.

z = w1x1 + w1x2 + ... + wnxn = XTW

The step function compares this weighted sum to the threshold, which outputs 1 if
the input is larger than a threshold value and 0 otherwise, is the activation function
that perceptrons utilize the most frequently.

A perceptron has a single layer of threshold logic units with each TLU connected to

all inputs.
The output of the fully connected layer can be:
 where X is the input W is the weight for each inputs neurons and b is the bias and h is
the step function.
During training, The perceptron’s weights are adjusted to minimize the difference
between the predicted output and the actual output. Usually, supervised learning algorithms
like the delta rule or the perceptron learning rule are used for this.

Steps in the Perceptron Learning Algorithm

1. Weighted Sum Calculation:

The algorithm calculates a weighted sum of inputs plus a bias term:
z=∑i=1nwixi+bz=∑i=1nwixi+b
where wiwi are weights, xixi are input values, and bb is the bias

2. Activation:
The weighted sum is passed through an activation function to produce an output:

• If z>thresholdz>threshold, output is 1 (or true).

• Otherwise, output is 0 (or false)

3. Error Calculation:
The difference between predicted and actual outputs is computed to find the error
term: E=y−y^E=y−y^ where yy is the actual output and y^y^ is the predicted output
4. Weight Update:
Weights are adjusted based on the error using a learning rate αα:
wi= wi+αExiwi=wi+αExi
 This step aims to minimize future errors by refining the model's parameters

5. Iteration:
The process repeats over multiple epochs until convergence, meaning that weights
stabilize and further adjustments yield minimal changes in output accuracy.
Input:
 Output:

Conclusion:
 EXPERIMENT NO.: 5

Aim: To study Development of ADALINE algorithm with bipolar inputs and outputs
Software requirement: SCILAB

Theory:

An Artificial Neural Network inspired by the human neural system is a network used to
process the data which consist of three types of layer i.e input layer, the hidden layer, and the
output layer. The basic neural network contains only two layers which are the input and output
layers. The layers are connected with the weighted path which is used to find net input data.
In this section, we will discuss two basic types of neural networks Adaline which doesn’t have
any hidden layer, and Madaline which has one hidden layer.
1. Adaline (Adaptive Linear Neural) :
• A network with a single linear unit is called Adaline (Adaptive Linear Neural). A unit
with a linear activation function is called a linear unit. In Adaline, there is only one
output unit and output values are bipolar (+1,-1). Weights between the input unit and
output unit are adjustable. It uses the delta rule i.e. where and are the weight, predicted
output, and true value respectively.
• The learning rule is found to minimize the mean square error between activation and
target values. Adaline consists of trainable weights, it compares actual output with
calculated output, and based on error training algorithm is applied.
Workflow:

First, calculate the net input to your Adaline network then apply the activation function to its
output then compare it with the original output if both the equal, then give the output else send
an error back to the network and update the weight according to the error which is calculated
by the delta learning rule. i.e, where and are the weight, predicted output, and true value
respectively.

Architecture:

In Adaline, all the input neuron is directly connected to the output neuron with the weighted
connected path. There is a bias b of activation function 1 is present. Algorithm: `

Step 1: Initialize weight not zero but small random values are used. Set learning rate α.
Step 2: While the stopping condition is False do steps 3 to 7.
Step 3: for each training set perform steps 4 to 6.
Step 4: Set activation of input unit xi = si for (i=1 to n).
Step 5: compute net input to output unit Here, b is the bias and n is the total number of
neurons .
Step 6: Update the weights and bias for i=1 to n and calculate when the predicted output
and the true value are the same then the weight will not change.
Step 7: Test the stopping condition. The stopping condition may be when the weight
changes at a low rate or no change.
Implementations
Problem: Design OR gate using Adaline Network?
Solution:
• Initially, all weights are assumed to be small random values, say 0.1, and set learning
rule to 0.1.
• Also, set the least squared error to 2.
• The weights will be updated until the total error is greater than the least squared error.
 x1 x2 t

1 1 1

1 -1 1

-1 1 1

-1 -1 -1

• Calculate the net input (when x1=x2=1)

• Now compute, (t-yin)=(1-0.3)=0.7
• Now, update the weights and bias

• calculate the error

Similarly, repeat the same steps for other input vectors and you will get.

x t (tyin) ∆w1 ∆w2 ∆b w1 (0. w2 (0. b

1 1) 1) (0.1) (tyin)^
2
x2
yin

1 1 1 0.3 0.7 0.07 0.07 0.07 0.17 0.17 0.17 0.49

1 -1 1 0.17 0.83 0.083 - 0.083 0.253 0.087 0.253 0.69

0.083

- 1 1 0.087 0.913 - 0.091 0.091 0.161 0.178 0.344 0.83

1 0.091 3 3 7 3 3
3
 - -1 - 0.004 - 0.100 0.100 - 0.262 0.278 0.243 1.01
1 1 3 1.004 4 4 0.100 1 7 9
3 4

This is epoch 1 where the total error is 0.49 + 0.69 + 0.83 + 1.01 = 3.02 so more epochs will
run until the total error becomes less than equal to the least squared error i.e 2.
• Python3

# Import necessary libraries import numpy as np

#Adaline neural network def Adaline(Input, Target, lr=0.2, stop=0.001): weight =

np.random.random(Input.shape[1]) bias = np.random.random(1)

Error=[stop +1]
# check the stop condition for the network while Error[-1] > stop or Error[-1]-Error[-2] >
0.0001:
error = [] for i in range(Input.shape[0]):
Y_input = sum(weight*Input[i]) + bias

# Update the weight for j in range(Input.shape[1]):

weight[j]=weight[j] + lr*(Target[i]-Y_input)*Input[i][j]
 # Update the bias bias=bias + lr*(Target[i]-Y_input)

# Store squared error value error.append((Target[i]-

Y_input)**2)
# Store sum of square errors
Error.append(sum(error)) print('Error
:',Error[-1]) return weight, bias

# Input dataset x = np.array([[1.0,

1.0, 1.0],
[1.0, -1.0, 1.0],
[-1.0, 1.0, 1.0],
[-1.0, -1.0, -1.0]])
# Target values t = np.array([1,
1, 1, -1])

w,b = Adaline(x, t, lr=0.2, stop=0.001)

print('weight :',w) print('Bias :',b)

Output:
Error : [2.33228319]
Error : [1.09355784]
Error : [0.73680883]
Error : [0.50913731]
Error : [0.35233593]
Error : [0.24384625] Error : [0.16876305] Error : [0.01283534]
Error : [0.00888318]
Error : [0.00614795]
Error : [0.00425492]
Error : [0.00294478]
Error : [0.00203805]
Error : [0.00141051] Error : [0.0009762] weight
: [0.01081771 0.01081771
0.98675106]
Bias : [0.01081771] Predictions:
Predict from the evaluated weight and bias of Adaline
Python3

# Predict from the evaluated weight and bias of adaline

def prediction(X,w,b):
y=[] for i in range(X.shape[0]):
x = X[i]
y.append(sum(w*x)+b) return
y prediction(x,w,b)

output: [array([1.0192042]),
array([0.99756877]),
array([0.99756877]), array([-
0.99756877])]

conclusion:
 EXPERIMENT NO.:6

Aim: Fuzzy Logic-Based PID Controller for Pressure Regulation in Gas Pipeline

Software: PYTHON

Theory:

To develop a Fuzzy Logic-based tuning mechanism for a PID controller to maintain

pressure in a gas pipeline. The fuzzy system will dynamically adjust the PID parameters
(Kp, Ki, Kd) based on process conditions, ensuring optimal performance under varying
load and pressure disturbances.

Steps:

1. Problem Definition:

Pressure regulation in a gas pipeline system is a critical task. Traditional PID controllers
often struggle to maintain precise control due to varying demand, changes in gas flow,
and environmental conditions, resulting in oscillations or overshoot. A Fuzzy Logic
system will be used to tune the PID parameters in real-time to handle these complexities.

2. System Overview:

• Input: Error (difference between setpoint and current pressure), and rate of change
of error (derivative of error).
• Output: PID parameters (Kp, Ki, Kd) adjustment.
• Controller: A PID controller tuned by Fuzzy Logic.

3. Fuzzy Logic Controller Design:

a. Fuzzification:

Define the fuzzy sets for inputs (error, rate of change of error) and outputs (Kp, Ki,
Kd):
• Input 1: Pressure error (Pa): Negative Large (NL), Negative Medium (NM),
Negative Small (NS), Zero (Z), Positive Small (PS), Positive Medium (PM),
Positive Large (PL).
• Input 2: Rate of change of pressure error (Pa/s): Negative (N), Zero (Z), Positive
(P).
• Outputs: PID controller parameters (Kp, Ki, Kd): Kp (Proportional gain): Small
(S), Medium (M), Large (L).
 Ki (Integral gain): Small (S), Medium (M), Large (L).

Kd (Derivative gain): Small (S), Medium (M), Large (L).

b. Fuzzy Membership Functions:

Create membership functions for each fuzzy variable:

• Pressure error: Use triangular membership functions for NL, NM, NS, Z, PS,
PM, PL.
• Rate of change of error: Use triangular membership functions for N, Z, P.
• PID parameters: Use triangular membership functions for S, M, L. c. Rule

Base:

Define a set of rules that adjust the PID gains based on the error and rate of change of
error:

• Kp: Adjust the proportional gain based on the error:

If (error is NL) and (rate of change is N), then (Kp is L). If
(error is Z) and (rate of change is Z), then (Kp is S).
• Ki: Adjust the integral gain based on error:
If (error is PS) and (rate of change is P), then (Ki is M).
• Kd: Adjust the derivative gain to control oscillations:
If (error is PM) and (rate of change is P), then (Kd is L).
d. Inference Engine:
Use the Mamdani inference method to apply the rules and calculate the new values of Kp,
Ki, and Kd based on the current process conditions.

e. Defuzzification:

Use the centroid method to defuzzify the fuzzy outputs and generate the crisp values of
Kp, Ki, and Kd that will be used by the PID controller.

4. Neural Network Enhancement (Optional):

You can use a Neural Network to predict the optimal PID parameters by learning from
historical data. This would act as an adaptive mechanism to improve the fuzzy logic
system’s performance over time.

5. Simulation and Testing:

a. Simulation Tool:
Use MATLAB with the Fuzzy Logic Toolbox and Simulink to simulate the gas pipeline
pressure control system and the fuzzy-tuned PID controller.

b. Testing:

1. Create different test scenarios with varying pressure setpoints and external
disturbances.
2. Observe how the Fuzzy-PID system responds to changes, focusing on response
time, stability, and overshoot.
3. Compare the performance of the Fuzzy-PID controller with a traditional PID
controller.
c. Performance Metrics:
• Settling Time: Time taken to reach the steady state after a disturbance.
• Overshoot: Measure the maximum deviation from the setpoint.
• Steady-State Error: Difference between the desired pressure and actual pressure at
steady state.

6. Results:

• Display plots showing the control response under different conditions.

• Compare the performance of the Fuzzy Logic tuned PID controller to a standard
PID controller.
• Highlight the improvements in terms of faster response, reduced overshoot, and
better adaptability to disturbances.
Code Example for Fuzzy Logic Tuning of PID Controller in PYTHON:
import numpy as np import
skfuzzy as fuzz
from skfuzzy import control as ctrl

# Define the fuzzy variables (input and output)

error = ctrl.Antecedent(np.arange(-15, 16, 1), 'error') # Expanded range rate_of_error = ctrl.Antecedent(np.arange(-7,
8, 1), 'rate_of_error') # Expanded range
kp = ctrl.Consequent(np.arange(0, 101, 1), 'kp') ki = ctrl.Consequent(np.arange(0, 101, 1),
'ki') kd =
ctrl.Consequent(np.arange(0, 101, 1), 'kd')

# Define fuzzy membership functions for Error error['NL'] =

fuzz.trimf(error.universe, [-15, -10, -5]) error['NM'] =
fuzz.trimf(error.universe, [-10, -5, 0]) error['Z'] = fuzz.trimf(error.universe, [-
2, 0, 2]) error['PM'] = fuzz.trimf(error.universe, [0, 5, 10]) error['PL'] =
fuzz.trimf(error.universe, [5, 10, 15])

# Define fuzzy membership functions for Rate of Error

rate_of_error['N'] = fuzz.trimf(rate_of_error.universe, [-7, -5, 0]) rate_of_error['Z'] =
fuzz.trimf(rate_of_error.universe, [-1, 0, 1]) rate_of_error['P'] = fuzz.trimf(rate_of_error.universe, [0, 5, 7])

# Define fuzzy membership functions for Kp, Ki, Kd kp['S'] =

fuzz.trimf(kp.universe, [0, 0, 50]) kp['M'] =
fuzz.trimf(kp.universe, [25, 50, 75]) kp['L'] =
fuzz.trimf(kp.universe, [50, 100, 100]) ki['S'] =
fuzz.trimf(ki.universe, [0, 0, 50]) ki['M'] =
fuzz.trimf(ki.universe, [25, 50, 75]) ki['L'] =
fuzz.trimf(ki.universe, [50, 100, 100])
kd['S'] = fuzz.trimf(kd.universe, [0, 0, 50]) kd['M'] = fuzz.trimf(kd.universe, [25,
50, 75]) kd['L'] =
fuzz.trimf(kd.universe, [50, 100, 100])

# Define the fuzzy rules

rule1 = ctrl.Rule(error['NL'] & rate_of_error['N'], [kp['L'], ki['L'], kd['S']])
rule2 = ctrl.Rule(error['Z'] & rate_of_error['Z'], [kp['S'], ki['M'], kd['M']]) rule3
= ctrl.Rule(error['PL'] & rate_of_error['P'], [kp['M'], ki['S'], kd['L']])
rule4 = ctrl.Rule(error['PM'] & rate_of_error['P'], [kp['L'], ki['M'], kd['L']]) rule5 = ctrl.Rule(error['NM'] &
rate_of_error['Z'], [kp['M'], ki['M'], kd['M']]) rule6 = ctrl.Rule(error['Z'] & rate_of_error['N'], [kp['M'],
ki['M'], kd['S']])
rule7 = ctrl.Rule(error['PL'] & rate_of_error['N'], [kp['L'], ki['S'], kd['S']])

# Create the control system and simulation

pid_ctrl = ctrl.ControlSystem([rule1, rule2, rule3, rule4, rule5, rule6, rule7]) pid_simulation
= ctrl.ControlSystemSimulation(pid_ctrl)

# Example input values for error and rate of change of error error_input =
5.841275134629758 rate_of_error_input = 6.965029156946181
# Pass inputs to the Fuzzy Logic system pid_simulation.input['error'] = error_input
pid_simulation.input['rate_of_error'] = rate_of_error_input

# Print input values for debugging print(f"Input Error:

{error_input}") print(f"Input Rate of Error:
{rate_of_error_input}")

# Compute the output (Kp, Ki, Kd) pid_simulation.compute()

# Check if the output exists before accessing it if 'kp' in pid_simulation.output and 'ki' in
pid_simulation.output and 'kd' in pid_simulation.output: print(f"Kp:
{pid_simulation.output['kp']:.2f}") print(f"Ki:
{pid_simulation.output['ki']:.2f}") print(f"Kd:
{pid_simulation.output['kd']:.2f}") else:
print("Fuzzy logic system did not produce an output for Kp, Ki, or Kd. Please check the rules or input
values.")

Output:

Input Error: 5.841275134629758

Input Rate of Error: 6.965029156946181
Kp: 62.61
Ki: 37.39
Kd: 75.22
Conclusion:
 EXPERIMENT NO.: 7

Aim: Development of Fuzzy Membership Functions .

Theory: A function that specifies the degree to which a given input belongs to a set is
membership function. Degree of membership is the output of a membership function; this
value is always limited to between 0 and 1, also known as a membership value or membership
grade.

Membership functions are used in the fuzzification and defuzzification steps of a FLS (fuzzy
logic system), to map the non-fuzzy input values to fuzzy linguistic terms and vice versa.

Formal Definition of membership function

Let us consider fuzzy set A,

A = {(x, µA(x))| x Є X}
Where, µA(x) is called the membership function for the fuzzy set A. X
is referred to as the universe of discourse.
The membership function associates each element x Є X with a value in the interval [0, 1].
In fuzzy sets, each elements is mapped to [0,1] by membership function.

That is, μA: X €[0, 1], where [0,1] means real numbers between 0 and 1 (including 0,1).
Consequently, fuzzy set is with „vague boundary set‟ comparing with crisp set.

Membership functions: Parameterization and MATLAB command Formulation

1) Triangular Membership function (trimf)

2) Trapezoidal MF (trapmf)
3) Gaussian MF (gaussmf)
4) Generalized bell MF (gbellmf)
5) Sigmoid membership function (sigmf)
Implementation:
Conclusion:
 EXPERIMENT NO.:8

Aim: Development of logic for fuzzy relations.

Software: SCILAB
Theory:
Fuzzy relations extend the concept of classical relations by introducing the notion of partial
membership, inspired by fuzzy set theory. In classical set theory, elements either belong to a
set or do not, represented by binary values (0 or 1). In fuzzy relations, elements have degrees
of membership, ranging from 0 to 1, allowing for the modeling of vagueness and uncertainty.
The logic for fuzzy relations builds on this concept.
This overview captures the key components and logic of fuzzy relations. It builds on
fundamental principles of fuzzy sets, while extending relational concepts to handle degrees
of association between elements:
1.Fuzzy Sets Overview:
Unlike classical sets, where elements either belong or don’t belong to a set (binary
membership), fuzzy sets allow partial membership, described by a membership function µ:X
→ [0,1], where each element x in X has a degree of membership between 0 (not a member)
and 1 (full member).
2. Fuzzy Relations:
A fuzzy relation R is a fuzzy subset of the Cartesian product of two or more sets. If A and B
are fuzzy sets, a fuzzy relation R on A × B, is described by a membership function µR : A×B
→ [0,1], indicating the degree of relation between elements of A and B.
Binary Fuzzy Relations: When dealing with binary fuzzy relations between two fuzzy sets A
and B, the relation R can be represented as a matrix with degrees of membership for each
pair (a, b), where a є A and b є B.
3.Operations on Fuzzy Relations:
Operations in fuzzy relations parallel operations in classical relations but incorporate the idea
of fuzzy membership:
3.1 Union: The union of two fuzzy relations combines their degrees of membership. The
resulting membership function takes the maximum degree of membership from either
relation.
 • Formula: µR (a, b) = max (µR1 (a, b), µR2 (a, b))
3.2 Intersection: The intersection of two fuzzy relations considers the minimum degree of
membership. This operation captures the extent to which both relations hold true
simultaneously.

• Formula: µR (a, b) = min (µR1 (a, b), µR2 (a, b))

3.3 Complement: The complement of a fuzzy relation reverses the degree of membership,
reflecting the non-membership.

• Formula: μ¬R (a, b) = 1- µR (a, b)

3.4 Composition: Fuzzy composition allows you to combine fuzzy relations into a new
relation. This operation can be performed using several methods, with the most common
being the max-min composition.

• Formula: For two fuzzy relations RRR and SSS:

μC (a, c) = max (min (μR (a, b),μS(b, c)))c

4. Properties of Fuzzy Relations:

Many classical relation properties can be extended to fuzzy relations with some
modifications:

• Fuzziness: Unlike traditional binary relations that are either true or false, fuzzy
relations assign degrees of membership ranging from 0 to 1. This allows for partial
truths and reflects the ambiguity inherent in many real-world situations.

• Symmetry: A fuzzy relation RRR is symmetric if, for any elements xxx and yyy, the
degree of relation R(x,y)R(x, y)R(x,y) is equal to R(y,x)R(y, x)R(y,x). This property
is important in contexts where the relationship is inherently bidirectional.

• Reflexivity: A fuzzy relation is reflexive if, for every element xxx, the degree of
relation R(x,x)R(x,
x)R(x,x) is 1. This means that every element is related to itself to the fullest extent.

• Transitivity: A fuzzy relation is transitive if, for any elements xxx, yyy, and zzz, the
degree of relation R(x,z)R(x, z)R(x,z) is at least the minimum of R(x,y)R(x, y)R(x,y)
and R(y,z)R(y, z)R(y,z). This property helps to maintain consistency in relationships
across multiple elements.
5. Fuzzy Equivalence Relations:
A fuzzy relation is an equivalence relation if it satisfies reflexivity, symmetry, and
transitivity. Fuzzy equivalence relations are crucial for clustering and classification tasks
where elements are grouped based on their degree of similarity.

7. Applications:
 Decision-making

 Control systems

 Image processing

 Natural language processing

8. Algorithm:

To develop logic for a fuzzy relations program in Python are following

• Step 1:Define fuzzy sets: Define the fuzzy sets for the input and output variables.

• Step2:Define fuzzy rules: Define the fuzzy rules based on the given conditions.

• Step 3:Use domains, sets, and rules: Use these three parts to model your system.

• Step 4:Use set operations: Combine sets with set operations like &, |, and ~.

• Step 5:Use rules to map domains: Use rules to map the input domain to the output
domain.

INPUT
Class Fuzzy Set:
def __init__(self, name, membership):
self.name = name self
membership =
membership

def get membership (self, element):

 return self membership.get(element, 0.0)

def __repr__(self):
return f"{self.name}: {self.membership}"

class Fuzzy Relation:

def __init__(self, relation_name,
matrix): self.relation_name =
relation_name self.matrix = matrix

def get_degree(self, x, y):

return self.matrix.get((x, y), 0.0)

def __repr__(self):
return f"{self.relation_name}: {self.matrix}"
def fuzzy_union(relation1,
relation2):
result_matrix = {} for key in
set(relation1.matrix.keys()).union(relation2.matrix.keys()
):
result_matrix[key] = max(relation1.get_degree(*key), relation2.get_degree(*key))
return FuzzyRelation("Union", result_matrix)
def fuzzy_intersection(relation1,
relation2):
result_matrix = {} for key in
set(relation1.matrix.keys()).union(relation2.matrix.keys()
):
result_matrix[key] = min(relation1.get_degree(*key), relation2.get_degree(*key)) return
FuzzyRelation("Intersection", result_matrix) def fuzzy_composition(relation1, relation2):
result_matrix = {}
for (x, z) in
relation1.matrix.keys():
for (z2, y) in
relation2.matrix.keys():
if z == z2:
degree = min(relation1.get_degree(x, z), relation2.get_degree(z,
y)) if (x, y) in result_matrix:
result matrix[(x, y)] = max(result_matrix[(x, y)], degree)
else:
result matrix[(x, y)] = degree return
Fuzzy Relation("Composition",
result_matrix)

# Define fuzzy sets

A = Fuzzy Set("A", {'a': 0.7, 'b': 0.5, 'c': 0.0})
B = Fuzzy Set("B", {'a': 0.8, 'b': 0.6, 'd': 0.2})
# Define fuzzy relations
R1 = Fuzzy Relation("R1", {('a', 'x'): 0.6, ('b', 'y'): 0.4})
R2 = Fuzzy Relation("R2", {('x', '1'): 0.5, ('y', '2'): 0.3})

# Perform operations union_relation =

fuzzy_union(R1, R2)
intersection_relation =
fuzzy_intersection(R1, R2)
composition_relation =
fuzzy_composition(R1, R2)

# Print results
print(A)
 print(B)
print(R1
)
print(R2
)
print(uni
on_relati
on)

Output

A: {'a': 0.7, 'b': 0.5, 'c': 0.0}

B: {'a': 0.8, 'b': 0.6, 'd': 0.2}
R1: {('a', 'x'): 0.6, ('b', 'y'): 0.4}
R2: {('x', '1'): 0.5, ('y', '2'): 0.3}
Union: {('a', 'x'): 0.6, ('b', 'y'): 0.4, ('x', '1'): 0.5, ('y', '2'): 0.3}
Intersection: {('a', 'x'): 0.0, ('b', 'y'): 0.0, ('x', '1'): 0.0, ('y', '2'): 0.0}
Composition: {('a', '1'): 0.3, ('b', '2'): 0.0}

CONCLUSION: