Unit IV 1

Competitive learning Neural Network

A competitive learning neural network is a type of artificial neural

network that involves a competitive process in which neurons compete with
each other to become the "winner" or "firing" neuron.

The winning neuron is the one that produces the highest output in response
to a given input pattern, while the other neurons remain inactive.

The architecture of a competitive learning neural network typically involves

a single layer of neurons that are fully connected to the input layer.

Each neuron in the output layer is associated with a weight vector, which is
adjusted during training to represent a specific pattern or category in the
input data.

The learning process in a competitive learning network is unsupervised,

which means that the network does not require explicit labels or categories
for the input data.

Instead, the network learns to recognize patterns in the input data by

adjusting the weights of the neurons based on the competition between
them.

One of the most common types of competitive learning neural network is

the Kohonen Self-Organizing Map (SOM) network, which is named after
its inventor, Teuvo Kohonen.

In a Kohonen SOM network, the neurons are arranged in a

two-dimensional grid, with each neuron representing a specific region in the
input data space.

During training, the weights of the neurons are adjusted based on the
similarity between the input patterns and the weight vectors of the neurons.
The end result is a topographic map of the input data space, in which
1
Prof. P. B. Koli,Assistant Professor,SNJB COE,Chandwad
similar input patterns are represented by nearby neurons in the output
layer.

Competitive learning neural networks are commonly used in applications

such as clustering, data visualization, and pattern recognition. They
are particularly useful for identifying underlying structures or relationships in
complex data sets, and for reducing the dimensionality of high-dimensional
data.

Pattern clustering and feature mapping network is a type of neural network

that is used for unsupervised learning tasks such as pattern clustering and
feature mapping. This type of network is also known as a Self-Organizing
Map (SOM) or Kohonen network, after its inventor Teuvo Kohonen.

In a pattern clustering and feature mapping network, the neurons are

arranged in a two-dimensional grid, with each neuron representing a
specific region in the input data space. During training, the weights of the
neurons are adjusted based on the similarity between the input patterns
and the weight vectors of the neurons. The end result is a topographic map
of the input data space, in which similar input patterns are represented by
nearby neurons in the output layer.

The learning process in a pattern clustering and feature mapping network is

unsupervised, which means that the network does not require explicit
labels or categories for the input data. Instead, the network learns to
recognize patterns in the input data by adjusting the weights of the neurons
based on the competition between them.

One of the main applications of a pattern clustering and feature mapping

network is to reduce the dimensionality of high-dimensional data. By
representing the input data in a lower-dimensional space, it becomes
easier to visualize and analyze the underlying patterns and relationships.

Another application of pattern clustering and feature mapping networks is in

image processing and computer vision, where they can be used for tasks
such as image compression, feature extraction, and object recognition.
Overall, pattern clustering and feature mapping networks are a powerful
tool for exploratory data analysis and unsupervised learning tasks, and they
have been used successfully in a wide range of applications in fields such
as finance, medicine, and engineering.

Adaptive Resonance Theory (ART) networks are a family of neural

networks that were developed by Stephen Grossberg and his colleagues in
the 1980s and 1990s. These networks are designed to model how the brain
processes information and learn from experience, and they are particularly
useful for tasks such as pattern recognition, classification, and clustering.

ART networks are based on the concept of "resonance", which refers to the
ability of a system to selectively respond to certain inputs while ignoring
others. This concept is used to develop networks that can learn to
recognize patterns and classify them into different categories based on
their similarity to previously learned patterns.

There are several types of ART networks, including ART-1, ART-2, and
ART-3. Each type of network is designed for different types of tasks and
input data, but they all share some common features.

One of the key features of ART networks is their ability to dynamically

adjust their internal parameters in response to changes in the input data.
This means that the networks can learn new patterns without forgetting
previously learned ones, which is particularly important for applications
where the input data may be constantly changing.

Another important feature of ART networks is their ability to deal with noisy
or incomplete input data. The networks can use a process called "vigilance"
to determine whether an input pattern is similar enough to a previously
learned pattern to be classified as belonging to the same category. This
helps to ensure that the network does not become over-generalized and
can adapt to new situations.
Overall, ART networks are a powerful tool for a wide range of applications
in fields such as image recognition, speech recognition, and data analysis.
They are particularly useful for tasks that involve dynamic input data and
require the ability to learn and adapt over time.

Adaptive Resonance Theory (ART) models are a family of neural

networks that have several key features that distinguish them from other
types of neural networks. Some of the main features of ART models are:

1.​ Adaptive learning: ART models are designed to adapt to new input
data in a way that is similar to how the human brain adapts to new
experiences. They can learn to recognize new patterns and
categories without forgetting previously learned ones.
2.​ Vigilance control: ART models use a process called "vigilance" to
ensure that new input patterns are only classified into existing
categories if they are sufficiently similar. This helps to prevent
over-generalization and ensures that the network can adapt to new
situations.
3.​ Top-down feedback: ART models use a feedback mechanism from
higher-level categories to lower-level ones, which helps to refine the
category boundaries and improve the accuracy of the network's
classifications.
4.​ Competition and cooperation: ART models use a competitive process
to select the winning category for each input pattern, but also allow
for cooperation between categories to improve the overall
performance of the network.
5.​ Unsupervised learning: ART models are often used for unsupervised
learning tasks, where the network must learn to recognize patterns
and categories in the input data without explicit labels or categories.
6.​ Robustness to noise: ART models are designed to be robust to noisy
or incomplete input data, which makes them useful for real-world
applications where the input data may not be perfect.

Overall, the combination of adaptive learning, vigilance control, top-down

feedback, competition and cooperation, unsupervised learning, and
robustness to noise make ART models a powerful tool for a wide range of
applications in fields such as image recognition, speech recognition, and
data analysis.

Character recognition using Adaptive Resonance Theory (ART)

networks involves training the network to recognize a set of input patterns
representing characters and then using the network to classify new input
patterns into the appropriate category.

To train an ART network for character recognition, the network is presented

with a set of input patterns, typically binary images of characters. The
network then adjusts its internal parameters, such as the vigilance level
and the weights of the neurons, based on the similarity between the input
patterns and the category prototypes. The prototypes represent the
average values of all the input patterns in each category.

Once the network is trained, it can be used to classify new input patterns
into the appropriate category. The input pattern is presented to the network,
and the network calculates the similarity between the input pattern and the
category prototypes. The network then selects the category with the
highest similarity value as the classification result.

The ART network is particularly useful for character recognition because it

can handle noisy or incomplete input patterns and can adapt to new
patterns without forgetting previously learned ones. The vigilance level of
the network can be adjusted to control the trade-off between sensitivity to
new patterns and specificity to existing categories, which makes the
network highly customizable for different applications.

Overall, character recognition using ART networks is a powerful tool for a

wide range of applications, including optical character recognition (OCR),
handwriting recognition, and document analysis. It has been used
successfully in fields such as finance, healthcare, and education.
There are two basic feature mapping models in neural networks: the
Self-Organizing Map (SOM) and the Learning Vector Quantization (LVQ)
network.

1.​ Self-Organizing Map (SOM): The SOM is a type of unsupervised

neural network that is used for clustering, visualization, and feature
extraction. It is based on the idea of competitive learning, where each
neuron in the network competes with the other neurons to represent a
certain region of the input space. The SOM consists of a 2D or 3D
grid of neurons, where each neuron is connected to the input space.
During training, the weights of the neurons are adjusted to match the
input patterns. The neurons that are close to each other in the grid
represent similar patterns, which allows the SOM to visualize the
high-dimensional input space in a low-dimensional grid.
2.​ Learning Vector Quantization (LVQ) network: The LVQ network is a
type of supervised neural network that is used for classification and
pattern recognition. It is based on the idea of competitive learning and
winner-takes-all, where each neuron in the network represents a
class or category. During training, the weights of the neurons are
adjusted to match the input patterns, and the neuron with the highest
activation is selected as the winner. The weights of the winning
neuron and its neighbors are adjusted to improve the classification
accuracy. The LVQ network can handle noisy or incomplete input
patterns and can adapt to new patterns without forgetting previously
learned ones.

Overall, both SOM and LVQ networks are powerful tools for feature
mapping, clustering, visualization, and classification tasks in a wide range
of applications, including image recognition, speech recognition, and data
analysis.
A Self-Organizing Map (SOM) is a type of unsupervised artificial neural
network that is used for clustering, visualization, and feature extraction. It is
also known as a Kohonen map, after its inventor, Teuvo Kohonen.

The SOM consists of a 2D or 3D grid of neurons, where each neuron is

connected to the input space. During training, the weights of the neurons
are adjusted to match the input patterns. The neurons that are close to
each other in the grid represent similar patterns, which allows the SOM to
visualize the high-dimensional input space in a low-dimensional grid.

The training process in a SOM consists of the following steps:

1.​ Initialization: The weights of the neurons are initialized randomly.

2.​ Input presentation: An input pattern is presented to the network.
3.​ Neuron activation: Each neuron in the grid computes its activation
level, which is based on the similarity between its weights and the
input pattern.
4.​ Neuron competition: The neuron with the highest activation level is
selected as the winner, and its weights are adjusted to match the
input pattern.
5.​ Neighborhood adaptation: The weights of the winning neuron and its
neighbors in the grid are adjusted to reflect the similarity between
their weights and the input pattern.
6.​ Repeat: Steps 2-5 are repeated for each input pattern.

After training, the SOM can be used for a variety of tasks, including
clustering and visualization of high-dimensional data. The neurons in the
grid represent similar patterns, so the SOM can be used to group similar
patterns together. In addition, the SOM can be visualized as a
low-dimensional map, which allows users to explore and understand the
structure of the high-dimensional input space.

Overall, the SOM is a powerful tool for unsupervised learning tasks, and
has been used successfully in a variety of applications, including image
recognition, speech recognition, and data analysis.

The algorithm for training a Self-Organizing Map (SOM) is as follows:

1.​ Initialize the weights: The weights of each neuron in the grid are
initialized randomly. Each neuron has a weight vector with the same
dimensionality as the input patterns.
2.​ Select an input pattern: A training pattern is randomly selected from
the input data.
3.​ Compute the best matching unit (BMU): The BMU is the neuron
whose weight vector is closest to the input pattern. The distance
between the input pattern and the weight vector of each neuron is
calculated using a distance metric such as Euclidean distance.
4.​ Update the weights of the BMU and its neighbors: The weights of the
BMU and its neighbors are updated based on the difference between
the input pattern and their current weights. The amount of update is
proportional to the distance between the neurons and the BMU, with
closer neurons receiving larger updates.
5.​ Decrease the learning rate: After each iteration, the learning rate is
decreased to reduce the amount of weight update. This allows the
network to converge to a stable solution.
6.​ Repeat: Steps 2-5 are repeated for each training pattern in the input
data.

The output of the SOM is a 2D or 3D grid of neurons, where each neuron

represents a region of the input space. Neurons that are close to each
other in the grid have similar weight vectors, which allows the SOM to
visualize the high-dimensional input space in a low-dimensional grid.
After training, the SOM can be used for various tasks, such as clustering
and visualization of high-dimensional data. The neurons in the grid
represent similar patterns, so the SOM can be used to group similar
patterns together. In addition, the SOM can be visualized as a
low-dimensional map, which allows users to explore and understand the
structure of the high-dimensional input space.

Here are some of the key properties of feature maps:

1.​ Topological structure: Feature maps have a topological structure,

meaning that the relationships between the neurons in the map
reflect the relationships between the features in the input data. This
topological structure is created during the learning process and is
preserved in the trained network.
2.​ Unsupervised learning: Feature maps are often trained using
unsupervised learning algorithms, meaning that the network does not
require labeled data to learn. Instead, the network learns to extract
relevant features from the input data and map them to the feature
map.
3.​ Dimensionality reduction: Feature maps are often used for
dimensionality reduction, where high-dimensional input data is
mapped to a lower-dimensional feature space. This can help to
reduce the computational complexity of subsequent processing steps
and can aid in visualizing and interpreting the data.
4.​ Clustering: Feature maps can be used for clustering, where similar
input patterns are mapped to neighboring neurons in the feature map.
This allows the network to group similar patterns together and can aid
in discovering underlying patterns in the data.
5.​ Generalization: Feature maps can generalize to new, unseen input
patterns by mapping them to the appropriate region of the feature
map. This allows the network to recognize and classify new patterns
based on their similarity to previously learned patterns.
Overall, feature maps are a powerful tool for unsupervised learning and
dimensionality reduction tasks. They can aid in discovering underlying
patterns in the data, clustering similar patterns together, and generalizing to
new, unseen patterns.

Computer simulations refer to the use of computer programs to model

and analyze complex systems or processes. These simulations allow
researchers and engineers to study and test various hypotheses and
scenarios in a controlled, virtual environment.

Computer simulations can be used in a wide range of fields, including:

1.​ Science: Computer simulations can be used to model complex

physical, chemical, and biological systems, allowing researchers to
better understand the underlying processes and test hypotheses.
2.​ Engineering: Computer simulations are used to design and optimize
systems and processes, such as aircraft, buildings, and
manufacturing processes.
3.​ Social sciences: Computer simulations are used to model and study
social phenomena, such as the spread of diseases, economic
systems, and voting behavior.
4.​ Gaming: Computer simulations are used to create realistic and
immersive gaming environments, allowing players to experience
complex scenarios and challenges.

Some of the benefits of using computer simulations include:

1.​ Cost-effective: Computer simulations can be less expensive than

physical experiments or prototypes, allowing researchers and
engineers to test and optimize systems and processes before
investing in physical models.
2.​ Controlled environment: Computer simulations allow researchers to
control variables and test various scenarios in a controlled
environment, reducing the risk of error or bias.
 3.​ Speed and efficiency: Computer simulations can be run quickly and
efficiently, allowing researchers to test and analyze large amounts of
data in a short amount of time.
4.​ Flexibility: Computer simulations can be easily modified and adapted
to test new hypotheses or scenarios, allowing researchers to explore
a wide range of possibilities.

Overall, computer simulations are a powerful tool for modeling and

analyzing complex systems and processes in a variety of fields, offering
numerous benefits over traditional physical experimentation.

Learning Vector Quantization (LVQ) is a type of artificial neural network

used for supervised learning, classification, and pattern recognition. LVQ is
a modification of the Self-Organizing Map (SOM) algorithm and works by
dividing input patterns into different classes or clusters.

LVQ consists of a two-layer neural network, where the input layer is

connected to a layer of neurons called the output layer. The output layer
neurons are arranged in a grid-like structure, and each neuron represents a
different class or cluster. During the training process, the weights of the
output layer neurons are adjusted to minimize the error between the
predicted class and the actual class of the input pattern.

The learning process in LVQ is supervised, meaning that each input pattern
is labeled with its correct class or cluster. The network is trained using a set
of labeled input patterns, and the weights of the output layer neurons are
updated based on the error between the predicted class and the actual
class. This process is repeated for multiple iterations until the network
achieves a satisfactory level of accuracy.

Some of the advantages of LVQ include:

1.​ Efficient learning: LVQ can be trained with a relatively small number
of labeled data compared to other supervised learning algorithms.
 2.​ Class separability: LVQ can handle input patterns with overlapping
classes by defining decision boundaries between the classes.
3.​ Robustness: LVQ is less susceptible to noisy input patterns
compared to other classification algorithms.
4.​ Interpretability: LVQ is transparent and easy to interpret, making it
useful for decision-making tasks.

Overall, Learning Vector Quantization is a powerful tool for supervised

learning and classification tasks, offering advantages such as efficient
learning, robustness, and interpretability.

Adaptive pattern classification is a type of machine learning algorithm

that uses statistical models to identify and classify patterns in data. These
algorithms adapt to changing input data and continuously improve their
performance over time.

The basic process of adaptive pattern classification involves the following

steps:

1.​ Data preprocessing: The input data is cleaned, normalized, and

transformed into a format suitable for analysis.
2.​ Feature extraction: Relevant features are extracted from the input
data to reduce its dimensionality and capture the most important
information.
3.​ Model training: A statistical model is trained using a set of labeled
data, where each input pattern is associated with a known class or
category.
4.​ Model testing: The trained model is evaluated using a separate set of
test data to assess its performance and generalization ability.
5.​ Model adaptation: The model is updated or retrained using new input
data to adapt to changes in the underlying patterns.

Some examples of adaptive pattern classification algorithms include:

 1.​ Adaptive Boosting (AdaBoost): A type of ensemble learning algorithm
that combines multiple weak classifiers to create a strong classifier
that can adapt to changes in the input data.
2.​ Online Learning: A type of machine learning algorithm that
continuously updates the model parameters as new data arrives,
allowing the model to adapt to changes in the underlying patterns.
3.​ Bayesian Learning: A type of statistical inference that uses Bayes'
theorem to update the model's probabilities as new data arrives,
allowing the model to adapt to changing input patterns.

Adaptive pattern classification algorithms have a wide range of

applications, including image recognition, speech recognition, and natural
language processing. These algorithms are particularly useful in dynamic
environments where the underlying patterns are constantly changing and
traditional classification algorithms may not be effective.