Module 4 Trends in Computing

Module IV

Soft computing: soft computing vs. hard computing: Introduction to Neural Network-
Intelligence, Neurons, Artificial Neural network, Application scope of neural network,
Brain vs computer. Problem areas, Training of artificial networks- supervised and
Unsupervised; From ordinary set to fuzzy sets- Basics of Fuzzy set logic Theory,
Foundation of fuzzy logic- fuzzy sets; Application.

SOFT COMPUTING

Soft computing is the reverse of hard (conventional) computing. It refers to a group of

computational techniques that are based on artificial intelligence (AI). It is a branch of
computer science by which we can create intelligent machines which can behave like a
human, think like humans, and able to make decisions.] and natural selection. It provides
cost-effective solutions to the complex real-life problems for which hard computing solution
does not exist. It can map a human mind and the human mind is a role model for soft
computing.

Zadeh coined the term of soft computing in 1992. The objective of soft computing is to
provide precise approximation and quick solutions for complex real-life problems.

Characteristics of Soft computing

• Soft computing provides an approximate but precise solution for real-life problems.
• The algorithms of soft computing are adaptive, so the current process is not affected
by any kind of change in the environment.

1
Module 4 Trends in Computing

• The concept of soft computing is based on learning from experimental data. It means
that soft computing does not require any mathematical model to solve the problem.
• Soft computing helps users to solve real-world problems by providing approximate
results that conventional and analytical models cannot solve.
• It is based on Fuzzy logic, genetic algorithms, machine learning, ANN, and expert
systems.

Example

Soft computing deals with the approximation model.

Let's consider a problem. Suppose string1 = "xyz" and string2 = "xyw"

1. Problem 1
2. Are string1 and string2 same?
3. Solution
4. No, the solution is simply No. It does not require any algorithm to analyze this.

Let's modify the problem a bit.

1. Problem 2
2. How much string1 and string2 are same?
3. Solution
4. Through conventional programming, either the answer is Yes or No. But these strings
might be 80% similar according to soft computing.

Applications of Soft Computing

• It is widely used in gaming products like Poker and Checker.

• In kitchen appliances, such as Microwave and Rice cooker.
• In most used home appliances - Washing Machine, Heater, Refrigerator, and AC as
well.
• Apart from all these usages, it is also used in Robotics work (Emotional per Robot
form).
• Image processing and Data compression are also popular applications of soft
computing.
• Used for handwriting recognition.

2
Module 4 Trends in Computing

Elements of soft computing

Soft computing is viewed as a foundation component for an emerging field of conceptual

intelligence. Fuzzy Logic (FL), Machine Learning (ML), Neural Network (NN), Probabilistic
Reasoning (PR), and Evolutionary Computation (EC) are the supplements of soft computing.

Also, these are techniques used by soft computing to resolve any complex problem.

Any problems can be resolved effectively using these components. Following are three types
of techniques used by soft computing:

• Fuzzy Logic
• Artificial Neural Network (ANN)
• Genetic Algorithms

Fuzzy Logic (FL)

Fuzzy logic is basically designed to achieve the best possible solution to complex problems
from all the available information and input data. Fuzzy logics are considered as the best
solution finders. It makes it easy to obtain an array of precise conclusions.

3
Module 4 Trends in Computing

Neural Network (ANN)

Neural networks were developed in the 1950s, which helped soft computing to solve real-
world problems, which a computer cannot do itself. We all know that a human brain can
easily describe real-world conditions, but a computer cannot.

An artificial neural network (ANN) emulates a network of neurons that makes a human brain
(means a machine that can think like a human mind). Thereby the computer or a machine can
learn things so that they can take decisions like the human brain. Artificial Neural Networks
(ANN) are mutually connected with brain cells and created using regular computing
programming. It is like as the human neural system.

Genetic Algorithms (GA)

A genetic algorithm is a subset of a large branch of computation. Genetic algorithm is almost

based on nature and takes all inspirations from it. There is no genetic algorithm that is based
on search-based algorithms. GA find its roots in natural selection and the concept of genetics.

SOFT COMPUTING VS HARD COMPUTING

4
Module 4 Trends in Computing

Parameters Soft Computing Hard Computing

Computation Takes less computation time. Takes more computation time.

time

Dependency It depends on approximation and It is mainly based on binary logic and

dispositional. numerical systems.

Computation Parallel computation Sequential computation

type

Result/Output Approximate result Exact and precise result

Example Neural Networks, such as Any numerical problem or traditional

Madaline, Adaline, Art Networks. methods of solving using personal
computers.

NEURAL NETWORKS

Neural networks or also known as Artificial Neural Networks (ANN) are networks that
utilize complex mathematical models for information processing. They are based on the
model of the functioning of neurons and synapses in the brain of human beings. Similar to the
human brain, a neural network connects simple nodes, also known as neurons or units. And a
collection of such nodes forms a network of nodes, hence the name "neural network."

Similar to the human brain, in a neural network, an array of algorithms is used to identify and
recognize relationships in data sets. Neural networks are designed to adapt to dynamic input
scenarios; with the result, the best possible outcomes are provided by the network without
having to rework the design of the output for further processing.

From a utilization standpoint, Neural Networks are being used on a variety of technologies
and applications such as video games, computer vision, speech recognition, social network
filtering, playing board, machine translation, and medical diagnosis. Surprisingly, neural
networks are being used for activities that are traditional and creative, like painting and art.

5
Module 4 Trends in Computing

BASIC COMPONENTS OF NEURAL NETWORK

At this point, it is important to know and understand what constitutes a neural network and its
components.

1. NEURONS

A neuron is the basic unit of a neural network. It receives data input and then combines the
input with its internal activation state as well as with an optional threshold activation
function. Then by using an output function, it produces the output.

The initial inputs are data from various external sources, such as voice files, images, and
documents. The final outputs could be recognizing a voice input or an object in an image or
text. The significance of the activation function is that it provides a seamless and
differentiable transition as input values change dynamically. So, a small change or shift in
input data produces a small change in the output.

2. CONNECTIONS AND WEIGHTS

Neural Network consists of connections and weights, where each connection t hrows an
output of one neuron, which becomes an input to another neuron in the network. A weight is
assigned to each connection, and it represents its relative importance on the neural network.
Any given neuron can have many to many relationships with multiple inputs and output
connections.

3. ORGANIZATION

It is an organization of the neurons into multiple layers. This aptly applies in the areas of
Deep Learning. It is designed in such a way that Neurons are connected to the immediate
neighbouring layers of neurons. This means is that neurons of one layer connect only to
neurons of the immediately preceding and immediately following layers. The input layer is
one that receives external data, and the layer that delivers the final result is the output layer.
There can be more hidden layers or no layers in between them. In some scenarios, un-layered
and single- layered networks are also possible, and multiple connection patterns are possible
between two layers. It is so versatile and maximal that a fully connected neuron set is
possible where every neuron in one layer is connected to every neuron in the next layer. So
based on the way to make the connections, there are many types of Neural networks.

6
Module 4 Trends in Computing

4. LEARNING

Learning is the process by which the network adapts itself to handle a task better by factoring
in sample data observations. Learning involves calibrating the weights and optional threshold
values of the network to obtain more accurate results. This is performed and achieved by
minimizing the errors that are observed. The process of learning reaches an optimum when
the error rate is reduced. It must be noted that even after the learning process is complete, the
error rate in most scenarios does not reach "0". If the error rate is too high even after the
learning process, the network requires to be redesigned.

5. LEARNING RATE

The learning rate for each observation defines the size of the corrective steps that the model
takes to adjust for errors. A high learning rate could reduce the training time, but the output
can be less accurate. So, in practice, a lower learning rate is preferred, which takes longer, but
has the potential to deliver greater accuracy. Optimization techniques are primarily targeted
at enhancing the speed of error minimization. Other learning improvement techniques mainly
attempt to achieve higher reliability of scores.

7
Module 4 Trends in Computing

Learning Techniques in Neural Networks

1. Supervised Learning: In this learning, the training data is input to the network, and the
desired output is known weights are adjusted until production yields desired value.
2. Unsupervised Learning: Use the input data to train the network whose output is
known. The network classifies the input data and adjusts the weight by feature
extraction in input data.
3. Reinforcement Learning: Here, the output value is unknown, but the network provides
feedback on whether the output is right or wrong. It is Semi-Supervised Learning.
4. Offline Learning: The weight vector adjustment and threshold adjustment are made
only after the training set is shown to the network. It is also called Batch Learning.
5. Online Learning: The adjustment of the weight and threshold is made after presenting
each training sample to the network.

APPLICATIONS OF ARTIFICIAL NEURAL NETWORKS

1. Social Media

Artificial Neural Networks are used heavily in social media. For example, let’s take the
‘People you may know’ feature on Facebook that suggests you people that you might know
in real life so that you can send them friend requests. Well, this magical effect is achieved by
using Artificial Neural Networks that analyse your profile, your interests, your current
friends, and also their friends and various other factors to calculate the people you might
potentially know. Another common application of Machine Learning in social media is facial
recognition. This is done by finding around 100 reference points on the person’s face and
then matching them with those already available in the database using convolutional neural
networks.

2. Marketing and Sales

When you log onto E-commerce sites like Amazon and Flipkart, they will recommend your
products to buy based on your previous browsing history. Similarly, suppose you love Pasta,
then Zomato, Swiggy, etc. will show you restaurant recommendations based on your tastes
and previous order history.

8
Module 4 Trends in Computing

This is true across all new-age marketing segments like Book sites, Movie services,
Hospitality sites, etc. and it is done by implementing personalized marketing. This uses
Artificial Neural Networks to identify the customer likes, dislikes, previous shopping history,
etc. and then tailor the marketing campaigns accordingly.

3. Healthcare

Artificial Neural Networks are used in Oncology to train algorithms that can identify
cancerous tissue at the microscopic level at the same accuracy as trained physicians. Various
rare diseases may manifest in physical characteristics and can be identified in their premature
stages by using Facial Analysis on the patient photos. So the full-scale implementation of
Artificial Neural Networks in the healthcare environment can only enhance the diagnostic
abilities of medical experts and ultimately lead to the overall improvement in the quality of
medical care all over the world.

4. Personal Assistants

You all have heard of Siri, Alexa, Cortana, etc. and also heard them based on the phones you
have!!! These are personal assistants and an example of speech recognition that uses Natural
Language Processing to interact with the users and formulate a response accordingly. Natural
Language Processing uses artificial neural networks that are made to handle many tasks of
these personal assistants such as managing the language syntax, semantics, correct speech,
the conversation that is going on, etc.

THE BRAIN VS. THE COMPUTER

Similarity Difference
The brain uses chemicals to transmit information; the
computer uses electricity. Even though electrical signals
Both use electrical signals
to send messages. travel at high speeds in the nervous system, they travel even
faster through the wires in a computer.
A computer uses switches that are either on or off ("binary").
In a way, neurons in the brain are either on or off by either
firing an action potential or not firing an action potential.
Both
However, neurons are more than just on or off because the
transmit information.
"excitability" of a neuron is always changing.

9
Module 4 Trends in Computing

This is because a neuron is constantly getting information

from other cells through synaptic contacts. Information
traveling across a synapse does NOT always result in an
action potential. Rather, this information alters the chance
that an action potential will be produced by raising or
lowering the threshold of the neuron.

Computer memory grows by adding computer chips.

Memories in the brain grow by stronger synaptic
connections.
Both have a
memorythat can grow.
It is much easier and faster for the brain to learn new things.
Yet, the computer can do many complex tasks at the same
time ("multitasking") that are difficult for the brain. For
example, try counting backwards and multiplying 2 numbers
Both can at the same time. However, the brain also does some
adapt andlearn. multitasking using the autonomic nervous system. For
example, the brain controls breathing, heart rate and blood
pressure at the same time it performs a mental task.
The human brain has weighed in at about 3 pounds for about
the last 100,000 years. Computers have evolved much faster
than the human brain. Computers have been around for only
a few decades, yet rapid technological advancements have
Both have evolved made computers faster, smaller and more powerful.
over time.
The brain needs nutrients like oxygen and sugar for power;
the computer needs electricity to keep working.

Both need energy.

It is easier to fix a computer - just get new parts. There are no
new or used parts for the brain. However, some work is being
done with transplantation of nerve cells for certain
neurological disorders such as Parkinson's disease. Both a
Both can be damaged. computer and a brain can get "sick" - a computer can get a
"virus" and there are many diseases that affect the brain. The
brain has "built-in back up systems" in some cases.
Scientists understand how computers work. There are
thousands of neuroscientists studying the brain. Nevertheless,
there is still much more to learn about the brain. "There is
more we do NOT know about the brain, than what we do
know about the brain"
Both brains
and computers are studied
by scientists

10
Module 4 Trends in Computing

TRAINING AN ARTIFICIAL NEURAL NETWORK

Once a network has been structured for a particular application, that network is ready to be
trained. To start this process the initial weights are chosen randomly. Then, the training, or
learning, begins. There are two approaches to training -supervised and unsupervised.
Supervised training involves a mechanism of providing the network with the desired output
either by manually "grading" the network's performance or by providing the desired outputs
with the inputs. Unsupervised training is where the network has to make sense of the inputs
without outside help.

The vast bulk of networks utilize supervised training. Unsupervised training is used to
perform some initial characterization on inputs. However, in the full-blown sense of being
truly self-learning, it is still just a shining promise that is not fully understood, does not
completely work, and thus is relegated to the lab.

SUPERVISED TRAINING

In supervised training, both the inputs and the outputs are provided. The network then
processes the inputs and compares its resulting outputs against the desired outputs. Errors are
then propagated back through the system, causing the system to adjust the weights which
control the network. This process occurs over and over as the weights are continually
tweaked. The set of data which enables the training is called the "training set." During the
training of a network the same set of data is processed many times as the connection weights
are ever refined.

The current commercial network development packages provide tools monitor how well an
artificial neural network is converging on the ability to predict the right answer. These tools
allow the training process to go on for days, stopping only when the system reaches some
statistically desired point, or accuracy. However, some networks never learn. This could be
because the input data does not contain the specific information from which the desired
output is derived. Networks also don't converge if there is not enough data to enable complete
learning.

11
Module 4 Trends in Computing

Ideally, there should be enough data so that part of the data can be held back as a test. Many
layered networks with multiple nodes are capable of memorizing data. To monitor the
network to determine if the system is simply memorizing its data in some nonsignificant way,
supervised training needs to hold back a set of data to be used to test the system after it has
undergone its training.

If a network simply can't solve the problem, the designer then has to review the input and
outputs, the number of layers, the number of elements per layer, the connections between the
layers, the summation, transfer, and training functions, and even the initial weights
themselves. Those changes required to create a successful network constitute a process
wherein the "art" of neural networking occurs. Another part of the designer's creativity
governs the rules of training. There are many laws (algorithms) used to implement the
adaptive feedback required to adjust the weights during training.

The most common technique is backward-error propagation, more commonly known as back-
propagation. These various learning techniques are explored in greater depth later in this
report. Yet, training is not just a technique. It involves a "feel," and conscious analysis, to
ensure that the network is not overtrained. Initially, an artificial neural network configures
itself with the general statistical trends of the data. Later, it continues to "learn" about other
aspects of the data which may be spurious from a general viewpoint.

When finally, the system has been correctly trained, and no further learning is needed, the
weights can, if desired, be "frozen." In some systems this finalized network is then turned
into hardware so that it can be fast. Other systems don't lock themselves in but continue to
learn while in production use.

UNSUPERVISED, OR ADAPTIVE TRAINING

The other type of training is called unsupervised training. In unsupervised training, the
network is provided with inputs but not with desired outputs. The system itself must then
decide what features it will use to group the input data. This is often referred to as self-
organization or adaption. At the present time, unsupervised learning is not well understood.
This adaption to the environment is the promise which would enable science fiction types of
robots to continually learn on their own as they encounter new situations and new
environments. Life is filled with situations where exact training sets do not exist.

12
Module 4 Trends in Computing

Some of these situations involve military action where new combat techniques and new
weapons might be encountered. Because of this unexpected aspect to life and the human
desire to be prepared, there continues to be research into, and hope for, this field. Yet, at the
present time, the vast bulk of neural network work is in systems with supervised learning.
Supervised learning is achieving results.

One of the leading researchers into unsupervised learning is Tuevo Kohonen, an electrical
engineer at the Helsinki University of Technology. He has developed a self-organizing
network, sometimes called an auto-associator, that learns without the benefit of knowing the
right answer. It is an unusual looking network in that it contains one single layer with many
connections. The weights for those connections have to be initialized and the inputs have to
be normalized. The neurons are set up to compete in a winner-take-all fashion. Kohonen
continues his research into networks that are structured differently than standard,
feedforward, back-propagation approaches.

Kohonen's work deals with the grouping of neurons into fields. Neurons within a field are
"topologically ordered." Topology is a branch of mathematics that studies how to map from
one space to another without changing the geometric configuration. The three-dimensional
groupings often found in mammalian brains are an example of topological ordering.

13