Basics of Soft Computing08 - Chapter1 PDF
Basics of Soft Computing08 - Chapter1 PDF
Basics of Soft Computing08 - Chapter1 PDF
1.0 INTRODUCTION
1
developing Expert Systems which learn from the data and for designing and
cellular automata which derived its inspiration from self reproduction, evolutionary
computational methods, which are largely due to Darwinian Evolution of species.
All the computational techniques such as Neural computation, mimicking the
functioning of the brain and swarm intelligence methods, motivated by the group
behavior of organisms have drawn their inspiration only from Nature and Soft
Computing Techniques is one of the successful techniques for developing models
for Expert system. ( Gupta & Kulkarni 2013 )
Artificial Intelligence (AI) is the field that studies how machine can be
made to act intelligently. AI research is progressing rapidly. The potential benefits
of AI are very high. Undoubtedly everything that civilisation has to offer is a
product of human intelligence by the tools that AI may provide. (Hawking 2014).
Success in creating AI would be the biggest event in human history. AI is to
discover how to program a computer to perform the remarkable functions that make
up human intelligence for which deeper understanding of human intelligence and
human mind is needed. ( Nath 2009)
2
1.1.1 Machine Learning:
3
1.1.2 Soft Computing
Computers are used in almost all disciplines that include science, technology
and medical science etc. Computing techniques are used to find exact solutions of
scientific problems. The solutions are attempted on the basis of two valued logic and
classical mathematics. However, all real life problems cannot be handled by
conventional methods. Zadeh, who is known as the father of Fuzzy Logic has
mentioned that humans are able to resolve tasks of high complexity without
measurement or computation. Hence the need arose for developing systems that
work on Artificial Intelligence
During the year 1955 John McCarthy organised Summer Research Project on
Artificial Intelligence (AI) to initiate research programmes in AI with a proposal that
AI research will proceed on the basis of the conjuncture that every aspect of learning
or any other feature of intelligence can in principle be so precisely described that a
machine can be made to stimulate it [McCarthy et al., 1955]. AI became a field of
research to build models and systems that act intelligently without human
intervention. In mid 1980s Zadeh focused on building systems or making computers
think like humans [Zadeh,1984]. For this purpose, the Machines’ ability to compute
with numbers which is named as hard computing has to be supplemented by an
additional ability more similar to human thinking which is named as soft computing
[Seising, et al., 2013].
4
The idea of soft computing was evolved in 1981 when Zadeh published his
paper on soft data analysis (Zadeh ,1997). Soft computing is a term coined by
Zadeh, combing a collection of computing techniques, spanning many fields that fall
under various categories in computational intelligence. Soft computing has three
main branches, Fuzzy system, Evolutionary computation, Artificial Neural
computing which sub-sums Machine Learning (ML), Probabilistic Reasoning (PR),
Belief Network, Chaos Theory, parts of Learning Theory and Wisdom Expert
System(WES).
5
1. Impossibility of extracting adequate set of rules from professionals in a given
application area (which has been called the knowledge acquisition
bottleneck, a mournful phrase which has been repeated in literature). As a
result there is a considerable degradation, in the performance of the
modelling of disease by the rules and in addition it is difficult to construct
automatic system to provide classification or pattern recognition tools to help
specialists to make a decision.
6
1.2.1 Fuzzy Logic
Zadeh continued to think about the basic issues in system analysis, especially
the issue of un-sharpness of class boundaries. The beginning of the genesis of the
Fuzzy set theory emerged out of these thoughts,(Zadeh,1965). He introduced this
concept of Fuzzy set, that is a class in which there may be a continuous infinity of
7
membership values with the grade of membership value of an object in a fuzzy set A
represented by a number ݂ ሺݔሻ in the interval [0,1] (Zadeh, 1965,Seising,2007). In
1968 he presented a paper ‘Fuzzy Algorithm’ to Information and Control in which
he has generalised the conception of an algorithm with the concept of Fuzzification
(Zadeh,1968). This became an inspiration for the common people and they started
practising it in their daily life for activities such as preparing recipes for cooking,
following prescriptions to illness, for getting guidance to park the car which be
regarded as a very crude form of Fuzzy Algorithms.
Parallel to the study of the applications of fuzzy logic, Professors Takagi and
Sugeno developed the first approach to construct fuzzy rules (Fuzzy Rules), from
training data or training The applications of fuzzy logic in everyday life since then
grow rapidly. The Fuzzy Rules, or rules of a fuzzy system, define a set of
overlapping patches that relate a full range of inputs to a full range of outputs. In that
sense, the fuzzy system approximates some mathematical function or equation of
cause and effect.
8
A very important result says that fuzzy systems can approximate any
continuous math function. Bart Kosko (1993) proved this uniform convergence
theorem by showing that enough small fuzzy patches can sufficiently cover the
graph of any function or input/output relation. The theorem also shows that we can
pick in advance the maximum error of the approximation and be sure there exists a
finite number of fuzzy rules that achieve it (Garrido, 2012).
9
a simple proposition.”. Because electric on-off switches can be interconnected such
that each Boolean statement can be realized, McCulloch and Pitts now ‘realized’ the
entire logical calculus of propositions by ‘neuron nets’. They arrived at the
following assumptions:
ii) a certain fixed number of synapses must be excited within the period of
latent addition in order to excite a neuron at any time, and this number is
independent of previous activity and position on the neuron;
iii) the only significant delay within the nervous system is synaptic delay;
iv) the activity of any inhibitory synapse absolutely prevents excitation of the
neuron at that time;
v) the structure of the net does not change with time (McCulloch, 1943);
10
Rosenblatt and Charles Wightman at Cornell University developed a first machine
for pattern 741 classification. Rosenblatt described this early artificial neural
network, called Mark I Perceptron, in an essay for the Psychological Review
(Rosenblatt, 1958). The euphoria came to an abrupt halt in 1969, however, when
Minsky and Seymour Papert completed their study of perceptron networks and
published their findings in a book (Minsky et al., 1987).
In the 1970s, Grossberg (1976), Von der Malsburg (1973), and Fukushima
(1975) conducted pioneering work on competitive learning and self organization,
based on the connection patterns found in the visual cortex.. Fukushima (1988)
proposed his neocognitron models [Fukushima, 1988], under the competitive
learning paradigm. The neocognitron is a neural network specially designed for
visual or character pattern recognition. Kohonen (1990). proposed his self-
organizing maps (SOM) The SOM algorithm adaptively transforms incoming signal
patterns of arbitrary dimensions into one- or two-dimensional discrete maps in a
topologically ordered fashion. A Kohonen network is a structure of interconnected
processing units that compete for the signal.
11
(Ackley et al., 1985). The Boltzmann learning is based on a method called
Simulated Annealing (SA) (Kirkpatrick et al., 1983). These works revived the
interest in neural networks. Kosko extended the ideas of Grossberg and Hopfield and
proposed the adaptive bidirectional associative memory (BAM) (Kosko,1987). The
Hamming network was proposed by Lippman in the mid-1980s (Lippman ,1987). It
is composed of a similarity feed forward subnet with an n-node input layer and an
m-neuron memory layer and a winner-take-all (WTA) subnet with a fully connected
m-neuron topology. The network is the most straightforward associative memory.
The Hamming network calculates the Hamming distance between the input pattern
and each memory pattern, and selects the memory with the smallest Hamming
distance, which is declared as the winner.
The landmark of the field is the multilayer perceptron (MLP) model trained
with the back propagation (BP) learning algorithm published in 1986 by Rumelhart
et al. [1986]. Hopfield (1982) published the paper ‘Neural networks and physical
systems with emergent collective computational abilities’ on his invention of an
associative neural network .
Broomhead and Lowe (1988) proposed the radial basis function network
(RBFN) model. The RBFN has equivalent capabilities as the MLP model, but with a
much faster training speed. The cellular neural network (CNN) model was proposed
Chua and Yang (1988), has a unique network architecture. CNNs are especially
useful in image and video processing, and are most suitable for very large scale
integrated (VLSI) implementation.
Neural Networks have certainly come a long way from early days of
McCulloh and Pitts and they will continue to grow in theory, design and application.
12
1.2.3 Evolutionary Computation
The origin of evolutionary computation started in the year 1950 through the
contributions of researchers Bremernam (1962), Friedberg(1958,1959), Box(1957)
and others but it gained momentum only during the year 1970,due to the
fundamental work of Holland (1962), Rechenberg(1965), Schwefel(1968) and Fogel
(1962). In 1973, Rechenberg’s work expanded the idea of evolutionary computing
approach to deal with numerical optimization (Rechenberg, 1973). In 1975,
Schwefel also contributed to application of evolutionary strategies to deal with
numerical/parametric optimization and also, formalized it as it is known nowadays
(Schwefel ,1975). During 1980’s due to the advancement in the field of computers,
the application of evolutionary algorithms to solve difficult real world problems has
started to receive significant attention. The researchers in various disciplines of
evolutionary computation remained isolated from each other until early 1990’s
(Belew et al., 1991). In mid 1960’s itself the bases for the 3 main forms of
Evolutionary Algorithms namely, Evolutionary Programming, Genetic Algorithms
and Evolutionary Strategies were identified. The roots of Evolutionary Programming
(EP) were laid by Lawerence Foyal et al (1966) and those of Genetic Algorithms
(GA) were developed at the University of Michigan by Holland (1967). On the other
side of the Atlantic Ocean, Evolutionary Strategies (ES’s) were joint development of
group of three students, Bienert, Rechenberge and Schewefel in Berlin,1965 (Jong
1992). Holland showed how to use Genetic search algorithms to solve real world
problems in his book Adaptation in Natural and Artificial system which was
published in 1975 (Holland ,1975). John Koza’s book Genetic Programming: On the
programming of computers by means of natural selection was published in the year
1992 (Koza,1992). However in 1990, there was an organized effort to provide a
forum for interaction among the various EP research Committees and an
International Workshops on Parallel problem solving from Nature at Dortmend
(Jong 1992). From then on the interaction and co-operation among EA researchers
around the world continued to grow (Jong 1992). The need for an organized EC
handbook was evident due to the dramatic growth of interest. The growth of the field
is reflected by many EC events and related activities each year and also by the
increasing number of books and articles about EC (Jong,1992).
13
1.3 AIM AND OBJECTIVES OF THE THESIS
The aim of the thesis is to develop new algorithms based on Soft Computing
techniques for disease classification and prediction. The objective of the thesis is
a. to collect and unify the recent advances in Soft Computing techniques with
special reference to disease classification.
The Tuberculosis and HIV disease databases are obtained from clinical trial
database at NIRT, ICMR, Chennai. Some of the real and synthetic databases are
obtained from the UCI repository of the University of Irvine. The Lung Cancer
Dataset and Prostate Cancer data set are obtained from http://datam.i2r.a-
star.edu.sg/datasets/krbd/
14
Chapter - 2 – discusses the fuzzy logic concepts, fuzzy set theory and fuzzy
relations and also artificial neural network and its’ various architecture. This chapter
also explains the technique of evolutionary computing, particularly the genetic
algorithm and its applications to biomedical database.
15
gene expression measurements. They are being used to gather information from
tissue and cell samples which are used to diagnose the disease. Futschik et al
(2003) have classified cancer tissue using fuzzy logic rules which have been
extracted from the trained net work to infer knowledge about classification process.
Güler & Übeyli (2004) have developed an ANFIS model for detection of electro
cardiograph graphic changes in patients with partial epilepsy. A neuro fuzzy model
for diagnosis of psychosomatic disorders is proposed by Aruna et al . A neural
fuzzy decision support system for paediatric ALL cancer subtype identification
using gene expression data was the work developed byTung, & Quek (2005). The
model of feature selection based on mutual information criteria of max-dependency,
max-relevance and minimum-redundency was the model developed by Peng et al
(2005). Temurtas,et al (2009) have compared the performance of the multilayer
neural network structure trained by Levenberg-Marquardt algorithm as a classifier
for identifing diabetes with few other performances. Anitha, et al. (2009). have
proposed a hybrid neural network for identifying abnormal retinal image
classification. Avci & Turkoglu (2009) have used principle component analysis and
ANFIS to diagnose heart valve disease. Ince, et al (2010)'s work is on evaluation of
global and local training techniques over feed-forward neural network architecture
spaces for computer-aided medical diagnosis. Karthik, et all (2011 have made an
attempt for diagnosis of liver disease and its types using Rough Set. Karegowda et
al (2011) have propsed that an application of genetic algorithm optimised neural
network connecion weights for medical diagnosis of PIMA Indians. Uzoka, et al
(2011) have used the. Clinical decision support system (DSS) in the diagnosis of
malaria: Petković et al (2013) had studied on how Automatic Nervous System
(ANS) branches affect the most relevant heart variability, by using ANFIS network..
Castanho et al. (2013). have developed Fuzzy expert system for predicting
pathological stage of prostate cancer. Shilaskar & Ghatol (2013) have promoted a
system for medical diagnosis of cardiovascular diseases by feature selection.
Samuel et al. (2013) have developed a web based decision support system driven by
fuzzy logic for the diagnosis of typhoid fever. Elshazly et al. (2013) have developed
a Hybrid system for lymphatic diseases diagnosis. Seera & Lim (2014) have
developed a hybrid intelligent system for medical data classification.
16