1414 IEEE TRANSACTIONS ON SYSTEMS, MAN, AND CYBERNETICS, VOL. 22, NO.

6, NOVEMBEWDECEMBER 1992

Generating Fuzzy Rules by Learning from Examples

Li-Xin Wang, Fellow, IEEE, and Jerry M. Mendel, Fellow, IEEE

Abstract- A general method is developed to generate fuzzy

rules from numerical data. This new method consists of five
steps: Step 1 divides the input and output spaces of the given
numerical data into fuzzy regions; Step 2 generates fuzzy rules
from the given data; Step 3 assigns a degree of each of the
generated rules for the purpose of resolving conflicts among ( so complicated that no mathematical model exists
the generated rules; Step 4 creates a combined fuzzy rule base or, the mathematical model is severely non-linear )
based on both the generated rules and linguistic rules of human
experts; and, Step 5 determines a mapping from input space
to output space based on the combined fuzzy rule base using a
defuzzifying procedure. The mapping is proved to be capable of
approximating any real continuous function on a compact set to
arbitrary accuracy. Applications to truck backer-upper control
and time series prediction problems are presented. For the truck
control problem, the performance of this new method is compared
with a neural network controller and a pure limited-rule fuzzy
(I states
Human Controller 1 (control
controller; the new method shows the best performance. For ( inputs outputs
the time series prediction problem, results are compared by
using the new method and a neural network predictor for the
Mackey-Glass chaotic time series.

I. INTRODUCTION Task : Design a control system to replace the human controller.

F OR MOST real-world control and signal processing prob-

lems, the information concerning design, evaluation, re-
alization, etc., can be classified into two kinds: numerical
Fig. 1. A practical problem: Design a control system to replace the human
controller.

information obtained from sensor measurements, and, linguis-

tic information obtained from human experts. Most current that no mathematical model exists for it, or, the mathematical
intelligent control and signal processing approaches are heuris- model is strongly nonlinear so that a design method does not
tic in nature, i.e., they combine some standard control or signal exist. The task here is to design a control system to replace
processing methods with expert systems in an ad hoc way for a the human controller (see Fig. 1).
specific problem; simulations are then performed to show that To design such a control system, we first need to see
the new approaches work well for the specific problem. This what information is available. We assume that there is no
kind of approach has two weakpoints: 1) it is quite problem mathematical model, i.e., we consider a model-free design
dependent, i.e., a method may work well for one problem but problem. Since there already is a human controller who
is not suited for another problem; and, 2) there is no common is successfully controlling the system, there are two kinds
framework for modeling and representing different aspects of of information available to us: 1) the experience of the
control or signal processing strategies, which makes theoretical human controller; and, 2) sampled input-output (state-control)
analyses for these approaches very difficult. In this paper, we pairs that are recorded from successful control by the human
propose a general method for combining both numerical and controller. The experience of the human controller is usually
linguistic information into a common framework-a fuzzy rule expressed as some linguistic “IF-THEN” rules that state in
base. what situation(s) which action(s) should be taken. The sampled
Suppose we have the following problem: there is a complex input-output pairs are some numerical data that give the
control system in which a human controller is an essential part; specific values of the inputs and the corresponding successful
the environment facing this human controller is so complicated outputs.
Manuscript received March 3, 1991, revised April 20, 1992. Each of the two kinds of information alone is usually
L. X. Wang was with the Signal and Image Processing Institute, Department incomplete. Although the system is successfully controlled
of Electrical Engineering Systems, University of Southern California, Los by a human controller, some information will be lost when
Angeles, CA 90089-2564 and is now with the Department of Electrical
Engineering and Computer Science, University of California, Berkeley, CA human controllers express their experience by linguistic rules.
94720. Consequently, linguistic rules alone are usually not enough for
J. M. Mendel is with the Signal and Image Processing Institute, Department designing a successful control system. On the other hand, the
of Electrical Engineering Systems, University of Southern California, Los
Angeles, CA 90089-2564. information from sampled input-output data pairs is usually
IEEE Log Number 9202126. also not enough for a successful design, because the past
0018-9472/92$03.00 0 1992 IEEE

Authorized licensed use limited to: INDIAN INSTITUTE OF TECHNOLOGY KANPUR. Downloaded on February 01,2021 at 06:41:21 UTC from IEEE Xplore. Restrictions apply.
 WANG AND MENDEL: GENERATING FUZZY RULES BY LEARNING FROM EXAMPLES 1415

operations usually cannot cover all the situations the control pairs of (l), and use these fuzzy rules to determine a mapping
system will face. If expert linguistic rules and numerical data f : (51,52) + Y.
pairs are the only information we can get for such a control Our approach consists of the following five steps:
system design, the most interesting case for us is when the
combination of these two kinds of information is sufficient for Step 1-Divide the Input and Output Spaces into Fuzzy Regions
a successful design. Assume that the domain intervals of x1,x2 and y are
Fuzzy control is an effective approach to utilizing linguistic
[x, , xt], [x;,x;] and [y- , y+], respectively, where “domain
rules [3], [4], whereas neural control is suited for using interval” of a variable means that most probably this variable
numerical data pairs (i.e., desired input-output pairs) [l], [4]. will lie in this interval (the values of a variable are allowed to
Present fuzzy controllers only use linguistic rules, whereas lie outside its domain interval). Divide each domain interval
present neural controllers only use numerical data pairs. This
leads to the following question: “Is it possible to develop a
+
into 2 N 1 regions ( N can be different for different variables,
and the lengths of these regions can be equal or unequal),
general approach that combines both kinds of information into denoted by SN (Small N ) , . . . , S1 (Small l), CE (Center),
a common framework, and uses both information, simultane-
B1 (Big l ) , . . ,BN (Big N ) . and assign each region a fuzzy
ously and cooperatively, to solve the control design or similar membership function. Fig. 2 shows an example where the
problems?” In this paper, we develop such a general approach. domain interval of x1 is divided into five regions ( N = 2); the
The key ideas of our new approach are to generate fuzzy domain region of x2 is divided into seven regions ( N = 3),
rules from numerical data pairs, collect these fuzzy rules and and the domain interval of y is divided into five regions
the linguistic fuzzy rules into a common fuzzy rule base, and, ( N = 2). The shape of each membership function is triangular;
finally, design a control or signal processing system based on one vertex lies at the center of the region and has membership
this combined fuzzy rule base. value unity; the other two vertices lie at the centers of the
In Section 11, we propose a five step procedure for gener- two neighboring regions, respectively, and have membership
ating fuzzy rules from numerical data pairs and show how values equal to zero. Of course, other divisions of the domain
to use these fuzzy rules to obtain a mapping from input regions and other shapes of membership functions are possible.
space to output space. Step 1 divides the input and output
spaces into fuzzy regions; Step 2 generates fuzzy rules from
Step 2 4 e n e r a t e Fuzzy Rules from Given Data Pairs
given desired input-output data pairs; Step 3 assigns a degree
to each generated rule; Step 4 forms the combined fuzzy First, determine the degrees of given x ~ ) , xand ~ ’ y(‘) in
rule base; and, Step 5 presents a defuzzifying procedure different regions. For example, ”(11) in Fig. 2 has degree 0.8 in
for obtaining a mapping based on the combined fuzzy rule B1. degree 0.2 in B2. and zero degrees in all other regions.
base. In Section 111, we prove that the resulting mapping is Similarly, x r ) in Fig. 2 has degree 1 in C E . and zero degrees
capable of approximating any nonlinear continuous function in all other regions.
on a compact set to arbitrary accuracy using the well-known Second, assign a given x p ) , x f ’ or y(‘) to the region with
Stone-Weierstrass theorem in analysis [ 5 ] . In Section IV, we maximum degree. For example, xy) in Fig. 2 is considered to
apply our new method to a truck backer-upper control problem be B l , and xf’ in Fig. 1 is considered to be CE.
[l], [4]. We compare this new approach with pure neural and Finally, obtain one rule from one pair of desired
fuzzy approaches. The power of our new approach becomes
input-output data, e.g.,
apparent when it is used in the case where neither linguistic
fuzzy rules nor input-output pairs are sufficient to successfully (xf),x?);y(l))j[x!)(o.ginBl, m a x ) , x t ) ( 0 . 7 i n ~ lmax);
,
control the truck to a desired position, but the combination y(l)(0.9in C E , max)] 3 Rule 1:
of both is sufficient. In Section V, we show that our new
method can be used for time-series prediction; and, we use it to IF x1 is B1 and 22 is 5’1. THEN y is C E ;
predict the Mackey-Glass chaotic time series, and compare the
results with those obtained using a neural network predictor. (xf),xf);y(2))+ [xy)(0.6in B1, max). xf’(1 in C E , max);
Conclusions are given in Section VI. ~ ( ~ ) ( 0 . 7 i n Bmax)]
l, + Rule2:
IF x1 is B1 and x2 is C E , THEN y is B1.
11. GENERATING
FUZZY RULESFROM NUMERICAL
DATA The rules generated in this way are “and” rules, i.e., rules in
Suppose we are given a set of desired input-output data which the conditions of the IF part must be met simultaneously
pairs: in order for the result of the THEN part to occur. For the
problems considered in this paper, i.e., generating fuzzy rules
from numerical data, only “and” rules are required since the
antecedents are different components of a single input vector.
where 2 1 and 2 2 are inputs, and y is the output. This simple
two-input one-output case is chosen in order to emphasize
and to clarify the basic ideas of our new approach; extensions Step 3 4 s s i g n a Degree to Each Rule
to general multi-input multi-output cases are straightforward Since there are usually lots of data pairs, and each data pair
and will be discussed later in this section. The task here is generates one rule, it is highly probable that there will be some
to generate a set of fuzzy rules from the desired input-output conflicting rules, i.e., rules that have the same IF part but a

Authorized licensed use limited to: INDIAN INSTITUTE OF TECHNOLOGY KANPUR. Downloaded on February 01,2021 at 06:41:21 UTC from IEEE Xplore. Restrictions apply.
 1416 IEEE TRANSACTIONS ON SYSTEMS, MAN, AND CYBERNETICS, VOL. 22, NO. 6, NOVEMBEWDECEMBER 1992

kl=l#d
1.0

0.0
* x1 B1
B2
x;
(a)

Nx,)
1. S3 S2 SI CE 81 82 B3 s1

52

s3

S2 S1 CE 01 B2

XI

1.0.. Fig. 3 . The form of a fuzzy rule base.

i.e., the degree of a rule is defined as the product of the

degrees of its components and the degree of the data pair that
generates this rule. This is important in practical applications,
because real numerical data have different reliabilities, e.g,
Fig. 2. Divisions of the input and output spaces into fuzzy regions and the
some real data can be very bad (“wild data”). For good data
corresponding membership functions. (a) n a ( s 1 ) .(b) m ( s p ) . (c) n7(y). we assign higher degrees, and for bad data we assign lower
degrees. In this way, human experience about the data is used
in a common base as other information. If one emphasizes
different THEN part. One way to resolve this conflict is to
objectivity and does not want a human to judge the numerical
assign a degree to each rule generated from data pairs, and
data, our strategy still works by setting all the degrees of the
accept only the rule from a conflict group that has maximum
data pairs equal to unity.
degree. In this way not only is the conflict problem resolved,
but also the number of rules is greatly reduced. Step 4-Create a Combined Fuzzy Rule Base
We use the following product strategy to assign a degree to
The form of a fuzzy rule base is illustrated in Fig. 3.
each rule: for the rule: “IF z1 is A and 22 is B, THEN y is
We fill the boxes of the base with fuzzy rules according
C,” the degree of this rule, denoted by D(Rule), is defined as
to the following strategy: a combined fuzzy rule base is
D(Rule) = mA(21)mB(z2)mC(y)* (2) assigned rules from either those generated from numerical
data or linguistic rules (we assume that a linguistic rule
As examples, Rule 1 has degree also has a degree that is assigned by the human expert and
D(Rule1) = mB1 (zl)msl(52)mcE(Y) reflects the expert’s belief of the importance of the rule); if
= 0.8 x 0.7 x 0.9 = 0.504
there is more than one rule in one box of the fuzzy rule
(3) base, use the rule that has maximum degree. In this way,
(see Fig. 2) and Rule 2 has degree both numerical and linguistic information are codified into
a common framework-the combined fuzzy rule base. If a
D(Rule2) = mBl(21 )mCE(22)mBl (Y) linguistic rule is an “and” rule, it fills only one box of the
= 0.6 x 1 x 0.7 = 0.42. (4) fuzzy rule base; but, if a linguistic rule is an “0r” rule (i.e., a
In practice, we often have some a priori information about rule for which the THEN part follows if any condition of the
the data pairs. For example, if we let an expert check given IF part is satisfied), it fills all the boxes in the rows or columns
data pairs, the expert may suggest that some are very useful corresponding to the regions of the IF part. For example,
and crucial, but others are very unlikely and may be caused suppose we have the linguistic rule: “IF 21 is S1 or 22 is
just by measurement errors. We can therefore assign a degree C E , THEN y is B2” for the fuzzy rule base of Fig. 3; then
to each data pair that represents our belief of its usefulness. In we fill the seven boxes in the column of S1 and the five boxes
this sense, the data pairs constitute a fuzzy set, i.e., the fuzzy in the row of C E with B 2 . The degrees of all the B2’s in
set is defined as the useful measurements; a data pair belongs these boxes equal the degree of this “or7’rule.
to this set to a degree assigned by a human expert. Step 5 4 e t e r m i n e a Mapping Based on
Suppose the data pair (~(11)~
2 ; ) ; y(l)) has degree m(’), then the Combined Fuzzy Rule Base
we redefine the degree of Rule 1 as
We use the following defuzzification strategy to determine
D (Rulel) = m B 1(21)ms1(.2 (Y ) m ( l ) (5) the output control y for given inputs (21, ~ 2 ) first,
: for given

Authorized licensed use limited to: INDIAN INSTITUTE OF TECHNOLOGY KANPUR. Downloaded on February 01,2021 at 06:41:21 UTC from IEEE Xplore. Restrictions apply.
 WANG AND MENDEL: GENERATING FUZZY RULES BY LEARNING FROM EXAMPLES 1417

inputs (51,52),we combine the antecedents of the ith fuzzy that when new inputs are presented the mapping continues
rule using product operation to determine the degree, m&, of to give desired or successful outputs. Hence, our new method
the output control corresponding to (51, ZZ), i.e., can be viewed as a very general model-free trainable fuzzy
system for a wide range of control and signal processing
mb. = mq ( 5 1 ) w ; ( 5 2 1 , (6) problems, where: “Model-Free” means no mathematical model
where Oa denotes the output region of Rule i , and I; denotes is required for the problem; “Trainable” means the system
the input region of Rule i for the j t h component, e.g., Rule learns from “examples” and expert rules, and can adaptively
1 gives change the mapping when new “examples” and expert rules
are available; and, “Fuzzy” denotes the fuzziness introduced
m&E = m m (21>ms1(.2) (7) into the system by linguistic fuzzy rules, fuzziness of data, etc.
then, we use the following centroid defuzzification formula to
determine the output 111. FUZZY SYSTEM AS A UNIVERSAL APPROXIMATOR
The five step procedure of the last section generates a fuzzy
system, i.e., a mapping .from input space to output space.
(8) Specifically, this mapping is represented by (6) and (8) for
the two-input one-output case. Using simplified notations, we
rewrite (6) and (8),for the general n-input one-output case, as

where jjz denotes the center value of region O i (the center

of a fuzzy region is defined as the point that has the smallest
absolute value among all the points at which the membership
function for this region has membership value equal to one),
and K is the number of fuzzy rules in the combined fuzzy
rule base.
From Steps 1 to 5 we see that our new method is simple
and straightforward in the sense that it is a one-pass build-
up procedure that does not require time-consuming training;
hence, it has the same advantage that the fuzzy approach has
over the neural approach, namely, it is simple and quick to where mi is the membership function of the ith rule for the
construct. jth component of the input vector, and is the center value
This five step procedure can easily be extended to general of the output region of the ith rule. We will prove that this
multi-input multi-output cases. Steps 1 to 4 are independent of generated fuzzy system, i.e., (ll),is a universal approximator
how many inputs and how many outputs there are. In Step 5, from a compact set Q c R” to R , i.e., it can approximate any
we only need to replace m;. in (6) with m& , where j denotes real continuous function defined on Q to any accuracy, where
the jth component of the output vector (0: is the region of the compact set Q is defined as
Rule i for the jth output component; mi is the same for all
j ) , and change (8) to
0; Q = [ ~ i , b l x] [ ~ 2 , b 2 x] . . . x [an>bn]C R”. (12)

For notational convenience, we represent Rule z(i =

1 , 2 , . . . ,K ) in the fuzzy rule base as: “IF x1 is RG;, 2 2 is
(9) RG;, . . ‘ x, is RG;, THEN y is RGb,” where RGJ( j =
~

1,2, . . . ,n) denotes the region for the g’th input antecedent
of Rule i (e.g., it can be S2. C E . B1. etc.), and RGb denotes
the outupt region of Rule i.
where $ denotes the center of region 03. Let F be the family of functions of the form of (11) on
If we view this five step procedure as a block, then the the compact set Q. There are three factors that determine a
inputs to this block are “examples” (desired input-output data member of F : 1) the definition of fuzzy regions, i.e., how to
pairs) and expert rules (linguistic IF-THEN statements), and define and divide the domain intervals; 2) the specific form
the output is a mapping from input space to output space. of membership functions mi; and, 3) the specific statements
For control problems, the input space is the state of the plant of fuzzy rules in the fuzzy rule base. By fixing fuzzy regions,
to be controlled, and the output space is the control applied membership functions, and fuzzy rules, we obtain an element
to the plant. For time-series prediction problems, the input of F. If f1 and f2 are different elements of F, then at least one
and output spaces are subsequences of the time series such of the three factors for fl and f2 must be different. In order
that the input subsequence precedes the output subsequence to analyses the family F, we make the following assumptions
(details are given in Section V). Our new method essentially for these three factors:
“learns” from the “examples” and expert rules to obtain a AS.1: The fuzzy regions for the input and output spaces
mapping that, hopefully, has the “generalization” property can be arbitrarily defined.

Authorized licensed use limited to: INDIAN INSTITUTE OF TECHNOLOGY KANPUR. Downloaded on February 01,2021 at 06:41:21 UTC from IEEE Xplore. Restrictions apply.
 1418 IEEE TRANSACTIONS ON SYSTEMS, MAN, AND CYBERNETICS, VOL. 22, NO. 6, NOVEMBERDECEMBER 1992

AS.2: The membership functions m: can be any continuous uses AS.4 and the definition of "active rule" that will be given
functions from [a,, b,] to [0, 11 for J = 1.2, ,n (i.e., for later in this section; hence, we suggest the reader read the rest
+ e .

inputs) and from (-30,c-o) to [0, 11 for j = 0 (i.e., for of this section before going to the proof of Theorem 1. The
output); however, mi must satisfy the following constraint: rest of this section will not use the result of Theorem 1.)
m ; ( z , ) # O f o r x , E RG~.z=1,2,....K.~=O,l,...,n, Theorem 1 is an existence theorem showing that there
with xo = y. This constraint means that the membership value exists a way of defining fuzzy regions, a way of choosing
of an antecedent for a rule cannot equal zero if the actual input membership functions, and a way of assigning fuzzy rules
value of this antecedent falls into the required region of the to the boxes of the fuzzy rule base, such that the resulting
rule. mapping, (ll),approximates an arbitrary nonlinear continuous
AS.3: Any rule can be assigned to any box of the fuzzy mapping from Q to R to any accuracy. This Theorem is
rule base. similar to the results of [6] and [7], which showed that a three-
These assumptions are usually satisfied in practice. Specifi- layer feedforward neural network is a universal approximator
cally, we have total freedom in defining fuzzy regions; we can provided that there are sufficiently large numbers of hidden-
choose any membership functions subject to the constraint of layer neurons. Theorem 1 provides the theoretical basis for
AS.2; and, we can assign any rule to any box of the fuzzy successful applications of our new method to many different
rule base. practical problems.
To analyses the properties of the function family F, we In many applications of fuzzy systems (e.g., [3], [4]), the
must first establish that the mapping defined by (11) is well- membership functions are triangular. We now study some
defined, i.e., for any input z E Q. (11) will generate an properties of the fuzzy systems that use the specific form of
output f ( z ) E R. The following two lemmas give sufficient membership functions that are defined as follows.
conditions for (11) to be well-defined. AS.4: The membership function for any intermediate fuzzy
Lemma 1: If all the membership functions m i are nonzero, region (i.e., not the smallest or the largest region) is a triangle
and there is at least one rule in the fuzzy rule base, then the whose vertices are at ( x . m ) = (x-l,O), (20,1). and (xl,O),
mapping defined by (11) from Q to R is well-defined. where the x-axis denotes a coordinate of the input or output
Proofs of lemmas and theorems are given in Appendix I. space, the m-axis denotes the corresponding membership
Lemma 2: If every box in the fuzzy rule base has a rule value, xo denotes the center of the region, and x P 1 ( x 1 )denotes
associated with it, i.e., there are no empty boxes in the fuzzy the center of the left (right) region. See Fig. 11 for an example.
rule base, then the mapping defined by (11) from Q to R is The membership functions for the smallest and largest regions
well-defined under AS.2. are determined by the way shown in Fig. 11.
In practice, the input space is usually high dimensional, If every box of the fuzzy rule base has a rule and [a3.b3] is
whereas the given successful data pairs and expert rules are divided into r3 fuzzy regions ( j = 1.2. . . . ,n ) ,then there are
often quite limited; as a result, many boxes of the fuzzy rule N = T I x 7-2 x ... x T , rules in the fuzzy rule base. N can
base may be empty. However, it is possible to fill up these be a huge number if the T ; S and n are large. However, under
empty boxes based on the limited given rules using the method the situation of AS.4, there are only a small fraction of these
of Section 11. Specifically, Steps 1-4 are first used to generate rules that are really used in (11) for any given z E Q.
a fuzzy rule base based on the limited data pairs and linguistic Definition: The zth fuzzy rule in the fuzzy rule base is
rules; then, the output for some typical input for which the active for z E Q if m)(x3) # 0 for all J = 1 , 2 , . . . ,n.
box in the fuzzy rule base is empty can be determined based Referring to ( l l ) , we see that a rule is active means that it
on the limited fuzzy rule base; finally, the range in which the will be used in (11).
output has the maximum degree is assigned to the empty box Lemma 3: Under AS.4, the following is true:
as a new rule. This can be an iterative procedure, i.e., when
1) There are at most 2" active rules for any z E Q.
a new rule is generated, this new rule and the existing rules
2) If T components of z E Q are at the centers of some
are combined into a fuzzy rule base that is used to generate
fuzy regions ( T = 0 , 1 . 2 , . . . , n ) , there are at most 2"-'
the next new rule. We can start the procedure from the empty
active rules at the z (the center of a fuzzy region is
boxes that are the nearest neighbors of the full boxes; in this
defined in Step 5 of Section 11).
way, the fuzzy rule base expands from existing rules until all
3) If T components of z E Q are at the centers of some
the boxes are filled up. This procedure always works if we
fuzzy regions, and if q components of the 5 are smaller
choose the nonzero regions of the membership functions to be
(greater) than the center values of the smallest (largest)
large enough such that the values of the membership functions
regions of the corresponding components, then there are
will not be zero for some points of their nearest neighbors. We
at most 2"-'-q active rules at the z.
will not study this procedure in detail in this paper; we gave
the basic ideas of the procedure in order to show that the Lemma 3 is useful in practice. Although we may need a
conditions of Lemma 2 can be satisfied. huge memory to store the fuzzy rule base, when we use the
Now we state the main result of this section. fuzzy rule base for a given input z E Q, only a relatively
Theorem 1: If the mapping defined by (11) is well-defined, small number of rules are used. In practice, we may store the
and if AS.l-AS.3 are true, then the mapping defined by (11) is fuzzy rule base in a cheap external memory; when we have an
capable of approximating any real continuous function over the input, we only take the active rules from the fuzzy rule base
compact set Q to arbitrary accuracy. (The proof of Theorem 1 into the host computer.

Authorized licensed use limited to: INDIAN INSTITUTE OF TECHNOLOGY KANPUR. Downloaded on February 01,2021 at 06:41:21 UTC from IEEE Xplore. Restrictions apply.
 __

WANG AND MENDEL GENERATING FUZZY RULES BY LEARNING FROM EXAMPLES 1419

loading dock x=10 $=90

rear

x=o x=20
Fig. 4. Diagram of simulated truck and loading zone

PPLICATION TO TRUCKBACKER-UPPER
CONTROL input-output pairs to simulate neural 2 d imerical-fuzzy
Backing a truck to a loading dock is a difficult exercise. It controllers, and compare their final control performance.
is a nonlinear control problem for which no traditional control Statement of the Truck Bucker-Upper Control Problem: The
system design methods exist. In [l], Nguyen and Widrow simulated truck and loading zone are shown in Fig. 4 [l],
develop a neural network controller for the truck backer-upper [4]. The truck corresponds to the cab part of the neural truck
in the Nguyen-Widrow [11 neural truck backer-upper system.
problem; and, in [4], Kong and Kosko propose a fuzzy control
The truck position is exactly determined by the three state
strategy for the same problem. The neural network controller
variables 4 , x , and y, where 4 is the angle of the truck
[ l ] only uses numerical data, and cannot utilize linguistic rules
with the horizontal as shown in Fig. 4. Control to the truck
determined from expert drivers; on the other hand, the fuzzy
is the angle 6 . Only backing up is considered. The truck
controller of [4] only uses linguistic rules, and cannot utilize
moves backward by a fixed unit distance every stage. For
sampled data. Since the truck backer-upper control problem
simplicity, we assume enough clearance between the truck and
is a good example of the control system design problem
the loading dock such that y does not have to be considered
discussed in the Introduction of this paper (i.e., replace a
as an input. The task here is to design a control system, whose
human controller by a machine), it is interesting to apply the
inputs are 4 E [-90°,2700]and x E [0,20], and whose
approach developed in Section I1 to this problem. In order to
output is 0 E [-4Oo,4O0], such that the final states will be
distinguish these methods, we call the method of [4] the “fuzzy
( X f : d f ) = (10,9O0).
approach,” the method of [ l ] the “neural approach,” and our
Generating Desired Input-Output Puirs(x,4;0): We do
new method the “numerical-fuzzy approach.”
this by trial and error: at every stage (given q5 and x) starting
The results of [4] demonstrated superior performance of the
from an initial state, we determined a control 6 based on
fuzzy controller over the neural controller; however, the fuzzy
common sense (i.e., our own experience of how to control
and neural controllers use different information to construct the steering angle in the situation); after some trials, we chose
the control strategies. It is possible that the fuzzy rules used in the desired input-output pairs corresponding to the smoothest
[4] to construct the controller are more complete and contain successful trajectory.
more information than the numerical data used to construct the The following 14 initial states were used to generate desired
neural controller; hence, the comparison between the fuzzy and input-output pairs: (zo,&j) = (1, 0), (1, 90), (1, 270); (7,
neural controllers, from a final control performance point of O), (7, 90), (7, 180), (7, 270); (13, O), (13, 901, (13, 1801,
view, is somewhat unfair. If the linguistic fuzzy rules were (13, 270); (19, 90), (19, 180), (19, 270). Since we performed
incomplete, whereas the numerical information contained lots simulations, we needed to know the dynamics of the truck
of very good data pairs, it is highly possible that the neural backer-upper procedure. We used the following approximate
controller would outperform the fuzzy controller. kinematics (see [8] for details):
Our new numerical-fuzzy approach provides a fair basis
for comparing fuzzy and neural controllers (the numerical- z(t + 1) = z ( t )+ cos[$(t)+ 6(t)] + sin[0(t)]sin[4(t)] (13)
fuzzy approach can be viewed as a fuzzy approach in the
sense that it differs from the pure fuzzy approach only in the
way it obtains fuzzy rules). We can provide the same desired
input-output pairs to both the neural and numerical-fuzzy
y(t + 1) = y(t) + sin[$(t) + 0 ( t ) ] - sin[O(t)]cos[4(t)] (14)
approaches; consequently, we can compare the final control
performances of both controllers fairly since they both use the
same information.
Example 1: In this example, we use the same set of desired

Authorized licensed use limited to: INDIAN INSTITUTE OF TECHNOLOGY KANPUR. Downloaded on February 01,2021 at 06:41:21 UTC from IEEE Xplore. Restrictions apply.
 1420 IEEE TRANSACTIONS ON SYSTEMS, MAN, AND CYBERNETICS, VOL. 22, NO. 6, NOVEMBEIUDECEMBER 1992

TABLE I
DESIRED
TRAJECTORY
STARTING = (1.0')
FROM (IOQO)

I 0 I 1.00 I 0.00 I1 -19.00 I

0 5 10 IS M X

Fig. 5. Truck trajectories using the neural controller and the numerical-fuzzy
controller.

In the simulations, we normalized [-40°,40'1 into [-1,1].

Similarly, the inputs to the neurons were also normalized into
[-I, 11.
Our neural network controller is different from the
Nguyen-Widrow neural controller [ 11. First, we have only
one neural network that does the same work as the Truck
Controller of the Nguyen-Widrow network; the truck emulator
of the Nguyen-Widrow network is not needed in our task.
Second, and more fundamentally, we train our neural network
where b is the length of the truck. We assumed b = 4 in using desired input-output (state-control) pairs, which are
the simulations of this paper. Equations (13)-(15) were used obtained from the past successful control history of the truck,
to obtain the next state when the present state and control are whereas Nguyen and Widrow [l]connect their neural network
given. Since y is not considered a state, only (13) and (15) were stage by stage and train the concatenated neural networks by
used in the simulations. We wrote (14) here for the purpose of back-propagating the error at the final state through this long
showing the complete dynamics of the truck. Observe, from network chain (the detailed algorithm is different from the
(13)-(15), that even this simplified dynamic model of the truck standard error back-propagation algorithm in order to meet the
is nonlinear. The 14 sequences of desired (.,$;e) pairs are constraint that the neural networks at each stage perform the
given in [8]; we only include one such sequence in this paper same transformation; for details see [l]). Hence, the training of
(Table I). our neural network is simpler than that of the Nguyen-Widrow
Neural Control and Simulation Results: We used a two- network. Of course, we need to know some successful control
input single-output three-layer back-propagation neural net-
trajectories (state-control pairs) starting from some typical
work [9, 101 for our control task. Twenty hidden neurons
initial states; this is not required in the Nguyen-Widrow neural
were used, and a sigmoid nonlinear function was used for network controller.
each neuron. The output of the third-layer neuron represents
We trained the neural network using the standard error back-
the steering angle 8 according to a uniform mapping from
propagation algorithm [9], [lo] for the generated 14 sequences
[0, 11 to [-40",40"], i.e., if the neuron output is g ( t ) . the
of desired (z, $; 8) pairs. We used the converged network to
corresponding output O(t) is
control the truck whose dynamics are approximately given by
O(t) = 80g(t) - 40. (16) (13)-(15). Three arbitrarily chosen initial states, (xo,&) =

Authorized licensed use limited to: INDIAN INSTITUTE OF TECHNOLOGY KANPUR. Downloaded on February 01,2021 at 06:41:21 UTC from IEEE Xplore. Restrictions apply.
 WANG AND MENDEL GENERATING FUZZY RULES BY LEARNING FROM EXAMPLES 1421

-90 -45 0 45 90 135 180 225 270 -

A
y
n
a b

A
0 1.5 4 7 9 1011 13 16 18.5 20

40 -33 -20 -14 -7-4 0 4 7 14 20 33 40

Fig. 6. Fuzzy membership functions for the truck backer-upper control problem.

(3, -30), (10, 220), and (13, 30), were used to test the neural A

controller. The truck trajectories from the three initial states are CE B1 62
shown in Fig. 5. We see that the neural controller successfully
53
controls the truck to the desired position starting from all three
initial states. S2
Numerical-Fuzzy Control and Simulation Results: We used
the five-step procedure of Section I1 to determine the control s1
law f : (x,+)4 0, based on the 14 generated sequences of @ CE
successful (x,$; 0) pairs. For this specific problem, we used
membership functions shown in Fig. 6, which are similar to 81
those used in [4] for fuzzy control of the problem based only 82
on linguistic rules. The fuzzy rules generated from the desired
input-output pairs and their corresponding degrees are given 83
in [8]; we show only the generated rules for the data pairs
of Table I (in this paper) in Table 11. The final fuzzy rule Fig. 7. The final fuzzy rule base generated from the numerical data for the
base is shown in Fig. 7 (this is the result of Step 4 of our truck backer-upper control problem.
method in Section 11; here we assume that no linguistic rules
are available). We see from Fig. 7 that there are no generated (3, -30), (10, 220), and (13, 30), which are the same states
rules for some ranges of x and 4. This shows that the desired used in the simulations of the neural controller. The final
trajectories from the 14 initial states do not cover all the trajectories of the truck have no visible difference from Fig.
possible cases; however, we will see that the rules in Fig. 5; hence, Fig. 5 also shows the track trajectories using the
7 are sufficient for controlling the truck to the desired state numerical-fuzzy controller.
starting from some given initial states. We simulated the neural and numerical-fuzzy controllers
Finally, Step 5 of our numerical-fuzzy method was used for other initial truck positions, and observed that the truck
to control the truck from the three initial states, ( X O , ~ ; ) = trajectories using these two controllers were also almost the

Authorized licensed use limited to: INDIAN INSTITUTE OF TECHNOLOGY KANPUR. Downloaded on February 01,2021 at 06:41:21 UTC from IEEE Xplore. Restrictions apply.
 1422 IEEE TRANSACTIONS ON SYSTEMS, MAN, AND CYBERNETICS, VOL. 22, NO. 6 , NOVEMBEWDECEMBER 1992

TABLE I1
FUZZYRULESGENERATED FROM THE DESIRED
INPUT-OUTPUT
PAIRS OF TABLE1 AND THE DEGREES
OF THESERULES

0 5 10 15 ? o x

Fig. 8. Truck trajectories using the fuzzy rules from the truncated data pairs
only.

there are no rules in the three center boxes outlined by the

heavy lines. The fuzzy rule base of linguistic rules for the
ending part was chosen to have only three rules that are the
same as the three center rules of Fig. 7.
We simulated the following three cases in which we used
the: 1) fuzzy rule base generated from only the truncated data
pairs; 2) fuzzy rule base of selected linguistic rules; and, 3)
20 fuzzy rule base which combined the fuzzy rule bases of 1) and
2). We see that for Case 3 the fuzzy rule base is the same as
in Fig. 7; hence, the truck trajectories for this case must be the
same. This is not surprising because both controllers used the same as those using the fuzzy rule base of Fig. 7. For each of
same information to construct their control laws. the cases, we simulated the system starting from the following
Example 2: In this example we consider the situation where three initial states: (q,,&) = (3, -30), (10, 220), and (13,
neither linguistic fuzzy rules alone nor desired input-output 30). The resulting trajectories for cases l), 2), and 3) for the
pairs alone are sufficient to successfully control the truck to three initial states are shown in Figs. 8, 9 and 5 , respectively.
the desired position, i.e., neither the usual fuzzy controller We see very clearly from these figures that, for cases (1)
with limited fuzzy rules nor the usual neural controller can and (2) the truck cannot be controlled to the desired position,
control the truck to the desired position, but a combination whereas for case (3) we successfully controlled the truck to
of linguistic fuzzy rules and fuzzy rules generated from the the desired position.
desired inputkoutput data pairs is sufficient to successfully
control the truck to the desired position. TO TIME-SERIES
V. APPLICATION PREDICTION
We consider the case where the beginning part of the Time-series prediction is a very important practical problem
information comes from desired input-output pairs whereas [ 2 ] . Applications of time-series prediction can be found in
the ending part of the information comes from linguistic rules. the areas of economic and business planning, inventory and
To do this we used only the first three pairs of each of the production control, weather forecasting, signal processing,
14 desired sequences, and generated fuzzy rules based only control, and lots of other fields. Let z ( k ) ( k = 1 , 2 , 3 , . . .) be
on these truncated pairs. The fuzzy rule base generated from a time series. The problem of time-series prediction can be
these truncated data pairs is the same as Fig. 7 except that + +
formulated as: given z ( k - m l),z ( k - m 2), . . , z ( k ) ,

Authorized licensed use limited to: INDIAN INSTITUTE OF TECHNOLOGY KANPUR. Downloaded on February 01,2021 at 06:41:21 UTC from IEEE Xplore. Restrictions apply.
 WANG AND MENDEL: GENERATING FUZZY RULES BY LEARNING FROM EXAMPLES 1423

1.2

0.8

0.6

0.4

0.2 1 I
0 100 200 300 400 500 600 7GQ 800 900 lo00

Fig. 10. A section of the Mackey-Glass chaotic time series.

m(x)

S3 S2 SI CE B1 B2 83

( 10.22d ,

/ 03 05 0.7 0.9 1.1 1.3 1.5

Fig. 11. The first choice of membership functions for the chaotic time series
* x(t)

prediction problem.

Example 3: Now we apply our numerical-fuzzy approach

0 5 10 15 X to predict the Mackey-Glass chaotic time-series [ll].Chaotic
time series are generated from deterministic nonlinear sys-
Fig. 9. Truck trajectories using the selected linguistic rules only
tems and are sufficiently complicated that they appear to be
“random” time series; however, because there are underlying
+
determine z ( k I ) , where m and 1 are fixed positive integers; deterministic maps that generate the series, chaotic time series
i.e., determine a mapping from [ z ( k - m l ) , z ( k - m+ + are not random time series. In [Ill, feedfonvard neural net-
+
2),...,~(k)]E R” to [ ~ ( k l ) ] E R. works were used for chaotic time-series prediction, and were
A feedforward neural network can also be used for this prob- compared with conventional approaches, like linear predictive
lem [ 111. For example, we can use a three-layer feedforward method, Gabor polynomial method, etc. The results showed
neural network, which has m input neurons and one output that the neural network approach gave the best prediction,
neuron, to represent the mapping from [z(k - m l),z ( k - + and the accuracy obtained using the neural network approach
+ +
m 2 ) , . . . ,z(k)] to [ z ( k l ) ] . The network is trained for the was orders of magnitude higher than that obtained using the
known z ( k ) ’ s ,and then the converged network is used for the conventional approaches. Here we use our numerical-fuzzy
prediction. Specifically, assume that z(1), z(2), . . . , z ( M ) are approach to the same Mackey-Glass chaotic time series in
given; then we form M - m desired input-output pairs: [ll], and compare the results obtained with those obtained
using the neural network approach.
[z(M - m), , z ( M - 1); .(Ad)] The Mackey-Glass chaotic time series is generated from the
[z(M-m- I ) , . . . , Z ( M - ~ ) ; Z ( MI)]
- following delay differential equation:
... dz(t) - 0.22(t - T )
- - - O.lz(t).
I)].
[z(l),...,z(m);z(m+ (17) dt +
1 ~ “ (-tT ) (18)

We train the neural network to match these M - m pattern When 7 > 17, (18) shows chaotic behavior. Higher values of
pairs using the error back-propagation algorithm [9], [lo]. 7 yield higher dimensional chaos. In our simulation, we chose
Our numerical-fuzzy method in Section I1 can also be used the series with T = 30. Fig. 10 shows 1000 points of this
a for this time series prediction problem. Similar to the neural chaotic series that we used to test both the numerical-fuzzy
network approach, we assume that z ( l),z(2), . . . , z ( M ) are and neural approaches.
given, and we form the M - m desired input-output pattern We chose m = 9 and 1 = 1 in our simulation, i.e., nine point
pairs in (17). Steps 1-4 of our numerical-fuzzy approach are values in the series were used to predict the value of the next
used to generate a fuzzy rule base based on the pattern pairs time point. The membership functions for any point are shown
(17); then this fuzzy rule base is used to forecast z(A4 p ) + in Fig. 11 for the numerical-fuzzy predictor (later, we will use
-
for p = 1 , 2 , . . using the defuzzifying procedure of Step 5 of other membership functions). 40 hidden-layer neurons were
our numerical-fuzzy method, where the inputs to the network used for the neural network predictor. The first 700 points
+
are z ( M + p - m ) , z ( M + p - m l), , z ( M + p - 1). of the series were used as training data, and the final 300

Authorized licensed use limited to: INDIAN INSTITUTE OF TECHNOLOGY KANPUR. Downloaded on February 01,2021 at 06:41:21 UTC from IEEE Xplore. Restrictions apply.
 1424 IEEE TRANSACTIONS ON SYSTEMS, MAN, AND CYBERNETICS, VOL. 22, NO. 6, NOVEMBEWDECEMBER 1992

0.2

OO SO 100 150 200 2SO 300

Fig. 12. Prediction of the chaotic time series from x( 701j to r( 1000j using Fig. 15. Prediction of the chaotic time series from ~ ( 7 0 1to
) x( 1000) using
the numerical-fuzzy predictor when 200 training data (from ~ ( 5 0 1to) ~ ( 7 0 0 ) the neural predictor when 700 training data (from x(1) to ~ ( 7 0 0 )are
) used.
are used.

1.4
1.4
1.2
1.2
1
1
0.8
0.8
0.6
0.6
0.4
0.4

0.2

00 50 100 1so 200 250 300

Fig. 16. Prediction of the chaotic time series from S(701) to x ( 1000) using
) ,r( 1000) using
Fig. 13. Prediction of the chaotic time series from ~ ( 7 0 1 to the updating fuzzy rule base procedure.
the neural predictor when 200 training data (from s(501) to .r( 700)) are used.

this “adaptive” procedure we use all the available information

to predict the next value of the series. We simulated this
1.4
adaptive procedure for the chaotic series of Fig. 10: we started
1.2
with the fuzzy rule base generated by the data ~ ( 1 to) ~ ( 7 0 0 ) ,
1
made a prediction of ~ ( 7 0 1 ) ,then used the true value of
0.11
~ ( 7 0 1 )to update the fuzzy rule base, and this updated fuzzy
0.6
rule base was then used to predict ~ ( 7 0 2 ) .This adaptive
0.4
procedure continued until ~ ( 1 0 0 0 ) .Its results are shown in
0.2

0
50 100 150 ZOO 250
1
BOO
Fig. 16. Comparing Figs. 16 and 14 we see that we obtain
only a slightly improved prediction.
Fig. 14. Prediction of the chaotic time series from ~ ( 7 0 1 to
) ~ ( 1 0 0 0using
) Finally, we show that prediction can be greatly improved
the numerical-fuzzy predictor when 700 training data (from ~ ( 1 ) to x ( 7 0 0 j ) by dividing the “domain interval” into finer regions. We
are used. performed two simulations: one with the membership function
shown in Fig. 17, and the other with the membership function
points were used as test data (for additional cases, see [SI). shown in Fig. 18. We used the adaptive fuzzy rule base
We simulated two cases: 1) 200 training data (from 501 to procedure for both simulations. The results are shown in Figs.
700) were used to construct the fuzzy rule base and to train the 19 and 20, for the membership functions of Figs. 17 and
neural network; and, 2) 700 training data (from 1 to 700) were 18, respectively. Comparing Figs. 16, 19 and 20 we see very
used. Figs. 12 and 13 show the results of the numerical-fuzzy clearly that we obtain better and better results as the “domain
and neural predictors respectively for case 1); and, Figs. 14 interval’’ is divided finer and finer. Fig. 20 shows that we
and 15 show similar results for case 2). As in [ll],the “past” obtained an almost perfect prediction when we divided the
data needed to perform prediction is obtained from observing “domain interval” into 29 regions. Of course, the price paid
the actual time series; thus, one makes a prediction and uses for doing this is a larger fuzzy rule base.
the actual values to make the next prediction. We see from
Figs. 12 to 15 that our new numerical-fuzzy predictor gave VI. CONCLUSION
about the same results as the neural network predictor. In this paper, we developed a general method to generate
One advantage of the numerical-fuzzy approach is that it fuzzy rules from numerical data. This method can be used
is very easy to modify the fuzzy rule base as new data as a general way to combine both numerical and linguistic
become available. Specifically, when a new data pair becomes information into a common framework-a fuzzy rule base.
available, we create a rule for this data pair and add the new This fuzzy rule base consists of two kinds of fuzzy rules: some
rule to the fuzzy rule base; then, the updated (i.e., adapted) obtained from experts, and others generated from measured
fuzzy rule base is used to predict the future values. By using numerical data using the method of this paper. We proved that

Authorized licensed use limited to: INDIAN INSTITUTE OF TECHNOLOGY KANPUR. Downloaded on February 01,2021 at 06:41:21 UTC from IEEE Xplore. Restrictions apply.
 WANG AND MENDEL: GENERATING FUZZY RULES BY LEARNING FROM EXAMPLES 1425

m(x) from that of the pure neural network controller; and, 2) in the
case where neither numerical data nor linguistic rules contain
1 S7 S6 S5 S4 S3 S2 S1 CE B1 B2 83 84 B5 B6 B7 enough information, both the pure neural and pure fuzzy
methods failed to control the truck to the desired position, but
our new method succeeded. We also applied our new method
to a chaotic time-series prediction problem, and the results
showed that our new method worked quite well.
The main features and advantages of the new method
02 0.4 0.6 0.8 1.0 12 1.4 1.6
XU) developed in this paper are: 1) it provides us with a gen-
eral method to combine measured numerical information and
Fig. 17. The second choice of membership functions for the chaotic time human linguistic information into a common framework-a
series prediction problem.
combined fuzzy rule base; this could be viewed as a first step
to develop some theoretically analyzable control algorithms
that use both numerical and linguistic information; 2) it is
a simple and straightforward one-pass build-up procedure;
SI4 SI3 SI2 S11 SI0 S9 -....* B9 B10 811 BIZ 813 814 hence, no time-consuming iterative training is required, so
that it requires much less construction time than a comparable
neural network; 3) there is lots of freedom in choosing the
membership functions; this provides us with lots of flexibilities
to design systems according to different requirements; and, 4)
it can perform successful control for some cases where neither
a pure neural network control nor a pure fuzzy control can.
Fig. 18. The third choice of membership functions for the chaotic time series
prediction problem.
APPENDIX
I
ProofofLemma I : Since O 5 mi(x3) 5 1, it is suf-
ficient to prove that for any x E Q there exist some r such
that II1~,<~[m;(x~)] # 0. Under the condition of this lemma,
there exists at least one i such that II1~j~n[m;(x,)] # 0 for
any x E Q, hence (11) is well-defined. Q.E.D.
Proof of Lemma 2: Since every box in the fuzzy rule
base has a rule, for any x E Q there must be a rule, say Rule 2,
0.2
such that x3 E RG; for j = 1 , 2 , . . . , n. By AS.2, m;(x,) # 0
for all J = 1 , 2 , . . . ,n, hence II1,,~,[mg(x),)] # 0, i.e., (11)
OO 50 100 150 200 150 300
is well-defined. Q.E.D.
Fig. 19. Prediction of the chaotic time series from ~ ( 7 0 1to
) s(1000) using In order to prove Theorem 1 we need some definitions.
the updating fuzzy rule base procedure with the membership functions of Fig.
18.
A family F of real functions defined on a set E is an
algebra if F is closed under addition, multiplication, and scalar
multiplication. The family F separatespoints on E if for every
x,y E E,x # y, there exists a function f E F such that
-
1.4
f ( s )# f(y). The family F vanishes at no point of E if for
each x E E there exists f E F such that f (x)# 0. Our proof
of Theorem 1 is based on the Stone-Weierstrass Theorem [5],
which we state here for convenience of the reader.
Stone-Weierstrass Theorem: Let F be an algebra of real
continuous functions on a compact set K. If F separates points
0.2,-
on K and if F vanishes at no point on K , then the uniform
so 100 110 200 250 300 closure B of F consists of all real continuous functions on K .
Fig. 20. Prediction of the chaotic time series from ~ ( 7 0 1 to
) z(1000) using The uniform closure B of F is the union of F and its limit
the updating fuzzy rule base procedure with the membership functions of Fig. points; hence, if B consists of all real continuous functions on
19. K , then F is capable of approximating any real continuous
function on K to arbitrary accuracy.
the generated fuzzy system is capable of approximating any Proof of Theorem 1: Let F be the family of well-defined
nonlinear continuous function on a compact set to arbitrary functions of the form of (11) on the compact set Q under
accuracy. We applied our new method to a truck backer-upper AS.1, AS.2, and AS.3. If we prove that F is an algebra of
control problem [l], [4], and observed that: 1) for the same real continuous functions, F separates points on Q, and F
training set (i.e., the same given input-output pairs), the final vanishes at no point of Q , then the Stone-Weierstrass Theorem
control performance of our new method is indistinguishable guarantees the conclusion of Theorem 1.

Authorized licensed use limited to: INDIAN INSTITUTE OF TECHNOLOGY KANPUR. Downloaded on February 01,2021 at 06:41:21 UTC from IEEE Xplore. Restrictions apply.
 1426 IEEE TRANSACTIONS ON SYSTEMS, MAN, AND CYBERNETICS, VOL. 22, NO. 6, NOVEMBERDECEMBER 1992

By AS.2, the m;(x,)'s are assumed to be real continuous that has the smallest absolute value among all the points at
functions; hence, F is a family of real continuous functions. which the membership function for this region has membership
Let f l , f2 E F, so that we can write them as value equal to one; see Section 11, Step 5). Then, we choose
K1 the membership functions for the input space Q to be of the
specific triangular form defined by AS.4 of Section 111. By
m l l 3 l n b l ; (x,)I
z=1 such a choice of fuzzy regions and membership functions, we
f l k )= K , (19) have m3(xJ)= m:(z,) = 1 for each active Rule z at x, and
each active Rule 1 at z , and all j = 1 , 2 , . . .n (the definition
of active ruZe is given in Section 111); additionally, there is one
K2 and only one active rule for x and one and only one active
~",,,n[m2;(~,)1 rule for z,because (11) is well-defined (which guarantees that
a= 1
fdx) = K 2 (20) there is at least one active rule for x and at least one active
rule for z ) , and since only the membership functions for the
I I l l , l n [ma:(x,11 regions with centers at the components of x or z are nonzero
a=1
at x or z , whereas all other membership functions are zero at
Thus we have (21), shown at the bottom of the next page. Now x and z (which guarantees that there is at most one active rule
Define m1i1(x,)m2;2(x,) as a new membership function of for z and at most one active rule for 2). Since z # z , there
, define 3'' +$'* as the output center must be at least one j such that x, # z, , hence, the only active
x,, say m ~ ' " ( x , ) and
of a new rule, say gal"; then, (21) is of the form of (11); rule for x and the only active rule for z are at two different
+
hence, f l f 2 E F. Similarly, f l ( x ) f ~ ( x can ) be written as boxes of the fuzzy rule base. Since we are free to assign any
rules to the boxes of the fuzzy rule base (AS.3), we just assign
K1 K 2
two different rules to these two boxes, and obtain the required
~'1~2n11J,n[7111;1(.J)..,21;2(.J)I f E F with f(x) = Tjz # 5' = f ( z ) (see (ll)), where Tjz((y')
fl(Z)f2(2) = 21=1a;;1 K 2 is the center of the output region of the active rule for x ( z ) .
Finally, we prove that F vanishes at no point of Q . By
n l < , l n [ q l(x,)m2$(x,)I
21=122=1 AS.l and AS.2, we can make all the j j z > 0. Since (11) is well-
(22) defined, there exists at least one z such that IIl,,,,[m;(x,)] #
which is of the form (11); hence, f l f 2 E F. Finally, for any 0 for any x E Q. Since (11) is a weighted average of positive
c E R: y s with some nonzero weights, the result is also positive,
1 3

K1 i.e., we obtain f E F such that f ( x ) # 0 (in fact, f ( x ) > 0)

c32n11,1n[m1;(41 for any x E Q. Q.E.D.
Proof of Lemma 3: For arbitrary z E Q and fixed j 9there
cf1(z) = z=l K 1 (23)
are at most two m i ' s which are nonzero at x, under AS.4.
*ll,,n[ml) (341 Since a rule, say Rule 2. is active only when mi(z,) # 0 for
r=l all 7 = 1 , 2 , . , n, there are at most 2" active rules for any
+ e

which is also of the form of (11); hence cfl(x) E F. In 2 E Q; this Proves (1). If r components of x E Q are at the
summary, F is an algebra of real continuous functions. centers of some fuzzy regions, there is only one m:, which
Next, we prove that F separates points on Q. Let 2, z E Q is nonzero at each of these r components (in fact, these m)s
and x # z. We now construct f E F with f ( x ) # f ( z ) . First, are equal to unity at these r components), and for each of the
we define the fuzzy regions of the input space Q such that other n - r components there are at most two nonzero mj's,
each element of x and z is at the center of a fuzzy region hence the total number of active rules is at most 2n-T; this
(recall that the center of a fuzzy region is defined as the point proves (2). If r components of x E Q are at the centers of

Authorized licensed use limited to: INDIAN INSTITUTE OF TECHNOLOGY KANPUR. Downloaded on February 01,2021 at 06:41:21 UTC from IEEE Xplore. Restrictions apply.
 WANG AND MENDEL: GENERATING FUZZY RULES BY LEARNING FROM EXAMPLES 1427

some fuzzy regions and q components of the z are smaller (or Jerry M. Mendel (S’59-A’61-M’72-F778) re-
greater) than the center values of the corresponding smallest ceived the B.S. degree in mechanical engineering
and the M.S. and Ph.D. degrees in electrical
(or the corresponding largest ) fuzzy regions, then there is only engineering from the Polytechnic Institute of
+
one nonzero m:. for each of these r q components, and the Brooklyn, Brooklyn, NY, in 1959, 1960, and 1963,
other n - r - q components have two nonzero mj ’s associated respectively.
His experience has included teaching courses in
with each of them; hence, the total number of active rules is Electrical Engineering at the Polytechnic Institute of
at most 2 n - - r - q ; this proves 3). Q.E.D. Brooklyn, from 1960 to 1963, and has also included
various consulting positions. From July 1963 to
REFERENCES
Januarv 1974 he was with McDonnell Douglas -
Astronautics Company. Currently he is Professor of Electrical Engineering-
Systems at the University of Southern California in Los Angeles, and is
[ l ] D. Nguyen and B. Widrow, “The truck backer-upper: An example of Director of the Signal & Image Processing Institute. He was Chairman of
self-learning in neural network,” IEEE Contr. Syst. Mag., vol. 10, no. the EE-Systems Department from March 1984 to August 1991.
3, pp. 18-23, 1990. He teaches courses in estimation theory, deconvolution, and higher-
[2] G. E. P. Box and G. M. Jenkins, Time Series Analysis: Forecasting and
order statistics. He has published more than 240 technical papers and is
Control. Oakland, C A Holden-Day, 1976. author of the monographs Maximum-Likelihood Deconvolution: a Journey
[3] Y. F. Li and C. C. Lan, “Development of fuzzy algorithms for servo into Model-Based Signal Processing (Springer-Verlag, 1990) and Optimal
systems,” IEEE Contr. Syst. Mag., vol. 9, no. 3, pp. 65-72, 1989.
[4] S. G. Kong and B. Kosko, “Comparison of fuzzy and neural truck Seismic Deconvolution: An Estimation-Based Approach (Academic Press,
backer upper control systems,” in Proc. IJCNN-90, vol. 3, June 1990, 1983), the texts Lessons in Digitaf Estimation Theory (Prentice-Hall, 1987),
and Discrete Techniques of Parameter Estimation: The Equation Error
pp. 349-358.
[5] W. Rudin, Principles of Mathematical Analysis. New York: McGraw- Formulation (Dekker, 1973), and, co-editor (with K. S. Fu (deceased)) of
Hill, 1964. Adaptive, Learning and Pattern Recognition Systems (Academic Press, 1970).
[6] K. Hornik, M. Stinchcombe and H. White, “Multilayer feedforward He is also author of the IEEE Individual Learning Program, Kalman Filtering,
networks are universal approximators,” Neural Networks, vol. 2, pp. and Other Digital Estimation Techniques. He served as Editor of the IEEE
359-366, 1989. Control Systems Society’s IEEE TRANSAC~IONS ON AUTOMATIC CONTROL, and
[7] G. Cybenko, “Approximations by superpositions of a sigmoidal func- is on the Editorial Board of the IEEE PROCEEDINGS.
tion,” Mathematics of Control, Signals, and Systems, 1989. Dr. Mendel is a Fellow of the IEEE, Distinguished Member of the
[8] L. X. Wang and J. M. Mendel, “Generating fuzzy rules from numerical IEEE Control Systems Society, member of the IEEE Geoscience and
data, with applications,” USC SIP1 Rep. No. 169, Univ. Southern Calif., Remote Sensing Society, the IEEE Signal Processing Society, the Society of
Los Angeles, 1991. Exploration Geophysicists, the European Association for Signal Processing,
[Y] P. Werbos, “New tools for predictions and analysis in the behavioral Tau Beta Pi and Pi Tau Sigma, and a registered Professional Control Systems
science,” Ph.D. dissertation, Harvard Univ. Comm. Appl. Math., 1974. Engineer in California. He was President of the IEEE Control Systems Society
[lo] D. E. Rumelhart and J. L. McClelland, Eds., Parallel Distributed in 1986. He received the SEG 1976 Outstanding Presentation Award for a
Processing, Vol. 1. Cambridge, M A MIT Press, 1986. paper on the application of Kalman Filtering to deconvolution; the 1983 Best
[ l l ] A. Lapedes and R. Farber, “Nonlinear signal processing using neural Transactions Paper Award for a paper on maximum-likelihood deconvolution
networks: Prediction and system modeling,” LA-UR-87-2662, 1987. in the IEEE TRANSACTIONS ON GEOSCIENCE AND REMOTE SENSING; a Phi Kappa
Phi book award for his 1983 research monograph on seismic deconvolution;
a 1985 Burlington Northern Faculty Achievement Award; and a 1984 IEEE
Centennial Medal.

Li-Xin Wang received the B.S. and M.S. degrees

from the Northwestern Polytechnical University,
Xian, People’s Republic of China, in 1984 and
1987, respectively, and the Ph.D. degree from the
University of Southern California, Los Angeles in
1992, all in electrical engineering.
From 1987 to 1989, he was with the Department
of Computer Science and Engineering, Northwest-
ern Polytechnical University, Xian, People’s Repub-
lic of China. From Fall 1989 to Spring 1992, he was
a ResearchFeaching Assistant in the Department of
Electrical Engineering-Systems at the University of Southern California, Los
Angeles. He is now a Postdoctoral Fellow in the Department of Electrical
Engineering and Computer Science, University of California at Berkeley. His
research interests are fuzzy systems and neural computing.
Dr. Wang received the Phi Kappa Phi Student Recognition Award for his
work on fuzzy systems.

Authorized licensed use limited to: INDIAN INSTITUTE OF TECHNOLOGY KANPUR. Downloaded on February 01,2021 at 06:41:21 UTC from IEEE Xplore. Restrictions apply.