1414 IEEE TRANSACTIONS ON SYSTEMS, MAN, AND CYBERNETICS, VOL. 22, NO.

6, NOVEMBER/DECEMBER 1992

Generating Fuzzy Rules by Learning from Examples

Li-Xin Wang, Fellow, IEEE, and Jerry M. Mendei, Fellow, IEEE

Abstract— A general method is developed to generate fuzzy

rules from numerical data. This new method consists of five
steps: Step 1 divides the input and output spaces of the given
numerical data into fuzzy regions; Step 2 generates fuzzy rules
from the given data; Step 3 assigns a degree of each of the
generated rules for the purpose of resolving conflicts among
the generated rules; Step 4 creates a combined fuzzy rule base
based on both the generated rules and linguistic rules of human
experts; and, Step 5 determines a mapping from input space
to output space based on the combined fuzzy rule base using a
defuzzifying procedure. The mapping is proved to be capable of
approximating any real continuous function on a compact set to
arbitrary accuracy. Applications to truck backer-upper control
and time series prediction problems are presented. For the truck
control problem, the performance of this new method is compared
with a neural network controller and a pure limited-rule fuzzy
controller; the new method shows the best performance. For
the time series prediction problem, results are compared by
using the new method and a neural network predictor for the
Mackey-Glass chaotic time series.

I. Introduction Task : Design a control system to replace the human controller.

F ORMOST real-world control and signal processing prob¬

lems, the information concerning design, evaluation, re¬
alization, etc., can be classified into two kinds: numerical
Fig. 1. A practical problem: Design a control system to replace the human
controller.

information obtained from sensor measurements, and, linguis¬

tic information obtained from human experts. Most current that no mathematical model exists for it, or, the mathematical
intelligent control and signal processing approaches are heuris¬ model is strongly nonlinear so that a design method does not
tic in nature, i.e., they combine some standard control or signal exist. The task here is to design a control system to replace
processing methods with expert systems in an ad hoc way for a the human controller (see Fig. 1).
specific problem; simulations are then performed to show that To design such a control system, we first need to see
the new approaches work well for the specific problem. This what information is available. We assume that there is no
kind of approach has two weakpoints: 1) it is quite problem mathematical model, i.e., we consider a model-free design
dependent, i.e., a method may work well for one problem but problem. Since there already is a human controller who
is not suited for another problem; and, 2) there is no common is successfully controlling the system, there are two kinds
framework for modeling and representing different aspects of of information available to us: 1) the experience of the
control or signal processing strategies, which makes theoretical human controller; and, 2) sampled input-output (state-control)
analyses for these approaches very difficult. In this paper, we pairs that are recorded from successful control by the human
propose a general method for combining both numerical and controller. The experience of the human controller is usually

base.
—
linguistic information into a common framework a fuzzy rule expressed as some linguistic “IF-THEN” rules that state in
what situation(s) which action(s) should be taken. The sampled
Suppose we have the following problem: there is a complex input-output pairs are some numerical data that give the
control system in which a human controller is an essential part; specific values of the inputs and the corresponding successful
the environment facing this human controller is so complicated outputs.
Manuscript received March 3, 1991, revised April 20, 1992. Each of the two kinds of information alone is usually
L. X. Wang was with the Signal and Image Processing Institute, Department incomplete. Although the system is successfully controlled
of Electrical Engineering Systems, University of Southern California, Los by a human controller, some information will be lost when
Angeles, CA 90089-2564 and is now with the Department of Electrical
Engineering and Computer Science, University of California, Berkeley, CA human controllers express their experience by linguistic rules.
94720. Consequently, linguistic rules alone are usually not enough for
J. M. Mendel is with the Signal and Image Processing Institute, Department designing a successful control system. On the other hand, the
of Electrical Engineering Systems, University of Southern California, Los
Angeles, CA 90089-2564. information from sampled input-output data pairs is usually
IEEE Log Number 9202126. also not enough for a successful design, because the past
0018-9472/92S03.00 © 1992 IEEE

Authorized licensed use limited to: INDIAN INSTITUTE OF TECHNOLOGY KANPUR. Downloaded on March 24,2025 at 05:54:53 UTC from IEEE Xplore. Restrictions apply.
 WANG AND MENDEL: GENERATING FUZZY RULES BY LEARNING FROM EXAMPLES 1415

operations usually cannot cover all the situations the control pairs of (1), and use these fuzzy rules to determine a mapping
system will face. If expert linguistic rules and numerical data f : (x1,x2) -> y.
pairs are the only information we can get for such a control Our approach consists of the following five steps:
system design, the most interesting case for us is when the
—
combination of these two kinds of information is sufficient for Step 1 Divide the Input and Output Spaces into Fuzzy Regions
a successful design.
Assume that the domain intervals of ati,at2 and y are
Fuzzy control is an effective approach to utilizing linguistic
[if ,s+], [x2, x J] and [?/_,t/+], respectively, where “domain
rules [3], [4], whereas neural control is suited for using interval” of a variable means that most probably this variable
numerical data pairs (i.e., desired input-output pairs) [1], [4]. will lie in this interval
(the values of a variable are allowed to
Present fuzzy controllers only use linguistic rules, whereas lie outside its domain interval). Divide each domain interval
present neural controllers only use numerical data pairs. This into 2N 1 regions (TV can be different for different variables,
+
leads to the following question: “Is it possible to develop a and the lengths of these regions can be equal or unequal),
general approach that combines both kinds of information into denoted
by SN (Small TV), ••■.ST (Small 1), CE (Center).
a common framework, and uses both information, simultane¬
Bl (Big 1), • • • ,BN (Big TV), and assign each region a fuzzy
ously and cooperatively, to solve the control design or similar
membership function. Fig. 2 shows an example where the
problems?” In this paper, we develop such a general approach.
domain interval of Xi is divided into five regions (TV = 2), the
The key ideas of our new approach are to generate fuzzy
domain region of x2 is divided into seven regions (TV = 3),
rules from numerical data pairs, collect these fuzzy rules and and the domain interval of y is divided into five regions
the linguistic fuzzy rules into a common fuzzy rule base, and,
finally, design a control or signal processing system based on one =
(TV 2). The shape of each membership function is triangular;
vertex lies at the center of the region and has membership
this combined fuzzy rule base.
value unity; the other two vertices lie at the centers of the
In Section II, we propose a five step procedure for gener¬ two neighboring regions, respectively, and have membership
ating fuzzy rules from numerical data pairs and show how values
equal to zero. Of course, other divisions of the domain
to use these fuzzy rules to obtain a mapping from input
regions and other shapes of membership functions are possible.
space to output space. Step 1 divides the input and output
spaces into fuzzy regions; Step 2 generates fuzzy rules from
given desired input-output data pairs; Step 3 assigns a degree —
Step 2 Generate Fuzzy Rules from Given Data Pairs
to each generated rule; Step 4 forms the combined fuzzy First, determine the degrees of given and y^ in
rule base; and, Step 5 presents a defuzzifying procedure different regions. For example, in Fig. 2 has degree 0.8 in
for obtaining a mapping based on the combined fuzzy rule Bl, degree 0.2 in B2, and zero degrees in all other regions.
base. In Section III, we prove that the resulting mapping is Similarly, in Fig. 2 has degree 1 in CE, and zero degrees
capable of approximating any nonlinear continuous function in all other regions.
on a compact set to arbitrary accuracy using the well-known Second, assign a given ar^a:^ or to the region with
Stone-Weierstrass theorem in analysis [5]. In Section IV, we maximum degree. For example,
apply our new method to a truck backer-upper control problem be Bl, and
a^ in Fig. 2 is considered to

[1], [4]. We compare this new approach with pure neural and a:^ in Fig. is considered to be CE.
1
Finally, obtain one rule from one pair of desired
fuzzy approaches. The power of our new approach becomes
input-output data, e.g.,
apparent when it is used in the case where neither linguistic
fuzzy rules nor input-output pairs are sufficient to successfully (a;®, x®; y® ) => [z® (0.8 in Bl, max), at® (0.7 in SI, max);
control the truck to a desired position, but the combination t/^^O.OinCE, max)] => Rulel:
of both is sufficient. In Section V, we show that our new
method can be used for time-series prediction; and, we use it to IF Xi is Bl and x2 is SI, THEN y is CE;
predict the Mackey-Glass chaotic time series, and compare the
results with those obtained using a neural network predictor. (a:®, x®;y®) => [a:® (0.6 in Bl, max), ^(l in CE, max);
Conclusions are given in Section VI. t/®(0.7inBl, max)] => Rule 2:

II. Generating Fuzzy Rules from Numerical Data

IF an is Bl and x2 is CE, THEN y is Bl.
The rules generated in this way are “and” rules, i.e., rules in
Suppose we are given a set of desired input-output data which the conditions of the IF part must be met simultaneously
pairs: in order for the result of the THEN part to occur. For the
problems considered in this paper, i.e., generating fuzzy rules
(^i1’,^; vm), ••• CO from numerical data, only “and” rules are required since the
antecedents are different components of a single input vector.
where and x2 are inputs, and y is the output. This simple
two-input one-output case is chosen in order to emphasize
and to clarify the basic ideas of our new approach; extensions —
Step 3 Assign a Degree to Each Rule
to general multi-input multi-output cases are straightforward Since there are usually lots of data pairs, and each data pair
and will be discussed later in this section. The task here is generates one rule, it is highly probable that there will be some
to generate a set of fuzzy rules from the desired input-output conflicting rules, i.e., rules that have the same IF part but a

Authorized licensed use limited to: INDIAN INSTITUTE OF TECHNOLOGY KANPUR. Downloaded on March 24,2025 at 05:54:53 UTC from IEEE Xplore. Restrictions apply.
 1416 IEEE TRANSACTIONS ON SYSTEMS, MAN, AND CYBERNETICS, VOL. 22, NO. 6, NOVEMBER/DECEMBER 1992

Fig. 3. The form of a fuzzy rule base.

i.e., the degree of a rule is defined as the product of the

degrees of its components and the degree of the data pair that
generates this rule. This is important in practical applications,
because real numerical data have different reliabilities, e.g,
Fig. 2. Divisions of the input and output spaces into fuzzy regions and the
some real data can be very bad (“wild data”). For good data
corresponding membership functions, (a) (b) m(x2). (c) m(y). we assign higher degrees, and for bad data we assign lower
degrees. In this way, human experience about the data is used
in a common base as other information. If one emphasizes
different THEN part. One way to resolve this conflict is to
objectivity and does not want a human to judge the numerical
assign a degree to each rule generated from data pairs, and
data, our strategy still works by setting all the degrees of the
accept only the rule from a conflict group that has maximum
data pairs equal to unity.
degree. In this way not only is the conflict problem resolved,
but also the number of rules is greatly reduced.
We use the following product strategy to assign a degree to
—
Step 4 Create a Combined Fuzzy Rule Base
The form of a fuzzy rule base is illustrated in Fig. 3.
each rule: for the rule: “IF is A and is B, THEN y is
We fill the boxes of the base with fuzzy rules according
C,” the degree of this rule, denoted by P(Rule), is defined as
to the following strategy: a combined fuzzy rule base is
£>(Rule) = mA(x1)mB(x2)mc(,y'). (2) assigned rules from either those generated from numerical
data or linguistic rules (we assume that a linguistic rule
As examples, Rule 1 has degree also has a degree that is assigned by the human expert and
D(Rulel) = mBi(x1)msi(x2)mcE(y') reflects the expert’s belief of the importance of the rule); if
there is more than one rule in one box of the fuzzy rule
= 0.8 x 0.7 x 0.9 = 0.504 (3)
base, use the rule that has maximum degree. In this way,
(see Fig. 2) and Rule 2 has degree both numerical and linguistic information are codified into
£>(Rule2) = mBi(x1)mCE(x2)mB1(y) —
a common framework the combined fuzzy rule base. If a
linguistic rule is an “and” rule, it fills only one box of the
= 0.6 x 1 x 0.7 = 0.42. (4) fuzzy rule base; but, if a linguistic rule is an “or” rule (i.e., a
In practice, we often have some a priori information about rule for which the THEN part follows if any condition of the
the data pairs. For example, if we let an expert check given IF part is satisfied), it fills all the boxes in the rows or columns
data pairs, the expert may suggest that some are very useful corresponding to the regions of the IF part. For example,
and crucial, but others are very unlikely and may be caused suppose we have the linguistic rule: “IF xj is SI or x2 is
just by measurement errors. We can therefore assign a degree CE, THEN y is Bl” for the fuzzy rule base of Fig. 3; then
to each data pair that represents our belief of its usefulness. In we fill the seven boxes in the column of SI and the five boxes
this sense, the data pairs constitute a fuzzy set, i.e., the fuzzy in the row of CE with B2. The degrees of all the 52’s in
set is defined as the useful measurements; a data pair belongs these boxes equal the degree of this “or” rule.
to this set to a degree assigned by a human expert.
Suppose the data pair (a:/ , a^;
t/1)) has degree mW, then
—
Step 5 Determine a Mapping Based on
the Combined Fuzzy Rule Base
we redefine the degree of Rule 1 as
We use the following defuzzification strategy to determine
D(Rulel) = (5) the output control y for given inputs (^1,3:2): first, for given

Authorized licensed use limited to: INDIAN INSTITUTE OF TECHNOLOGY KANPUR. Downloaded on March 24,2025 at 05:54:53 UTC from IEEE Xplore. Restrictions apply.
 WANG AND MENDEL: GENERATING FUZZY RULES BY LEARNING FROM EXAMPLES 1417

inputs (11,2:2), we combine the antecedents of the ith fuzzy that when new inputs are presented the mapping continues
rule using product operation to determine the degree, of to give desired or successful outputs. Hence, our new method
the output control corresponding to (a:i , 12), i.e., can be viewed as a very general model-free trainable fuzzy
system for a wide range of control and signal processing
m'o, = (6)
problems, where: “Model-Free” means no mathematical model
is required for the problem; “Trainable” means the system
where O' denotes the output region of Rule i, and TJ
denotes
learns from “examples” and expert rules, and can adaptively
the input region of Rule i for the j th component, e.g., Rule
1 gives change the mapping when new “examples” and expert rules
are available; and, “Fuzzy” denotes the fuzziness introduced
(7) into the system by linguistic fuzzy rules, fuzziness of data, etc.
then, we use the following centroid defuzzification formula to
determine the output III. Fuzzy System as a Universal Approximator
A' The five step procedure of the last section generates a fuzzy
system, i.e., a mapping from input space to output space.
y=
^—
^2mo‘
(8) Specifically, this mapping is represented by (6) and (8) for
the two-input one-output case. Using simplified notations, we
rewrite (6) and (8), for the general n-input one-output case, as
Z=1

where y‘ denotes the center value of region O' (the center m’ = (10)
of a fuzzy region is defined as the point that has the smallest
absolute value among all the points at which the membership
function for this region has membership value equal to one), K
A
and K is the number of fuzzy rules in the combined fuzzy
rule base.
From Steps 1 to 5 we see that our new method is simple = — (11)
and straightforward in the sense that it is a one-pass build¬
up procedure that does not require time-consuming training; 52
i=l
52
i=l
hence, it has the same advantage that the fuzzy approach has
over the neural approach, namely, it is simple and quick to where is the membership function of the ith rule for the
mJ
construct. jth component of the input vector, and fa is the center value
This five step procedure can easily be extended to general of the output region of the ith rule. We will prove that this
multi-input multi-output cases. Steps 1 to 4 are independent of generated fuzzy system, i.e., (11), is a universal approximator
how many inputs and how many outputs there are. In Step 5, from a compact set Q C Rn to R. i.e., it can approximate any
we only need to replace in (6) with mloi , where j denotes real continuous function defined on Q to any accuracy, where
the jth component of the output vector (O’ is the region of the compact set Q is defined as
Rule i for the jth output component; is the same for all
Q = [ai , x [a2, b2] x x [an,6n] C Rn. (12)
j), and change (8) to
K For notational convenience, we represent Rule ? (i =
1,2, •,7T) in the fuzzy rule base as: “IF Xi is RG\,x2 is
= (9) RG^-'^Xn is RGfa THEN y is RG^,” where RG^j =
* 1,2, denotes the region for the j’th input antecedent
52
i=l
mO‘J of Rule i (e.g., it can be S2.CE, Bl, etc.), and RG'O denotes
the outupt region of Rule i.
where fa denotes the center of region OJ. Let F be the family of functions of the form of (11) on
If we view this five step procedure as a block, then the the compact set Q. There are three factors that determine a
inputs to this block are “examples” (desired input-output data member of F: 1) the definition of fuzzy regions, i.e., how to
pairs) and expert rules (linguistic IF-THEN statements), and define and divide the domain intervals; 2) the specific form
the output is a mapping from input space to output space. of membership functions mJ: and, 3) the specific statements
For control problems, the input space is the state of the plant of fuzzy rules in the fuzzy rule base. By fixing fuzzy regions,
to be controlled, and the output space is the control applied membership functions, and fuzzy rules, we obtain an element
to the plant. For time-series prediction problems, the input of F. If /1 and f2 are different elements of F, then at least one
and output spaces are subsequences of the time series such of the three factors for fa and f2 must be different. In order
that the input subsequence precedes the output subsequence to analyses the family F, we make the following assumptions
(details are given in Section V). Our new method essentially for these three factors:
“learns” from the “examples” and expert rules to obtain a AS.l: The fuzzy regions for the input and output spaces
mapping that, hopefully, has the “generalization” property can be arbitrarily defined.

Authorized licensed use limited to: INDIAN INSTITUTE OF TECHNOLOGY KANPUR. Downloaded on March 24,2025 at 05:54:53 UTC from IEEE Xplore. Restrictions apply.
 1418 IEEE TRANSACTIONS ON SYSTEMS, MAN, AND CYBERNETICS, VOL. 22, NO. 6, NOVEMBER/DECEMBER 1992

AS.2: The membership functions mJ can be any continuous uses ASA and the definition of “active rule” that will be given
functions from [aj,6j] to [0, 1] for j = 1,2, ,n (i.e., for later in this section; hence, we suggest the reader read the rest
inputs) and from (—00,00) to [0, 1] for j = 0 (i.e., for of this section before going to the proof of Theorem 1. The
output); however, mJ must satisfy the following constraint: rest of this section will not use the result of Theorem 1.)
mj(at;) 0 for Xj e RGj.i = 1, 2, • • • , K, j = 0,1, • , n, Theorem 1 is an existence theorem showing that there
with x0 = y. This constraint means that the membership value exists a way of defining fuzzy regions, a way of choosing
of an antecedent for a rule cannot equal zero if the actual input membership functions, and a way of assigning fuzzy rules
value of this antecedent falls into the required region of the to the boxes of the fuzzy rule base, such that the resulting
rule. mapping, (11), approximates an arbitrary nonlinear continuous
AS.3: Any rule can be assigned to any box of the fuzzy mapping from Q to R to any accuracy. This Theorem is
rule base. similar to the results of [6] and [7], which showed that a three-
These assumptions are usually satisfied in practice. Specifi¬ layer feedforward neural network is a universal approximator
cally, we have total freedom in defining fuzzy regions; we can provided that there are sufficiently large numbers of hidden-
choose any membership functions subject to the constraint of layer neurons. Theorem 1 provides the theoretical basis for
AS.2; and, we can assign any rule to any box of the fuzzy successful applications of our new method to many different
rule base. practical problems.
To analyses the properties of the function family F, we In many applications of fuzzy systems (e.g., [3], [4]), the
must first establish that the mapping defined by (11) is well- membership functions are triangular. We now study some
defined, i.e., for any input x Q, (11) will generate an properties of the fuzzy systems that use the specific form of
output f(x) E R. The following two lemmas give sufficient membership functions that are defined as follows.
conditions for (11) to be well-defined. AS.4: The membership function for any intermediate fuzzy
Lemma 1: If all the membership functions mJ are nonzero, region (i.e., not the smallest or the largest region) is a triangle
and there is at least one rule in the fuzzy rule base, then the whose vertices are at (x.m) = (a:_i, 0), (xq, 1), and (ri.O),
mapping defined by (11) from Q to R is well-defined. where the z-axis denotes a coordinate of the input or output
Proofs of lemmas and theorems are given in Appendix I. space, the m-axis denotes the corresponding membership
Lemma 2: If every box in the fuzzy rule base has a rule value, x0 denotes the center of the region, and 1 (rr 1 ) denotes
associated with it, i.e., there are no empty boxes in the fuzzy the center of the left (right) region. See Fig. 11 for an example.
rule base, then the mapping defined by (11) from Q to R is The membership functions for the smallest and largest regions
well-defined under AS.2. are determined by the way shown in Fig. 11.
In practice, the input space is usually high dimensional,
whereas the given successful data pairs and expert rules are
often quite limited; as a result, many boxes of the fuzzy rule
base may be empty. However, it is possible to fill up these
N = ri x
—
If every box of the fuzzy rule base has a rule and [aj, bj] is
divided into rj fuzzy regions (j 1,2, • • ,n), then there are
x • • x rn rules in the fuzzy rule base. N can
be a huge number if the rjs and n are large. However, under
empty boxes based on the limited given rules using the method the situation of ASA, there are only a small fraction of these
of Section II. Specifically, Steps 1-4 are first used to generate rules that are really used in (11) for any given x & Q.
a fuzzy rule base based on the limited data pairs and linguistic Definition: The ith fuzzy rule in the fuzzy rule base is
rules; then, the output for some typical input for which the active for x & Q if mlj[xj) / 0 for all j = 1,2,
box in the fuzzy rule base is empty can be determined based Referring to (11), we see that a rule is active means that it
on the limited fuzzy rule base; finally, the range in which the will be used in (11).
output has the maximum degree is assigned to the empty box Lemma 3: Under ASA, the following is true:
as a new rule. This can be an iterative procedure, i.e., when
1) There are at most 2” active rules for any x 6 Q.
a new rule is generated, this new rule and the existing rules
2) If r components of x e Q are at the centers of some
are combined into a fuzzy rule base that is used to generate
fuzy regions (r 0, 1, 2, • • • , n), there are at most 2n~r
=
the next new rule. We can start the procedure from the empty
active rules at the x (the center of a fuzzy region is
boxes that are the nearest neighbors of the full boxes; in this
defined in Step 5 of Section II).
way, the fuzzy rule base expands from existing rules until all
3) If r components of x £ Q are at the centers of some
the boxes are filled up. This procedure always works if we
fuzzy regions, and if q components of the x are smaller
choose the nonzero regions of the membership functions to be
(greater) than the center values of the smallest (largest)
large enough such that the values of the membership functions
regions of the corresponding components, then there are
will not be zero for some points of their nearest neighbors. We
at most 2n~r~i active rules at the x.
will not study this procedure in detail in this paper; we gave
the basic ideas of the procedure in order to show that the Lemma 3 is useful in practice. Although we may need a
conditions of Lemma 2 can be satisfied. huge memory to store the fuzzy rule base, when we use the
Now we state the main result of this section. fuzzy rule base for a given input x 6 Q, only a relatively
Theorem 1: If the mapping defined by (11) is well-defined, small number of rules are used. In practice, we may store the
and if AS.1-AS.3 are true, then the mapping defined by (11) is fuzzy rule base in a cheap external memory; when we have an
capable of approximating any real continuous function over the input, we only take the active rules from the fuzzy rule base
compact set Q to arbitrary accuracy. (The proof of Theorem 1 into the host computer.

Authorized licensed use limited to: INDIAN INSTITUTE OF TECHNOLOGY KANPUR. Downloaded on March 24,2025 at 05:54:53 UTC from IEEE Xplore. Restrictions apply.
 WANG AND MENDEL: GENERATING FUZZY RULES BY LEARNING FROM EXAMPLES 1419

Fig. 4. Diagram of simulated truck and loading zone.

IV. Application to Truck Backer-Upper Control input-output pairs to simulate neural and numerical-fuzzy
Backing a truck to a loading dock is a difficult exercise. It controllers, and compare their final control performance.
is a nonlinear control problem for which no traditional control Statement of the Truck Backer-Upper Control Problem: The
simulated truck and loading zone are shown in Fig. 4 [1],
system design methods exist. In [1], Nguyen and Widrow
develop a neural network controller for the truck backer-upper [4]. The truck corresponds to the cab part of the neural truck
in the Nguyen-Widrow [1] neural truck backer-upper system.
problem; and, in [4], Kong and Kosko propose a fuzzy control
The truck position is exactly determined by the three state
strategy for the same problem. The neural network controller
variables <f>, x, and y, where if is the angle of the truck
[1] only uses numerical data, and cannot utilize linguistic rules
with the horizontal as shown in Fig. 4. Control to the truck
determined from expert drivers; on the other hand, the fuzzy
is the angle 6. Only backing up is considered. The truck
controller of [4] only uses linguistic rules, and cannot utilize
moves backward by a fixed unit distance every stage. For
sampled data. Since the truck backer-upper control problem
simplicity, we assume enough clearance between the truck and
is a good example of the control system design problem
the loading dock such that y does not have to be considered
discussed in the Introduction of this paper (i.e., replace a
as an input. The task here is to design a control system, whose
human controller by a machine), it is interesting to apply the
inputs are <4 [-90°, 270°] and x e [0,20], and whose
approach developed in Section II to this problem. In order to
distinguish these methods, we call the method of [4] the “fuzzy
output is 9 [-40°,40°], such that the final states will be
approach,” the method of [1] the “neural approach,” and our = (10,90°).
Generating Desired Input-Output Pairs{x,4>-, 9): Vie do
new method the “numerical-fuzzy approach.”
this by trial and error: at every stage (given f and a;) starting
The results of [4] demonstrated superior performance of the
from an initial state, we determined a control 9 based on
fuzzy controller over the neural controller; however, the fuzzy common sense (i.e., our own experience of how to control
and neural controllers use different information to construct the steering angle in the situation); after some trials, we chose
the control strategies. It is possible that the fuzzy rules used in the desired input-output pairs corresponding to the smoothest
[4] to construct the controller are more complete and contain successful trajectory.
more information than the numerical data used to construct the The following 14 initial states were used to generate desired
neural controller; hence, the comparison between the fuzzy and input-output pairs: (xq>0o) = (L ®)> O’ 90), (1, 270); (7,
neural controllers, from a final control performance point of 0), (7, 90), (7, 180), (7, 270); (13, 0), (13, 90), (13, 180),
view, is somewhat unfair. If the linguistic fuzzy rules were (13, 270); (19, 90), (19, 180), (19, 270). Since we performed
incomplete, whereas the numerical information contained lots simulations, we needed to know the dynamics of the truck
of very good data pairs, it is highly possible that the neural backer-upper procedure. We used the following approximate
controller would outperform the fuzzy controller. kinematics (see [8] for details):
Our new numerical-fuzzy approach provides a fair basis
for comparing fuzzy and neural controllers (the numerical- x(t + 1) = x(t) -I- cos[^(t) + 0(f)] + sin[0(t)] sin[<4(t)] (13)
fuzzy approach can be viewed as a fuzzy approach in the
sense that it differs from the pure fuzzy approach only in the
way it obtains fuzzy rules). We can provide the same desired
input-output pairs to both the neural and numerical-fuzzy
y(t + 1) = y(t) + sin[0(t) + 0(f)] - sin[0(0] cos[0(t)] (14)
approaches; consequently, we can compare the final control
performances of both controllers fairly since they both use the
same information. W + 1) = ^(0 - sin’1 (15)
Example 1: In this example, we use the same set of desired

Authorized licensed use limited to: INDIAN INSTITUTE OF TECHNOLOGY KANPUR. Downloaded on March 24,2025 at 05:54:53 UTC from IEEE Xplore. Restrictions apply.
 1420 IEEE TRANSACTIONS ON SYSTEMS, MAN, AND CYBERNETICS, VOL. 22, NO. 6, NOVEMBER/DECEMBER 1992

TABLE I
Desired Trajectory Starting from (t0oo) = (1-0°)
t X e°
0 1.00 0.00 -19.00
1 1.95 9.37 -17.95
2 2.88 18423 -16.90
3 3.79 26.59 -15.85
4 4.65 34.44 -14.80

5 5.45 41.78 -13.75

6 6.18 48.60 -12.70

7 7.48 54.91 -1135
8 7.99 60.71 -10.60
9 8.72 65.99 -9.55
10 9.01 70.75 -8.50

11 938 74.98 -7.45

12 9.46 78.70 -6.40
13 9.59 81.90 -5.34
14 9.72 84.57 -430 Fig. 5. Truck trajectories using the neural controller and the numerical-fuzzy
controller.
15 9.81 86.72 -335
16 9.88 8834 -230
In the simulations, we normalized [— 40°,40°] into [—1,1]-
17 9.91 89.44 0.00 Similarly, the inputs to the neurons were also normalized into
H.i]-
18 Our neural network controller is different from the
19 Nguyen-Widrow neural controller [1]. First, we have only
one neural network that does the same work as the Truck
20 Controller of the Nguyen-Widrow network; the truck emulator
of the Nguyen-Widrow network is not needed in our task.
Second, and more fundamentally, we train our neural network
where b is the length of the truck. We assumed & = 4 in using desired input-output (state-control) pairs, which are
the simulations of this paper. Equations (13)-(15) were used obtained from the past successful control history of the truck,
to obtain the next state when the present state and control are whereas Nguyen and Widrow [1] connect their neural network
given. Since y is not considered a state, only (13) and (15) were stage by stage and train the concatenated neural networks by
used in the simulations. We wrote (14) here for the purpose of back-propagating the error at the final state through this long
showing the complete dynamics of the truck. Observe, from network chain (the detailed algorithm is different from the
(13)-(15), that even this simplified dynamic model of the truck standard error back-propagation algorithm in order to meet the
is nonlinear. The 14 sequences of desired (x,^>;0) pairs are constraint that the neural networks at each stage perform the
given in [8]; we only include one such sequence in this paper same transformation; for details see [1]). Hence, the training of
(Table I). our neural network is simpler than that of the Nguyen-Widrow
Neural Control and Simulation Results: We used a two-
network. Of course, we need to know some successful control
input single-output three-layer back-propagation neural net¬
trajectories (state-control pairs) starting from some typical
work [9, 10] for our control task. Twenty hidden neurons
initial states; this is not required in the Nguyen-Widrow neural
were used, and a sigmoid nonlinear function was used for network controller.
each neuron. The output of the third-layer neuron represents
We trained the neural network using the standard error back-
the steering angle 0 according to a uniform mapping from
propagation algorithm [9], [10] for the generated 14 sequences
[0, 1] to [— 40°,40°], i.e., if the neuron output is g(t), the of desired (x, 4>~, 9) pairs. We used the converged network to
corresponding output 9(t) is
control the truck whose dynamics are approximately given by
0(f) = 80p(t) - 40. (16) (13)-(15). Three arbitrarily chosen initial states, =

Authorized licensed use limited to: INDIAN INSTITUTE OF TECHNOLOGY KANPUR. Downloaded on March 24,2025 at 05:54:53 UTC from IEEE Xplore. Restrictions apply.
 WANG AND MENDEL: GENERATING FUZZY RULES BY LEARNING FROM EXAMPLES 1421

m(x)

m(0)

e
-40 -33 -20 -14 -7-4 0 4 7 14 20 33 40

Fig. 6. Fuzzy membership functions for the truck backer-upper control problem.

(3, —30), (10, 220), and (13, 30), were used to test the neural
controller. The truck trajectories from the three initial states are
shown in Fig. 5. We see that the neural controller successfully
controls the truck to the desired position starting from all three
initial states.
Numerical-Fuzzy Control and Simulation Results: We used
the five-step procedure of Section II to determine the control
law / : (x, 0) — 0, based on the 14 generated sequences of
successful (at, </>;#) pairs. For this specific problem, we used
membership functions shown in Fig. 6, which are similar to
those used in [4] for fuzzy control of the problem based only
on linguistic rules. The fuzzy rules generated from the desired
input-output pairs and their corresponding degrees are given
in [8]; we show only the generated rules for the data pairs
of Table I (in this paper) in Table II. The final fuzzy rule Fig. 7. The final fuzzy rule base generated from the numerical data for the
base is shown in Fig. 7 (this is the result of Step 4 of our truck backer-upper control problem.
method in Section II; here we assume that no linguistic rules
are available). We see from Fig. 7 that there are no generated (3, -30), (10, 220), and (13, 30), which are the same states
rules for some ranges of x and This shows that the desired used in the simulations of the neural controller. The final
trajectories from the 14 initial states do not cover all the trajectories of the truck have no visible difference from Fig.
possible cases; however, we will see that the rules in Fig. 5; hence, Fig. 5 also shows the track trajectories using the
7 are sufficient for controlling the truck to the desired state numerical-fuzzy controller.
starting from some given initial states. We simulated the neural and numerical-fuzzy controllers
Finally, Step 5 of our numerical-fuzzy method was used for other initial truck positions, and observed that the truck
to control the truck from the three initial states, (zo,^) = trajectories using these two controllers were also almost the

Authorized licensed use limited to: INDIAN INSTITUTE OF TECHNOLOGY KANPUR. Downloaded on March 24,2025 at 05:54:53 UTC from IEEE Xplore. Restrictions apply.
 1422 IEEE TRANSACTIONS ON SYSTEMS, MAN, AND CYBERNETICS, VOL. 22, NO. 6, NOVEMBER/DECEMBER 1992

TABLE II
Fuzzy Rules Generated from the Desired Input-Output
Pairs of Table I and the Degrees of These Rules

Fuzzy IF THEN
rules Degree
for t= x is 4 is 0 is

0 S2 S2 S2 1.00

1 S2 S2 S2 0.92
2 S2 S2 S2 0.35
3 S2 S2 S2 0.12
4 S2 S2 S2 0.07
5 S1 S2 S1 0.08
6 S1 S1 S1 0.18

7 S1 S1 S1 0.52
8 S1 si S1 0.56
9 S1 S1 SI 0.60
10 CE S1 S1 0.35
11 CE S1 S1 0.21
12 CE S1 CE 0.16
13 CE CE CE 0.32
only.
14 CE CE CE 0.45
15 CE CE CE 0.54 there are no rules in the three center boxes outlined by the
16 CE CE CE 0.88 heavy lines. The fuzzy rule base of linguistic rules for the
ending part was chosen to have only three rules that are the
17 CE CE CE 0.92 same as the three center rules of Fig. 7.
18 We simulated the following three cases in which we used
the: 1) fuzzy rule base generated from only the truncated data
19 pairs; 2) fuzzy rule base of selected linguistic rules; and, 3)
20 fuzzy rule base which combined the fuzzy rule bases of 1) and
2). We see that for Case 3 the fuzzy rule base is the same as
in Fig. 7; hence, the truck trajectories for this case must be the
same. This is not surprising because both controllers used the same as those using the fuzzy rule base of Fig. 7. For each of
same information to construct their control laws. the cases, we simulated the system starting from the following
Example 2: In this example we consider the situation where three initial states: ($0,^0) = (3,-30), (10, 220), and (13,
neither linguistic fuzzy rules alone nor desired input-output 30). The resulting trajectories for cases 1), 2), and 3) for the
pairs alone are sufficient to successfully control the truck to three initial states are shown in Figs. 8, 9 and 5, respectively.
the desired position, i.e., neither the usual fuzzy controller We see very clearly from these figures that, for cases (1)
with limited fuzzy rules nor the usual neural controller can and (2) the truck cannot be controlled to the desired position,
control the truck to the desired position, but a combination whereas for case (3) we successfully controlled the truck to
of linguistic fuzzy rules and fuzzy rules generated from the the desired position.
desired input-output data pairs is sufficient to successfully
control the truck to the desired position. V. Application to Time-Series Prediction
We consider the case where the beginning part of the Time-series prediction is a very important practical problem
information comes from desired input-output pairs whereas [2]. Applications of time-series prediction can be found in
the ending part of the information comes from linguistic rules. the areas of economic and business planning, inventory and
To do this we used only the first three pairs of each of the production control, weather forecasting, signal processing,
14 desired sequences, and generated fuzzy rules based only control, and lots of other fields. Let z(k)(k = 1,2,3, be
on these truncated pairs. The fuzzy rule base generated from a time series. The problem of time-series prediction can be
these truncated data pairs is the same as Fig. 7 except that — —
formulated as: given z(k m + 1), z(k m + 2), • • • , z(k),

Authorized licensed use limited to: INDIAN INSTITUTE OF TECHNOLOGY KANPUR. Downloaded on March 24,2025 at 05:54:53 UTC from IEEE Xplore. Restrictions apply.
 WANG AND MENDEL: GENERATING FUZZY RULES BY LEARNING FROM EXAMPLES 1423

Fig. 10. A section of the Mackey-Glass chaotic time series.

Fig. 11. The first choice of membership functions for the chaotic time series
prediction problem.

Example 3: Now we apply our numerical-fuzzy approach

to predict the Mackey-Glass chaotic time-series [11], Chaotic
time series are generated from deterministic nonlinear sys¬
tems and are sufficiently complicated that they appear to be
“random” time series; however, because there are underlying
determine z^k + l), where m and I are fixed positive integers; deterministic maps that generate the series, chaotic time series
i.e., determine a mapping from [a(fc - m + l^z^k - m + are not random time series. In [11], feedforward neural net¬
2) , • , z(k)] e Rm to [z(fc + 0] e R- works were used for chaotic time-series prediction, and were
A feedforward neural network can also be used for this prob¬ compared with conventional approaches, like linear predictive
lem [11]. For example, we can use a three-layer feedforward method, Gabor polynomial method, etc. The results showed
neural network, which has m input neurons and one output that the neural network approach gave the best prediction,
—
neuron, to represent the mapping from [z(k m + 1), z(k —
m + 2), • • • , z(k)] to [z(/c + /)]. The network is trained for the
and the accuracy obtained using the neural network approach
was orders of magnitude higher than that obtained using the
known z(k)’s, and then the converged network is used for the conventional approaches. Here we use our numerical-fuzzy
prediction. Specifically, assume that 2(1), z(2), ,z(M) are approach to the same Mackey-Glass chaotic time series in
—
given; then we form M m desired input-output pairs: [11], and compare the results obtained with those obtained
[z(M
[z(M
— m), • , z(M - 1); z(M)]
- m - I),- • ,z(M - 2)-,z(M - 1)]
using the neural network approach.
The Mackey-Glass chaotic time series is generated from the
following delay differential equation:
dx(t) _ -
0.2x(t t)
O.ltrfi). (18)
[.z(l), • • • z(m + 1)]. (17) dt 1 + s10(t r)—
We train the neural network to match these M m pattern—
pairs using the error back-propagation algorithm [9], [10].
When r
t
> 17, (18) shows chaotic behavior. Higher values of
yield higher dimensional chaos. In our simulation, we chose
Our numerical-fuzzy method in Section II can also be used the series with r = 30. Fig. 10 shows 1000 points of this
for this time series prediction problem. Similar to the neural chaotic series that we used to test both the numerical-fuzzy
network approach, we assume that z(l),z(2),- ,z(M) are and neural approaches.
given, and we form the Af - m desired input-output pattern We chose m = 9 and I = 1 in our simulation, i.e., nine point
pairs in (17). Steps 1-4 of our numerical-fuzzy approach are values in the series were used to predict the value of the next
used to generate a fuzzy rule base based on the pattern pairs time point. The membership functions for any point are shown
(17); then this fuzzy rule base is used to forecast z(M + p) in Fig. 11 for the numerical-fuzzy predictor (later, we will use
for p = 1, 2, • • using the defuzzifying procedure of Step 5 of other membership functions). 40 hidden-layer neurons were
our numerical-fuzzy method, where the inputs to the network used for the neural network predictor. The first 700 points
— —
are z(M + p m), z(M + p m + 1), • • • ,z(M + p - 1). of the series were used as training data, and the final 300

Authorized licensed use limited to: INDIAN INSTITUTE OF TECHNOLOGY KANPUR. Downloaded on March 24,2025 at 05:54:53 UTC from IEEE Xplore. Restrictions apply.
 1424 IEEE TRANSACTIONS ON SYSTEMS, MAN, AND CYBERNETICS, VOL. 22, NO. 6, NOVEMBER/DECEMBER 1992

Fig. 12. Prediction of the chaotic time series from t(701) to t(1000) using Fig. 15. Prediction of the chaotic time series from x(701) to x(1000) using
the numerical-fuzzy predictor when 200 training data (from j*(501) to t(700) the neural predictor when 700 training data (from z(l) to ;r(700)) are used.
are used.

Fig. 16. Prediction of the chaotic time series from x(701) to .r( 1000) using
Fig. 13. Prediction of the chaotic time series from z(701) to x(1000) using the updating fuzzy rule base procedure.
the neural predictor when 200 training data (from a*(501) to .r(700)) are used.

this “adaptive” procedure we use all the available information

to predict the next value of the series. We simulated this
adaptive procedure for the chaotic series of Fig. 10: we started
with the fuzzy rule base generated by the data x(l) to a:(700),
made a prediction of z(701), then used the true value of
a;(701) to update the fuzzy rule base, and this updated fuzzy
rule base was then used to predict r(702). This adaptive
procedure continued until «(1000). Its results are shown in
Fig. 16. Comparing Figs. 16 and 14 we see that we obtain
only a slightly improved prediction.
Fig. 14. Prediction of the chaotic time series from t(701) to z(1000) using Finally, we show that prediction can be greatly improved
the numerical-fuzzy predictor when 700 training data (from z(l) to j-(700)) by dividing the “domain interval” into finer regions. We
are used. performed two simulations: one with the membership function
shown in Fig. 17, and the other with the membership function
points were used as test data (for additional cases, see [8]). shown in Fig. 18. We used the adaptive fuzzy rule base
We simulated two cases: 1) 200 training data (from 501 to procedure for both simulations. The results are shown in Figs.
700) were used to construct the fuzzy rule base and to train the 19 and 20, for the membership functions of Figs. 17 and
neural network; and, 2) 700 training data (from 1 to 700) were 18, respectively. Comparing Figs. 16, 19 and 20 we see very
used. Figs. 12 and 13 show the results of the numerical-fuzzy clearly that we obtain better and better results as the “domain
and neural predictors respectively for case 1); and, Figs. 14 interval” is divided finer and finer. Fig. 20 shows that we
and 15 show similar results for case 2). As in [11], the “past” obtained an almost perfect prediction when we divided the
data needed to perform prediction is obtained from observing “domain interval” into 29 regions. Of course, the price paid
the actual time series; thus, one makes a prediction and uses for doing this is a larger fuzzy rule base.
the actual values to make the next prediction. We see from
Figs. 12 to 15 that our new numerical-fuzzy predictor gave VI. Conclusion
about the same results as the neural network predictor. In this paper, we developed a general method to generate
One advantage of the numerical-fuzzy approach is that it fuzzy rules from numerical data. This method can be used
is very easy to modify the fuzzy rule base as new data as a general way to combine both numerical and linguistic
become available. Specifically, when a new data pair becomes
available, we create a rule for this data pair and add the new
—
information into a common framework a fuzzy rule base.
This fuzzy rule base consists of two kinds of fuzzy rules: some
rule to the fuzzy rule base; then, the updated (i.e., adapted) obtained from experts, and others generated from measured
fuzzy rule base is used to predict the future values. By using numerical data using the method of this paper. We proved that

Authorized licensed use limited to: INDIAN INSTITUTE OF TECHNOLOGY KANPUR. Downloaded on March 24,2025 at 05:54:53 UTC from IEEE Xplore. Restrictions apply.
 WANG AND MENDEL: GENERATING FUZZY RULES BY LEARNING FROM EXAMPLES 1425

from that of the pure neural network controller; and, 2) in the

case where neither numerical data nor linguistic rules contain
enough information, both the pure neural and pure fuzzy
methods failed to control the truck to the desired position, but
our new method succeeded. We also applied our new method
to a chaotic time-series prediction problem, and the results
showed that our new method worked quite well.
The main features and advantages of the new method
developed in this paper are: 1) it provides us with a gen¬
eral method to combine measured numerical information and
Fig. 17. The second choice of membership functions for the chaotic time
series prediction problem.
human linguistic information into a common framework a —
combined fuzzy rule base; this could be viewed as a first step
to develop some theoretically analyzable control algorithms
that use both numerical and linguistic information; 2) it is
a simple and straightforward one-pass build-up procedure;
hence, no time-consuming iterative training is required, so
that it requires much less construction time than a comparable
neural network; 3) there is lots of freedom in choosing the
membership functions; this provides us with lots of flexibilities
to design systems according to different requirements; and, 4)
it can perform successful control for some cases where neither
a pure neural network control nor a pure fuzzy control can.
Fig. 18. The third choice of membership functions for the chaotic time series
prediction problem.
Appendix I
Proof of Lemma 1: Since 0 < m'^Xj) 1, it is suf¬
ficient to prove that for any x G Q there exist some i such
that 0. Under the condition of this lemma,
there exists at least one i such that IIi<y<n[m“ (atj)] / 0 for
any x G Q, hence (11) is well-defined. Q.E.D.
Proof of Lemma 2: Since every box in the fuzzy rule
base has a rule, for any x G Q there must be a rule, say Rule i,
such that Xj G BG' for j = 1, 2, • • , n. By AS.2, mfaj) / 0
for all j = 1, 2, • • ;n, hence ni<y<n[mj(a:)j)] / 0, i.e., (11)
is well-defined. Q.E.D.
Fig. 19. Prediction of the chaotic time series from o-(701) to t( 1000) using In order to prove Theorem 1 we need some definitions.
the updating fuzzy rule base procedure with the membership functions of Fig.
18. A family F of real functions defined on a set E is an
algebra if F is closed under addition, multiplication, and scalar
multiplication. The family F separates points on E if for every
x,y G E,x / y, there exists a function f G F such that
f(x) f(y). The family F vanishes at no point of E if for
each x G E there exists f G F such that f(x) 0 0. Our proof
of Theorem 1 is based on the Stone-Weierstrass Theorem [5],
which we state here for convenience of the reader.
Stone-Weierstrass Theorem: Let F be an algebra of real
continuous functions on a compact set K. If F separates points
on K and if F vanishes at no point on K, then the uniform
closure B of F consists of all real continuous functions on K.
Fig. 20. Prediction of the chaotic time series from ;r(701) to ;r(1000) using The uniform closure B of F is the union of F and its limit
the updating fuzzy rule base procedure with the membership functions of Fig. points; hence, if B consists of all real continuous functions on
19. K, then F is capable of approximating any real continuous
function on K to arbitrary accuracy.
the generated fuzzy system is capable of approximating any Proof of Theorem 1 : Let F be the family of well-defined
nonlinear continuous function on a compact set to arbitrary functions of the form of (11) on the compact set Q under
accuracy. We applied our new method to a truck backer-upper AS.l, AS.2, and AS.3. If we prove that F is an algebra of
control problem [1], [4], and observed that: 1) for the same real continuous functions, F separates points on 0, and F
training set (i.e., the same given input-output pairs), the final vanishes at no point of Q, then the Stone-Weierstrass Theorem
control performance of our new method is indistinguishable guarantees the conclusion of Theorem 1.

Authorized licensed use limited to: INDIAN INSTITUTE OF TECHNOLOGY KANPUR. Downloaded on March 24,2025 at 05:54:53 UTC from IEEE Xplore. Restrictions apply.
 1426 IEEE TRANSACTIONS ON SYSTEMS, MAN, AND CYBERNETICS, VOL. 22, NO. 6, NOVEMBER/DECEMBER 1992

By AS.2, the m^XjYs are assumed to be real continuous that has the smallest absolute value among all the points at
functions; hence, F is a family of real continuous functions. which the membership function for this region has membership
Let /i , /2 F, so that we can write them as value equal to one; see Section II, Step 5). Then, we choose
AT the membership functions for the input space Q to be of the
52^n1^<n[mlj(a:J)] specific triangular form defined by AS.4 of Section III. By
such a choice of fuzzy regions and membership functions, we
(19) have m'^Xj) = ml(zj) = 1 for each active Rule i at x, and
52
2= 1
each active Rule I at z, and all j = 1, 2, • •• , n (the definition
of active rule is given in Section III); additionally, there is one
K2 and only one active rule for x and one and only one active
rule for z, because (11) is well-defined (which guarantees that
= — (20) there is at least one active rule for x and at least one active
rule for z), and since only the membership functions for the
regions with centers at the components of x or z are nonzero
2 =1 at x or z, whereas all other membership functions are zero at
Thus we have (21), shown at the bottom of the next page. Now x and z (which guarantees that there is at most one active rule
Define mlj1(xj)m2j2(xj) as a new membership function of for x and at most one active rule for z). Since x z, there
Xj, say rnF'^^Xj), and define t/1'1
+ t/2l as the output center must be at least one j such that Xj / z, , hence, the only active
of a new rule, say t/11’'2: then, (21) is of the form of (11); rule for x and the only active rule for z are at two different
hence, /1 + ^6 F Similarly, fi(x)f2(x) can be written as boxes of the fuzzy rule base. Since we are free to assign any
rules to the boxes of the fuzzy rule base (AS.3), we just assign
Al A2
two different rules to these two boxes, and obtain the required
52 52 f e F with f(x) /yl = /(z) (see (11)), where ^(y1)
is the center of the output region of the active rule for x(z).
Finally, we prove that F vanishes at no point of Q. By
52= 52 ni<^„[ml}1(2;>)m2}2(a:>)]
21 1 22=1 AS.l and AS.2, we can make all the y1 > 0. Since (11) is well-
(22) defined, there exists at least one i such that ni<j<„[mj(a:y)]
which is of the form (11); hence, £ F. Finally, for any 0 for any x e Q. Since (11) is a weighted average of positive
c e R: y"s with some nonzero weights, the result is also positive,
Al i.e., we obtain f £ F such that f(x) / 0 (in fact, f(x) > 0)
52cyl'n1<J<„[ml’.(a:J)] for any x E Q. Q.E.D.
Proof of Lemma 3: For arbitrary x £ Q and fixed j, there
c/ift = (23)
are at most two mJ’s which are nonzero at Xj under AS.4.
5211i^<n[ml*(a;;)] Since a rule, say Rule i, is active only when f 0 for
2
=1 all j = 1, 2, • • • , n, there are at most 2" active rules for any
which is also of the form of (11); hence cfi(x) 6 F. In z 6 Q; this proves (1). If r components of x £ Q are at the
summary, F is an algebra of real continuous functions. centers of some fuzzy regions, there is only one mJ, which
Next, we prove that F separates points on Q. Let x. z g Q is nonzero at each of these r components (in fact, these mjs
and x 0 z. We now construct f £ F with f(x) f(z). First, are equal to unity at these r components), and for each of the
we define the fuzzy regions of the input space Q such that other n - r components there are at most two nonzero mJ ’s,
each element of x and z is at the center of a fuzzy region hence the total number of active rules is at most 2n~r; this
(recall that the center of a fuzzy region is defined as the point proves (2). If r components of x £ Q are at the centers of

A'l A2 A'l A'2

»1=122=1
52 52 52 52
il=l «2=1
Tf^x)
+t f2(x) =
f

52 52 ni^x„[mlJ1(a;>)m2J2(a:J)]
21=1 22=1
KI K2
+ ^2‘2)ni^Xn[mlJ1(a;>)m2J2(a:J)]
_ 2152
“
52
= 1 22 =1 (21)
KI K2

=
21=1 22 1

Authorized licensed use limited to: INDIAN INSTITUTE OF TECHNOLOGY KANPUR. Downloaded on March 24,2025 at 05:54:53 UTC from IEEE Xplore. Restrictions apply.
 WANG AND MENDEL: GENERATING FUZZY RULES BY LEARNING FROM EXAMPLES 1427

some fuzzy regions and q components of the x are smaller (or Jerry M. Mendel (S’59-A’61-M’72-F’78) re¬
greater) than the center values of the corresponding smallest ceived the B.S. degree in mechanical engineering
and the M.S. and Ph.D. degrees in electrical
(or the corresponding largest ) fuzzy regions, then there is only engineering from the Polytechnic Institute of
one nonzero mJ for each of these r + q components, and the Brooklyn, Brooklyn, NY, in 1959, 1960, and 1963,
—
other n r — q components have two nonzero mJ ’s associated
with each of them; hence, the total number of active rules is
respectively.
His experience has included teaching courses in
Electrical Engineering at the Polytechnic Institute of
at most 2n~r~q] this proves 3). Q.E.D. Brooklyn, from 1960 to 1963, and has also included
various consulting positions. From July 1963 to
January 1974 he was with McDonnell Douglas
References Astronautics Company. Currently he is Professor of Electrical Engineering-
Systems at the University of Southern California in Los Angeles, and is
[1] D. Nguyen and B. Widrow, “The truck backer-upper: An example of Director of the Signal & Image Processing Institute. He was Chairman of
self-learning in neural network,” IEEE Contr. Syst. Mag., vol. 10, no. the EE-Systems Department from March 1984 to August 1991.
3, pp. 18-23, 1990. He teaches courses in estimation theory, deconvolution, and higher-
[2] G. E. P. Box and G. M. Jenkins, Time Series Analysis: Forecasting and order statistics. He has published more than 240 technical papers and is
Control. Oakland, CA: Holden-Day, 1976.
[3] Y. F. Li and C. C. Lan, “Development of fuzzy algorithms for servo author of the monographs Maximum-Likelihood Deconvolution: a Journey
systems,” IEEE Contr. Syst. Mag., vol. 9, no. 3, pp. 65-72, 1989. into Model-Based Signal Processing (Springer-Verlag, 1990) and Optimal
[4] S. G. Kong and B. Kosko, “Comparison of fuzzy and neural truck Seismic Deconvolution: An Estimation-Based Approach (Academic Press,
backer upper control systems,” in Proc. 1JCNN-90, vol. 3, June 1990, 1983), the texts Lessons in Digital Estimation Theory (Prentice-Hall, 1987),
pp. 349-358. and Discrete Techniques of Parameter Estimation: The Equation Error
[5] W. Rudin, Principles of Mathematical Analysis. New York: McGraw- Formulation (Dekker, 1973), and, co-editor (with K. S. Fu (deceased)) of
Hill, 1964. Adaptive, Learning and Pattern Recognition Systems (Academic Press, 1970).
[6] K. Hornik, M. Stinchcombe and H. White, “Multilayer feedforward He is also author of the IEEE Individual Learning Program, Kalman Filtering,
networks are universal approximators,” Neural Networks, vol. 2, pp. and Other Digital Estimation Techniques. He served as Editor of the IEEE
359-366, 1989. Control Systems Society’s IEEE Transactions on Automatic Control, and
[7] G. Cybenko, “Approximations by superpositions of a sigmoidal func¬ is on the Editorial Board of the IEEE Proceedings.
tion,” Mathematics of Control, Signals, and Systems, 1989. Dr. Mendel is a Fellow of the IEEE, Distinguished Member of the
[8] L. X. Wang and J. M. Mendel, “Generating fuzzy rules from numerical IEEE Control Systems Society, member of the IEEE Geoscience and
data, with applications,” USC SIPI Rep. No. 169, Univ. Southern Calif., Remote Sensing Society, the IEEE Signal Processing Society, the Society of
Los Angeles, 1991. Exploration Geophysicists, the European Association for Signal Processing,
[9] P. Werbos, “New tools for predictions and analysis in the behavioral Tau Beta Pi and Pi Tau Sigma, and a registered Professional Control Systems
science,” Ph.D. dissertation. Harvard Univ. Comm. Appl. Math., 1974. Engineer in California. He was President of the IEEE Control Systems Society
[10] D. E. Rumelhart and J. L. McClelland, Eds., Parallel Distributed in 1986. He received the SEG 1976 Outstanding Presentation Award for a
Processing, Vol. 1. Cambridge, MA: MIT Press, 1986. paper on the application of Kalman Filtering to deconvolution; the 1983 Best
[11] A. Lapedes and R. Farber, “Nonlinear signal processing using neural Transactions Paper Award for a paper on maximum-likelihood deconvolution
networks: Prediction and system modeling,” LA-UR-87-2662, 1987. in the IEEE Transactions on Geoscience and Remote Sensing; a Phi Kappa
Phi book award for his 1983 research monograph on seismic deconvolution;
a 1985 Burlington Northern Faculty Achievement Award; and a 1984 IEEE
Centennial Medal.
Li-Xin Wang received the B.S. and M.S. degrees
from the Northwestern Polytechnical University,
Xian, People’s Republic of China, in 1984 and
1987, respectively, and the Ph.D. degree from the
University of Southern California, Los Angeles in
1992, all in electrical engineering.
From 1987 to 1989, he was with the Department
of Computer Science and Engineering, Northwest¬
ern Polytechnical University, Xian, People’s Repub¬
lic of China. From Fall 1989 to Spring 1992, he was
a Research/Teaching Assistant in the Department of
Electrical Engineering-Systems at the University of Southern California, Los
/Angeles. He is now a Postdoctoral Fellow in the Department of Electrical
Engineering and Computer Science, University of California at Berkeley. His
research interests are fuzzy systems and neural computing.
Dr. Wang received the Phi Kappa Phi Student Recognition Award for his
work on fuzzy systems.

Authorized licensed use limited to: INDIAN INSTITUTE OF TECHNOLOGY KANPUR. Downloaded on March 24,2025 at 05:54:53 UTC from IEEE Xplore. Restrictions apply.