CHAPTER 6

Design of Expert Systems

6.1 INTRODUCTION
In the previous chapters, we have discussed the general concepts and theory of
expert systems and other intelligent decision making-systems. A number of papers
have attempted to summarize the properties of intelligent systems. There are many
pros and cons for each type (Begley 03). This chapter presents general guidelines
for building practical expert systems designed for real-world applications, not re-
search prototypes. A software engineering methodology is described so that an
expert system can be a quality product developed in a cost-effective and timely
manner.
Many books, papers, and software tools have been devoted to the design of
expert systems, covering all aspects in great detail. So this one chapter will not
make you an instant expert. (You'll have to read the chapter first). However by
understanding the principles that explain why expert systems are designed the
way they are, you'll be better able to take advantage of sophisticated tools and
methodologies. In fact the design of expert systems is part of a more general
field called Knowledge Management (KM) which deals with al the knowledge
assets available to an organization (Becerra-Fernandez 03).
KM is connected to the Information Management (IM) which is connected
to the Information Processing (IP) which is connected to the Information Sys
tems (IS) which is connected to the Information Techuology (IT). (On second
thought, if you can say KM, IM, IP, IT, three times in a row without mispro-
nouncing, you are an Instant Expert!) KM is a major problem considering all
the different types of resources for many different types of audiences such as of-
fice, web, FAQs, Help Desk (human and automated), email, fax, phone, product,
program, developer, manager, end-user, and publicity.
Large companies have their own very detailed methods, documents, books,
and software
they developed to create, manage, maintain, and sell knowledge
assets because it is such a profitable field (Conway 02). This is especially true

353
 of Expert Systems
354 Chapter 6: Design

that can greatly reduce hum

tools are created
when intelligent
even cheaper
than global outsourcing.
penscs be-
Iefthat's
caus you think how much time a typical person spends documents
usingco
created
puters,
by com
it'sa
to manage all
the
need computers
tautology that
we
are widely used in business ecause they
because they are so com-
puters. Expert systems
increasing
amount of information
knowledge essential
available
to the dramatically linkS to software and other onli
the web (Malhotra 01). As usual,
on G.
listed in Appendix
for this chapter are

PROBLEM
6.2 SELECTING THE APPROPRIATE
and resources by which you can build a knowlede. -based
There are many ways
There are even móre ways you can waste time and.
your d money
expert system.
or
to the Starving Authors and Copyeditors Fund af
(for example, donating for
Wisdom states that hefo.
the Fourth Law of before
profit organization). However,
should know your destination.
you set out on a journey, you
Before you build an expert system, you should select an appropriate Droh
software project, there are a number
lem, discussed in Section 1.6. Like any
as
of general considerations that should be made before a large commitment of

people, resources,and time to a proposed expert system. These general consid.

erations are typical of project management concerns in conventional programs,
but must be customized for the special requirements of expert systems. A very
high-level view of project development is shown in Figure 6.1. The three gen
eral stages of Activity, Product Configuration, and Resource Management have
some more specific considerations, shown below. These more specific consider-
ations will be discussed as questions and answers to serve as guidelines for ex-

pert systems projects.

Selecting the Appropriate Paradigm

Why are we building an expert system?
This is probably the most important question to be asked of any project 1
most desirable answer is that the president of the company wants it built.Itni
not the case, see if the general advantages of expert systems as described inn
ter 1, Section 2 apply. Most of all, remember only management can authore
system and technical personnel needed. If you decide to build it on your own
to the company you were right, and then realize what a great
show proaue that
and decide to quit and form your own company, there is the small matter o
Intellectual Property Agreement you signed, Most IPAs state that whatever you
the
come up whether related or not to
with, yourjob
duties, 24x7x52, the belongsh
company. In particular, the answerto this question must eventually be given
be a
owners or stockholders funding development. Before starting, there shoud
clear identification of the problem, expert, and users.
pest
Note that the singular term expert is used, not
tne ext
plural experts. Just
Way to guarantee trouble is to have multiple spouses at once, more than o n
as

pert will also guarantee trouble. Even the President only sees one
a
ulole
time, so it's not a matter of cost. Trying to model the expertise or l
 6.2 Selecting the Appropriate Problem 355

Management Tasks
co 6.1 Project

Project Management

Activity Management
Product Configuration
Resource Management
Management

Analysis Forecast
Scheduling Acquire Resource
Resourcees
Needs
Chronicling Product Change
Planning
Management Management Minimize Assigning
Resource Resource
Bottlenecks Responsibilities

experts in one system is a bad idea. However, modeling the expertise of multiple
experts in multiple systems using a blackboard architecture to reach at léast a
majority opinion is good. This does bring up what the fuzzy term good means.
If nine out of ten doctors say the patient will die, the
patient will inevitably
choose the doctor that offers a treatment for life (unless the patient is really
cheap and the treatment costs money). Trying to decide how to rank multiple ex-
pert opinions is a daunting task, usually more formidable than building a single
expert system for a single expert. So we'll stick with the single-system-single-
expert approach in this chapter and leave the multiple-experts-multiple-systems
as a class project.

Payoff
What is the payoff?
This question is related to the first question. However, this question is more
pragmatic by asking for a specific return on investment of people, resources.
time, and money. The payoff may be in money, increased efticiency, or any of
the other advantages of expert systems described in Chapter 1. It is also impor-
tant to remember that if no one uses the system, there will be no payoff. Because
expert systems is different from conventional programming it is more ditticult
to answer this question.

Tools
What tools are available to build the system?
There are many expert systems tools available today with advantages and disad-
vantages as discussed in Appendix G. However, this should be taken only as a
guide since software tools develop so rapidly. Generally, you can count onan
enhancement every couple years in each t0ol and a major revision every five
356 Chapter 6: Design of Expert Systems

years. Of course there's møre money to be made in major bug fixes of

ware that are introduced with a new name as a new version. If you ha. soft-
doubts, count the number of new names for the same operating system
nave any
yor
been using the last ten years.
have
The best guide to picking an expert system toolfollows from the Third
of Wisdom: if you're lost, ask someone. Check the web for applications a
have been built using the software. Try to find the failures, not just the s
Although hard statistics are very difficult to come by, most estimates place hs.
number of successful software implementations at only 20-30%. This prohle
is compounded if you have to work with a busy expert and you are not familiar
with terms from the expert's domain.
Which brings us to the Second Law of Wisdom: if you're going to ask
someone for help, it's a good idea to know the language. While there are many
human and computer foreign language translators available, it's a lot more difi.
cult to learn terms in a knowledge domain.
However just learning terms or memorizing dictionary definitions of words
does net of relationships on which a rule-based
not create the semantic expert
system is based. While each rule does represent a chunk of knowledge, there is
both strong and weak coupling between rules in the knowledge base. If the fir-
ing of one rule is guaranteed to make another rule fire, that is strong-coupling
or to put it another way, a strong link in the inference chain that leads from valid
facts to a valid conclusion. If one rule firing leads to multiple rules being avail-
able for triggering by the inference engine, that is weak-coupling. This means
the inference chain is not as
strong; we are not as sure which path to follow be
cause there is more than one.
The more rules that can potentially fire indicate that the inference chain is
not as strong as in strong coupling. However this is not bad. If every rule was

strongly coupled to another rule, there would be no need for an inference

or
expert system because a procedural enginc
A programming
procedural language is designed to operate language is perfect for at
uas
ment, in the order entered unless there
sequentially, statement by sta
there is
control decisions such as IF teso It
are
an
algorithm that can solve a
problem efficaciously, then there S no
for an Al solution.
Ideally, an expert system is a balanced mix of and
weakly coupled rules, just as humans may use strouabilis-
tic, and other methods to reach a deductive, inductive, proD
Consider the following semantic
conclusion.
list of terms in a The
term semantic list is used
here because no knowledge do lt
stead, one word naturally follows graph links these words tog
from the
are, the more
you'll understand the complete previous. (Note that the O
derstand terms like: overdue list!) Think how long it lo0
delinquent, account closed, debtmovie, overdue notices, overdue c c o u n t

We know
m
collection agency, phone calls ("ArC home
hon
you're home because
at y
email, bad credit, low you're not at work"), huge increzasc pam
credit,
due, taxes plus interest on no
credit, "we can finance taxes

taxes, you, in a lobby

ist, campaign attorney fees, foreclosure, sherit
contribution, rider ology fromsheri
apology from company, huge cashon appropriations bill, apol
aged buyout of video rental award for mental pain and
ring, l e v e r

company. sufrer"E
 6.2 Selecting the Appropriate Problem 357

Semantic lists are often associated with other signs. The field of semiotics
studies signs and their meanings. An example that is familiar to everyone is
learning your "A, B, Cs as a child. You should still remember the melody that
ooes along with children singing the "A, B, Cs" because there is a semiotic rela-
fionship between the song, i.e., lyrics and music. Semiotic relationships are used
all the tinme by advertisers in famous TV jingles. Some jingle creators are so
good at this art that you can't get a jingle out of your head for days; it just keeps
repeating over and Over in a loop.
When you are mterviewing an expert, it's essential to also create a written
transcript of the interview. By reading the transcript you can spot the semantic
relationships between terms. That is what is important. Knowledge terms are not
isolated words; they fom semantic clusters that link to form the entire semantic
net of knowledge in the expert's mind. The mind is a good example of an asso-
ciative semiotic menmory in which one word, smell, touch, sight, or sound may
elicit all kinds of other memories.
The classic tool for eliciting knowledge the standard interview and much
work has been done on this (Novak 98). In fact the interview technique is usu-
ally augmented by drawing diagrams to make sure the concepts and relation-
ships. Visual metaphors are useful because the developers and experts can both
see the relationships and meanings at the same time and agree on what is to be
represented and how. Software tools are also available that can represent the en-
tire ontology. Recall from Chapter 1 that an ontology is the complete descrip-
tion of a domain represented in a formal way.
One tool which has been used very successfully is Protégé, an ontology edi-
tor and knowledge-base editor. This is a very sophisticated tool is available free
from Stanford University and well-supported on their website with tutorials on cre-

ating your first ontology and many other topics. Protégé is used with TixClips,
an

essential when
integrated development environment for CLIPS. Ontologies are
building very large systems having thousands of rules.

Cost
How much will it cost?
This is like asking how much a divorce will cost. It's not the up-tront costs that
you can see in email spam for do-it-yourself-divorce or do-your-own-brain-sur-
and up. The cost of
gery kits. It's the costs that come later which keep adding up
and time devoted to
building an expert system depends on the people, resources,
ts construction. Also, just like any other piece of software or hardware, you
have to figure in the cost of maintenance. This brings us to
the First Law of
of gas, it's a
Wisdom: If you've bought a car and are not worried about the cost
Zero Law of Wisdom: It's better
g00d idea to own your own oil well. And the
to use someone else's car and gas. Maitenance of an expert system is also much

more difficult if your expert is no longer

available.
much help since experts are quite competi-
Bringing in another expert is not
tive. It's a lot easier to build a knowledge-based system where the knowledge
other sources that are publicly available.
Comes from people, documents, and
have done this because they have found these systems
are
Many companies
 Systems
358 Chapter 6: Design of Expert
the problems with a few
few questions.
useful screening tools
that get rid of 90% of center.
telephone call
lot chcaper than funding
a
This is a to run an expert system

the hardwareand software required tems tool,

Besides If your personnel have
considerable cost in training.
. e or little
there may also be to train them. Professional
no experience
with a tool, it can
be costly
person. Of course
traini
to $2500/week per
software often runs 1s
c o u r s e s for any
companies can make far
it used to be. Now
no longer the
disadvantage oniine courses, certificas
in classes and
providing for their products n
money how toO be an instructor for certi
exams, preparing
for certification
exams, ica-
and in-home support. Todav
tion exams, books, seminars,
workshops, putting
and bug free is a recipe for fail
easy to use,
out a product that is powerful,

IN THE DEVELOPMENT OF AN EXPERT SYSTEM

6.3 STAGES
How will the system be developed?
of an expert system will depend the re. on
To a large extent, the development
sources provided. However, like any other project, the development will also de-
on how the process is organized
and managed. A good reference based on
pend
an expert system developed for the Federal Highway Administration that used
over 300 rules is (Wentworth 94). If you're familiar with a standard project plan-
will make it easier to
ning tool like Microsoft Project, then stick to that tool. It
explain to management what is going on and why the project is a little behind
schedule (as you get your resume up-to-date).

Project Management
Standard project management techniques and software tools are expected to pro-
vide the following.

Activity Management
planning
- define activities

specify priority of activities

-resource requirements
milestones
-durations
-responsibilities
scheduling
assign starting and ending times
resolve conflicts in
scheduling tasks of equal priority
chronicling
-monitor project performance
analysis
-

analyze plans, schedules, and chronicled

activities
Product Configuration Management
product management
manage the different versions of the
product
 6.3 Stages in the Development of an Expert System 35

change management
manage change proposals and impact evaluations
assign personnel to make changes
install new product versions

Resoure Management
forecast needs for resources
acquire resources
identify expert or knowledge base
arrange your schedule for convenience of expert
assign responsibilities for optimum use of resources
provide critical resources to minimize bottlenecks

In particular, note that in the acquisition of resourees, the point is made that you
should adjust your schedule to that of the expert and not vice versa. Unless you
are paying the expert full-time, there will be other duties and their time is very
valuable.
Figure 6.2 shows the ideal high-level view of the activities required to pro-
duce a system in terms of the stages that a system goes through.

in the Development of an Expert System

Figure 6.2 General Stages

to show
Paper or comparison study
Feasibility Study project is feasible

Expert system quickly put together

ideas, arouse enthusiasm,
to demonstrate
Rapid Prototype
and impress upper-level management

In-house verification of the expert systemn

Refined System on real problems by knowledge engineers
(o-test) and expert

selected userS not

Field Testable System tested by
or expert
(5-test) knowledge engineers

Validated and tested

User documentation
Commercial Quality
System Training and/or electronic mail
Fast user support by telephone

Maintenance and Fix bugs

Enhance capabilities
Evolution
360 Chapter 6: Design of Expert Systems

In the ideal view, the product is not deployed for end users until all the H
are worked out. In the case of military products, this is a really good idea since bugs
it S not that easy to recall a nuclear missile once it's been launched.
In contrast to government contracts, things are ditterent in the commercial
World since there are no guaranteed contracts that pay money on a regular basic
and cost overruns if the project is latc. The commercial paradigm followed to
for the product every quarter until i
day is to spenda lot of money on publicity
1s released several years later. This has two purposes. First, private develonera
pers
who want to use your product in their products are willing to pay money for
pre
release versions even with known bugs. So this provides a continuing source of
revenue as the product is being developed instead of your company having to
for several years. A secondary benefit is that vou
fund all the development costs
don't need to do much (if any) in-house testing.
crashes, it asks the user if it
Simply make sure that every time the product
clicks OK, the message will he
can send an automatic bug report. When the user
with
sent information as to where the fault occurred
diagnostic
automatically
will store the bug report in a
An automated email program at your company
database. Standard data mining tools will then look for patterns to notify your
for a particular aspect of the product. Dif.
particular developer team responsible
levels will be set up so that only when the number of bugs has ex-
ferent trigger
notification be sent to the teams. Some
will
ceeds their particular trigger level a

teams work is more important

than others so it is not necessary that the same
on the number and severity
of the bugs, re-
trigger level be used. Depending
sources may then be allocated to
fix them for the next quarterly release. A spe-
cial website should be set up for prerelease
versions, with a subscription fee
version.
charged to access each new quarterly preview announcements should be timed
Second, and important, the quarterly
most
the Se-
to coincide with the quarterly reports
to public stockholders required by
curities and Exchange Commission (SEC) for public companies. Even if youre
still private and have not yet released an initial public offering
(IPO) of stock,
increase the value of your IPO.
this will attract venture capital to your site and

The Delivery Problem

How will ihe system be delivered?
to be deployed, the deliver
Depending on the number of expert systems
factor in development. Ine
problem of the developed systems may be a major
earliest stage of developmen
delivery problem should be considered in the
cost far more than the system
Development systems in the 20th Century low co
users. Now with
which the product was to be deployed to end
been solved.
machines, that aspect of the deliyery problem has
of hardware that your prou
However, another problem is the wide range with u
accounted for the initial popularity of Java
is supposed to run on. This
and that applets could only Pl
Write once, anywhere," slogan
use guarantees
different hardware is still a problem
in the "sandbox." The deployment
on
on every dittet
a virtual machine is installed
less an interpreter is used and
That has worked successfully since bytecode
interpreters w
hardware.
type of
 6.3 Stages in the Development of an Expert System 36

introduced with Pascal in the 1970s so that it could run on any microcomputer.
This same approach was used by Java.
Although CLIPS does not generate bytecodes, it is designed to produce a
standardC version of the finished expert system. This can then be compiled us
ing a standard C compiler for the specific hardware platform and the executable
deployed for that particular platform. No "CLIPS Virtual Machine" is required.
One of the original design goals of CLIPS was to produce an expert system
tnat
could easily be deployed on any new hardware that may arise. If this is not ade-
quate, other versions of CLIPS such as Jess have been developed as mentioned
in the Preface. Jess is written in Java and althougth it does not support multiple
inheritance, it has all of the advantages of Java in being easily deployed to dif-
ferent hardware.
In many cases the expert system must be integrated with other existing pro-
grams. Consideration should be given to the communication and coordination
of the expert system input/output with these programs. It may also be desirable
to call the expert system as a procedure from a conventional programming lan-
guage, and the system should support this. CLIPS is designed so that it can be
called from a host language such as a C++ program, do its function, and then
return control to the host program. Such a hybrid system was also one of the
main design considerations of CLIPS since the performance of an expert systemn
can decrease quickly depending on how many rules are in the system and how
much pattern matching must be done. While the Rete Algorithm is good, an ex-
pert system is still not as fast as compiled code.
Although you could use CLIPS to simulate a calculator, spreadsheet, or even
a word processor, its performance would be not as good as a program writen in
a third generation language such as C, C++, or C#. For example, you could
create an intelligent data mining tool by calling CLIPS from a standard database
such as Oracle or MySql and return recommendations to the database user. In
is designed to let other applications use data and then return
Oracle, the cursor
control to Oracle. CLIPS follows this same principle of being a good host ora
good guest. Another alternative is to use/modify the CLIPS source code and in-
tegrate it into your application, then compile the result into a new application
for deployment.

Maintenance and Evolution

How will the system be maintained and evolve?
The maintenance and evolution of an expert system is more of an open-ended
than with conventional programs. Because expert systems are not based
activity
on algorithms, their performance is dependent on knowledge and expertise. As
new knowledge is acquired and old knowledge moditied, the performance of the

system should ideally remain constant.

However there is a problem in that conventional programs based on algo-
rithms will not have rule conflicts like expert systems. A rule conflict is nothing to
be afraid of. In fact that is why inference engines were developed. All a rule con-
flict means is that more than one rule has matched the pattern on its left-hand-side
and can potentially be fired. People do thinking of this type all the time.
 Systems
362 Chapter 6: Design of Expert
Paradox. It you re sitting
ng on
on th.
the
For example,
consider the Game
and a handful of.
f
couch
in one hand
game on
a
TV and have a beer
One rule is saying, In " chips
watching in your mouth?
do put
"If beer is available then
which you
the other hand.
available then eat." Another rule is saying,
have a rule conflict.
drink
and drink are available you
Since both food book, the Minus One ne Law of
who have read this
Fortunately for those destination, go everywhere
becarse
a specific
Wisdom states: you don't have
If The obvious solution to the Ca
n e v e r know
what you're missing. eat-drink. (After all, where wou
you beer and then
in
dox is to dunk the chips been so busy gambling he cau ldn't
of Sandwich had not
some meat in between twa si
world be if the Earl
for dinner so he just put
lices
time to sit down
spare
became famous?)
of bread and 1s also more of a con.
system atter delivery
of an expert
n-
The enhancement
of a commercial Svat
ystem
system. The developers
cern in a commercial-quality to what the users want and
success. This means listening
it be financial
want to a
In the commercial world, it has been
to pay for in improvements.
are willing will continually pay for software up-
proven over and again that people
over
there will be fewer bugs.
grades in the desperate hope as new features
are added. In fact it is
this will never happen as long
Naturally hundreds or thousands of known bugs.
release products with
accepted practice to software piracy since legitimate
users will be
This is actually a way of fighting
download patches, and patches, and
manufacturer's website and
able to go to the another service
then more patches until
until a service pak is available,
patches. release its new product or face stiff
a company must
pak is available, until finally
Like all such software, an
stock price manipulation.
penalties from the SEC for finished-it only keeps getting
better.
never really be
expert system may

6.4 ERRORS IN DEVELOPMENT STAGES

canbe classitied by
The potential major errors of expert systems development
These er
illustrated in Figure 6.3.
the most likely stages in which they occur, as
rors include the following:

Expert's Errors. The expert is the expertise source of the expert systenl
expert's knowledge is erroneous, the results may be propagated througn t
ire development process. As mentioned in Chapter 1, a side benetit of b u s
ex
an expert system is the potential detection of erroneous knowledge when tne not
pert's knowledge is made explicit. Just because someone is an expert u
mean they will never make a mistake. It is just that they are far less nA
ertorm

an ordinary person in their domain of expertise. For example, you could P

yo
brain surgery on someone. However unless you are a trained neurosurec pro-
patient is less likely to survive. On the other hand, we have all heard T Va
claiming that if you have been injured by an expert (or anyone wi
there are plenty of lawyers willing to help you receive compensation
For mission-critical projects in which human life and property a ledge
may be necessary to set up formal procedures to certify an expert'si
One approach that NASA has successfully used for space flight is ngne
 6.4 Errors in
Development Stages 363

Figure 6.3 Major Errors in Expert Systems and Some

Causes

Errors in the
Expert expert's knowledge, such as incorrect and
incomplete knowledge

Knowledge Semantic errors of meaning between

engineer and expert
knowledge
Engineer
Incomplete elicitation of knowledge from expert

Syntax errors of form

Knowledge Errors in
Base content due to incorrect and incomplete
knowledge and uncertainty in rules and facts

Inference Bugs in the inference engine, and other software of the

Engine expert systems tool

Errors of inference due to incorrect priority of rules,

Inference
interactions of rules, and errors of the knowledge-base
Chain Errors due to nonmonotonic inference

review problem solutions and the analysis

Technique Panels, which regularly inde-
solutions. The panels consist of system users,
techniques used to develop the and managers to ensure coverage
of
pendent domain experts, system developers,
all areas affecting development. is
approach is that the expert's knowledge
The advantage of a group panel when
the development process,
under close scrutiny at the beginning of
placed The longer that e r r o n e o u s knowledge
is easier to correct.
erroneous knowledge
to correct. If the expert's knowledge is
goes undetected, the
more expensive it is The
validation of the expert system.
the ultimate test is the satisties
not initially verified, demonstrates whether
the system
Iinal validation of the expert system solutions.
of
correctness and completeness
requirements, especially Sometimes they are called
time in large companies.
Panels are used all the neutral term. During
the 1970s and 80s, a
more
teams or a
performance review
tried in software development by allowing pro-
variation of the panel review was
it quickly degenerated into
code. Unfortunately
grammers to review
each other's started equating a
programmer's
manager's
review tool once lines of code per day,"
or "errors
a bad management
like "correct
in terms of
metrics
was not supposed
to know the de-
performance
management
code." Although
some
would be honest,
per 100 lines of so that programmers

review process stake. Panels can

tails of this peer and raises were at
when promotions factor
as peerful but the "peerful"
peers were not when there is appropriate oversight information.
be a useful tool the reliability of
when it c o m e s to
considered
Should always be
 6.5 Software Engineering and Expert Systems 367

theirperformance degrades (hopefully) gracefully at the boundaries of their 1g-

norance. Human experts should be honest enough to admit more uncertainty in
their conclusions near the boundaries. However, unless an expert system is
specifically programmed to handle uncertainty, it may continue to supply an-
swers even if the inference chain and evidence are very brittle and unreliable.
The worse thing is that the human may believe the conclusions are reliable.

Vs
6.5 SOFTWARE ENGINEERING AND EXPERT SYSTEMS
In the previous sections we have discussed the
general considerations in using
the expert systems paradigm. Now let's examine the stages of expert systems de-
velopment from the more technical perspective of the knowledge engineer who
actually builds the system.
Since expert systems have moved out of the research stage, there is a real
need to deliver quality software up to the standards of conventional software.
The accepted methodology for developing quality software for commercial, in-
dustrial, and governmental standards is software engineering.
It is important to follow good standards in the development ofa product or
the product will probably not be of good quality. Expert systems are products
just as any other software product such as a word processor, payroll program, or
computer game.
However, there is a significant difference of mission between expert systems
and typical consumer products such as word processors and video games. The term
mission refers to the overall purpose of a project or a technology. Expert systems
technology generally has a serious mission of supplying expertise in high-
performance and possibly hazardous situations where human life and property
are at stake. These are the mission-critical applications mentioned in the previ-
ous section. Knowledge-based systems do not have to meet such high standards.
Mission-critical applications are very different from the more relaxed mis-
sion of word processors and video games, which are developed to increase effi-
ciency and provide recreation. Human life is not jeopardized by a bug in a word
processor or a video game (or at least it shouldn't).
Expert systems are high-performance systems that must be of high quality
or they will be prone to bugs. Software engineering provides methodologies for
building quality software, as illustrated in Figure 6.4
Quality is a difficult term to describe in a general sense because it means dif-
ferent things to different people. One way of defining quality is as the required
or desirable attributes of an object determined on some scale. The term object is
used here to mean any software or hardware, such as a software product. The at-
tributes and their values are calléd metrics because they are used as measures of
an object. For example, the measured reliability of a disk drive is a metric of its
quality. One measure of this attribute is the mean time between failures
(MTBF) of the drive. A reliable drive might have an MTBF of 50,000 hours of
use between crashes, while a cheap drive might have a MTBF of 10,000 hours.
In the 20th century, people might use their computers a few hours at work or at
home. Now it's customary to leave the computer on 24x7x52 and the hours do
368 Chapter 6: Design of Expert Systems

Methodology
Engineering
Figure 6.4 The Software
PROBLEMS

Wide Variation in Lack of Programmer

High Cost of
Development
Productivity
Development Methodology

SOFTWARE ENGINEERING

Software
Plans. Schedules Life
Documentation Requirements, Reports
Cycle
and Design

PRODUCT

Cost On Time Well

Validated Documented
Verified, Effective
Easily Maintainable
and Tested
and Enhanceable

add up. Of course you can set your operating system to power down the drve

justlike the monitor screen. However these on and off a dozen times a
turning
day may introduce new risks.
What's important is not just the time, but the error rate in writing. Suppose
disk drive has a quoted error of one bit in every billion writes. In the 1990swnen
common disk drives were 20 Megabytes, that meant you could fill yourdnve
Limes over without a single write error. Now with drives in the 200 Gigaoyi
range, one write error in a billion means there may be 200 errors on your ha
drive, assuming the same error rate.
If
the error occurs in writing a compressed image such as a jpg it willpe
ably not be noticed. But if thie error occurs in a line of code, a sins
terror

could turn a program statement like X=A to X=8, with potentially very Daudisk
sults. Modern drives have a built-in sensor that ntinuously monitors
lech-
health through feature called Self-Monitoring, Analysis, and Reporting
a

nology (SMART). The SMART technology can be accessed by Sh such

as Norton prognive
System Uilities when the Disk Doctor Utility is run. Tn can

However you must

advance warning of most
potential disk hardware failures. However
heed the warning by backing up and replacing the drive before data is losit ifthe
SMART is a tool that predicts
your disk drive heath but cannot
problem 1s With deteriorating magnetic media, since nothing 1s
e,The
P
 6.5 Software Engineering and Expert Systems 369

smaller a microscopic imperfection in the magnetic coating of the hard disk

suror
face created during manufacturing or operation, thé greater is the possibility
a write error, and thus an error in the software. As more data is read and written
onto the magnetic surface ofa hard disk, and the longer the drive runs, parts wear
down. Newer drives may run down faster since they spin at higher speeds and the
magnetic coating must be even more uniform and without imperfections.
In the 1990s the standard was 3600 RPM, then
5400 RPM, and now 7,200
RPM with premiunm drives available at 10,000 RPM. As an analogy, assume the
tire of your car is 2 ft. in diameter and so is about 6 feet in circumference. If your
tire turned 3,600 RPM, you'd be going about 250 mph, while a 10,000 RPM tire
would make you go 700 MPH, thus breaking the sound barrier if your disk ro-
tated as fast as a car tire! It used to be that the weakest spot in a computer system
was the moving components because of the friction and heat
they generate.
However a new point of failure has arisen with 64-bit processors available and
generating a lot more heat than the old 32-bit ones. This is such a critical problem
that new motherboards come with temperature sensors, a heat-sink fan right on
top of the microprocessor, a case with 6 fans for cooling, and software to shut
down the system if the temperature rises too high. It's also a good idea to physi-
cally locate the disk drive and memory as far away from the CPU as possible.
Table 6.1 gives a checklist of some metrics that may be used in assessing the
guality of an expert system. These metrics should be taken only as a guide since
a specific expert system may have more or fewer of these. However, the impor-
tant concept is to have a required list of metrics that can be used in describing
quality.

Table 6.1 Some Software Quality Metrics for Expert Systems

Software Metric3
Correct output given correct input
Complete output given correct input
Consistent output given the same input again
Reliable so that it does not crash (often) due to bugs
Usable for people and preferably user-friendly
Maintainable
Extensible
Validated to prove it satisfies the user's needs and requests
Tested to prove correctness and completeness
Cost effective
Reusable code for other applications
Portable to other hardware/software environments
Interfaceable with other software
Understandable code
Accurate
Precise
Graceful degr lation at the boundaries of knowledge
Embedded capability with other languages
Validated knowledge base
Explanation facility
370 Chapter 6: Design of Expert Systems

A list of metrics allows you to more easily prioritize metrics since

may be in conflit with others. For example, increasing the testing of aneny
system to assure its validation will increase the cost. Deciding when testinPert
is generally a complex decision involving the factors of schedules, cost ends
quirements. 1deally, all three of these requirements should be satisfiedd. r
In e
prac-
tice, some may be judged more important than others and the constraints.
of sat-
isfying all factors will be weakened.

6.6 THE EXPERT SYsTEM LIFE CYCLE

One of the key concepts of software engineering is the life cycle. The softu
life cycle is the period of time that starts with the initial concept of the softu
and ends with its retirement from use. Rather than thinking of development an
maintenance separately, the life cycle concept provides a continuity that eon
nects all stages. Planning for maintenance and evolution early in the life cvcle
reduces the cost of these stages later.

Maintenance Costs
For successful conventional software that keeps getting modified and enhanced
every few years, maintenance can easily be ten times the initial development
costs when the product was first released and grows with every new version.
Even though this may seem a lot, as mentioned earlier there's a lot of money to
be made in maintenance. However the Minus Three Law of Wisdom states: If
you don't love to journey, then love what you've got. So if you keep coming out
with bug fixes of your product, don't change the name to something entirely
new. Stick with a variation of the old name, but don't say "Doors BugYix
No.10," say "Duors 2010."
on
Expert systems tend to be more knowledge based these days than based
human expertise. There are significant exceptions such as intelligent trainers tor
special personnel such as doctors or pilots. Expert systems are used very suc
cessfully as training systems to reduce the amount of time experts spend teaei
ing introductory material. However if enough time and resources are devo
the expert system, it can teach a student to be ready for the real thing
As technology changes, there is less demand to preserve a retiring pe bused
knowledge. Expert systems require more maintenance because they do a
on much knowledge that is heuristic and experiential. Expert systens i e
lot of inference under uncertainty are even more susceptible to n g mam
itg
e

nance and evolution costs. With the availability of automatic tod n a i n t a i n

and importing ontologies into expert systems, it is becoming easier lo solt

the systems. However the systems also tend to become larger, just IKC
ware, so it's always a running battle.

Waterfall Model
ventional

A number of different life cycle models have been developed for co And is
software. The classic waterfall model was the original life cycle mo
illustrated in Figure 6.5.
 6.6 The Expert System Life Cycle 371

Flgure 6 . 5 7 erfall Model of the Sotftware Life

Cycle

System
Feasibility

Validation
Software Plans
and Require-
ments
Validation

Product
Design
Verification

Detailed
Design
Verification

Code

Unit Test

Integration
Product
Verification
Implementation

System Test
Operations and
Maintenance

Revalidation

of conventional software. In the wa-

This model is familiar to programmers
veritication and validation (V & V) activity
terfall model each stage ends with a
Also, notice that the arrows go back and
to minimize any problems in that stage.
the iterative developmentbetween
forth only one stage at a time. This represents
minimize the cost compared to the much higher
two adjacent stages in order to
over several stages. The
cost of going back stages
cost of iterating development
but grows exponentially harder.
1S not a simple linear function
372 Chapter 6: Design of Expert Systems

Another term for life cycle is process model because it is concerned

the following two fundamental issues of software development: with

(1) What should be done next?

be performed?
(2) How long should the next stage
The process model is actually a metamethodology because it determi
the order and duration that the common software methods are applied. The co
mines
m
mon software development methods (or methodologies):
such as
specific methods to accomplisha stage
planning
requirements
knowledge acquisition
testing
representation of stage products
documentation
code
diagrams
Code-and-Fix Model
A number of process models have been used for software development. The ear
liest "model" is the infamous code-and-fix model, in which some code is w
ten and then fixed when it doesn't work right. This is usually the method of
choice for new programming students both in conventional programming and
expert systems.
By 1970 the deficiencies in the code-and-fix approach were so obvious that
the waterfall model was developed to provide a systematic methodology that
was especially useful for large projects. However, there were difficulties with
the waterfall model because it assumed that all the information necessary fora
stage was known. In practice, it was often not possible to write the complete re
quirements until a prototype had been built. This led to the do-it-twice concept
in which a prototype was built, the requirements determined, and then the tin
system was built.

Incremental Model
The ineremental waterfall model is a refinement of the waterfall and ot
Standard top-down approach. The basic idea of ineremental development istto
develop software in increments of functional capability. The incrementa The model

has been used very

successfully in large conventional software projectS.
incremental model is also useful for expert
systems development, in wn
addition of rules increases the
capabilities of the system from assistant n
league and finally expert level. Thus in an expert system the major incre
are from assistant, to
colleague, and from colleague to expert.
The minor increments n
each

level that offer some

correspond to increments of
expertise wae in
significant increase. Amicroincrement is the cua
expertise by adding or refining an individual rule.
 6.7 A Detalled Life Cycle Model 373

oiral
Figure 6.6 A Spire Model of Expert System Development

Planning Knowledge Acquisition

Requirements Validation
Design
Validation

Delivered System
Time

Evaluating the Coding

Expert System Verification
Testing Testing
Verification Documentation
Integration
Validation

The primary advantage of the incremental model is that the increases in

functional capability are easier to test, verify, and validate than the products of
individual stages in the waterfall model. Each functional increment can be
tested, verified, and validated immediately with the expert rather than waiting to
do the entire validation at the end. This decreases the cost of incorporating cor-
rections in the system. In essence, the incremental model is similar to a continu
ous rapid prototype that extends over the entire development. Rather than just a
rapid prototype of the initial stages to determine requirements in the do-it-twice
approach, the evolving prototype is the system.

Spiral Model
One way of visualizing the incremental model is an adaptation of the Barry
Boem's standard spiral model for software engineering, as shown in Figure 6.6.
Each circuit of the spiral adds some functional capability to the system. The
ending point label "Delivered System" is actually not the end of the spiral. In-
stead, a new spiral begins with maintenance and evolution of the system. The
spiral can be further refined to specify more precisely the general stages of
Knowledge Acquisition, Coding, Evaluation, and Planning.

6.7 A DETAILED LIFE CYCLE MODEL

A life cycle model that has been successfully used in a number of expert sys-
tems projects is the Linear Model, illustrated in Figure 6.7. This life eycle
sists of a number of stages from Planning to System Evaluation and describes
 374 Chapter 6: Design of Expert Systems

Figure 6.7 The Linear Model of Expert System Development Life Cycle

Knowledge Design Product

Baseline Baseline Baseline

Knowledge Definition Knowledge Design Knowledge Validation

Code
Source Acquisition Detailed& Formal Test System
Planning Jdent. & Analysis & | Definition Checkout Evaluation
Design Test Analysis
Selection Extraction

Knowledge Preliminary Knowledge Test Test

Work Final
Review Data System Readiness Audit
Plan
Review
Review
Review Design Review
Review

the development of the system to some point at which its functional capabili
ties will be evaluated. After this, the life cycle repeats this same sequence from
Planning to System Evaluation until the system is delivered for routine use.
The life cycle is then used for subsequent maintenance and evolution of the
system. Although not explicitly shown, verification and validation proceed in
parallel with the stages. Rather than just fixing up some bugs, it is important
1o follow through with the same sequence of stages to maintain the quality ot
the expert system. Skipping stages, even to fix one little bug, impairs the en
tire quality.
This life cycle shown may be considered one circuit of the spiral model.
Each stage consists of tasks. Not all tasks may be necessary for a stage, espe
cially once the system goes into maintenance and evolution. Instead, the tasks
are meant o serve as a composite of all tasks for the entire lite eycle, trom in
tial concept to software retirement. The tasks will also depend on the exact
of application being built and so should be considered only as a guide, ran
be
than absolute requirements that must be performed for each stage o
completed.
This life cyclemodel will be discussed in detail to illustrate the manyfac
I re
should be considered for a large quality
tors
that expert system. Forsma "even

search-type prototypes that are not intended for general use, not all tasks or c
stages are necessary. However, it is amazing how much software that is de
oped for personal or research use gets released to associates and then ntogen
eral use.

Planning
the
ex
The purpose of the Planning Stage is to produce a formal work plan
to ed used

pert system development. The work plan is a set of documents that wil
to guide and evaluate development. Table 6.2 illustrates the tasks in thisstage
>
 6.7 A Detailed Life Cycle Model 375

Table 6.2 Planning Stage Tasks

Task Objective
Feasibility assessment
Determine if it is
worthwhile
whether expert
to build the system and if so.
Resource management Assess systems technology should be used.
resources of
ware
people, time. money. software, and hard-
required. Acquire and manage the
Task phasing Specify tasks and their order in the required resources.
the
Schedules Specify the stages.
Preliminary functional layout starting and delivery dates of
Define what the tasks in the stages.
system should accomplish by
of the system. This taskspecifying the
the
high-level functions
purpose of the system. specifies
High-level requirements Describe in high-level terms how the
functions of the system
will be accomplished.

The feasibility assessment is the most

important task in the life cycle. The
assessment must answer the questions of whether the project is worthwhile and
the related question of whether an
expert system is the appropriate paradigm.
The answers to these two questions determine if the project should proceed us-
ing an expert systems approach. Many factors are involved in feasibility assess-
ment. As discussed in Section 6.1, these factors include selection of an
appro-
priate expert systems domain, cost, payoff, and others.
Knowledge Definition
The object of the knowledge definition stage is to define the knowled
Tequirements of the expert system. The Knowledge Definition Stage consists of
two main tasks, as follows:

Knowledge sourceidentification and selection

Knowledge acquisition, analysis, and extraction
Each of these major tasks is composed of other tasks. Table 6.3 describes the
tasks involved in source identification and selection.

Table 6.3 Knowledge Source ldentification and Selection Tasks

Task Objective
Source identification Who and what are the knowledge sources, without regard to

availability.
Prioritized list of knowledge sources in order of importance
Source importance
to development.
List of knowledge sources ranked in order of availability.
Source availability
The Web, books and other documents are generally much
human experts.
more available than
Source selection Select the knowledge sources based on importance and
availability.
376 Chapter 6: Design of Expert Systems
tasks are described in1Ta
Table 6.4,
and extraction
Acquisition, analysis,

Extraction Tasks
Acquisition, Analysis, and
Table 6.4 Knowledge
Objective
Task
Specifyhow knowledge will be acquired by
Acquisition strategy for interviewing experts, reading document ethods
rule
induction, repertory grids, and so forth,

Knowledge element identification Pick out the specific knowledge from sources
will be useful in this iteration ofthe life cycle
Classify and organize the knowledge to aid in know.
Knowledge classification system edge verification and understanding by developers
Use hierarchical groups whenever possible.

Specify the functional capabilities of the svstemi

Detailed functional layout detail. This level is at a more technical level while
the preliminary functional layout was at a manaoe.
rial level.
Describe general phases that the expert system wil
Preliminary control flow
execute. Phases correspond to logical groups of
rules that are activated/deactivated in groups to
control execution flow.
user's viewpoint. An often ig-
Preliminary user's manual Describes system from
nored, but essential part of the system. It is ab-
solutely important to involve users as soon as pos-
sible for feedback. If they don't use the system, it's
worthless.
Define exactly what the system is supposed to do.
Requirements specifications The expert system will be validated using these re-
quirements.
Knowledge baseline Baseline knowledge for the system. Any changes
must now be done by a formal change request. The
high-level knowledge is now adequate for the next
stage of knowledge design.
www. www.w.

naly
The main objective of the knowledge acquisition task, the knowledge
sis task, and the knowledge extraction task is to produce and verity tne
ined,
edge required by the system. By the time that the knowledge is baseil
should be correct and suitable for the next stage of knowledge design. l
tion to thecustomary method of interviewing experts, many software
s are may
pro
pro
be used to perform automated knowledge acquisition. Often these neer t
vided by the tool vendor.
Knowledge acquisition by a knowledge enginee
ing to elicit knowledge from an expert requires a great deal of skill au is dis
1 is dis-

cussed further in
(Grzymala-Busse 91).
Knowledge Design
design
The objective of the knowledge design stage is to tailed

for an expert system. There

produce the ael
are two main tasks that stage:
comprise this sig
Knowledge definition
Detailed design
 6.7 A Detailed Life Cycle Model 377

Table 6.5 describes the tasks associated

with knowledge definition.
Definition Tasks
Table 6.5 Knowledge

Task Objective
Knowledge representation Specify how knowledge will be represented, such as rules,
frames, or logic. Dependent upon what the
tool will support. expert systems
Detailed control structure
Specify three general control structures: (1) if the system is
embedded in procedural code, how it will be called; (2)
control of related
groups of rules within an executing sys-
tem: (3) metalevel control structures for rules.
Internal fact structure Specify the internal structure of facts in a consistent manner
to aid in
Preliminary user interface
understanding and good style.
Specify a preliminary user interface. Get feedback from
users about the interface.
Initial test plan Specify how code will be tested. Define test data, test driv-
ers, and how test results will be analyzed.

The internal fact structures described in Table 6.5 are discussed in much more
detail in the chapters on CLIPS. The basic idea of specifying fact structures is to
adapt good style. For example, a fact such as "10" is not very meaningful by it-
self. What does the "10" represent? If additional information is included with the
fact such as "price 10" or, better still, "gold price 10:" then the gold price is mean-
ingful. Notice that this form of the fact is in conventional object-attribute-value
form and so is convenient for people to read and understand. CLIPS supports this
throughthe deftemplate construct for rules, and also objects.
In some expert systems languages the fields may have strong typing so that
not al-
onlycertain values are allowed. If a rule tries to specify a value that is
detailed design stage of
lowed, the inference engine flags this as an error. The
knowledge is shown in Table 6.6. document
The product of the detailed design stage is the baselined design
document undergoes a
from which coding can proceed. The baselined design
check before coding begins.
knowledge system design review as a final

Table6.6 Detailed Design ofKnowledge Tasks

Task Objective
in the
Specify how knowledge is logically organized
Design structure what is in the knowledge base.
knowledge base and
how the system is to be implemented.
Implementation strategy Specify user interface after receiving user
Detailed user interface Specify the detailed user interface design.
feedback from the preliminary
Document the design.
Design specifications and report the code will be tested and verified.
Detailed test plan Specify exactly how
 Systems
378 Chapter 6: Design of Expert

Code and Checkout

code and checkout
stage, which the act
begins the actual code
Table 6.7 describes the
implementation.

Checkout Tasks
Table 6.7 Code and

Objective
Task
Implement coding.
Coding test data, test drivers, and test analysis
Test code using
Tests procedures.
documented source code.
Produce commented,
Source listings manual so experts and users
Produce working user's
can
User manual feedback on system.
provide
Document installation/operation
of system for users.
Installation/operations guide overall expert system functionality, limita-
System description document Document
tions, and problemns.

test readiness review, which determines if the

This stage terminates with the verification.
system is ready for the next stage of knowledge
expert

Knowledge Verification
of determining the cor
The knowledge verification stage has the objective
of the system. This stage is divided into
rectness, completeness, and consistency
two main tasks:

Formal tests
Test analysis
Table 6.8 describes the formal test task of the knowledge verification stagc

Table 6.8 Formal Test Tasks of the Knowledge Verification Stage

Task Objective
Test procedures Implement formal test procedures.
Test reports Document test results.

looksfor

The test analysis tasks are shown in Table 6.9. The test analysiS T
the following major problems:
incorrect answers
incomplete answers
inconsistent answers
 6.8 Summary 379

and determines if the problem lies in rules, inference chains, uncertainty, or

some combination of these three factors. If the problem cannot be pinned down
to the expert systenm, then it is
necessary to analyze the expert systems tool sort
ware for bugs, which should only be done as a last resort. Unlike other tools,
CLIPS does provide all the source code and manuals so that you can do this.

Tasks
Table 6.9 Test Analysis

Task Objective
Results evaluations Analyze test results.
Recommendations Document recommendations and conclusions of tests.

System Evaluation
As described in Table 6.10, the final stage in the development life cycle is the
evaluation stage. The purpose of this stage is to summarize what has
system
been learned with recommendations for improvements and corrections.

Table 6.10 System Evaluation Stage Tasks

Task Objective
Results evaluation Summarize the results of testing and verification.
Recommend any changes to the system.
Recommendations
Validate that the system is correct with respect to
user
Validation
needs and requirements.
If
Interim or final report If the system is complete, then issue final report. not,
issue an interim report. Ask for more money.

built up in iterations, the report from the

Since expert system is usually
an
in-
will usually be an interim report describing the
System Evaluation Stage added. However, the
as new knowledge is
creased functionality of the system
and also as part of the previous
must be verified by itself
new system capability must also be per-
in the system, That is, the system verification
knowledge knowledge, not just the new knowl-
formed in conjunction with all the systerm time at this stage
the system should also be validated each
edge. Ideally expert the risk of irritating
iteration. However this runs
rather than waiting for the final
the expert.

6.8 SUMMARY
construction
discussed a software engineering approach to the
In this chapter we interview with
of expert systems. Principles
were given about conducting good for
a
are widely used solving
an expert. Now that expert
systems technology
quality be products, especially since
real-world problems, expert
systems muSt
and by credit bureaus.
So many are used in
defense systems