· Introduction

.· , ""

...
PreiiD. ____,_ . . dia LV!?ow@fr® ~O[M]Dfr®~ ·
ew Delhi- 110 001 · .
2002 ' .:
This Thirteenth. lndl•n Reprlnt-Rs. ,50.00
(Original U.S. Edition--As. 3017.00)

INTRODUCTION TO ARTIFICIAL INTELLIGENCE AND ' EXP~RT . SYSTEMS

by Dan W. Patterson .

C 1990 by Prentice-Hall, Inc., Englewood Cliffs, N.J., U.S.A. All rights reserved . No part of' this book
may be reproduced in any form, by mimeograph or any other means. without permission ih writing
from the publisher. · . · · . ' ·

The al,llhor and publisher of this book have used their best elforla In preparing this book. These efforts Include
the development, research, and tesUng of the theories and programo to determine their effecttv.ne .... The 8Uihort
and publisher make no warranty of any kind , ·expre,sed or implied, with regard to these programs or th.,
documentation con,ained in this book. The author and publisher. aha• not be liable in any evenr for incidental 01
consequential damages in connection with, ?' arising out of. the fumilhing. perlormance, or use of these progratT'IS'.
Personal Consultant and Personal Consultant Plus are Regl1tered Trademarks of Texas· Instruments.
Rulemaster Is a Registered Trademark.
Kee 11 a Regi~ered Trademark.

ISBN-81-203-G777-1

The export rights of this book · are vested solely with the publisher.

This Eastern Economy Edition is the authorized, complete and unabridged pholo·offset reproduction
of the latest American edition specially published "nd priced lor sale only in Bangladesh, Burma,
Cambodia, China, Fiji, Hong Kong, India, Indonesia, Laos, MalaYsia. Nepal, Pakistan, Philppines,
Singapore, South Korea, Sri Lanka, Taiwan, ·Thailand, and Vietnam . . ·

Reprinted in India by special arrangement with Prentice-Haft, Inc., Englewood Cliffs, N.J., t).S .A.

Thirteenth Printing Janui.ry, 2002

Published by Asoke K. Ghosh, Prentice·Hall of India PriviJ!fe · ~ed. M-97, Connaught Circus.
New Oe)hi-110001 and .Printed by Mohan Makhijani at Rekha Pr-inters Private Limited,
New Delhi·110020.
 To Neslie
for her generous love and encouragement

..

- •.
 Contents

PREFACE Ku,

Part I Introduction to Artificial Intelligence I

I OVERVIEW OF ARTiFICIAL INTELLIGENCE I
1.1 What isAl? 2
1.2 The Importance of Al 3
1.3 Early Work in Al 5
1.4 Aland Related Fields 7
1.5 Summary 8

2 KNOWLEDGE: GENERAL CONCEPTS 9

2.1 Introduction 9
2.2 Definition and Importance of Knowledge 10
2.3 Knowledge-Based Systems 13

V
 Vi
Contents

2.4 Representation of Knowledge 14

2.5 Knowledge Organization 16
2.6 Knowledge Manipulation 16
2.7 Acquisition of Knowledge 17
2.8 Summary 17
Exercises 17

3 LISP AND OTHER Al PROGRAMMING LANGUAGES 19

3.1 Introduction to LISP: Syntax and Numeric
Functions 19
3.2 Basic List Manipulation Functions in LISP 22
3.3 Functions, Predicates, and Conditionals 25
3.4 Input, Output, and Local Variables 29
3.5 Iteration and Recursion 33
3.6 . Property Lists and Arrays 35
3.7 Miscellaneous Topics 38
3.8 PROLOG and Other Al Programming Languages 40
3.9 Summary 43
Exercises 44

Part 2 Knowledge Representation 47

4 FORMALIZED SYMBOLIC LOGICS 47

4.1 Introduction 47
4.2 Syntax and Semantics for Propositional Logic 49
4.3 Syntax and Semantics for FOPL 55
4.4 Properties of Wffs 60
4.5 Conversion to Clausal Form 62
4.6 Inference Rules 65
4.7 The Resolution Principle 66
4.8 Nondeductive Inference Methods 73
 vii
Contents

4.9 Representations Using Rules 75

4.10 Summary 76
Exercises 77
80
5 DEALING WITH INCONSISTENCIES AND UNCERTAINTIES

5.1 Introduction 81

5.2 Truth Maintenance Systems 82

5.3 Default Reasoning and the Closed World

Assumption 87

5.4 Predicate Completion and Circumscription 90

5.5 Modal and Temporal logics 92

5.6 Fuzzy Logic and Natural Language Computations 97

5.7 Summary 104

Exercises 105
107
6 PROBABILISTIC REASONING

6.1 Introduction 107

6.2 Bayesian Probabilistic Inference 109

6.3 Possible World Representations 113

6.4 Dempster-Shafer Theory 115

6.5 Ad-Hoc Methods 119

6.6 Heuristic Reasoning Methods 122

6,7 Summary 123

Exercises 124

7 STRUCTURED KNOWLEDGE: GRAPHS, FRAMES, AND 126

RELATED STRUCTURES

7.1 Introduction 126

7.2 Associative Networks 127

7.3 Frame Structures 136

7.4 Conceptual Dependencies and Scripts 140

 VIII
Contents
7.5 Summary 144
Exercises 145

B OBJECT-ORIENTED REPRESENTATIONS 147

8.! Introduction 147
8.2 Overview of Object-Oriented Systems 149
8.3 Objects. Classes, Messages, and Methods 150
8.4 Simulation Example Using an OOS Program 155
8.5 Object Oriented Languages and Systems 161
8.6 Summary 164
Exercises 165

Part 3 Knowled'ge Organization and Manipulation 167

9 SEARCH AND CONTROL STRATEGIES 167
9.1 Introduction 17
9.2 Preliminary Concepts 168
9.3 Examples of Search Problems lb
9.4 Uniformed or Blind Search 174
9.5 Informed Search 178
9.6 Searching And-Or Graphs 184
9.7 Summary 185
Exercises .186

10 MATCHING TECHNIQUES 188

10.1 Introduction 188
10.2 Structures Used in Matching 191
10.3 Measures for Matching 194
10.4 Matching Like Patterns 198
10.5 Partial Matching 201
10.6 Fuzzy Matching Algorithms 204
10.7 The RETE Matching Algorithm 205
 Contents
ix
10.8 Summary 209
Exercises 209

11 KNOWLEDGE ORGANIZATION AND MANAGEMENT 211

11.1 Introduction 212
11.2 Indexing and Retrieval Techniques 215
11.3 Integrating Knowledge in Memory 219
11.4 Memory Organization Systems 220
11.5 Summary 225
Exercises 225

Part .4 Perception, Communication, and Expert Systems

227
12 NATURAL LANGUAGE PROCESSING
227
12.1 Introduction 228
12.2 Overview of Linguistics 228
12.3 Grammars and Languages 231
12.4 Basic Parsing Techniques 240
12.5 Sematic Analysis and Representation
Structures 255
12.6 Natural Language Generation 259
12.7 Natural Language Systems 264
12.8 Summary 266
Exercises 267

13 PATTERN RECOGNITION 271

13.1 Introduction 272
13.2 The Recognition and Classification Process 273
13.3 Learning Classification Patterns 277
13.4 Recognizing and Understanding Speech 281
13.5 Summary 282
Exercises 283
K Contents

14 VISUAL IMAGE UNDERSTANDING 285

14.1 Introduction 285

14.2 Image Transformation and Low-Level
Processing 290
14.3 Intermediate-Level Image Processing 299
14.4 Describing and Labeling Objects 304
14.5 High-Level Processing 312
14.6 Vision System Architectures 317
14.7 Summary 323
Exercises 323

15 EXPERT SYSTEMS ARCHITECTURES 326

15.1 Introduction 327

15.2 Rule-Based System Architectures 330
15.3 Nonproduction System Architectures 337
15.4 Dealing with Uncertainty 347
15.5 Knowledge Acquisition and Validation 347
15.6 Knowledge System Building Tools 349
15.7 Summary 354
Exercises 354

Part 5 Knowledge Acquisition 357

16 GENERAL CONCEPTS IN KNOWLEDGE ACQUISITION 357

16.1 Introduction 357

I62 Types of Learning 359
16.3 Knowledge Acquisition Is Difficult 360
16.4 General Learning Model 361
16.5 Performance Measures 364
16.6 Summary 365
Exercises 366
Con xi

17 EARLY WORK IN MACHINE LEARNING 367

17.1 Introduction 367

17.2 Perccptrons 368
17.3 Checker Playing Example 370
17.4 Learning Automata 372
17.5 Genetic Algorithms 375
17.6, Intelligent Editors 378
17.7 Summary 379
Exercises 379

18 LEARNING BY INDUCTION 381

18.1 Introduction 381

18.2 Basic Concepts 382
18.3 Some Definitions 383
18.4 Generalization and Specialization 385
18.5 Inductive Bias 388
18.6 Example of an Inductive Learner 390
18.7 Summary 398
Exercises 399

19 EXAMPLES OF OTHER INDUCTIVE LEARNERS 401

19.1 Introduction 401

19.2 The 1D3 System 401
19.3 The LEX System 405
19.4 The INDUCE System 409
19.5 Learning Structure Concepts 412
19.6 Summary 413
Exercises 414
 xli Contents

20 ANALOGICAL AND EXPLANATION-BASED LEARNING 416

20.1 Introduction 416
20.2 Analogical Reasoning and Learning 417
20.3 Examples of Analogical Learning Systems 421
20.4 Explanation-Based Learning 426
20.5 Summary 430
Exercises 431

REFERENCES 432
INDEX 441
 Preface

A major turning point occurred in the field of artificial intelligence with the realization
that in knowledge lies the power. ' This realization led to the development of a
new class of systems: knowledge-based systems. Knowledge-based systems use spe-
cialized sets of coded knowledge to "reason" and perform limited intelligent tasks.
This is in constrast with more conventional type programs which rely on data and
general algorithms (weak methods) to solve less intelligent tasks. Knowledge-based
systems proved to be much more successful than the earlier, more general problem
solving systems. They proved to be more effective in most areas of Al including
computer vision, natural language understanding, planning, and problem solving
using the newly developed rule-based expert systems.
In concert with the knowledge-base theme, this book is mainly about knowledge
and the role it plays in creating effective Al programs. It focuses on all aspects
of 'knowledge: knowledge representation methods, knowledge acquisition tech-
niques, knowledge organization, and knowledge manipulation. It illustrates the basic
knowledge-system approach and emphasizes the important use of knowledge in
such systems.
This book was written as a text for my classes in artificial intelligence at the
University of Texas at El Paso. These classes are for upper division undergraduate
and first year graduate students. The courses assume prerequisites of basic computer
science courses (like programming languages) and a general maturity in mathematics.

xl"
 Preface
xlv

The material may be used as a one semester survey course in Al or as a two

semester sequel with basic Al principles and tools being taught the first semester
and special topics such as vision, natural language understanding, machine learning,
and expert systems architectures taught the second semester.
The book is comprehensive in its coverage of all the important topic areas of
Al. and no particular bias is given to any special area or approach. The treatment
of knowledge acquisition and machine learning is much more comprehensive than
that found in other introductory texts on Al. And computer vision, natural language
processing, and pattern recognition are also covered in some depth. A significant
part of the text is devoted to the important topics of knowledge representation,
including methods of dealing with uncertain, incomplete, and vague knowledge
(such as methods related to nonmoflotonic lcgics and commonsense reasoning).
Currently, there is a debate being waged among artificial intelligence practi-
tioners over the best approach to Al computations: the neural network approach
vs. the symbolic computation approach. We recognize the importance of this debate
because the future direction of Al will be determined by its outcome. But whatever
the outcome, the successes of symbolic computation on knowledge structures suggest
that this approach will last for some time to come. Because of that, most of the
text has been devoted to this approach. Even so, the recent successes of the biologi-
cally inspired neural network approach suggests that there is an important place in
Al for systems based on these paradigms. Consequently we have included introduc-
tory material on this important subject as well.
This book is about the differenç areas of study which make up the field of
Al. abqut the techniques and tools of Al. and about the products Al has produced.
The book is also about knowledge, an essential component of Al. For this reason,
the material has been organized around knowledge and the roles it plays in each of
the component areas of study.
The book has been divided into five parts or general topic areas related to
'knowledge: Introduction to Artificial Intelligence, Knowledge Representation,
Knowledge Organization and Manipulation. Perception and Communication, and
Knowledge Acquisition.
Part I is a general introductory section composed of three chapters. Chapter
I presents a general overview of Al in which the importance of the field is discussed,
sor,ie important terms are introduced, and a brief summary of early work is presented.
This is followed with a chapter which defines knowledge, what it is, and the important
roles it plays in Al afid in the development of knowledge-based systems. Chapter
3 offers a concise summary of the most important programming languages used by
Al practitioners, with particular emphasis on LISP.
Part II covers the important areas of knowledge representation. It consists of
five chapters. Chapter 4 presents the important topics of propositional and first
order predicate 'logics. An area that has come to play a preeminent role in Al.
Chapter 5 discusses problems and solutions for the representation of inconsistent
and uncertain knowledge. Chapter 6 continues this theme with a treatment of fuzzy
and modal logic. In Chapter 7, structured representation schemes are introduced
Preface xv

with the notion of associative networks, conceptual graphs, and frames. Chapter 8
completes Part II with an introduction to systems which are based on object oriented
represefltaUOfl structures.
Part Ill covers topics related to the organization and manipulation of knowledge.
This part contains three chapters. Chapter 9 discusses the important problems associ-
ated with search and control. Chapter 10 presents a comprehensive treatment of
matching techniques, an essential function of most Al programs. This part concludes
with Chapter II which covers memory organization and management techniques.
Part IV contains three chapters related to perception and Communication. The
first chapter. Chapter 12 covers the subfield of natural language processing. Although
only a single chapter has been devoted to this subject, the treatment is thorough.
Chapter 13 presents a condensation of important topics from pattern recognition.
Chapter 14 presents a comprehensive treatment of the important topic of computer
vision. And, Chapter 15 has an introduction to Expert System architectures and
related topics.
Part V. the final section, presents an up-to-date, comprehensive view of knowl
edge acquisition/machine learning. All of the in-iportmnt learning paradigms are cov-
ered in this part. Chapter 16 begins with general concepts related to knowledge
acquisition. This is followed in Chapter 17 with a summary of early work in machine.
learning. Chapter 18 introduces inductive learning concepts and presents a detailed
example of an inductive learning system. Chapter 19 continues inductive learning
with examples of recent systems. Chapter 20, the final chapter, covers analogical
and explanation-based learning paradigms.
We hope the reader will experience many enjoyable and rewarding sessions
reading from the exciting material to be found in the text.

ACKNOWLEDGMENTS

In writing this text, a number of individuals have been helpful with their suggestions
and comments. They include the following students: Teow Kiang Chew, Teck Huat
Goh. Julie Lemen, Sergio Felix, Ricardo Martinez, Vincente Fresquez, 1-tun-Ming
Hsu, Rudy Velasquez, and Jose Najera-Mora. Special thanks are given to E. Louise
(Neslie) Patterson for proofreading most of the manuscript and offering many useful
suggestions. Thanks are also given to the following reviewers for their valuable
suggestions: Christopher K. Carlson, George Mason University; Daniel Chester,
University of Delaware; Karen L. McGraw, Cognitive Technologies; and Gordon
Novak, University of Texas, Austin. Finally, I wish to thank the Electrical Engineer-
ing and Computer Science Department of the University of Texas at El Paso for
the generous use of their facilities.