Explanation Patterns

Understanding Mechanically and Creatively

ROGER C. SCHANK
Yale University

LAWRENCE ERLBAUM ASSOCIATES, PUBLISHERS

1986 Hillsdale, New Jersey London

Copyrighted Material
Copyright © 1986 by Lawrence Erlbaum Associates, Inc.
All rights reserved. No part of this book may be reproduced in
any form, by photostat, microform, retrieval system, or any other
means, without the prior written permission of the publisher.

Lawrence Erlbaum Associates, Inc., Publishers

365 Broadway
Hillsdale, New Jersey 07642

Library of Congress Cataloging in Publication Data

Schank, Roger C., 1946-
Explanation patterns.

Bibliography: p.
1. Artificial intelligence. 2. Cognition.I.Title.
Q335.S3863 1986 006.3 86-19872
ISBN 0-89859-768-4

Printed in the United States of America

10 9 8 7 6 5 4 3 2

Copyrighted Material
 Contents

Preface v
Chapter 1: The Explanation Game 1
Chapter 2: What Needs To Be Explained 25
Chapter 3: Explanation Goals 51
Chapter 4: The Process of Explanation:
Explanation Questions 80
Chapter 5: The Process of Explanation:
Explanation Patterns 109
Chapter 6: Creativity Through the
Misapplication of Explanation Patterns 140
Chapter 7: Question-Driven Understanding 178
Chapter 8: Some Final Questions 208
Appendix: SWALE: A Program That Explains 232
References 255

iii

Copyrighted Material
Copyrighted Material
 Preface

In the age of the computer, conjecture about things mechanical has natu-
rally led to the question of whether machines can think. As the emphasis
on Artificial Intelligence (AI) has grown rapidly, questions about machine
intelligence have begun to have a certain urgency. There is, at the same
time, an opposite, almost whimsical type of question that arises when one
begins to consider whether machines can think, namely, Can people think?
People seem willing enough to grant machines the possibility of doing
calculations of tremendous complexity with utmost ease. They seem willing
to believe that machines will be able to talk, walk, hear, and evaluate, but
when it comes to the possibility that machines might be creative, on their
own, most people deny the possibility? Why? What magic power do we
believe people to possess that we also believe machines to be inherently
incapable of possessing?
This is why the question of whether machines can think is actually one of
whether people can think, or, to put this another way, if the present pursuit
of algorithms that describe the process of thinking in sufficient detail
actually enables a machine to emulate that process, we can expect, based
on past reactions to machine intelligence, that people will begin to worry
about the nature of their own humanness. If a machine can do something
"human," then that "something" has been reduced to a mere mechanical
process that no longer seems worth doing.
The question we are concerned with in this book is: If we can find a set
of processes that machines can slavishly follow, and if by so doing, these
machines can come up with creative thoughts, what would that tell us
about human beings? If the machine's procedure was adapted from a
v

Copyrighted Material
vi PREFACE

human procedure, that is, if all the machine was doing was what we know
people are doing, would we abandon our inherent skepticism about the
abilities of machines, or would we demystify our inherent admiration for
things human?
In a sense, these are the issues I deal with in this book. I say "in a sense"
because this book is no way a philosophical treatise. Rather it is an exercise
in Artificial Intelligence and in Cognitive Science. It is an attempt to come
to understand one of the most complex problems of mind by examining
some of the mechanisms of mind: to define the apparatus that underlies
our ability to think. In other words, this book is about people. But, it is not
only about people, it is about simulating people doing what they consider
to be their most human-specific act, being creative. Thus, it is also a book
about machines and machine-capabilities.
Why discuss creativity at all? Why, in an era of great difficulty in getting
machines to do the simplest understanding tasks, should one even begin to
consider creativity, certainly one of the most complex issues of all? My
premise is that the simplest mental tasks and the most complex of mental
tasks are all interrelated. Understanding the simplest sentence is a creative
act. Seen as a simple process involving merely the syntactic parsing of a
sentence, it is hard to see where the creativity might lie. But, understanding
a sentence also involves relating what you have understood to what you
already knew, and that task involves memory search. Memory search, as I
attempted to show in Schank (1982) involves learning since memory is
never quite the same as it was the last time it was used and will not be the
same after the new sentence is processed. And, learning involves creativity
since it isn't always so clear what should be learned. All these things are
really part of the same process. Thus it follows that AI will never exist in
any important way until machines are capable of some basic creativity.
This premise has been at the base of most of the work being done at the
Yale AI laboratory in recent years. I wish to thank my graduate students
who have helped me formulate these ideas by serving as a sounding board,
by writing programs based on these ideas, by revising some of these ideas
with their own better worked out ones, and by reading and commenting on
the manuscript. I also wish to thank the Advanced Research Projects
Agency of the Department of Defense, the Air Force Office of Scientific
Research, the Office of Naval Research, and the National Science Founda-
tion for its support of this work.

Copyrighted Material
 To Joshua
who is more creative
than today's computers

Copyrighted Material
Copyrighted Material
 1 The Explanation Game

THE TURING TEST

The question of whether machines can think is perhaps a bit tired at this
point in the history of computing, but the question of what would be
required in order to make machines think is just beginning to be explored.
Of course, each question relates to the other, so, in order to begin the
process of building a thinking machine, we must consider not only if such a
project is possible, but also what elements of the human thinking process
are inextricably bound up with our sense of what we mean when we talk
about a thinking machine. To build a machine with Artificial Intelligence,
we must come to understand, in a profound way, what it means to have
intelligence. For years, researchers in AI could content themselves with
adding yet another cute feature to their programs, allowing machines to do
new things, one by one. But, it is becoming increasingly clear that to really
make intelligent machines, as opposed to machines that exhibit one or two
aspects of intelligence, one must attack the basic issues of the nature of
human thought and intelligence head on.
There have, of course, been many discussions of machine thought, by far
the most famous of these by the British mathematician, Alan Turing
(Turing, 1963), when computers were in their infancy. Turing proposed a
test, which he called the Imitation Game, that has become a common but
often misunderstood way of judging the ability of a machine to understand.
Turing's major argument was that the question of whether a machine can
think is meaningless. He suggested the following alternative: If a person
failed to distinguish between a man imitating a woman (via teletype) and a
1

Copyrighted Material
2 1. THE EXPLANATION GAME

computer imitating a man imitating a woman, then the machine succeeded

in the Imitation Game. Turing argued that success in this game was, in the
long run, inevitable.
What Turing actually said, and what has been made of it, have not always
been the same thing. He did not say that machines are thinking if they
played the Imitation Game successfully. He merely said that successful
imitation was the real issue for computer scientists. In the subsequent
history of Artificial Intelligence, two general lines of argument on this issue
have developed. One has been that machines will never duplicate human
thought processes; the other is that no matter how well machines imitate
human behavior, they cannot be said to truly understand in the same way
as a human being. To put this another way, the claims made are first, that
the Turing test will never be passed; and second, that even if it is passed, it
does not prove that machines understand. Of course, Turing himself disagreed
with the first claim; he regarded the second one as meaningless.
It is now more than thirty years since Turing's paper. With the exception
of Kenneth Colby (Colby, 1975), a psychiatrist who found that other
psychiatrists could not tell his computer version of a paranoid from the real
thing, no researcher has claimed that he has created a program that would
pass the Turing test. And, no matter how many times people have affirmed
that Turing was correct in his assessment of the validity of the question, it
has failed to go away. Further, there is little reason to believe that it will go
away. No achievement in building intelligent software can dispel it because
there are always those who believe that there is something more to the
nature of intelligence than any amount of software displays.
Critics of Artificial Intelligence seem always to be able to "move the
line" that must be crossed. "It would be intelligent if it did X" is nearly
always followed by "No, that was wrong; it would be intelligent if it did Y"
when X has been achieved. But the problem of the ultimate possibility of
an intelligence that is different from our own and possibly even superior to
our own, embodied in something not made of flesh and blood, will not be
determined by critics with arbitrary standards. It is time to look again at
the twin questions of the criteria for the ultimate thinking ability of com-
puter programs and the nature of what is meant by understanding.

ON MEN AND WOMEN

One of the interesting, if not serendipitous, facets of Turing's Imitation

Game is that, in its initial conception, the problem is to find the difference
between a man and a woman via teletype. The seeming implicit assump-
tion is that there are differences between men and women that are discernable
by teletype. On the other hand, given that the problem is merely to get the

Copyrighted Material
 ON MEN AND WOMEN 3

computer to do as well at the Imitation Game as the man did, and assuming
that there is no difference between men and women that would be recogniz-
able via teletype, the task of the machine is to duplicate a human in its
answers. Thus, Turing's test doesn't actually depend upon men and women
being discernably different. But, the nature of what it means to understand
may best be illustrated by that distinction.
To see what I mean, let us consider the question of whether men can
really understand women, (or alternatively, whether women can really
understand men). It is common enough in everyday experience for men
and women to both claim that they really do not understand their opposite
number. What can they mean by this? And, most importantly, how is what
they mean by it related to the problem of determining whether computers
can understand?
When the claim is made that men and women are really quite different
(mentally, not physically), what is presumably meant is that they have
different beliefs, different methods of processing information, different
styles of reasoning, different value systems, and so on. (It is not my point
here to comment on the validity of these assertions. I am simply attempting
to use the principle of these assertions in my argument. These same
assertions might be made about different ethnic groups, cultures, nations
and so on; I am simply using Turing's domain.)
The claim that I assume is not being made by such assertions is that men
and women have different physical instantiations of their mental processes.
(Of course, it is possible that men and women do have brains that differ
physically in important respects, but that would be irrelevant for this
argument.) So, what is it that makes men and women feel they have difficulty
understanding each other? Empathy. Understanding involves empathy. It is
easier to understand someone who has had similar experiences—and who,
because of those experiences, has developed similar values, beliefs, mem-
ory structures, rules-of-thumb, goals, and ideologies—than to understand
someone with very different types of experiences.
Understanding consists of processing incoming experiences in terms of
the cognitive apparatus one has available. This cognitive apparatus has a
physical instantiation (the brain, or the hardware of the computer) and a
mental instantiation (the mind, or the software of the computer). When an
episode is being processed, a person brings to bear the totality of his
cognitive apparatus to attempt to understand it. What this means in prac-
tice is that people understand things in terms of their particular memories
and experiences. People who have different goals, beliefs, expectations and
general life styles will understand identical episodes quite differently.
Therefore, no two people understand in exactly the same way or with
exactly the same result. The more different people are from one another,
the more their perception of their experiences will differ. On the other

Copyrighted Material
4 1. THE EXPLANATION GAME

hand, when people share certain dimensions of experience, they will tend
to perceive similar experiences in similar ways. Thus, men tend to under-
stand certain classes of experiences in ways that are different from women.
It is unlikely that an experience that in no way bears upon one's sex will
be understood differently by men and women. Recall that the assumption
here is that the baseline cognitive apparatus is the same regardless of sex.
Any experience that does relate to the sex of the observer in some way will
be processed differently. This can involve obvious issues, such as the
observance of an argument between a man and a woman. There, we would
expect a man to observe the episode from the point of view of the man and
a woman to observe it from the point of view of the woman. In addition,
such identification with different characters in a situation can extend to
observations of situations where the feuding characters are of the same sex,
but one displays attributes more traditionally male and the other displays
traditional female behavior. Identification, and thus perception, can thus
be altered by one's understanding of the goals, beliefs, or attitudes underly-
ing or perceived to underlie the behavior of the characters in an episode
one is observing.
Thus, for example, one's perception of the validity and purpose behind a
war can be altered by whether one is the mother of a son who is about to be
drafted or whether one previously fought in a war and found it an enno-
bling experience. In general, one's sense of what is important in life affects
every aspect of one's understanding of events.
The claim then, is that men and women, as examples of one division of
human beings, do not, and really cannot, understand each other. The same
argument can be put forward, with more or less success, depending upon
the issue under consideration, with respect to Arabs and Israelis or intellec-
tuals and blue collar workers. In each of these cases, differing values can
cause differing perceptions of the world.

COMPUTERS AND PEOPLE

Now let us return to the question of whether computers can understand.

What exactly does it mean to claim that an entity—either a person or a
machine—has understood? On the surface, it appears that there are two
different kinds of understanding to which people commonly refer. We talk
about understanding another human being, or animal, and we talk about
understanding what someone has told us or what we have seen or read.
This suggests that there are actually two different issues to confront when
we talk about computers understanding people. One is to determine whether
computers will ever really understand people in the deep sense of being
able to identify with them or empathize with them. The other is whether

Copyrighted Material
 COMPUTERS AND PEOPLE 5

computers will be able to comprehend a news story, interact in a conversa-

tion or process a visual scene. This latter sense of understanding comprises
the arena in which AI researchers choose to do battle. Often the critics of
AI (e.g. Weizenbaum, 1976; Dreyfuss, 1972) choose to do battle in the
former.
In fact, these two seemingly disparate types of understanding are really
not so different. They are both aspects of the same continuum. Examining
both of them allows us to see what the ultimate difficulty in AI is likely to
be and what problems AI researchers will have to solve in order to create
machines that understand.
Weizenbaum claims (Weizenbaum, 1976) that a computer will never be
able to understand a shy young man's desperate longing for love, expressed
in his dinner invitation to the woman who is the potential object of his
affection, because a computer lacks experience in human affairs of the
heart. In some sense this seems right. We cannot imagine that machines
will ever achieve that sort of empathy because we realize how difficult it is
for people who have not been in a similar situation to achieve that level of
understanding. In other words, Weizenbaum's statement is, in some sense,
equivalent to saying that no person can understand another person without
having experienced feelings and events similar to those one is attempting to
understand. Where does this leave the poor computer? Where does this
leave a child? Where does this leave a happily married man who met his
wife when they were both small children and married her when they
graduated high school, before he ever had to cope with asking her out to
dinner?
Weizenbaum is correct as far as he goes, but he misses a key point. Under-
standing is not an all-or-none affair. People achieve degrees of understanding
in different situations, depending upon their level of familiarity with those
situations. Is it reasonable to expect a level of empathy from a machine that
is greater than the level of empathy we expect from human beings?
The important point for researchers in AI, psychology, or philosophy, is
not whether machines will ever equal humans in their understanding
capabilities. The important scientific questions are about people, not
computers. What processes does a person go through when he is attempting
to understand? How does our recognition that understanding involves
empathy and the ability to relate and draw upon common experiences help
us to better understand the nature of mental processes? It is this issue that
is of primary importance.
So, there are really three main questions:

How can we understand enough about the nature of what it

means to understand so that we can make the issue of compre-
hension, by people or computers, more concrete?

Copyrighted Material
6 1. THE EXPLANATION GAME

How can we get a better grasp on exactly how particular experi-

ences affect the understanding process?
What abilities must a machine have before it could possibly be
considered to be intelligent?
This book is about the second and third of these questions, but this
chapter is about the first. Understanding involves a multiplicity of types
and levels of understanding; and different people understand things
differently. These statements are obvious and seemingly well-known. But in
reality, AI research, as well as criticism of that research and expectations
about the progress of that research, has proceeded as if those facts were
hardly even viable possibilities. Explaining how a better understanding of
what is involved in being intelligent can change our view of what AI is, and
perhaps even what psychology should be, is the goal of this book.

THE NATURE OF UNDERSTANDING

The easiest way to understand the nature of understanding is to think of it

in terms of a spectrum. At the far end of the spectrum we have what I call
COMPLETE EMPATHY. This is the kind of understanding that might
obtain between twins, very close brothers, very old friends, and other such
combinations of very similar people.
At the opposite end of the spectrum we have the most minimal form of
understanding, which I call MAKING SENSE. This is the point where
events that occur in the world can be interpreted by the understander in
terms of a coherent (although probably incomplete) picture of how those
events came to pass.
Now let us step back for a moment. Before we complete this spectrum, it
would be worthwhile to discuss both what the spectrum actually represents
and what the relevance of current AI research is to this spectrum.
There is a point on this spectrum that describes how an understander
copes with events outside his control. The end points of this spectrum can
be loosely described as, on the one hand, the understander saying to
himself, yes, I see what is going on here, it makes some sense to me, and, on
the other hand, his saying, of course, that's exactly what I would have done,
I know precisely how you feel.
In our recent research (for example, Schank & Riesbeck, 1981; Schank,
1982) we have been concerned with the nature of understanding because
we are trying to get computers to read and process stories. In the course of
that research, we have considered various human situations that we wished
to model. We built various knowledge structures, that attempt to character¬
ize the knowledge people have of various situations. The restaurant script,

Copyrighted Material
 THE NATURE OF UNDERSTANDING 7

for example (Schank & Abelson, 1977), was used in an attempt to understand
restaurant stories, and it was that script which prompted Weizenbaum's
criticism about understanding love in a restaurant. Since that earlier research
we have come to realize that these knowledge structures function best if
they are dynamic. That is, they must be able to change as a result of new
experiences. In other words, we expect that as knowledge is used, it
changes. Or, to put this another way, as we undergo experiences we learn
from them.
At the core of such an hypothesis is the notion that in attempting to
understand we are attempting to relate our new experiences to our prior
experiences by utilizing knowledge structures that contain those previous
experiences. Consider, for example, the following situation. Imagine that
you are hungry and that someone suggests a marvelous restaurant to you
called Burger King. You happily go to this restaurant, armed as you always
are, with a set of expectations about what will happen there. Specifically,
you expect that you will: ENTER; BE SEATED; GET & READ MENU;
ORDER; BE SERVED; EAT; PAY; and EXIT. The items written in bold
face are called scenes and can be best understood as bundles of expecta-
tions themselves that concern the particulars of how those scenes will
actually take place. The assertion put forth in Schank (1982) is that such
scenes are derived from experience and are subject to constant change by
experience.
You were told that you were going to a restaurant, so you brought out your
Memory Organization Package (or MOP) for restaurants (M-RESTAURANT)
that told you what scenes to expect. What you find in Burger King,
however, is a different sort of thing altogether. The order of scenes is
ENTER; ORDER (without a menu exactly); PAY; SERVE (but it doesn't
look much like the other SERVE); BE SEATED (but on your own); EAT;
LEAVE. So what is a processor to do?
The obvious thought is that you have been fooled. This is not a
restaurant at all. Or maybe this is just a weird restaurant. Without
other evidence, it is hard to know what to do. What a processor can
do is mark the expectation failures. That is, we expected that scenes
would come in a certain order, and we expected that scenes would
proceed in a certain manner, but they didn't. What we do in this case
is to index the exceptions so that we will be able to recognize them when
they occur again.
Now suppose that after you complain to your friend about Burger King,
he suggests that you try another restaurant, called McDonald's, instead.
You are confronted with a new situation which you must attempt to
"understand." And it is reasonable to say that you have understood
McDonald's if you recall the Burger King experience (in other words, are
reminded of Burger King) and use it to help create expectations about what

Copyrighted Material
8 1. THE EXPLANATION GAME

will happen next. The key point, however, is what happens to your memory
as a result of these two experiences.
You have now encountered two exceptions to a MOP that were them¬
selves quite similar. It is reasonable, therefore, to create a new MOP that
has expectations in it that correspond to these new experiences. We might
call that new MOP, M-FAST-FOOD. We can index this MOP as an excep-
tion to M-RESTAURANT so that it will be available to change and grow
as new experiences relevant to it occur.
The important point here then is that when we are reminded of some
event or experience in the course of undergoing a different experience, this
reminding behavior is not random. We are reminded of this experience
because the structures we are using to process this new experience are
the same structures we are using to organize memory. Thus, we cannot
help but pass through the old memories while processing a new input.
There are an extremely large number of such high level memory struc-
tures. Finding the right one of these (that is, the one that is most
specific to the experience at hand) is one of the things that we mean by
understanding.
In other words, an important part of what we mean by understanding is
the accessing of extant knowledge structures in which to place our new
experiences. We feel that we have understood when we know where a new
input belongs in memory. Sometimes understanding is more profound than
that, however. The creation of new knowledge structures in memory is also
part of understanding. Such structures are created in terms of old ones.
The more these new structures differ from the old, the more complicated
understanding can be.
In this view then, understanding is finding the closest higher level structure
available to explain an input and creating a new memory structure for that
input that is in terms of the old closely related higher level structure.
Understanding is a process that has its basis in memory then, particularly
memory for closely related experiences accessible through reminding and
expressible through analogy. Further, the depth of understanding will
increase if there are many available relevant personal experiences in terms
of which inputs can be processed. Lastly, understanding means finding
some memory, any memory at all sometimes, that will help one cope with
an experience that initially seems novel. We want to feel that we have
understood, and we feel that way to the extent that we can believe that
what we have just seen really is like something else we have seen previously.
With this definition of the nature of understanding then, let us now
return to our understanding spectrum.

Copyrighted Material
 THE SPECTRUM OF UNDERSTANDING 9

THE SPECTRUM OF UNDERSTANDING

COMPLETE EMPATHY exists when individuals have many shared experi-

ences that have in turn created very similar memory structures. The
consequence of this is that, given a set of similar goals and beliefs, new
episodes would be processed in the same way. The above caveat is very
important. Similar experiences, but different goals and beliefs, would still
result in differing perceptions of the events, or to put it another way, in a
lack of COMPLETE EMPATHY in understanding of each other's actions.
The point to be made about the understanding spectrum then, is: The
more that goals, beliefs, and prior experiences and memories are shared, the
more complete the level of understanding that can take place. On the
opposite end of the spectrum, MAKING SENSE involves finding out what
events took place and relating them to a perception of the world that may
be quite different from that in the mind of the actor in those events.
Let us now consider what may well be a midpoint on the understanding
spectrum. We discussed earlier the problem of men and women understand-
ing each other, in general. The point there was that despite a cognitive
apparatus that was identical, something was preventing complete under-
standing. This mid-point I label COGNITIVE UNDERSTANDING. By
this I mean that although a man may be able to build an accurate model of
a given woman, he may still not really understand what her motivations,
fears, and needs are. That is, he lacks COMPLETE EMPATHY with her,
but he still understands a great deal about her. To claim that he doesn't
understand her can only mean understanding in its deepest sense. Certainly
by any measure of understanding less than COMPLETE EMPATHY he
could rightly claim to understand what she does and not be accused by
Weizenbaum of failing the understanding test.
Thus, there are obviously many different kinds of understanding. Where
do computers fit in? The claim I want to make is that, given a spectrum as
we have described:
MAKING SENSE-COGNITIVE UNDERSTANDING-COMPLETE EMPATHY

today's work on computer understanding can only reasonably claim the left
half of the spectrum as its proper domain. It is a legitimate argument that
computers will never understand, if what is meant by that is that they
would be unlikely to understand much at the right hand side of the
spectrum. Computers are not likely to feel the same way as people do, are
not likely to have the same goals as people do, and will never, in fact, be
people. Given that most people fail to understand each other at points on
the right hand side of this spectrum, it seems no great loss to admit to the
likely failure of computers to achieve this level of understanding. Com-

Copyrighted Material
10 1. THE EXPLANATION GAME

puters are unlikely to do better than women at understanding men or vice

versa.
On the other hand, computers can and will share some experiences with
people. They may be able to read newspapers and build models of behavior
almost as well as people do. And, perhaps most important (with respect to
what follows in this book, in any case), computers should be capable of
explaining unforeseen events in terms of their own experiences, even to the
point of being creative in their interpretation of new events. In order for
them to do this, we will have to learn a great deal more about what it means
to understand.
In reading a news story about terrorism, just because the terrorist and I
share the feature of being human (and even perhaps the same age and sex),
it does not follow that I will understand the story better than a computer. It
seems safe to say that I can understand such stories better than a computer
can, as of now at least, but this is due to my experience with such stories
and my general level of interest in human affairs, both political and
sociological. If a computer were to attain a similar level of background
knowledge and interest, it too would be able to understand such stories.
The real question is: Is it possible, and if so, exactly how would one go
about, getting a machine to have such knowledge, cause the interaction of
new knowledge with old knowledge to generate the need to know more,
and use that need to know to guide its processing of news stories?
This is only one kind of processing, however, and is thus really an
example of only one kind of understanding. Another kind of understanding
is shown by what we have already done with respect to computers and
newspapers. Cullingford (1978) in SAM, DeJong (1979) in FRUMP, Lebowitz
(1980) in IPP, and Lytinen (1984) in MOPTRANS, all demonstrated com-
puter programs that could process news stories at various levels. SAM
summarized, answered questions, and translated a variety of fairly simple
news stories. FRUMP did a more superficial analysis of a wider range of
stories, providing summaries of the key points. IPP did an in-depth analysis
of stories in one domain (terrorism), attempting to modify its memory and
make generalizations from what it read. MOPTRANS also processed terror-
ism stories with an eye towards translating them from any of three or four
languages into any other of those languages.
Did these programs understand? Certainly they performed some tasks
that humans perform when they are said to be understanding. At the level
of understanding measured by unsophisticated language comprehension
tests, they understood in the sense that they could answer the who, what,
where, and when questions. Is this understanding? Or, to really understand
do we have to be much more sophisticated in our analysis of a story?
One things seems clear: understanding is not a yes or no, all or none
affair. Granting that there are many levels to understanding, the key

Copyrighted Material
 THE REVISED TURING TEST 1 1

question for AI is not whether the levels we have already achieved can be
construed to be examples of understanding, but rather how we can achieve
the levels of understanding that are still eluding us. MAKING SENSE has
been achieved by computers in certain domains. On the other side of the
spectrum, it may well be the case that computers cannot, in principle,
completely understand people, any more than men can completely under-
stand women or vice versa. COMPLETE EMPATHY is just an end point
on the continuum. There is a lot of space in between the end points. The
real issue is not whether computers can understand, but how to make them
understand better. To do this, we must get a better picture of what it means
to understand.
How different are these three points on the spectrum in terms of what
processes would be necessary in order to construct machines that met
those criteria? What does a machine do in order to achieve the ability to
MAKE SENSE, and what would it have to do in order to COGNITIVELY
UNDERSTAND or show COMPLETE EMPATHY?

THE REVISED TURING TEST

Turing's Imitation Game has not left everyone in AI thrilled by the pros-
pects of having to meet its criteria as a measure of success. Colby (1973)
argued that his paranoid simulation program did indeed pass Turing's test.
He found that psychiatrists who were presented with output from Colby's
PARRY program and output in a similar form from an actual human
patient, were unable to effectively distinguish between them. This passing
of the Turing test failed to convince AI people that Colby's program
understood or thought, nor should it have done so. Despite whatever
validity Colby's program might have as a model of paranoia, it would seem
that the failure of experts to distinguish between imitations and the real
thing should not be taken as much more than a statement of the compe-
tence of the experts. In fact, in the case of the psychiatrists, the Imitation
Game was a particularly poor test since a psychiatrist's choices upon facing
a non-normal patient are not that extensive. PARRY was not brain-damaged
or schizophrenic, so "paranoid" was, given the presence of a few paranoid
signs, a reasonable diagnosis. What Colby seems to have done is effectively
simulated various aspects of the output of a paranoid, which may or may
not reflect accurate processes underlying the production of that output.
The issue then, is one of the distinction between good output and good
simulation.
Ideally, the test of an effective understanding system, if I may use that
word, is not the realism of the output it produces, as Turing would have it,
but rather the validity of the method by which that output is produced.

Copyrighted Material
12 1. THE EXPLANATION GAME

Unfortunately, we cannot create a test that depends upon the evaluation of

methods for two reasons, one practical and one philosophical. The practi-
cal reason is that it is difficult to cut open either the machine or the human
to see what is going on inside. Examination of hardware doesn't tell us
much anyway. From a software point of view, people are extremely hard to
test in a controlled experiment, given the wide range of their possible
experiences prior to that experiment. Programs, on the other hand, are
hard to evaluate with respect to establishing exactly what claims they are
making.
The philosophical point is that we do not examine the innards of people
in order to establish that they are understanding. It might seem that we
grant them an ability to understand based on their humanity. But, it often
happens, in schools, offices, and other places that do such evaluations, that
we do assess a human's ability to understand. It seems sensible therefore,
that the methods that are good for the human are also good for the
computer.
To see what I mean here, let's consider the spectrum of understanding
again. For the sake of argument, assume that it is possible, in theory, to
produce a program that meets the output requirements of each of the three
levels of understanding noted earlier. To make this argument more concrete,
I list here some possible outputs for each of these points on the spectrum:
MAKING SENSE
input: news from the UPI wire
output: a summary of a newspaper story—a translation of a
speech into another language
COGNITIVE UNDERSTANDING
input: a set of stories about airplane crashes, complete with
data about the airplanes and the circumstances
output: a conclusion about what may have caused the crash
derived from a theory of the physics involved and an under-
standing of the design of airplanes, used in conjunction with
algorithms that can do creative explanation
COMPLETE EMPATHY
input: I was very upset by your actions last night.
output: I thought you might have been, it was a lot like the way
you treated me last week.
input: But I meant you no harm.

Copyrighted Material
 THE REVISED TURING TEST 13

output: Do you remember the way your father used to treat you
on holidays when he made you call all your relatives? He
meant no harm either.
input: I see what you mean.
output: I thought you might; there's no friend like an old friend.
Assuming that these input/output pairs are not entirely fanciful, I would
now like to draw some conclusions about them that are reflective of my
view of what a reasonable test should comprise. My conclusions are effectively
summarized with the following words:
ACCURACY; SURPRISE; EMOTION
The claim I am making is, to the extent that output is an effective way of
characterizing degree of understanding (although that is to a very limited
extent indeed, it may well be our only choice), we can judge the signifi¬
cance of that output in terms of its place on the understanding spectrum
with respect to the following features:
The extent that that output accurately accomplishes a task that a
competent human could do.
The extent that that output characterizes an especially original or
important result that most humans cannot easily accomplish.
The extent that that output effectively seems to replicate a real
live human being with whom someone is familiar.
The above three standards reflect the essence of the three points on the
understanding spectrum that I have been discussing.
Turing's Imitation Game then, can be seen, as a result of further insights
into AI since the time that Turing first concerned himself with these issues,
to be somewhat outdated. Merely fooling people is not the true task in AI.
Rather we must, in addressing fundamental questions about the nature of
intelligence, devise a set of requirements that any intelligent entity ought to
be able to meet. And, if we are talking about intelligent beings with
linguistic capabilities, any test we devise can have an added level of
complexity. That is, the true value of language in an intelligence-testing
framework is that we can transcend the limitations of only being able to get
solutions to problems or answers to questions. Linguistically equipped
intelligent beings can explain their own actions.

Copyrighted Material
14 1. THE EXPLANATION GAME

THE EXPLANATION TEST

In the end, any system, human or mechanical, is judged on its output. We

do not take apart a human to look at his insides in an effort to establish that
his understanding mechanisms are of the right sort. Nor is it clear what the
right sort of mechanisms are. We are faced with a dilemma then. We cannot
use output to tell us if a system really understands. On the other hand,
output is all we can reasonably expect to get.
To resolve this dilemma we must address the question of self-awareness.
The issue of consciousness is a vast one and I do not mean to address it
here. Rather, I wish to claim that the fundamental difference between a
system that can produce reasonable output and one that meets the crite¬
rion that is implicit in the term understanding system is that an understand¬
ing system should be able to explain its own actions.
One might ask: explain its own actions to whom? There are two answers
to this question, each of them quite important in their own way. The first
answer is, of course, to those who inquire about how the system works. The
second answer is, to itself. It is this latter answer that represents the essence
of the understanding process, although, quite naturally, it is the latter
answer which is also the most difficult to assess.
A system that not only does interesting things, but can explain why it did
them can be said to understand at any point on the understanding spectrum
where that explanation is sufficient. In other words, being able to say what
sequences of actions were followed in a chain of reasoning passes the
explanation test at the level of making sense. SHRDLU (Winograd, 1972),
MYCIN (Shortliffe et al., 1973) and other programs in AI therefore, can be
said to understand at the level of making sense since they can explain why
they did what they did to the extent of being able to describe the steps that
they went through in performing their respective tasks.
SHRDLU manipulated blocks in a blocks world by responding to Eng-
lish commands. One of the most impressive aspects of that program was
that when asked why the program had performed any given action, the
program could respond with a causal sequence of actions. It could say that
it had done X to do Y in order to do Z and so on. At the end of its chain of
reasoning, it had only the initial command given by the user. In that case, it
would respond, I did that because you asked me to. This latter response
became well-known and was pschologically very appealing. (For example,
Restak (1979) used the phrase "Because you asked me to" as the title of a
chapter in a popular book on the brain. That chapter was on various
aspects of AI and only touched upon Winograd's work.) Although it was
not put this way at the time, one of the reasons that Winograd's program
was much appreciated was because it understood at the MAKING SENSE
level on the understanding spectrum. It understood its world of blocks as

Copyrighted Material
 THE EXPLANATION TEST 15

well as a human would. Now, it would not have passed Turing's test,
because it understood nothing but blocks. But, I claim, it should be
recognized as having passed the Explanation Test at the level of MAKING
SENSE.
Each point on the understanding spectrum has essentially the same
requirements in the Explanation Test. For COGNITIVE UNDERSTAND-
ING, the program must be able to explain why it came to the conclusions it
did, what hypotheses it rejected and why, how previous experiences influ-
enced it to come up with its hypotheses and so on. We do not let a human
being come up with innovative ideas, generalizations, correlations and so
on, unless he can explain himself. Creative scientists are supposed to be
able to tell us what they do—not what they do while they are being creative,
but what the reasoning is behind what they have discovered. We may not
expect Einstein to know how he came up with his creative ideas, but we
certainly expect him to be able to explain the physics behind them. We will
put up with unexplained brilliance for a while, but eventually we object, we
believe that we are somehow being fooled. We cannot put up with a
scientist who says: E = MC2,I don't know what that means, but I am sure it
is true. There is no reason why we should let a machine off more easily.
The machine must be able to answer the question, How do you know? to
the satisfaction of an examiner in a school who would expect no less from
his students on a similar accomplishment in a similar domain.
The last point we have presented on our spectrum, that of COMPLETE
EMPATHY, has no easier test. Any system purporting to satisfy this level of
understanding must satisfy its examiner in the same way that a human
would satisfy him in a similar situation. I am claiming that this is not likely
in its fullest sense, and that this improbability is what all the uproar is about
with respect to the assessment of the possibilities for Artificial Intelligence
by laymen. No machine will have undergone enough experiences, and
reacted to them in the same was as you did, to satisfy you that it really
understands you. One is extremely lucky if one meets one person in one's
lifetime who satisfies that criterion.
The second aspect of explanation is much more complex. What would it
mean for a system to explain something to itself, and why does that matter?
The argument put forward in this book is that what is important about
understanding is how it changes over time. A machine that understands at
the level of making sense fails to convince us that it really has understood
because it continues to understand in the same way every time. The
computer programs that we built at Yale during the 1970's, all had the
feature, as do most AI programs, of behaving exactly the same way every
time.
But, people, especially intelligent people, do not behave this way. They
adapt to new circumstances by changing their behavior. How do they do

Copyrighted Material
16 1. THE EXPLANATION GAME

this? The argument put forward in this book is that the method that they
use for this adaptation is also based upon explanation, but explanation to
oneself. Explaining something to yourself means, in essence, attempting to
correct an initial misunderstanding by finding relevant experiences in one's
memory that might account for the incomprehensible event. The result of
such explanation can be a new understanding of old information as well. In
other words, explanation, learning, and also creativity, are inextricably
bound together.
To explain something to yourself at the making sense level, one need
only find a relevant knowledge structure in which to place a new input.
Seeing a new event as an instance of an old event constitutes understanding
at a very simple level. But, trying to find the theory that explains a group of
events can be a much more complicated and more profound type of
understanding. Explaining a group of events to yourself can thus mean
coming up with a theory of those events. Thus, cognitive understanding
really requires a good grasp of what would constitute a reasonable explana-
tion for something in general.
From the perspective of a test of a computer's understanding ability
then, this spectrum allows us to set some criteria that will allow us to
examine whether an understanding system is understanding by looking at
that systems's powers of explanation. But, more important is a system's
ability of self-explanation. That is, one cannot easily examine how a system
changes itself as a result of its experiences, one can just observe it over
time. But the claim is that the concept of explanation is key in bothe cases.
Explaining the world, to others and to oneself, is, in essence, the heart of
understanding.

QUESTIONS AND EXPLANATIONS

It was suggested by Riesbeck (personal communication) that one way to

make the distinction between passing the Explanation Test at each level is
to use the example of a joke. A computer understander that simply under-
stood the joke, in that it could explain what had happened, would be
understanding at the level of MAKING SENSE. A program that actually
found the joke funny, to the extent that it could explain what expectations
had been violated, what other jokes it knew that were like it that it was
reminded of, and so on, would be understanding at the level of COGNI-
TIVE UNDERSTANDING. Finally, a program that belly laughed because
of how that joke related to its own particular experiences and really
expressed a point of view about life that the program was only now
realizing, would have understood at the level of COMPLETE EMPATHY.
In all of these cases, perhaps the most important aspect of understanding is

Copyrighted Material
 QUESTIONS AND EXPLANATIONS 17

left untestable, however. Was the program changed in some way by the
joke? This is not an irrelevant question for a person, nor should it be for a
machine, though it might a bit difficult to answer in either case.
What will it take to get a machine to the level where it satisfies people
that it understands at the level of COGNITIVE UNDERSTANDING? Or,
to put this question another way, what is it that people do with each other
that makes us believe that they are understanding at that level? The answer
can, I believe be reduced to two words: questions and explanations.
People generate questions about those things that they did not fully
understand. Sometimes those questions cause people to find explanations,
sometimes novel and sometimes mundane, for the odd, unexpected, situa-
tions that they encounter. Sometimes those questions remain, to be answered
at a later date, or to be combined with new explanations as the seeds of
more creative explanations.
This book is about what it would take to achieve COGNITIVE UNDER-
STANDING. But it is not a philosophical work. I am interested in how to
push the level of intelligence that computers have already achieved to the
next step. I am also interested in exactly what it is that people must be
doing when they do intelligent things.
Because of that, this book is about questions and explanations. It is a
book about people and machines. It addresses the questions of how humans
formulate questions and how humans formulate explanations. The idea
here is that if we can find out what it is that humans do when they are
thinking and learning, then maybe we can model our machines on them.
The real intent of Artificial Intelligence, is, I claim, to find out what
intelligence is all about. We tend to say that a person is intelligent to the
extent that he is insightful, creative, and in general, able to relate apparently
unrelated pieces of information to come up with a new way of looking at
things. We tend to claim that a person is unintelligent to the extent that his
behavior is thoroughly predictable with reference to what we know that he
knows. Thus, when a person does things the way he was told to do them,
never questioning and thus never creating new methods, we tend to see him
as unintelligent.
I mention this here because I see the Explanation Test as a kind of
intelligence test. We are not asking the computer to simply replicate
intelligent behavior because we have no knowledge of which aspects of
such behavior are more intelligent than others. Is composing a sonnet a
more or less an intelligent act than playing chess? There is no way to
answer this objectively because it isn't the acts themselves that are at issue
here, but rather the quality of those acts. Turing could not differentiate
between these feats because he did not have the experience of trying to
build programs to do each task. But now, as a result of years of AI research,
such a question is easier to answer.

Copyrighted Material
18 1. THE EXPLANATION GAME

We can make a program write bad sonnets or play poor chess fairly
easily. Neither of these feats seem much of a mark of intelligence. Indeed,
working on either of them would not be considered AI any more, although
such work was done by AI researchers not so long ago. Today, work on
computer poetry or computer chess falls within the domain of AI only to
the extent that it mimics the complex cognitive processes associated with
the creativity inherent in both acts. Thus, if the computer poetry program
started with a set of feelings and was able, by relating such feelings to its
personal experiences, to create poetry, particularly poetry of some new
type, we would be legitimately impressed. Similarly, if our computer chess
program was capable of improving its playing ability by inventing a new
strategy or employing an old one that it recalled having seen in a match it
knew about, that would be an AI-type feat.
We have come to understand in AI that it isn't the tasks themselves that
are interesting. What matters is how they are done. Thus, I claim, the only
way to know if our machines are intelligent is to make them do what we
would expect a human to do in a similar situation. We must expect them to
be able to explain how they did it. Furthermore, those explanations should
have some connection with how the task in question actually was performed.
Often this is a difficult task for people to perform. We do not always know
where our creative powers come from or how they were employed in any
given instance. But, we can attempt to give rational explanations. We
should demand no less from machines.

HOW THE EXPLANATION TEST WORKS

It should be clear that there is no passing or failing of the explanation test

as such. The reason for this is that the test refers to a continuum of
understanding and thus it is possible to pass the test at one point and fail it
at a point immediately to its right.
The test itself is simple. It revolves around the completion of a specific
task. A mental task is given to a machine on the one hand, and a person on
the other, as in Turing's Imitation Game. The interviewer is asked to
question the machine or person about how each came up with the behavior
that it or he did. If the interviewer judges one subject's answers to be more
insightful and explanatory than the other's, then that subject is judged to
have passed the Explanation Test. If that subject happens to be the machine,
then the machine can be said to be understanding at the level of explana-
tion that the task itself was rated. In other words the degree of passing is
related to the complexity of the task.
As long as people give better explanations than machines for given tasks,
then we can say they understand better than machines. When machines

Copyrighted Material
 CHANGING MACHINES 19

outstrip people in their explanatory ability, however, machines will be

safely claimed to be better understanders, and hence more intelligent, than
people in that area of knowledge. To give a simple example, machines can
already out-computer humans. What they cannot do is explain the compu-
tation processes that they use because they do not understand them any
better than a hand-held calculator can be said to understand its operations.
Thus, while we might prefer to use a computer for our calculations, until
we prefer to use a computer over a mathematician to explain the nature of
the operations in mathematics, machines will not be able to pass the
Explanation Test for mathematics. Here too, however, the different points
on the spectrum are operating. We may be able to enable the machine to
accurately explain what it does mathematically and thus achieve the
MAKING SENSE level of explanation in mathematics. But, we would have
to get a machine to understand why it did what it did to come up with some
new mathematical idea in order to claim the COGNITIVE UNDER-
STANDING level.
The Explanation Test then, is not really a question of imitation. Our
question is not so much whether a person could fool us into believing that
he is a machine or whether a machine could fool us into believing that it is
a person. Rather, we are interested in finding out whether anybody or
anything that we talk to can be coherent in its understanding and explanation;
can be creative and self-referential in its understanding and explanation;
can be truly insightful in its understanding and explanation; and can be
thinking about things on its own.
To understand is to satisfy some basic desire to make sense of what one is
processing, to learn from what one has processed, and to formulate new
desires about what one wants to learn. Our question is how people do this
and how machines might do this.

CHANGING MACHINES

The issue then is change. Intelligent beings change themselves as a result of

experience. They accomplish this change through the process of attempting
to explain what they do not understand. In other words, they change when
a given change facilitates understanding. How can we get computers to
change themselves over time? How can machines adapt to new experiences
that they cannot initially understand? The answer I provide in this book
can be expressed in three words: Explanation, Questions, and Creativity.
The task is to get computers to explain things to themselves, to ask
question about their experiences so as to cause those explanations to be
forthcoming, and to be creative in coming up with explanations that have
not been previously available.

Copyrighted Material
20 1. THE EXPLANATION GAME

During the course of processing a new event, information from old

events is called into play. We understand new events by using our prior
experiences. We interpret the new in terms of the old. To do this, we must
be able to retrieve old events by matching aspects of new events to indices
that enable old events to be retrieved.
When this retrieval occurs successfully, we think nothing of it at all and
simply go on with our processing. In those cases we are understanding in
terms of fairly standard and nonspecific old memories, ones that cause no
particular notice because they are so ordinary.
On other occasions, we take immediate note of the old episode that we
have suddenly found in our consciousness. We say that we have been
"reminded" of this old event and often we tell others about that event if
there is someone to tell, or we ponder the old event for a while. The
difference between these two circumstances is representative of the differ-
ence between processing that enables learning and processing that does
not. And, most importantly for our purposes here, the difference reflects
when explanation is necessary and when it is not.
A key element in reminding is explanation. The most important thing we
can find out about human memory is what the indices to specific memories
are like. The basis from which those indices often seem to be constructed
are explanations that have been developed to differentiate one experience
from another. In other words, if indexing in memory is the key problem in
AI, then explanation is an important part of the solution. When we explain
things to ourselves, other things come to mind. Creative thought to a large
extent depends upon our ability to explain things.
Let me illustrate the explanation/reminding problem with a true story:

I was having dinner with a friend of mine who eats only

kosher food. He ordered a pasta dish made with cream sauce
and mushrooms on the grounds that even though he was not in
a kosher restaurant, the ingredients were all kosher, so it would
be all right to eat it. When the dish arrived, it had small red bits
of things in it which looked suspisciously like meat. Since he
could not eat it if it were meat (and probably could not eat
the entire dish because of the meat), I asked him if he wanted
me to taste the red things to see what they were. He laughed,
said no, and then told me a story of which he had just been
reminded. He said he had a cousin who decided to become
more orthodox in his religious practices. The more he studied,
the more he found out about orthodox Jewish rules. These
included the fact that it was only all right for a man to have sex
with his wife during two weeks out of every month. At this point
he stopped studying. He told my friend that there were some

Copyrighted Material
 CHANGING MACHINES 21

things he just didn't want to know. My friend said that this

applied exactly to the current case. He just didn't want to find
out that he couldn't eat the dish. He decided to not eat the red
things, but to eat the rest as he just didn't want to know.
The concept of memory organization that we are talking about here is
very well illustrated by this example. Once one has concocted an explana-
tion of an event (in this case, my friend explained his own behavior with
"there are just some things I don't want to know"), memories get activated.
Because explanations themselves are such strong indices, other memories,
especially ones in similar contexts (in this case, Jewish religious prohibitions)
become activated whenever an explanation is concocted. Their value?
Well, sometimes it is to illustrate a point by telling a story. But they have a
much more significant use. They allow us to check the validity of the
explanations themselves. And, they often force us to think of new explana-
tions when old ones seem to no longer suffice.
If we constantly predict something will happen and it never does, or the
opposite always happens, we feel inclined to change our predictions. We
need to remember events that show our predictive rules to be in error. Now,
this doesn't mean that we always do remember them. People are very good
at forgetting bad things that happen to them, thus enabling them not to
have to revise their expectation rules. But, when they feel that their
expectations really were wrong, they attempt to recall past errors to help
them formulate new expectations.
In order to do this, it is necessary to explain what went wrong initially
and, to do that, one needs evidence. Since prior evidence must be around
for consideration when a new expectation is to be formed, it is important
that events that did not conform to that expectation were stored in terms of
the failed expectation. Thus, one kind of index is a failed expectation itself.
Events are stored in terms of the expectations that they violated.
Because humans are intelligent enough to wonder about why something
has gone wrong when it goes wrong, it is important for them to attempt an
initial characterization of what exactly the problem was. Thus, they con-
struct initial explanations, which serve as indices as well. The reason is
clear enough. If explanations were not available, they would be lost. That
is, what we were trying to learn at any given point would be immediately
lost as soon as we stopped considering it for a moment. To avoid this,
memory keeps available tentative explanations. Where to store them? With
the expectations that failed, naturally.

Copyrighted Material
22 1. THE EXPLANATION GAME

EXPLANATION IS UNDERSTANDING

A first assumption in this book, then (first stated in Schank, 1982), is that
when our expectations are found to be in error we must attempt to explain
why. Failures lead to attempts to explain failures. We want to avoid making
the same mistake twice. How? By understanding what principles underlie
the expectations that we had in the first place. We must understand them
so we can fix them.
In the remindings that we examined, one thing that we found (Schank,
1982) was that each pair of events that we connected by a reminding had in
common not only an identical expectation failure, but an identical explana-
tion of that failure as well.
Explanation is critical to the understanding process. As understanders,
we want to know what others around us are doing, and will do. Furthermore,
we want to know why others do what they do. Thus, people are constantly
seeking explanations for the behavior of others. One of the simplest kinds
of explanations is a script (Schank & Abelson, 1977). We believe that we
understand why a person is doing what he is doing if we can point to a
script that he is following. Knowing that someone is doing a particular
action because it is part of the behavior prescribed by a commonly adhered
to set of rules makes us feel comfortable that we have explained his
behavior. We can, of course, look for explanations that are more profound
than saying "he is doing that because he (or someone in his situation)
always does that." We usually feel that we have understood without going
that deep, however. And, it is this script-type of understanding that we are
referring to when we talk about Making Sense. But, when script-based
understanding is not enough, when we really want to understand something
in a deep way, the explanation process must come into play. It is the
creation of new explanations that produces Cognitive Understanding.
At the root of our understanding ability is our ability to seek and create
explanations. What we are trying to do when we seek an explanation for an
actor's behavior, is to find a set of beliefs that the actor could hold with
which the actions that he took would be consistent. So, what is understand-
ing then? In this view, understanding can be seen to be no more than, and
no less than, explanation.
We have, in AI, gotten used to speaking of "computer understanding,"
meaning by that term that an input sentence has been combined with a
memory structure of some sort. To put this graphically, we have believed
that:

Copyrighted Material
 EXPLANATION IS UNDERSTANDING 23

AN INPUT SENTENCE
+
A MEMORY STRUCTURE THAT INDICATES WHERE TO
PLACE THAT SENTENCE
results in
AN UNDERSTOOD SENTENCE
In other words, the process of understanding, for computers in the early
stages of AI research, was embodied in the notion that understanding
something means putting it in its proper context, or Making Sense of it.
The major problem with this point of view is that, on occasion, the
proper context either does not previously exist, or, it is difficult to deter-
mine exactly what that context is. Graphically:
AN INPUT SENTENCE
but
NO MEMORY STRUCTURE IS PREPARED TO ACCOMMODATE
THAT SENTENCE
results in
A SENTENCE THAT NEEDS EXPLANATION
The premise here is that, in principle, what we are doing when we
understand is explaining what we have heard. Most of the time those
explanations are very mundane, so we don't feel as if we are explaining
anything. Rather, we are simply placing the new information in a previously
extant structure. In other words, we don't have to explain why someone is
reading a menu in a restaurant since we know exactly when and where such
actions take place. Explanations are necessary when an action fails to come
in the way we expected, or cannot be placed into an existing structure.
But, the point is that we are always needing to explain every action we
encounter. We feel that we are explaining when we are doing something
other than accessing an everyday, run-of-the-mill, structure, or when, while
using such a structure, we have still found ourselves incapable of deciding
exactly what to do. Thus, explaining and understanding are the same thing,
one is just more conscious a process than the other. And, explanation must
be a more conscious process, because it occurs when something has gone
wrong and needs to be fixed.

Copyrighted Material
References
Abelson, R. , Carroll, J. (1965). Computer Simulation of Individual Belief Systems, American
Behavioral Scientist. 8, pp. 24–30.
Carbonell, J. (1979). Subjective Understanding: Computer Models of Belief Systems, Ph.D. Thesis,
Yale University.
Colby, K. M. (1973). The Structure of Belief Systems, Schank, R.C. , and Colby, K.M. eds., Computer
Models of Thought and Language, W.H. Freeman and Company, San Francisco, pages 251–286.
Colby, K.M. , & Smith, D.C. (1969). Dialogues between Humans and an Artificial Belief System,
Walker, D.E. and Norton, L.M. eds., Proceedings of the International Joint Conference on Artificial
Intelligence, IJCAI, Boston.
Cullingford, R. (1978). Script Application: Computer Understanding of Newspaper Stories, Ph.D.
Thesis, Yale University. Research Report #116.
DeJong, G. (1979). Skimming Stories in Real Time: An Experiment in Integrated Understanding, Ph.D.
Thesis, Yale University.
Dreyfuss, H.L. (1972). What Computers Can't Do: A Critique of Artificial Reason, Harper & Row, New
York.
Hammond, K. (1984). Indexing and Causality: The organization of plans and strategies in memory.
Technical Report 351, Yale University Department of Computer Science.
Kogos, F. (1970). Book of Yiddish Proverbs. Poplar Books, Secaucus, NJ.
Lebowitz, M. (1980). Generalization and Memory in an Integrated Understanding System. Ph.D.
Thesis, Yale University.
Lytinen, S. (1984). The Organization of Knowledge In a Multi-lingual, Integrated Parser, Ph.D. Thesis,
Yale University. Research Report #340.
Newell, A. , & Simon, H. (1972). Human Problem Solving. Prentice-Hall, Englewood Cliffs, NJ.
Restak, R.M. (1979). The Brain: The Last Frontier. Doubleday, Garden City, NY.
Rieger, C. (1975). Conceptual Memory and Inference, Conceptual Information Processing. North-
Holland, Amsterdam.
Schank, R.C. (1972). Conceptual Dependency: A Theory of Natural Language-Understanding.
Cognitive Psychology. 3/4, pp. 552–631.
Schank, R.C. (1975). Fundamental Studies in Computer Science, Volume 3: Conceptual Information
Processing, North-Holland, Amsterdam.
Schank, R.C. (1982). Dynamic Memory: A Theory of Learning in Computers and People, Cambridge
University Press.
Schank, R.C. , & Abelson, R. (1977). Scripts, Plans, Goals and Understanding, Lawrence Erlbaum
Associates, Hillsdale, New Jersey.
Schank, R.C. , & Riesbeck, C. (1981). Inside Computer Understanding: Five Programs with
Miniatures, Lawrence Erlbaum Associates, Hillsdale, New Jersey.
Searle, J.R. (1980). Minds, Brains and Programs, The Behavioral and Brain Sciences, 3, pp. 417–424.
Shortliffe, E.H. , Axline, S.G. , Buchanan, B.G. , Merigan, T.C. , & Cohen, N.S. (1973). An Artificial
Intelligence Program to Advise Physicians Regarding Antimicrobal Therapy, Computers and
Biomedical Research, 6, pp. 544–560.
Turing, A.M. (1963). Computing Machinery and Intelligence, Feigenbaum, E.A. , and Feldman, J. eds.,
Computers and Thought, McGraw-Hill, New York.
Weizenbaum, J. (1976). Computer Power and Human Reason: From Judgement to Calculation,
Freeman, San Francisco.
Wilensky, R. (1978). Understanding Goal-Based Stories, Ph.D. Thesis, Yale University. Research
Report #140.
Winograd, T. (1972). Understanding Natural Language, Academic Press, New York.