Children’s Literature in Education (2012) 43:273–291

DOI 10.1007/s10583-012-9163-6

ORIGINAL PAPER

Reading Other People’s Minds Through Word

and Image

Maria Nikolajeva

Published online: 1 March 2012

 Springer Science+Business Media, LLC 2012

Abstract This article considers how emotions can be conveyed through the
interaction of word and image in picturebooks addressed to young readers. The
theoretical framework employed in the article develops ideas from cognitive literary
theory, adapting it to the specific conditions in which there is a significant difference
between the sender’s and the recipient’s cognitive levels. The concept of emotion
ekphrasis is used to demonstrate the various ways of representing emotions, and
special attention is paid to the issues of mind-reading, empathy and other aspects of
recipients’ affective engagement. The theoretical argument is illustrated by pic-
turebooks by Max Velthuijs, Shaun Tan, Anthony Browne, and Maurice Sendak.

Keywords Picturebooks
Emotions
Cognitive poetics
Affective response

Eight years ago, when I first met Siobhan, she showed me this picture
L
and I knew that it meant ‘‘sad’’, which is what I felt when I found the dead dog.
Then she showed me this picture
,

Maria Nikolajeva is a Professor of Education at the University of Cambridge, UK, previously a Professor
of comparative literature at Stockholm University, Sweden, where she taught children’s literature and
critical theory for twenty-five years. She is the author and editor of several books, among them
Children’s Literature Comes of Age: Toward the New Aesthetic (1996), How Picturebooks Work,
co-authored with Carole Scott (2000), From Mythic to Linear: Time in Children’s Literature (2000), The
Rhetoric of Character in Children’s Literature (2002), Aesthetic Approaches to Children’s Literature
(2005), and Power, Voice and Subjectivity in Literature for Young Readers (2010). In 1993-1997 she was
the President of the International Research Society for Children’s Literature. She was also one of the
senior editors for The Oxford Encyclopedia of Children’s Literature (2006) and received the
International Grimm Award in 2005 for a lifetime achievement in children’s literature research.

M. Nikolajeva (&)
Faculty of Education, University of Cambridge, 184 Hills Road, Cambridge CB2 8PQ, UK
e-mail: [email protected]

123
274 Children’s Literature in Education (2012) 43:273–291

and I knew that it meant ‘‘happy’’, like when I’m reading about the Apollo space missions…

(Haddon, 2003, p. 2)

The protagonist and narrator of Mark Haddon’s (2003) The Curious Incident of the
Dog in the Night-time suffers from Asperger syndrome. Like many autistic children
and adolescents, Christopher Boon is extraordinarily bright, but he lacks the
elementary social skills such as understanding other people’s feelings. Physically,
he is a teenager, intellectually he is superior to most adults, yet socially and
emotionally he is no older than three or four and completely solipsistic. But he is a
quick learner. His teacher uses his ability to read emoticons to develop empathy. He
knows that the word ‘‘sad’’ describes what he felt in certain circumstances. He
cannot generalise from this knowledge, but the verbal signifier ‘‘sad’’ will for him
always be associated with his feeling of sadness. The teacher utilises this association
to create a further connection, matching the word ‘‘sad’’ with the emoticon of
sadness, L. She creates a link between the experienced emotion, the word and the
icon. Skipping the word, Christopher is now conditioned to interpret the emoticon as
representing sadness. Whenever he sees a human face resembling the emoticon he
reads it as sadness. He is able to read another person’s face, interpret it as an
expression of an emotion and connect this second-hand emotion to his real,
experienced one.
Christopher is a literary character, but his behaviour is an accurate description not
only of Asperger sufferers, but also of the process young children go through when
they learn the indispensable social skill of understanding other people’s feelings
(Blakemore and Frith, 2005). It is believed that visual stimuli play a stronger part in
this process than verbal, since our visual skills are hard-wired in the brain, while
linguistic skills are not (e.g. Wolf, 2007; Carr, 2010). Reading a person’s facial
expression or bodily posture purportedly sends a stronger signal to the brain than the
verbal statement ‘‘This person is happy, sad or frightened.’’
The knowledge and understanding of other people’s minds are essential social
skills, and if literature can help children develop these skills it would be a major tool
for socialisation. It has been repeatedly claimed that literature contributes to
socialisation; yet it has never been thoroughly examined exactly how it works. Why
would even the most profound knowledge of fictional people who do not exist and
have never existed, with their non-existing personal problems and public networks,
their non-existing opinions and non-existing emotions, be of any relevance
whatsoever for our knowledge and understanding of real people either in our
immediate environment or further away? Cognitive criticism (also known as
cognitive narratology, cognitive poetics or literary cognitivism) has recently
provided some fascinating answers to these questions, combining recent achieve-
ments in brain research with the most profound issues of literary criticism, including
why reading offers aesthetic pleasure (e.g. Tsur, 1992; Turner, 1996; Crane, 2001;
Stockwell, 2002; Gavins and Steen, 2003; Hogan, 2003a, b; Gibson et al., 2007;
Vermeule, 2010; Hogan, 2011). Most of this research comes from cognitive
psychology, social linguistics, or psycholinguistics, where scholars have noted that
the data typically collected through time-consuming and high-cost fieldwork can,

123
Children’s Literature in Education (2012) 43:273–291 275

with some reservations, be supplanted by the easily available and inexhaustible

source of fiction (e.g. Gibbs, 1994). In children’s literature scholarship, the
possibilities of a cognitive approach are so far little explored and focused mainly on
image schemes (e.g. Stephens, 2011). In particular, one of the central directions of
cognitive criticism, readers’ affective engagement with fiction, has not been given
due attention, even though, paradoxically, studies of real and implied readers are
considerably more common within children’s literature research, especially its
educationally oriented aspects.
There is no room within this essay to delve into details of brain research or the
various approaches to emotions (for a concise summary, see Evans, 2001; O’Shea,
2005). As a literary scholar, I am interested in whether fiction can convey to readers
the knowledge and understanding of other people’s emotions, and if so, how exactly
this happens. As a children’s literature scholar, I am also interested in whether
young readers respond to texts in ways different from mature readers, due to a
different (not necessarily inferior) level of cognitive and emotional development.
The numerous empirical studies about children’s engagement with fiction can only
provide collateral evidence of children’s ability to infer emotions from literary texts.
My cognitive approach to children’s fiction is therefore purely theoretical; thus,
when I speak of ‘‘reader responses’’ I refer to potential reactions provoked by the
text. In this article I am looking in particular at how understanding of other people
in the actual world is enhanced by multimedial texts, where verbal and visual
components can support or contradict each other. I have chosen a number of well-
known examples to illustrate my argument. However, some preliminary theoretical
discussion, based on the above-mentioned cognitive studies, is necessary.
My two foremost sources of inspiration are Blakey Vermeule’s Why Do We Care
about Literary Characters? (2010) and Patrick Colm Hogan’s What Literature
Teaches Us About Emotions (2011). Both describe empathy and mind-reading in
actual life as the human capacity to attribute mental states to other people, based on
actions and reactions, facial expressions, body language and other external signals
(this capacity is also referred to as the Theory of Mind). In fiction, the skills of
mind-reading can be trained over and over again, with little risk of error. Blakey
and Hogan illustrate their arguments with classic literature that is essentially
monomedial. Yet much of the reasoning can be successfully applied to multimedial
texts, exactly because in real life mind-reading depends heavily on visual
perception. Multimedial narratives frequently make use of ambiguity created in
the interaction between media.
Although neither Vermeule nor Hogan considers the educational implications of
their arguments, these are of overall importance. Empirical research demonstrates
that mind-reading normally develops at the age of five and is slower or even totally
impeded in autistic children (e.g. Blakemore and Frith, 2005). It typically starts with
recognising basic emotions: sadness, joy, fear, anger and disgust (Oatley, 1992;
Evans, 2001). In cognitive science, basic emotions are explained through complex
processes in the brain which neuroscience has recently managed to register, for
instance, through magnetic resonance imaging. On a very primitive level, emotions
are responses of goal-oriented systems in our brains. Thus happiness occurs when a
goal is achieved or likely to be achieved. Sadness is caused by an ultimate loss of a

123
276 Children’s Literature in Education (2012) 43:273–291

goal; fear occurs when a goal is threatened; anger comes from frustration over a
goal, and so on. High-cognitive emotions (also referred to as social emotions), such
as love, hatred, contempt, envy, jealousy, pity or guilt, involve two or more agents’
common goals, into which individual goals have to be incorporated. It is easy to find
striking examples of all these emotions in literature; moreover, most literary plots
are clearly built around at least one of them. However, since literary characters are
created of words and thus have no brains to generate emotions, why would we care
about this artificial and empty construction?
Fiction creates situations in which emotions are simulated; we engage with
literary characters’ emotions because our brain can, through mirror neurons,
simulate other people’s goals in the same manner as it can simulate our own goals. It
can be pointed out that since literary characters are fictional they have no real goals.
However, as cognitive critics argue, in reading fiction we engage with possible
outcomes of the fictive situations, including the final desired outcome of the whole
story. Cognitive criticism purports that the reason we can engage with fictive
characters is because of the connections between the mediated experience of the text
and emotional memories stored in the brain. We engage with literary characters
through vicarious, or proxy experience.

Affective Response

The most primitive way to render an emotion in a literary text is to state that
somebody is happy, sad or angry. This can be conveyed through a narrator—‘‘He
was happy’’—or by a character’s direct speech or thought: ‘‘I am happy.’’ The
former is an objective statement, the latter is a subjective perception. In both cases
words trigger readers’ memories stored in the brain that are associated with the
respective emotions, which enable us to read a character’s mind.
Our emotional memories, however, are fragmentary and imprecise. They do not
necessarily render an event as it was experienced, but rather as it was stored when it
was transferred from the working memory to the long-term memory. Reading a
literary text activates the long-term memory, and the emotions connected with it are
activated as well. In making sense of fiction, readers frequently relate fictional
events to their personal experience and ‘‘understand’’ characters’ emotions by
connecting them to relevant emotionally charged memories. Cognitive theory refers
to this process as ‘‘misattribution’’: we attribute our own emotions to those of fictive
characters. Another way of expressing it is to say that we project our own emotions
onto fictive characters.
A central argument of cognitive criticism is to question the relationship between
literal and figurative language, challenging the conventional belief that the latter is
effectively redundant in everyday communication. Based on extensive empirical
research, Mark Turner (1996) argues that narrative precedes language, and
Raymond Gibbs (1994) argues that metaphor is omnipresent in everyday life. If
so, projecting life experience onto fictional narratives comes naturally, and building
our personal stored memories on earlier vicarious experience may amplify the
projection: we make connections between disparate fictive characters’ emotions. For

123
Children’s Literature in Education (2012) 43:273–291 277

a young reader, who may have limited or no experience of real-life emotions,

literary, second-hand experience is particularly valuable.
In reading statements such as ‘‘He was happy’’ or ‘‘She was sad’’ we do not
necessarily experience the same emotion directly and to the same degree, but we can
empathise with a character who is happy or sad, provided that we have a stored
memory of the emotion, either from life experience or from a previous literary
experience. We have emotional knowledge of what it means to be happy or sad. Yet
empathy does not imply that we fully share the character’s state of mind. On the
contrary, mature readers should be able to empathise with a character without
necessarily sharing the emotion, as well as to keep their own emotions separate from
the character’s (cf. Feagin, 2007). Moreover, a mature reader will be aware that any
emotion discourse is a representation and, as such, merely an approximation of an
actual emotion. For instance, we can understand that a character is angry, but see no
reason for the anger or judge whether the anger is misdirected. The balance of
engagement and disengagement is a prerequisite for successful mind-reading
(cf. Keen, 2008).
When the emotion has been given a literal (non-metaphorical) verbal label, such
as ‘‘He was happy,’’ it can be expanded or deepened by further words or images. A
word expanded by other words is a description of the mental state or an explanation
of what it means to be happy or sad. There is no succinct literary term to cover this
phenomenon, so I will employ the concept of emotion ekphrasis: that is, verbal,
visual or multimedial discourse used to describe an emotion. Emotion ekphrasis can
provide an expansion of the mental state (‘‘He was so happy that he wanted to sing
and dance’’), or an explanation of what it means to be happy (‘‘He was as happy as
you are when you open a birthday present’’). It may include a temporal and causal
indication (‘‘after that, he was happy’’ or ‘‘he is happy because…’’). It can also be
expressed metaphorically rather than literally, as, for instance, in a sentence like,
‘‘His heart missed a beat.’’ It may extend to many pages, as it often does in
psychological novels. It may, further, employ complex narrative devices, such as
free indirect speech. What is problematic, however, is that emotions are not
necessarily as clearly delineated as the labels make them: there are degrees of
happiness and sorrow; there are nuances between happy, glad and exhilarated, sad
and upset, angry and furious. Emotions are by definition non-verbal, and language
can never convey an emotion adequately. Metaphors are a powerful device to
circumvent this dilemma, while visual images carry a stronger potential still. Images
can substantially enhance the meaning expressed by the words approximating the
vague and indefinable emotion. Emotion ekphrasis is not merely an explanation of
an emotion (‘‘he was sad because his pet rabbit had died’’), but an evocation of it.
Visually represented happiness or sadness also evokes a memory in the brain that
simulates the experience of happiness or sadness. This, however, can only be
possible if there is a stored memory of a particular emotion, which may seem to
preclude emotions that we have not experienced, or experienced only to a lesser
degree. For instance, few of us—fortunately—have been exposed to extreme
horrors, yet we can relate to the feeling of horror, not least in a multimedial text in
which an image has an immediate impact on the viewer. Emotional memories are
not coherent, but disjunctive and fragmentary. The way a certain experience is

123
278 Children’s Literature in Education (2012) 43:273–291

stored in the brain may be not actual, but distorted, affected by a subjective
perception or by particular circumstances. Pieces of different memories may
become randomly interconnected as they are stored. Memories can also be
suppressed and re-emerge if provoked by a strong emotion, whether actual or
vicarious (e.g. Nalbantian, 2003). All this implies that a visual image can potentially
evoke a wide range of emotions circumventing the relative precision of words. For a
pre-literate reader, such experience is invaluable.

Sadness

Let us start with a seemingly simple narrative, Frog is Sad, by Max Velthuijs
(2003). The cover, the first element of the narrative we meet, conveys information
on two levels: the verbal statement ‘‘Frog is sad’’ and an image of a character whose
posture and facial expression suggest sadness. Note that although the character is
not human, we tend to anthropomorphise animals, inanimate objects, natural
phenomena and even abstract notions, in real life as well as in fiction. In the actual
world, we may, for instance, believe that birds sing because they are happy. In a
fictional world, we read Frog’s emotions as if they were human. In this case, the
most prominent visual detail suggesting sadness is the shape of his mouth. This
visual representation of sadness is both a convention, like an emoticon, and
something we may have from our personal experience; this, of course, is how
Christopher Boon learned to understand sadness. In semiotic terms, the image is
indexical: it does not represent the signified, but points at it. The verbal statement
strongly supports our interpretation of the image. As a thought experiment, we could
remove the title and try to read the image on its own. Our interpretation would
arguably be less precise; for instance, we could read it as ‘‘The character is
meditative’’ or ‘‘The character is bored.’’ Both emotions border on sadness, but
since sadness is a basic emotion, it is ostensibly the first choice in our interpretive
strategy. Nuances of the emotion can be conveyed verbally with the range of sad,
upset, distressed, melancholy, pensive, miserable, gloomy, unhappy, anxious, and so
on. From images, we cannot exactly determine the nuance (Fig. 1).
The title of Velthuijs’s book, however, gives us little or no choice. A title or an
image caption is a very strong authoritative statement, hard to ignore. (When a
caption declares: ‘‘This is not a pipe’’ we at least need to reflect upon it and decide
whether to trust our vision or the statement.) Mature readers tend to rely more on
verbal statements than on mind-reading from images, possibly because we are, from
early school years, manipulated to believe that language has more weight than
images; but then mature readers will also be able to perceive the irony in the tension
between words and images (Stockwell, 2002; Keen, 2008). Pre-literate readers are
likely to trust images more than words, since images are direct and immediate,
while words need processing. This is pure speculation; we know too little about it
yet, the recent achievements of neuroscience notwithstanding. (For instance, it has
been confirmed that the ‘‘emotional’’ part of the brain responds to a visual stimulus
nano-seconds before it is processed by the ‘‘rational’’ part; see Evans, 2001, 25ff).
However, it is contestable whether visual images have a stronger impact on our

123
Children’s Literature in Education (2012) 43:273–291 279

Fig. 1 The cover of Frog is Sad by Max Velthuijs, reproduced by permission of Andersen Press Ltd

minds than the verbal, and I would claim that experiments to prove either might be
impossible since processing of words and images happens differently in our brain.
All we know is that the visual cortex is evolutionarily hard-wired in our brain, while
language is not. This should logically imply that reading images comes naturally,
while understanding verbal statements, whether oral or written, must be learned.
However, on the cover of Frog is Sad, there is no overt contradiction between the
verbal and visual information, which makes our interpretation unequivocal: Frog is
sad. Throughout the book, Frog’s body language, including his mouth shape, repeats
and thus amplifies our understanding of his emotional state as sadness. From the
images alone, we should not only be able to reconstruct the story, but also
contemplate the nuances of Frog’s emotions; for instance, irritation and perplexity
at his friends’ attempts to entertain him. We have no problems reading the change in
mood as Frog begins to smile, stretches out his arms and starts to dance. However,
the words consistently enhance the images: ‘‘Frog woke up feeling sad… he felt like
crying, but he didn’t know why.’’ The latter clause is slightly more complex than the
simple statement, ‘‘He was sad.’’ It invites us to reflect upon the cause of the
emotion. If the character does not know why he is sad, can we think of any reasons?
Or do we recognise that one can be sad without obvious reasons?
In a book such as Frog is Sad, marketed for very young readers, the verbal text
accompanying the images is a pedagogical device. This might be because the
author, or publisher, has assumed that a young child might not be able to read the

123
280 Children’s Literature in Education (2012) 43:273–291

emotions without verbal support. However, this would be a dubious assumption. We

know from experimental psychology that even infants a few weeks old respond to
and thus presumably understand both actual and represented facial expressions.
There is vast anecdotal evidence of young children responding adequately to visual
images of emotions long before they can talk. The verbal information in the Frog
book may seem redundant. On the other hand, communication theory maintains that
redundancy is an important communicative factor. More important, however, is the
fact that the book title verbalises the emotion before readers are given a chance to
identify it for themselves. The verbal text here has the same function as voice-over
in film. A film-maker would perhaps avoid voice-over to articulate characters’
emotions, trusting viewers to be able to interpret them from the visual narrative
alone.
This is what happens in more complex picturebooks, where the verbal narrative
provides little support for mind-reading, if at all. Shaun Tan’s (2001) The Red Tree
offers exactly the same story as Frog is Sad: a person is sad, but becomes glad
again. However, this simple emotional transformation is conveyed through complex
and highly ambiguous imagery. Neither the title nor the cover image provides any
guidance as to what the text will be about, except that the contrasting colours
suggest the opposites, warm–cold, sunshine–rain, and by extension, happy–
unhappy. The character’s reflection in water is symbolic rather than iconic, and
contributes to the anticipation of a complex narrative.
The image on the frontispiece emphasises the character’s inability to express
herself with words as nothing intelligible comes out of her mouth. All that follows
apparently takes place in her mind, while the accompanying words are vague. It is
as if the character were struggling to explain her emotions in words; the absence of
punctuation or capital letters, the uneven and wavy text, the use of varying font size
and of bold type—everything accentuates the fragmentary nature of the verbal
narrative, corresponding to the vague, unspoken, unutterable emotion. Yet the
verbal text is sufficiently evocative.
We cannot unequivocally identify the emotion experienced by the character: it is
a combination of sorrow, fear, loneliness, anxiety, despair and plain horror. In
reading the character’s mind, we do not rely exclusively on her facial expression
and body language, exactly because we are supposed to understand that her
mindscape is externalised through the images. In fact, the girl’s facial expression
does not change throughout the book, and the absence of a mouth makes it almost
impossible to read her mind through her face. When happiness breaks through on
the final page, her mouth appears and clearly indicates joy. The ekphrasis primarily
employs figurative language: metaphor. Cognitive criticism would argue that the
literal meaning of ‘‘she was sad’’ is not necessarily more straightforward or easily
comprehended than the sophisticated, metaphorical, multimedial, emotion
ekphrasis.
Each doublespread in the narrative would have demanded pages upon pages of
verbal ekphrasis, and perhaps still be inadequate, while the images produce an
immediate and direct effect, even though each image also affords lengthy
contemplation and repeated reading. Surrealism generally affects our senses,
including disgust, which is evoked powerfully in The Red Tree. We do not only

123
Children’s Literature in Education (2012) 43:273–291 281

empathise with the character, but relate directly to the imagery. This implies that we
engage with the emotion both diegetically, as experienced by the character, and
extradiegetically, as experienced by an extratextual agency, which presumably
produces a synergetic effect. Rich intertextuality further amplifies the impact. The
red leaf, which we first encountered on the cover, appears framed over the girl’s
bed. It is inconspicuously present on every doublespread as a promise of hope, and
indeed both the character and reader are rewarded at the end for having endured the
pain. Empathising with the girl, we are expected to recognise her unspeakable state
of mind, relating it to our own similar memories, even though we may have never
experienced such intense emotions as the character. We are frustrated, and we feel
relief when the narrative ends on a positive note. Compared to Frog is Sad,
multimedial ekphrasis in The Red Tree not only differs in degree of complexity, but
also in nature: the former is literal as opposed to metaphorical.

Fear

Most narratives in Velthuijs’s Frog series are built around basic emotions.
Sometimes these are announced in the title, such as Frog is Frightened (1994) (the
cover actually contradicts the title, since the cover image shows three characters
safely asleep in bed). Sometimes the emotions are explicated within the story, so
that Frog is Frog (1996) equals ‘‘Frog is happy.’’ Frog is Frog is a more complex
statement than Frog is Sad or Frog is Frightened (arguably, just slightly more
complex, but enough not to explicitly refer to an emotion and thus render it
metaphorical rather than literal even on the verbal level).
The cover image of Frog is Frog, showing Frog trying on wings, suggests a sense
of happy anticipation; however, with some elementary notion about the nature of
frogs and the laws of gravity, or with an intertextual knowledge of the Icarus myth,
we are expected to infer that although Frog feels happy right now, he will soon be
disappointed. Readers who lack such previous knowledge will be unable to predict
the development of the plot and thus understand the futility of Frog’s ultimate goal,
to be happy. The statement ‘‘frog is frog’’ is a metaphor connected with the
happiness that comes out of discovering and affirming one’s own identity. Frog
becomes unhappy because he fails to achieve his goals, even though they are
obviously inappropriate. Inability to achieve one’s goals typically results in the
basic emotion of anger, ranging from irritation to despair and frustration. Since the
verbal text does not provide direct clues (apart from one instance of the adverb
‘‘sadly’’), readers have a wider scope for interpretation. We know that Frog’s
attempts are futile. We empathise with him because we may have had a similar
experience that we project onto the character. However, we are also disengaged
from him because we know—or are supposed to know—that he is being silly. This
is the balance between empathy and identification, engagement and disengagement
that I mentioned before. The change of Frog’s moods within the narrative, from
happiness through sadness and despair back to happiness, is clearly shown through
the images. Yet it is also conveyed through minimal ekphrasis, including the final,

123
282 Children’s Literature in Education (2012) 43:273–291

Fig. 2 The cover of Frog is Frog by Max Velthuijs, reproduced by permission of Andersen Press Ltd

literal, ‘‘Suddenly Frog felt very happy,’’ which may feel redundant, yet for a very
young reader might prove necessary (Fig. 2).
When the verbal and the visual ekphrasis contradict each other, the result is
irony. Our temporary disengagement from Frog’s attempts to fly is based on a
recognition of irony. It is the subject of recent heated debates about whether young
children understand and appreciate irony and at what age such understanding occurs
(Gibbs, 1994; Walsh, 2011). There is too little empirical research to make definitive
claims. Immature readers may trust visual rather than verbal statements, that is, infer
from the posture or facial expression that a character is glad even though the verbal
text might say something else. The young brain has limited ability to synthesise
complex and contradictory information. Research with autistic young people shows
that they can ‘‘read’’ only straightforward, literal, non-ironic images.
If this is true, the plain, literal emotion discourse, for instance, expressed in the
title Frog is Frightened, is a rather primitive way of representing a complex
emotion. Likewise, the verbal ekphrasis in Anthony Browne’s (1987) The Tunnel,
‘‘She was frightened,’’ is straightforward. In both cases, literal verbal statements are
expanded into visual metaphorical ones. Naturally, we must neglect the obvious
differences in visual style, but compositionally, the two images corresponding to the
literal statement, ‘‘was frightened,’’ are quite similar, and both convey rapid
movement from left to right, the way we in Western culture read verbal texts. Yet
the abundance of details in Browne’s image slows down the processing of visual

123
Children’s Literature in Education (2012) 43:273–291 283

information and provides the brain with more time to access stored emotional
memories. Moreover, the image is, as in The Red Tree, overloaded with metaphors
and intervisual elements. Again, the difference is in degree as well as in nature.
Moreover, the resolution of the situation, and thus the expected affective reader
engagement, is also different. Frog runs into the adjacent page where he is relatively
safe, while the girl runs into the next doublespread and into the unknown. This
subsequent doublespread is wordless, as if the verbal narrator becomes mute when
confronted with the character’s horror. Intuitively we would say that the image from
The Tunnel is substantially more complex, but let us consider in what way it is more
complex and what emotions the two images evoke in the reader.
Frog is a comic character, by virtue of being a hybrid between animal and human,
but also because we recognise the situation of being scared by monsters under the
bed as comic—a situation that also turns out to meet our expectations. The narrative
is based on dramatic irony, that is, on the discrepancy between what the character
knows and what the reader knows. We may sympathise with the character, but the
feeling the narrative evokes in us is most likely a sense of superiority, even for
readers who are very young. Superiority implies that readers are disengaged from
the character and can laugh at him. (This judgement might be wrong; it would
certainly be interesting to test this empirically and see at what age such
disengagement occurs. I have anecdotal evidence of this particular image being
frightening for a two-year-old, who apparently experienced extradiegetic rather than
diegetic fear.)
In The Tunnel, the character is human and thus arguably invites stronger empathy
than a non-human figure. There is nothing comical in the girl’s fears, even though
we know from the previous events that she too is scared of things under her bed. Her
fears are amplified by anxiety about her brother. She has crossed the boundary and
entered a world (arguably, the world of her unconscious) where she knows that she
is no longer safe. Readers engage with her because the image appeals to our
unpronounced fears, dreams, nightmares and generally strong, negative emotions,
including disgust. The rich intervisuality of the image may directly evoke our
emotions, circumventing the character, as we have seen in The Red Tree. In a
thought experiment, we can delete the figure of the girl from the doublespread and
consider whether we would still interpret the image as the girl’s emotion ekphrasis,
that is, on a diegetic level. I would argue that we would, since, after the preceding
story, we are already strongly engaged with the character, asking ourselves, ‘‘What
is she feeling?’’ or, projected from our actual or fictional experience, ‘‘What would I
feel in her place?’’ Moreover, the earlier imagery in the book foreshadows the
doublespread and makes the reader alert to its multi-layered meaning.
My suggestion is that while Velthuijs’s image might evoke a momentary sense of
danger and fear, this shifts to a sense of security, anticipated from the cover,
whereas Browne’s image remains threatening and unresolved. A magnetic
resonance imaging analysis might show the difference—or lack thereof—between
readers’ affective response to the two images, depending on the informants’ age and
various other factors. It would doubtless show the difference between responses to
the verbal statements, ‘‘I am frightened’’ or ‘‘She was frightened,’’ and the visual
representation of fear.

123
284 Children’s Literature in Education (2012) 43:273–291

Fig. 3 The cover of Frog in Love by Max Velthuijs, reproduced by permission of Andersen Press Ltd

Love

So far we have been looking at basic emotions. Frog in Love (1989) refers to a
higher-cognitive, or social emotion requiring that two individuals’ ultimate goals, to
be happy, become equally valuable for both. The title signals the central theme,
amplified by the image of the moon, a recurrent romantic trope (or image scheme;
see Turner, 1996). The narrative, however, starts with an ambivalent emotion
ekphrasis: ‘‘Frog…felt funny. He didn’t know if he was happy or sad.’’ There is a
slight contradiction between the words and the image, since the latter rather
suggests that Frog is sad, judged by his mouth shape and his arms hanging down as
if in resignation. Further on, it is stated that Frog ‘‘was worried.’’ The accompanying
image can be read as exhibiting perplexity, uncertainty, or even anxiety. Another
character reads Frog’s mind stating that, from the symptoms Frog describes, he must
be in love. This realisation makes Frog happy, the image clearly indicating his
emotion, while the words state, in a more subdued manner, that ‘‘he was so
pleased…’’ (Fig. 3).
Yet as a social emotion, love demands that happiness be shared by two agents. In
real life, we often need to read not only one mind, but several at once, and in
interaction with each other. Vermeule (2010) speaks about embedded mind-reading
orders occurring in literature, of the type: ‘‘We think that A thinks that B thinks that
A thinks….’’ In texts with one focaliser we have access to one mind; in texts with

123
Children’s Literature in Education (2012) 43:273–291 285

multiple focalisers, we have access to several minds and have to sort them out,
without necessarily ascribing any of them a higher degree of ‘‘truth.’’ In multimedial
texts, embedded mind-reading is in addition conveyed through the interplay of
verbal and visual information.
Let us first return to Frog, who is sad. Frog is the focaliser in most of the
narrative: the verbal narrator wants us to think (first order) that Frog is sad (second
order); the visual narrator supports this intention. In Frog’s encounter with other
characters, another mind-reading order is added, which is not as easily inferred from
the images alone: ‘‘Little Bear was worried.’’ Here, the reader is asked to understand
that Bear understands Frog’s emotions, which creates another emotion in him:
worry. Unlike internal sadness, worry is a social and directed emotion: we worry
about someone. The text spells out the reason for Bear’s worry; Bear’s empathy
amplifies the reader’s empathy toward Frog, but it also invites the reader to
empathise with Bear.1 In the statement, ‘‘He wanted Frog to be happy,’’ we are two
orders of mind-reading from Frog: we read his mind through Bear’s mind.
Similarly, in Frog in Love, Frog thinks that Duck is not interested in him, which
makes him unhappy; his distress is caused by his inability to read Duck’s mind. As
long as Frog’s happiness is not shared, it turns into unhappiness. Readers’ insight
into Duck’s mind through Frog’s mind is supplemented by direct access to Duck’s
mind: she is happy about her secret suitor, but simply does not know who he is—an
elementary misunderstanding that so much of world literature is built upon. This is
another case of dramatic irony: the readers know more than either of the two
characters. When the (unlikely) lovers are united, the reader is supposed to
experience proxy happiness. This is what happy endings are about.
The narrative in The Tunnel involves two agents, a brother and sister, nameless
until the penultimate doublespread. The story follows their emotional interaction
through words as well as images, which, to begin with, are nearly symmetrical: ‘‘not
at all alike… they were different….’’ The two characters have separate goals
because they have different notions of happiness: ‘‘The sister stayed inside…. The
brother played outside….’’ Their diverse goals lead to mutual disgust—‘‘they fought
and argued noisily’’—and to both characters’ unhappiness as they are forced to
endure each other’s presence. The conflict peaks with a powerful image of discord
(the dump), followed by an intricate sequence of embedded mind-reading in front of
the tunnel. The frontal portrait of the sister leaves no doubt about her feelings: she is
scared and upset because, her appeals notwithstanding, her brother has abandoned
her and crawled into the tunnel. The second order of mind-reading thus involves the
following: the (visual) narrator wants us to think that the sister is scared and upset;
the verbal narrator simply states that she ‘‘was frightened of the tunnel.’’ We can,
however, add several mind-reading orders by asking: what does the narrator want us
to think (first order) the sister thought (second) that her brother thought (third) that
she felt (fourth) when he challenged her to enter the tunnel? According to Vermeule
(2010), our brains can instantly process three to four mind-reading orders, while any

1
I have chosen to use the concept of empathy rather than sympathy, although both terms are used in
cognitive science and cause disagreement between scholars (e.g. Evans, 2001). Cognitive literary
criticism tends to use empathy.

123
286 Children’s Literature in Education (2012) 43:273–291

additional order demands a special effort. My guess is that an unsophisticated

reader, such as a young child, will not go beyond the second order (what the sister is
thinking), but it is impossible to say without substantial research. Presumably, the
mind-reading skill is mastered gradually as we learn to add more orders to our mind-
reading.
Unravelling this complex mind-reading, we can go step by step. What is the sister
feeling, as suggested by the image? Probably: ‘‘I am frightened, upset and angry
with my brother because he has left me behind.’’ We infer this from the girl’s facial
expression, but perhaps also because in our life experience, we too would be angry
and disappointed in a similar situation. Does she know what her brother is thinking
and, if so, how does she know it? Presumably, because she knows that he is always
teasing her, and we know it because we have been told that ‘‘Sometimes he crept
into her room to frighten her, for he knew that she was afraid of the dark.’’ This
phrase contains advanced mind-reading: the narrator wants us to know (first order)
that the girl knows (second) that her brother knows (third) what she feels (fourth).
The girl thus knows that her brother believes that she will be scared of the tunnel. In
order to make this inference, we must go beyond the single image, of the girl in
front of the tunnel, and take the whole text into consideration, just as we can only
fully understand the girl’s feelings in the wordless spread, discussed above, against
previous knowledge. What we by inference learn about the brother is that while he
is capable of empathy, knowing how his sister feels, he still deliberately makes her
frightened and upset. This is contradictory information that prompts an ethical
question: why would anyone deliberately want to harm another person, especially a
person whom he is supposed to love (based on the assumption that siblings love
each other)? Further, wouldn’t a person who inflicts suffering on someone he is
supposed to love feel guilty and thus unhappy? However, these questions only occur
if high-order mind-reading is in place.
Whatever judgement we make about the brother, if any, we still need to decide
what the sister thinks about him and what the narrator wants us to think. Is she angry
with him, besides being frightened? Does she hate him? Hatred is a social emotion
corresponding to the basic emotion of disgust. Is the girl disgusted when she creeps
through the ‘‘dark, and damp, and slimy’’ tunnel, and does she re-direct her disgust
at the brother? Does she secretly admire him? Is she envious of him? Is she afraid of
him? Her genuine emotion toward the brother is never verbalised, but we finally
interpret it as ‘‘Rose loves her brother.’’ Love, as we remember, is a social emotion
implying that we want the object of our affection to be happy, but we also want the
object of our affection to want us to be happy. This is the goal Rose achieves in the
end. The image of her smiling face is confirmed by the words: ‘‘Rose smiled at her
brother. And Jack smiled back.’’ Smiling is not a direct emotion ekphrasis, but, like
an emoticon, an image of a smiling face is an indexical sign, strongly connected
with happiness. Although we do not see Jack’s face, we read his happiness reflected
in Rose’s happy face. The reunion and reconciliation of the siblings, who have
previously been in discord, is an emotionally satisfactory ending for the reader
because the ultimate goal of happiness has been achieved. Since the discord is
stronger than the simple misunderstanding between Frog and Duck, our proxy
happiness is also stronger. Unlike the Frog books, The Tunnel explores a wide range

123
Children’s Literature in Education (2012) 43:273–291 287

of mutually contradictory basic and higher-cognitive emotions within a single

narrative.

Anger and Guilt

Yet both the Frog books and The Tunnel explore emotional interaction between
peers. Maurice Sendak’s (1963) Where the Wild Things Are involves two agents in
an asymmetrical power relationship. In addition, the empowered agent, the mother,
is only represented verbally. Max is the sole focaliser of the verbal narrative and
partial visual focaliser (in the image in which we share his literal point of view). The
mind-reading pattern of the narrative will be, according to Vermeule’s argument, as
follows. The narrator wants the reader to believe (first order) that Max knows
(second) that his mother will be aggravated by his mischief (third). The text utilises
the verbal narrator who expresses the assessment of the situation through stating that
Max ‘‘made mischief’’—a judgemental statement, employing an adult perspective.
The visual narrator expands the verbal statement into a succession or iteration of
images that evoke much stronger emotions than mere ‘‘mischief’’: aggressive and
destructive behaviour is a better description; yet since the emotions are not
articulated they remain vague and open to interpretation. Max’s facial expression
and posture provide a good indication of his state of mind; in fact, a better indication
than a verbal ekphrasis might afford.
The verbal narrator wants the reader to believe that Max’s mother is angry. The
narrator also wants the reader to believe that Max knows that his mother is angry,
and further that the mother knows that Max knows that she is angry. Since we never
see the mother in any image but only hear her voice, we cannot read her emotions
directly (as we do with Max’s face) and can only make inferences from what we
think Max thinks. The verbal utterances of the two agents are meaningless if
interpreted literally; for they are both metaphorical: the mother’s ‘‘WILD THING!’’
and Max’s ‘‘I’LL EAT YOU UP!’’
Max’s face in the image is seemingly easy to read: he is angry. However, the
scope of emotions is more complex, and readers are invited to use their own life
experience and emotional memory to make further inferences. Explicitly, Max is
angry, but implicitly, he is also unhappy. We assume that he loves his mother
because our experience suggests that small children love their mothers. We then
infer that since Max loves his mother and knows that she is angry with him, he
should be unhappy. Further, unhappiness as a basic emotion is associated with the
social emotion of guilt. We think that Max feels guilty because he knows that his
mother, whom he loves, will not only be angry with him, but upset. We know that
deliberately upsetting someone whom we love makes us unhappy and perhaps
makes us feel guilty. The images do not directly suggest this long chain of
inferences, but can still evoke memories of relevant emotions. Such ‘‘deep’’ mind-
reading may seem too complicated to be feasible at all; yet it is exactly because of
the ambiguous nature of visual narrative that several embedded orders are available.
We assume that the mother loves Max because our experience suggests that
mothers love their children. We can bring other experiences into our interpretation,

123
288 Children’s Literature in Education (2012) 43:273–291

as has been done repeatedly, based primarily on the images: that the mother has
encouraged Max’s wildness by giving him a wolf suit; that she carelessly allows
him to play with dangerous objects; or that she neglects the signals he is
communicating with his aggressive behaviour. This advanced level of interpretation
may prompt the reader to believe that the mother does not love Max as much as she
should since she does not care whether he is happy or not. Such interpretation
disputably demands a higher cognitive experience than the primary audience of this
text may possess. Therefore, let us for the sake of argument dismiss this
interpretation and assume that the author wants the reader to believe that the mother
loves her son, as she should.
The reader knows that Max knows that his mother loves him. This knowledge
comes from a later statement in the narrative referring to ‘‘someone who loved him
best of all,’’ which is Max’s inference about his mother’s feelings (third order). At
the beginning of the story, the reader must leap over the temporary mind-reading—
‘‘Max knows that his mother is angry with him’’—in order to get to the permanent
mind-reading—‘‘Max knows that his mother loves him’’ (third order)—and further,
to ‘‘Max knows that his mother knows that he knows that she loves him’’ (fifth
order). The latter inference demands two additional orders, and most readers
probably do not get that far; yet the depth of the narrative certainly allows it.
In the following sequence of images, mind-reading is relatively easy, since Max
becomes the sole agent, and we only need to read his mind. We can clearly see the
changes in Max’s emotions through his facial expression and posture, despite the
lack of any verbal evidence. He is happy sailing in his boat, scared of the first
monster, and angry with the next set of monsters. In addition, his emotions are
represented metaphorically through visual images: the monsters. The verbal text
stops altogether when emotions become too strong, and the three wordless
doublespreads convey Max’s state of mind on a subverbal level. Then the aggressive
emotions are subdued, and Max’s mood changes again. We know that Max is sad
because of his physical appearance, supported by the statement that he ‘‘was
lonely.’’ Loneliness is a social emotion caused by separation from the object of our
love. We believe that Max is sad because he thinks that his mother is still angry with
him, even though the words say that he wanted to be with someone who loved him
best of all. We are allowed the choice of believing that the mother is still angry with
him or that she is not angry any more. Moreover, we must reconcile these two
contradictory ideas: that the mother can love Max and be angry with him at the same
time, something that young readers allegedly are unable to do. Here a complex
social (and permanent) emotion is pitched against a basic and temporary one.
Presumably, immature readers will go with the basic emotion. Further, we must
acknowledge the co-existence of two emotions, yet allow for the possibility that
Max, as a young child, is unable to reconcile them. Since we have no visual support
for reading the mother’s mind, we are wholly dependent on our cognitive and social
skills.
Back in his room, Max is smiling, and we are supposed to believe that he is
happy because he thinks that his mother loves him. What his mother really thinks is
beyond our knowledge, yet an immature reader will most likely infer that since she
has given him his supper she loves him (based on the assumption that a person who

123
Children’s Literature in Education (2012) 43:273–291 289

loves someone wants to make them happy). The symbolic representation of love
through food in children’s literature has been noted repeatedly. Yet the disturbing
part of this narrative is that the mother leaves supper for Max rather than staying in
his room and giving him a hug. She thus substitutes material gratification for
emotional engagement. Her absence from the image guides my implicit mind-
reading of the mother, derived exclusively from her actions: that she is indifferent
toward her child or in fact still angry with him, which he does not know and which
an immature reader will, most likely, fail to consider. Interestingly enough, just as
the verbal narrative stopped when Max’s emotions grew too vivid, the visual
narrative is cut off when there are no emotions involved. The ultimate emotion the
readers are left with depends on how much they delve into the embedded mind-
reading orders. Although Vermeule (2010) does not state this, it could be implied
from her argument that the quality of a literary work can be judged by the number of
plausible mind-reading orders it offers. Where the Wild Things Are, published in
1963, is still considered one of the most psychologically complex multimedial
narratives, presumably not least because of its intricate mind-reading orders which
only now can we analyse in detail, with the emergence of cognitive narratology.
(I could argue that the recent film transmediation of Sendak’s text has reduced the
number of orders and thus failed to convey the complexity of the book.)

I started my argument by claiming that empathy is an indispensable social skill

and that fiction can provide a training field for young readers’ development of mind-
reading. As brain research demonstrates, empathy is not a natural capacity. The
complexity of emotion ekphrasis in fiction puts high demands on the maturity of the
reader. Readers are expected to be able to empathise even with characters whose
emotions they do not recognise directly, and sharing emotions is one of those
vicarious experiences that fiction can offer us as readers. Moreover, emotion
ekphrasis inevitably leads to ethical issues, since readers need to assess characters’
actions by probing into their mental states (Nikolajeva, 2012). If so, our
understanding of young readers’ affective responses becomes an essential ideolog-
ical and educational issue.
My intention in the present article has been to show how fiction can potentially
teach even very young children to read other people’s minds, and how multimedial
texts in particular can stimulate this skill. I have also shown how even in seemingly
simple narratives there is room for various interpretations of emotion ekprasis as
long as the verbal text is not too intrusive. The main problem with representing
emotions in a verbal literary text is that language is not sufficiently adequate to
convey complex emotional states, something that Jacques Lacan and his followers
emphasise, and that narratologists such as Dorrit Cohn (1978) have discussed.
Multimedial studies show that images are, in Cohn’s terms, subverbal or preverbal;
they can take over when words become ineffective. Thus multimedial texts offer
excellent opportunities for mind-reading skills to readers without accomplished
verbal literacy, but certainly also contribute to the development of empathy in any
reader. We have always known by intuition that reading is valuable; cognitive
criticism suggests a neurological basis for the value of reading: it provides a way of
helping us understand other human beings.

123
290 Children’s Literature in Education (2012) 43:273–291

As already pointed out, my argument is speculative and has no empirical work to

fall back on. Hopefully, it will inspire scholars who work with books and children to
test the ideas developed here. I anticipate many surprises from such research.

References

Primary sources

Browne, Anthony. (1987). The Tunnel. London: Julia McRae.

Haddon, Mark. (2003). The Curious Incident of the Dog in the Night-time. London: Jonathan Cape.
Sendak, Maurice. (1963). Where the Wild Things Are. New York: Harper.
Tan, Shaun. (2001). The Red Tree. Sydney: Hachette.
Velthuijs, Max. (1989). Frog In Love. London: Andersen.
Velthuijs, Max. (1994). Frog Is Frightened. London: Andersen.
Velthuijs, Max. (1996). Frog Is Frog. London: Andersen.
Velthuijs, Max. (2003). Frog Is Sad. London: Andersen.

Secondary sources

Blakemore, Sarah-Jayne, and Frith, Uta. (2005). The Learning Brain: Lessons for Education. London:
Wiley Blackwell.
Carr, Nicholas. (2010). The Shallows: How the Internet is Changing the Way we Read, Think and
Remember. New York: Atlantic Books.
Cohn, Dorrit. (1978). Transparent Minds: Narrative Modes for Presenting Consciousness in Fiction.
Princeton, NJ: Princeton University Press.
Crane, M.T. (2001). Shakespeare’s Brain: Reading with Cognitive Theory. Princeton, NJ: Princeton
University Press.
Evans, Dylan. (2001). Emotions: A Very Short Introduction. Oxford: Oxford University Press.
Feagin, Susan L. (2007). ‘‘How Could You?’’: Deeper Understanding Through Fiction. In John Gibson,
et al. (Eds.), A Sense of the World. Essays on Fiction, Narrative and Knowledge (pp. 55–66). New
York: Routledge.
Gavins, Joanna, and Steen, Gerald (Eds.). (2003). Cognitive Poetics in Practice. London: Routledge.
Gibbs, Raymond W. (1994). The Poetics of Mind: Figurative Thought, Language and Understanding.
Cambridge: Cambridge University Press.
Gibson, John, Huemer, Wolfgang, and Pocci, Luca (Eds.). (2007). A Sense of the World: Essays on
Fiction, Narrative and Knowledge. New York: Routledge.
Hogan, Patrick Colm. (2003a). Cognitive Science, Literature and the Arts: A Guide for Humanists. New
York: Routledge.
Hogan, Patrick Colm. (2003b). The Mind and Its Stories: Narrative Universals and Human Emotion.
Cambridge: Cambridge University Press.
Hogan, Patrick Colm. (2011). What Literature Teaches Us about Emotions. Cambridge: Cambridge
University Press.
Keen, Susanne. (2008). Empathy and the Novel. Oxford: Oxford University Press.
Nalbantian, Suzanne. (2003). Memory in Literature from Rousseau to Neuroscience. Basingstoke:
Palgrave.
Nikolajeva, Maria. (2012). Guilt, Empathy and the Ethical Potential of Children’s Literature. Barnboken,
35(1).
Oatley, Keith. (1992). Best Laid Schemes: The Psychology of Emotions. Cambridge: Cambridge
University Press.
O’Shea, Michael. (2005). The Brain. A Very Short Introduction. Oxford: Oxford University Press.

123
Children’s Literature in Education (2012) 43:273–291 291

Stephens, John. (2011). Schemas and Scripts: Cognitive Instruments and the Representation of Cultural
Diversity in Children’s Literature. In Kerry Mallan, and Clare Bradford (Eds.), Contemporary
Children’s Literature and Film: Engaging with Theory (pp. 12–35). Basingstoke: Palgave.
Stockwell, Peter. (2002). Cognitive Poetics: An Introduction. London: Routledge.
Tsur, Reuven. (1992). Towards a Theory of Cognitive Poetics. Amsterdam: North Holland.
Turner, Mark. (1996). The Literary Mind: The Origins of Thought and Language. New York: Oxford
University Press.
Vermeule, Blakey. (2010). Why Do We Care about Literary Characters? Baltimore: The Johns Hopkins
University Press.
Walsh, Susan. (2011). Irony and the Child. In Karin Lesnik-Obertein (Ed.), Children in Culture,
Revisited: Further Approaches to Childhood. Basingstoke: Palgrave.
Wolf, Maryanne. (2007). Proust and the Squid: The Story of Science and the Reading Brain. New York:
HarperCollins.

123