See discussions, stats, and author profiles for this publication at: https://www.researchgate.

net/publication/281069602

When image and text meet - teaching with visual and multimodal texts

Article · January 2011

CITATIONS READS

0 1,887

1 author:

Jon Callow
The University of Sydney
40 PUBLICATIONS   193 CITATIONS   

SEE PROFILE

All content following this page was uploaded by Jon Callow on 26 May 2016.

The user has requested enhancement of the downloaded file.

 PETAA
Paper 181
When image
and text
meet:
teaching with visual
and multimodal texts
Jon Callow
http://www.advertolog.com/wwf/print-outdoor/lungs-11711955/

Being immersed in a visual culture, we can sometimes not even This means that we need to make sure
notice how we interact with images and words. This photo of we don’t just read a text like a picture
the logged forest catches our eye, summoning the image of a book, talk about one or two literary
pair of lungs. The poster comes together with the conservation features, discuss which picture we like
group’s logo and the tag line ‘Let the planet breathe’, making the best then move on to the next activity.
a powerful point. Multimodal texts make meaning because
all the elements work together to create
Engaging with various texts can sometimes seem a simple,
a whole text. This is the case for picture
everyday act, belying the complexity involved. Until something
books, using only image and text, or for
catches our eye, jarring us, or delighting us with its simplicity,
video and multimedia, where image,
its beauty or its drama, we may not stop and reflect more
gesture, movement, words and sound all
deeply about what we have seen or experienced. It is when
work together to create the final piece.
we do stop that we wonder how the author, artist or designer
communicated their message so effectively.

The concept of multimodal texts is integrated in the Australian

This e-update will:
Curriculum: English, across the language, literature and literacy
strands. From understanding social contexts, expressing and ●● explore multimodality, with a
developing ideas through language and literature to creating focus on image and text
new texts, the multimodal nature of English is deeply embedded ●● review key visual metalanguage
in the rationale and strands of the curriculum. Teamed with
●● demonstrate how to plan
a knowledge of written language is the inclusion of a visual
effective modelled viewing
language thread (ACARA, 2011:66), as well as assessment lessons using picture books
linking to multimodal texts, expressed in the achievement
standards from Foundation to year 10. ●● present examples of annotated
picture book images to guide
A key concept to understand when working with multimodal classroom practice
texts is that the whole is greater than the sum of the parts.

© 2011 PETAA – Primary English Teaching Association Australia. PO Box 3106 Marrickville Metro NSW 2204 Tel 61 (0)2 8020 3900 www.petaa.edu.au ISSN 1442603X 1 ......
 http://www.advertolog.com/wwf/print-outdoor/shark-13804905/

Consider this advertisement. Initially we is powerful because it sets up a question in the viewer’s mind,
are drawn to the fin on the left, perhaps a hook to get attention. Reading left to right is also important
reacting with a sense of fear. The caption here. We move from the shark being present – the ‘given’,
reinforces this, yet when we look to the to its apparently sudden disappearance – the ‘new’. This new
right, the fin is gone, but the caption information is possibly more unsettling. The dark blue colours
suggests this is ‘more horrifying’. Isn’t further add to a fearful atmosphere, as well as the fact that the
this a contradiction? Is it because the fin in the first photo (and by association the shark) is framed
shark is under the water? What could the to appear quite close to us as the viewer.
advertisement mean? Reading the text on
the bottom, the purpose becomes clearer The placement of the large photos at the top, with the more
– the loss of wildlife has an impact on the detailed text and logo down the bottom is also common. Many
ecosystem. This is the same ecosystem advertisements put their main ideas at the top, considered the
on which humans also depend. The ‘ideal’ in western culture, while the more everyday aspects of
poster cleverly plays with fear of sharks the product, such as the details, ingredients or specific features,
and the juxtaposition of image and word, are often at the bottom, symbolically grounded in what we can
in order to convey its message. term the ‘real’.

The written text here is also powerful – ‘Exploiting the ecosystem

We can learn through multimodal text. Our
also threatens human lives’. Strong action verbs like exploiting,
emotion and intellect have been engaged.
and threatens create an immediate emotional reaction, especially
However, we can also learn about this
when linked to the noun group human lives.
advertisement, by understanding how
both the written and visual elements work, From a critical literacy aspect, we can also ask about their
as well as reflecting on what world-view world-view as advocates for wildlife. Is the information true –
or ideology is being presented (Callow, would the loss of sharks really impact us as humans? Should we
1999; Kress & Van Leeuwen, 2006). think about supporting this organisation? Reflecting critically
allows us the space to step back and consider the points of
Visually the fin is the most salient
view presented. (See Callow, 2008.)
element in the poster, with the adjective
‘horrifying’ shaping our interpretation Exploring multimodal texts can range from advertisements,
and response, as well as augmenting the posters and videos to one of the mainstays of any English
image. This juxtaposition of two images classroom – the picture book.

...... 2 © 2011 PETAA – Primary English Teaching Association Australia. PETAA Paper 181: When image and text meet
Modelled viewing and step and explore how they work together
(Walsh, 2010). Pedagogically this will
picture books include reading the whole text, revisiting
key pages and events in the story,
Teaching viewing using picture books provides both exposure
explicitly teaching an aspect of the visual
to quality literature as well as a stimulating variety of visual
and written text and how they work
images and techniques. The following provides a guide for
together, then consolidating the concepts
choosing and using pictures books with a focus on the visual
and skills in a concluding activity.
and written text interplay.
In a modelled viewing lesson (whole class
Read and enjoy or small group), a teacher demonstrates
key visual and written features as well
Having chosen a piece of quality literature, make sure you
as involves their learners in a variety
read it and enjoy it, as your own knowledge and enthusiasm is
of discussions and hands-on activities.
important and will always positively influence your teaching.
Within one lesson, a teacher may move
across a scaffolding continuum, from
Literary themes and features more teacher-regulated practices (reading
As you are teaching a whole text, consideration of the broader the text, making statements about key
literary features should inform your lesson planning (Saxby, features, asking focused questions about
1997). These elements are also key aspects of the Australian features) to more scaffolded activities,
Curriculum: English. Features include: where students take some responsibility
for practising the demonstrated skills or
● Theme – ideas that hold the story together, such as
knowledge (Hammond, 2001). Ideas for
friendship, survival, family, courage
activities can include:
● Plot – traditional narrative structures or variations on this
● sorting a selection of images into
● Characters – major and minor characters, heroes and shot distances or types of angles
villains, animals as characters
● listing key features such as salient
● Point of view – first person or third person points, characters that demand
viewer attention or effective use
● Place or setting – realistic (urban, city, bush), fantasy
of colour
or imaginary, historical
● re-drawing a character so that they
Choose a key page from the story either look at the viewer (demand)
or away from them (offer)
There are often key moments in a story, which are represented
in word and image. When planning, choose a key page, note ● creating a frozen moment scene
down the main ideas and the visual and verbal features that are from the story and using a digital
important in creating the ideas and feel of the page (Refer to camera to experiment with
the samples on the following pages). Table 1 gives a summary angles, framing or shot distance.
of features that may be present. Only choose the features that
are relevant or strongly represented on the page.
In the classroom
Scaffolding multimodal meanings Consider the example of Fox on the
The middle column in the table brings together the visual following page, which shows an overview
and written aspects of the texts to ensure that the multimodal of a chosen double page spread. The
aspects are explored. Rather then just listing specific visual following ideas are suggestions for
features and grammatical aspects, we need to take that extra planning a modelled viewing lesson.

© 2011 PETAA – Primary English Teaching Association Australia. PETAA Paper 181: When image and text meet 3 ......
 TABLE 1 ● MULTIMODAL MEANINGS – IMAGE AND TEXT
Visual Multimodal meanings Written
(Concepts of print and screen; Visual Guiding principles to consider (Structures of different types of
Language*) when image and text meet texts; Sentence and clause level
grammar; Word level grammar;
Vocabulary*)

Happenings In the context of the unfolding Events and Ideas

Actions and settings – how line and shape story, how does the page Sentence and clause structures
represent people, objects and events, develop the characters, story – building happenings and ideas
including speech and thought events and build themes and through different verb types
conceptual and symbolic –images showing concepts? How do the visual and adverbs/ adverbials
ideas and written work together? – description of characters and
setting through noun groups
and adjectives

Reacting and relating How do the visual and written Interacting

shot distance – long, mid or close text shape our feelings and statements, questions and
angle – high, low or eye level reactions to the characters? commands
gaze of characters – offers and demands Do we react differently to the attitudes and opinions
colours – moods and symbolism visuals than to the written text? modality – degrees of certainty
shapes and textures and their impact Do the literary features literary features – alliteration ,
realism of images – photo, artwork, drawings, complement or contradict the repetition, rhyme, personification,
diagrams visual elements? metaphor; interesting vocabulary
choices

Design and layout How does the layout of Text organisation

balance – symmetry and contrast of elements the visual and written text genre/ text type –literary
top/bottom, left/right, centre/margin – influence the reader/viewer? (narrative, ballad, recount etc.) or
placement of elements Does the layout privilege factual
salience – what attracts attention particular visual or written paragraphs
reading paths – vectors directing our gaze elements? Why might the sentence order and structure
framing – how text and image may frame author/illustrator have made
each other these choices?

* - denotes thread from Australian Curriculum: English (ACARA, 2011)

For more details on visual grammar, see Callow, 1999, and for more details on written grammar, see Derewianka, 2011

Before Reading

Building prior knowledge During Reading

Discuss the cover, finding out what Have a balanced number of questions and think-aloud statements that
preconceptions learners have about foxes. allow the story to unfold, as well as prompt comprehension for events
List words associated with foxes and themes.
Explain the concept of a fable: a short story
with a moral Think aloud statements and questions
Dog and Magpie build a strong friendship at the start, as Dog is faithful
Prediction and kind.
Could this Fox be a hero or villain? Why is fox always watching? How does he tempt Magpie? Should
Look at the endpapers - what does the Magpie go with Fox?
harsh red paint suggest? The descriptions of Fox as having ‘haunted eyes’ and a ‘rich red coat’
Why is there a dog with a magpie in its are very effective.
mouth before the title page? Why does Fox leave her in the desert?
Is Fox’s final cry a scream of triumph or despair?
After viewing the final endpapers, discuss whether Magpie gets home
safely or not.

...... 4 © 2011 PETAA – Primary English Teaching Association Australia. PETAA Paper 181: When image and text meet
Fox by Margaret Wild and Ron Brooks

Themes: friendship, loyalty, betrayal

Synopsis: A fable of friendship and treachery where Dog and Magpie’s

relationship is challenged by Fox’s arrival in the bush.

Key features and context of image

As the orientation moves into the complication of this narrative, Fox
arrives after Dog and Magpie have become close friends. A feral animal,
he doesn’t belong either in the bush or in Dog and Magpie’s friendship.
This page foreshadows the betrayal by Magpie, with Fox coming between the two friends, visually and
in the written text. Fox’s flattery sees Magpie initially recoil in fear but her feelings change as his words
influence her in the unfolding story.

TABLE 2 ● MULTIMODAL MEANINGS – AN ANALYSIS OF FOX

Visual Multimodal meanings Written

Happenings – Magpie is pulling away, Fox says one thing but does he Events and Ideas – “You looked
laying on back, showing her fear. Dog is mean another? Does his image extraordinary” – a flattering statement
looking off the page, appearing trusting match his words or contradict but is Fox telling the truth?
or possibly naive? Fox stares at Magpie – them? What do the words Dog beams but Magpie shrinks away –
are his intentions friendly or evil? and image of Dog and Magpie use of ‘but’ joining the clauses shows
suggest about their future contrast between the two characters.
friendship?

Reacting and relating – The colours How does the use of colour Interactions – Strong verb choices
give an earthy feel, with Fox’s red build up our feelings about build tension in the story.
standing out as he dominates the page. each character? Are the strong Magpie shrinks away – shows her fear
Red is symbolic of danger, influencing verb choices reflected in the Dog beams – shows his innocence
our emotional response, as does the image? Or do the images tell us Fox staring at her burnt wing –
messy, handwritten lettering. something more? suggests evil

Design and layout Why did Ron Brooks put the Text organisation – No clear
Fox and Dog on the left, opposite Magpie speech box vertically? Because paragraphing or text layout adds to
on the right, gives an unbalanced feel. it is askew, does it make the confusion and the conflict that is
Magpie is now separated, small and words in it mean something growing. Fox’s words and actions
vulnerable at the bottom of the page. different? (look for other create fear in Magpie.
Fox is at the top, in the ideal position, examples in the story where the
appearing dominant. text isn’t horizontal. What does
Fox’s speech box is vertical. Both Fox it signal about the characters’
and his speech box come between Dog relationships?)
and Magpie. His words are the centre of
the page and the centre of attention too.

Teaching strategies Colour

Show the double page spread, either using the book or Discuss possible symbolic meanings of
projected on a screen or an interactive whiteboard. the colour red – danger, power, strength,
romance, passion. Do any of these apply
Vectors and gaze to Fox?

Ask a student to trace along the gaze of Fox to Magpie (or draw
an arrow on the IWB). Re-read Fox’s statement. Pose some of
the questions in the multimodal meanings column (see Table 2).

© 2011 PETAA – Primary English Teaching Association Australia. PETAA Paper 181: When image and text meet 5 ......
 Layout Written grammar
Discuss Fox’s flattery. Point out how Box some of the strong verbs and discuss their impact using
his words and body are curving around the multimodal meanings questions.
Magpie. How does she react? How do we
feel about Magpie? Key narrative point
How do the text and pictures mark a change in the story? How
Reactions does this scene change the friendship between Magpie and
Circle or label the eyes. Do they look at Dog? How does it make us feel about each character?
us (demand) or elsewhere (offer)? What
does this tell us about the character’s
feelings or intent?

- Circle each character’s eyes - who is Fox and his words are in the centre & at the top of the page.
looking at who? Do they look at us? Both are framing Magpie, trapping her in the corner. How is
Magpie reacting? How do we feel about Magpie?

Strong vector from Fox’s gaze - who has the power?

Draw a line down the middle of the page.
Dog and Fox are on the left, separated from Magpie. - Box the powerful verbs e.g beams, shrinks,
Why might Ron Brooks have done this? staring. Do they complement or contrast
what we see?

Visual literacy and Interactive White Boards

The interactive tools and techniques available on Interactive White Boards extend the creative
ways of teaching about multimodal and visual texts (see Goodwin, K. e-update 017, 2011)
Pens and highlighters Manipulate elements
Ask students to annotate an image, circle key Using elements from a page (such as
features or draw vectors and reading paths characters or objects), re-design the page to
tell a different story, re-size them to make
Drag and drop one element salient, or create text boxes to
Match labels to parts of the page such as manipulate the font, size and the position of
description of angles or shot distances, noun text on the screen.
groups or verb types.

...... 6 © 2011 PETAA – Primary English Teaching Association Australia. PETAA Paper 181: When image and text meet
Gleeson, L., & Blackwood, F. (2009). Clancy & Millie and the
very fine house
Themes: friendship, play, imagination

Synopsis: Clancy moves to a new house, which is much too big and very
lonely. However, Clancy meets Millie and together they find friendship,
building towers, trains and houses for the three little pigs.

Key features and context of image: This second double page spread
of the book contrasts strongly with the first more colourful one, where
the three little pigs cavort across the sky. As orientation blends into
complication, this page plays with angles, proportions and colour, placing Clancy in a home that is
overwhelming in its size and loneliness. Visually this pattern is repeated over the next four pages.

Table 3 presents the multimodal meanings that can be read into this text.

TABLE 3 ● MULTIMODAL MEANINGS – AN ANALYSIS OF CLANCY & MILLIE AND

THE VERY FINE HOUSE
Visual Multimodal meanings Written

Happenings – Clancy and his parents, The written text provides the Events and Ideas – The second
appearing very small, stand outside and dialogue here, while the visual part of the sentence from the
gaze upwards at the imposing new house. gives more information about the previous page (Clancy has moved
The cloudy shape of a pig leaps back to the characters. What do the pictures from this house ) is completed
previous page, with the faint outline of a tell us about what they are doing here, to reinforce the contrast
wolf chasing it. and where they are looking? between homes (.. .to this house).

Reacting and relating – Like the How do the colours make us Interacting – Each character
characters, we are at a low angle, with the react? Why do you think Freya makes a statement about the
buildings purposely drawn with tall, angled Blackwood drew the buildings house. Clancy’s parents describe
walls leaning ominously forward. Using a on an angle? We are low down, the house as new, best, and a very
long shot, Clancy and his parents appear looking up at them – how does it fine dwelling. Tellingly, Clancy
small and powerless. The colours are drab make us feel? whispers rather than says that
with dirty browns, further affecting our it is ‘too big’. Clancy’s mother’s
feelings towards the house. positive attitude and comments
continue over the next four pages,
while Clancy yearns sadly for his
old home.

Design and Layout – The very small The words describing the house Text organisation – The written
characters are in the lower left-hand side, are on the left. The picture of the text is also in the lower left-hand
looking powerless, while the buildings, house is on the right – do they side, where the description sits
made salient by their size, are on the complement or contrast each in opposition to the house being
right. Visually, the right-hand side can other? Why are the houses made described.
be understood as what is ‘new’ in the to look so big? How do you think
unfolding narrative. it makes Clancy feel?

Teaching strategies Reactions - Clancy’s parents sound

confident when they describe the house.
After having read the book, come back to the first two pages.
Do they look confident?
Use the annotated image to explore angle, size and descriptions.
Key narrative point - How do the
Vectors and gaze - Circle Clancy and his parents. Trace
story and pictures tell us about Clancy’s
the gaze of their eyes. What are they looking at? Why?
feelings? Are the next pages similar? Keep
reading to see if there is a pattern.

© 2011 PETAA – Primary English Teaching Association Australia. PETAA Paper 181: When image and text meet 7 ......
 Circle Clancy and his parents - how big or small are they
Can you see a wolf in the clouds? What feel does he give?
compared to the house? Why might they be drawn this way?

Box the noun groups describing the house - compare Trace or draw along the vertical lines of the houses - how do
them to the pictures. Why might they be different? you think the angle of the houses makes the family feel?
A low angle shot can often make the viewer feel powerless

Conclusion
Working with multimodal texts provides literacy skills as part of engaging with and enjoying the rich
opportunities to develop our learners’ visual culture we live and learn in.
love and interest in literature, enhance
For further information and resources for teaching about visual
their knowledge of visual and written
and multimodal texts, visit the website at www.petaa.edu.au
language, as well as develop their visual
Refere nc es
ACARA. (2011), The Australian Curriculum: English Available from www.australiancurriculum.edu.au/English/Curriculum.
Callow, J. (1999), Image Matters – Visual Texts in the Classroom, PETAA, Newtown, Sydney.
Callow, J. (2008), ‘Show Me: Principles for Assessing students’ visual literacy’, The Reading Teacher, 61(8), 616-626.
Derewianka, B., (2011), A new grammar companion for teachers (2nd ed.), PETAA, Newtown, Sydney
Goodwin, K. (2011), ‘Engaging students in literacy learning with interactive whiteboards’, e:update, 017, PETAA, Newtown, Sydney
Hammond, J. (2001), Scaffolding : teaching and learning in language and literacy education, PETAA, Newtown, Sydney
Kress, G. R. & Van Leeuwen, T. (2006), Reading images: the grammar of visual design (2nd ed.), Routledge, London, New York.
Saxby, H. M. (1997), Books in the life of a child : bridges to literature and learning, Macmillan Education Australia, South Melbourne.
Walsh, M. (2010), Multimodal literacy: researching classroom practices, PETAA, Newtown, Sydney
While every effort has been made to trace and acknowledge copyright and ownership of all included works, should any infringement have
occurred, the publishers offer their apologies and invite copyright owners to contact them.

About the author

Dr Jon Callow is an experienced educator and teacher, having worked in primary schools, universities and in
professional development for teachers. He currently teaches at the University of Sydney, in the areas of English and
multiliteracies and he has worked alongside teachers in their classrooms, both in Australia and the United States.
His research interests include visual literacy, children’s literature, social justice and pedagogy and digital media.

PE TAA Papers
Originally published as PENs (Primary English Notes 1 to 162), and more recently known as e:updates (1 to 18), these informative monographs
or articles, are now known as PETAA Papers. Following the 2011 change to PETAA (Primary English Teaching Association Australia) and to
recognise the rich history of the 180 previously published PENs and e:updates, it was decided to continue the number progressively from PEN 1
and thus the first PETAA Paper is numbered 181.

...... 8 © 2011 PETAA – Primary English Teaching Association Australia. PETAA Paper 181: When image and text meet
View publication stats