Visualization Viewpoints

Editor:
Theresa-Marie Rhyne

More Than Telling a Story: Transforming

Data into Visually Shared Stories
Bongshin Lee and Nathalie Henry Riche
Microsoft Research

Petra Isenberg
Inria

Sheelagh Carpendale
University of Calgary

V isualizations are increasingly used “in the

wild” to tell compelling stories supported
by data. Examples include Hans Rosling’s
talks on human development trends using Gap-
minder (gapminder.org) and the use of visualiza-
covering the aspects of the visual data storytell-
ing process that have so far received less research
attention. We discuss the entire process of trans-
forming data into visually shared stories, incorpo-
rating steps involved in finding insights (explore
tions on the Web (see Figure 1 for an example) data), turning these insights into a narrative
including news media such as the New York Times (make a story), and communicating this narrative
and the Guardian. Given the practical power visu- to an audience (tell a story). We conclude with
alizations can have in communicating facts and research opportunities in visualization as a story-
opinions, the visualization research community telling medium in the light of this broader process.
has begun to pay more attention to the need and
use of visualization as a storytelling medium to Storytelling Scope
tell compelling data stories. The word “storytelling” has a long history during
In this article, we take a closer look at how the which it has been used in many ways in differ-
visualization community has discussed visual sto- ent domains. As a result, it invokes meaning and
rytelling. We argue that so far the community nuance without having one single agreed-upon
has been using the term “storytelling” in a broad definition. Yet, most descriptions of storytelling
way, without a clear consensus or discussion on require some sort of controlled delivery or pre-
what a visual data story encompasses. Yet, despite sentation of information.1 This delivery can be
this wide view on what constitutes a data story, with or without a predefined temporal or narra-
visualization research on storytelling has mainly tive structure, but it always contains components
centered on how data visualization components that form the “story” (structures, elements, and
contribute to communication. Instead, we argue concepts) and those that influence the “telling”
for first narrowing the scope of what visual data part of storytelling (people, tools, and channels).
stories encompass and then expanding our re- Similarly, no single crisp definition of a “vi-
search focus to the whole process of transforming sual data story” has emerged. We argue against
data into visually shared stories, including forma- using the broadest sense, which would consider
tive processes such as crafting narrative structures. any shared visualization to be a data story, given
Narrowing the scope of what is termed a data that visualization is by definition focused on mak-
story, for instance, by distinguishing between a ing data visible—and thus on presenting data to a
visual data story and a data visualization, helps viewer. A visualization can be thought of as hav-
us open the door for a more detailed examination, ing both design elements that form a story and

84 September/October 2015 Published by the IEEE Computer Society 0272-1716/15/$31.00 © 2015 IEEE

g5vis.indd 84 8/26/15 12:04 PM

 presentation methods that have been used to tell
it.2 If we take this broadest view on visual data
stories, however, any images containing even sim-
ple charts with little explanations or reading aids
would be called visual data stories. To facilitate a
more practical discussion on the structure of vi-
sual data storytelling in the visualization commu-
nity, we need to consider narrowing the scope of
visual data stories.
Several research papers have considered this
problem of defining what storytelling in the do-
main of visualization encompasses. Edward Segel
and Jeffrey Heer introduced the term narrative vi-
sualization, emphasizing visual stories that include
a narrative, or a series of causally related events.3
They also showed that storytelling in the context
of visualization cannot be separated from talk- Figure 1. Example visual data story. A full Flash presentation, “Human
ing about visual narratives, but it still needs to be Development Trends, 2005,” is available at www.gapminder.org/
more concretely defined—in particular because the downloads/human-development-trends-2005.
differences and commonalities to traditional sto-
rytelling mechanisms and strategies are not well the author’s high-level communication goal,
understood yet. They attempted to develop a bet- which can range from educating or entertaining
ter understanding of visual data stories by formu- the viewers with illustration of facts to convinc-
lating the design space of narrative visualization. ing or persuading them with thought-provoking
Jessica Hullman and Nicholas Diakopoulos began opinions.
with Segel and Heer’s term narrative visualization
and defined it as a genre that combines interac- Taking this narrower view of a visual data story,
tion techniques for exploratory control over in- we can exclude several types of stories not related
sights gained and communicative, rhetorical, and to visualization, allowing us to uncover research
persuasive techniques for conveying an intended opportunities relevant to the visualization com-
“story.”4 Similar to Segel and Heer, the authors munity. For example, typical stories told in books
did not clearly define what constitutes a visual and movies are not pertinent to our discussion un-
data story but aim to better describe the visual- less they are focused around data visualizations. In
ization techniques involved in communicating an addition, we exclude Web-based interactive visual-
intended message. Although these two approaches izations that support completely free exploration
are a great first step, as they are valuable for un- without any guidance. We also do not consider
derstanding communication mechanisms for visu- visual data stories to include, for example, charts
alization in general, it is still unclear exactly what posted on the Web unless they are enriched with
is or is not a visual data story. written explanations or annotations that help the
In discussing the process of transforming data viewer capture the intended message. For example,
into visually shared stories, we propose that visual many visualizations that include only a title above
data stories can be thought of as follows: the chart usually leave the interpretation of the
content entirely to the viewer.
■■ A visual data story includes a set of story pieces— It is important to note that these last examples—
that is, specific facts backed up by data (such as simple charts created from data—can be turned into
how energy consumption has changed over the visual data stories according to our narrower scope
years). if the author added appropriate annotations for a
■■ Most of the story pieces are visualized to support simple story or combined several to form a longer,
one or more intended messages. The visualiza- more complex story. However, this process of turn-
tion includes annotations (labels, pointers, text, ing data into a visual story has so far received little
and such) or narration to clearly highlight and attention. We argue that it is just as important to
emphasize this message and to avoid ambiguity consider research on the process of creating a visual
(especially for asynchronous storytelling). data story as it is to understand which features best
■■ Story pieces are presented in a meaningful order communicate it. We describe such a possible pro-
or with a connection between them to support cess and open areas for research next.

IEEE Computer Graphics and Applications 85

g5vis.indd 85 8/26/15 12:04 PM

 Visualization Viewpoints

Explore data Make a story Tell a story

Gather facts

Data analyst Scripter Editor Presenter and audience

Provide
input

Data Excerpts Story pieces Plot Story material Shared story Perceived story
Make
logical Build Share
connection presentation story
Explore and analyze Order story pieces Respond to input

External factors
Settings

Target audience Medium

Figure 2. Storytelling process. The visual data storytelling process involves transforming data into visually shared stories via three
main components (exploring data, making a story, and telling a story), although the progression need not be linear.

Visual Data Storytelling Process the visual data story content. Pertinent data ex-
In the past, few research papers have considered cerpts are collected through exploratory analysis.
a wider visual storytelling process. An exception These may be simple, such as recorded data facts or
is the article by Robert Kosara and Jock Mackin- steps from the analysis process. They may be more
lay,5 which aimed to provide a high-level overview complex, such as derived data insights, interesting
of the state of storytelling research. They briefly sets or sequences within the data, and/or process
mentioned a working model for how stories are details and variations. They may include the first
constructed based on how journalists work and ar- quick externalizations of the data, such as charts
gued that visualization tools for analysis are rarely from spreadsheets or hand sketches made during
usable for presentation purposes. the analysis. At this point in the process, this col-
Here, we contribute a much more detailed de- lection of excerpts may or may not be tied to any
scription of the storytelling process in visualiza- specific visual representation. The result of data
tion with regard to activities, artifacts, and roles exploration when making a visual data story is a
involved to develop a more encompassing look at collection of the chosen data excerpts.
the visual storytelling process and to uncover open
areas for research. We propose the visual data sto- Making a Story
rytelling process (VDSP) (see Figure 2) as a work- To make a story, the data excerpts gathered in step
ing model, which is derived from other models in one need to be assembled into a storyline that is
the data journalism literature.6 The VDSP summa- interesting, illuminating, and compelling. The se-
rizes the main roles and activities that visualiza- quence plays a critical role in a story; the same
tion storytellers engage in as they turn raw data set of excerpts can have impact or can fall flat. A
into a visually shared story, along with the types of significant part of making a story is the process
artifacts that result from these activities. of constructing the storyline or plot. The activities
While the storytelling process is linear in Figure involved are ordering, establishing logical connec-
2 for illustration purposes, it does not have to begin tions, developing flow, formulating a message, and
on the left and move to the right; it can contain creating the denouement. These activities that are
many loops and multiple occurrences of each com- often intertwined may be achieved sequentially,
ponent. For the sake of simplicity, the three main simultaneously, or through multiple iterations.
components—exploring data, making a story, and Furthermore, while developing the storyline, it
telling a story—are introduced with their respective may be necessary to go back to the explore data
artifacts in a linear order. stage to gather more excerpts (insights or evi-
dence). The final outcome of this step in the pro-
Exploring Data cess is the plot of a story that describes how the
Exploring data involves the set of activities cen- story pieces are related (in time, cause and effect,
tered around exploring and analyzing data. Data patterns, and so on) and what they mean in an
is the raw material that constitutes the source of overall context.

86 September/October 2015

g5vis.indd 86 8/26/15 12:04 PM

 Table 1. Four common storytelling settings.
Example scenario Time Place Audience participation level
Live presentations Synchronous Colocated Low
Dynamic discussions Synchronous Colocated High
Recorded videos, static infographics Asynchronous Distributed Low
Guided tours, interactive infographics Asynchronous Distributed High

Telling a Story Note that it is possible (and often likely) that one
Telling a story is the general process of material- person plays multiple roles. For example, one per-
izing the abstract plot and delivering the story. It son can analyze the data, build a plot, create the
consists of the following activities: building a pre- story material, and then present it. Yet, in many
sentation (creating story material with the chosen professional settings, it is also possible that the
medium), sharing the story using the story mate- roles are taken on by different people. For exam-
rial, and finally receiving and handling the feed- ple, professional analysts or statisticians may be
back from the audience. hired to dig through the data and provide inter-
In the building phase of telling a story, a plot esting excerpts, a journalist may take this input
and story pieces are taken and turned into story and build a plot, a graphic designer may prepare
material. Story material is the materialization of infographics using the plot with the prepared story
each piece of this abstract content through the pieces, and a professional typesetter will include
development of visual representations, interac- the infographics in the news medium.
tions, animations, annotations, or narration. For
example, story material could be one or more vi- External Factors: Audience, Setting, and Medium
sualizations assembled in a slide deck, a video with A set of external factors may impact the story-
narration, an infographic presented on a poster, or telling process. In every step (when searching for
a demo planned with an interactive system for the data, creating a plot with story pieces, building
live presentation. story material, or delivering a visual story), con-
This story material turns into a shared story siderations regarding the target audience at the re-
once it is delivered to at least one person. Ulti- ceiving end of the shared story are typically taken
mately, the perceived story is what the audience into account. A next factor concerns the setting
understands through the storytelling experience. (context), which determines the way a visual story
We acknowledge that, in the current practice, is presented. On a higher level, one can character-
making a story is often merged with the presen- ize the setting by time and place where the story
tation building phase of the telling a story com- is conveyed. The setting can be further character-
ponent. For example, a storyteller may sequence ized by the level of possible audience participation.
specific visual representations to build a plot while Table 1 shows how different combinations of the
making a presentation. However, by making the setting can lead to commonly encountered story-
distinction explicit in the process we can capture telling scenarios with data.
many other ways these two components can inter- The medium is another factor that influences
act. Also, making a story and building story mate- how the story material will be created and pre-
rial require different skill sets, involving the two sented as well as consumed. For example, a story
different roles we describe next. consumption experience with a static image sig-
nificantly differs from that with an interactive
Roles in the Visual Storytelling Process infographics, supporting a simple, controlled in-
A number of different roles are involved in turning teraction. The choice of media (for example, video,
data into a visually shared story: images, text, narration, and interactive system)
comes with presentation parameters that are more
■■ The data analyst engages in the process of ex- or less appropriate (such as color palettes or fonts).
ploring and analyzing the data that is the foun-
dation of the story. Ethics in Visual Data Storytelling
■■ A scripter builds the plot using the chosen excerpts. When visualizations are leveraged to make in-
■■ The editor prepares the story material. tended messages more comprehensible and per-
■■ The presenter is responsible for delivering the suasive, visual representations can be misused
story. unintentionally, offer controversial emphasis, or
■■ The audience experiences the story and provides even be abused intentionally. Alexander B. Howard
feedback. argues that, given the rapidly increasing amount of

IEEE Computer Graphics and Applications 87

g5vis.indd 87 8/26/15 12:04 PM

 Visualization Viewpoints

digital media, data journalists must be more con- ever, there is no explicit support for selecting and
scious of ethics than ever before.7 It seems sensible ordering these elements to form a coherent plot
that the ethics maintained in journalism should with a beginning and an end.
be upheld in visual data storytelling as well. To Because the structure and sequence in stories
increase the transparency and credibility of visual (the plot) can influence the reception of a story,
data stories throughout the visual data storytelling Hullman and her colleagues recently investigated
process, we need to make it easier and more desir- the forms and reactions to sequencing in linear,
able for authors to share underlying data and their slideshow-style presentations using visualiza-
analysis process. tions.8 There are research opportunities in further
exploring ways to provide guidelines for making a
Research Opportunities good story, suggesting different plots, and letting
The storytelling process we have described here helps people experiment with several plots and select
us identify and articulate several research opportu- the most compelling one. For example, as work
nities. We first discuss missing efforts for the two on Show Me9 suggests an appropriate visualiza-
tion for given data properties such as data type
and dimension with other possibilities, a system
We need to make it easier and more for story building can automatically suggest pos-
sible data stories for given story properties (such
desirable for authors to share underlying as story point, intended message, and goal). This
could benefit novices as well as people with ex-
data and their analysis process. tensive training in communication (such as data
journalists) or with inherent storytelling skills
(the ability to create a compelling story).
components of the visual data storytelling process—
making a story and telling a story. We then reflect Make It Easier to Tell a Story
on whether it is desirable to build an encompassing Even authors inherently skilled at creating com-
tool that supports the entire visual data storytelling pelling story structures face the challenge of pro-
process. Finally, we underline the need to develop ducing the story material. This step may prove
novel methodologies to assess the quality of both extremely difficult and time consuming for people
authoring and experiencing visual stories. with low programming skills. Several online tools
(such as Many Eyes) are accessible to nonexperts,
Help People Make a Compelling Story offering a set of visualizations that are based on
While the visualization community has a pleth- predefined templates. They usually support the
ora of techniques supporting the exploration and creation of visualizations through three main
analysis of data, not much attention has been put steps: import data, select the visualization type,
on helping a scripter make a story—that is, iden- and configure the visual attributes (such as color
tifying a sequence of compelling story pieces. In palettes). Although there is some support for sim-
practice, people often make a story (even if just ple interactions, these are usually limited to things
in their minds) while they are either exploring the such as mouse over and click, making these tools
data or building a presentation. Yet, visualization largely only suitable for rather simple stories.
systems do not usually incorporate help for the To support more sophisticated interactions and
general making a story phase, and they particu- animations, people have resorted to programming
larly fall short in supporting people in the steps to using libraries such as D3 (www.d3.org). These
collect and organize excerpts from data explora- libraries have been adopted by many designers
tion that are potentially interesting for the final and data analysts, but they require programming
visual data story. Furthermore, visualization sys- skills. This motivated recent research on helping
tems do not usually provide support for building people with little or no programming skills cre-
a plot from such a collection. As a result, people ate custom visualizations. A recent example is
have to keep track of a set of interesting visual- Ellipsis, a graphical interface for story authoring
izations, often resorting to capturing them on with preexisting visualizations and the Ellipsis
an auxiliary storage such as creating a document domain-specific language (DSL).10 It is an interest-
with screen captures and notes. IBM’s i2 Analyst’s ing research space to support the easy creation of
Notebook is a rare exception in that it is aimed at custom annotations through direct manipulation
streamlining this process (see www-03.ibm.com/ and reuse of existing story elements.
software/products/en/analysts-notebook). How- Most of the existing tools are meant to help peo-

88 September/October 2015

g5vis.indd 88 8/26/15 12:04 PM

 ple create material that can be shared on the Web to learn and use, or limited in functionality and,
asynchronously. However, a presentation based on thus, only useful for certain types of stories with
an interactive visualization could allow the pre- certain types of data.
senter to be more responsive to the audience by in-
teracting with the data and showing the action in Beyond Evaluating a Story
real time. For example, Tableau introduced a fea- Kosara and Mackinlay discussed evaluation as a re-
ture called Story Points11 that allows people to cap- search challenge with a focus on understanding the
ture a set of visualizations with specific states as effectiveness of a story in conveying its message.5
a dashboard (a story point) within Tableau. These Although we agree that this question is important,
can be recorded with captions and reordered with evaluation challenges exist beyond studying the
simple drag-and-drop interaction. Designing an story itself. Understanding where, when, and why
easy-to-use, yet powerful experience for authoring the visual storytelling process is successful is an-
the story material for interactive visualization sys- other fundamental challenge to tackle.
tem is an interesting future research topic. Understanding why a story is or is not effective
Delivering a story live also presents a unique may require an answer that goes beyond looking
opportunity to perceive the audience’s reaction as at story delivery and the story’s reception by the
the story is told, potentially allowing the presenter audience. Several factors in the visual data sto-
to adjust the presentation style in real time to pro- rytelling process can influence a story’s success
vide a better experience. There are opportunities and can lead to poorly delivered stories still be-
to help the presenter monitor audience feedback ing successful or brilliantly delivered stories fail-
and assess their engagement as well as to facilitate ing. Designing evaluation methodologies across
or suggest alteration to the ongoing presentation. the process we described earlier is, thus, a major
In addition, other recent research has focused on research question. It is important to derive study
techniques to better engage the audience. For ex- protocols, new methods, and analysis practices for
ample, SketchStory helps presenters perform an each phase of the visual data storytelling process
engaging storytelling by attracting attention and and attempt to evaluate the process as a whole.
creating anticipation.12 However, it only offers Because storytelling involves either people taking
limited annotation and interaction with the data on multiple roles (data analyst, scripter, editor, and
during the presentation. More research on sup- presenter) or several people working together, the
porting advanced features such as emphasizing story creation and the delivery process can become
different components via annotation, highlight- a collaborative activity. As such, a wealth of re-
ing, and zooming during a live presentation is a search opportunities open up. For example, collab-
promising avenue for research. orative evaluation metrics could be applied or we
could study communication successes and failures
One Tool for the Whole Process? between different people involved in the process.
One research opportunity is an attempt to com-
bine data analysis, scripting, editing, and present- Emerging Scenarios
ing functionality all into one tool or a suite of Our visual data storytelling process supports sev-
tools. However, we need to reflect on whether this eral scenarios involving storytelling with data by
is a desirable outcome. taking different paths through the components
On the one hand, a holistic tool may facilitate and different manifestations of each of the compo-
every requirement of each of the storytelling com- nents. We envision that other storytelling opportu-
ponents and aid people in taking on the different nities exist that include visualization support, and
roles in the process. For example, the scripter’s it is interesting to consider whether our visual data
work may be significantly lessened as new or storytelling process could be expanded, by adding
updated data is processed by the analyst. Thus, steps or activities, or modified to more clearly cap-
the design of an encompassing tool or a suite of ture emerging scenarios. For example, our story-
tools that allows a fluid switch between differ- telling process could support the audience actually
ent storytelling components and activities, keeps taking on the role of data analyst, scripter, and/or
track of the data, and updates story material as editor. Involving the audience in the story creation
updated data appears seems like a promising en- process by opening up the editing and scripting
deavor. On the other hand, such an encompass- roles is an exciting emerging scenario. For example,
ing tool may either be extremely feature-rich to Sprint’s Global Water Experiment (www.visualizing
support various data types, visualization styles, .org/sprint/global-water-experiment), where the
and presentation techniques, making it difficult audience could edit an initial visualization and

IEEE Computer Graphics and Applications 89

g5vis.indd 89 8/26/15 12:04 PM

 Visualization Viewpoints

data, illustrates the potential of collaborative story 3. E. Segel and J. Heer, “Narrative Visualization: Telling
creation. Such emerging scenarios can empower Stories with Data,” IEEE Trans. Visualization and
the audience and make the raw data and the story Computer Graphics, vol. 16, no. 6, 2010, pp. 1139–1148.
creation process more transparent. 4. J. Hullman and N. Diakopoulos, “Visualization
Rhetoric: Framing Effects in Narrative Visualization,”

W
IEEE Trans. Visualization and Computer Graphics, vol.
e aim to facilitate better structured discus- 17, no. 12, 2011, pp. 2231–2240.
sions around compelling techniques for 5. R. Kosara and J. Mackinlay, “Storytelling: The Next
storytelling with data visualization by drawing a Step for Visualization,” Computer, vol. 46, no. 5, 2013,
line between a visual data story and a general data pp. 44–50.
visualization and narrowing the scope of what a 6. J. Gray, L. Chambers, and L. Bounegru, The Data
visual data story is. Considering the entire process Journalism Handbook, O’Reilly Media, 2012.
of transforming data into visually shared stories 7. A.B. Howard, The Art and Science of Data-Driven
along with this more focused definition, we believe Journalism, Tow Center for Digital Journalism;
it is possible to widen the scope of research around http://towcenter.org/wp-content/uploads/2014/05/
visual data stories. By pursuing these new avenues Tow-Center-Data-Driven-Journalism.pdf.
of research in visual data storytelling process, vi- 8. J. Hullman et al., “A Deeper Understanding of
sualizations can enable more effective storytelling Sequence in Narrative Visualization,” IEEE Trans.
with data. Visualization and Computer Graphics, vol. 19, no. 12,
2013, pp. 2406–2415.
9. J. Mackinlay, P. Hanrahan, and C. Stolte, “Show Me:
References Automatic Presentation for Visual Analysis,” IEEE
1. B. Alexander, The New Digital Storytelling: Creating Trans. Visualization and Computer Graphics, vol. 13,
Narratives with New Media, Praeger, 2011. no. 6, 2007, pp. 1137–1144.
2. T. Strothotte and C. Strothotte, Seeing between the 10. A. Satyanarayan and J. Heer, “Authoring Narrative
Pixels, Springer-Verlag, 2003. Visualizations with Ellipsis,” Computer Graphics Forum
(Proc. EuroVis), vol. 33, no. 3, 2014, pp. 361–370.
11. E. Fields, “8.2 Preview: Story Points,” Tableau
Software, 20 May 2014; www.tableausoftware.com/
about/blog/2014/5/82-preview-tell-story-your-data

CONFERENCES
-story-points-30761.
12. B. Lee, R.H. Kazi, and G. Smith, “SketchStory: Telling
in the Palm of Your Hand More Engaging Stories with Data through Freeform
Sketching,” IEEE Trans. Visualization and Computer
Let your attendees have: Graphics, vol. 19, no. 12, 2013, pp. 2416–2425.

• conference schedule
• conference information
• paper listings Bongshin Lee is a researcher in Microsoft Research’s neXus
• and more
Group. Contact her at [email protected].
The conference program mobile app
works for Android devices, iPhone,
iPad, and the Kindle Fire. Nathalie Henry Riche is a researcher in Microsoft Re-
search’s neXus Group. Contact her at [email protected].

Petra Isenberg is a research scientist in Inria’s Aviz project.

Contact her at [email protected].

Sheelagh Carpendale is a professor at the University of

For more information please contact Calgary. Contact her at [email protected].
Conference Publishing Services (CPS) at
[email protected]
Contact department editor Theresa-Marie Rhyne at
[email protected].

Selected CS articles and columns are also available

for free at http://ComputingNow.computer.org.

90 September/October 2015

g5vis.indd 90 8/26/15 12:04 PM